General

iHEARu-PLAY is a crowdsourcing-based scientific game, where players listen to collections of short audio recordings, watch collections of video recordings or images and provide answers to multiple questions regarding the content of the samples. Further, players can perform prompted recording tasks for different topics. Depending on their own choices and those of other players, players are awarded with game points, can reach different experience levels and are ranked on the leaderboard accordingly.
In order to improve state-of-the-art speech analysis methods and the underlying machine learning techniques, researchers require huge amounts of annotated audio data (i.e. audio samples with corresponding metadata) to train and test their systems. Up until now, the collection, recording and annotation of audio samples has been a very inefficient, costly and time-consuming process. By playing iHEARu-PLAY, volunteers are able to directly contribute valuable data to researchers and help advance science just by solving the tasks in the game.
Recently, automatic speech and speaker recognition has matured to the degree that it entered the daily lives of thousands of Europe's citizens, e.g., on their smart phones or in call services. During the next years, speech processing technology will move to a new level of social awareness to make interaction more intuitive, speech retrieval more efficient, and lend additional competence to computer-mediated communication and speech-analysis services in the commercial, health, security, and further sectors. To reach this goal, rich speaker traits and states such as age, height, personality and physical and mental state as carried by the tone of the voice and the spoken words must be reliably identified by machines.
The iHEARu project, short for "Intelligent systems' Holistic Evolving Analysis of Real-life Universal speaker characteristics", aims to push the limits of intelligent systems for computational paralinguistics by considering Holistic analysis of multiple speaker attributes at once, Evolving and self-learning, deeper Analysis of acoustic parameters - all on Realistic data on a large scale, ultimately progressing from individual analysis tasks towards universal speaker characteristics analysis, which can easily learn about and can be adapted to new, previously unexplored characteristics.

How to start

There is absolutly no foreknowledge needed! The game was specifically designed to allow as many people as possible to play the game. Our game rules are simple enough to allow anyone to get into the game within minutes. See our short tutorial for further information.
You can play the game without the need for an account and the data you submit will help us just like the data of regular players. Nevertheless, most of its functionality will be disabled until you sign up, so you will not be able to create a profile, earn points for annotating and recording or track your progress on the leaderboard.
All you need is a way to playback audio, preferably via headphones or external speakers and/or a standalone or build-in microfone to record your speech. Make sure you are in a quiet environment, so your speakers or headphones allow you to listen to the audio samples and/or your microfone is able to record your speech in good quality.
For a quick start, please check out our tutorial and watch our tour video.

Game-related questions

This is by design. Don't be confused by any choices that may not make any sense in the given context of the task. Select the choice(s) you think best represent your answer.

The gamerscore is an indication of how long a player has played iHEARu-PLAY and how well he or she has been doing overall. New players start with a score of zero. For each submitted annotation and recorded prompt, players get awarded a certain number of points.

For the annotation mode, the number of points depends on how well the submitted answer agrees to the answers of other players. Choosing an answer that fits well to the answers of other players will give more points than an uncommon answer.

In recording mode, the number of points a player will get for uploading a recording depends on the effort required to record the prompt, i.e. a recording task that takes longer to record will give more points than a short task. For any additional recordings uploaded for a prompt, players will only get a reduced number of additional points.

Any points awarded are multiplied by the player's current score multiplier before being added to the player's gamerscore. In addition to gaining points from completing annotation or recording tasks, players with the rank Expert gain bonus points for any other Expert player that agrees with their answer on the same question if they are the first to propose a new unique answer.

Points may also be awarded through the discovery of bonus items that are occasionally hidden behind questions.

The score multiplier weights points awarded from completed annotation or recording tasks on the basis of four aspects: the players rank, the number of tasks completed on the current day, a trustability rating of the player that incorporates how reliable we regard the player in answering questions faithfully, and a dataset multiplier that may lead to some databases giving you more points than others.
Bonus items are random items that will add extra points to your score by awarding you with an additional score multiplier for a fixed time, extra points and more. At the moment we have the following bonus items available:

Daily Bonus: It's created every single day and lasts for the first 25 questions you play on that day.

New Member: Whenever you register with your account for the first time you get the bonus item, which lasts for your first 25 questions. It's attainable only once.

Hard Worker: It's for the players who are eager to annotate more than others. It is valid for 50 questions on that day you got the bonus item. This is a terrific opportunity to increase your rank in the leaderboard.

Skipping Questions: If you are lucky, you will find this Bonus Item, which enables you to skip a question you don't like answer. Whenver you skip the question you gain also points to increase your score. In Addition to that, your progress on the database will increase as if you answered the question. It's valid only on the day of winning it.

Players start as Beginners, and are promoted to next higher rank by reaching the specified points for that rank. The best rank a player can be awarded with is the Legendary rank after reaching a score of 3 000 000 points or higher. All achievable ranks with their according points are summarized in the table below:
Rank Points needed
Newbie 0
Beginner 1,000
Novice 5,000
Intermediate 20,000
Adept 100,000
Expert 250,000
Veteran 500,000
Master 1,000,000
Ace 2,000,000
Legendary 3,000,000
Badges are achievements listed on your profile page and are visible to other players. They are awarded to players fulfilling special requirements that are unique to each badge, e.g. reaching a certain number of points or completing your profile information. Some badges can be earned in the 3 different categories bronze, silver and gold. A bronze badge will have the easiest conditions to fullfil, wheras gold has the hardest conditions. You can be awarded with the following badges:
Badge Description Availability
Autobiographer Complete your profile Gold
Nothing To Hide Fill out the metadata form Bronze, Silver, Gold
First Steps as Annotator Submit your 100th annotation Bronze, Silver, Gold
First Steps as Recorder Upload your 50th recording Bronze, Silver, Gold
Blabbermouth Submit recordings totalling 4 hours in length Bronze, Silver, Gold
Busy Bee Make it into the top 3 players of the last 7 days Bronze, Silver, Gold
Busy Beaver Make it into the top 3 players of the last month Bronze, Silver, Gold
Champion Make it into the top 3 players of all time Bronze, Silver, Gold
Treasure Hunter Collect 50 bonus items on a single day Bronze, Silver, Gold
Regular Customer Answer at least 20 questions every day for 14 days in a row Bronze, Silver, Gold
On a Roll Answer 25 questions with excellent answers in a row Bronze, Silver, Gold
Streak of Bad Luck Do not find any bonus item for 28 questions in a row Bronze, Silver, Gold
Early Bird Answer 200 questions in one morning between 4 am and 8 am Bronze, Silver, Gold
Night Owl Answer 200 questions during a single night between 23 pm and 3 am Bronze, Silver, Gold
On The Go Play iHEARu-PLAY on a mobile device Gold
Feedback Provide us feedback by filling out a survey Gold
Bug Reporter Report a previously unknown website bug or recording issue Gold
No, you can also use your keyboard to annotate files. Press spacebar to start/stop the audio or video file from playing. Select an appropriate answer with your arrow keys and confirm it with the enter key.
  • Single-Choice:This question type is mainly about choosing only one possible answer from the choices.
  • Multiple-Choice: From the set of all possible answers, you may find more than one choice as a suitable answer. Therefore, this question type allows you to choose more than one answer for the questions.
  • Scaled Values: Your answer of the question will be given as a discrete (numeric) data.
  • Continuous numeric: For this question type, your answer will be given on a continuous (numeric) scale.
  • 2D Continuous numeric:This question type gives you the possibility to answer the question in form of a chessboard with it's boxes to choose only one box from the two-dimensional graph.
  • Manikins: This question type is about choosing one possible figure as an answer from the given choices.
  • Pairwise Comparison: With this type of question you will hear two audio files. Select the one that matches the given description best.

Troubleshooting

Please check your spam mails. If there is no verification mail in your spam mails, we can re-send you the verification mail. If you still have not received our verification mail, please contact us here.
Unfortunately, our recorder is just supported by the latest versions of Mozilla Firefox and Google Chrome. We are working on making it available to all browsers in future. If you use one of these browsers, make sure you allow your browser to get access to the microphone.

Privacy

Your personal data will help us to establish statistical information about research with user annotations, e.g. differences in performances of different user groups. Your data is only used for stated purposes and is anonymized in the process. For more information, please see our Privacy Policy.
Data security is very important to us! Your personal data are stored in our database at a secured server at the University of Passau. Only the administrators of the service have fully access to your data. For more information, please see our Privacy Policy.
We will use your personal data only anonymised and only for statistical results of the experiments. Your user name and e-mail address is only used and stored in order to provide this service and will never be shared with third parties. For more information, please see our Privacy Policy.
Any submitted answer and recorded audio file are stored in our database and will be used for scientific research only. For this purpose the pseudonymized data may be shared with other research bodies working in the same fields or as part of collaborations, but only on a licence basis. For more information, please see our Privacy Policy.
No! If you decide to sign up with your Facebook account, your data on Facebook will not be used by us in any way. We have no rights to access your information whatsoever.

Contact and additional information

If you encounter any inconsistencies or bugs on our site, or if you have any other question, please feel free to contact us via our contact form. We will come back to you as soon as possible.
The iHEARu project website contains information on current activities and an extensive list of publications of the iHEARu team.
If you want to delete your account, go to your profile page and click on "My account" and "Delete account". This will remove your account from all public areas on the website but we will keep the annotations and recordings you submitted as part of the game. In order to have all your submitted data from annotations and recordings deleted, as well, please do not delete your account yet but contact us via the contact form.
You can delete your annotations and recordings without deleting your account, just contact us via our contact form any time.
Your registered e-mail adress will be deleted and the other data will be stored anonymised. If you further like to delete all data, just contact us via our contact form.