Paralinguistic Challenge, Interspeech 2010

Most paralinguistic analysis tasks resemble each other not only by means of processing and ever-present data sparseness, but by lacking agreed-upon evaluation procedures and comparability, in contrast to more “traditional” disciplines in speech analysis; at the same time, this is a rapidly emerging field of research, due to the constantly growing interest on applications to human behaviour analysis, and technologies for human-machine communication and multimedia retrieval.

In these respects, the INTERSPEECH 2010 Paralinguistic Challenge shall help bridging the gap between excellent research on paralinguistic information in spoken language and low compatibility of results, by addressing three selected tasks. The aGender and the the AVIC corpora will be provided by the organisers. The first consists of 46 hours of telephone speech, stemming from 954 speakers, and will serve to evaluate features and algorithms for the detection of speaker age and gender. The second features 2 hours of human conversational speech recording (21 subjects), annotated in different levels of interest. The corpus further features a transcription of spoken content with word boundaries by forced alignment, non-linguistic vocalisations, single annotator tracks, different well-defined chunkings, and the sequence of these. Both corpora are given with distinct definition of test, development, and training partitions, incorporating speaker independence and different miking, as needed in most real-life settings. Benchmark results of the most popular approaches will be provided.

Three sub-challenges are addressed:

• In the Age Sub-Challenge, the age of speakers has to be determined in four different age groups as classification task. The challenge measure will be the unweighted average recall per class to better compensate for imbalance between these four age groups. However, in the training and development sets also the actual speaker age in years is provided. This may be used as additional information for model construction or reporting of more precise results in submitted papers.
• In the Gender Sub-Challenge, a three-class classification task has to be solved: apart from female/male, a third "gender" is added: children. While this is strictly speaking a combined age/gender task the reason is obvious: determining childrens' gender by speech is beyond this challenge. As for the age challenge, the competition measure is unweighted average recall of these three classes.
• Finally, the Affect Sub-Challenge asks for determination of speakers’ interest in ordinal representation in this year’s challenge as opposed to last INTERSPEECH’s Emotion Challenge, which dealt with emotion in a broader sense. The competition measure will thus be the cross-correlation between the annotators' mean "Level of Interest" annotation ground truth and the regression prediction. However, as a secondary measure the mean linear error will be calculated. Here, participants will also be given the individual labelers' tracks for model construction.

All sub-challenges allow contributors to find their own features with their own classification algorithm. However, a standard feature set will be given per corpus that may be used. The labels of the test set will be unknown, and participants will have to stick to the definition of training, development, and test sets. They may report on results obtained on the development set, but have only one trial to upload their results on the test set, whose labels are unknown to them. Each participation will be accompanied by a paper presenting the results that undergoes peer-review. The organisers preserve the right to re-evaluate the findings, but will not participate themselves in the challenge. Participants are encouraged to compete in multiple sub-challenges.

Overall, contributions using the provided or an equivalent database are sought in (but not limited to) the following areas:

• Participation in any of the sub-challenges
• Combined determination of paralinguistic information under mutual information exploitation
• Novel features and algorithms for the detection of paralinguistic information
• Novel corpora and evaluations for paralinguistic tasks

The results of the Challenge will be presented at a Special Session of Interspeech 2010 in Makuhari, Japan.
Prizes will be awarded to the sub-challenge winners and a best paper.
If you are interested and planning to participate in the Paralinguistic Challenge, or if you want to be kept informed about the Challenge, please send the organisers an e-mail to indicate your interest.