Personal tools
You are here: Home SIGs and LIGs Speech SIG is17-compare

is17-compare

 

Computational Paralinguistics Challenge (ComParE), Interspeech 2017

Addressee, Cold, Snoring



Last updated:

12 September 2017

Last addition:

Winners of the Challenge announced.



Organisers:

Björn Schuller (University of Passau, Germany AND Imperial College London, UK)
Stefan Steidl (FAU Erlangen-Nuremberg, Germany)
Anton Batliner (University of Passau, Germany)
Elika Bergelson (Duke University, USA)
Jarek Krajewski (University of Wuppertal, Germany)
Christoph Janott (TUM, Germany)

Sponsored by:

Association for the Advancement of Affective Computing (AAAC)
audEERING GmbH
iHEARu



Officially started:

31 January 2017 (Cold Sub-Challenge)
10 February 2017 (Addressee Sub-Challenge)
16 February 2017 (Snoring Sub-Challenge)



*FINISHED* Winners of the INTERSPEECH 2017 Computational Paralinguistics Challenge:

  • The Addressee Sub-Challenge is awarded to:
    No award was given, as the baseline was not surpassed (for the first time in the Challenge series).
  • The Cold Sub-Challenge is awarded to:
    Gábor Gosztolya, Róbert Busa-Fekete, Tamás Grósz, László Tóth:
    DNN-based Feature Extraction and Classifier Combination for Child-Directed Speech, Cold and Snoring Identification
  • The Snoring Sub-Challenge is awarded to:
    Heysem Kaya, Alexey A. Karpov:
    Introducing Weighted Kernel Classifiers for Handling Imbalanced Paralinguistic Corpora: Snoring, Addressee and Cold

The organisers congratulate the winners and thank all participants for their outstanding contributions - Overall the best result could be reached by fusion of these in the Sub-Challenges.

If you are interested in using the corpora of the Challenge outside this event, please see here for new licenses and download.





Deadlines:

14 March 2017 Paper submission to INTERSPEECH 2017
1 June 2017 Final result upload
5 June 2017 Camera-ready paper




Baseline results are available:

Download the Paper on the Challenge.




Call for Participation as PDF.

*Get started:*
License Agreement (Addressee) for the dataset download (Addressee Sub-Challenge)
License Agreement (Cold) for the dataset download (Cold Sub-Challenge)
License Agreement (Snoring) for the dataset download (Snoring Sub-Challenge)



*Read about it:* Paper on the Challenge to be cited.
*Participate:* Result submission is now open. Participants were contacted by email for the result submission procedure.
*FAQ:* Frequently asked questions.



The Challenge

The Interspeech 2017 Computational Paralinguistics ChallengE (ComParE) is an open Challenge dealing with states and traits of speakers as manifested in their speech signal’s acoustic properties. There have so far been eight consecutive Challenges at INTERSPEECH since 2009 (cf. the repository), but there still exists a multiplicity of not yet covered, but highly relevant paralinguistic phenomena. Thus, we introduce three new tasks by the Addressee Sub-Challenge, the Cold Sub-Challenge, and the Snoring Sub-Challenge.

For the tasks, the data are provided by the organisers.

For the Addressee Sub-Challenge, Elika Bergelson, Marisa Casillas, Amanda Seidl, Melanie Soderstrom, Andrei Amatuni, and Anne Warlaumont are providing the Homebank Child/Adult Addressee Corpus (HB-CHAAC). This corpus is comprised of recordings of conversations involving babies/toddlers and their caregivers and siblings in their natural home environment using the LENA recording device. A random selection of 20 conversations from each of 61 homes are included. These include 2523 minutes of home recording, with 288 minutes of adult speech hand-labelled as 'child-directed' or 'adult-directed'. The task is to determine whether the adult is speaking to a child (child-directed speech) or another adult (adult-directed speech).

The Cold Sub-Challenge will base its results on the Upper Respiratory Tract Infection Corpus (URTIC) which is kindly given by Jarek Krajewski and colleagues. This dataset provides 30k segments of speech and non-verbal vocalisations by 700+ subjects under cold or control group subjects with a total duration of 152 hours. The task will be to recognise speech under cold based on acoustic information.

Finally, in the Snoring Sub-Challenge, comparisons will be run on the Munich-Passau Snore Sound Corpus (MPSSC) given by Christoph Janott and colleagues. Four “VOTE” snore sound types (excitation at the velum, oropharyngeal area, tongue base, or epiglottis) – overall 800+ single, isolated snores with a length 0.2s – 1.5s per event of 200+ subjects from three different medical centres recorded between 2006 and 2015, using varying sets of audio equipment are contained. The task will be to determine the excitation location based on acoustics.

The INTERSPEECH 2017 Computational Paralinguistics Challenge (ComParE) shall help bridging the gap between excellent research on paralinguistic information in spoken language and low compatibility of results.

Overall, three Sub-Challenges are thus addressed:

  • In the Addressee Sub-Challenge, the conversational partner (child/adult) has to be identified.
  • In the Cold Sub-Challenge, speech under cold has to be determined for the first time.
  • In the Snoring Sub-Challenge, four types of snoring sounds have to be recognised.

All Sub-Challenges allow contributors to find their own features with their own machine learning algorithm. However, a standard feature set will be provided that may be used. Participants will have to stick to the definition of training, development, and test sets as given. They may report results obtained on the development sets, but have only five trials to upload their results on the test set per Sub-Challenge, whose labels are unknown to them. Each participation has to be accompanied by a paper presenting the results that undergoes the normal Interspeech peer-review and has to be accepted for the conference in order to participate in the Challenge. The organisers preserve the right to re-evaluate the findings, but will not participate themselves in the Challenge.

We encourage both - contributions aiming at highest performance w.r.t. the baselines provided by the organisers, and contributions aiming at finding new and interesting insights w.r.t. these data. Overall, contributions using the provided or equivalent data are sought for (but not limited to):

  • Participation in a Sub-Challenge
  • Contributions focussing on Computational Paralinguistics centred around the Challenge topics

The results of the Challenge will be presented at Interspeech 2017 in Stockholm, Sweden.
Prizes will be awarded to the Sub-Challenge winners.
If you are interested and planning to participate in the Computational Paralinguistics Challenge, or if you want to be kept informed about the Challenge, please send the organisers an e-mail to indicate your interest.




To get started: Please obtain the License Agreement (Addressee Sub-Challenges) or License Agreement (Cold Sub-Challenge) or License Agreement (Snoring Sub-Challenge) to get a password and further instructions for the download of the datasets: Please fill it out for the dataset(s) you wish to obtain, and print, sign, scan, and email accordingly. The agreement(s) has/have to be signed by a permanent staff member.
After downloading the data you can directly start your experiments with the data sets. Once you found your best method you should write your paper for the Special Session. At the same time you can compute your results per instance and Sub-Challenge task on the test set and upload them: we will then let you know your performance result.

Paper on the Challenge: The introductory Paper on the Challenge (the paper will soon be available) provides extensive descriptions and baseline results. All participants will be asked to avoid repetitions of Challenge, data, or feature descriptions in their submissions - of course, they have to describe shortly the essentials of the databases dealt with - but include the following citation:

Björn Schuller, Stefan Steidl, Anton Batliner, Elika Bergelson, Jarek Krajewski, Christoph Janott, Andrei Amatuni, Marisa Casillas, Amanda Seidl, Melanie Soderstrom, Anne S. Warlaumont, Guillermo Hidalgo, Sebastian Schnieder, Clemens Heiser, Winfried Hohenhorst, Michael Herzog, Maximilian Schmitt, Kun Qian, Yue Zhang, George Trigeorgis, Panagiotis Tzirakis, Stefanos Zafeiriou: "The INTERSPEECH 2017 Computational Paralinguistics Challenge: Addressee, Cold & Snoring", Proceedings INTERSPEECH 2017, ISCA, Stockholm, Sweden, 2017

Result Submission: Will be opened in due course. Registered participants will be contacted by email.

Paper Submission (all participants): Please be reminded that a paper submission and at least one upload on the test set are mandatory for the participation in the Challenge. However, paper contributions within the scope are also welcome if the authors do not intend to participate in the Challenge itself. In any case, please submit your paper until 14 March 2017 (and final results by 1 June 2017) using the standard style info and respecting length limits, and submit to the regular submission system. Important: as topic you should choose only this Special Session (Computational Paralinguistics Challenge 2017). Please note that

  • The deadline for submission of the papers and results is the INTERSPEECH 2017 paper submission deadline: 14 March 2017. Remaining result upload trials can be saved for new Challenge results until 1 June 2017.
  • The papers will undergo the normal review process.
  • Papers shall not repeat the descriptions of database, labels, partitioning etc. of the Sub-Challenge corpora but cite the introductive paper :

    Björn Schuller, Stefan Steidl, Anton Batliner, Elika Bergelson, Jarek Krajewski, Christoph Janott, Andrei Amatuni, Marisa Casillas, Amanda Seidl, Melanie Soderstrom, Anne S. Warlaumont, Guillermo Hidalgo, Sebastian Schnieder, Clemens Heiser, Winfried Hohenhorst, Michael Herzog, Maximilian Schmitt, Kun Qian, Yue Zhang, George Trigeorgis, Panagiotis Tzirakis, Stefanos Zafeiriou: "The INTERSPEECH 2017 Computational Paralinguistics Challenge: Addressee, Cold & Snoring", Proceedings INTERSPEECH 2017, ISCA, Stockholm, Sweden, 2017

  • Participants may contribute in all Sub-Challenges at a time.
  • A training and development partitioning will allow for tests and results to be reported by the participants apart from their results on the official test set.
  • Papers may well report additional results on other databases.
  • An additional publication is planned that summarises all results of the Challenge and results combination by ROVERING or ensemble techniques. However, this publication will be post INTERSPEECH 2017.



Frequently asked questions: you might find an answer to your questions here:
  • Q: Are there scripts to extract the ComParE feature set with openSMILE available?
    A: Please see here.
  • Q: Do I have to include test set results for the paper submission?
    A: Yes. However, you can update these in the final camera ready version. In any case, at least one result submission has to be made before the paper deadline.
  • Q: Can we submit one paper to Interspeech describing participation in several Sub-Challenges?
    A: This is possible - obviously the paper should read well and contain all necessary details, as it needs to pass peer-review as condition for successful Challenge participation.
  • Q: Can we submit more than one paper to Interspeech describing participation in different Sub-Challenges?
    A: This is possible - you can submit, e.g., separate papers for the participation in different Sub-Challenges or for different approaches in the same Sub-Challenge. Each one will undergo peer-review independently.
  • Q: Can we have richer meta-data for the databases, so that we can also look at other things?
    A: Items not available in the current download package will only be available after the Challenge. Then, however, more info will partially be given.
  • Q: My results are below the baseline - does it make sense to submit?
    A: Of course it does. We do not know whether the baseline will be surpassed and different experiences with the tasks on the same dataset will be of interest. Please remember that all submissions to the Challenge go through the normal reviewing process. Although it is very likely that the reviewers do know - and take into account - the baselines, the criteria are the usual, i.e., scientific quality; surpassing any baseline - be this the one given for this challenge, or another one known from the literature - is just one of the criteria. A paper reporting results above the baseline, but poorly written, runs high risks *not* to be accepted; in contrast, a paper which is well written, contributing to our knowledge, but with results below the baseline, has high chances to be accepted.
  • Q: When is the deadline for submitting the results?
    A: You will need to submit results by 1 June 2017 prior to camera ready paper submission to INTERSPEECH as a result on test needs to be included in your final paper version if you want to compete for the Sub-Challenge awards. All except one (which has to be used before the paper deadline on 14 March 2017) of the five result submissions per Sub-Challenge and participant can be saved for submission as late as 1 June 2017.
  • Q: How will the data be distributed to the participants? Are you sending the test data at the same time with training and development partitions?
    A: Yes, we do. However, labels are only given for training and development partitions. Further, we are not sending the data - you will need to download the data. Please first download, print, and sign the license agreements - one per data set once the agreements are available (cf. above) - and scan and mail or fax these to the addresses given on the agreements. You will then receive an email with download instructions.
  • Q: Can I use additional databases within the Challenge?
    A: You can - as long as these are well known and accessible to everybody and you clearly state the details in your paper.
  • Q: May I participate in several Sub-Challenges?
    A: Yes, of course. Every site may participate in every Sub-Challenge and the five upload trials are per Sub-Challenge. In how many Sub-Challenges one site participates is decided at the moment you submit your paper by the results you include. Please send us an additional e-mail to indicate which Sub-Challenges you participate in for us to know.
  • Q: Do I have to submit a paper in order to participate?
    A: Yes, the submission and acceptance of a paper is mandatory. Please make sure to select the special event during the submission procedure as first option. The prizes will be awarded during INTERSPEECH 2017 in Stockholm/Sweden.
  • Q: May I include results on other corpora in my paper?
    A: Yes, of course. As long as it fits the general focus these are of course very welcome.
  • Q: What are the other formalities with respect to the paper of mine?
    A: Please make sure to reference the official Paper on the Challenge (now available) and avoid repetition of the general data and Challenge description.
  • Thank you and welcome to the Challenge!



More Information will follow on a regular basis.



Document Actions
Powered by Plone

Portal usage statistics