Personal tools
You are here: Home SIGs and LIGs Speech SIG is18-compare

is18-compare

Homepage of the Interspeech 2018 Computational Paralinguistics Challenge

 

Computational Paralinguistics Challenge (ComParE), Interspeech 2018

Atypical & Self-Assessed Affect, Crying & Heart Beats



Last updated:

7 February 2018

Last addition:

Challenge started.

Organisers:

Björn Schuller (University of Augsburg, Germany / Imperial College London, UK / audEERING)
Stefan Steidl (FAU Erlangen-Nuremberg, Germany)
Anton Batliner (University of Augsburg, Germany)
Peter Marschik (University of Graz, Austria / University Medical Center Göttingen, Germany / Karolinska Institutet Stockholm, Sweden)
Harald Baumeister (University of Ulm, Germany)
Fengquan Dong (Shenzhen University General Hospital, P.R. China)

Sponsored by:

Association for the Advancement of Affective Computing (AAAC)
audEERING GmbH
iHEARu



Officially started:

1 February 2018 (Crying & Heart Beats Sub-Challenges)





Deadlines:

16 March 2018 Paper abstract registration at INTERSPEECH 2018
23 March 2018 Final paper submission to INTERSPEECH 2018
11 June 2018 Final result upload
17 June 2018 Camera-ready paper


Call for Participation as PDF.

*Get started:*
License Agreements (All Sub-Challenges) to get a password and further instructions for the download of the datasets: Please fill out electronically the PDF for the dataset(s) you wish to obtain, and print, sign, scan, and email both the electronic version and the scans accordingly to ease handling. The agreement(s) has/have to be signed by a permanent staff member.
After downloading the data you can directly start your experiments with the data sets.

*FAQ:* Frequently asked questions.





Baseline results will follow soon.





The Challenge

The Interspeech 2018 Computational Paralinguistics ChallengE (ComParE) is an open Challenge dealing with states and traits of speakers as manifested in their speech signal’s acoustic properties. There have so far been nine consecutive Challenges at INTERSPEECH since 2009 (cf. the repository), but there still exists a multiplicity of not yet covered, but highly relevant paralinguistic phenomena. Thus, we introduce four new tasks by the Atypical Affect Sub-Challenge, the Self-Assessed Affect Sub-Challenge, the Crying Sub-Challenge, and the Heart Beats Sub-Challenge.

For the tasks, the data are provided by the organisers.

For the Atypical Affect Sub-Challenge, Björn Schuller, Simone Hantke, and colleagues are providing the EMOTASS Corpus. This unique corpus is the first to give access to recordings of affective speech from disabled individuals encompassing a broader variety of mental, neurological, and physical disabilities. It comprises recordings of 15 disabled adult individuals (ages range from 19 to 58 years with a mean age of 31.6 years). The task will be classification of five emotions from their speech facing atypical display. Recordings were made in their everyday working environment. Overall, around 11k utterances and around nine hours of speech are included.

In the Self-Assessed Affect Sub-Challenge, comparisons will be run on the USOMS database given by Harald Baumeister, Eva-Maria Rathner and colleagues. It comprises four times five-minute sessions from around 150 individuals. They had to speak spontaneously twice about negative, and twice about positive experiences in their life. Before and after, the speaker reports a self-assessment of their own state of mind (Arousal and Valence on a ten-point Likert scale). The task will be to determine the emotion of individuals as was assessed by themselves.

The Crying Sub-Challenge will base its results on the CRIED dataset which is kindly given by Peter Marschik, Florian Pokorny, and colleagues. It comprises >5k vocalisations of 20 healthy infants (10 females) recorded within a study on postnatal neuro-functional and neuro-behavioural changes and adaptations. The infants were recorded 7 times in bi-weekly intervals, with the first assessment at 4 weeks and the last one at 16 weeks post-term age. All vocalisations were extracted from sequences of up to 5 minutes in duration in which the infants were awake, lying in supine position in a cot. During these sequences the infants were not exposed to external stimuli or manipulation. Vocalisation segmentation was based on the criterion to assign a vocalisation to a distinct vocal breathing group. Vegetative sounds, such as breathing sounds, smacking sounds, hiccups, etc., were not segmented and, thus, not included in the dataset. The vocalisations were categorised into the following three classes: (i) neutral/positive mood vocalisations, (ii) fussing vocalisations, and (iii) crying vocalisations. The categorisation process was done on the basis of audio-video clips by two experts in the field of early speech-language development. Automatic classification into these classes has to be performed by the participants.

The Heart Beats Sub-Challenge bases on the Heart Sounds Shenzhen (HSS) database which is kindly given by Fengquan Dong and colleagues. It is a data set of heart sounds gathered from 170 (55 female, age ranges 21 – 88 years) subjects with various ages and health conditions. The data set was collected by Shenzhen University, Shenzhen, China. There are three classes to be recognised for the data: normal, mild, and moderate/severe as diagnosed by physicians specialized in heart disease. These will have to be recognised from the acoustics of the signal.

The INTERSPEECH 2018 Computational Paralinguistics Challenge (ComParE) shall help bridging the gap between excellent research on paralinguistic information in spoken language and low compatibility of results.

Overall, four Sub-Challenges are thus addressed:

  • In the Atypical Affect Sub-Challenge, emotion of disabled speakers is to be recognised.
  • In the Self-Assessed Affect Sub-Challenge, self-assessed affect shall be determined.
  • In the Crying Sub-Challenge, mood-related types of infant vocalisation have to be classified.
  • In the Heart Beats Sub-Challenge, types of Heart Beat Sounds need to be distinguished.

All Sub-Challenges allow contributors to find their own features with their own machine learning algorithm. However, a standard feature set will be provided that may be used. Participants will have to stick to the definition of training, development, and test sets as given. They may report results obtained on the development sets, but have only five trials to upload their results on the test set per Sub-Challenge, whose labels are unknown to them. Each participation has to be accompanied by a paper presenting the results that undergoes the normal Interspeech peer-review and has to be accepted for the conference in order to participate in the Challenge. The organisers preserve the right to re-evaluate the findings, but will not participate themselves in the Challenge.

We encourage both - contributions aiming at highest performance w.r.t. the baselines provided by the organisers, and contributions aiming at finding new and interesting insights w.r.t. these data. Overall, contributions using the provided or equivalent data are sought for (but not limited to):

  • Participation in a Sub-Challenge
  • Contributions focussing on Computational Paralinguistics centred around the Challenge topics

The results of the Challenge will be presented at Interspeech 2018 in Hyderabad, India.
Prizes will be awarded to the Sub-Challenge winners.
If you are interested and planning to participate in the Computational Paralinguistics Challenge, or if you want to be kept informed about the Challenge, please send the organisers an e-mail to indicate your interest.




To get started: Please obtain the License Agreements (All Sub-Challenges) to get a password and further instructions for the download of the datasets: Please fill out electronically the PDF for the dataset(s) you wish to obtain, and print, sign, scan, and email both the electronic version and the scans accordingly to ease handling. The agreement(s) has/have to be signed by a permanent staff member.
After downloading the data you can directly start your experiments with the data sets. Once you found your best method you should write your paper for the Special Session. At the same time you can compute your results per instance and Sub-Challenge task on the test set and upload them: we will then let you know your performance result.

Paper on the Challenge: The introductory Paper on the Challenge (the paper will soon be available) provides extensive descriptions and baseline results. All participants will be asked to avoid repetitions of Challenge, data, or feature descriptions in their submissions - of course, they have to describe shortly the essentials of the databases dealt with - but include the following citation:

Björn W. Schuller, Stefan Steidl, Anton Batliner, Peter B. Marschik, Harald Baumeister, Fengquan Dong, Simone Hantke, Florian Pokorny, Eva-Maria Rathner, Katrin D. Bartl-Pokorny, Christa Einspieler, Dajie Zhang, Alice Baird, Shahin Amiriparian, Kun Qian, Zhao Ren, Maximilian Schmitt, Panagiotis Tzirakis, Stefanos Zafeiriou: "The INTERSPEECH 2018 Computational Paralinguistics Challenge: Atypical & Self-Assessed Affect, Crying & Heart Beats", Proceedings INTERSPEECH 2018, ISCA, Hyderabad, India, 2018.

Result Submission: Will be opened in due course. Registered participants will be contacted by email.

Paper Submission (all participants): Please be reminded that a paper submission and at least one upload on the test set are mandatory for the participation in the Challenge. However, paper contributions within the scope are also welcome if the authors do not intend to participate in the Challenge itself. In any case, please register your paper by 16 March 2018 and submit your paper until 23 March 2018 (and final results by 17 June 2018) using the standard style info and respecting length limits, and submit to the regular submission system. Important: as topic you should choose only this Special Session (Computational Paralinguistics Challenge 2018). Please note that

  • The deadline for submission of the papers and results is the INTERSPEECH 2018 paper submission deadline: 23 March 2018 (but registration of papers by 16 March 2018!). Remaining result upload trials can be saved for new Challenge results until 11 June 2018.
  • The papers will undergo the normal review process.
  • Papers shall not repeat the descriptions of database, labels, partitioning etc. of the Sub-Challenge corpora but cite the introductive paper :

    Björn W. Schuller, Stefan Steidl, Anton Batliner, Peter B. Marschik, Harald Baumeister, Fengquan Dong, Simone Hantke, Florian Pokorny, Eva-Maria Rathner, Katrin D. Bartl-Pokorny, Christa Einspieler, Dajie Zhang, Alice Baird, Shahin Amiriparian, Kun Qian, Zhao Ren, Maximilian Schmitt, Panagiotis Tzirakis, Stefanos Zafeiriou: "The INTERSPEECH 2018 Computational Paralinguistics Challenge: Atypical & Self-Assessed Affect, Crying & Heart Beats", Proceedings INTERSPEECH 2018, ISCA, Hyderabad, India, 2018.

  • Participants may contribute in all Sub-Challenges at a time.
  • A training and development partitioning will allow for tests and results to be reported by the participants apart from their results on the official test set.
  • Papers may well report additional results on other databases.
  • An additional publication is planned that summarises all results of the Challenge and results combination by ROVERING or ensemble techniques. However, this publication will be post INTERSPEECH 2018.



Frequently asked questions: you might find an answer to your questions here:
  • Q: While using auDeep I get the error message that the parser for a sub-challenge is missing. What should I do?
    A: Please update your local branch whenever a new challenge package is released. git pull https://github.com/auDeep/auDeep
  • Q: Are there scripts to extract the ComParE feature set with openSMILE available?
    A: Please see here.
  • Q: Do I have to include test set results for the paper submission?
    A: Yes. However, you can update these in the final camera ready version. In any case, at least one result submission has to be made before the paper deadline.
  • Q: Can we submit one paper to Interspeech describing participation in several Sub-Challenges?
    A: This is possible - obviously the paper should read well and contain all necessary details, as it needs to pass peer-review as condition for successful Challenge participation.
  • Q: Can we submit more than one paper to Interspeech describing participation in different Sub-Challenges?
    A: This is possible - you can submit, e.g., separate papers for the participation in different Sub-Challenges or for different approaches in the same Sub-Challenge. Each one will undergo peer-review independently.
  • Q: Can we have richer meta-data for the databases, so that we can also look at other things?
    A: Items not available in the current download package will only be available after the Challenge. Then, however, more info will partially be given.
  • Q: My results are below the baseline - does it make sense to submit?
    A: Of course it does. We do not know whether the baseline will be surpassed and different experiences with the tasks on the same dataset will be of interest. Please remember that all submissions to the Challenge go through the normal reviewing process. Although it is very likely that the reviewers do know - and take into account - the baselines, the criteria are the usual, i.e., scientific quality; surpassing any baseline - be this the one given for this challenge, or another one known from the literature - is just one of the criteria. A paper reporting results above the baseline, but poorly written, runs high risks *not* to be accepted; in contrast, a paper which is well written, contributing to our knowledge, but with results below the baseline, has high chances to be accepted.
  • Q: When is the deadline for submitting the results?
    A: You will need to submit results by 11 June 2018 prior to camera ready paper submission to INTERSPEECH as a result on test needs to be included in your final paper version if you want to compete for the Sub-Challenge awards. All except one (which has to be used before the paper deadline on 23 March 2018) of the five result submissions per Sub-Challenge and participant can be saved for submission as late as 11 June 2018.
  • Q: How will the data be distributed to the participants? Are you sending the test data at the same time with training and development partitions?
    A: Yes, we do. However, labels are only given for training and development partitions. Further, we are not sending the data - you will need to download the data. Please first download, print, and sign the license agreements - one per data set once the agreements are available (cf. above) - and scan and mail or fax these to the addresses given on the agreements. You will then receive an email with download instructions.
  • Q: Can I use additional databases within the Challenge?
    A: You can - as long as these are well known and accessible to everybody and you clearly state the details in your paper.
  • Q: May I participate in several Sub-Challenges?
    A: Yes, of course. Every site may participate in every Sub-Challenge and the five upload trials are per Sub-Challenge. In how many Sub-Challenges one site participates is decided at the moment you submit your paper by the results you include. Please send us an additional e-mail to indicate which Sub-Challenges you participate in for us to know.
  • Q: Do I have to submit a paper in order to participate?
    A: Yes, the submission and acceptance of a paper is mandatory. Please make sure to select the special event during the submission procedure as first option. The prizes will be awarded during INTERSPEECH 2018 in Hyderabad/India.
  • Q: May I include results on other corpora in my paper?
    A: Yes, of course. As long as it fits the general focus these are of course very welcome.
  • Q: What are the other formalities with respect to the paper of mine?
    A: Please make sure to reference the official Paper on the Challenge (soon available) and avoid repetition of the general data and Challenge description.
  • Thank you and welcome to the Challenge!



More Information will follow on a regular basis.



Document Actions
Powered by Plone

Portal usage statistics