Personal tools
You are here: Home SIGs and LIGs Ethics SIG ETHI-CA2


ETHI-CA² 2016: ETHics In Corpus Collection, Annotation and Application

ETHI-CA² 2016: ETHics In Corpus Collection, Annotation & Application

An LREC 2016 workshop.

24 May 2016, Portoroz, Slovenia

Workshop Description:

ETHI-CA²’s focus spans ethical aspects around the entire processing pipeline from speech and language as well as multimodal resource collection and annotation to system development and application.
In the recent time of ever-more collection “in the wild” of individual and personal multimodal and –sensorial “Big Data”, crowd-sourced annotation by large groups of individuals with often unknown reliability and high subjectivity, and “deep” and autonomous learning with limited transparency of what is being learnt, and how applications such as in health or robotics depending on such data may behave, ethics have become more crucial than ever in the field of language and multimodal resources making it a key concern of the LREC community. There is, however, a surprising if not shocking white spot in the landscape of workshops, special session, or journal special issues in this field, which ETHI-CA² aims to fill in.
The goal is thus to connect individuals ranging across LREC’s fields of interest s uch as human-machine and –robot and computer-mediated human-human interaction and communication, affective, behavioural, and social computing whose work touches on crucial ethical issues (e.g., privacy, tracability, explainability, evaluation, responsibility, etc.). According systems increasingly interact with and exploit data of humans of all ranges (e.g., children, adults, vulnerable populations) including non-verbal and verbal data occurring in a variety of real-life contexts (e.g., at home, the hospital, on the phone, in the car, classroom, or public transportation) and act as assistive and partially instructive technologies, companions, and/or commercial or even decision making systems. Obviously, an immense responsibility lies at the different ends from data recording, labelling, and storage to its processing and usage.


Emerging interactive systems have changed the way we connect with our machines, modifying how we socialize, our reasoning capabilities, and our behavior. These areas inspire critical questions centering on the ethics, the goals, and the deployment of innovative products that can change our lives and society. - Many current systems operate on private user data, including identifiable information, or data that provides insight into an individual’s life routine. The workshop will provide discussions of user consent and the notion of informed data collection. - Cloud-based storage systems have grown in popularity as the scope of user-content and user-generated content has greatly increased in size. The workshop will provide discussions on best practices for data annotation and storage and evolving views on data ownership. - Systems have become increasingly capable of mimicking human behavior through research in affective computing. These systems have pr ovided demonstrated utility, for interactions with vulnerable populations (e.g., the elderly, children with autism). The workshop will provide discussions on considerations for vulnerable populations. - The common mantra for assistive technology is, “augmenting human care, rather than replacing human care.” It is critical that the community anticipates this shift and understands the implication of machine-in-the-loop diagnostic and assessment strategies.

Topics of Interest:

Topics include, but are not limited to:
  • Ethics in recording of private content
  • Ethics in multimodal, sensorial data collection
  • Ethics in annotation (crowd-sourced) of private data
  • Data storage/sharing/anonymization
  • Transparency in Machine Learning
  • Ethics in Affective, Behavioural, and Social Computing
  • Responsibility in Educational Software and Serious Games
  • Human-machine interaction for vulnerable populations
  • Computer-mediated Human-Human Communicatio n
  • Responsibility in Decision-Support based on Data
  • The role of assistive technology in health care

Summary of the Call:

The ETHI-CA² 2016 workshop is crucially needed first edition in a planned for longer series. The goal of the workshop is to connect individuals ranging across LREC’s fields of interest such as human-machine and –robot and computer-mediated human-human interaction and communication, affective, behavioural, and social computing whose work touches on crucial ethical issues (e.g., privacy, tracability, explainability, evaluation, responsibility, etc.). These areas inspire critical questions centering on the ethics, the goals, and the deployment of innovative products that can change our lives and consequently, society. It is critical that our notion of ethical principles evolves with the design of technology. As humans put increasing trust in systems, we must understand how best to protect privacy, explain what information the systems record, the implications of these recordings, what a system can learn about a user, what a third party could learn by gaining access to the data, changes in human behavior resulting from the presence of the system, and many other factors. It is important that technologists and ethicists maintain a conversation over the development and deployment lifecycles of the technology. The ambition of this workshop is to collect the main ethics, goals and societal impact questions of our community including experts in sociology, psychology, neuroscience or philosophy. At LREC 2016, the workshop shall encourage a broad range of its community’s researchers to reflect about and exchange on ethical issues inherent in their research, providing an environment in which ethics co-evolve with technology.


  • Deadline for 1500-2000 words abstract submission: 29 February 2016 (extended)
  • Notification of acceptance: 17 March 2016 (new)
  • Final version of accepted paper: 24 March 2016 (extended)
  • Workshop: 24 May 2016.


09:00 – Introduction
Laurence Devillers

09:10 – Keynote (Chairperson: Laurence Devillers)
Edouard Geoffrois
Interactive System Adaptation: Foreseen Impacts on the Organisation and Ethics of System Development

09:50 – Talk 1 (Chairperson: Björn Schuller)
Teresa Scantamburlo and Marcello Pelillo
Contextualizing Privacy in the Context of Data Science

10:10 – Talk 2 (Chairperson: Björn Schuller)
Kevin Bretonnel Cohen, Karen Fort, Gilles Adda, Sophia Zhou and Dimeji Farri
Ethical Issues in Corpus Linguistics And Annotation: Pay Per Hit Does Not Affect Effective Hourly Rate For Linguistic Resource Development On Amazon Mechanical Turk

10:30 – Coffee and Poster Session (Chairperson: Laurence Devillers)
Jana Diesner and Chieh-Li Chin
Gratis, Libre, or Something Else? Regulations and Misassumptions Related to Working with Publicly Available Text Data

Wessel Reijers, Eva Vanmassenhove, David Lewis and Joss Moorkens
On the Need for a Global Declaration of Ethical Principles for Experimentation with Personal Data

Agnes Delaborde and Laurence Devillers
Diffusion of Memory Footprints for an Ethical Human-Robot Interaction System

Björn Schuller, Jean-Gabriel Ganascia and Laurence Devillers
Multimodal Sentiment Analysis in the Wild: Ethical considerations on Data Collection, Annotation, and Exploitation

Jocelynn Cu, Merlin Teodosia Suarez and Madelene Sta. Maria
Subscribing to the Belmont Report: The Case of Creating Emotion Corpora

Lucile Bechade, Agnes Delaborde, Guillaume Dubuisson Duplessis and Laurence Devillers
Ethical Considerations and Feedback from Social Human-Robot Interaction with Elderly People

11:10 – Talk 3 (Chairperson: Joseph Mariani)
Joss Moorkens, David Lewis, Wessel Reijers and Eva Vanmassenhove
Language Resources and Translator Disempowerment

11:30 – Talk 4 (Chairperson: Joseph Mariani)
Simone Hantke, Anton Batliner and Björn Schuller
Ethics for Crowdsourced Corpus Collection, Data Annotation and its Application in the Web-based Game iHEARu-PLAY

11:50 Workshop Closing

Author Information

1500-2000 words extended abstracts are needed at first for submission. The full papers will be published as workshop proceedings along with the LREC main conference by ELRA. For these, the instructions of LREC need to be followed. The submission will be via the START conference system - a link will follow shortly.

A journal special issue in a related journal is planned for inviting the best papers, but remaining open to further submissions.

Further, a best paper award will be given.

Submissions of extended abstracts need to be made at

Organizing Committee:

  • Laurence Devillers, LIMSI-CNRS/Paris-Sorbonne University, France,
  • Björn Schuller, Imperial College London, UK/University of Passau, Germany,
  • Emily Mower Provost, University of Michigan, USA,
  • Peter Robinson, University Cambridge, UK,
  • Joseph Mariani, IMMI/LIMSI-CNRS/University Paris-Saclay, France,

Contact: Laurence Devillers

Program Committee:

  • Gilles Adda, LIMSI-CNRS, France
  • Jean-Yves Antoine, University of Tours, France
  • Rafael Banchs, I2R, Singapur
  • Nick Campbell, TCD, Ireland
  • Raja Chatila, ISIR, France
  • Alain Couillault, GFII, France
  • Anna Esposito, UNINA, Italy
  • Karen Fort, Université Paris-Sorbonne, France
  • Jean-Gabriel Ganascia, UPMC, France
  • Alexei Grinbaum, CEA, France
  • Hatice Gunes, Queen Mary University of London, UK
  • Dirk Heylen, University of Twente, Netherlands
  • François Hirsch, INSERM, France
  • Christophe Leroux, CEA, France
  • Catherine Tessier, ONERA, France
  • Isabelle Trancoso, INESC, Portugal







Document Actions
Powered by Plone

Portal usage statistics