AAMAS 2004 WS - Perception in ECAs

WORKSHOP ANNOUNCEMENT AND CALL FOR PAPERS

Balanced Perception and Action in ECAs http://www.iut.univ-paris8.fr/~pelachaud/AAMAS04/

In conjunction with

AAMAS 2004 Workshop http://www.aamas-conference.org/

DESCRIPTION: ON THE ROLE OF PERCEPTION IN ECAS Conversation is by no means a sequence of one-way message sending, with the role of participants alternating between that of “sender” and “receiver”. Rather, the conversants adapt continuously to each other during the conversation: what a participant says, and how and when she says it, depends on her perception of the others’ behaviour, their (perceived and inferred) mental and communicational capabilities and preferences, attention, understanding, emotional state, etc. Conversation is thus a two-way street of multiple message types, no matter who has the turn. From a technical point of view, face-to-face conversation can be described as layered feedback loops, each at various time-scales. As a result, the roles of “speaker” and “listener” are much more hard to distinguish than may seem at first: While speaking, the one who has the turn is also a “receiver”, paying close attention to the listener, who continuously produces paraverbals such as “aha”, “hmhm”, non-verbal behaviors such as gaze, facial expressions, signals about willingness to take the turn, interrupt, etc., and other actions relevant to the progression of the dialogue, all of which is synchronized to the actions of the other participant to the decisecond. Perception, in addition to motor skill, is thus critical in organizing movement and communication. The topic of the workshop is interaction between humans and Embodied Communicative Agents (ECAs) where production of output is interwoven with perception. Creating a real-time communication loop implies consideration of information provided to one or more generation modules by one or more perception modules. A well-tuned multi-layered communication loop allows an ECA to perceive the user’s state, action and environment, and use these factors to decide what, how, and when to present communicative content and behavior. In the workshop we do not wish to address (the techniques of) perception per se, but rather how perceptual data should be used to enhance the interaction, and how it should be used to feed into different levels of the ECA’s behaviour, like dialogue management, language usage, emotional reaction, manual, facial and body gestures, and so on. Specific robust vision, speech and language techniques, especially suited and/or used for ECAs are of course of interest. Perception of the user’s characteristics is viewed in the broadest sense, including perception of presence, both static information (age, gender, culture, personality) and dynamic characteristics such as emotional and dialogue state (misunderstanding, repetition, etc.). Perception of the environment and of non-conversational parties may also be relevant, as well as the monitoring of the agent’s own gesture and facial expression, when produced consciously in order to achieve a special effect. Besides the traditional perception-action loop, mixed and virtual reality scenarios are of interest too, and can both complicate and simplify the set-up.

WORKSHOP AIMS For the workshop we wish to bring together researchers and developers of embodied conversational characters with different background and expertise, to exchange ideas and experiences on the various aspects involved in the creation of ECAs in the loop: - linguistic - speech (voice, intonation) - cognitive, personality and emotion models - nonverbal communication - animation - graphics - vision - dialogue capabilities and social interaction with other agents Contributions from related fields like cognitive science and psychology (e.g. on guidelines for models of adaptivity) are welcome, as well as presentation and evaluation studies of ECAs which operate with complex perception-action loops.

MAJOR TOPICS Issues to be addressed: - Perception and adaptation in human-human communication - Integration and interwoven usage of multiple input and output modalities - Perception techniques used in, or specially suited for ECA applications (being robust, fast, cheap as of hw required) - Models integrating perception of the user: physical presence, reaction, behaviour, inferred communicative intent, as well as cognitive and emotional states - Adaptation to the user (age, gender, language, culture, personality, …) - Adaptation to the operational environment (noise, public place) - Evaluation studies of/for perceptive and adaptive ECAs

WORKSHOP FORMAT The full-day workshop will give the opportunity to present papers, show demos, and discuss issues in scheduled panel sessions.

SUBMISSION FORMAT Since we aim at a highly focused event on balanced perception and action in ECAs, we kindly ask authors to prepare their submissions according to the following structure: 1. Brief description of application area character type(s) and research goals; 2. Description of their requirements for an ECA in the loop; 3. Description of the actual aspects of perception being used in their system(s); 4. Report on experiences and on evaluation studies with the creation of a particular agent in the loop – what are the useful features, what are the necessary steps toward such a creation; 5. Other aspects relevant to the topic of this workshop.

Papers length should be 4 to 8 pages long (using 11pt, single space, all margins of 2cm) and should be accompanied as much as possible with an animation or URL’s showing multimedia content (actual systems, screenshots, animations, etc) describing the work presented. Submissions should be emailed to Catherine Pelachaud (c.pelachaud@iut.univ-paris8.fr) in PS or PDF format by April 4th, 2004. If email is not possible, please sent two copies of your paper to (though, email is much preferred):

Catherine Pelachaud IUT of Montreuil University of Paris 8 140 rue de la Nouvelle France 93100 Montreuil France

TIMELINE April 17 Deadline for submissions of contributions May 10 Authors notification May 30 Submission of camera ready contribution for the workshop notes July 19/20 Workshop

ORGANIZING COMMITTEE Primary Contact: Catherine Pelachaud (LINC, IUT de Montreuil - University of Paris 8, 140 rue de la Nouvelle France, 93100 Montreuil, France) c.pelachaud@iut.univ-paris8.fr

Zsofia Ruttkay (Center for Mathematics and Computer Science, INS2.2, Holland) ruttkay@cwi.nl

Kris Thórisson, Communicative Machines kris@media.mit.edu

PROGRAM COMMITTEE Elisabeth Andre (DE) Norman Badler (USA) Antonio Camurri (I) Doug De Carlo (USA) Sylvie Gibet (F) Bjorn Granstrom (SW) Katherine Isbister (USA) Stacy Marsella (USA) Jean-Claude Martin (F) Catherine Pelachaud (F) Isabella Poggi (I) Helmut Prendinger (J) Thomas Rist (DE) Zsofia Ruttkay (NL) Candy Sidner (USA) Mark Steedman (UK) Matthew Stone (USA) Kris Thórisson (Iceland)