post-doc multimodal dialog

We are looking for a post-doc to develop a multimodal dialog system. The research takes place within the ANR project MyBlog-3D.

_Context:_
Virtual worlds are developing rapidly over the internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person while ECAs are autonomous 3D human-like entities. Each person controls one or several avatars and receives feedback from the virtual world on an audio-visual display.

In the project MyBlog-3D, the avatar’s animation is driven by the user’s behaviour in an unobtrusive manner. A small webcam detects the arm movements as well as the facial expressions a user displays. These detected motions are reproduced in real-time onto the avatar. On the other hand, the ECA animation is driven by a multimodal dialog system.

_Work to be done:_
Within MyBlog-3D we are developing a keyword spotting algorithm. Detecting keywords allows us to get ‘important’ information for the ECA to have a ‘sense’ of what is being said. Doing a semantic analysis is out of reach for a real-time application and when the discourse domain is simply too vast. Keyword spotting allows us to get some information related to the conversation.

The aim of the present research is to develop a multimodal dialog system that would allow the ECA to answer to the user. The aim is to develop a template-based dialog system. As input the system will take the recognized keywords. The output of the dialog is communicative intentions. These intentions are then converted into multimodal nonverbal behaviours that are sent to an ECA system.

*Required knowledge*: C++, dialog system
*Location*: ENST – Paris
37/39, rue Dareau
75014 Paris
*Project length*: 12-18 month
*Stipend*: around 2000 euros per month
*Contact*: send a CV to
catherine.pelachaud@inria.fr
http://www.iut.univ-paris8.fr/~pelachaud/