Special session on 'Emerging Multimodal Interfaces' during the Artificial Intelligence Applications and Innovations conference

Recent research results in the area of multimodal interfaces have been focusing on the development of natural, adaptive and intelligent interfaces enabling machines to communicate with humans in ways much close to the way humans communicate among themselves. Among the characteristics of such interfaces is that they respond to speech and language, vision gesture, haptics and other senses. Moreover, they enable interaction between and among humans and the virtual and physical environment, through intuitive multimodal interfaces that are autonomous and capable of learning and adapting to the user environment in dynamically changing contexts. Furthermore, they recognize emotive user reaction and have robust dialogue capabilities with unconstrained speech and language input.

In order to augment natural interactivity between humans and the physical or virtual environment, research is now carried out towards autonomous interfaces that are capable of learning and adapting to user intentions and behavior. Emphasis is also put on multimodal interfaces that feature unconstrained, robust and ergonomic interaction, recognize user reactions and respond to them in the most natural way. Natural interaction is facilitated through appropriate selection and combination of modalities taking into account user’s preferences and context. Combining modalities hinges on innovative information fusion mechanisms, which include efficient synchronization schemes. The later are required given the real-time and diverse nature of sensor and contextual information streams. The overall goal of multimodal interfaces research is to enable a new wave of ubiquitous, pervasive, context-aware and human centric services. Such services are likely to incorporate integrated and multidisciplinary interface systems, which can cognitively adapt to that status of users and the surrounding environment. The European Commission (EC) has greatly boosted this research in the scope of the 1^st and 5^th IST calls, which included strategic (research) objectives on ‘Multimodal Interfaces’. The three projects supporting this workshop, namely CHIL, HUMAINE and SIMILAR, are among the most prominent EC co-funded projects in the area of Multimodal Interfaces.

• Robust Multimodal Audio-Visual Processing for Advanced Context Awareness in Smart Spaces
• Toward supporting group dynamics
• Multimodal Integration of Sensor Network
• Multimodal Identity Tracking in a Smartroom
• Multimodal Focus Attention and Stress Detection and feedback in an Augmented Driver Simulator
• A Fuzzy Expert System for the Early Warning of Accidents Due to Driver Hypo-Vigilance
• Mixed Reality Cane Simulation
• 3D content-based search using sketches
• Manual Annotation and Automatic Image Processing of Multimodal Emotional Behaviors in TV Interviews
• MPEG-4 Facial Expression Synthesis based on Appraisal Theory
• Towards On- and Off-line Search, Browse and Replay of Home Activities