calendar | intranet

;

MULTIMODAL HMI

Description : natural multimodal communication in virtual reality, an application for collaborative tasks

Team / Agency :
- I3S Laboratory (Computer Science, Signals, Systems)
- LPL (Speech and Language Laboratory)
- LIA (Computer Science Laboratory of Avignon)
- JENsys Site
- ISM (Institute of Movement Sciences)

Principal Authors : Farre Jacques, Philippe Blache

Year : 2008

Multimodal communication, having long been limited to specific applications, begins to address the question of its treatment in a natural situation. This requires taking into account not only aspects directly related to communication, but also to environment. The problem of multimodal communication is beginning to be better described. However, its implementation in a controlled environment is in its infancy. Virtual reality serves as a privileged testing ground in that it allows you to control most of the environmental parameters while approximating the complexity of natural situations.

The project’s objective is to examine implementation conditions of natural multimodal communication in a type of complex situation (collaborative tasks), under immersion in a virtual reality facility. For this we propose to test natural human-human interaction models that we developed focusing on the gesture-speech interaction.