Background
- Director Principal Researcher Takashi Matsuyama (Graduate School of Informatics, Kyoto University)
- Principal Researcher Atsushi Iriki(Section of Cognitive Neurobiology, Department of Maxillofacial Biology, Tokyo Medical and Dental University)
- Principal Researcher Masato Sasaki(Graduate school of Interdisciplinary lnformation Studies/Graduate, School of Education, The University of Tokyo)
- Principal Researcher Yasuo Kuniyoshi(Graduate School of Information Science and Technology, The University of Tokyo)
The figure shown below illustrates the social structure in the 21st century, where our every day life activities are conducted in two different types of societies:
- Physical Real World :
- This is the society where we, human beings, breath, eat, walk, sleep, and so on with other objects and living things.
- Cyber Network Society :
- This is the society that has been established these ten years on top of information network systems, where important social activities such as economical business, education, entertainment, communication and so on are being conducted.
Figure1: The social structure in the 21st century
With the rapid growth of the cyber network society, gaps between these two societies have been incurring serious problems.
One way to fill the gaps would be to integrate these two societies with real-time bi-lateral multimodal information processing, which includes
- Object Digitization :
- Multimodal sensing and recognition of objects and human activities in the physical real world and
- Media Rendering :
- Multimedia rendering and presentation of digital data in the cyber network society.
By conducting these two types of processing in real time, data in the cyber network society and situations in the physical real world are cross-checked to keep mutual integrity.
To make the integration smooth and casual, dynamic multimodal interaction / communication protocols between human beings in the physical real world and intelligent machines in the cyber network society are crucial as well as the above two media processing capabilities: at what timing pauses are introduced, at what speed responses are given, how reactive and proactive controls are conducted, and in what media information is represented.
