Versatile Image Generation by Cooperative Observation Stations

In remote conference and lecturing systems as illustrated in Fig. 18, people at different locations should share virtual and/or real spaces and temporal events as well as multi-media information stored in databases. CDV systems offer fundamental platforms for such application systems. For example, those motion understanding methods described in Section 4.3 can be used to capture interesting and useful images for remote conference and lecturing: tracking a speaker and a teacher.

  
Figure 18: Remote lecturing system with CDV.

In the project we are studying the following technical issues for real time versatile image generation by cooperative observation stations.

Virtual 3D Walk Through by Integrating Multiple APS Images

:
An APS image represents an omnidirectional view of a 3D scene from one viewpoint (Fig. 11), so that all images of the scene viewed from that viewpoint can be generated from the single APS image. Thus, 3D virtual walk through can be realized by integrating multiple APS images taken at different locations.

Image Completion Using Object Models

:
Since the view of an observation station is limited, it often happens that some portions of an object cannot be seen due to occlusion. When the object model is given a priori, a complete image of the object can be generated by integrating an observed image and the object model (Fig. 19). Note that the former may be incomplete but it is live and natural. The latter, on the other hand, can be complete but may be obsolete. The integration of such complementary information also helps to realize the 3D virtual walk through.

Optimal Camera Layout

:
Where to place a group of cameras is an important problem to take vivid and interesting video images. Fig. 20 illustrates three basic camera layouts:

1. Parallel View: accurate stereo, tracking a moving object
2. Convergent View: accurate 3D object modeling, focus of attention
3. Divergent View: environment monitoring, panoramic view generation

  
Figure 19: Image completion using an object model.

  
Figure 20: Basic camera layouts.

Utilities of the camera layout changes depending on the size and shape of the space covered by cameras, object behaviors, and tasks. Conversely, given such information a priori, we can find the optimal camera layout. In TV Studios and theaters, for example, a group of cameras are set up to take attractive images based on scenarios. We are studying optimal camera layout algorithms for such applications.

Context Dependent Dynamic Camera Switching and Motion Control

:
As is well known, video images without editing are not interesting, appealing, or artistic. That is, to take attractive video images, we should conduct dynamic camera switching and motion control as well as the camera layout. We are analyzing movie videos to classify and understand types of camera works employed and their effectiveness. Based on this analysis we will be able to design a camera work description language, with which multiple cameras are dynamically controlled to capture attractive video images.