Method to create realistic virtual worlds from 360-degree videos.

What is it about?

Creating realistic virtual worlds is very difficult for ordinary people because it requires specialized skills and knowledge about 3D models. In addition, 3D models are far from reproducing real objects. Therefore, we propose a system to automatically create realistic virtual worlds from 360-degree videos. In this world, users can explore via their own avatars.

Featured Image

Why is it important?

Our system detects intersections from videos and divide them by intersections. Changing videos in each intersection enable users to explore the area recorded in videos. When using videos for drawing realistic background, there is no 3D model and collision. Therefore, avatars can be easily superimposed on buildings. In our system, determine movable areas and unmovable areas by semantic segmentation. Thanks to this, we can set pseudo collision to the videos and prevent avatars from overlaying buildings.