High‐resolution 360 Video Foveated Stitching for Real‐time VR. (13th October 2017)

• Record Type: Journal Article
• Authors: Lee, Wei‐Tse; Chen, Hsin‐I; Chen, Ming‐Shiuan; Shen, I‐Chao; Chen, Bing‐Yu
• Is Part Of: Computer graphics forum. Volume 36:Number 7(2017)
• Page Start: 115
• Page End: 123
• Publication Date: 2017-10-13
• Subjects: Categories and Subject Descriptors (according to ACM CCS) -- I.3.3 [Computer Graphics]: Picture/Image Generation—Line and curve generation; Computer graphics -- Periodicals; 006.605
• DOI: 10.1111/cgf.13277
• Languages: English
• ISSNs: 0167-7055

Abstract

Abstract: In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real‐world scene. Although many capture devices are being released, getting high‐resolution panoramas and displaying a virtual world in real‐time remains challenging due to its computationally demanding nature. In this paper, we propose a real‐time 360° Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high‐resolution panoramic Video in real‐time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.