Predictive display for teleoperation with virtual reality fusion technology. Issue 5 (1st October 2021)

• Record Type: Journal Article
• Authors: Fu, Yongqing; Lin, Weiyang; Huang, Junsong; Gao, Huijun
• Is Part Of: Asian journal of control. Volume 23:Issue 5(2021)
• Page Start: 2261
• Page End: 2272
• Publication Date: 2021-10-01
• Subjects: feature point detection -- predictive display -- teleoperation -- virtual reality fusion; Automatic control -- Periodicals; Control theory -- Periodicals; 629.805
• DOI: 10.1002/asjc.2690
• Languages: English
• ISSNs: 1561-8625

Abstract

Abstract: A virtual reality (VR) fusion technology of matching a virtual environment of simulated graphics models with actual camera views of the slave station task environment has been developed. This fusion technology enables high‐fidelity predictive displays with calibrated graphics overlay on the live video. Reliable and accurate parameters are achieved by camera calibration and feature point detection. Since the object pose becomes known through the virtual model, the operator can use the teleoperation system effectively. Predictive displays are very useful for both noncontact and contact tasks, providing an effective VR interface with immediate visual prediction to the operator. Besides, the feature points detection avoids the exacerbation of maneuverability caused by outside interference. Lastly, the experiment of endoscopic penetration is conducted to validate the system.