Communicating and controlling robot arm motion intent through mixed-reality head-mounted displays. (October 2019)

• Record Type: Journal Article
• Authors: Rosen, Eric; Whitney, David; Phillips, Elizabeth; Chien, Gary; Tompkin, James; Konidaris, George; Tellex, Stefanie
• Is Part Of: International journal of robotics research. Volume 38:Number 12/13(2019)
• Page Start: 1513
• Page End: 1526
• Publication Date: 2019-10
• Subjects: Mixed reality -- motion planning -- human–robot interaction; Robots -- Periodicals; Robots, Industrial -- Periodicals; 629.89205
• DOI: 10.1177/0278364919842925
• Languages: English
• ISSNs: 0278-3649

Abstract

Efficient motion intent communication is necessary for safe and collaborative work environments with co-located humans and robots. Humans efficiently communicate their motion intent to other humans through gestures, gaze, and other non-verbal cues, and can replan their motions in response. However, robots often have difficulty using these methods. Many existing methods for robot motion intent communication rely on 2D displays, which require the human to continually pause their work to check a visualization. We propose a mixed-reality head-mounted display (HMD) visualization of the intended robot motion over the wearer's real-world view of the robot and its environment. In addition, our interface allows users to adjust the intended goal pose of the end effector using hand gestures. We describe its implementation, which connects a ROS-enabled robot to the HoloLens using ROS Reality, using MoveIt for motion planning, and using Unity to render the visualization. To evaluate the effectiveness of this system against a 2D display visualization and against no visualization, we asked 32 participants to label various arm trajectories as either colliding or non-colliding with blocks arranged on a table. We found a 15% increase in accuracy with a 38% decrease in the time it took to complete the task compared with the next best system. These results demonstrate that a mixed-reality HMD allows a human to determine where the robot is going to move more quickly and accurately than existing baselines.