Creating Metrics for Human-Agent Teams. (December 2020) - Cavanah, Elizabeth; Ford, Zachary; Jasper, Angelica; Stonewall, Jacklin; Gilbert, Stephen B.; Dorneich, Michael
This paper describes the requirements and overall design of a testbed for a human-agent team. A robust, flexible testbed will enable researchers to evaluate the effectiveness of human-agent teaming concepts and issues. In the testbed, MazeWorld, multiple agents play different roles in which tasks are interdependent. Each role can be served by a human or an autonomous agent. Metrics were developed to capture individual and team effectiveness and allow researchers to compare different types of teams and teamwork protocols. Examples of the current task design are presented with the discussion of the future development of the system.