Automation reliability, human–machine system performance, and operator compliance: A study with airport security screeners supported by automated explosives detection systems for cabin baggage screening. (July 2020)

• Record Type: Journal Article
• Authors: Huegli, David; Merks, Sarah; Schwaninger, Adrian
• Is Part Of: Applied ergonomics. Volume 86(2020)
• Publication Date: 2020-07
• Subjects: Human–automation interaction -- Airport security baggage screening -- Visual search; Human engineering -- Periodicals; 620.82
• DOI: 10.1016/j.apergo.2020.103094
• Languages: English
• ISSNs: 0003-6870

Abstract

Abstract: Using a simulated X-ray screening task, we tested 122 airport security screeners working with the support of explosives detection systems for cabin baggage screening (EDSCB) as low-level automation. EDSCB varied systematically on three automation reliability measures: accuracy, d', and positive predictive value (PPV). Results showed that when unaided performance was high, operator confidence was high, and automation provided only small benefits. When unaided performance was lower, operator confidence was lower, and automation with higher d' provided large benefits. Operator compliance depended on the PPV of automation: We found lower compliance for lower PPV. Automation with a high false alarm rate of 20% and a low PPV of .3 resulted in operators ignoring about one-half of the true automation alarms on difficult targets—a strong cry-wolf effect. Our results suggest that automation reliability described by d' and PPV is more valid than using accuracy alone. When the PPV is below .5, operators should receive clear instructions on how to respond to automation alarms. Highlights: Benefits of low-level automation depend on unaided performance and automation reliability. Automation reliability measures are accuracy, d', and the positive predictive value (PPV). When unaided performance is low, higher automation d' results in better target detection. Operator compliance depends on the PPV of automation. Operators should be given clear instructions if the PPV of automation is below 0.5.