High variability in Summative Usability Test methods & reporting among clinical informatics vendors complying with Federal certification requirements. (September 2014)

• Record Type: Journal Article
• Authors: Buchanan, Christine; Threatt, Anthony; Weinger, Matthew B.; Miller, Anne
• Is Part Of: Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting. Volume 58:Part 1(2014)
• Page Start: 1491
• Page End: 1495
• Publication Date: 2014-09
• Subjects: Human engineering -- Congresses; 620.8205
• DOI: 10.1177/1541931214581311
• Languages: English
• ISSNs: 1541-9312

Abstract

The Office of the National Coordinator for Health Information Technology (ONC) has been tasked with managing the integration of usability best practices into electronic health record development but, has provided little guidance on how to conduct Summative Usability Testing (SUT). We reviewed three aspects of SUT in four publicly available SUT vendor reports. Significant deviations from human factors engineering (HFE) and user-centered design (UCD) best practices and clear methodological errors were found in all four reports. The main sources of variation involved non-representative participant selection, instructions that led participant performance, and varied interpretation of results reporting requirements. Although ONC has made significant gains in implementing best practice UCD approaches, more explicit guidance may be needed to enhance integrity and reduce variation.