Are accident analysis methods fit for purpose? Testing the criterion-referenced concurrent validity of AcciMap, STAMP-CAST and AcciNet. (December 2021)

• Record Type: Journal Article
• Authors: Hulme, Adam; Stanton, Neville A.; Walker, Guy H.; Waterson, Patrick; Salmon, Paul M.
• Is Part Of: Safety science. Volume 144(2021)
• Publication Date: 2021-12
• Subjects: Accident analysis -- AcciMap -- STAMP-CAST -- AcciNet -- Validity; Industrial accidents -- Periodicals; Accident Prevention -- Periodicals; Safety -- Periodicals; Travail -- Accidents -- Périodiques; 363.11
• DOI: 10.1016/j.ssci.2021.105454
• Languages: English
• ISSNs: 0925-7535

Abstract

Highlights: Reliability and validity testing of accident analysis models and methods is rarely undertaken in HFE and safety science despite its recognised importance. We tested and compared three systems-based accident analysis methods in terms of their capacity to accurately model the cause of a fatal vehicle-pedestrian collision incident. The Signal Detection Theory (SDT) paradigm which accounts for true positives, false negatives, false positives and true negatives was used to compute the validity results. Systems-based accident analysis methods did not produce satisfactory validity results in this study compared to a referent expert analysis. Abstract: Systems-based methods such as AcciMap, STAMP-CAST and more recently AcciNet, are used to uncover the multifactorial cause of accidents within complex systems to support the development of new safety interventions. Despite this, there has been a lack of research that has formally examined the reliability and validity of systems-based accident analysis methods. This gap is addressed in this study. The criterion-referenced concurrent validity of AcciMap, STAMP-CAST and AcciNet was compared in a repeated measures design using Signal Detection Theory (SDT). A process of analytical fragmentation was adopted to understand how the individual phases of each method contributed to their overall validity. The results of the overall analyses indicate that the three methods achieved comparable and moderate results. It is concluded that state-of-the-art systems-based accident analysis methods did not produce satisfactory validity results in this study compared to a referent expert analysis. Further research needs to be conducted to demonstrate the efficacy of systems-based accident methods if they are to be used with confidence and work as expected.