Testing the impartiality of surveys to measure differential risk perception. (January 2015)

• Record Type: Journal Article
• Authors: Thomas, P.J.
• Is Part Of: Measurement. Volume 60(2015:Feb.)
• Page Start: 155
• Page End: 173
• Publication Date: 2015-01
• Subjects: Differential risk perception -- Structural view independence -- Jensen's inequality -- Valuation Index -- Second Valuation Index -- Rail Safety and Standards Board; Weights and measures -- Periodicals; Measurement -- Periodicals; Measurement; Weights and measures; Periodicals; 530.8
• DOI: 10.1016/j.measurement.2014.10.001
• Languages: English
• ISSNs: 0263-2241

Abstract

Graphical abstract: How the Valuation Index is biased in favour of certain views. Weighting of view, π k, versus the number of deaths in scenario A judged to be as bad as 10 deaths in scenario B, example of Section 3. Highlights: UK safety policy on severe railway accidents is based on opinion measurements filtered through the Valuation Index. The paper tests the Valuation Index against the criterion of structural view independence. The Valuation Index fails the test and is therefore a biased statistic. An equally invalid Second Valuation Index can be constructed using equivalent logic. Cutting spending against fatalities in severe railway accidents by a factor of three since 2003 is brought into question. Abstract: The paper addresses the validation of opinion measurement by survey, specifically how the "readings" or views are consolidated for policy use into a single figure held to be representative of the population as a whole. While the sample mean might seem the obvious choice, a number of influential safety studies have employed a more complicated metric, the Valuation Index, as discussed within, to assess whether more should be spent to protect against some hazards than others. The question arises as to whether the Valuation Index treats the views of the different people in the survey sample impartially or not. The Valuation Index possesses the property of reciprocality, in the sense that the Index of the reciprocal is equal to the reciprocal of the Index. Jensen's inequality reveals that this reciprocality comes at the price of an inconsistent treatment of the individual's view whenever there is any difference of opinion in the sample. The Valuation Index is found to be non-unique, in that a Second Valuation Index can be constructed using equivalent arguments. Thus if the Valuation Index is valid, then so must be the Second Valuation Index, but the latter returns a different value, implying a contradiction that can be resolved only by regarding neither as valid. Crucially, both the Valuation Index and the Second Valuation Index fail the test of structural view independence. Both indices are biased and thus neither is a valid measure for consolidating human opinions. The finding has particular importance for the UK because opinion surveys interpreted using the Valuation Index have been influential in the changes to safety policy brought in for UK rail transport, where the amount recommended to avert death in multiple fatality accidents since 2003 has been reduced to about a third of what it used to be.