How objective are quantitative risk assessments and how subjective are qualitative assessments?
This image is another example from a presentation of mine, highlighting a nice quote from Gerd Gigerenzer.
He aptly observes the lack of critical evaluation of quantitative risk models, statistics, predictive algorithms etc.
Our unwavering support of models is partially reflected by what John Downer called out as the “ideal of mechanical objectivity”. That is, as John notes, the belief that “the math is the math” and models and algorithms are calculations, not opinions and as such are relatively immune from subjectivity.
We have a lot of research that challenges this view. Organisations like NASA or BP learnt this lesson the hard way in their use of particular software models (CRATER model for NASA which was designed to assist in estimating potential damage to the tiles from impacts or via the OptiCem software used to help model safe well design parameters in the case at Macondo).
Although there’s the argument that the software were appropriate to be used to assist decision making in conjunction with other means (including engineering judgements) and the software was rather misused, the Macondo example highlights how risk estimations can be tweaked until they produce the answer that we were either expecting or hoping for.
Starbuck & Milliken (1988) called this “fine-tuning the odds until something breaks”.
As we argued in our 2018 conference paper, many aspects of organisational practice (including risk systems) may give the appearance of being based on objective truths—contrasted against the subjective aspects of judgements and expert knowledge. But the dichotomy is a myth: the incontrovertible data and technical conclusions mask the underlying ambiguities and social processes that shape these methods (Wynne, 1988).
The “hard data” used in QRA is still based on subjective/intersubjective “justified beliefs” (Aven, 2017, p. 255). In times of high uncertainty, use of QRA may entice analysts to model information not present in the source data (Aven, 2017).
Link to the LinkedIn post: https://www.linkedin.com/posts/benhutchinson2_how-objective-are-quantitative-risk-assessments-activity-6907828069936283648-plBB?utm_source=share&utm_medium=member_desktop
SafetyInsights.org 