Oof, a real banger Master’s from Colette Alexander, exploring the role of Quantitative Risk Assessments (QRA) as fantasy documents and enabling devices in cybersecurity.
Can’t do this justice, so just a few extracts:
- “Even though it is often touted as a value of producing QRA, organizational learning was not an objective for any of the interviewees”
- “As the complexity of a system increases, the accuracy of any single agent’s own model of that system decreases rapidly” (p. 47). Because of this degraded understanding, models of the system or predictions of risk inherent in the system are also flawed”
- “When viewed as an artifact in culture, QRA can be classified as a type of ‘fantasy document’, a term coined by sociologist Lee Clarke who defined a class of documents that were created with the intention of mitigating risk in case of disaster, but which often in practice contain unrealistic assumptions and beliefs that make them useless when disaster actually strikes”
- “QRA is an ‘enabling device’ (Hutchinson et al., 2022). Like a fantasy document, an enabling device creates risk in organizations when people attribute objective truth to it that it does not actually contain”
- “Where fantasy documents focus on mitigating or managing potentially high-risk scenarios, enabling devices are more a part of everyday work. The purpose of an enabling device is to facilitate work – as in, greenlight a construction project or move to the next phase of a plan, by decreasing uncertainty with various partners (internal or external customers, leaders, government institutions, the public) around safety concerns”
- “QRA utilizes expertise to project control over complex and even dangerous technology (Downer, 2013). The transformation of subjective knowledge into numbers serves to soothe uncertainty through its appearance of objectivity, which allows business activity to commence
- Numbers have an appearance of objectivity and can rise above qualitative arguments when advocating for a particular position or activity
- “When uncertainty about key aspects of a task is high, rationalistic plans and rational-looking planning processes become rationality badges, labels proclaiming that organizations and experts can control things that are, most likely, outside the range of their expertise”
- “QRA clearly functions as an enabling device in cybersecurity for every single one of the people who were interviewed”, with the “most common activity QRA enabled for interviewees were budget approval for their cybersecurity organizations”
- “QRA’s enabling power lies in its perceived objectivity”
- “Even when interviewees were aware of the lack of objectivity underlying the production of QRA, they believed that the perception of objectivity by other key decision makers was what ultimately made it a powerful tool”
- Stakeholders “were convinced of its persuasiveness even if they were not all convinced it held objective truth
- While information security officers saw QRA more as a pragmatic tool, risk managers “had higher expectations of the objective truth of their QRAs, feeling that they should approach an accurate, predictive model”
- “Clarke’s warning is that the expression of risks as probabilities, quantified and given the appearance of objectivity, allows risk to be written off as safe, or in the case of cybersecurity: controlled”
- “As a tool to advocate for budget and the acceptance of risk appetite, QRA operates as an enabling device, convincing the organization that approved projects can control against the complex, uncertain and ambiguous world”
- “In a situation where the nuance and lack of objective truth of the numbers is understood by practitioners but not by consumers of the fantasy document, organizations could be set up for dramatic failures that impact their customers and employees”
- “This is a potential side effect of the alchemy of QRAs as enabling devices in cybersecurity: they turn risks, combined with systems knowledge into solid investments, controlled for by seemingly objective expressions of expertise”
Study link: http://lup.lub.lu.se/student-papers/record/9148570/file/9148571.pdf
My site with more reviews: https://safety177496371.wordpress.com
SafetyInsights.org 