SafetyInsights.org

This study used a content analysis approach to compare management stakeholders’ perspectives on procedures versus that of operators, with a focus to reveal differences between work-as-imagined (WAI) and work-as-done (WAD).

Both groups were asked questions like: what percentage of the time they thought operators departed from procedures, reasons for procedural departures, definitions of what a high quality procedure was and some other stuff. 11 operators and 11 stakeholders were involved in the interviews.

Some background on WAI/WAD and procedure abstraction is described in the paper. SOPs are usually intended to be written prescriptions of how work is expected to be performed. In reality though, they are often “abstract and often decontextualized expectations of work” where “the writers of the procedures create SOPs based on WAI” (p1350).

WAI often reflects more stable assumptions around the work environment and can “ignore the nuances introduced by the users of SOPs and their work environments” (p1350). Moreover, because it’s near impossible for SOPs to factor in an individual’s cognitive and physical capabilities, the abstraction of work in WAI “can reduce the effectiveness of SOPs in in non-idealized contexts” (p1351).

The paper also discusses the model 1 / model 2 framework of procedures. Model 1 takes a more static view of compliance to SOPs, where adherence to SOPs developed in advance are seen as the ideal method to control risks, and minimising deviations is a key goal. It’s more prescriptive and more closely aligned to WAI. Model 1 has and does undoubtedly lead to improvements in safety. Model 2 in contrast, is a more dynamic interpretation of rules as resources for actions. These resources help guide and extend cognitive capabilities and are more embedded in WAD.

Although both views have their place, previous research hinted that deviations from procedures were not the most strongly correlated factors for incidents & near misses. A stronger predictor in these studies was perceived procedure quality rather than procedure departure.

[Note: the model 1 / model 2 view isn’t Safety-I and Safety-II.]

Too much to cover here, so just a few points.

Results
It was found that, overall, management stakeholders had a view of procedures that tended towards a model 1 view – where procedures are high quality and describe an ideal way of working, whereas model 2 was more common for operators. While there were several areas of alignment between the groups, there were also some key differences.

For stakeholder interpretations, a large variation was evident. Some thought that there was more or less 100% adherence to procedures by operators, but most recognised at least some departures. A common estimation was around 90-95% adherence to procedures.

Similar ranges were evident for operators. Interestingly, operators often said little to no procedural departures for them, but somewhere around 75-80% for others, particularly the experienced operators.

Departures were likely to occur with external or situational variables where the SOP can’t be completed as written.

For reasons why, stakeholders cited complacency, repetition, efficiency, experience and training as factors. I’ve skipped most of the discussion here but many of the factors appear to be internalised factors relating to the operator, although some did indicate external pressures contributing.

Operators provided similar reasons to the above and particularly emphasised the role that routine and repetition played. One notable point was the role redundancy played, as many procedures were seen to involve excess redundancy, so some things could be skipped. A drive for efficiency was also a key reason, similarly to the reason cited by stakeholders. Lack of efficiency was a perceived challenge with SOPs.

Of interest is that operators didn’t cite external pressures.

Relating to the quality of procedures, stakeholders focused on ensuring adherence to procedures whereas operators cared more about the level of detail for each step in the process. One operator noted that if procedures were followed exactly as they were written, the process would become too inefficient.

At the task level, stakeholders were interested in the tasks being understandable whereas operators were concerned with having more autonomy and also in procedures having more accurate content, focused on task goals, and provided some clarity on safety and hazards. The latter there at prompts for some operators at certain tasks.

For reasons not to report, something unique to the operator feedback was a reluctance to report necessary SOP amendments. Other reasons were the well-known ones, like perceiving that management doesn’t care, blame, escalation is futile, reporting is hard to do, reporting stalls completion and others.

The cited factors by operators link to the climate of the environment, where “operators may not believe that their leadership is highly committed to safety” (p1353).

Overall, it’s said that stakeholder and operator responses were fairly aligned but there were some notable differences. Because of the varying estimations of deviations in inter- and intra-individual situations, it’s said that “deviations in WAD are pervasive” (p1353).

Also, since the responses were fairly well aligned between stakeholders and operators, it’s concluded that the stakeholders that write procedures are at least aware of the gap (to some degree).
There were more differences between the groups when it came to how each group defined high quality procedures. Both groups wanted the work to be performed safely, correctly and efficiency, they differed on how to achieve that.

Stakeholders “clearly preferred a high level of control and adherence over the completion of each step, and wanted the overall tasks to be understandable, which closely adheres to the Model 1 perspective” (p1353). For operators, “step-level usability was more important for operators who focused on how much detail each step contained and were concerned with more autonomy, efficiency, clarity, accuracy, and safety at the task-level. These task-level desires reveal that operators desire more Model 2 flexibility and increased voice in the procedures they use daily” (p1353).

Calls from operators for more voice and agency was said to be contrasted with their “apathetic perceptions” of how the procedure amendment process worked. This indicated varying levels of their understanding how it worked. Stakeholders, as expected, could provide far more detail on the review process. For barriers to reporting issues, operators noted perceived barriers like being blamed, seen as incompetent or reporting may delay completion of the task. So, despite operators wanting more voice and agency, the authors state that “the costs and the perceived lack of benefits to reporting (e.g., management apathy, futility of reporting), appear to discourage operator engagement in the review and revision of procedures” (p1353).

In concluding, it’s said that understanding WAI and WAD should be a critical concern for any high-risk organisation. Whereas stakeholders stressed that anybody has the authority to report an issue to management, operators “appear to believe that stakeholders expect them to perform the procedure as-is, and that pointing out a needed change is a high-cost, low-benefit exercise” (p1353).

Authors: Mendoza, A. M., Liu, S. N. C., Dumlao, S. V., Hendricks, J. W., Son, C., Sasangohar, F., & Peres, S. C., 2020, Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Vol. 64. No. 1.

Study link: https://doi.org/10.1177%2F1071181320641322

Link to the LinkedIn article: https://www.linkedin.com/pulse/where-two-ends-meet-operator-stakeholder-procedures-ben-hutchinson