SafetyInsights.org

This meta-analysis determined the relative effectiveness of different methods of worker safety and health training aimed at improving safety knowledge and performance and to reduce negative outcomes (incidents etc).

A large dataset was included – 95 studies from 1971 to 2003 across 15 countries, including >20k participants. Note: while this had a large dataset, the age of the studies (2003 and prior) is a limitation and largely misses the AR/VR shift as seen today.

Providing background, they note:

• Methods of training range from passive information based techniques, to learner-centred and performance-based methods (hands-on)
• Passive methods, like lectures, videos and computer training is linked to being the least engaging, and therefore likely the least effective for knowledge gain and behavioural change
• Moderately engaging methods include feedback interventions where performance information is provided in small groups, allowing learners to correct their performance. Feedback is an example of “programmed instruction”, a method used to present information in a standardised manner like on a computer
• The most engaging methods of training focus on developing knowledge in stages and emphasise principles of behavioural modelling. Behavioural modelling involves “observation of a role model, modeling or practice, and feedback designed to modify behavior” (p315). This can include hands-on demonstrations, work simulations and the like and requires active participation from the trainee

The core findings won’t be a surprise to anybody (that more engaging methods are more effective), but the correlations and specific findings may interest some.

Results

Key findings included:

• More engaging training methods had greater effects on knowledge acquisition – effect sizes ranged from 0.55 for least engaging (passive means), 0.74 for moderately engaging and 1.46 for most engaging (most active means)
• Training durations were on average longer for the most engaging methods, training duration and level of engagement were only weakly and non-significantly associated with knowledge acquisition and performance; thus, not really a factor here
• A small number of studies (n = 7) explored maintenance or decay of knowledge based on training characteristics. Knowledge decayed 50% over periods of time in the low engagement methods (e.g. ranging in studies from 1 week to 1 year periods)
• Only a single study explored decay in moderately engaging methods, finding a 15% drop over 4 weeks. A single study explored highly engaging methods, finding no decay after 4 weeks (** research on knowledge decay/maintenance was a limitation when this review was completed)
• For behavioural safety performance resulting from different degrees of training engagement, no significant difference was observed

Discussing the findings, they note that as expected, as training becomes more engaging, greater knowledge acquisition results and potentially greater gains in behavioural performance and less negative outcomes (although little data could be evaluated from the incident perspective).

They found that “the most engaging methods of safety training are, on average, approximately 3 times more effective than the least engaging methods in promoting knowledge and skill acquisition” (p315).

An unexpected finding was that the least engaging and moderately engaging methods had fairly comparable overall mean levels of performance for behavioural markers. The authors suspect this difference relates to how each training method is used.

For instance, the least and moderately engaging methods were more likely used for more fundamental and routine tasks, like using PPE, clearing up work areas, applying sunscreen etc., whereas the most engaging and active methods were more likely used for complicated activities or under more complex operating environments (managing bloodborne hazards in healthcare and the like).

They believe that “differences in the complexity of performance tasks, coupled with suboptimal measures of more complex tasks, influenced our results” (p320).

These findings support the “design and implementation of learner-centered, participatory approaches to worker safety and health training” (p321) and the findings are consistent with calls “advocating for the active involvement of workers so that the advanced knowledge necessary for fault prevention can be developed (e.g., anticipatory responses to problem situations in manufacturing context” (p321).

Moreover, the findings indicate that wherever possible, computer-based and distance learning methods, tending to be the least engaging and most passive methods, “should, in some manner, include active participation on the part of learners (e.g., modeling, feedback, and dialogue) to enhance their knowledge acquisition and increase their preparedness” (p321).

Feedback is to date a limitation of a lot of training, being rather “directional feedback rather than facilitating the types of dialogue that would engender action-focused reflection” (p321).

Further, while moderately and highly engaging training methods are likely to be, on average, more time consuming and possibly more expensive than the least engaging methods – this is likely to be a short-term cost but potentially less expensive and more effective in the long term (considering also the likely higher effects of knowledge decay in less engaging methods).

Authors: Burke, M. J., Sarpy, S. A., Smith-Crowe, K., Chan-Serafin, S., Salvador, R. O., & Islam, G. (2006). American journal of public health, 96(2), 315-324.

Study link: https://doi.org/10.2105/AJPH.2004.059840

Link to the LinkedIn article: https://www.linkedin.com/pulse/relative-effectiveness-worker-safety-health-training-ben-hutchinson