Relative Effectiveness of Worker Safety and Health Training Methods

Abstract

Objectives. We sought to determine the relative effectiveness of different methods of worker safety and health training aimed at improving safety knowledge and performance and reducing negative outcomes (accidents, illnesses, and injuries).

Methods. Ninety-five quasi-experimental studies (n=20991) were included in the analysis. Three types of intervention methods were distinguished on the basis of learners’ participation in the training process: least engaging (lecture, pamphlets, videos), moderately engaging (programmed instruction, feedback interventions), and most engaging (training in behavioral modeling, hands-on training).

Results. As training methods became more engaging (i.e., requiring trainees’ active participation), workers demonstrated greater knowledge acquisition, and reductions were seen in accidents, illnesses, and injuries. All methods of training produced meaningful behavioral performance improvements.

Conclusions. Training involving behavioral modeling, a substantial amount of practice, and dialogue is generally more effective than other methods of safety and health training. The present findings challenge the current emphasis on more passive computer-based and distance training methods within the public health workforce.

An understanding of how best to implement worker safety and health training is a critical public need in light of the tragic events of September 11, 2001, as well as ongoing efforts to prepare emergency responders and professionals in related areas to do their jobs safely and effectively.1 The need to gain a better understanding of the effectiveness of safety and health training is also apparent in a broader context given that millions of injuries and illnesses are reported annually in private industry workplaces,2 and health and safety training is globally recognized as 1 means of reducing the costs associated with such events.3 Indeed, researchers from different fields, including business, psychology, engineering, and public health, have long recognized the need for comprehensive, systematic evaluations of safety and health training to address these types of critical public- and private-sector concerns.4^–7

The conclusion from several narrative reviews has been that most training interventions lead to positive effects on safety knowledge, adoption of safe work behaviors and practices, and safety and health outcomes.5^,8^,9 However, these qualitative reviews are speculative as to the specific factors that enhance the relative effectiveness of safety and health training interventions in reducing or preventing worker injury or illness.10^–12 Notably, a fundamental question remains unresolved within the scientific literature: What is the relative effectiveness of different methods of safety and health training in modifying safety-related knowledge, behavior, and outcomes?

Attempts to address similar broad-based questions related to the benefits of work-related health and safety interventions13 have revealed the need for a large-scale, quantitative analysis of the extant literature. Results from such an analysis would not only help improve safety and health training programs but also provide evidence of the benefits of these programs, securing both new and continued support from the public as well as the private sector.

With these ends in mind, this study was designed to meta-analytically examine the effectiveness of different types of worker safety and health training, across industries and occupations, from 1971 to the present. In the section to follow, we describe different methods of worker safety and health training and offer hypotheses concerning the relative effectiveness of these methods.

SAFETY AND HEALTH TRAINING STRATEGIES

Methods of safety and health training range from passive, information-based techniques (e.g., lectures) to computer-based, programmed instruction and learner-centered, performance-based techniques (e.g., hands-on demonstrations). Lectures, one of the least engaging methods of safety and health training, are commonly used to present health- and safety-related information. Other common passive techniques include videos and pamphlets or other types of written materials.

Methods of training that can be categorized as moderately engaging incorporate knowledge of results, for example feedback interventions in which performance information is provided in small groups, allowing learners to correct their mistakes. Feedback is also a characteristic of programmed instruction, a method of training designed to present information in a standardized manner, such as on a personal computer or in a workbook format. An extensively used moderately engaging method, computer-based instruction, has been created for the entire gamut of work-place health and safety topics, including occupational safety, industrial safety, systems safety, fire protection, hazardous materials and waste disposal and storage, industrial hygiene, risk management, and safety engineering and design.14

The most engaging methods of safety and health training focus on the development of knowledge in stages15 and emphasize principles of behavioral modeling.16 Behavioral modeling involves observation of a role model, modeling or practice, and feedback designed to modify behavior. These methods also include hands-on demonstrations associated with behavioral simulations, which require active participation from the trainee.

In the case of behavioral simulations and hands-on training, interactions between trainees and trainers will frequently go beyond 1-way feedback to engage trainees in dialogue concerning knowledge acquired or actions taken. Such dialogue, in either a virtual or actual context, is important because it is posited to enhance quality of reflection (thinking) with respect to actions taken.17^–19 This action-focused reflection is regarded as the key to knowledge acquisition and transfer of training, in that it forces the trainee to infer causal and conditional relations between events and actions, leading to development of strategies for handling unforeseen events and initiating and promoting self-regulatory motivational processes (e.g., self-monitoring and self-efficacy expectations).

Consistent with these arguments, there is ample evidence in the training literature that active approaches to learning are superior to less active approaches.20 Therefore, as training moves along the continuum from more passive information-based methods (e.g., lectures) to the most engaging methods (e.g., behavioral modeling and hands-on demonstrations), we hypothesize that greater knowledge acquisition and more transfer of training to the work setting will occur (thereby improving behavioral safety performance and reducing negative safety and health outcomes).

METHODS

Search and Inclusion Criteria

We identified relevant studies published between 1971 (i.e., subsequent to passage of the US Occupational Safety and Health Act of 1970 [29 USC §651–678]) and 2003 by searching the PsycInfo, PubMed, and ABI-Inform electronic databases using phrases such as “health and safety training,” “safety training intervention,” and “error management and intervention.” In addition, we manually searched 19 journals and the reference sections of relevant publications. This process yielded 709 studies from a wide variety of fields, including occupational medicine, industrial hygiene, management, and applied psychology. We assessed all reports of an empirical nature to determine whether they met our criteria for inclusion in the meta-analysis.

Inclusion criteria were as follows. First, the study had to involve a quasi-experimental design (i.e., a study approximating a true experiment but not allowing for control of all relevant variables because of its field setting).21 Second, participants had to be recruited from a working population (this population could include youth workers). Third, the method of intervention (e.g., lecture, programmed instruction, behavioral modeling, or simulation) had to be clearly identified and had to involve the development of job-relevant safety knowledge.

Fourth, the study was required to include at least 1 of the following types of dependent variables: safety knowledge (i.e., self-rating or test of knowledge), safety performance (i.e., self-ratings or supervisor, coworker, or observer ratings of safety-related behavior), or safety and health outcome (i.e., measure of accidents, illnesses, or injuries). Fifth, the training intervention and data had to be assessed at the individual level of analysis. Finally, the statistical information necessary to calculate an effect size (d ) had to be available. A large number of studies were excluded because they contained inadequate statistical information or were not field experiments of health and safety training effectiveness. Of the originally identified 709 studies, 95 met the inclusion criteria.

Coding of Studies

An extensive coding protocol was developed to include the following information: (1) method of safety and health training, (2) duration of training, (3) dependent variable (i.e., safety knowledge, behavioral safety performance, or safety and health outcomes) used in all posttraining assessments, (4) reliability of dependent variable, (5) occupational classification, and (6) country of study. All reliability estimates22 (i.e., estimates of the consistency with which variables were measured) for knowledge tests were internal consistency estimates23 (i.e., α coefficients), and the majority of reliability estimates for the performance measures were interrater estimates (e.g., correlation between 2 trained observers’ assessments of workers’ performance).

In terms of classification of training methods, lectures, films, and video-based training were classified as the least engaging methods; programmed instruction techniques, including computer-based instruction and feedback techniques, were classified as moderately engaging training methods; and behavioral modeling, simulation, and hands-on training were categorized as the most engaging training methods. All study characteristics pertaining to hypothesis tests were double coded; disagreements between coders were settled by the first author.

Statistical Analyses

Initially, we computed d statistics using the procedures described by Shadish and colleagues24 and Lipsey and Wilson.25 In the case of studies in which gains or losses because of training were expressed as a proportion or a percentage, we estimated d statistics via an arcsine transformation, which results in a conservative estimate of d values.25

Subsequently, we employed the Raju et al.26 meta-analysis procedure because this procedure allows effects to be corrected for unreliability associated with the dependent variable. Such corrections produce more accurate estimates of population-level effects and permit more appropriate comparisons of mean effects across different types of dependent variables.27 In our set of studies, mean reliability values weighted according to sample size were 0.67 for safety knowledge, 0.86 for safety performance, and 0.96 for safety-related outcomes. Notably, the Raju et al. procedure allows computation of asymptotically derived standard errors for mean corrected (disattenuated) correlations in fixed-effect and random-effect forms.28 Therefore, before using this procedure, we transformed d statistics to correlations via maximum-likelihood estimates.27

For most studies that reported multiple effects within our dependent variable categories (e.g., effects for 2 dimensions of behavioral performance), we computed an average effect. In a few exceptions, we judged 1 effect more appropriate (e.g., because it was based on a more clearly defined dependent variable), and we included this effect in our analyses. This procedure ensured independence of study effects within any particular effect size distribution.

A number of effect sizes for combinations of training method and dependent variable were based on within-subject designs. To examine the possible effects of study design on our results, we conducted separate meta-analyses of studies involving within-subject designs and studies involving between-subjects designs (we also conducted separate analyses for distributions that included both types of study designs). In addition, because of the lack of pretraining information in many studies, effects for between-subjects studies were based on posttest-only comparisons of control and training groups. In a few cases in which the control or comparison group was non-comparable (e.g., the groups had different amounts of work experience) or the comparison group was trained with a less engaging method than the focal trained group (and in which pretraining and posttraining data were available), study effects were based on within-subject data for the trained group or groups.

RESULTS

Description of Studies

Ninety-five studies29^–123 conducted between 1971 and 2003 in 15 countries were included in the present meta-analyses (Table 1 ▶). These studies comprised 126 independent samples, 20991 participants (the sum of the independent samples), and 147 safety training effect sizes. The 43 samples from the health care occupations represented the largest occupational group.

TABLE 1—

Effects From Each Study and Each Dependent Variable: Meta-Analysis of Safety and Health Training Effectiveness, 1971–2003

	Disattenuated Effect Size
Study	Safety Knowledge	Safety Performance	Safety and Health Outcomes
Alavosius and Sulzer-Azaroff 29	. . .	0.42	. . .
Albers et al.30	0.90	. . .	. . .
Arcury et al.31	0.32	0.39	. . .
Arnetz and Arnetz 32	0.20	. . .	. . .
Askari and Mehring33	0.43	. . .	. . .
Azizi et al.34	. . .	0.96	. . .
Baker 35	. . .	2.33	. . .
Barnett et al.36	0.30	. . .	. . .
Bosco and Wagner 37	1.45	. . .	. . .
Calabro et al.38	2.44	. . .	. . .
Caparez et al.39	1.53	. . .	. . .
Carlton40	3.67	0.56	. . .
Carrabba et al.41	. . .	0.29	0.20
Chaffin et al.42	. . .	0.19	. . .
Chhokar and Wallin43	. . .	0.86	. . .
Cohen and Jensen44
Plant 1	. . .	0.12	. . .
Plant 2	. . .	0.42	. . .
Cole et al.45	. . .	0.60	. . .
Coutts et al.46	0.82	. . .	. . .
Curwick et al.47	1.27	. . .	. . .
Daltroy et al.48	0.99	. . .	. . .
Daltroy et al.49
Mailhandlers	. . .		–0.02
Clerks	. . .		0.01
DeVries et al.50	. . .	0.60	. . .
Dortch and Trombly 51
Group 1 vs group 3	. . .	1.60	. . .
Group 2 vs group 3	. . .	1.51	. . .
Eckerman et al.52	2.35	. . .	. . .
Evanoff et al.53	. . .	. . .	0.39
Ewigman et al.54	0.43	0.69	. . .
Feldstein et al.55	. . .	. . .	0.37
Fox and Sulzer-Azaroff 56	. . .	0.72	. . .
Froom et al.57	. . .	1.55	. . .
Gerbert et al.58	1.04	0.23	. . .
Girgis et al. 59	0.10	0.37	. . .
Goldrick60
78 nurses	1.65	. . .	. . .
66 nurses	1.78	. . .	. . .
Haiduven et al.61	. . .	. . .	0.61
Hopkins62	. . .	0.59	. . .
Hultman et al.63	. . .	1.61	. . .
Hurlebaus and Link64	0.94	. . .	. . .
Infantino and Musingo65	. . .	. . .	0.58
Inman and Blanciforti 66	. . .	0.21	. . .
Knobloch and Broste67	. . .	1.02	. . .
Komaki et al.68
Makeup department	. . .	0.85	. . .
Wrapping department	. . .	0.76	. . .
Komaki et al.69
Department 1	. . .	0.37	. . .
Department 2	. . .	0.29	. . .
Department 3	. . .	0.20	. . .
Department 4	. . .	0.26	. . .
Leslie and Adams70	. . .	0.18	0.91
Ludwig and Geller 71	. . .	0.06	. . .
Ludwig and Geller 72
Participative goal setting	. . .	1.15	. . .
Assigned goal setting	. . .	1.07	. . .
Lueveswanij et al.73	1.21	. . .	. . .
Luskin et al.74	0.91	. . .	. . .
Lynch et al.75
Combined sample	0.72	. . .	. . .
Medical personnel	. . .	0.58	. . .
Nursing personnel	. . .	0.43	. . .
Lynch and Freund76	0.64	0.30	. . .
Maples et al.77	. . .	. . .	0.79
Marsh and Kendrick78	0.76	. . .	. . .
Martyny et al.79	0.10	. . .	. . .
Materna et al.80	. . .	1.68	. . .
Mattila81	. . .	0.29	. . .
Mattila and Hyodynmaa82
Office building site		0.36	. . .
Apartment building site		0.39	. . .
McCauley 83	. . .	1.27	. . .
Melhorn 84
Standard rivet gun training	. . .	. . .	0.34
Vibration-dampening rivet gun training	. . .	. . .	0.13
Michaels et al.85
Carpenters	0.92	. . .	. . .
Construction laborers and pipe caulkers	0.52	. . .	. . .
Custodial assistants	0.40	. . .	. . .
Dental assistants/hygienists and dentists	0.80	. . .	. . .
Electricians	0.79	. . .	. . .
Engineers and high-pressure plant tenders	0.78	. . .	. . .
Plumbers	0.78	. . .	. . .
Print shop workers	0.56	. . .	. . .
Traffic device maintainers	0.94	. . .	. . .
Nasanen and Saari 86	. . .	0.55	0.63
Parenmark et al.87	. . .	. . .	0.73
Parkinson et al.88	0.67	0.51	. . .
Peters89	0.35	. . .	. . .
Porru et al.90	0.40	. . .	. . .
Ray et al.91	. . .	0.69	. . .
Ray et al.92	. . .	0.38	. . .
Reber and Wallin93	. . .	0.20	. . .
Reber et al.94	. . .	0.95	0.68
Reddell et al.95	. . .	. . .	0.88
Rhoton96	. . .	. . .	0.98
Rundio97	1.85	. . .	. . .
Saarela98	. . .	. . .	0.20
Saarela et al.99
Tankers	. . .	. . .	–0.27
Ferries	. . .	. . .	–0.21
Saari100	. . .	0.46	. . .
Saari and Nasanen101	. . .	0.55	0.31
Sadler and Montgomery 102
Leader-directed group	. . .	0.39	. . .
Standard lecture	. . .	0.25	. . .
Schwartz103	. . .	. . .	1.39
Seto et al.104	1.45	1.48	. . .
Streff et al.105	. . .	0.29	. . .
Sulzer-Azaroff and de Santamaria106
Department 1	. . .	. . .	0.43
Department 2	. . .	. . .	0.66
Department 3	. . .	. . .	0.62
Department 4	. . .	. . .	0.16
Department 5	. . .	. . .	0.53
Department 6	. . .	. . .	0.13
Symes et al.107	0.45	. . .	. . .
Troup and Rauhala108	. . .	0.97	. . .
Uwakwe109	0.57	0.07	. . .
van Poppel et al.110	. . .	. . .	0.17
Vaught et al.111
Hands-on training	. . .	0.51	. . .
Computer-based training	. . .	–0.33	. . .
Videman et al.112	. . .	1.33	0.11
Wang et al.113	1.74	0.73	0.58
Wertz et al.114	0.47	. . .	. . .
Whitby et al.115	0.25	0.83	. . .
Williams and Geller 116	. . .	0.21	. . .
Williams and Zahed 117
Lecture method	3.67	. . .	. . .
Computer-based training	3.37	. . .	. . .
Wolford et al.118	. . .	0.44	. . .
Wong et al.119	. . .	0.23	. . .
Wynn and Black 120	0.97	. . .	. . .
Yarall121
Worksite B	0.67	1.12	. . .
Worksite C	0.80	2.07	. . .
Yassi et al.122	. . .	0.29	0.09
Zohar et al.123	. . .	1.21	. . .

Note. Effects are expressed as d statistics corrected for dependent variable unreliability.

Safety and Health Training Methods

Table 2 ▶ presents the results of tests of our hypotheses based on data gathered within the first posttraining assessment. For reporting purposes, mean effect sizes (d statistics), along with number of effects (k) and total sample sizes (n) pertaining to hypothesis tests, are presented for (1) studies involving between-subjects designs, (2) studies involving within-subject designs, and (3) the combined (overall) distributions of effects. As a result of the complex nature of r-to-d transformations for d values in our overall analysis (which generally fell outside the straightforward transformation range for d of −0.41 to 0.41),27 limits of the confidence intervals for the mean effects and variances of effect size distributions are presented only in correlation (r) form in Table 3 ▶.

TABLE 2—

Training Method Results: Meta-Analysis of Safety and Health Training Effectiveness, 1971–2003

	Safety Knowledge			Safety Performance			Safety and Health Outcomes
Training Method/Study Design	n	k	MΔ	n	k	MΔ	n	k	MΔ
Least engaging (overall)	4097	18	0.55	2356	20	0.63	1950	3	0.2
Between-subjects studies	1071	7	0.58	1509	12	0.65	. . .	. . .	. . .
Within-subject studies	3026	11	0.54	847	8	0.58	. . .	. . .	. . .
Moderately engaging (overall)	3021	19	0.74	1864	31	0.62	4528	19	−0.13
Between-subjects studies	1121	5	0.66	1044	10	0.74	3846	7	−0.04
Within-subject studies	1900	14	0.79	820	21	0.47	682	11	−0.66
Highly engaging methods (overall)	886	12	1.46	2019	16	0.74	3068	9	−0.48
Between-subjects studies	609	7	1.27	1914	12	0.72	1588	6	−0.25
Within-subject studies	277	5	1.89	105	4	1.14	1480	3	−0.74

Note. n = total number of individuals; k = number of effects; M_Δ = estimated mean d statistic corrected for dependent variable unreliability (mean Δ).

TABLE 3—

Training Method Results in Correlation Form: Meta-Analysis of Safety and Health Training Effectiveness, 1971–2003

	Safety Knowledge					Safety Performance					Safety and Health Outcomes
Training Method/Study Design	n	k	Mρ	95% CI	Vρ	n	k	Mρ	95% CI	Vρ	n	k	Mρ	95% CI	Vρ
Least engaging (overall)	4097	18	0.26	0.22, 0.29	0.01	2356	20	0.28	0.24, 0.32	0.02	. . .	. . .	. . .	. . .	. . .
Between-subjects studies	1071	7	0.27	0.20, 0.34	0.02	1509	12	0.29	0.24, 0.33	0.04	1950	3	0.1	0.15, 0.06	0.01
Within-subject studies	3026	11	0.25	0.21, 0.29	0.01	847	8	0.28	0.21, 0.35	0.00	. . .	. . .	. . .	. . .	. . .
Moderately engaging (overall)	3021	19	0.33	0.30, 0.37	0.03	1864	31	0.28	0.23, 0.32	0.03	4528	19	–0.06	−0.09, −0.03	0.01
Between-subjects studies	1121	5	0.29	0.23, 0.35	0.06	1044	10	0.32	0.26, 0.37	0.05	3846	7	–0.02	–0.05, 0.01	0.00
Within-subject studies	1900	14	0.36	0.32, 0.41	0.01	820	21	0.22	0.15, 0.27	0.00	682	12	–0.32	−0.38, −0.23	0.04
Highly engaging methods (overall)	886	12	0.58	0.52, 0.64	0.03	2019	16	0.34	0.30, 0.38	0.01	3068	9	–0.23	−0.26, −0.19	0.01
Between-subjects studies	609	7	0.52	0.45, 0.60	0.02	1553	10	0.33	0.29, 0.38	0.01	1588	6	–0.12	−0.17, −0.07	0.00
Within-subject studies	277	5	0.71	0.59, 0.83	0.02	105	4	0.46	0.31, 0.61	0.03	1480	3	–0.34	−0.38, −0.29	0.01

Note. n = total number of individuals; k = number of study effects; M_ρ = estimated mean correlation corrected for dependent variable unreliability (mean ρ); CI = confidence interval around estimated M_ρ; V_ρ = estimated variance of effects.

The findings reported in Table 2 ▶ are consistent with the expectation that the more engaging a method of training, the greater the effects of safety and health training on knowledge acquisition. Overall, mean knowledge acquisition effect sizes for the least engaging, moderately engaging, and the most engaging safety training interventions (for both types of study designs combined) were 0.55, 0.74, and 1.46, respectively. As indicated by the confidence intervals for these effects (Table 3 ▶), the effects were significantly different from each other. Furthermore, although training durations were, on average, greater in the case of more engaging training methods, training duration and level of engagement were only weakly (and nonsignificantly) associated in the knowledge category studies (as well as the performance category studies). These findings rule out training duration and a strict observational learning effect as a plausible rival explanation for the present results.124

A small subset of studies (i.e., 7) that included knowledge measures allowed us to examine maintenance or decay in terms of the effectiveness of safety training. In 5 studies involving training at low levels of engagement, the average effect decreased approximately 50% (i.e., from 0.55 to 0.28) during periods ranging from 1 week to 1 year after the initial assessment. The effect in the lone study involving moderately engaging training decreased approximately 15% (from 3.37 to 2.85) over 4 weeks, and the effect in the single study involving highly engaging training was maintained at 1.84 over a 4-week period. More research and better reporting of primary study results are needed before definitive conclusions can be reached about decay of safety and health training effectiveness over time.

With respect to improvements in behavioral safety performance, the mean overall effects associated with safety and health training interventions in the least engaging, moderately engaging, and most engaging categories were 0.63, 0.62, and 0.74, respectively. Although these effects were not significantly different from each other, it is notable that the confidence interval for the latter effect was predominantly outside the range of the respective confidence intervals for the effects of the least engaging and moderately engaging training methods.

Our findings are generally consistent with the expectation that as level of engagement in training increases, training will have greater effects in terms of reductions in negative safety and health outcomes. For the overall distributions, the mean effects associated with the least engaging, moderately engaging, and the most engaging safety and health training methods were 0.20, −0.13, and −0.48, respectively, and these effects were significantly different from each other. It should be noted that the least engaging and moderately engaging distributions were each influenced greatly by a single study involving a large sample size and a small effect.

DISCUSSION

Here we assessed theoretical expectations concerning the relative effectiveness of different methods of worker safety and health training aimed at modifying safety-related knowledge, behaviors, and outcomes. This is the first investigation focusing on such training, to our knowledge, that has included all studies published since 1971 and has involved a scientifically rigorous approach. Although the number of studies examining illnesses, injuries, and accidents was not sufficient to allow separate consideration of these categories of safety and health outcomes, the quality of the database was adequate for testing general hypotheses.

As mentioned, our results are consistent with the proposition that as the method of safety and health training becomes more engaging, the effect of training is greater in terms of knowledge acquisition and reductions in negative outcomes. Our results concerning behavioral performance were more equivocal but nevertheless provided consistent support, in the case of both between-subjects and within-subject study designs, for the effectiveness of more engaging training methods. Together, these findings address calls for research on safety and health interventions, including those of the National Occupational Research Agenda.10^–12^,125 More specifically, our results speak to the goals of the intervention effectiveness research agenda, including not only what interventions are most effective in enhancing worker safety and health but also why they are effective.

Our findings indicate that the most engaging methods of safety training are, on average, approximately 3 times more effective than the least engaging methods in promoting knowledge and skill acquisition. An alternative way to differentiate the effects of the most engaging methods on knowledge gain from the effects of other methods is to compute “common language” effect sizes.126 In a given study, the probability of a randomly selected individual from the most engaging training group exceeding a randomly selected individual from the least engaging training group in terms of knowledge acquired was 0.74; the analogous probability was 0.70 in a comparison of randomly selected individuals from the most engaging and moderately engaging groups. The magnitudes of such differences alone have broad organizational and public policy implications for the manner in which safety and health training—in particular, mandated training—is delivered.

Unexpectedly, the least, moderate, and most engaging safety and health training methods had somewhat comparable overall mean levels of effectiveness with respect to improvements in behavioral performance. We cautiously interpret this pattern of results to be a function of the fact that the training methods classified as least engaging and moderately engaging were often associated with more fundamental, routine types of tasks (e.g., applying sunscreen, inserting hearing devices, keeping work areas clear of obstacles), whereas the methods classified as most engaging often involved advanced, complex work activities (e.g., properly handling needles to avoid exposure to blood-borne pathogens, selecting and using respirators to avoid neurotoxic exposures). We suspect that differences in the complexity of performance tasks, coupled with suboptimal measures of more complex tasks, influenced our results.4^,127

Our findings indicate that the most engaging methods of safety training are, on average, most effective in reducing negative outcomes such as accidents. The greater effectiveness of more engaging, hands-on training in reducing negative outcomes and increasing knowledge acquisition lends support to the calls of researchers and practitioners advocating the design and implementation of learner-centered, participatory approaches to worker safety and health training,74^,128^–130 and such a finding is consistent with the results of previous meta-analytic studies of training evaluation in other domains.131 Furthermore, our findings are consistent with recommendations in other areas of the literature advocating for the active involvement of workers so that the advanced knowledge necessary for fault prevention can be developed (e.g., anticipatory responses to problem situations in manufacturing contexts).132^,133

In a broader sense, the present results provide guidance for the design and delivery of educational interventions targeted toward the public health workforce.134^–136 Efforts to increase the capacity of this workforce as well as the capacity of the public to respond to threats, react to emergency events, and simply engage in safe behavior must be achieved, in part, through continued education programs.1^,137^–139 Designing and implementing effective training is central to these efforts.

Our results have important implications that should be considered in light of the current push toward greater use of distance learning training in preparing the public health workforce.140^–143 Our findings suggest that, to the extent possible, computer-based and distance learning methods should, in some manner, include active participation on the part of learners (e.g., modeling, feedback, and dialogue) to enhance their knowledge acquisition and increase their preparedness. To date, most computer-based and distance safety training has been rather passive, including directional feedback rather than facilitating the types of dialogue that would engender action-focused reflection.144 Our recommendations concerning active worker participation and dialogue as accompaniments to computer-based and distance learning methods of health and safety training are fully consistent with theoretical models concerning distance learning and education.145^,146

Another important finding of this study relevant to the design and evaluation of safety and health training was that between-subjects and within-subject study designs yielded similar results with respect to knowledge acquisition. Despite cautionary issues concerning potential threats to the internal and external validity of within-subject study designs,147 our results demonstrate that studies involving such designs provide theoretically interpretable findings that are consistent with findings from between-subjects studies in the domain of worker safety and health training. Given that within-subject designs generally involve greater statistical power than between-subjects designs27 and that withholding safety and health training from a comparison group (or locating a control/comparison group) for the purpose of program evaluation is often ethically questionable in safety-related work, our findings encourage greater use of within-subject designs in evaluating safety and health training.

The meta-analytic results described here are also necessary building blocks for any effort aimed at estimating the incremental costs or benefits of different types of safety and health training.148 Such information is particularly important given today’s increased pressures to justify and improve health and safety investments.12 Our results suggest that moderately and highly engaging training methods are, on average, more time consuming and probably more expensive in the short term but that they are potentially less costly and more effective in the long term while better ensuring worker and public safety.

In addition, the trends in the magnitudes of our results across dependent variable categories are consistent with predictions from job performance theories. Job performance theories posit that interventions (e.g., safety and health training) are expected to have their greatest impact on more proximal outcome variables such as knowledge acquisition and their least impact on more distal, low-base-rate phenomena such as accidents. The reasoning is that knowledge acquisition is expected to mediate the relationships between such interventions and their more distal outcomes.8^,149 Accordingly, in evaluating the effectiveness of interventions, safety and health training researchers and practitioners need to focus much more on the development of well-designed, standardized measures of safety knowledge. Furthermore, because training effects for relatively distal outcomes such as on-the-job performance and injuries are likely to be more affected by intervening, time-related variables than training effects for relatively proximal measures such as knowledge assessments, we stress the need for future research examining the influence of situational variables (i.e., organizational safety climate, opportunities to apply knowledge and skills, type of work, country/culture, and so on)150^,151 on safety and health training effectiveness.

We also encourage future primary empirical research addressing some of the limitations of the present meta-analysis (e.g., primary studies related to distributions with small numbers of effects). Moreover, we encourage primary and meta-analytic research designed to extend our study and examine safety and health training relative to more specific safety knowledge, safety performance, and safety and health outcome variables, in addition to examining the role of individual difference variables (e.g., worker motivation, work experience). Our future success in promoting safe work behaviors and reducing the negative consequences of unsafe behaviors will largely depend on our ability to improve our conceptualizations and communications of the effectiveness of safety and health training interventions.