An Intervention Effectiveness Study of Hazard Awareness Training in the Construction Building Trades

SYNOPSIS

Objective

We evaluated knowledge, attitudes, and self-reported work practices among apprentice and journeyman trainees in two construction trades at baseline and three months after participation in two training sessions as part of a 10-hour Occupational Safety and Health Administration hazard awareness training program. We developed preliminary assessment of prior and current training impact, accounting for demographics, trade, and construction site safety climate.

Methods

Participants were recruited prior to union-delivered safety training, self-completed a baseline survey prior to class, and completed a follow-up interviewer-administered telephone survey three months later. Discrimination (D) testing evaluated knowledge questions, paired t-tests examined differences in pre- and post-intervention knowledge, and attitude responses were tested with the Wilcoxon signed rank test. Linear regression analysis and logistic regression were used to assess the contribution of different categorical responses to specific sub-questions.

Results

Of 175 workers completing the baseline survey, 127 were born in the U.S. and 41 were born in Mexico; 40% of those who reported ethnicity were Hispanic. Follow-up surveys were completed by 92 (53%) respondents and documented significant increases in both fall safety and electrical safety knowledge. The most recent safety climate was associated with improvement in fall safety attitudes (slope = 0.49, p<0.005) when adjusted by country of birth (slope = 0.51, p<0.001). Workers born in Mexico had less formal education than U.S.-born workers and lower baseline knowledge scores, but more positive attitude scores at baseline and greater improvements in attitude at follow-up.

Conclusion

Knowledge and attitude improvement following a one-hour safety class was measurable at three months in both U.S.-born and Mexican-born construction workers.

The construction industry consistently ranks as one of the most hazardous in the U.S. for both fatal and non-fatal traumatic injury rates.^1,2 Construction work accounts for the highest number of fatal occupational injuries in the U.S., and this number is increasing. Falls from heights remain the leading cause of fatal traumatic injury in this industry, followed by struck-by incidents, and contact with an electric current. Significant challenges to reducing injury rates include worksite and job task characteristics, work organization, worker training, and an increase in the informal work sector, which has been accompanied by an increase in the proportion of immigrant workers. In 2000, the proportion of self-identified Hispanic workers in construction reached 15%, of whom 74% were born outside the U.S. and nearly one-third spoke only Spanish. The rate of fatal occupational injury in this population was almost twice that of other construction workers (relative risk [RR] = 1.84, 95% confidence interval [CI] 1.60, 2.10).²

The effect of immigration status overlaps with reports of ethnicity. Of the 2.3 million foreign-born construction workers in the U.S. in 2005, 59% were born in Mexico and an additional 25% were born in other countries in Latin America. By 2005, 27% of construction workers self-identified as Hispanic and one-quarter of construction workers reported speaking a language other than English in the home.³ Loh and Richardson explored all traumatic occupational fatalities reported through the Census of Fatal Occupational Injury between 1996 and 2001, and determined that the excess rate of fatal injury among Hispanic workers occurred only among foreign-born Hispanic workers, while U.S.-born Hispanic workers had fatal injury rates that were identical to the rest of the U.S. workforce.⁴

Occupational safety and health training programs and effectiveness research have explored primarily the effectiveness of safety and health training in fixed-site manufacturing and among hazardous waste workers. This research was spurred chiefly by the Occupational Safety and Health Administration (OSHA) Hazard Communication Standard and by the Superfund Amendment and Reauthorization Act of 1986, both of which expanded requirements for worker training. Active participation in problem identification and problem solving contribute to effecting measurable change.^5–8 However, peer-reviewed evaluation of the effectiveness of training in the construction sector is limited.

A study of plumbers and pipefitters in western Ohio demonstrated a reduction in injuries among workers who had received jobsite safety orientation, although the impact of safety awareness training (equivalent to the OSHA 10-hour course) could not be determined.⁹ Dong et al. evaluated health insurance records, union training records, and workers' compensation data for the two-year period 1993–1994 for more than 8,000 laborers in Washington State. Laborers who had completed an OSHA training program experienced a 12% decrease in workers' compensation costs (95% CI 0.75, 1.02).¹⁰

Studies of workplace health and safety also emphasize the importance of factors such as supervisory support, provision of appropriate equipment, training on use of equipment, and active worker participation, among other considerations that determine safety climate—the perception by workers that safety is valued in their organization.^11–13 The importance of reinforcing classroom learning with on-site job safety has been emphasized for younger workers in particular. In one study of youth in farm settings, safety training of the youth alone failed to have any impact.¹⁴ A recent study of apprentice (novice) carpenters beginning on-the-job training in residential construction demonstrated wide variability in the quality of jobsite mentoring that often resulted in a lack of support for safe work practices learned in the classroom and consequent failure to follow them.¹⁵

A growing body of information evaluating the role that safety climate plays in general industry has more recently been applied to construction sites.^16–22 Safety climate has been associated with injury occurrence and with injury severity in construction workers,^19,20 and appears to be linked to injury and illness in Latino poultry workers.²¹

To address the unique challenges posed by construction sites, OSHA adapted its 10-hour hazard-awareness training program to specifically target construction workers. CPWR (formerly the Center to Protect Workers' Rights, now the Center for Construction Research and Training) subsequently developed the Smart Mark curriculum for the OSHA 10- and 30-hour training programs for the members of building trades unions under the guidance of the Construction Industry Partnership in 1997–1998. Smart Mark hazard-awareness training is frequently incorporated into the apprenticeship (or pre-apprenticeship) training programs for the building trades, and is presented differently to apprentices (i.e., students enrolled in formal training that includes practical experience as well as classroom teaching) and journeymen (i.e., workers who have entered a trade and are working for others). The 10-hour program includes up to 10 of the 13 available training programs or modules, selected on the basis of relevance to the particular trade. Apprenticeship programs vary in length and content by trade, but in general alternate classroom, hands-on controlled workshop activities, and actual on-the-job experience in work settings.

This pilot study was conducted to (1) assess the impact of hazard awareness training on baseline knowledge, attitudes, and work practices among roofers and pipefitters and (2) identify potential changes in safety climate. Because one of the unions included a significant proportion of members who were born in Mexico, demographic differences were explored in this group. This article describes baseline demographics of participating workers; the relationship to baseline knowledge, attitudes, and self-reported practices regarding fall and electrical safety hazards; and changes in knowledge and attitudes following the training intervention among a paired subset of workers.

METHODS

All respondents agreed to participate via informed consent using protocols approved by the Institutional Review Board of the Office to Protect Research Subjects of the University of Illinois at Chicago. The study was conducted between November 2004 and January 2006 at two sites: the Pipe Fitters' Training Center Local Union 597 in Mokena, Illinois, and the Joint Apprentice Training Center of the United Union of Roofers and Allied Workers Local 11 in Indian Head Park, Illinois.

Survey design and data collection

A survey obtaining basic demographic information as well as work experience and safety knowledge, attitudes, and self-reported work practices was developed in English through iterative discussions and pilot testing among master trainers (instructors who train other trainers) in the construction trades, including those who used the Smart Mark curricular materials and those who used other materials. The survey design was modified according to sequential evaluation of items as part of the overall pilot project, with sequential modifications of survey administration methods as well.

The survey incorporated existing knowledge questions and developed attitude and practice questions based on interviews with union master trainers, including one item that asked whether the worker had previously identified a fall hazard at the worksite, and a follow-up question to clarify whether any remediation action was taken. Knowledge questions were selected through several iterations by starting with work practice questions included in the 10-hour curriculum and selecting those items rated to be meaningful, likely to be remembered, and of consequence on the worksite by four master trainers serving on the CPWR board of advisors, as well as two content and training experts. Items were subsequently reduced to those items for which discrimination (D) scores were identified as successful in distinguishing high performers from low performers. The D-test scores were produced for individual test items by comparing the ratio of high scorers who answered the item correctly with low scorers who answered the item correctly; the test indicates how well an item discriminates between high and low scorers, assuming that a flawed test item is one that is correctly answered more often by individuals performing poorly overall when compared with those performing well.

In addition, a previously validated workplace safety climate scale developed by the National Institute for Occupational Safety and Health (NIOSH) was adapted for use in the construction sector, modified with input from one of the union partners, translated, and evaluated. Questionnaire reliability for the safety climate scale has been previously reported.²³ Although fluency in English had been described as a prerequisite for union membership during the project's development phase, initial recruitment included a number of workers for whom Spanish was the primary language, prompting translation of study materials into Spanish, with back-translation into English; the use of dual-language survey instruments; the addition of bilingual research assistants; and additional review of self-completed surveys.

Workers received a letter from the union in advance of the training session that described the research project and partnership. On the morning of training, voluntary participation was solicited and informed consent was obtained. A home carbon monoxide detector was provided to participants as a modest incentive upon completion of the baseline survey. Among those attending the training sessions, the overall participation rate for the project was 79% (71% for the roofers and 99% for the pipefitters). The latter included only apprentices, and scheduling difficulties required exclusion from some of the analyses. The survey was administered to union roofers in a group setting and to union pipefitters just prior to a safety training class. The survey was read in English, and the participants had approximately 30 minutes to complete it. Bilingual facilitators were available to answer individual questions for the roofers.

Between November 2004 and November 2005, eight groups of roofers attending classes were offered the survey at the Roofers Joint Apprenticeship Training Center in suburban Chicago, Illinois, enrolling a total of 158 participants. Scheduling difficulties caused by the move of the pipefitter training center to Mokena during the course of the project reduced the number of pipefitter training classes to three, held between June 2005 and January 2006. All of the pipefitter classes were part of the apprenticeship training, and only one class was surveyed prior to the relevant OSHA hazard awareness class. The total number of participants for baseline assessment and demographics was 87, of whom only 28 were eligible for pre/post comparison testing (i.e., completed the survey prior to and following the training session).

Along with informed consent at baseline, participants were asked to provide contact information for subsequent telephone follow-up. Three months after the baseline survey and training class, bilingual research assistants attempted follow-up telephone contact, with a maximum of 10 tries at different evening/weekend time periods. Several workers declined to participate, including one who was out of work and in litigation for a serious work-related injury. The addition of a modest follow-up incentive ($10 gift card) that was announced at the time of recruitment failed to improve follow-up participation. For roofers, the follow-up rate among eligible participants was 52% (76 of 147); for pipefitters, the follow-up rate was 57% (16 of 28). Follow-up participation was limited to 53% of baseline (92 of 175) primarily because of difficulty in making follow-up telephone contact.

Participation, questionnaire completion, evaluation, D-score generation, and demographic and baseline comparisons among unions were conducted using the entire enrollment (n=245). However, a number of exclusions were applied for subsequent paired analysis. Surveys with high numbers of missing responses and those provided by trainees who were accessed only after they had received the initial hazard awareness program were excluded, reducing the baseline eligible population from 245 to 175 participants. A question about prior safety training was asked of the first 88 survey respondents, but because of wording inconsistency (“ever” vs. “past three months”), the question was removed when post-training questionnaire administration demonstrated confusion.

Data entry, description, and analysis

Data from the completed questionnaire survey were entered into two separate but identical Microsoft® Access tables.²⁴ These tables were exported to Microsoft Excel files and the responses were compared by subtraction. When mismatched responses were identified, the completed questionnaire was reviewed and reentered to ensure quality control. Data were then imported into SAS® software for analysis.²⁵

Outcome categories targeting fall safety knowledge, attitudes, work practices, and safety climate were examined against potential confounders including age, years in the trade, previous formal safety training, journeyman or apprentice status, and education. These variables were also compared by ethnicity/birthplace.

We performed univariate analysis followed by multivariate analysis. The paired t-test and Kruskal-Wallis test were used for comparisons of continuous variables. The Chi-square and Fisher's exact tests were used to compare categorical variables. Logistic regression and multiple linear regression modeling were used to assess the contribution of different independent factors on the dependent variable outcomes of interest, following the approach in Kleinbaum and Klein.²⁶ Only factors with a potentially significant association at the p≤0.20 level in the univariate analysis were introduced into the initial multivariate models. The final regression model was chosen according to a backward elimination technique.

A chunk test using the likelihood ratio test (p<0.05) was performed on the full model (all independent variables of interest and all two-way interaction terms) and a reduced model (all independent variables of interest and the two-way interaction terms between the exposure variable [e.g., ethnicity/nativity] and all other variables). When the model was reduced to independent variables and exposure interaction terms, the interaction terms were eliminated using backward elimination. When only potential variables of interest remained, they were eliminated one by one and assessed for confounding. If the effect (i.e., the slope in linear regression or the odds ratio [OR] in logistic regression) of the variable of interest changed by <10%, this variable was considered a possible confounder and kept in the model as a controlling variable.

ORs with corresponding 95% CIs were used to determine the magnitude of association between independent variables and hazard identification as the dependent variable.

Exclusions

Two groups of participants were included for questionnaire review and discrimination analysis, but were excluded from pre/post comparisons. These included the 10 participants born in Mexico who volunteered during the initial roofers' session (prior to implementation of the translated survey and assistance from bilingual facilitators) and the 60 pipefitters who participated in sessions that took place after training had already occurred. In all, there were 175 participants at baseline and 92 paired for the follow-up survey. Because of missing answers, the numbers available for each analysis varied.

RESULTS

Table 1 shows the distribution of demographic variables among the subset of participants available for pairing who were analyzed. The paired comparisons demonstrated that participants who completed follow-up surveys were almost identical to the dropouts for these variables, suggesting that there was no systematic difference for these factors between those who completed the follow-up survey and those who did not.

Table 1.

Distribution of demographic characteristics among construction workers completing baseline and follow-up evaluation of hazard awareness training

^aMost Hispanic respondents did not identify a race or identified their race as “other” (e.g., Mexican or Hispanic/Latino).

SD = standard deviation

Knowledge

Fall hazard knowledge scores were based on six questions determined by D-scoring to adequately distinguish between high- and low-scoring individuals. For the 175 respondents who completed the baseline questionnaire, the mean score was 2.4 (standard deviation [SD] =1.4) out of a maximum of six. For the 92 participants who completed both the baseline and the three-month follow-up surveys, the mean improvement was 0.7 (SD= 1.6) (paired t-test, t=4.46, p<0.0001).

Using linear regression, fall knowledge at baseline was modeled for associations with age, education, birth country, mean safety climate score, union status, years in the trade, and years in the union. There was statistically significant interaction between age and union status, so the model was stratified into apprentice and journeyman-level strata. Among journey workers (n=46), no associations with age and union status of p<0.05 were found. Among apprentices (n=99), the number of years in the union was significant (slope = 0.1, p<0.05) when adjusted for age, climate, and birth country. Among apprentices (but not journeymen), baseline knowledge scores improved as the number of years of work experience increased.

Also using linear regression, the improvement in fall knowledge after the safety training was modeled for associations with age, education, birth country, trade, union status, years in the union, and years in the trade. When stratified by union status, no characteristics of journey-level workers were associated with fall knowledge improvement. For apprentices, only trade was associated with knowledge improvement (p<0.0001) and had a large effect (slope = 1.56). However, because pipefitters differed significantly from roofers in having higher educational attainment, this difference may have masked a difference in ability to learn in a classroom setting based on prior educational experience.

Electrical hazard knowledge scores were based on seven of the original 13 questions and ranged from 0 to 7. These seven questions were selected for their ability to distinguish high scorers from low scorers using the D-test. For 165 respondents at baseline, the mean score was 3.7 (SD=1.8). For 83 respondents at follow-up, the mean improvement from baseline was 0.7 (SD=1.6) (paired t-test, t=3.61, p=0.0005).

Attitude

Attitude is the personal priority that the individual attaches to the importance of workplace safety. It differs from safety climate, which is the individual's perception of how the organization in which he or she is working values safety. Fall safety attitude was measured using three questions: (1) “How important is fall protection?” (2) “Are OSHA requirements regarding fall protection more of a hindrance or a help to you being able to do your work well?” and (3) “If you had the option of receiving an extra $50 per day to work without paying attention to fall protection requirements, would it be worth it to you?” All three questions had four possible answers, which ranged from the least safety-conscious attitude to the most safety-conscious attitude. Respondents could also select “don't know,” which was counted as a missing response. The distribution of responses for the first question was very skewed (positive), so it was discarded. For each of the remaining questions, a majority indicated a positive attitude.

Electrical safety attitude was measured with the same set of three questions, with the subject being electrical safety rather than fall safety. The distribution of responses to the first question was highly skewed (positive). The other two questions showed greater variability than the first. For each question, a majority indicated a positive safety attitude.

Baseline safety attitudes (included in the analysis as the mean of the last two questions) did not correlate with baseline knowledge. Both fall safety and electrical safety attitudes performed similarly when baseline values were compared with follow-up assessment, using the Wilcoxon signed rank test for paired observations not normally distributed.

There were no significant improvements in the safety attitude questions describing OSHA regulations as a hindrance rather than a benefit. However, significantly fewer workers stated they would accept $50 to work less safely after training both for fall safety (p=0.005) and for electrical safety (p=0.004).

Self-reported work practices

Self-reported fall and electrical work practices were separately assessed with a series of questions that encompassed the identification of workplace hazards at any time in the past, the action taken by the participant, and the results of the action, or, if no action was taken, the reason why. Because the “ever” time frame was used to identify hazards, the responses at follow-up were not directly comparable, and are therefore not shown. This problem could be corrected in the future by selecting a specified time frame, which would require a larger sample size.

Table 2 describes responses to the question “Have you ever seen a situation in your worksite in which fall protection was not being used in an appropriate and safe way?” and to subsequent questions about whether action was taken to correct the hazard. Fewer respondents had identified an electrical safety hazard, but the action responses were similar. Using logistic regression, associations between fall hazard identification at baseline and age, education, union status, and birth country were assessed. Only age was associated with fall hazard identification (OR=1.06, 95% CI 1.02, 1.09). The older workers were more likely to identify fall hazard than younger workers. For the subset (n=63) that was asked about prior training, training was also associated with the likelihood of identifying a fall hazard (OR=5.97, 95% CI 1.68, 21.2).

Table 2.

Distribution of baseline responses by construction workers to fall hazard identification, and corresponding remediation action questions^a

^an=175 for question 1, but diminishes for subsequent questions based on prior responses.

^bThe total number of respondents answering this question was less than 175 based on answers to the previous question or missing responses.

^cPercentages may add up to > 100 because respondents could choose all responses that applied.

OSHA = Occupational Safety and Health Administration

Safety climate

Mean safety climate at the most recent job was determined for all respondents who missed no more than one of the items. Items included statements such as, “I feel free to report safety violations where I work,” for which the participant checked a score ranging from 1 (strongly disagree) to 4 (strongly agree). Two versions of the climate scale were used—one with six items and one with seven—and mean scores were computed.²³

For 164 respondents at baseline, the mean score was 3.2 (SD=0.56). For purposes of regression modeling, mean safety climate was turned into a dichotomous variable divided at the 45th percentile due to the highly skewed distribution. Using logistic regression, no associations were found between the dichotomous (better vs. worse) safety climate score at baseline (as a dependent variable) and age, education, union status, and years in the union. The association between safety climate and birth country disappeared with the dichotomized version. Safety climate as an independent variable at baseline was not associated with knowledge or attitude scores. Safety climate at baseline did not predict changes in knowledge or attitude at follow-up.

Safety climate at the most recent jobsite was also measured at follow-up, largely at different work locations because of the transient nature of construction work, and did not change overall from baseline to follow-up. For the 84 paired respondents, there was no improvement from baseline to follow-up (mean = 0.01; p=0.84 from a paired t-test).

Linear regression analysis was used to explore the relationship between the mean safety climate score on the follow-up survey and improvements in safety attitude (n=83). The most recent safety climate was associated with an improvement in fall safety attitude (slope = 0.49, p<0.005) when adjusted by birth country (slope = 0.51, p<0.001). Workers born in Mexico had more positive attitudes at baseline and experienced a greater improvement in attitude than workers born in the U.S. Because birth country also had some association with safety climate, adjustment for it was included to get the best estimate of the effect of safety climate.

DISCUSSION

Measurable improvements in knowledge and attitudes were demonstrated three months after a one-hour hazard awareness training session that was provided in the context of a union-based apprenticeship or journeyman training program. The setting implied continued, hands-on reinforcement of information either through continued employment as a journeyman, or through the apprenticeship learning process. These findings are consistent with results from an earlier survey of union trainers. In this prior research, trainers who reported an ongoing relationship with their trainees were more likely than trainers who did not report such ongoing contact to estimate that the training had a positive impact on trainee safety.²⁷ However, Lipscomb and others have described a lack of safety reinforcement by jobsite mentors in a union-based apprenticeship that led to a lack of actual safety performance on the job, emphasizing the importance of factors other than formal training.¹⁵

Although we were unable to assess self-reported work practices during follow-up, at baseline both self-reported hazard identification and subsequent action to seek remediation of the hazard improved with age as well as with previous training. Safety climate at baseline did not predict subsequent knowledge gains, although safety climate at the current job on follow-up was associated with improved individual attitudes toward safety.

Because the study population included a substantial number of foreign-born Hispanic workers as well as a smaller number of U.S.-born Hispanic workers, we were able to explore the association between demographics and various outcomes within the population of union roofers. Two findings were of particular interest. First, both at baseline and at follow-up, foreign-born Hispanic workers had more positive safety attitudes than U.S.-born workers, and achieved knowledge gains despite classroom instruction given in English. (In these settings, groups of workers were observed sitting together for peer translation.) Second, when asked at baseline whether they had identified and acted upon hazards at work at any time in the past, foreign-born workers were as likely as U.S.-born workers to have done so.

Limitations

This study was designed as a pilot project and was limited by a small sample size and a lack of objective verification of reported work practices. Because it was conducted within a union-based partnership in which all workers were in the process of receiving 10-hour OSHA training, we did not control for union status and had no control subjects who did not receive training. As such, it was impossible to distinguish between specific knowledge and attitude changes that could reasonably be attributed to the training intervention and those that may have been acquired during the course of the hands-on apprenticeship training—a question that should be addressed in future studies.

While the knowledge increases were statistically significant, the magnitude of increase (e.g., from 40% correct answers to 52% correct answers on fall hazards) was modest. However, given that these items included only those rated to be meaningful for worker safety, the improvement is both heartening and likely to be meaningful.

CONCLUSION

Within a unionized workforce, which in all likelihood represents a select immigrant worker population, immigrant workers gained from hazard awareness training and reported similar hazard-identification and remediation practices as their U.S.-born counterparts. Characteristics of both union membership and of training should be further explored through externally verified outcome measures to (1) better understand their impact as interventions and (2) determine whether and how these interventions might reduce the disproportionately high rate of work-related injuries among foreign-born Hispanic workers.