EVALUATING GOALS IN WORKER HEALTH PROTECTION USING A PARTICIPATORY DESIGN AND AN EVALUATION CHECKLIST

Abstract

Spanish-speaking immigrant workers in construction are considered hard to reach and at high risk for work-related injury and fatality. This evaluation study describes the use of participatory methods and an evaluation checklist to consider a health and safety (H&S) training program for these workers. A previously developed training manual and model were disseminated to eight worker centers (WCs) through participatory research collaboration. It incorporated H&S training for workers while strengthening the role of WCs as sources of leadership development and worker empowerment. Design, delivery, reaction, application, and extension were assessed through individual interviews with participants, trained trainers, and center staff and through observation of training sessions and partner debriefs; pre- and post-training tests assessed participant learning. Results indicate moderate learning and application by participants and strong evidence for structural gains in and among WCs. We conclude that such partnerships and models are valuable tools for collaborating with hard-to-reach workers.

This article describes the evaluation of a health and safety training program for immigrant Latino workers using worker centers (WCs). Our intervention was built on a successful model used in union-based construction apprenticeship. It was then upgraded to meet the requirements of the Occupational Safety and Health Administration (OSHA) 10-Hour Construction Hazard Awareness training certification program, and further adapted to include Freirian methods of adult participation and peer education [1-3]. In the current project, we aimed to integrate construction occupational health and safety hazard awareness training into the activities of WCs and to provide the centers with additional tools. We also assessed our ability to disseminate the program. University researches and WC staff collaborated in this effort.

Immigrant Latino construction workers are at significantly greater risk of death from falls, the leading cause of construction deaths, compared both to all U.S. workers and to U.S.-born Latino workers. Furthermore, workers in enterprises employing fewer than 10 workers are also at significantly increased risk of fatal falls [4]. Documentation status may place immigrant workers at greater risk of entering the informal work sector, dominated by small enterprises and contingent working arrangements.

Day labor is the most extreme example of informal working arrangements. It is short-term, involving workers in a broad range of general and some specialized tasks, mostly in manual labor in construction, moving and materials transport, landscaping, and painting. Day laborers most frequently work for homeowners/ renters and contractors. Valenzuela and colleagues [5] conservatively estimated that 117,600 workers were seeking employment or working in day labor in all regions of the country in 2004. Day labor is also completely contingent and highly unpredictable; while many day laborers are paid decent hourly wages, the instability of the work and wage theft together place most firmly in the category of working poor. Verbal employment “contracts” can be made and broken by the hour, and employer responsibility for health and safety is diffused or completely lost because of the informality of the work. Work lacking in legal protection, social benefits, and worker power has been termed precarious [6, 7, 8], and has been linked to poor occupational health outcomes in immigrant workers [9]. Recent poor economic conditions in the United States have badly affected day laborers, leaving many in a position of severe economic insecurity10]. In a study of day laborers at two WCs and one street corner hiring site, Seixas and colleagues [11] found that after controlling for type of work, immigrant day laborers were 1.5 to two times more likely than non-immigrant day laborers to report exposure to hazardous conditions. From their data, they estimated an injury rate of 31 recordable injuries per 100 full-time employees, as compared to a recordable injury rate for construction and warehouse employees as a whole in 2004 of 6.2 to 9.3 per 100 FTE.

Worker centers have developed in response to the extreme forms of exploitation often experienced by immigrant workers. They focus largely on empowerment and leadership development to address wage theft and discrimination, as well as to allow mutual support for work, learning, and other activities [12]. These activities make them natural locations for our training intervention; WCs are in direct contact with immigrant Latino construction workers who often (but not always) engaged in day labor or other informal working arrangements.

In a research setting, the gold standard for testing an intervention is demonstrating that it caused the hoped-for outcomes. The clearest way to do this is through experimental and quasi-experimental research designs. In the evaluation tradition, however, it is acknowledged that these may not be feasible in a community setting, and, given their highly controlled design, also may not be able to capture the full extent of the intentions and outcomes of complex initiatives [13, pp. 293-304]. Our pilot efforts at quasi-experimental design for assessing trainee learning quickly highlighted oft-cited challenges to carrying out such projects in community-based partnerships, including stakeholder frustration [14], multiple orientations toward the endeavor [15, 16, 17], and a desire to make definitive statements about the success of a program at an inappropriate point in its maturation [17, 18]. In response, we adopted a simpler design, using a hybrid of the spirit of empowerment evaluation [19, 20, pp. 321-350] and the structure of practical participatory evaluation [21], both forms of participatory evaluation. Data collection and analyses were guided by checklist tools [22, 23]. Although the checklist tool dictates that all checkpoints are outcomes to be measured, we have incorporated the needs assessment into the introduction, and arranged the remaining checkpoints in a way that better reflects the traditional structure of a research report. Greater methodological detail on results of participant learning and application checkpoints are available elsewhere [24]. Here our aim is to provide an outline of all evaluative activities to promote discussion [18] of how to implement, judge, and use partnered efforts at protecting worker health.

METHODS

Evaluation

Scriven’s Training Evaluation Checklist (TEC) [23] describes 11 points that should be considered in assessing the value of training programs. Table 1 provides the checklist, our data sources, and basic notes on analyses for each item. As dictated by a practical participatory evaluation framework, a team of evaluators (university researchers, advisory board) and a team of expert practitioners (WC staff, health and safety trainers authorized by OSHA) worked together. Evidence garnered at each point sometimes relied more heavily on one team or another. Especially in implementation and piloting phases, we focused on making the program more accessible and useful. We considered the individual level (reaction, learning, retention, applicability) as well as the group-center and supra-center levels (design, delivery, extension, value, alternatives) and modified details of the intervention and evaluation as the project progressed. This project was approved by the University of Illinois at Chicago Institutional Review Board.

Table 1.

Components, Data Sources, and Analyses Used in Evaluation of More than Training Program, 2008-2011

Checkpoint	Assessed	Source(s) of data	Analyses
Need	Yes	Published literature; OH&S injury statistics	Narrative review; N/Aa
Design	Yes	Training manual; published literature; funding proposal; partnership meeting notes	N/A
Delivery	Yes	Observation of train-the-trainer sessions, pre-training preparation, training sessions; de-brief with trainers; attendance logs; semi-structured interviews with WC staff	Review of observation and debrief notes; simple counts compared to goals; standard memo, coding, typology development
Reaction	Yes	Observation of training; training manual section assessments; post-training debrief	Review of observation notes; simple counts; review of debrief notes
Learning	Yes	Pre- and post-training trainee questionnaires; semi-structured three month follow-up telephone interviews with trainees	Descriptive statistics, McNemar’s non-parametric test of matched pairs, code development and simple pre/post counts; directed content analysis—code development, simple before/after counts of frequency and content, comparison pre-post
Retention	Not assessed	—	—
Application	Moderately assessed	Semi-structured three month follow-up telephone interviews with trainees	Directed content analysis—code development, simple post/follow-up counts and comparisons of content and frequency
Extension	Yes	Semi-structured interviews with WC staff; logs of interaction with WCs; reported independent H&S initiatives at WCs	Conventional content analysis—standard memo, coding, typology development; description from lists
Value	Yes	All data sources	Discussion with partners using all data sources
Alternatives	Yes	Published literature; partnership meeting notes; logs of interaction with WCs; meetings with enforcement agencies	Narrative review; review of meeting notes
Return on Investment	Moderately assessed	All data sources	Discussion with partners using all data sources

^aN/A = not applicable

Delivery of Training Program

The Day Laborers’ Health and Safety Workbook (DLH&SW) [3] was developed by New Labor with researchers at Rutgers University Occupational Training and Education Consortium (OTEC), and the University of Medicine and Dentistry of New Jersey’s School of Public Health. It provides solid factual information that satisfies the requirements for subject and time dedicated to each theme for the OSHA 10-hour construction hazard safety card. It has an additional section, titled “Job Fear,” which deals with the social and power contexts of day labor. The training is provided over two full days. The workbook uses the Small Group Activity Method, which is learner-centered, and employs three learning pathways: from facilitator to facilitator, facilitator to trainee, and trainee to facilitator. The workbook activities are to be led by trained facilitators. Each includes fact sheets on a specific health and safety topic and one or more problem-solving activities that require using the fact sheet information. Working in small groups, trainees acquire safety information, discuss challenges, and define their responses. In this way the training encourages health and safety knowledge communication and teamwork skills, as well as thought about application at worksites.

Influenced by the broader health promotion literature and our partners’ experiences with peer health and safety trainers [1, 25], we incorporated Worker Leaders (WLs) as facilitators. Worker leaders were selected by WC staff using their existing strategies for leadership development. They were trained regionally on the content in the DLH&SW as well as on organization, recruitment, and facilitation skills over a three-day period by one of our OSHA-authorized trainers. The training was rooted in the values and methods of Popular Education, which contextualizes learning by class, social structure, and explicit political struggle. Worker centers and WLs were also responsible for recruiting participant trainees to their own sessions. Inclusion criteria for trainees in our intervention were to work in construction activities and be a Spanish speaker.

Pre-training preparation followed a standard format the day immediately prior to the start of a training session, and was facilitated and observed by one of three partner OSHA-authorized trainers. All training sessions were supported by authorized trainers and observed by at least one member of the research team. Presence of an authorized trainer is required for granting the 10-hour completion card; more important, participating authorized trainers also played key roles in creating an atmosphere of trust and open discussion, and in capacity-building, at participating WCs. Two of the authorized trainers had played central roles in development of the workbook and training model used.

Data Collection and Analyses

Notes on data sources and analyses are listed in Table 1. Detail and rationales for some of the checkpoints are described below.

Trainee Learning

We used pre- and post-training questionnaires to gather data on demographics, work history, injury history, specific content knowledge, approximated worksite decision-making, and intended future health and safety practices (post-test only). After consent, the Spanish-language learning assessments were administered by a researcher. Multiple-choice questions inquired about concrete facts from the most emphasized sections of the training manual. Two other questions assessed knowledge in an applied way. In one, participants were asked to point out errors in health and safety practices depicted in a photograph of a worker on a ladder. Criteria we used to assess answers are described in Table 2. In the other question, participants read a short vignette describing a worker’s arrival at a worksite with inadequate safety measures. Workers were queried with the following prompt: “Even though Juan is worried about some of the conditions at the worksite, he wants to work. Do you have any advice for Juan?”

Table 2.

Categorical Criteria for Safe Ladder Use Critique, Year 3, More than Training Project, 2008-2011

Preferred answers	Other answers
Answers: Ladder is too short for task; worker is standing on the top rung; the ladder is not obviously secured; the worker’s center of gravity is outside the ladder; he is not wearing a harness but also not maintaining three points of contact with the ladder	Answers: Answers other than the preferred (i.e., worker’s dress or lack of personal protective equipment)
	Incorrect = clearly factually incorrect, or unintelligible, or possible but not visible in photo
Weak = the idea is assumable with some inference by analyst but language is non-specific or incomplete	Weak = not patently incorrect but perhaps not related directly to ladders, not among the most obvious important answers, not specific enough to judge more strongly, any reference to general PPE, object unclear
Adequate = clearly and unequivocally indicates an error in a factually correct manner—broad concept	Adequate = clearly indicates a legitimate problem, though may not be among the most obvious
Strong = answer contains all components/details necessary to fully describe the problem indicated—concepts, context, and words	Strong = answer fits criteria for strong, specific, but relates not to the most obvious pictured concerns

Application

Short three-month follow-up telephone interviews were attempted for each participant who consented to be re-contacted, querying if and how participants had used the information and skills learned in training. Answers were compared to predicted use from the post-training test.

Extension:

We considered three stages: the Year 1 extension of the program in Site 1; Year 2 maintenance in Site 1 and expansion to Sites 2, 3, and 4; Year 3 maintenance of Year 1 and Year 2 sites, and expansion to Sites 5, 6 and 7.

Value:

We considered the training, itself, as well as aspects of partnerships developed and strengthened. We created a list of strengths and weaknesses of the training effort garnered from the viewpoints of the evaluation and practitioner teams, and the formal data collection. Since we see some strengths and weaknesses as more important than others, we weighted them using a simple system. Next, items weighted equally in the strengths and weaknesses categories canceled each other out, as in an algebraic equation.

Alternatives:

We assessed alternatives by considering other possible approaches to the same training.

Return on Investment:

The following aspects of return on investment were considered: social and physical environment improvements, ethical and legal considerations, the value of the model in other contexts or uses, and the comparative value of this approach versus others.

RESULTS

Design

Resources already present in WCs, though vital, were not sufficient. Interviews prior to the project and after its completion indicated that most WCs lacked both monetary resources and staff time to launch health and safety (H&S) initiatives, and in many cases also lacked health and safety expertise. Through the partnership with a center with significant health and safety experience, and with university health and safety researchers, WCs gained content training and resources as well as funding to support staff time. Centers were specifically responsible for management of their portions of the project budget; for providing space for training, advertising, and recruitment; for securing food; and for choosing WLs. University researchers took the lead on administrative facilitation, evaluation of the project, and development of learning assessment tools. The simpler design alleviated WC partner burden and frustration. Worker center and university partners maintained regular communication about the project to discuss implementation, challenges, successes, and modification of learning assessment tools.

Delivery

Each worker center selected and provided transportation for WLs to train-the-trainer (TTT) sessions and supported them throughout the project. Centers generally already had their own frameworks for leadership development, and in interviews with our evaluator team, they described those strategies. Most centers had had the opportunity to observe regular participants in center activities, and had some idea of whom they might approach. One center director’s characterization of initial selection as a “logical choice” was typical. Other frequently cited reasons for choosing WLs were experience working in construction, availability to travel to regional TTT sessions, and ability to commit to the time required of trainers.

Many also went on to describe a more systematic and strategic process. For example, one center director described observation of an individual’s leadership potential over time, gauging of possible interest in health and safety, and a “purposeful moment” wherein the director expressed respect for and confidence in the member, and asked for help in the mission. Then the opportunity for a leadership role was offered. Other center directors expressed this process in terms of the potential WL’s understanding of the mission of the center, and how health and safety training fit within that larger mission. Though there were a few attrition challenges given the mobility of the workforce, most WLs remained throughout the project’s duration.

One center was a notable standout in terms of its strategy. Rather than selecting WLs from among its membership, it used staff members as trainers, citing the extreme difficulty of getting involvement in the center, and even getting its target audience to come out for meetings, because of a prosecutorial atmosphere toward immigrant workers in that city.

Our debriefings with center staff and WLs, and formal interviews with center staff, indicate that WCs used a wide variety of strategies for recruiting trainees, and that they modified these over the course of the project using lessons learned. For example, some centers had regular meetings at which they announced the trainings. Others placed fliers in churches and Latino-owned groceries and businesses, and advertised the events on community calendars. After realizing that referring to OSHA 10-hour training meant little to potential participants who didn’t know what that was, one committee composed of a volunteer, the WLs, and an organizer employee leveraged their good relationships with a local radio station to provide brief educational and recruitment spots on the air; as workers were being invited to attend, they were also told what OSHA was and what such trainings entailed. Center directors agreed that after initial training sessions had been completed, word-of-mouth aided recruitment efforts, as did offering small incentives, such as grocery store gift certificates. Desired attendance was not reached at every training session. One center director commented that it was “beautiful to see how this [ease of recruitment] changed over time—self-worth reaches a different level; they could develop themselves more as people [by] seeking opportunities.” Participation in training across the seven project sites is summarized in Table 3.

Table 3.

Years of Participation, Number of Worker Leaders, Worker Trainees Receiving 10-hour Card, and Person-Hours of Training by Site, More than Training Project, 2008-2011

Site	Years of participation	Worker leaders trained/person-hours of training	Worker participants trained/person-hours of traininga
Site 1	1,2,3	6/60	133/1370
Site 2	2,3	8/80	55/560
Site 3	2,3	4/40	79/800
Site 4	2,3	3/30	58/595
Site 5	3	11/110	58/585
Site 6	3	3/30	29/290
Site 7	3	4/40	32/325
Total		32/320	444/4525

^a17 worker participants completed only one day of training. They are included in counts of total workers trained (because they did not receive the 10-hour card) but hours of training received are included in counts of person-hours of training.

Socio-demographic information obtained from learner assessments indicate that most participants had moderate levels of formal education (Years 2 and 3 mean 9.4 years, range 0-17 years), though educational attainment ranged from elementary school to college. Participants with education levels greater than or equal to high school totaled 41 percent of the group. Trainees were largely, but not exclusively, male, and most had been working in the United States for some time (Years 2 and 3 participants: mean 13.2 years, median 11 years, range 0-42 years). A wide range of ages was represented, though the mean and median ages were 38.5 and 37.5, and most participants were born in Mexico. About two-thirds felt they spoke English very well or well, while close to 30 percent said they could not speak English well or at all (data not shown). Fewer than half had prior experience with health and safety training, especially in Spanish. Likewise, work in “construction activities” encompassed a wide range of work activities for this population.

Reaction to Training

All participants were enthusiastic about the training program, and also made suggestions for improvement, which were incorporated where possible. Both participants and center leaders cited the small group methods as very positive, for team-building and for allowing participation from workers with varied levels of workplace experience, formal schooling, and literacy. In three-month follow-up interviews, many participants felt positive not only about the training itself, but about what it signified to them: an interest in “workers like us.”

Participant Learning

On discrete questionnaire items, we found a statistically significant correct answer gain on two of the four items. Responses to applied questions showed modest improvement in identifying hazards and responding to an unsafe work site with assertive, systematic suggestions [see 24].

Application

We have limited but positive evidence for application of knowledge gained by trainees at their worksites. We reached 33 percent of participants (N = 84) in Year 3 (Figure 1). Most participants reported that they had been able to use knowledge gained at worksites. When a follow-up question probed what they had found most useful, they reported that the Job Fear, Ladders, and Electrical Hazards sections were useful, and that in general, they were “más concienciado” (more aware) of hazards at their work sites. Of interest from an organizing angle, many report talking to their co-workers and friends about construction site hazards, and approaching supervisors about their concerns.

Figure 1.

Planned (post-training) and reported application of knowledge at follow-up (three months), Year 3 participants, More than Training project, 2008-2011.

Worker leaders have found application for their facilitation and organizing training in other contexts. For example, leaders at one WC meet regularly to discuss strategic ways that they might engage workers around health and safety. Others have secured leadership positions on advisory boards as a result of the leadership development learned through this project. Finally, one center reported filing its first OSHA complaint after gaining expertise and understanding through the center’s participation in this project.

Extension

Extension may also be thought of as dissemination. In one sense, this project itself was the result of dissemination of a program used on the East Coast to a broader geographic area. From that perspective, it has been successfully extended. The workbook was subtly adapted for use in each geographic area, more photographs and hands-on activities were added, regional trainers were trained and had the opportunity to form networks among themselves throughout the program period, and the training program was delivered as intended, at different times of the year and at different sites at different levels of organizational development, using different trainers (who were regionally trained and attached to a WC) each time.

This is not the only way in which the training has been successfully extended. Worker centers alone and in partnership, and WLs, have extended the training and model far beyond expectations. This began quite early, when WLs who had been trained but could not offer the certification to trainees without the presence of an OSHA-authorized trainer began asking why they could not be authorized. They were already thinking about how they might extend their learning to larger groups of people, and beginning to occupy a social role as organizers and leaders. They also understood the value of connecting with fellow WLs in other cities; rather than provision of TTT sessions in their own city, the WLs requested regional training sessions involving several centers, traveling very long distances to participate in training. Worker leaders also report a closer relationship to WCs as a result of their participation.

Worker leaders reported training workers on their own time through outreach at work sites of which they are aware. One center’s leaders have become especially creative in these efforts. For example, they have taken personal protective equipment to street corners where people wait for work and given the equipment away through a raffle. Subsequently, they used that event as a teaching moment to explain the equipment’s proper use. Worker leaders have also built skills that can serve other aspects of center work. Several now serve on advisory boards, both at their home centers and in national networks.

Individual staff at WCs also extended the reach of this project. In one example, an organizer became an OSHA-authorized trainer. This designation allows the center to provide monthly OSHA 10-hour trainings that local contractors pay for, for which the center breaks even financially, but through which it is able to recruit new members. All centers have found the project to be a valuable recruitment tool, and several centers stated that it is their most valuable tool.

Worker centers as organizations report that the framework used in this project has systematized their approach to other issues of priority for their groups. A center director described improvements in the center’s work on wage theft, using strategies gained from this training program and the connections made to those in leadership positions at other centers for brainstorming and strategic thinking. Another center reported that, though it had long focused on leadership development, this project had driven home the point that leadership development requires intentional programs. This center reported realizing that the ability to pull workers into activities where they build specific skills and can move on to bigger things had changed its approach to leadership development. The center director also stated that his group had recognized the critical importance of evaluation and debriefing as components for ongoing organizational improvement. Another center invited its advisory board to attend the trainings at its center as a way to integrate health and safety into board development.

Several centers reported that this project had strengthened their relationship to gatekeepers in the health and safety world. For example, two of the Midwest WCs now have representatives on the advisory board of the Illinois Occupational Surveillance Program. Another reported that it has formed relationships with health and safety specialists and a local university. Still another reported ongoing dialog with its regional OSHA office, and regular visits by representatives of that office to distribute materials and perform other outreach activities.

Finally, WCs have secured ongoing funding for their own health and safety initiatives. Two participating WCs have led successful OSHA funding requests for health and safety training at multiple centers through their national WC networks. In one of those initiatives, the same center that had expressed greater understanding of the importance of evaluation requested consulting help from a member of our evaluation team for the grant period. A third, having developed stronger relationships with another center in its state, contracted graphic designers, Popular Education artists, and photographers to help the center develop its own manual, and partnered with the other centers on a statewide initiative. It has also begun to consider developing training modules on more specific subject matter for construction workers. Centers have furthermore begun to think about extending this training model in terms of other priority worker groups. Several centers are actively considering the development of training programs for domestic workers and independent truck owner/drivers.

One WC, despite being involved in both of the aforementioned new funding initiatives, reported feelings of frustration that the center was “not intentional enough” to follow up with participating trainees as much as they would have liked to, and wonders about continued opportunity for this sort of work. A second center reported that demand for training outpaces the center’s available funding to provide it. A third center, which opted to train its staff rather than WLs to provide the training, indicated that health and safety is not actively on the center’s agenda. Staff who participated are no longer part of the organization, and their involvement in health and safety is on an “as requested” basis.

Value

Aspects of the program (strengths and weaknesses) considered and the weights (importance) assigned to them are shown in Table 4. Our assessment finds that strengths outweigh weaknesses in both number and importance.

Table 4.

Strengths and Weaknesses Considered, and Weights Assigned to Each in Determining Program Value, More than Training Project, 2008-2001

Strengths	Level of importance (weight)		Weaknesses
Training solid proposal given assessment of need and other alternatives	**
Very strong design, based on need, ways of working of organizations, and adult learning pedagogy	***	**	Tools used to assess learning and retention still developing
Good fidelity of delivery across sites, improved with time and feedback	**	**	Some difficulty getting appropriate attendance at some sites and in some seasons—learned with time
Very highly rated by participants and other stakeholders	*
Good observed and tested mastery of attitude and values, very good observed and reported development of networks	**	**	Moderate to limited assessment of knowledge gain (better in open-ended questions)
Self-reported and observed retention of attitudes and values	**	**	Difficult to gauge retention of knowledge—unclear if tools ideal; can’t observe worksites; self-report follow-up interviews; high attrition to follow-up
Good reported applicability	**	**	Difficult to judge frequency/quality of applicability (see above)
Very strong evidence for intended and unintended extension	***

Key:

^*Not very important

^**Important

^***Very important.

Alternatives

The OSHA 10-hour training program exists and is theoretically available independently of this project. However, normal presentation, in English and using didactic lecture methods, is not appropriate for this group for several reasons. Early in our project, we had hoped to build additional capacity by connecting WCs to local OSHA-authorized trainers, rather than sending those affiliated with the project to each. The dearth of Spanish-language trainers is impossible to over-emphasize, and no one was using a hands-on, participatory training style such as this one. Most trainers use lengthy, unidirectional slide-show presentations. We question the effectiveness of this style in general, and for this population specifically.

Employment relations can affect worker health through two different pathways: through conditions at a specific point of production, or through outcomes of the labor process—pay and benefits—that affect workers’ lives in and outside the workplace. Normally, the former is the purview of occupational health and safety (OH&S) professionals—but any attempt to modify that for this group of workers requires mindfulness about their conditions of employment. Working with similar groups and approaches, Ochsner and her colleagues [26] highlight the reality that training is a vital but partial solution for these workers, given the precarious employment conditions they experience. They describe ongoing efforts to provide additional resources in partnership with a WC for this group (moving safety audits to the work site itself, nascent relationships with area OSHA officers), aiming at continued relational and organizational strength-building. This is not a small challenge, and it is thus also a challenge to evaluating efforts to impact the health and safety of workers. De Souza and her colleagues [27] describe similar challenges to both conducting and evaluating a shorter training effort with day laborers. Their study revealed modest knowledge gain and some ability to apply knowledge and skills obtained in training; like ours, their efforts to assess application were limited by the need to use self-reported application data and by challenges to follow-up.

Another alternative is active outreach and enforcement on the part of federal and state OSHA. Both have recently begun, including national outreach to low-English-literacy populations, a Latino worker safety summit, a national emphasis program targeting residential construction, and increased funding for development of new training programs addressing the rights of workers through OSHA’s Susan Harwood Grants program. However, given the small resources these agencies have available and the need for wider reach, partnerships with stakeholders are critical components and are currently in the early stages. While several WCs cited closer relationships with area OSHA officers and the official health and safety world in general as a result of this project—undeniably positive developments—they are not enough to depend on exclusively.

Outreach to employers, which also recognizes the precarious nature of this sort of work and the power dynamics inherent in it [26, 28], is a valuable alternative, and one that many WCs engage in already. Likewise, we are aware of projects that intend to improve the safety climate at residential construction sites [29] by training employers. Employer training may be especially beneficial in the context of the capacity of WCs to serve as workforce development agents, and the frequency with which immigrant workers using WCs play the role of both employee and employer, by being hired and then recruiting their peers to a specific project. Given knowledge about management’s impact on workplace health and safety, such efforts are a valuable component of improving workplace health and safety for this workforce.

Our experience suggests that despite competing needs to work, the sustained training period (ongoing recruiting, leadership development and training provision for WLs, two days for participants) allows relationship- and trust-building among trainees, authorized trainers, and WLs necessary to gain full benefit from the program. It also fulfills an organizing goal in allowing the participants to begin to feel at home at the host WC. In addition, the workbook and training modality of this project explicitly acknowledges the importance of the intersection of other influences with the conditions at any given worksite in training style, the setting of objectives, and evaluation. We cannot understand working conditions and safety for this group of workers without understanding power relations, policies, and employment practices. Such an understanding, for this workforce, makes it clear that work in all three areas remains a priority.

Return on Investment

Economic

The total direct cost of the funded portion of this project was approximately $1,000,000, which included the supply, travel, and personnel costs incurred at the university, as well as an amount given to each center to attend regional train-the-trainer sessions and cover the costs of providing the training at their centers, as well as an extensive evaluation component. Initial work developing the partnership, adapting the workbook for local use, recruiting advisory committee members from diverse academic backgrounds, and analyzing and presenting baseline sources of information preceded the actual training program. Over three years, we trained 444 workers (4,525 person-hours), and 32 WLs (320 person-hours). A training grant provided by OSHA expects individuals to be trained for less than $500 per worker. Our per-trainee costs far exceed that benchmark, although the research component accounts for a portion of that. Working backwards, had each WC been separately funded to adapt workbook, materials, and evaluation procedures and to train the peer trainers, the provision of $500 per worker would be sufficient to sustain these programs to complete the OSHA 10-hour hazard- awareness training.

Most WC staff who were interviewed agreed that providing the training would not have been possible without grant funds. They also cited the logistical and program support provided by the university teams as highly valuable. Many also were able to leverage the existence of the training for donations of lunches during the training sessions, and safety equipment for the training and the center. In that sense, the economic investment was smaller than the return, though this may cause problems for the ongoing ability to sustain the training program in the absence of grant funding.

Environmental

Investment in the social environment was among the most important goals of the project. Workers gained greater connection to each other in terms of health and safety at work sites, WLs took their roles seriously, and WCs that did not previously focus on health and safety report that this is now on their agendas as a result of this project.

Ethical

Ethically speaking, humans have intrinsic value and should be protected in body and mind at work; such considerations are in line with the legal frameworks put forth by worker protections in this country [30]. This is not always the reality for many workers, day laborers included. This project works toward improving that situation through organization, training, consciousness-raising and capacity-building. It fulfills the requirements for OSHA 10-hour hazard-awareness training for the construction industry in part by letting workers know about their rights to a safe and healthy workplace, regardless of immigration status.

Extension

The existing extension and future potential far exceeded the imaginings and hopes of the planning group. Overall, most WCs already have found multiple ways to extend the training program, often with mentorship from staff at the WC who originally developed the workbook. Whether by securing additional funds to continue, by using the model in their ongoing activities, or by adapting it to better suit the needs of their membership, the program has been extended. Additionally, all but one center desire and have ongoing plans for continuing with a health and safety mission.

Exclusiveness

Given the circumstances this group of workers face, approaches such as the one used in this project appear to be among the best alternatives for increasing knowledge and values about health and safety in this workforce. Given that real protection involves individual as well as systemic change, complementary efforts such as those designed to work more closely with enforcement bodies and employers should also continue, ideally as a coordinated effort.

DISCUSSION

Many of the goals of this project were empowerment-focused, at the level of individual WLs and participants as well as at the organizational level. Because the timeframe and developing evaluation expertise on the part of the university research team precluded guided self-evaluation, it is perhaps most fair to say that our framework was empowerment-infused. We believe, however, that movement toward full empowerment evaluation at future moments in the life of this training program is a worthwhile goal, a strong fit with the organizing mission of WCs, and well within their capabilities. In unionized settings, worker-trainers have proven adept and valuable to improving the fit of health and safety training and the evaluative process; the authors suggest that such results would not be limited to a unionized work setting [31]. Defining a campaign or goal, taking stock, and planning for the future—the key phases of empowerment evaluation [20]—are precisely what high-functioning WCs already do. Moreover, the cyclic foci on reflection and action and the use of the simplest adequate data collection methods imperative in empowerment evaluation are not at all outside the realm of priorities in WCs. Additionally, as Fetterman [20] has noted, ongoing empowerment evaluation efforts do not preclude the use of external evaluations, and often provide useful data to such efforts.

The results of this evaluation should be interpreted with several limitations in mind. Tools used to assess participant learning are still not without challenges in administration, completeness, analysis, and interpretation. Likewise, follow-up data on application of training were self-reported and not the result of direct observation. It is important to note that our partners who originally developed this training program had found it to be both well-received and effective with day laborers in another geographic region [1]. Our work both assesses our ability to maintain positive reception and participant learning and skill-building, and builds on their methods for assessing these goals. In addition to providing a window to participant learning, these forms of assessment provided insight for future endeavors. Though photographs were used quite successfully to facilitate conversation in the training itself and were mirrored in learner assessment for consistency, our experience is identical to that noted by De Souza and her colleagues [27] when describing their use of photographs to assess learning in a pilot phase. Namely, photographs are too “busy,” and there are too many possible answers to point out. Our definitions for coding strong answers were met infrequently, and it is possible that the definition was too narrow and academic. In addition, in order to ask workers to focus on higher-order work practices (other than protective equipment), it would be important to show a work situation where the worker was using appropriate personal protective equipment. Given the worksite climate, in which workers must rely more heavily on themselves and on personal protective equipment than in a more managed environment where a hierarchy-of-controls approach is feasible, this emphasis on the part of workers may actually be an important finding in terms of self-efficacy. Alternatively, the evaluation may point out a weakness in the training—that the immediacy of the need engenders a focus on applying personal protective equipment among the participants, and that feedback to the facilitators to reinforce the superiority of higher levels of control is warranted. Though learner responses to applied questions indicated more critical ability to assess and act accordingly, a final lesson learned is that, for completely open-ended responses in written format, such as this one, assessing the answers may rely too heavily on the quite variable writing of the participants and distract from content. Future efforts should incorporate drawings but narrow the possibilities and ways in which participants may answer [27], and should consider expanded use of qualitative interviewing. Given those limitations, we have conservatively considered the results to constitute moderate evidence of learning and application. However, tool development up to the present point was based on pilot testing, feedback from users, and expert content knowledge, and theoretically was predicated on adult learning knowledge [3].

Gains in extension of the project model and in the social environment at WCs documented in this evaluation are potentially valuable. In a 2000 report on the education and training needs for occupational health and safety personnel, the Institute of Medicine suggested that, to address the health and safety needs of an increasingly diverse workforce laboring at scattered sites under conditions not well managed by our current system, we needed to add “a new and different model of practice” [32]. The committee encouraged the OH&S community to consider ways in which professionals might be trained by drawing from people within a non-standard work situation. It furthermore suggested an approach similar to other non-site-specific public health campaigns such as seatbelt use or smoking cessation, which employ population-level education, outreach, advocacy, training, and policy to create change. Worker leaders, trained and supported by WCs and OH&S professionals as they were in this project, may be one example of such a strategy for residential construction. Credentialing the WLs would aid this function.

Worker Leaders and WCs could also contribute as actors in building occupational injury and surveillance capacity for contingently employed immigrant workers, who are not well covered by current systems [33, 34]. This project did not include surveillance activities, but Lowry and her colleagues briefly explored active surveillance of injury at day labor hiring centers [35]. The authors suggested that structured, incentivized surveys at hiring centers might be a sufficiently systematic way to augment data on injury in this workforce. Strengthened academic/WC partnerships, inclusion of OH&S activities, and ability to independently obtain funds for this work within centers, as demonstrated in this project, could provide a basis from which to further explore and evaluate their suggestions.

CONCLUSIONS

When dealing with programs that ultimately aspire to large social and structural changes, it can be overwhelming to consider how to assess progress on any one of the more immediate goals, let alone the long-term ones. Practical participatory evaluation, with its explicit processes of co-generation of knowledge and its aim of capacity-building for all involved, turned out to be a very good framework for this program and the needs of all partners.

The Training Evaluation Checklist served two concrete and important purposes for our evaluators. First, it provided structure for evidence that otherwise felt difficult to organize, categorize, and explain to others [18]. Second, the checklist provides a way forward to future improvements and valuation of this training program. It highlights areas where better data are required for stronger evidence, and provides a comparable set of variables for benchmarking over time. We strongly encourage groups undertaking similar efforts to use these evaluative schemes for program improvement and capacity development.

This training workbook and adult learning method, administered in partnership through worker centers and using worker leaders, is a viable and valuable way to reach immigrant Latino construction workers, including many day laborers. Its design was a strong fit for use in a worker-center setting, and has provided benefit to participating centers beyond the granted project itself. Moreover, such an approach to “hard-to-reach” workers may also be a viable strategy for longer-term goals of reducing injuries and illnesses in this worker group, improving injury surveillance, and protecting the health of other excluded worker groups.

Biography

EMILY Q. AHONEN is an assistant professor with joint appointments in the Social and Behavioral Science and Environmental Health Science Departments at the Indiana University Richard M. Fairbanks School of Public Health in Indianapolis. She previously held a research faculty position in the Division of Environmental and Occupational Health Sciences at the University of Illinois at Chicago School of Public Health. Ahonen received MPH and PhD degrees from the Universitat Pompeu Fabra in Barcelona, Spain. Write to her at eqahonen@iupui.edu.

JOSEPH ZANONI, PhD, MILR, has 30 years of combined educational experience, developing and evaluating occupational health programs, and collaborating in research as a Research Assistant Professor in the Division of Occupational and Environmental Health Sciences, and is Director of Continuing Education and Outreach for the Illinois Occupational and Environmental Health and Safety Education and Research Center, at the University of Illinois at Chicago, School of Public Health. Write to him at jzanoni@uic.edu.

LINDA FORST is the head of Environmental and Occupational Health Sciences at the UIC School of Public Health. She has an MD from Michigan State University College of Medicine and heads the occupational surveillance program in Illinois. Write to her at forst-l@uic.edu.

MICHELE OCHSNER co-directs the Rutgers University Occupational Training and Education Consortium (http://smlr.rutgers.edu/research-and-centers/centers-and-programs/occupational-training-and-education-consortium). Her research and health and safety projects focus on training methods and organizational practices and policies that support worker participation and engagement. Write to her at mochsner@work.rutgers.edu.

LOU KIMMEL is a co-founder and Director of Operations and Membership for New Labor. He develops and leads train-the-trainer and leadership development courses, manages trainer networks and creates popular education curricula. Kimmel has trained hundreds of low-wage workers to be peer-trainers for worker centers and other community-based organizations throughout the country. He can be reached at lkimmel@newlabor.org.

CARMEN MARTINO co-directs the Rutgers Occupational Training and Education Consortium (OTEC) and conducts strategic research for employers, community-based organizations and unions at the national and local levels. He designs site-specific workforce development programs that build frontline leadership and organizational capacity. Contact him at cthndrrd@verizon.net.

ELISA RINGHOLM has been the Development Director of Latino Union of Chicago since 2009. She has a BS in Environmental Science and a BS in Anthropology from Loyola University Chicago. She is an award winning writer and internationally recognized researcher who has been a leader in anti-sexual violence, environmental, and social justice groups since her youth. Write to her at development@latinounion.org.

ERIC RODRÍGUEZ has been the Executive Director of Latino Union of Chicago, a nationally recognized workers’ center, since 2002. His focus on worker leadership development catalyzed Latino Union’s paradigm-shifting health and safety program, developed a workers’ cooperative social enterprise called Café Chicago, and sustained the Albany Park Workers’ Center as the only one of its kind in the Midwest. He was awarded MALDEF’s 2012 Community Service Award, a 2011 Chicago Community Trust Emerging Leader Fellowship, and a 35 Leaders Under 35 award in 2009. Eric is completing a degree in Non-profit Management at North Park University. Write to him at erodriguez@latinounion.org.

ADAM KADER is Worker Center Program Director at Arise Chicago. He has a Master’s degree in Urban Planning & Policy from the University of Illinois at Chicago. He is a member of the Labor Research Action Network and the Planners Network. He can be reached at adam@arisechicago.org.

ROSEMARY K. SOKAS, MD, MOH, is professor and chair, Department of Human Science, School of Nursing and Health Studies, Georgetown University. She previously directed the Division of Environmental and Occupational Health at the University of Illinois at Chicago School of Public Health, and most recently directed the Office of Occupational Medicine for the Occupational Safety and Health Administration. She holds an M.D. from Boston University School of Medicine, and an MOH and MSc from the Harvard School of Public Health. She can be reached at sokas@georgetown.edu.