Economic Evaluation of an Intervention Promoting Adoption of Occupational Sun Protection Policies

Abstract

OBJECTIVE:

Economic evaluation of an intervention promoting adoption of occupational sun protection actions by Colorado public-sector employers.

METHODS:

Randomized controlled trial with two-year follow-up conducted during 2010-2013. Thirty-three intervention and 30 attention-control worksites in final economic sample. Twenty-four-month intervention of personal contacts, training, and materials. Intervention delivery micro-costed. Costs of implemented actions from employer self-report.

RESULTS:

Twenty-four-month intervention costs: $121,789, 51.8% incurred by project staff (per-worksite mean=$1,732). Worksite costs: $58,631 (mean=$1,777). Per-employee costs: $118 project staff, $56 worksites. Materials cost: $5,990 (mean=$181). Intervention worksites implemented 72 non-training sun protection actions post-SSW (mean=2.18). Control worksites implemented 39 actions (mean=1.30). Total costs to intervention worksites of implementing the 72 post-SSW actions: $90,645 (mean=$2,747). Control worksite costs: $66,467 (mean=$2,216). Per-employee implementation costs are comparable to other worksite health interventions.

CONCLUSION:

SSW expanded adoption of sun protection actions at a reasonable per-employee cost.

Introduction

Skin cancer is the most common cancer in the U.S.^1,2 In 2012, 5.4M cases of keratinocyte cancers (i.e., basal or squamous cell) were diagnosed among 3.3M Americans. Invasive melanoma accounts for relatively fewer incident cases of skin cancer (96,480 estimated in 2019), but the melanoma incidence rate has increased sharply in the last 30 years, especially among Americans over age 50. Moreover, melanoma is responsible for a large majority of skin cancer deaths (7,230 estimated in 2019).³

Unprotected exposure to ultraviolet (UV) radiation is a major risk factor for all types of skin cancer.⁴ Outdoor workers are often exposed over many years to high levels of solar UV radiation⁵ and are at risk for skin cancer of all types^4,6, which is exacerbated by workers’ general reticence toward sun protection.^5,7,8 Melanoma mortality produced US$66.9 billion in productivity losses during 1990-2008 in the United States.⁹ Skin cancer prevention is therefore a priority due to its high prevalence³, recurrence, and treatment and lost-productivity costs.¹⁰

The objective of sun safety fits well with broader workplace initiatives to prevent injury, improve well-being, and address concerns over heat stress and climate change.¹¹ Research has established the value of workplace education in increasing workers’ interest in sun protection.^12-19 However, education may be even more useful as part of a comprehensive program in which employers adopt policies for routine sun safety training, environmental controls, and administrative procedures that place sun safety on organizational and individual agendas.

Sun Safe Workplaces (SSW) was a two-year randomized field trial of an intervention promoting employer adoption of sun safety policy and providing employee education on sun safety. Findings reported elsewhere^20-21 supported the hypotheses that employers receiving the SSW intervention adopted more occupational sun safety policies and implemented more sun safety actions for employees than employers receiving only basic sun safety communication. This paper presents the results of an economic evaluation of SSW.

Methods

The design and implementation of SSW are described elsewhere.^20-22 Briefly, a two-year randomized controlled trial enrolled 98 local Colorado government organizations (i.e., municipalities, counties, and special districts (public organizations providing water, sanitation, parks and recreation, and fire protection)) recruited between March 2011 and August 2011. The appropriate senior manager (e.g., city or county manager) provided organizational consent to participate. Eligible organizations had employees who worked outdoors in at least one of the service areas of public works, public safety, or parks and recreation. Organizations also provided formal written policies on employee safety and had at least five senior managers complete a baseline survey. Participating organizations were randomized to either the SSW intervention or an attention-control condition (i.e., basic sun safety communication). At the end of the intervention, project staff collected the most current policy documents by email or in person; senior managers who had completed the baseline survey (or their replacement as necessary) were post-tested online or by mail. Policy documents were coded for sun safety content by trained research assistants blind to condition.

Sun Safe Workplaces Intervention

SSW promoted adoption of workplace sun safety policies and provided training in personal sun protection for outdoor workers. The intervention began by project staff sending a Program Announcement Packet to the designated contact manager at each workplace and requesting an initial face-to-face meeting at which project staff covered: 1) Introduction to SSW; 2) Sun Safety Practices in the Workplace; 3) Sun Safety Policy for Outdoor Workers; 4) Sun Safety Policy Adoption; and 5) Sun Safety Policy Reinforcement and Maintenance. Staff also presented the SSW website, which described skin cancer and UV radiation, risk factors, personal risk assessment, and workplace prevention strategies (e.g., sunscreen, sunglasses, shade). The website also contained a Sun Safety Tool Box with a worksite audit, policy-writing tool, and fact sheets on sun safety and policy implementation barriers. Following the initial meeting, the contact manager scheduled sun safety training by project staff with various employee groups. The training fit into workplace safety/wellness and covered: 1) the U.S. skin cancer problem; 2) the sun, UV rays and skin cancer; 3) assessing personal risk; and 4) practicing sun safety and obtaining regular skin examinations. Workplace sun safety materials based on diffusion of innovation theory principles were sent in four sets (twice a year over two years) to the contact manager for distribution to employees, including 10 posters, six tip cards, a sticker, a sunscreen discount coupon, and a CD on total skin self-examination. Monthly follow-up contacts strengthened staff relationships with managers, supported managers’ decisions to write policies and train employees, and problem-solved barriers.

Attention-Control Intervention.

Control organizations received printed materials on occupational sun safety (one package annually) comprised of posters with personal protection messages and skin cancer incidence rates, risk assessment brochures, worksite guides, total skin self-examination CDs and bookmarks showing the ABCDEs of melanoma and skin self-examination, and an OSHA sun safety tip card. Project staff also made general sun safety presentations at state professional conferences that both control and intervention organizations may have attended.

Measures

Policy Measures.

Project staff obtained written policy documents pertaining to employee safety (e.g., safety and risk management manuals, employee handbooks, standard operating procedures, personnel manuals) from each organization. The policy coding protocol^22-23 covered 15 “content categories” in three domains: 1) environmental controls (provision of shade, schedule of outdoor work); 2) administrative procedures (risk assessment, training of employees and managers/supervisors, policy review/monitoring, resource allocation, communication with clients/patrons, compliance by contractors/subcontractors, and posting of UV Index); and 3) personal protection actions (use of UV-protective clothing [including long-sleeved shirt and long pants/skirt], hats, protective eyewear, and sunscreen [including sunscreen lip balm], and regular skin cancer screening [self-examination or clinical examination]). For each policy category, trained research assistants, blind to experimental condition, used the coding protocol to independently record the presence, strength, and intent of the policy, responsibility for policy implementation, and date adopted/revised. Specific “directives” within certain policies were also recorded to assess compliance with recommended best-practice sun safety^24-25 (i.e., work schedules that avoided midday hours outdoors [or not]; supervisor-delivered sun safety training [or self-directed training]; sunscreen with SPF 15 or higher [or not]; wide-brimmed hat [or any brim or no length specified]; long sleeves and long pants [or any sleeves/pants or length not specified]; UV-protective eyewear [or not]; clinical skin examination [or self-examination]; and interval of exam [annual, monthly, any interval, no interval specified]).

Policy Implementation Measures.

At posttest, senior managers reported on policy implementation (i.e., sun protection actions), along with job and demographic characteristics and attitudes toward workplace health and sun safety policy. They also indicated whether they were aware of a formal written sun protection policy (yes, no/don’t know), how well managers/employees were following the policy (1=very poorly, 5=very well), and whether employees were provided sun protection equipment (i.e., sunscreen, wide-brimmed hats, sunglasses, long-sleeved shirts, long pants, or outdoor shade) or adjusted work schedules to reduce their midday outdoor exposure (yes, no/don’t know).

Costs

Intervention costs, obtained from project accounting records, were defined as the value of resources used to develop, implement, and maintain the SSW intervention over the two-year trial period and were measured from an “inclusive” perspective that accounts for resources of both project staff and the participating worksites.²⁶ Research-related costs were excluded. All intervention components were classified as labor (e.g., meeting times) or non-labor (e.g., brochures). Total SSW intervention costs were organized into two broad components: “delivery” costs and “action” costs. Delivery costs were incurred to directly implement and maintain the SSW intervention and included costs of both project staff and the participating worksites. For example, an in-person meeting included the time of both project staff and worksite contacts. Action costs were costs incurred by the worksites themselves for implemented sun protection actions. Control worksites incurred only action costs, whereas intervention worksites incurred both delivery and action costs. Quantities of intervention delivery components (e.g., trainings, email and phone contacts) were obtained from a project tracking database and assigned an average duration based on consultation with project staff. For example, a response to a worksite email contact was assumed to take three minutes to compose, edit, and send. Time units were valued by the appropriate hourly wage rate, including fringe benefits, for a given worksite. Cost data came from expense reports and retrospective labor estimates. Unit cost multipliers were applied to quantities (e.g., letters, brochures) tracked in a project database. Action costs focus on self-reported sun protection actions that worksites implemented during or after the SSW intervention. Also, reported action costs excluded training of managers and training of employees because worksite respondents may have conflated the SSW intervention training with training programs that their sites initiated themselves, which was the intent of the post-test survey questions. Therefore, costs are reported for an array of 13 policies rather than 15.

Results

Profile of the Local Government Organizations

The 33 intervention worksites that were successfully followed up with two years post-intervention, reported an average of 185 employees (median=80, range=25-1,200). Comparable values for the 30 control worksites at the two-year follow-up were an average of 271, median of 100, and range of 10-2,400. Among intervention (control) worksites, 55% (47%) were municipal governments with 6% (13%) county governments, with most worksites in both groups located in metropolitan Denver or southwestern Colorado. Approximately 22% of managers in both groups were female. Intervention (control) employers were in counties with average annual household income of $79,258 ($76,689) and 38.4% (36.0%) of residents who had college degrees.

Intervention Delivery Costs

Two-year intervention delivery costs totaled $121,789 (monthly average=$5,075), of which $63,158 ($2,632) (51.8%) was incurred by project staff (per-worksite mean=$1,732 ($72), median=$1,052 ($44)) (Table 1). Worksite delivery costs totaled $58,631 ($2,443) (per-worksite mean=$1,777 ($74), median=$1,637 ($68)). Non-labor costs (e.g., brochures, posters) were $5,990 ($250) (9.5% of project staff delivery costs; per-worksite mean=$181 ($8), median=$179 ($8)). Proportions of SSW labor costs attributed to intervention delivery were comparable between project and worksite staffs with 87.5% of project staff labor in email contacts, in-person contacts, and training preparation (93.2% for worksite staff) (Table 1). Delivery costs per employee were $118 (monthly average=$5) for project staff labor (median=$42 ($2), range=$2-$1,200 (< $1-$50)); non-travel delivery costs per employee averaged $23 ($1) (median=$12 (< $1), range=$1-$95 (<$1-$4)). Worksite labor delivery costs averaged $56 ($2) per employee (median=$44 ($2), range=$2-$190 (< $1-$8)); non-training labor delivery costs per employee were $14 (< $1) (median=$9 (< $1), range=$1-$51 (< $1-$2)).

Table 1.

Intervention Costs (000s)

Labor Costs
	Project Staff	%	Monthly Average*	Worksite Staff	%	Monthly Average*	Total	Monthly Average*
Email Contacts	$4.1	25	$173	$2.5	23	$104	$6.6	$277
Phone Contacts	$0.7	4	$30	$0.5	4	$19	$1.2	$49
In-Person Meetings	$5.3	31	$219	$4.3	39	$179	$9.6	$398
Mailings	$1.4	8	$58	$0.3	3	$12	$1.7	$69
Training	$5.4	32	$224	$3.3	31	$140	$8.7	$364
Direct Delivery Costs	$16.9	100	$704	$10.9	100	$454	$27.8	$1,158
Employee Training Time				$47.7		$1,989	$47.7	$1,989
Project Staff Travel	$40.3		$1,678				$40.3	$1,678
	$57.2			$58.6		$2,443	$115.8	$4,825
Non-Labor Costs
Posters/Brochures/Kits	$6.0		$250	$0			$6.0	$250
Total Intervention Delivery Costs	$63.2		$2,632	$58.6		$2,443	$121.8	$5,075
Post-Intervention Action Costs (Intervention Group)				$90.6		$3,777	$90.6	$3,777
Total Intervention Costs (Intervention Group)							$212.4	$8,852
Post-Intervention Action Costs (Control Group)				$66.5		$2,769	$66.5	$2,769
Incremental Total Costs (Intervention over Control)							$146.0	$6,082
Intervention Delivery Costs (staff travel and employee training time removed)							$33.8	$1,407
Incremental Total Costs (staff travel and employee training time removed)							$58.0	$2,415

^*Monthly averages over 24 months in whole dollars.

Two points should be emphasized about the delivery cost estimate. First, project staff costs included $40,272 (monthly average=$1,678) in travel to participating worksites around Colorado. These costs included both actual expenditures (e.g., air travel) and the value of staff travel time and were not resources directly allocated to delivery of the intervention itself (per-worksite mean=$1,220 ($51), per-worksite median=$456 ($19)). (A similar phone-based or online intervention would not incur these costs.) Total non-travel project staff delivery costs were $16,897 ($704) (per-worksite mean=$512 ($21), median=$458 ($19)). Second, worksite delivery costs included $47,742 ($1,989) in employee attendance at sun safety training. These costs were not necessarily additional dollars invested by worksites in sun safety. Employees are paid for “non-productive” activities, such as department meetings and training. SSW training was often incorporated into existing training sessions, so the budgetary effect of training was reduced. However, this value acknowledges the opportunity cost to worksites of having their employees devote time to sun safety training, rather than more directly “productive” work. Worksite delivery costs without employee training were $10,888 ($454) (per-worksite mean=$330 ($14), median=$539 ($22)).

Action Implementation and Costs

The 33 reporting intervention worksites implemented 72 non-training sun protection actions over the 24 months post-SSW (per-worksite mean=2.18, median=2) (Table 2). The 30 control worksites implemented 39 such actions (per-worksite mean=1.30, median=1) (difference in per-worksite means: p=.04). The largest positive differences in action implementation between intervention and control worksites were in regular skin exams, provision of sunscreen, provision of sun-protective hats, and risk assessment of sun exposure. Provision of permanent outdoor shade and work scheduling to avoid midday sun exposure were each one less among intervention worksites than among control worksites.

Table 2.

Distribution of Implemented Actions

	Intervention (N=33)	Control (N=30)	Difference
Provision of permanent outdoor shade	5	6	−1
Provision of portable shade	9	6	3
Scheduling of work to avoid midday sun exposure	2	3	−1
Risk assessment of sun exposure or protection	7	2	5
Sun safety policy review/changes and monitoring	7	4	3
Resource allocation for sun safety personal protection equipment	5	3	2
Provision of sunscreen	10	5	5
Provision of sun-protective hats	10	5	5
Provision of sun-protective clothing	4	2	2
Provision of sun-protective eyewear	4	1	3
Regular skin exams (e.g. health fair with a physician)	7	1	6
Contractors and subcontractors to comply with sun safety policies	1	1	0
Posting of the UV index	1	0	1
	72	39	33
Actions per worksite	2.18	1.30	p = 0.002

UV: ultraviolet

Total implementation costs to intervention worksites of the 72 post-SSW actions were $90,645 (monthly average=$3,777) (per-worksite mean=$2,747 ($114), median=$550 ($23)) (Table 3). Average per-action cost was $1,259 ($105), ranging from $9 (<$1) for simply posting the UV index publicly to $9,500 ($396) for permanent shade installation. Comparable costs to control worksites were $66,467 ($2,769) (per-worksite mean=$2,216 ($92), median=$128 ($5)). Average per-action cost was $1,704 ($71) with a range comparable to intervention worksites. Restricted to the 22 intervention worksites implementing at least one post-SSW action, the per-worksite mean was $4,120 ($172) (median=$1,413) ($59). Comparable estimates for the 21 control worksites implementing at least one post-SSW action were per-worksite mean=$3,165 ($132) and per-worksite median=$276 ($12). Estimated at $9,500 ($396), installation of permanent outdoor shade was by far the most expensive action within both groups (intervention: 52% of action costs; control: 86%). Within intervention worksites, provision of sun-protective clothing, hats, and eyewear had the highest costs per implementing worksite, each over $1,000. Costs per implementing worksite among controls were generally lower than among intervention worksites. Costs of implemented actions per employee across intervention worksites were $142 ($6) (median=$16 (< $1)). Across control worksites, per-employee costs per action were $183 ($8) (median=$8 (< $1)).

Table 3.

Costs of Implemented Actions

	Intervention	Cost per action	Monthly Average*	Control	Cost per action	Monthly Average*
Provision of permanent outdoor shade	$47,500	$9,500	$396	$57,000	$9,500	$396
Provision of portable shade	$2,889	$321	$13	$2,994	$499	$21
Scheduling of work to avoid midday sun exposure	$514	$257	$11	$387	$129	$5
Risk assessment of sun exposure or protection	$696	$99	$4	$141	$71	$3
Sun safety policy review/changes and monitoring	$550	$79	$3	$280	$70	$3
Resource allocation for sun safety personal protection equipment	$264	$53	$2	$117	$39	$2
Provision of sunscreen	$2,730	$273	$11	$610	$122	$5
Provision of sun-protective hats	$13,119	$1,312	$55	$3,415	$683	$28
Provision of sun-protective clothing	$15,573	$3,893	$162	$160	$80	$3
Provision of sun-protective eyewear	$4,334	$1,084	$45	$775	$775	$32
Regular skin exams (e.g. health fair with a physician)	$2,422	$346	$14	$541	$541	$23
Contractors and subcontractors to comply with sun safety policies	$46	$46	$2	$46	$46	$2
Posting of the UV index	$9	$9	$0	$-	$-	$0
	$90,645	$1,259	$52	$66,467	$1,704	$71
Cost per worksite	$2,747		$114	$2,216		$92
Median cost per worksite	$551		$23	$128		$5
Worksites implementing at least one action	22			21
Cost per implementing worksite	$4,120		$172	$3,165		$132

UV: ultraviolet

As noted above, the average cost of implemented actions is skewed significantly upward by the high cost of permanent shade structures that may be of limited value to worksites with mobile workforces. Excluding permanent shade structures lowers the overall action cost per intervention worksite by more than half from $2,747 (monthly average=$114) to $1,307 ($54). The average cost of 67 non-permanent shade actions was $644 ($27) with a range of $9 (< $1) for posting the UV index publicly to $3,893 ($162) for providing sun-protective clothing. Only provision of sun-protective clothing ($162), hats ($55), and eyewear ($45) cost over $15 per month. Among control worksites, the average cost of 33 non-permanent shade actions was $287 ($12). The average cost per month across actions among control worksites ranged from < $1 (UV index) to $32 (eyewear).

Table 4 lists the average incremental intervention costs per incremental action implemented, comparing the intervention worksites to the control worksites. Total intervention costs include costs of both delivery and implemented actions. With all costs of staff travel and employee training time included, the average cost of the 33 incremental actions implemented by intervention worksites was $4,423 (monthly average=$184) ($4,573 excluding permanent shade ($190)). Excluding all travel and employee training time lowered these average costs to $1,756 ($73) ($1,984 excluding permanent shade ($83)). Future implementing organizations may expect an average cost per implemented action within this range, depending on the extent of new employee training.

Table 4.

Incremental Ratios

	Including Permanent Shade		Excluding Permanent Shade
	Intervention	Control	Intervention	Control
Total Intervention Cost	$212,436	$66,467	$164,934	$9,467
Incremental Intervention Cost (Intervention over Control)	$145,969		$155,467
Actions Implemented (AI)	72	39	67	33
Incremental AI	33		34
Incremental Intervention Cost / Incremental AI	$4,423		$4,573
Total Intervention Cost (staff travel and employee training removed)	$124,420	$66,467	$76,920	$9,467
Incremental Intervention Cost (Intervention over Control)	$57,953		$67,453
Incremental Intervention Cost / Incremental AI	$1,756		$1,984

Discussion

Occupational sun safety programs are a critical component of national efforts to prevent skin cancer, given outdoor workers’ increased risk for skin cancer^27-29 attributed to their chronically high doses of solar UV and inadequate on-the-job sun protection.^30-34 Cost can be a potentially determining factor in the adoption and implementation of occupational safety³¹, making it a necessary and complex factor to consider when assessing program implementation and outcomes. The results of this study demonstrate the interactions of cost and implementation that explain and can possibly guide the adoption of and investment in occupational sun safety.

Consistent with our observation of the workplace²⁰, SSW successfully increased the number of sun protection actions that senior managers reported implementing at intervention worksites relative to control worksites and at a per-employee cost of SSW delivery ($174) that equates to $7.25 per month. This cost is comparable to other worksite health interventions such as reducing employee sitting time ($310 per employee)³⁵, weight management ($589-$601 per employee)³⁶, and nutrition improvement ($62 per employee).³⁷ The total (i.e., delivery plus action) per-employee costs of SSW are among the least expensive of these worksite-based programs. This may be a favorable feature of a skin cancer prevention program that worksite personnel may view as a lower priority than accident prevention and other risk behaviors such as alcohol and substance abuse. Accidents and substance abuse have more direct effects on workers than skin cancer and thus more harmful effects on worksite environments and worker performance, making them a higher priority for worksite leadership and staff. It is important to note that our analysis is limited to the cost implications of the SSW intervention’s ability to induce self-reported sun protection actions among participating worksites. It does not incorporate the effects of the SSW intervention on individual employee behavior, which we plan to address in future work. Still, changing organizational behavior to implement effective preventive health programs can be quite challenging and require substantial investments in interventions.

The estimate of delivery costs includes significant travel costs of project staff ($95 per employee). Reducing or eliminating the need for travel would enhance the efficiency of intervention delivery still further, as would integration of sun protection training into extant training programs. We are currently testing a strategy for scaling up SSW that eliminates travel in favor of virtual meetings and online training to motivate cost-effective distribution. However, it is likely to be more difficult to establish strong working relationships with senior managers using virtual as opposed to in-person interactions, which risks reducing implementation effects, making the balance between reach and effectiveness a key consideration in scale-up of SSW.

The per-employee cost of sun protection actions incurred by implementing worksites was of a similar magnitude to intervention delivery costs. In any worksite intervention, one should consider the burdens it places on employers both in tangible assets in this case for personal protection equipment or opportunity costs including time spent by employees in training rather than productive work. As we show, these costs are not insignificant and in our anecdotal experience during recruitment for the study, many employers requested information on these incurred costs. One benefit of our analysis is that these estimated costs can be provided to employers and in general they are modest and should not pose a substantial barrier to occupational sun safety, especially if an employer avoids installing permanent shade.

While permanent shade was relatively expensive when initially purchased compared to other personal protection equipment, this may not be a fair comparison. Shade has a much longer life than other personal protection equipment that degrades with repeated uses or can be damaged in the course of the job such as clothing or has a single use such as a dose of sunscreen. Shade may be more affordable when comparing it to the cost of supplying other personal protection equipment annually over several years than it appears by comparing its cost of purchase. Still, shade is not effective for workers who are mobile, unless it can be attached to equipment or delivered using temporary, portable shade structures.

Our study has several strengths. First, the SSW sample of worksites was diverse, but randomization controlled for important group differences. Second, a detailed observational coding protocol assessed all worksite sun protection policies. Third, the project team rigorously tracked all intervention delivery cost data, primarily labor events and associated wage and fringe rates. However, we acknowledge that the SSW study was conducted in Colorado, a mountainous state with often high UV radiation exposure. Sun safety may be a more salient health issue in Colorado than in other locations; therefore, the SSW results may represent somewhat of an upper bound on the potential for sun safety behavior modification or may apply best to southern states with high UV radiation. Additionally, SSW was conducted among a sample of local government organizations, which may, on balance, be somewhat less concerned with the budgetary implications of preventive health programs such as SSW than private businesses interested in profit maximization. Finally, SSW outcomes related to policy implementation are based on self-reported responses of organizational staff, which are subject to well-known demand and social desirability biases. However, policy scores were based on objective coding of actual written documents, which avoided these biases. Furthermore, the project team assigned costs to implemented programs using a detailed template outlining labor and non-labor resources for each action.

Conclusion

The SSW intervention successfully expanded the adoption of sun protection actions among intervention worksites at a cost per employee well within the range of other worksite health interventions. Future research should examine the longer-term effects of sun protection actions on employees’ individual sun protective behavior and ultimately, employee health.