Sustained use of an occupational sun safety program in a recreation industry: follow-up to a randomized trial on dissemination strategies

Abstract

The sustainability of evidence-based programs is needed to obtain long-term benefits. To assess sustainability of Go Sun Smart (GSS), an occupational skin cancer prevention program disseminated to the North American ski industry. Fifty-three of the 68 ski areas from the original dissemination trial participated in 2012 and 2013, 5 to 7 years after program distribution by enhanced or basic dissemination strategies. Sustained use was measured by: (1) on-site observation of sun protection communication and (2) an online survey with senior managers. In the sustainability assessment, sun safety communication had declined, and dissemination strategy did not affect continued use. Managers held weaker attitudes about skin cancer importance and program fit, but more managers provided free/reduced-cost sunscreen than in the dissemination trial. Manager turnover was a key factor in program discontinuance. Sustainability remains a challenge. Additional research is needed to determine the best strategies for sustainability.

INTRODUCTION

The true measure of a prevention intervention’s success is how much it reduces risky health behaviors over the long term [1]. While, a careful review of the literature reveals that many interventions are effective in the near term, few of them have taken steps to determine whether these effects were long lived [2–4]. For the past decade, the authors have been involved in the development and dissemination of a theory-based intervention for occupational sun protection among outdoor workers in the North American ski industry [5]. Under the Go Sun Smart brand (GSS), the intervention, comprised of signage and brochures, employee training, and a website, was introduced to the 365 ski areas in the National Ski Areas Association (NSAA), the leading professional association, and potentially reached over 100,000 employees, in a randomized trial comparing a Basic Dissemination Strategy of industry-standard distribution to an Enhanced Dissemination Strategy adding theory-based communication. GSS [6], in common with similar evidence-based interventions [7–9], was successful in improving workers’ sun protection practices in the near term when ski area managers implemented GSS with sufficient intensity [10, 11]. However, within 2 years’ time, many of these same managers were less likely to use GSS materials [12], raising doubts about whether the initial improvements had been reinforced over time. The only other long-term follow-ups for sun safety interventions were conducted over 3 years or less [13–15].

Sustained efforts to convince outdoor workers to practice sun protection on the job are needed for several reasons. Outdoor workers are chronically exposed to ultraviolet radiation (UV), a primary cause of skin cancer [16, 17]. Lifetime UV exposure is linked to squamous cell carcinoma (SCC) and possibly melanoma; intermittent, severe exposure (i.e., sunburn) may be related to melanoma, basal cell carcinoma (BCC), and SCC [18–21]. Outdoor work is associated with non-melanoma skin cancer (NMSC), i.e., BCC and SCC [22, 23] and possibly an elevated risk of melanoma [24]. Preventing NMSC is a priority due to its extremely high prevalence (over 3 million annually [25]), tendency to recur [26, 27], disfigurement [28], treatment costs [29], and association with other cancers [30, 31]. Unfortunately, many outdoor workers fail to practice sun safety [32, 33] and remain at high risk for skin cancer from occupational UV exposure.

As used here, sustainability refers to the continuation of a program, its activities, and structures, when financial, organizational, and technical support from external organizations has ended [1, 34]. It has been estimated that over 40 % of many health programs do not last beyond the first few years [35]. In diffusion of innovations theory (DIT) [36, 37], sustainability occurs when a program such as GSS becomes institutionalized [38, 39], the final stage in organizational adoption [38, 39] (i.e., routinizing the innovation so it becomes part of organization’s normal activities) [40]. Innovations such as GSS are more likely to be institutionalized [37, 41] (i.e., sustained) when there is significant administrative support for its adoption and when the changes within the organization that its adoption requires become an integral feature of the organization’s structure. By contrast, some organizational factors, such as management turnover [42, 43], can contribute to discontinuance.

This paper reports the results of a long-term follow-up examining the sustainability of GSS 5 to 7 years after a trial in which it was disseminated throughout North America. Best-practice recommendations for sustainability assessment were followed, including using a valid longitudinal design with randomization, measuring continuation of program components, and exploring newly created organizational procedures/policies [2, 44, 45]. Sustained program effects on employee sun protection are reported elsewhere [46].

METHODS

Sample

The senior-most manager (e.g., General Manager) or manager who served as the key contact for the 2004–08 GSS dissemination trial was re-contacted and invited to participate in this study. Ultimately, managers at 53 of the original 68 areas agreed to participate (77.9 %; 52 participated in 2012 and 1 delayed participation until 2013; see CONSORT diagram in Fig. 1). During recruitment, names and contact information of senior managers at the ski area were updated.

Fig 1.

CONSORT diagram for trial

Go Sun Smart program

GSS advised employees to practice sun protection on the job including using sunscreen, sunglasses, hats, and protective clothing, seeking shade, reducing time in the midday sun, and avoiding sunburning. As described previously, GSS was based on DIT and principles of persuasive communication and included posters, small decals, magnets, outdoor signage, signage for ski/snowboard schools, brochures for employees and guests, a training program with script and slides, newsletter articles, and brief messages (see materials at http://rtips.cancer.gov/rtips/programDetails.do?programId=308006]).

Procedures

Ski areas (n = 68) were originally recruited to a two-group randomized trial that compared two dissemination strategies in the fall of 2004–06 in three annual waves. All ski areas received the GSS from the industry’s leading professional association (i.e., National Ski Areas Association) in 2004–07 through a Basic Dissemination Strategy (BDS). Half the ski areas were randomized by the project statistician to an Enhanced Dissemination Strategy (EDS) (n = 33 ski areas; 12 in 2004, 11 in 2005, and 10 in 2006) based on principles of DIT. An immediate posttest assessment of GSS use (O₁) was performed using on-site observation in the winter during the season when they were first recruited. Senior managers were surveyed in the fall and winter, at the beginning and end of that season. All ski areas received GSS for up to 2 years after the immediate posttest. An intermediate-term follow-up assessment of GSS use (O₂) using the on-site observation was performed two seasons (n = 27 in winters 2006) or one season (n = 41 ski areas in winters of 2007 or 2008) later.

From January to April of 2012, sustained use of GSS was measured (O₃), which was 5 to 7 years after the immediate posttest (O₁) (sustained use was assessed at 1 ski area in March 2013) using three methods. First, project staff completed an on-site observation of GSS materials and other sun protection communication. Second, the contact manager or another senior manager was interviewed using a semi-structured interview protocol. Third, in March, senior managers at the ski areas were invited to complete an online survey. Managers received two reminders over 2 weeks. All procedures were approved by the Western Institutional Review Board (IRB) and the San Diego State University IRB.

Go Sun Smart dissemination strategies

Two dissemination strategies were compared in the earlier dissemination trial. The BDS used the normal procedures by the industry’s professional association to distribute safety programs. It included informational booths at national and regional trade shows, promotional materials for GSS (e.g., logo magnets, lip balm, and post-it notes), 1-page informational tip sheets, and free starter kits of materials (2 per year over 3 years in October/November and January). All ski areas received GSS through the BDS from 2004–07.

The EDS included all BDS activities plus face-to-face, as well as online and phone, contact between project staff and senior managers. Based on DIT, project staff attempted to reinforce the need for occupational sun safety, gain senior managers’ commitment to use GSS, plan its implementation, and provide continued support for using GSS. Staff also sought to reduce managers’ uncertainty about GSS by describing GSS’ fit with ski area operations and connecting GSS to the NSAA’s endorsement. Staff coached managers on how best to use GSS, advised them about the importance of identifying internal champions in their employ to provide vocal support to GSS and reinforced their decisions to use GSS [37, 47]. Project staff visited the ski areas from November to January and followed up with managers by telephone and email monthly or more often through the end of March. The BDS and EDS were described in more detail elsewhere [10].

On-site observation of sustained Go Sun Smart use

The on-site observation protocol used in the dissemination trial for O₁ and O₂ was used to assess the visible number of GSS items the ski areas [10] were using during the sustainability assessment (O₃). During a one-day visit, project staff recorded all printed GSS materials on display (i.e., 15 posters/signs, 3 brochures, 2 static, clings and 1 logo magnet were originally distributed in the earlier dissemination trial). Staff also recorded any other sun protection messages (e.g., employer-generated; commercial advertising). All locations were searched, including employee-only areas (e.g., offices, locker rooms, chair lift shacks). Staff recorded where each item was located and whether it was in an employee-only area. Three measures of program use were created by summing the counts: GSS items, non-GSS items excluding commercial advertising, and total sun protection communication. The observation protocol was validated in the earlier effectiveness trial on GSS [6].

Survey of senior managers on sun protection communication and actions

The questionnaire for managers measured their use of GSS, other sun protection messages for employees and guests, and administrative actions promoting sun safety. Using questions from the dissemination trial [10], managers reported whether they had communicated with employees about sun protection, and if so, whether they used a poster/sign, brochure, article, or message in an employee newsletter or email message or link to the GSS website. They reported on whether they conducted employee training on sun protection. In the dissemination trial, managers’ reports were validated by positive correlations with employees’ recall of sun safety messages [10]. Managers also were asked if the ski area made any of the following changes to support employee sun safety: provided free/reduced-cost sunscreen, sunglasses, or hats, conducted skin screenings, created a policy on sun safety, or included sun safety in a health/safety fair. Managers’ attitudes on occupational sun safety were assessed, i.e., employees’ susceptibility to and importance of skin cancer and characteristics of worksite sun protection education (see Table 4). Managers described their job characteristics (length of employment, work indoors/outdoors/both) and demographics (age, education, Hispanicity, race, gender).

Table 4.

Managers’ reports related to Go Sun Smart by assessment period and dissemination strategy

	Assessment period			Dissemination strategy
	(n = 547 managers at 51 ski areas)a			O1 and O3 combined (n = 547 managers at 51 ski areas)a			O3 only (n = 286 managers at 53 ski areas)b
	O1	O3	p	Basic	Enhanced	p	Basic	Enhanced	p
Reported use of Go Sun Smart for employees
Poster or sign	93.3 % (88.8 %,96.1 %)c	84.9 % (78.2 %,89.8 %)c	0.002	85.9 % (78.3 %,91.1 %)c	92.8 % (88.2 %,95.7 %)	0.016	76.8 % (64.7 %,85.7 %)c	88.9 % (81.2 %,93.7 %)c	0.017
Brochure	67.8 % (59.3 %, 75.2 %)	58.5 % (49.6 %, 66.8 %)	0.029	54.4 % (43.6 %, 64.9 %)	71.2 % (62.6 %, 78.6 %)	0.007	48.7 % (35.1 %, 62.5 %)	67.0 % (55.4 %, 76.8 %)	0.024
Article/messages in newsletter or flyer	65.8 % (55.6 %,74.7 %)	55.5 % (44.9 %, 65.7 %)	0.023	63.9 % (50.2 %, 75.6 %)	57.6 % (45.6 %, 68.8 %)	0.242	60.8 % (46.3 %, 73.6 %)	52.7 % (40.7 %, 64.3 %)	0.194
Message sent by email or on ski area website	40.3 % (31.4 %, 49.8 %)	30.8 % (22.9 %, 40.1 %)	0.028	38.9 % (27.9 %, 51.2 %)	32.1 % (23.3 %, 42.3 %)	0.186	31.4 % (21.1 %, 43.9 %)	33.1 % (24.2 %, 43.4 %)	0.410
Provision of sun protection items to employees
Free or reduced-cost sunscreen	63.8 % (53.1 %, 73.3 %)	70.8 % (61.2 %, 78.9 %)	0.057	69.7 % (56.3 %, 80.4 %)	65.0 % (52.7 %, 75.5 %)	0.290	69.6 % (54.2 %, 81.6 %)	69.5 % (56.3 %, 80.1 %)	0.495
Hat	76.4 % (64.9 %, 85.0 %)	56.8 % (43.6 %, 69.0 %)	<0.001	69.3 % (51.4 %, 82.8 %)	64.3 % (49.1 %, 78.6 %)	0.365	55.2 % (35.1 %, 73.8 %)	52.5 % (34.8 %, 69.5 %)	0.422
Sunglasses or goggles	13.3 % (8.8 %, 19.5 %)	7.6 % (4.7 %, 11.9 %)	0.014	10.2 % (6.1 %, 16.5 %)	9.9 % (6.3 %,1 5.2 %)	0.464	8.5 % (4.0 %, 17.2 %)	5.7 % (2.8 %, 11.4 %)	0.204
Shade cover in work areas	49.5 % (41.7 %, 57.3 %)	40.9 % (33.9 %, 48.3 %)	0.026	48.5 % (39.2 %57.8 %)	41.9 % (34.1 %, 50.1 %)	0.146	39.1 % (29.4 %, 49.9 %)	39.0 % (30.9 %, 47.9 %)	0.495
Reported communication with guests
Written message on a poster, sign, newsletter article, etc.	87.3 % (81.4 %,91.6 %)	66.5 % (57.9 %,74.1 %)	<0.001	73.1 % (62.7 %,81.4 %)	83.4 % (76.2 %, 88.8 %)	0.027	59.6 % (45.7 %,72.2 %)	73.4 % (62.4 %,82.0 %)	0.049
Susceptibility and importance of skin cancer
Employees likelihood of getting skin cancer (strongly agree [5]—strongly disagree [1])	3.09 (2.94, 3.24)	3.23 (3.08, 3.37)	0.054	3.14 (2.96, 3.31)	3.18 (3.02, 3.33)	0.364	3.30 (3.09, 3.52)	3.19 (3.01, 3.37)	0.207
Skin cancer is an important health concern to employees (strongly agree [5]—strongly disagree [1])	4.13 (4.02, 4.24)	4.16 (4.06, 4.27)	0.319	4.13 (4.00, 4.25)	4.17 (4.06, 4.28)	0.308	4.10 (3.95, 4.25)	4.12 (4.00, 4.25)	0.407
Skin cancer is an important health concern to me (strongly agree [5]—strongly disagree [1])	4.37 (4.26, 4.48)	4.25 (4.15, 4.36)	0.024	4.25 (4.12, 4.38)	4.37 (4.26, 4.49)	0.081	4.15 (3.98, 4.33)	4.31 (4.16, 4.46)	0.086
Perception of characteristics of sun protection education
Affordability (too expensive [1]—very affordable [5])	4.15 (4.02, 4.29)	3.88 (3.76, 4.01)	<0.001	3.96 (3.80, 4.12)	4.08 (3.94, 4.21)	0.128	3.79 (3.57, 4.02)	3.91 (3.72, 4.10)	0.222
Compatibility with workplace (compatible [5]—incompatible [1])	4.31 (4.11, 4.52)	3.68 (3.48, 3.88)	<0.001	3.97 (3.72, 4.22)	4.02 (3.80, 4.24)	0.375	3.78 (3.43, 4.12)	3.64 (3.35, 3.93)	0.275
Ease of implementation (easy [5]—difficult [1])	4.19 (4.04, 4.33)	3.96 (3.83, 4.10)	0.005	3.99 (3.82, 4.15)	4.16 (4.02, 4.31)	0.056	3.94 (3.70, 4.18)	4.01 (3.81, 4.21)	0.336
Fit with company values (fits [5]—does not fit [1])	4.43 (4.24, 4.63)	3.66 (3.48, 3.84)	<0.001	3.99 (3.76, 4.22)	4.10 (3.90, 4.30)	0.236	3.75 (3.40, 4.09)	3.68 (3.39, 3.98)	0.392
Simplicity (too simple [1]—too complicated [5])	2.39 (2.27, 2.50)	2.72 (2.61, 2.83)	<0.001	2.58 M (2.45, 2.70)	2.53 M (2.42, 2.64)	0.301	2.63 (2.45, 2.81)	2.81 (2.67, 2.95)	0.062
Appropriateness for employees (appropriate [5]—inappropriate [1])	4.66 (4.50, 4.82)	4.04 (3.89, 4.20)	<0.001	4.32 (4.13, 4.51)	4.38 (4.21, 4.55)	0.310	4.02 (3.70, 4.35)	4.06 (3.79, 4.33)	0.437

^aCompleters from the 51 ski areas that participated in both the dissemination trial and sustainability assessment

^bCompleters from the 53 ski areas that participated in the sustainability assessment only

^c95 % confidence interval

Ski area characteristics

Descriptive information for each ski area was collected by project staff. This included the number of employees, number of female senior managers, latitude and elevation, and mean annual number of hours of sunshine (from the National Weather Service).

Statistical analysis

On-site observations of GSS materials in use at the ski areas were compared across the dissemination trial (O₁ and O₂) and sustainability assessment (O₃). Also, the two dissemination strategies (BDS v. EDS) were compared. All comparisons were performed using PROC GLIMMIX in SAS and adjusted for organizational characteristics (number of employees was dropped because the model would not converge). Analyses were conducted first on the ski areas that participated in the sustainability assessment (i.e., completers analysis). Next, the effect of ski area dropout was assessed using imputation in which ski areas that dropped out were assigned a value of 2 × O₂ − O₁ to model the assumption that even though they dropped out, these ski areas continued to experience the change in program use witnessed between O₁ and O₂, not just the level of use at O₂.

Managers’ survey responses were compared between the dissemination trial (O₁) and the sustainability assessment (O₃), adjusting for clustering within resorts. Managers were not surveyed at O₂. This analysis used responses collected from managers successfully followed up (i.e., no imputation was made for missing values), as they were secondary measures.

When planning the dissemination trial, a 1-tailed alpha criterion was used to determine the needed sample size [10] because we planned to evaluate a directional hypothesis that that EDS was better than BDS. We did not plan on interpreting results indicative of the inferiority of EDS, because failure to reject the null hypothesis with this 1-tailed test would lead to the same conclusion as with a 2-tailed test: recommend the BDS. The same sample was used in the sustainability assessment, so a 1-tailed alpha was employed in all tests on sustainability of GSS.

RESULTS

Profile of ski areas and senior managers

The characteristics of participating ski areas are reported in Table 1. The resorts ranged in size (based on employee numbers), ownership, and elevation; were located across North America at a variety of latitudes; and a quarter of them received a large amount of sunshine hours. Reasons for non-participation included lack of interest, change in ownership/management, and low snow levels leading to late opening or early closing. A higher proportion of western ski areas (100 % west, 90 % southwest, 100 % Rocky Mountain, and 88.9 % northwest) participated than eastern ski areas (63.2 % northeast and 16.7 % central/southeast; p = 0.001). Thus, it was not surprising to find that participating ski areas had higher elevations (participated M = 5504 ft (base) and 8073 ft (summit); did not participate M = 3458 ft (base) and 4930 ft, p = 0.013 and 0.002, respectively) than non-participating ski areas.

Table 1.

Profile of ski areas and senior managers

Resort characteristics (n = 53)	%
Number of employees (mean = 467.3, sd = 374.2)
Less than 250	26.4
250–499	39.6
500–999	26.4
1000 or more	7.6
Percent of female managers	25.6
Manager turnover rate	52.1
Ownership type
Multi-resort ownership	20.8
Single-resort ownership	79.2
Region
West	15.1
Northwest	15.1
Southwest	34.0
Rocky Mountains	11.3
Northeast	22.6
Central and Southeast	1.9
Latitude of resort (mean = 42.0°N, sd = 4.3°)
Base elevation (mean = 5504.3 ft., sd = 2810.9 ft)
Summit elevation (mean = 8073.5 ft, sd = 3435.8 ft)
Annual mean number of sunshine hours
2400 h or less	28.3
2401–3200 h	45.3
3201 h or more	26.4
Manager characteristics (n = 286)	%
Work outdoors
Mostly outdoors	33.3
Outdoors and indoors equally	30.9
Mostly indoors	35.8
Involvement in decisions about workplace procedures and policy on safety and health of employees
Never or rarely	3.8
Some of the time	21.7
Most or all of the time	74.5
Age (mean = 47.8, sd = 11.3)
22–39	23.1
41–49	28.9
50–59	31.0
60–78	17.0
Education
High school diploma or less	8.2
Trade, technical or vocational education, or some college	39.0
College graduate or postgraduate degree	52.8
Hispanic	1.5
Race
American Indian/Alaska Native/First Nations	0.7
White	98.6
More than one race	0.7
Ever been told by doctor that you had skin cancer	15.7
Female	26.9

A total of 465 senior managers received an invitation to complete the online survey, with 286 complying (62.2 %; n = 182 online and 104 by mail; Fig. 1). Of these, 146 were interviewed in O₁ and 140 were new in O₃. Given this, analyses were conducted treating the O₁ versus O₃ comparison as between subjects. Response rate to the O₁ survey did not differ between resorts participating (69.2 %) or not participating (62.9 %) in the sustainability assessment (O₃; p = 0.543). Senior managers were older, highly educated, nearly all white, and mostly male. Two thirds worked outdoors at least some of the time, and nearly all were decision makers concerning safety and health of employees. Also, 15.7 % had been diagnosed with skin cancer. Respondents held jobs ranging from the senior-most manager (e.g., president, general manager, and chief operating officer) to departmental managers (e.g., accounting, base and lift operations, ski school, and ski patrol) to personnel and safety managers (e.g., human resources and risk management; see Table 1).

Sustained use of Go Sun Smart

At the sustainability assessment (O₃), the 53 ski areas were observed to be using 2.19 GSS items on average (M = 2.51 items using imputation, i.e., 2 × O₂ − O₁). Ski area use of GSS had declined substantially at the sustainability assessment in the analyses of both the ski areas completing the sustainability assessment and when imputing missing use values for those not followed up (Fig. 2 and Table 2).

Fig 2.

Observed number of Go Sun Smart items in use by Basic and Enhanced Dissemination Strategy at Immediate Posttest (0₁), 1- to 2-year follow-up (0₂), and 5- to 7-year sustainability assessment (0₃)

Table 2.

Mean number of messages in use at ski areas by assessment period

Items in usea	Completers (n = 53 ski areas)				Intent-to-treat (n = 68 ski areas)b
	O1	O2	O3	p	O1	O2	O3	p
Go Sun Smart only	6.77 (5.23, 8.32)c	5.11 (3.67, 6.55)c	2.19 (1.29, 3.09)c	<0.001	6.24 (4.97, 7.50)c	4.72 (3.48, 5.96)c	2.51 (1.45, 3.58)c	<0.001
All Sun Safety messages	7.68 (6.09, 9.27)	5.38 (3.94, 6.82)	4.94 (3.15, 6.73)	<0.001	7.12 (5.81, 8.42)	4.96 (3.71, 6.20)	4.65 (3.05, 6.24)	<0.001

^aExcluding retail messages (i.e., messages advertising sunscreen, lip balm, goggles, or clothing for sale)

^bFor intent-to-treat analyses, ski areas not followed up in the sustainability assessment (O₃) were assigned a score equal to 2 × O₂ − O₁ to represent the decline in use predicted by the difference between O₁ and O₂ assessments in the Dissemination Trial

^c95 % confidence interval

Use of any sun safety communication

When GSS materials and non-GSS sun safety messages observed at the ski areas were combined (M = 4.94 [sd = 6.50; range = 0–34] completers; M = 4.65 with imputation), the decline in sun safety messaging was smaller at the sustainability assessment but still statistically significant (Table 2).

Effect of dissemination strategy on sustained GSS use

The continued impact of the two dissemination strategies was explored at the sustainability assessment (O₃ only) and by collapsing across all three assessments (O₁, O₂, O₃). Dissemination strategy did not affect continued use of GSS (O₃: BDS M = 1.69, EDS M = 2.67, p = 0.14) or the presence of any sun safety messages (O₃: BDS M = 4.42, EDS M = 5.44, p = 0.29) at the sustainability assessment. However, across all three observations, use of the GSS items was higher at ski areas in the EDS than in the BDS conditions in the imputation analysis, as was use of any sun safety communication in both the completer and imputation analyses (Table 3). Further, it appears from Fig. 2 that more ski areas in the EDS group may have continued to use a relatively larger number of GSS items (9 or more) than in the BDS group.

Table 3.

Mean number of messages observed at ski areas by dissemination condition for all assessments combined

Items in usea	Completers (n = 53 ski areas)			Intent-to-treat (n = 68 ski areas)
	Basic	Enhanced	p	Basic	Enhanced	p
Go Sun Smart only	3.82 (2.77, 4.87)c	5.53 (4.32, 6.74)c	0.052	3.57 (2.75, 4.39)c	5.46 (4.31, 6.62)c	0.017
All Sun Safety messages	5.05 (3.73, 6.38)	6.91 (5.61, 8.22)	0.022	4.58 (3.56, 5.61)	6.63 (5.39, 7.86	0.006

^aExcluding retail messages (i.e., messages advertising sunscreen, lip balm, goggles, or clothing for sale)

^bFor intent-to-treat analyses, ski areas not followed up in the sustainability assessment (O₃) were assigned a score equal to 2 × O₂ − O₁ to represent the decline in use predicted by the difference between O₁ and O₂ assessments in the Dissemination Trial

^c95 % confidence interval

Effect of management turnover on sustained GSS use

Management turnover was a barrier to sustained GSS use: high-use ski areas in the dissemination trial had lower GSS use in the sustainability assessment when ≥40 % of managers had turned over from O₁ to O₃; areas with no turnover continued to have high use of GSS (GSS use at O₁ by management turnover interaction, p < 0.001).

Managers reports of sustained GSS use

Managers also reported a decline in GSS use at the sustainability assessment (O₃) relative to reports from the immediate posttest (O₁) in the dissemination trial. Managers were less likely to report using posters, brochures, newsletter articles or flyers, and messages sent by email or on ski area websites to communicate about sun safety with employees at the long-term follow-up (Table 4). Managers’ reported use of posters and brochures was higher at ski areas in the EDS than BDS condition, both at the sustainability assessment (O₃ only) and when combining the dissemination trial posttest (O₁) and the sustainability assessment (O₃) (Table 4). Managers also reported that their communication to guests about sun safety was lower at the sustainability assessment (O₃) than in the dissemination trial (O₁); however, this communication was higher in the EDS than in the BDS condition (Table 4).

At the same time, we observed that managers reported weaker attitudes about the importance of skin cancer and perceived that sun protection education for employees was not as good a fit with their organization 5 to 7 years after dissemination. Specifically, at the sustainability assessment (O₃), they perceived that skin cancer was less important for managers themselves and were less likely to believe that workplace sun safety education was affordable, compatible with the workplace, easy to implement, fit the company values, simple, and appropriate for employees than at the posttest of the dissemination trial (O₁; Table 4).

By contrast, more managers reported providing free or reduced-cost sunscreen to employees at the sustainability assessment (O₃) than in the dissemination trial (O₁) (Table 4). Provision of hats, sunglasses/goggles, and shade to employees did decline at the sustainability assessment. Still, about half of the managers reported providing hats and shade in work areas (Table 4).

DISCUSSION

The GSS occupational skin cancer prevention program demonstrated only modest sustainability 5 to 7 years after its distribution. Once promotional efforts by our team and the NSAA ended, most managers stopped using all but a few of the GSS items. Managers at a small group of ski areas, however, remained committed to extensive use of the program. This is not surprising. Ski area managers face an ongoing litany of organizational priorities, both health and non-health related. Many of these, moreover, were bound to reduce the level of commitment necessary to continue a program such as GSS. Also, some of the managers, who were quick to use GSS when first introduced, may have believed that their initial efforts were sufficient in addressing the sun safety need among their employees. As a result, sun safety may have been demoted on their safety agenda by more immediate and pressing issues.

When compared to data from our initial examination of sustainability [12], the decline in use was most pronounced in the first years following its distribution, with smaller declines in subsequent years. As others have noted [48], our findings suggest that active, long-term investments to promote heath programs such as GSS are essential for sustaining their initial effects.

That said, our team discovered that many of the ski areas provided sunscreen products to employees in their efforts to encourage employees to take precautions when working outdoors. This suggests that the dissemination of GSS may have contributed to a cultural shift in how ski area management viewed the importance of sun safety to their corporate health. As mentioned, research on DIT [36, 37] has found that the more an organization changes to accommodate an innovation, the more likely it will persist [41] and become institutionalized [37, 41].

On another front, the data gathered on sustainability also suggest factors that may have undermined sustainability, beginning with the high rate of turnover among managers. The ski industry, and perhaps similar enterprises in outdoor recreation, experiences a considerable mobility among managers. In the decade our team was involved with this industry, we grew accustomed to having to re-introduce ourselves to a new manager, scrambling to win their support, and introducing them to the GSS program. This suggests, at minimum, that the organizational stability necessary to increase the odds of an innovation such as GSS becoming an integral part of its policies about employee safety was missing.

Together, these findings call for further sustainability research to identify program components and dissemination strategies that can provide the financial, organizational, and technical support needed to sustain them. These could involve the provision external, and internal supports for program institutionalization. To be sustained, programs need to be designed to be modifiable [34] and inexpensive to continue, especially where materials have limited shelf life (e.g., signage can become damaged by weather and need replacement). Program supports might include providing message templates that can be manipulated and reprinted by employers and electronic versions of face-to-face trainings on sun safety. Also, managers should be taught the behavior change principles used in GSS so they can use them when creating their own sun protection communication.

Periodic refresher training and distribution of program materials conducted by industry professional associations could provide essential external support for program sustainability [34]. Ongoing training should prompt renewed interest in occupational sun safety, keep it on the workplace safety agenda, and restore managers’ commitment to implementation. Refresher sessions have reintroduced and maintained recommended behaviors in programs aimed at individuals [49, 50]. Further, holding this training at the annual or regional industry meetings that several employers can attend could make it cost effective and the ties with other organizations that use the program may help to sustain its use.

It also may be useful to encourage workplaces to adopt formal workplace sun safety policies as an internal administrative support for program sustainability. Policies persist even when organizational priorities shift and managers turnover by reminding managers to promote sun safety to employees and signaling the organization’s commitment to sun safety over the long term [2]. Policy development, including workplace health policies and policies requiring provision of health education programs [51, 52], has improved health outcomes [51, 53–57]. Managers should be advised to adopt policies that enact training in personal sun protection, plus environmental controls (e.g., shade and work schedules) and administrative procedures (e.g., risk assessment, supervisor training, policy review/monitoring, resource allocation, contractor compliance, and posting UV Index) [58–63]. Such policies will serve to establish both organizational and individual agendas for sun safety, integrate GSS into the workplace operations, and brand GSS as part of an organization’s culture. Finally, both refresher training and policy adoption would address two barriers to program use that appeared to produce discontinuance of GSS, i.e., management turnover and wavering commitments [42, 43]. Refreshers and policies can inform new managers about the need for occupational sun protection, show them how to use GSS to address it, and formalize the organization’s commitment to occupational sun safety. The longitudinal assessment of program sustainability by organizations is rare [13, 14, 45]. Most studies of workplace interventions have explored immediate impacts on worker knowledge and behavior [64], and no studies on skin cancer prevention have been assessed beyond 2 years.

This sustainability assessment had a few limitations, including the loss of 15 ski areas who did not agree to the 5- to 7-year follow-up and the use of self-report of program use by senior managers. However, the observational data corroborated these self-reports. While less than 20 % of the ski areas in the NSAA membership were assessed, we randomly selected from among those meeting eligibility criteria to minimize bias. Intent-to-treat methods are based on assumptions about drop outs that are not always accurate, as we found in our effectiveness trial [6]. The ski industry may limit the generalizability of the findings, e.g., operating in the winter, the season with low ambient UV, having long-standing efforts to address workplace safety, formal risk managers, and safety committees, controlling access to work areas, and having well-established communication channels. Still, it is an integral part of the large U.S. outdoor recreation industry [65] with diverse employers and workplace cultures, and many employees who routinely work (and recreate) outdoors at high elevation, in dry climates, and on reflective snow surfaces [66–69], where they receive substantial, chronic UV exposure [7]. Further, employees are mostly white with fair skin [6, 11] highly susceptible to skin cancer [70]; a large proportion are male and young, groups least likely to practice sun safety [71–74]; half reported sunburns [6, 11]; and many are long-term, low-paid seasonal workers [74], with limited health insurance. Thus, the results should generalize to other similar industries, too.

Sustainability remains a challenge for programs such as the GSS occupational sun protection program. Many employers continued to use only a small amount of program materials although several employers did supplement program materials with their own messages. Even with supplementation, program use at most workplaces was below the level that affected employees’ sun protection when it was originally disseminated. Health professionals need to plan efforts to maintain program use and overcome factors such as management turnover to make real change in the disease prevention behavior in settings such as workplaces.