Abstract
Objectives. We compared 2 strategies for disseminating an evidence-based skin cancer prevention program.
Methods. We evaluated the effects of 2 strategies (basic vs enhanced) for dissemination of the Pool Cool skin cancer prevention program in outdoor swimming pools on (1) program implementation, maintenance, and sustainability and (2) improvements in organizational and environmental supports for sun protection. The trial used a cluster-randomized design with pools as the unit of intervention and outcome. The enhanced group received extra incentives, reinforcement, feedback, and skill-building guidance. Surveys were collected in successive years (2003–2006) from managers of 435 pools in 33 metropolitan areas across the United States participating in the Pool Cool Diffusion Trial.
Results. Both treatment groups improved their implementation of the program, but pools in the enhanced condition had significantly greater overall maintenance of the program over 3 summers of participation. Furthermore, pools in the enhanced condition established and maintained significantly greater sun-safety policies and supportive environments over time.
Conclusions. This study found that more intensive, theory-driven dissemination strategies can significantly enhance program implementation and maintenance of health-promoting environmental and policy changes. Future research is warranted through longitudinal follow-up to examine sustainability.
Skin cancer is the most commonly diagnosed form of cancer in the United States, with 1 in 5 Americans expected to develop skin cancer during their lifetime.1–4 Although skin cancer rates are increasing, most skin cancers could be prevented by consistent use of sun protection, including sunscreen, hats, sunglasses, shirts, and shade.1–3,5,6 Behavioral recommendations for primary prevention of skin cancer include limiting time spent in the sun, avoiding the sun during peak hours (10:00 am to 4:00 pm), using sunscreen with a sun protection factor of 15 or higher when outside, wearing protective clothing (hats, shirts, pants) and sunglasses, seeking shade, avoiding sunburn, and making sun safety a habit.2,4
Although awareness about skin cancer is growing, preventive behaviors remain relatively low in the United States.7 Childhood exposure to the sun’s UV rays increases the risk for skin cancer later in life.8 Prevention programs for children in outdoor aquatic settings may influence youths, their parents, and swimming pool environments. These programs can achieve significant public health benefits if they are widely disseminated and successfully adopted, maintained, and continued.9
The Community Preventive Services Task Force10 recommends outdoor recreation and tourism setting–based interventions to prevent skin cancer on the basis of strong evidence of effectiveness in improving participants’ sun protective behaviors (e.g., use of sunscreen and sunglasses, avoidance of sun exposure), reducing UV exposure, and decreasing incidence of sunburn. Nevertheless, few skin cancer prevention programs that have been formally evaluated have continued for more than 1 year,11 and few evidence-based skin cancer prevention programs have been disseminated beyond an initial efficacy trial.12–14 Because long-term sun safety is critical to preventing the development of skin cancer, more evidence is needed about how best to disseminate interventions to vulnerable populations. Also, despite the growing body of research on the initial adoption of evidence-based public health interventions, less research has focused on strategies to improve the maintenance and sustainability of programs, organizational policies, and supportive environmental changes over time. Because effective cancer prevention programs must be sustained in the long term to have public health effect,15,16 it is critical to develop and evaluate efforts to create sustainable programs that would ensure long-term effects of public health initiatives.17 The Pool Cool Diffusion Trial was an innovative test of strategies to effect a lasting program and associated organizational changes within swimming pool environments across several years.
The main aims of the Pool Cool Diffusion Trial (2003–2010) were to evaluate the effects of 2 strategies (basic vs enhanced) for dissemination of the Pool Cool skin cancer prevention program on (1) program implementation, maintenance, and sustainability; (2) improvements in organizational and environmental supports for sun protection at swimming pools; and (3) sun protection habits and sunburns among children. We describe the main results of the trial at the swimming pool, or organizational, level.
METHODS
The Pool Cool skin cancer prevention program is a multicomponent educational and environmental intervention that was systematically developed, pilot tested, and evaluated in a randomized trial at 28 swimming pools in Hawaii and Massachusetts. The Pool Cool program had significant positive effects on children’s sun protection behaviors and on sun-safety environments at swimming pools18 and reduced sunburns among lifeguards and aquatic instructors18,19 in 2 racially and geographically distinct audiences. Across 2 seasons of dissemination pilot testing at 282 pools in the United States and Canada, the acceptability and feasibility of implementing Pool Cool in diverse settings were affirmed.20
Design Overview
The Pool Cool Diffusion Trial was conducted at swimming pools in metropolitan regions across the United States and used a 3-level nested experimental design across 3 years of intervention.20 The 3 levels of study were regional field coordinators, swimming pools, and children aged 5 to 10 years taking swimming lessons.20 Pools were assigned to field coordinators according to region and proximity. Field coordinators, and their pools, were then stratified according to latitude (north–south; > 40 degrees N and ≤ 35 degrees N) and size of pool (large–small) and randomized into 2 treatment conditions—basic (control) and enhanced (intervention). Field coordinators and pools were aware of the randomization but were not fully informed of the different strategies.
Details of the study design have been published previously.20 Although no explicit attempts to conceal any activities were made, the national scope of the study led to limited awareness of treatment-group differences.
Theoretical Foundations
The Pool Cool intervention and diffusion trial had foundations in social cognitive theory,21 diffusion of innovations theory,22 and theories of organizational change.23 Social cognitive theory was the basis for the educational and environmental program strategies and the enhanced treatment-group methods. Increased incentives, reinforcements, sun-safety skill-building, modeling of behavior from pool staff, and shade structures as environmental facilitators were all applications of social cognitive theory in the program. Diffusion of innovations was the basis for the conceptualization of implementation, maintenance and sustainability strategies, and measures. For example, concepts of compatibility and simplicity (less complexity) informed the design of staff training for the program and the carefully developed intervention kits of program materials. Furthermore, theories of organizational change informed the pool-level methods and measures as well as the sustainability-promoting strategies (guide to sustainability, environmental supports such as umbrellas in the enhanced arm).20
These models are complementary, with considerable overlap among them to increase initial and sustained use of the program components over time. The intent of the trial was not to test a single model but to apply the most promising constructs from each to the problem of skin cancer prevention, organizational change, and program diffusion in aquatics settings.
Sample and Recruitment
The Pool Cool Diffusion Trial was conducted across 4 calendar years with 2 consecutive cohorts of 3 years each, starting in 2003 and 2004, at outdoor swimming pools in 33 metropolitan areas across the United States. Pools in the first cohort were offered the option of continuing in the study for a fourth year. Pools were recruited in cooperation with the National Recreation and Park Association with multiple methods, including National Recreation and Park Association Web site notices, National Recreation and Park Association e-mail discussion lists, conference displays, and targeted advertisements in aquatic magazines and National Recreation and Park Association newsletters. Metropolitan regions were required to have a minimum population size of 100 000 and at least 4 outdoor swimming pools willing to participate. Recruited pools were both public (city, county, military) and private (Young Men’s Christian Association, country clubs). Pools were required to offer swim lessons to children aged 5 to 10 years and to be large enough to recruit at least 20 parents to fill out surveys.
Swimming pools were the main organizational level unit of study, and the staff at each pool were responsible for delivering the Pool Cool intervention to the primary audience—children aged 5 to 10 taking swim lessons—and for recruiting parents to fill out baseline surveys that included contact information so that research staff could conduct telephone follow-up interviews with parents at the end of the summer. The pool manager or aquatic supervisor at each pool was responsible for that pool’s participation and the level of participation of its staff members.
Because retention of participating field coordinators, pool managers, and aquatic staff was important to successful completion of the study, the project team built relationships with professional organizations and recreation sites at national, regional, and local levels. This was achieved by participating in aquatics and recreation conferences, developing career opportunities, encouraging local media coverage of program activities,24,25 and providing access to resources to conduct the program after research participation concluded.26,27
Intervention
The Pool Cool program is an educational and environmental policy intervention for skin cancer prevention at outdoor swimming pools.19 All participating pools, including the basic pool sites, received a Pool Cool toolkit and annual training for aquatics staff from the field coordinator. The toolkits were similar to the ones used in the efficacy trial and dissemination pilot tests and included a Leader’s Guide, 3 sun-safety signs, a gallon pump container of sunscreen, and materials for poolside activities (laminated lesson sheets).18,19
Pools in the enhanced condition received extra incentives, reinforcements, feedback, and skill-building guidance and tools to increase sustainability. They received additional sun-safety resources (game board, Mini Big Book with graphics for lessons), Pool Cool incentive items (e.g., hats/T-shirts, message pens, lanyards, UV-sensitive stickers to detect UV light, water bottles), sun-safety signs, and shade supports (e.g., umbrellas for lifeguard stands) or structures (awnings or canopies). The enhanced pools and field coordinators were also given manuals titled, How to Make Pool Cool More Effective and The Pool Cool Guide to Sustainability. The sustainability guide included suggestions and methods for securing continued funding and support, including developing partnerships with local organizations to continue the program after the end of the research study. Enhanced pools and field coordinators also participated in a Frequent Applier program to earn raffle points as incentives to encourage maximum participation in the program. Raffled items included conference registrations, extra Pool Cool logo items (e.g., hats, lanyards, pens), extra gallons of sunscreen, and shade structures. Enhanced-group field coordinators participated in 2 to 3 additional conference calls each summer and were actively engaged in discussions about program maintenance and sustainability that were not conducted with field coordinators in the basic group.
Data Collection Procedures
The pool manager at each participating pool was asked to fill out a survey prior to the lifeguard training and again at the end of the summer during each year of participation. Surveys were distributed to pools by the regional field coordinators, who collected them and mailed them back to the study office. Pool managers received gift cards ($5 and $10) for returning completed baseline and follow-up manager surveys.
Measures
The pool manager surveys asked about the respondents’ background; pool organizational characteristics; and pool-level measures of implementation, maintenance, and sustainability in each successive year. The survey measures were selected or adapted from our earlier studies of skin cancer prevention in aquatic settings.18–20 We collected data about the pool manager’s demographic characteristics, length of tenure at the pool, skin cancer risk factors,28 sun exposure, skin self-examination behaviors, and sunburns.29 Questions about pool and organizational characteristics included location, size, and type of community (urban vs suburban or rural); workforce stability and turnover; and number of pool staff and patrons.
Key study outcome variables were sun-safety policies and environments, obstacles and supporting factors for sun safety, and program implementation, maintenance, and sustainability. Because these variables were measured with multiple survey items, we created composite scores for each. We measured sun-safety policies and environments with 11 items at baseline and follow-up each year.25 The composite included 2 questions about whether the pool offered programs and policies for lifeguards and for swimmers and 9 questions about sun-safety policies for pool patrons. The items were on a 4-point scale (1 = rarely/never to 4 = usually/always), and missing responses were recoded as rarely/never. The 11 items were summed to create a policy and environment scale score (α = 0.71–0.78).
At baseline each year, we asked pool managers to rate the importance of 4 supporting factors (health concern, risk management of employees, community or citizen demand, and community relations) and 4 obstacles (limited budget, lack of information, pool facility design, and low priority) to sun safety at the pool. Items were on a 4-point scale (1 = not at all important to 4 = very important), and missing responses were recoded as not at all important. We summed each set of items to create a supporting factors scale score (α = 0.84–0.85).
Pool-level measures of implementation, maintenance, and sustainability21,30 of the Pool Cool program were assessed at follow-up in each successive year. Implementation of the Pool Cool program was assessed at follow-up each year with 10 items asking whether the main components of the Pool Cool program were used and at what level. Four questions were on a 4-point scale and asked about the frequency of educational activities in swimming lessons, sun-safety education programs, teaching of Pool Cool lessons, and use of the Pool Cool Leader’s Guide. These items were recoded (1 = sometimes, rarely, or never to 3 = usually or always). The other 6 questions were yes-or-no items and asked if the pool used the Mini Big Book, conducted poolside activities, displayed the sunscreen tips poster, displayed the aluminum sun-safety signs, used the sunscreen provided, and added shade structures or shaded areas this summer. These yes-or-no items were also recoded (1 = no and 3 = yes). Missing responses were recoded as a 1 (no), and the 10 items were summed to create a composite implementation score (α = 0.68–0.72). Maintenance of program implementation was defined as continuing the same level of program implementation during the second year of participation and was assessed with the implementation variables and composite scores from each pool’s second follow-up survey.
Sustainability of the Pool Cool program was measured at follow-up by 2 items asking how much the Pool Cool program was modified to better fit the pool and how much pool procedures and policies were modified to better fit the Pool Cool program (1 = not at all to 4 = a lot). Missing responses were recoded as 1 (not at all), and the 2 items were summed to create a composite sustainability score for 2005 and 2006 (α = 0.65). In 2006 (year 4), 9 additional sustainability questions were included on the pool manager follow-up survey. Six items were on a 5-point scale (1 = strongly disagree to 5 = strongly agree). These items asked whether managers at the pool actively advocated to continue Pool Cool, whether lead responsibility for coordinating and implementing the program had been assigned to a single lead person, whether the Pool Cool program had become a permanent part of the programming at the facility, whether funding was earmarked to continue the Pool Cool program at the pool, whether relationships with outside organizations to help continue the Pool Cool program at the pool had been established, and whether the Pool Cool program was widely recognized and accepted by the community and patrons. The other 3 items asked whether the pool had added any environmental supports, had clear goals and objectives, and had established a written schedule. These items were coded no = 1 and yes = 5 to match the other 6 sustainability items. We summed the 9 items to create an additional composite sustainability score for 2006 (α = 0.64).
Process Evaluation
We conducted an independent process evaluation each summer to measure program implementation. Each year, poolsite observations and onsite interviews were completed at a sample of 40 pools, and telephone interviews were conducted with an additional sample of 80 pools.24,31,32 The interview asked about the Pool Cool kit of materials, training of pool staff on the program, and receipt and use of the educational and environmental components. Site visit observations documented the pool environment and sun-safety practices of aquatic staff. Additional process evaluation included training surveys, field coordinator activity logs, tracking e-mails,25 and items on surveys of parents, lifeguards, and pool managers.24
Only data from the annual interview component of the process evaluation are reported here.
Statistical Analysis
Data analyses were completed with SPSS version 18.0 (IBM, Somers, NY). Preliminary analyses included computing descriptive statistics for all pool manager surveys, assessing nonresponse bias at each stage of the study, and evaluating treatment-group equivalence on key characteristics at baseline. For the main analysis, we analyzed data across the 3 phases of each pool’s participation in the study: first year, second year, and last year of participation (which could be year 4 for pools in the 2003 cohort). To be included in the implementation outcome analysis, the pool manager survey had to be completed at baseline and follow-up in year 1. Inclusion in the maintenance analysis required at least 1 baseline and 2 follow-up pool manager surveys, and for the sustainability analysis, the pool had to participate in the study for at least 3 years and complete a baseline and final year follow-up pool manager survey. We used repeated-measures analyses of covariance (ANCOVAs) to assess treatment-group differences in implementation, maintenance, sustainability, and sun-safety policies and environments. Preliminary analyses indicated that 2 covariates—pool attendance size and supporting factors—were used to control for significant organizational predictors in these models.
Analysis of the process evaluation interviews included descriptive statistics to summarize implementation and calculate implementation scores for each pool.32 We used the χ2 test and t test to examine differences in implementation between the treatment-group pools and between years. We computed an implementation score from 16 items about training, lessons, poolside activities, sun-safety signage, and sunscreen and calculated a 10-item supplementary score for use of program items.32
RESULTS
A total of 435 pools participated in the trial and returned a pool manager baseline survey in year 1, and 390 (89.7%) completed both a pool manager baseline and a pool manager follow-up survey and thus were included in analysis for implementation outcomes. There were 286 pools (73.3%) in the maintenance analysis and 296 pools (75.9%) in the sustainability analysis (Table 1). Treatment groups were nearly evenly represented at each stage of the outcome analyses.
TABLE 1—
Pool and Manager Demographics at Implementation, Maintenance, and Sustainability: Pool Cool Diffusion Trial, 33 US Metropolitan Areas, 2003–2010
| Variable | Implementation (n = 390), No. (%) | Maintenance (n = 286), No. (%) | Sustainability (n = 296), No. (%) |
| Group | |||
| Basic | 207 (53.0) | 138 (48.3) | 151 (51.0) |
| Enhanced | 183 (47.0) | 148 (51.7) | 145 (49.0) |
| Latitude | |||
| North | 192 (49.2) | 139 (48.6) | 151 (51.0) |
| South | 198 (50.8) | 147 (51.4) | 145 (49.0) |
| Community description | |||
| Urban | 155 (39.7) | 121 (42.3) | 123 (41.6) |
| Suburban or rural | 235 (60.3) | 165 (57.7) | 173 (58.4) |
| Average length of staff employment, y | |||
| < 1 | 46 (11.9) | 32 (11.2) | 35 (11.8) |
| 1–2 | 151 (39.2) | 113 (39.5) | 117 (39.5) |
| ≥ 3 | 188 (48.8) | 141 (49.3) | 144 (48.6) |
| No. of people in the community | |||
| ≤ 49 999 | 110 (29.8) | 80 (28.0) | 84 (28.4) |
| 50 000–99 999 | 94 (25.5) | 72 (25.2) | 77 (26.0) |
| 100 000–299 999 | 63 (17.1) | 43 (15.0) | 39 (13.2) |
| ≥ 300 000 | 102 (27.7) | 91 (31.8) | 96 (32.4) |
| No. of staff | |||
| 1–10 | 135 (34.6) | 103 (36.0) | 109 (36.9) |
| 11–22 | 125 (32.1) | 82 (28.7) | 91 (30.8) |
| ≥ 23 | 130 (33.3) | 101 (35.3) | 95 (32.2) |
| Attendance | |||
| < 500–999 | 156 (40.0) | 114 (39.9) | 119 (40.2) |
| 1000–1999 | 123 (31.6) | 85 (29.7) | 87 (29.4) |
| ≥ 2000 | 111 (28.5) | 87 (30.4) | 90 (30.4) |
| Gender | |||
| Male | 158 (40.5) | 116 (40.6) | 108 (37.5) |
| Female | 232 (59.5) | 170 (59.4) | 180 (62.5) |
| Years at pool | |||
| ≤ 1 | 103 (27.8) | 76 (27.9) | |
| 2–4 | 142 (38.4) | 96 (35.3) | |
| ≥ 5 | 125 (33.8) | 100 (36.8) | |
| Age, y | |||
| ≤ 21 | 106 (27.4) | 73 (25.6) | 84 (29.0) |
| 22–28 | 103 (26.6) | 77 (27.0) | 113 (39.0) |
| ≥ 29 | 178 (46.0) | 135 (47.4) | 93 (32.1) |
| Race/ethnicity | |||
| White | 331 (84.9) | 246 (86.0) | 245 (82.8) |
| Other | 59 (15.1) | 40 (14.0) | 51 (17.2) |
| Education | |||
| ≤ Some college | 185 (48.1) | 137 (48.8) | |
| Completed college | 159 (41.3) | 117 (41.6) | |
| Completed graduate school | 41 (10.6) | 27 (9.6) | |
| Marital status | |||
| Married or part of a couple | 167 (43.3) | 127 (44.9) | |
| Single | 219 (56.7) | 156 (55.1) | |
| Have children | |||
| Yes | 141 (36.5) | 100 (35.3) | |
| No | 245 (63.5) | 183 (64.7) | |
| No. of sunburns last summer | |||
| 0 | 119 (30.7) | 84 (29.6) | |
| 1 | 121 (31.2) | 95 (33.5) | |
| ≥ 2 | 148 (38.1) | 105 (36.9) | |
| Skin cancer risk category | |||
| Low | 132 (34.1) | 67 (23.6) | |
| Moderate | 133 (34.1) | 68 (23.9) | |
| High | 122 (31.5) | 149 (52.5) |
Note. Sample sizes vary based on missing responses.
In year 1, the pools were equally distributed between northern (50.8%) and southern (49.2%) latitudes, mostly located in suburban or rural areas (60.3%). Most pools (71.5%) admitted fewer than 2000 visitors to the pool each week, and they were evenly distributed among having a small staff (1–10 employees), medium staff (11–22 employees), and more than 23 employees. Pools in the South were more likely to be in larger communities, and pools in the North were more likely to have a pool manager with higher skin cancer risk. Pool managers were 59.5% females and predominantly White (84.9%), and 72.6% were older than 21. More than half (51.9%) had completed college, and all had some college education. Nearly three quarters (72.2%) had worked at the same pool for more than 2 years.
Treatment-group pools were equivalent on most pool manager and pool characteristics, with the exception of enhanced-group pool managers being significantly older and having a longer tenure at the pool (40.4% vs 28.1% working more than 5 years; P = .05). Completion of multiple pool manager follow-up surveys for the maintenance and sustainability analyses was associated with longer pool manager employment at the pool, with the pool manager being younger than 28, and with being located in a larger community.
Main Results
The main aims of this study were to evaluate the effects of 2 strategies (basic vs enhanced) for dissemination of the Pool Cool skin cancer prevention program on (1) program implementation, maintenance, and sustainability and (2) improvements in organizational and environmental supports for sun protection at swimming pools.
Program implementation after a summer of participation.
A simple analysis of variance (ANOVA) with treatment condition as the between-participants factor showed no significant difference between basic and enhanced study pools at the end of the first summer. We also conducted a 1-way ANCOVA that controlled for pool size and supporting factors. As can be seen in Table 2, this analysis found no statistically significant main effect for treatment condition at the end of the first summer of participation. After a single summer of participation in Pool Cool, both basic and enhanced pools began implementing the program, and there was no difference in implementation between treatment conditions (Figure 1).
TABLE 2—
Summary of Repeated-Measures Analyses of Covariance (ANCOVA) Results Showing Scale Scores by Treatment Group With Control for Pool Attendance Size and Supporting Factors: Pool Cool Diffusion Trial, 33 US Metropolitan Areas, 2003–2010
| Estimated Marginal Means (SE) |
|||||||
| Measure | Baseline | First Follow-Up | Second Follow-Up | Final Follow-Up | Significance, Treatment | Significance, Time | Significance, Treatment × Time |
| First-year completers, n = 390 | |||||||
| Implementationa | |||||||
| Basic (n = 207) | — | 23.08 (0.29) | F = 0.44; P = .51 | ||||
| Enhanced (n = 183) | — | 22.80 (0.31) | |||||
| Policy and environmentb | |||||||
| Basic (n = 207) | 21.61 (0.40) | 25.83 (0.36) | F = 0.11; P = .74 | F = 7.50; P = .006; partial η2 = 0.02 | F = 0.41; P = .52 | ||
| Enhanced (n = 183) | 21.24 (0.43) | 25.90 (0.38) | |||||
| Second-year completers (n = 286) | |||||||
| Maintenancec | |||||||
| Basic (n = 138) | — | 23.44 (0.35) | 23.53 (36) | F = 0.35; P = .55 | F = 6.52; P = .01; partial η2 = 0.02 | F = 4.11; P = .04; partial η2 = 0.02 | |
| Enhanced (n = 148) | — | 22.63 (0.34) | 23.86 (0.35) | ||||
| Policy and environment | |||||||
| Basic (n = 138) | 21.96 (0.51) | 26.77 (0.45) | F = 0.01; P = .92; partial η2 = 0.02 | F = 21.56; P < .001; partial η2 = 0.07 | F = 4.61; P = .03; partial η2 = 0.03 | ||
| Enhanced (n = 148) | 21.09 (0.49) | 27.74 (0.43) | |||||
| Final-year completersa (n = 296) | |||||||
| Maintenance | |||||||
| Basic (n = 151) | — | 23.71 (0.32) | 23.12 (0.34) | F = 0.27; P = .61 | F = 1.98; P = .16 | F = 7.13; P = .008; partial η2 = 0.02 | |
| Enhanced (n = 145) | — | 22.82 (0.32) | 23.61 (0.34) | ||||
| Policy and environment | |||||||
| Basic (n = 151) | 21.94 (0.49) | 26.43 (0.41) | F = 0.48; P = .49 | F = 15.50; P < .001; partial η2 = 0.05 | F = 8.88; P = .003; partial η2 = 0.03 | ||
| Enhanced (n = 145) | 21.04 (0.50) | 28.02 (0.42) | |||||
Implementation scores: range = 8–30; median = 23.00.
Policy and environment scores: range = 11–42; median = 22.00.
Maintenance scores: range = 9–30, median = 23.00.
FIGURE 1—
Implementation scores by condition over time: Pool Cool Diffusion Trial, 33 US metropolitan areas, 2003–2010.
Note. *P < .05; **P < .01.
Program maintenance at second follow-up.
We conducted a 2-way 2 (treatment) × 2 (time) repeated-measures ANCOVA that controlled for pool attendance size and supporting factors to assess maintenance of the program and possible differences by treatment conditions after the second summer. As reported in Table 2, this analysis found that there was a statistically significant main effect for program participation over time (F1,275 = 6.52; P = .01; partial η2 = 0.02). Those pools that continued with the program improved their implementation of the program over the second summer (Figure 1). Results also showed a significant interaction between treatment group and time (F1,275 = 4.11; P = .04; partial η2 = 0.02), with a greater increases among pools in the enhanced group.
Program maintenance at the last follow-up.
Finally, we assessed the program maintenance and treatment differences for pools that completed the course of the study. As shown in Table 2, results from the repeated-measures ANCOVA indicated that there was a statistically significant interaction for maintenance by treatment group (F1,291 = 7.13; P = .008; partial η2 = 0.02). Those pools in the enhanced condition had greater overall maintenance of the program over 3 summers of Pool Cool participation than did those in the basic group (Figure 1).
Sun-safety policies and environments in the first year.
A second aim of this study was to assess sun-safety policies and environmental improvements at study pools over the course of the program. Basic ANOVA analyses indicated no differences in these policies and pool environments between pools assigned to basic and enhanced study conditions at baseline. As seen in Table 2, this analysis showed that there was a statistically significant improvement in policy and environments for both basic and enhanced groups over the first summer (F1,377 = 7.50; P = .006; partial η2 = 0.02). Although this increase was slightly greater for the enhanced group of pools than for the basic group, the difference was not statistically significant (Figure 2).
FIGURE 2—
Policy and environment scores by condition over time: Pool Cool Diffusion Trial, 33 US metropolitan areas, 2003–2010.
Note. *P < .05; **P < .01.
Sun-safety policies and environments at second follow-up.
We assessed pool sun-safety policies and environmental improvements after pools participated in Pool Cool for a second summer. As shown in Table 2, this repeated-measures ANCOVA analysis indicated a statistically significant main effect for policy and environmental improvements over time (F1,275 = 21.56; P < .001; partial η2 = 0.07) and an interaction between treatment and time (F1,275 = 4.61; P = .03; partial η2 = 0.03), with those in the enhanced group increasing their scores more over the period. Pools in the enhanced group increased their sun-safety policies and environments more over the second year of the study (Figure 2).
Sun-safety policies and environments at the last follow-up.
Finally, we assessed the sun-safety policies and environmental improvements for pools that completed the course of the study. As can be seen in Table 2, this analysis showed a statistically significant main effect for policy and environmental scores between the 2 periods (F1,291 = 15.50; P = .001; partial η2 = 0.05). Pools in both conditions increased their sun-safety policies and environments. The results also showed no statistically significant main effect for treatment, but there was a significant interaction between policy and environments and treatment condition over time (F1,291 = 8.88; P = .003; partial η2 = 0.03), with those in the enhanced condition implementing and maintaining greater sun-safety polices over the course of the study (Figure 2).
Program sustainability.
We conducted a 1-way ANCOVA that controlled for pool attendance size and supporting factors for sustainability summary scores in 2006. This analysis showed no statistically significant main effect for treatment group. Pools in the basic group were more likely to report that an individual had been assigned the lead responsibility for coordinating and implementing Pool Cool at their sites (22.2% vs 3.8% strongly agree; P = .01). Pools in the enhanced group were more likely to report adding any environmental supports (56.3% vs 41.1%; P = .05).
Process Evaluation
The process evaluation data from 2003 and 2006 indicated fairly high implementation levels across pools and years, with teaching sun-safety lessons and using the supplied sunscreen as the most-often-implemented components. The primary mean implementation score for the pools ranged from 68.3% to 73.2% from 2003 to 2006, and the supplementary scores ranged from 30.7% to 36.9% out of 100%. Implementation scores between the basic and the enhanced conditions were similar with no significant differences between groups. For specific activities, pools in the enhanced condition scored higher than those in the basic condition on conducting lifeguard training and teaching the Pool Cool lessons; the differences were statistically significant only in 2005.
The key factors that facilitated Pool Cool program implementation included receiving the Pool Cool kit and materials, knowledge about skin cancer, field coordinator support, and the ease of conducting the program. The most-often-identified barrier to implementing Pool Cool was time pressure. In the last year of evaluation, pool contacts were asked how they would sustain the program in the future. More than 76% reported that they would continue the program by incorporating it into routine pool operations and securing management support.
DISCUSSION
This trial was one of the few health promotion studies to measure implementation, policy and environmental changes, and behavior over multiple years. Pools in both conditions improved their implementation of the program over time; however, the enhanced condition had significantly greater maintenance of program components over the 3 years. Also, although both groups increased their sun-safety policies and environments, there was a significant interaction between policy and environments and treatment over time, with those in the enhanced condition building and maintaining greater sun-safety polices over both follow-up periods.
Provision of a packaged and well-tested program kit, training of aquatic staff, and use of a Leader’s Guide describing how to implement the program facilitated program implementation. These components have been shown to assist with health program adoption in other organizations.33 In addition, the Pool Cool program was multicomponent and integrated individual and organizational change approaches. This approach to environmental and policy change is critical to program maintenance because structural change strategies can be more lasting than individual behavior change.34
We incorporated additional planning for sustainability into the enhanced dissemination program to build capacity among aquatic staff to sustain organizational change. This process is essential to maintaining the program elements over time. As the process evaluation data showed, the integration of Pool Cool into the operations of the pools and the presence of program champions contributed to its sustainability. These processes have been recommended as elements to increase program continuation.35,36
Few diffusion studies have explored implementation and sustainability and tested strategies to improve dissemination.14,37,38 Buller et al.14 tested the effects of an enhanced dissemination strategy with personal contact on program implementation and sustainability of an occupational sun-safety intervention in 69 ski areas and found it more effective than a basic dissemination strategy. Their results are consistent with those found in this study. We cannot say what, specifically, about the treatments is responsible for the differences; however, our extensive process evaluation indicated that the pool managers in the enhanced condition often used the 2 additional manuals that were provided, especially the sustainability guide, to aid in plans for maintaining the program.
The Pool Cool program can be easily replicated, and there has been interest and implementation among other pools that did not participate in the study.26,39 Strategies such as providing well-packaged, user-friendly program materials at low or no cost and strategic advertisement of the availability of program materials may increase program use.33,40 Future research is warranted through longitudinal follow-up to examine further sustainability. Recommendations are increasing for sustainability research as a next stage in the translation or dissemination of evidence-based interventions into practice.41
Strengths of this implementation research include the large sample of geographically dispersed pools, rigorous intervention and research methodology, and triangulation of process and outcome evaluations. This study was ambitious, and its national scope presented challenges for retention of participating pools and maintaining a high response rate. Reliance on self-report of program implementation might be considered a limitation, but this was less of a concern because of the monitoring conducted through independent observations of a sample of sites each year.24 The intervention continues to be implemented without a formal dissemination strategy, well into the second decade of the 21st century.27
Acknowledgments
The research reported here was supported by the National Cancer Institute (grant CA 92505).
The authors acknowledge the contributions of Frances McCarty, Dawn Hall Holman, Lucja Bundy, Kathy Spangler, the Aquatics Section of the National Recreation and Parks Association, and the participating pool managers and field coordinators.
Human Participant Protection
All study procedures were approved by the institutional review boards of the University of Hawaii, University of North Carolina, and Emory University.
References
- 1.Jemal A, Saraiya M, Patel P et al. Recent trends in cutaneous melanoma incidence and death rates in the United States, 1992-2006. J Am Acad Dermatol. 2011;65(5 suppl 1):S17–S25.e1–e3. doi: 10.1016/j.jaad.2011.04.032. [DOI] [PubMed] [Google Scholar]
- 2.American Cancer Society. Cancer Facts & Figures 2014. Atlanta, GA: American Cancer Society; 2014. [Google Scholar]
- 3.Oliveria SA, Saraiya M, Geller AC, Heneghan MK, Jorgensen C. Sun exposure and risk of melanoma. Arch Dis Child. 2006;91(2):131–138. doi: 10.1136/adc.2005.086918. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Centers for Disease Control and Prevention. Skin cancer. 2014. Available at: http://www.cdc.gov/cancer/skin. Accessed July 3, 2014.
- 5.Green AC, Williams GM, Logan V, Strutton GM. Reduced melanoma after regular sunscreen use: randomized trial follow-up. J Clin Oncol. 2011;29(3):257–263. doi: 10.1200/JCO.2010.28.7078. [DOI] [PubMed] [Google Scholar]
- 6.van der Pols JC, Williams GM, Pandeya N, Logan V, Green AC. Prolonged prevention of squamous cell carcinoma of the skin by regular sunscreen use. Cancer Epidemiol Biomarkers Prev. 2006;15(12):2546–2548. doi: 10.1158/1055-9965.EPI-06-0352. [DOI] [PubMed] [Google Scholar]
- 7.Saraiya M, Hall HI, Uhler RJ. Sunburn prevalence among adults in the United States, 1999. Am J Prev Med. 2002;23(2):91–97. doi: 10.1016/s0749-3797(02)00461-0. [DOI] [PubMed] [Google Scholar]
- 8.Gallagher RP, Lee TK, Bajdik CD, Borugian M. Ultraviolet radiation. Chronic Dis Can. 2010;29(suppl 1):51–68. [PubMed] [Google Scholar]
- 9.Saraiya M, Glanz K, Briss PA et al. Interventions to prevent skin cancer by reducing exposure to ultraviolet radiation: a systematic review. Am J Prev Med. 2004;27(5):422–466. doi: 10.1016/j.amepre.2004.08.009. [DOI] [PubMed] [Google Scholar]
- 10. Community Preventive Services Task Force. The Guide to Community Preventive Services: The Community Guide. 2014. Available at: http://www.thecommunityguide.org/index.html. Accessed February 19, 2014.
- 11.Glanz K, Mayer JA. Reducing ultraviolet radiation exposure to prevent skin cancer methodology and measurement. Am J Prev Med. 2005;29(2):131–142. doi: 10.1016/j.amepre.2005.04.007. [DOI] [PubMed] [Google Scholar]
- 12.Lewis E, Mayer JA, Slymen D et al. Disseminating a sun safety program to zoological parks: the effects of tailoring. Health Psychol. 2005;24(5):456–462. doi: 10.1037/0278-6133.24.5.456. [DOI] [PubMed] [Google Scholar]
- 13.Buller DB, Andersen PA, Walkosz BJ et al. Enhancing industry-based dissemination of an occupational sun protection program with theory-based strategies employing personal contact. Am J Health Promot. 2012;26(6):356–365. doi: 10.4278/ajhp.110113-QUAN-22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Buller DB, Walkosz BJ, Andersen PA et al. Sustainability of the dissemination of an occupational sun protection program in a randomized trial. Health Educ Behav. 2012;39(4):498–502. doi: 10.1177/1090198111420287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Tibbits MK, Bumbarger BK, Kyler SJ, Perkins DF. Sustaining evidence-based interventions under real-world conditions: results from a large-scale diffusion project. Prev Sci. 2010;11(3):252–262. doi: 10.1007/s11121-010-0170-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rohrbach LA, Ringwalt CL, Ennett ST, Vincus AA. Factors associated with adoption of evidence-based substance use prevention curricula in US school districts. Health Educ Res. 2005;20(5):514–526. doi: 10.1093/her/cyh008. [DOI] [PubMed] [Google Scholar]
- 17.Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17. doi: 10.1186/1748-5908-7-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Geller AC, Glanz K, Shigaki D et al. Impact of skin cancer prevention on outdoor aquatics staff: the Pool Cool program in Hawaii and Massachusetts. Prev Med. 2001;33(3):155–161. doi: 10.1006/pmed.2001.0870. [DOI] [PubMed] [Google Scholar]
- 19.Glanz K, Geller AC, Shigaki D, Maddock JE, Isnec MR. A randomized trial of skin cancer prevention in aquatics settings: the Pool Cool program. Health Psychol. 2002;21(6):579–587. [PubMed] [Google Scholar]
- 20.Glanz K, Steffen A, Elliott T, O’Riordan D. Diffusion of an effective skin cancer prevention program: design, theoretical foundations, and first-year implementation. Health Psychol. 2005;24(5):477–487. doi: 10.1037/0278-6133.24.5.477. [DOI] [PubMed] [Google Scholar]
- 21.Goodman RM, McLeroy KR, Steckler AB, Hoyle RH. Development of level of institutionalization scales for health promotion programs. Health Educ Q. 1993;20(2):161–178. doi: 10.1177/109019819302000208. [DOI] [PubMed] [Google Scholar]
- 22.Rogers EM. Diffusion of Innovations. 5th ed. New York, NY: Free Press; 2003. [Google Scholar]
- 23.Steckler A, Goodman RM, Kegler MC. Mobilizing organizations for health enhancement: theories of organizational change. In: Glanz K, Rimer B, Lewis FM, editors. Health Behavior and Health Education: Theory, Research and Practice. 3rd ed. San Francisco, CA: Jossey-Bass; 2002. pp. 335–360. [Google Scholar]
- 24.Escoffery C, Glanz K, Hall D, Elliott T. A multi-method process evaluation for a skin cancer prevention diffusion trial. Eval Health Prof. 2009;32(2):184–203. doi: 10.1177/0163278709333154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hall DM, McCarty F, Elliott T, Glanz K. Lifeguards’ sun protection habits and sunburns: association with sun-safe environments and skin cancer prevention program participation. Arch Dermatol. 2009;145(2):139–144. doi: 10.1001/archdermatol.2008.553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Hall DM, Escoffery C, Nehl E, Glanz K. Spontaneous diffusion of an effective skin cancer prevention program through Web-based access to program materials. Prev Chronic Dis. 2010;7(6):A125. [PMC free article] [PubMed] [Google Scholar]
- 27. University of Pennsylvania. Welcome to Pool Cool. 2010. Available at: http://www.pool-cool.org. Accessed February 19, 2014.
- 28.Glanz K, Schoenfeld E, Weinstock MA et al. Development and reliability of a brief skin cancer risk assessment tool. Cancer Detect Prev. 2003;27(4):311–315. doi: 10.1016/s0361-090x(03)00094-1. [DOI] [PubMed] [Google Scholar]
- 29.Glanz K, Yaroch AL, Dancel M et al. Measures of sun exposure and sun protection practices for behavioral and epidemiologic research. Arch Dermatol. 2008;144(2):217–222. doi: 10.1001/archdermatol.2007.46. [DOI] [PubMed] [Google Scholar]
- 30.Steckler A, Goodman RM, McLeroy KR, Davis S, Koch G. Measuring the diffusion of innovative health promotion programs. Am J Health Promot. 1992;6(3):214–224. doi: 10.4278/0890-1171-6.3.214. [DOI] [PubMed] [Google Scholar]
- 31.Glanz K, Isnec MR, Geller AC, Spangler K. Process evaluation of implementation and dissemination of a sun safety program at swimming pools. In: Steckler A, Linnan L, editors. Process Evaluation in Public Health Interventions. San Francisco, CA: Jossey-Bass; 2002. pp. 58–82. [Google Scholar]
- 32.Escoffery C, Glanz K, Elliott T. Process evaluation of the Pool Cool Diffusion Trial for skin cancer prevention across 2 years. Health Educ Res. 2008;23(4):732–743. doi: 10.1093/her/cym060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rohrbach LA, Grana R, Sussman S, Valente TW. Type II translation: transporting prevention interventions from research to real-world settings. Eval Health Prof. 2006;29(3):302–333. doi: 10.1177/0163278706290408. [DOI] [PubMed] [Google Scholar]
- 34.Brownson RC, Haire-Joshu D, Luke DA. Shaping the context of health: a review of environmental and policy approaches in the prevention of chronic diseases. Annu Rev Public Health. 2006;27:341–370. doi: 10.1146/annurev.publhealth.27.021405.102137. [DOI] [PubMed] [Google Scholar]
- 35.Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320–347. [Google Scholar]
- 36.Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13(1):87–108. doi: 10.1093/her/13.1.87. [DOI] [PubMed] [Google Scholar]
- 37.Hoelscher DM, Kelder SH, Murray N et al. Dissemination and adoption of the Child and Adolescent Trial for Cardiovascular Health (CATCH): a case study in Texas. J Public Health Manag Pract. 2001;7(2):90–100. doi: 10.1097/00124784-200107020-00012. [DOI] [PubMed] [Google Scholar]
- 38.Allicock M, Campbell MK, Valle CG et al. Evaluating the dissemination of Body & Soul, an evidence-based fruit and vegetable intake intervention: challenges for dissemination and implementation research. J Nutr Educ Behav. 2012;44(6):530–538. doi: 10.1016/j.jneb.2011.09.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Cariou C, Gonzales M, Krebill H. Adapting and implementing an evidence-based sun-safety education program in rural Idaho, 2012. Prev Chronic Dis. 2014;11:E77. doi: 10.5888/pcd11.130268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based interventions in health care: application of the replicating effective programs framework. Implement Sci. 2007;2:42. doi: 10.1186/1748-5908-2-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–2067. doi: 10.2105/AJPH.2011.300193. [DOI] [PMC free article] [PubMed] [Google Scholar]