Abstract
Objective:
The objective of this study is to compare three methods of administering a sleep health program (SHP) in fire departments.
Methods:
An SHP, comprising sleep health education and screening for common sleep disorders, was implemented in eight fire departments using three approaches: expert-led, train-the-trainer, and online. Participation rates, knowledge assessments, surveys, and focus group interviews were analyzed to assess the reach and effectiveness of the methodologies.
Results:
The Expert-led SHP had the highest participation rate, greatest improvement in knowledge scores, and prompted more firefighters to seek clinical sleep disorder evaluations (41%) than the other approaches (20 to 25%). Forty-two percent of focus group participants reported changing their sleep behaviors.
Conclusion:
All approaches yielded reasonable participation rates, but expert-led programs had the greatest reach and effectiveness in educating and screening firefighters for sleep disorders.
The public expects firefighters to be available around-the-clock and to perform flawlessly when called upon. The majority of career firefighters work 24-hour extended-duration shifts, which cause acute sleep deprivation, chronic sleep deficiency, and circadian misalignment.1 The negative consequences of sleep loss are of major concern to firefighters2 and more than half of firefighters report sleep disturbances.3 In addition, a significant proportion of firefighters suffer from undiagnosed sleep disorders, which further impair sleep and exacerbate fatigue.4 Sleep deficiency and sleep disorders are associated with an increased risk of crashes, injuries, and adverse mental and physical health outcomes,3–8 including a greatly increased risk of motor vehicle crashes and heart attacks, the two leading causes of death in firefighters.9 In addition, the annual direct costs to the United States from sleep disorders and sleepiness are estimated to be in excess of 15 billion dollars, and total costs including lost productivity may exceed 150 billion dollars.10 The total cost of accidents due to sleepiness has been estimated to be between 43 and 56 billion dollars.11
Fatigue education and awareness training programs are required in commercial aviation12 and similar programs have been instituted successfully for railroad workers,13 truckers,14 and nurses.15,16 Recently, we developed a sleep health program (SHP) for firefighters, which included some of the key components recommended for a comprehensive fatigue risk management program17: sleep health education, incorporating caffeine re-education, and advice on dealing with shiftwork, as well as screening for common undiagnosed sleep disorders. We educated and screened almost 7000 firefighters in 66 US fire departments across 32 states, with departments ranging in size from 22 to 1574 firefighters. Sleep disorders were highly prevalent in these active-duty career firefighters, with over one-third (37%) of firefighters screening positive for at least one sleep disorder, most of whom were previously undiagnosed. Twenty-eight percent screened positive for obstructive sleep apnea (OSA), 6% for insomnia, 9% for shift work disorder (SWD), and 3% for restless legs syndrome (RLS). Compared with those who did not screen positive, firefighters who screened positive for a sleep disorder were 3.5 times more likely to report getting injured, more than twice as likely to take a sick day or self-report falling asleep while driving, and had double the risk of reporting a motor vehicle crash.4
In the current study, we employed multiple approaches to evaluate the reach and effectiveness of three delivery methods for the SHP: expert-led (EL); train-the-trainer (TT); and online (OL). We characterized participation rates, knowledge gained by firefighters after completing the education, the rate of evaluation, and treatment for firefighters identified as being at risk for a sleep disorder, and the perceived strengths and challenges associated with each implementation methodology.
METHODS
We performed a study to compare the reach and effectiveness of three methodologies to deliver an SHP to firefighters. The SHP consisted of a 30-minute educational training session on healthy sleep and fatigue countermeasures, including video testimonials from firefighters previously treated for sleep disorders, and union and departmental leadership encouraging participation in the program, followed by a screening for common sleep disorders. We conducted this program in eight fire departments and compared the participation rates, knowledge gained, and effectiveness of the programs following three different implementation approaches: EL (two departments); TT (two departments); and OL (four departments).
Recruitment
Recruitment letters were sent to the 3511 city, county, and federal U.S. fire departments, which have three or more station houses, in order to gauge interest in participating in the SHP. A semi-structured phone interview was conducted with interested groups to determine eligibility and identify the most suitable form of program administration.
Sixty-six out of 117 departments interviewed were selected to participate in the survey study and eight of these were further selected to participate in the evaluation of implementation methodology. On the basis of the number of firefighters, level of departmental interest, experience implementing wellness programs, and the ability to meet implementation requirements (eg, Internet access, availability of trainers), two departments were assigned to EL, two departments to TT, and four departments to OL methodology. Departments were not assigned randomly. Two additional departments were included in the OL group in anticipation of lower participation rates.
Implementation
The sleep health education was delivered using three different methodologies. In the departments assigned to EL, a member of the research team presented the education and screening program in-person to groups ranging from 2 to 50 firefighters. Each department assigned to the TT methodology appointed 8 to 12 peer trainers who completed a 2-day training course taught by the research team. This course provided trainers with a background understanding of sleep health, and training in the logistics necessary to conduct the sleep disorders screening. The peer trainers subsequently led the sleep health educational training in their departments to groups of 3 to 16 firefighters. To ensure consistency and quality, a member of the investigative team evaluated approximately 10% of peer trainers’ presentations using an assessment checklist and provided summarized critiques to the trainers and supervisors. For the OL departments, a Web site was provided with a video of the education session and a toll-free helpline.
Sleep Disorders Screening
Following the education training session, firefighters were given the opportunity to participate in a voluntary screening program for common sleep disorders. Validated tools were used for identifying the risk of OSA,18 moderate to severe insomnia,19 and RLS.20 For SWD, we created a screening tool on the basis of The International Classification of Sleep Disorders-2 (ICSD-2) diagnostic criteria.21 This screening was conducted on paper in the EL and one TT departments and online in the OL and the second TT departments. Firefighters were informed of their sleep disorder risk either online (online questionnaire) or via a letter (paper questionnaire). In the EL departments, we referred at-risk firefighters to a sleep clinic near their location. In the TT and OL departments, at-risk firefighters were referred to a list of board-certified sleep specialists in their area provided by the American Academy of Sleep Medicine web site.
Surveys
To evaluate each firefighter's knowledge of the educational program content, anonymous assessments were administered to each firefighter before and following the educational training session. Assessments included eight multiple-choice or true/false questions that broadly covered the educational content. After approximately 1 year, firefighters were also asked to complete an end-of-program questionnaire, which asked them about their impressions of the SHP, to confirm that they were notified of their sleep disorder risk, and whether or not they sought further evaluation and treatment. Firefighters also rated the program using discrete 7-point Likert scales, with 1 anchored as “Not at all” and 7 anchored with “Very important or helpful.” All end-of-program surveys were completed online.
Confidentiality
In addition to institutional approval from the Partners Healthcare Research Committee, we obtained a Certificate of Confidentiality from the National Institutes of Health to ensure confidentiality of the sensitive data that were collected. Data from individual participants were assigned numerical identification numbers, which were used to link data records for each individual, and the remainder of the analysis record was de-identified. The name–code index was maintained in a secure location, in an encrypted file on a password-protected, IP-restricted computer system. Data were not shared with fire department administrators or union personnel.
Statistical Analysis
A Chi-square was used to compare participation rates, changes in assessment scores, firefighters’ rating of the program, and the rate of evaluation and treatment for firefighters identified as being at risk for a sleep disorder among program implementation methods. Further analysis using logistical regression, treating department as a random effect to account for inter-department variability under the same program implementation method, was also conducted. Alpha was set at 0.05.
Focus Groups
To evaluate the impact of the SHP further and to achieve a deeper understanding of the program's effectiveness from a firefighter's viewpoint, we conducted nine focus groups 4 to 15 months following the education sessions across three departments, each representing one of the three implementation approaches. Within each department, participants were divided into three separate focus groups: (1) firefighters who completed educational training and the sleep disorders screening questionnaire; (2) firefighters who completed educational training and did not complete the sleep disorders screening questionnaire; and (3) high ranking officers and fire department administrators, regardless of their participation. Focus group participants were given a short survey to assess their views of the SHP. The focus group lasted approximately 2 hours and the moderator followed a discussion guide to ensure consistency across groups and sessions. Each focus group was recorded by video and was notated using Verbatim Transcription. Using NVIVO 9.0 software (QSR International, Doncaster, Victoria, Australia), transcripts were analyzed to categorize comments on the basis of the core topics that emerged.22
RESULTS
Participation
Firefighters in the three implementation groups (Table 1) were comparable in self-reported demographics, health, and work-related characteristics (Table 2). A total of 6101 firefighters were eligible to participate in the SHP: 1630 in EL departments, 1519 in TT departments, and 2952 in OL departments. A total of 4603 firefighters (75.4%) attended the sleep health education. The participation rate was significantly higher in the EL (83.9%) and TT (88.5%) departments, than the OL departments (64.0%) (P < 0.0001, Table 1). The cooperation rate (ie, the number of firefighters who participated in the sleep disorders screening/the number of firefighters present for the educational training) was also significantly higher in the EL (73.2%) and TT (66.3%) departments than in the OL departments (39.7%) (P < 0.001, Table 1). The cooperation rate for the end-of-program questionnaire was 25.9% overall, and was significantly higher in the OL departments (43.0%) than in the EL (21.4%) and TT (16.7%) departments (P < 0.0001, Table 3).
TABLE 1.
Comparison of Departments Grouped by Program Administration Type
| Expert-Led | Train-the-Trainer | Online | Total | P | |
| Number of departments | 2 | 2 | 4 | 8 | |
| Number of stations | 38 | 87 | 119 | 244 | |
| Area covered (square miles) | 441.1 | 8780 | 1136 | 10,357.1 | |
| Number of firefighters | 1630 | 1519 | 2952 | 6101 | |
| Number of training sessions | 110 | 142 | — | — | |
| Training participation rate, n (%) | 1368 (83.9) | 1345 (88.5) | 1890 (64.0) | 4603 (75.4) | <0.0001 |
| Baseline survey participation rate*, n (%) | 1002 (61.5) | 892 (58.7) | 751 (25.4) | 2645 (43.4) | <0.0001 |
| Baseline survey cooperation rate*, n (%) | 1002 (73.2) | 892 (66.3) | 751 (39.7) | 2645 (57.5) | <0.0001 |
| End-of-program questionnaire cooperation rate*, n (%) | 214 (21.4) | 149 (16.7) | 323 (43.0) | 686 (25.9) | <0.0001 |
| Training knowledge assessment (n pre/n post-training) | 750/749 | 1252/1249 | 1381/910 | 3383/2908 | |
| Training, mandatory or with continuing education (CE) credit | Mandatory | Mandatory | Mixed | Mixed |
*Participation rate is the response rate that includes all potentially eligible participants. Cooperation rate is the response rate of participants that were contacted for a specific part of the study.
TABLE 2.
Self-Reported Participant Characteristics
| Characteristic | Expert-Led | “Train-the-Trainer” | Online | Total |
| N | 1002 | 892 | 751 | 2645 |
| Age, yrs | ||||
| Mean ± SD (range) | 42.9 ± 8.4 (22–1) | 38.0 ± 9.6 (20–5) | 40.8 ± 7.9 (21–2) | 40.6 ± 8.9 (20–1) |
| Sex, n (%) | ||||
| Women | 22 (2.2) | 65 (7.3) | 28 (3.7) | 115 (4.3) |
| Men | 955 (95.3) | 801 (89.8) | 705 (93.9) | 2461 (93.0) |
| Not known | 25 (2.5) | 26 (2.9) | 18 (2.4) | 69 (2.6) |
| Race, n (%)* | ||||
| White | 855 (85.3) | 687 (77.0) | 640 (85.2) | 2182 (82.5) |
| Black | 68 (6.8) | 56 (6.3) | 23 (3.1) | 147 (5.6) |
| Asian | 3 (0.3) | 9 (1.0) | 2 (0.3) | 14 (0.5) |
| Native American | 19 (1.9) | 17 (1.9) | 27 (3.6) | 63 (2.4) |
| Pacific | 3 (0.3) | 3 (0.3) | 3 (0.4) | 9 (0.3) |
| Other | 37 (3.7) | 82 (9.2) | 45 (6.0) | 164 (6.2) |
| Not known | 39 (3.9) | 59 (6.6) | 25 (3.3) | 123 (4.7) |
| Ethnicity, n (%) | ||||
| Hispanic | 60 (6.0) | 158 (17.7) | 137 (18.2) | 355 (13.4) |
| Other | 880 (87,8) | 633 (71.0) | 567 (75.5) | 2080 (78.6) |
| Not known | 62 (6.2) | 101 (11.3) | 47 (6.3) | 210 (7.9) |
| Body mass index (kg/m2), n (%) | ||||
| <25 | 144 (14.4) | 200 (22.8) | 108 (14.4) | 452 (17.1) |
| ≥25 and <30 | 522 (52.1) | 424 (48.2) | 373 (49.7) | 1319 (49.9) |
| ≥30 and <35 | 248 (24.8) | 200 (22.8) | 171 (22.8) | 619 (23.4) |
| ≥35 | 82 (8.2) | 55 (6.3) | 56 (7.5) | 193 (7.3) |
| Not known | 6 (0.6) | 13 (1.1) | 43 (5.7) | 62 (2.3) |
| Body mass index, kg/m2 | ||||
| Mean ± SD (range) | 28.8 ± 4.3 (15.8–55.2) | 28.1 ± 4.3 (16.7–48.8) | 28.7 ± 4.5 (17.6–53.2) | 28.5 ± 4.3 (15.8–55.2) |
| Health (subjective), n (%) | ||||
| Poor | 6 (0.6) | 3 (0.3) | 1 (0.1) | 10 (0.4) |
| Fair | 42 (4.2) | 26 (2.9) | 50 (6.7) | 118 (4.5) |
| Good | 398 (39.7) | 306 (34.3) | 274 (36.5) | 978 (37.0) |
| Very good | 418 (41.7) | 388 43.5) | 293 (39.0) | 1099 (41.6) |
| Excellent | 129 (12.9) | 162 (18.2) | 111 (14.8) | 402 (15.2) |
| Not known | 9 (0.9) | 7 (0.8) | 22 (2.9) | 38 (1.4) |
| Employed in the fire service, yrs | ||||
| Mean ± SD (range) | 15.6 ± 8.0 (0.3–40.4) | 11.3 ± 9.1 (0.1–38.0) | 14.4 ± 7.8 (1.0–38.2) | 13.9 ± 8.5 (0.1–40.4) |
| Scheduled shift duration, n (%) | ||||
| 8 | 122 (12.2) | 9 (1.0) | 20 (2.7) | 151 (5.7) |
| 24 | 759 (75.7) | 456 (51.1) | 584 (77.8) | 1799 (68.0) |
| 48 | 106 (10.6) | 341 (38.2) | 0 (0.0) | 447 (16.9) |
| Other: 12, 10/14, 72 | 10 (1.0) | 20 (2.2) | 33 (4.4) | 63 (2.4) |
| Unknown | 5 (0.5) | 66 (7.4) | 114 (15.2) | 185 (7.0) |
| Average scheduled hours per week Mean ± SD (range) | 50.0 ± 9.4 (3–144) | 58.2 ± 9.9 (40–127) | 60.5 ± 12.1 (2–127) | 55.4 ± 11.3 (2–144) |
| Average monthly work hours Mean ± SD (range) | 222.5 + 80.5 (24–648) | 253.0 + 90.6 (22–656) | 263.4 + 87.0 (30–608) | 242.7 + 87.4 (22–656) |
| Second job, n (%) | 430 (42.9) | 221 (24.8) | 286 (38.1) | 937 (35.4) |
*Respondents could select more than one race (57 reported more than one race).
TABLE 3.
End-of-Program Questionnaire Participation, Diagnosis, and Treatment Compliance
| Expert-Led | “Train-the-Trainer” | Online | Total | χ2 P | |
| Total participants in end-of-program questionnaire | 214 | 149 | 323 | 686 | — |
| Screened at high risk for a sleep disorder, n (%) | 87 (40.7) | 60 (40.3) | 138 (42.7) | 285 (41.5) | 0.84 |
| Reported in the end-of-program questionnaire that they screened at high risk for a sleep disorder, n (%) | 61 (70.1) | 37 (61.7) | 83 (60.1) | 181 (63.5) | 0.30 |
| Sought clinical evaluation as a result of the program, n (%)* | 36 (41.4*) | 15 (25.0*) | 27 (19.6*) | 78 (27.4*) | 0.0015 |
| Recommended treatment, n (%)† | 30 (83.3) | 8 (53.3) | 17 (63.0*) | 55 (70.5*) | 0.057 |
| Compliant with treatment, n (%)‡ | 23 (76.7) | 7 (87.5) | 14 (82.4*) | 44 (80.0*) | 0.81 |
*Number of subjects who screened at a high risk for a sleep disorder used as a denominator.
†Number who sought clinical evaluation as a result of the program used as a denominator.
‡Number recommended for treatment used as a denominator.
Knowledge Assessment
Overall, firefighters showed a 28.6% improvement in knowledge assessment scores after training. Firefighters in the EL departments showed the most improvement (34.0%), followed by TT (28.5%) and OL (24.2%) departments. There was a significant improvement in the knowledge gained by firefighters in the EL departments, when compared separately with the firefighters in the OL departments (posthoc P = 0.03) but when adjusted for multiple comparisons, only a trend was shown (P = 0.10).
Sleep Disorders Screening
Overall, 40.8% (1079/2645) of firefighters screened at risk for at least one sleep disorder. The risk of having a sleep disorder was consistent among methodology groups (Chi-square, P = 0.84; logistic model with fire department as a random effect, P = 0.91). Of those who completed the end-of-program questionnaire, 41.5% (285/686) had screened at risk for a sleep disorder. Of those who had screened positive and were notified that they were at risk for a sleep disorder, only 63.5% (181/285) answered affirmatively when queried as to whether they had been notified that they were at risk. The firefighters who screened positive in the EL departments reported a significantly higher notification rate (70.1%) than those firefighters in the TT (61.7%) and OL (60.1%) departments (P = 0.0015, Table 3).
Of the 285 firefighters who screened positive for a sleep disorder and completed the end-of-program questionnaire, 78 (27.4%) reported that they sought a clinical evaluation as a result of the program. Firefighters in EL departments were significantly more likely to seek a clinical evaluation (36/87; 41.4%) than the firefighters in TT (15/60; 25%) and OL (27/138; 19.6%) departments (P < 0.0015, Table 3). There was no difference in treatment compliance among groups (Table 3).
Program Impact
On the end-of-program questionnaire, 563 of 686 (82.1%) firefighters reported participating in the SHP. More than half (52.0%, 293/563) of firefighters rated the importance of the SHP as 5 or higher out of 7 and there was no statistical significant difference between EL, TT, and OL departments (P = 0.87). Overall, 42.6% (240/563) rated the program 5 or higher for “useful information” and similarly between the programs (P = 0.61), and 50.3% (283/563) rated it 5 or higher for “recommend to other fire departments” with no differences between the groups (P = 0.67).
Focus Groups
A total of 62 firefighters participated in the nine focus groups (three to nine participants in each) and completed a short survey to assess their views of the SHP (not all firefighters answered every survey question). Overall, 42% (24/57) of the firefighters reported changing their sleep behavior as a result of the program (31.6, 57.9, and 36.8% for EL, TT, and OL, respectively, P = 0.22) and 29% (17/58) of firefighters reported that the program had a positive impact on their life (Fig. 1) (29.4%, 35.3%, and 35.3%, for EL, TT, and OL, respectively, P = 0.94).
FIGURE 1.
Focus group participants reported, in an anonymous survey, the effects of the fatigue risk management program and specific behavioral changes. There was no difference among groups in the percentage of firefighters reporting behavior change.
In the focus group discussion, firefighters reported that the SHP was effective for all methods of implementation. When all implementation methods were described, the majority of firefighters thought that the EL format seemed the best for an occupational fatigue risk management program for firefighters. When asked to assess additional considerations such as program cost, logistics of scheduling, and confidentiality, however, it was difficult to find consensus as to which implementation method was best. Most firefighters agreed that individual departments would need to assess their own needs and assets before determining the best program implementation to employ. A list of the pros and cons of each implementation method was created on the basis of the focus group discussion (Table 4). Four major themes emerged from the content analysis: communication/confidentiality; awareness of fatigue and sleep-related topics; behavioral and lifestyle changes; and firefighter culture and the challenges it creates for an SHP.
TABLE 4.
Focus Groups: Pros and Cons of Program Administration Type
| Expert-Led | Train-the-Trainer | Online | |
| Pros | Professionalism | Peers can be trusted | Inexpensive |
| Material is solid and accurate | Less expensive than Expert-led | Easy to distribute | |
| Participant engagement is more likely | Trainers are in place | No scheduling necessary | |
| Individual questions possible | Individual questions possible | Wider Reach | |
| No need for technology or software | No need for technology or software | Participant can stay on duty | |
| Procedures are closely followed | Wider reach than Expert-led | Consistent messaging | |
| Personal approach | Personal approach | Immediate feedback from screening | |
| Cons | Expensive | More expensive than Online | Easy for participants not to engage |
| May be unfamiliar with the population | Participants answers may be less honest | Lower participation rates | |
| Smaller Reach | Paperwork can be lost | Impersonal | |
| Scheduling can be difficult | Not experts in material | No ability to ask individualized questions | |
| Takes participants out of service | Accuracy of training questioned | Technology required | |
| Outsiders mistrusted | Less respected than experts | Technical problems | |
| Fear of Intent | Confidentiality a concern | Increased preparation time required | |
| Topic dependent |
Communication/Confidentiality
In focus groups, firefighters expressed a great deal of initial skepticism and even some fear regarding occupational health programs in general. One expressed mixed feelings, saying, “I think there are some people in the department who don’t like the idea that their job and their profession are involved in their personal and medical information and then there are others who think it's a good, proactive way to stay on top of your health.” Firefighters stated that transparent and effective communication about the purpose and content of the program was extremely important. Firefighters recommended that union groups should communicate information about upcoming programs in-person at meetings and ensure adequate dissemination of information about the program. Having a local champion advocating for the program was seen as positive. Firefighters in the focus groups believed that the educational portion of the program was essential and it increased their interest in participating in the sleep disorders screening survey.
Despite the measures we took to ensure confidentiality, some firefighters also stated that they were concerned about it and how any data collected during the program could negatively impact their work or social life. Apprehensions mentioned by the firefighters included the potential for new work hours guidelines, lost work time or punitive effects on the job, the cost of sleep disorder treatment, and their ability to obtain life insurance if found to have a sleep disorder.
Most firefighters stated that confidentiality was adequately addressed and maintained during this SHP and their fears of providing sensitive and personal information were significantly reduced as a result. This opinion was especially true in EL department who completed the paper survey. Although data were not shared with fire departments, firefighters in one of the TT departments mentioned concerns about providing confidential information to their peers or entering such information on departmental computers. Some firefighters who did not participate in the sleep disorders screening said that they might have participated had the screening been anonymous rather than confidential.
Awareness of Fatigue- and Sleep-Related Topics
The majority of firefighters agreed that the most important outcomes of the SHP included an increased awareness of fatigue-related issues among firefighters, the opportunity to receive sleep disorder treatment, and information to help improve habits that impact sleep on and off the job. There was a small minority that disagreed with these observations, although these were typically individuals who opted out of completing the sleep disorders screening survey.
Firefighters stated that, following the program, they witnessed an impact on their own awareness as well as their colleagues regarding sleep and fatigue topics. Individuals raised specific elements discussed in the educational training, such as the dangers of fatigue, the negative health consequences of sleep deficiency, as well as demonstrating an increased understanding of sleep disorders. Others mentioned positive cultural effects on raising awareness around sleep, including facilitating discussions of the problems associated with sleep disorders, such that the stigma associated with having or being treated for a sleep disorder was diminished. One firefighter stated, “I heard of a lot of people having sleep studies. I think if you hadn’t come to town half these people wouldn’t have [had] sleep studies” and another commented “There's an engineer [who] used to be asleep, it seemed like the entire shift, because he was just so out of it. He got the CPAP and he's just awake all day long.”
Almost unanimously, firefighters expressed the opinion that the education and screening program should continue to be offered by the Federal Emergency Management Agency (who provided grant support for this work) for all fire departments and should be implemented in other industries that schedule shift work. Many believed that the results of this project would help to increase participation in future SHPs and some firefighters stated that this program could initiate a cultural change on the topic of sleep in the fire service.
Behavioral and Lifestyle Changes
Many firefighters in the focus groups commented that the SHP had a substantial impact on the behavior of firefighters in their department (43.5% EL, 36.8% TT, and 46.7% OL reported behavioral changes, P = 0.83) and that they believed that some of these changes may have a lasting effect. Some firefighters stated that they had adopted healthier habits and others reported that they were formally diagnosed and treated for sleep disorders. Although most changes noted were on the individual level (ie, “I take naps at the station now after this [program] simply because the department supports this;” “Knowing now that it is much more important, sleep is precious to me.”), some were noted at the departmental level (ie, “Station-wise, it has [changed]. The guys will do half caffeine per coffee. We take blood pressures once a week after fires. People have started to monitor themselves a little bit more.”).
Firefighters reported that sleep, sleep disorders, and fatigue were not common discussion topics before the implementation of the SHP. Many firefighters believed that the training session increased the likelihood that firefighters would seek medical advice about a sleep disorder concern. Some firefighters who received notification that they had a high risk of a sleep disorder did not seek further evaluation. Those firefighters reported that the cost, time, effort, fear of the unknown, spousal pressure, peer pressure, fear of punitive damages, aversion to lifelong treatment with continuous positive airway pressure (CPAP), and the impact on their ability to work were barriers to seeking treatment.
Firefighter Culture and Challenges
There are unique aspects to the firefighter culture that are important to consider in creating an SHP. Firefighters in the focus groups emphasized this uniqueness citing the close proximity of sleeping quarters and sharing of bunk rooms often led to coworkers being able to observe and impact each other's sleep quality and quantity directly. For example, if one firefighter snores loudly, he may disrupt the sleep of many other firefighters in the station (“Can you believe he's keeping us all awake? ... the guys who may not necessarily have the sleep problem are up because of the guys that have the sleep problem.”).
Firefighters also stated that their training schedules were extremely full and many felt overburdened at times by this. Thus, firefighters were sensitive to the addition of new training requirements such as the educational portion of this SHP. Nonetheless, given that optional trainings are often disregarded, many firefighters believed that the sleep health training would need to be made mandatory to be effective.
DISCUSSION
Firefighters must provide 24/7 coverage every day of the year. Although they traditionally work in 24-hour shifts,1 it is imperative that firefighters remain alert in order to conduct their work safely and effectively. Moreover, being predominantly middle-aged shiftworkers, firefighters have an increased risk of sleep disorders. We therefore developed and evaluated an SHP to educate firefighters on sleep health and screened for common sleep disorders. In order to inform the best practice for implementation of the program, we evaluated three different implementation methods.
Regardless of whether the program was conducted with EL, TT, or OL methodology, firefighters reported that the SHP was important, provided helpful information, and led to positive behavioral changes. A greater percentage of firefighters participated in the EL and TT departments. Firefighters increased their knowledge of sleep health in all departments, with EL departments showing the largest gains. Firefighters in EL departments also sought clinical follow-up and had higher rates of treatment compliance more than the TT and OL departments.
Education on healthy sleep was an important aspect of the SHP, as it engaged firefighters in the topic and promoted participation in the sleep disorders screening. In the focus groups, many firefighters reported that they had little or no interest in undergoing screening until they participated in an educational training session. Although the use of online learning is becoming increasingly popular, in this SHP, an in-person presentation from a sleep health expert or trained firefighter elicited more participation in the sleep disorders screening than did the online education. Firefighters who participated in the EL education also had a greater increase in knowledge, though we recognize that online learning can be more effective than face-to-face delivery in other settings.23 Our online educational module may have been strengthened by increased interaction with the user, a tactic that has been shown to enhance learning.23 In volunteer and career firefighters, an interactive, multimedia training tool improved knowledge transfer and retention compared with traditional classroom learning.24 Heightened interactive learning may convince more firefighters to be screened for common sleep disorders.
An important end-point of the SHP is for firefighters who are at a high risk for a sleep disorder to seek further evaluation, diagnosis, and treatment. Firefighters in the EL departments were twice as likely to seek evaluation as those in the OL departments. This difference might be accounted for by the closer working relationship that was established in the EL departments and more personal referrals to a sleep disorders clinic with which we had established a connection. In one EL department, investigators provided additional phone calls and reminders about clinic referrals and this effort likely led to the increased clinic follow-up in that group. Interestingly, although not significant (P = 0.057), treatment recommendation was higher in the EL departments. Again, this is likely due to the physicians in the clinic being familiar with the SHP in contrast to the less interactive process of having firefighters print out a letter explaining their risk to take to a clinic, as was recommended to firefighters in the OL departments.
Communication regarding the risk of a sleep disorder may need to be improved. Although 41.5% of firefighters who took the end-of-program questionnaire had been notified that they were at a high risk of having a sleep disorder, when asked “Did you receive notification that you were at high risk for any sleep disorder (s)?”, only 26.4% answered affirmatively. Thus, we did not communicate effectively to approximately 36% (104/285) of the at-risk firefighters, despite our extensive efforts. Alternatively, the inconsistency in the notification of high risk and the response from the participant in knowing he or she was at high risk may be due to forgetfulness in the intervening year or the discounting of the information as not something critical to remember. Our message, including the seriousness of the level of risk and the adverse health and safety consequences of untreated sleep disorders, needed to be more clearly conveyed. Messaging should be written in a language that is common to firefighters and it should not only inform but also motivate those at high risk to seek appropriate evaluation, diagnosis, and treatment, if necessary. We convened a firefighter research advisory committee, made up of firefighter administration, union, line firefighters, and a fire department physician to assist in the development of the SHP. Using such a group to craft a sleep disorders risk notification that better resonates with firefighters might improve future outcomes.
Senior leaders’ enthusiasm and support of the union members were key to the success of the SHP. Memoranda were signed by each department before the study to demonstrate their commitment to the funding agency and most made the training (but not survey completion) mandatory. It may be possible for some departments to offer continuing education credits to further encourage participation. Firefighters in the focus groups emphasized the importance of having a champion within the department advocate for a novel program such as the SHP. A champion can informally build trust amongst peers, emphasize the importance of the program, alleviate fear, and guide overall perceptions. The ability to have questions about the SHP answered by investigators is also important. Even though there was a “Frequently Asked Questions” page available in the online program, perhaps the ability to directly ask and have answered specific questions about the program increased the participation in the EL and TT departments.
Interestingly, there was a reversal in participation on the end-of-program questionnaire with participation rates in OL departments twice that of EL and TT departments. This incongruity might be due to the fact that firefighters in OL departments were accustomed to participating in the SHP online, whereas most firefighters in EL and TT departments completed the screening survey on paper and were switched to a Web-based format at the end of the program. Consistency in delivery of the program may be an important consideration to prevent attrition. Alternatively, those firefighters in OL departments who completed the online screening survey may have been more self-motivated than those who were encouraged to complete the screening survey by in-person experts or trainers, and thus were willing to continue to complete online surveys. Further, technical issues may have also played a role in EL and TT departments (eg, correct e-mail addresses, retrieving lost passwords, and Web site time-outs), whereas technology was proven at the start of the study in OL departments. Finally, the more intensive interactions in the initial training, both for EL and TT, may have used up the participants’ capacity for this topic and they may have felt overburdened and consequently less responsive when approached a second time. Perhaps future programs should consider this balance when designing an SHP and plan to collect any required information on first contact with participants.
Challenges and Limitations
Implementing an SHP in operational fire departments presents operational implementation challenges and consequent research limitations. Although we selected “interested” fire departments and required signed memorandums of understanding for inclusion in the study, the resultant level of interest was not always easy to predict. In the course of implementing a program in an operational field setting, there may be changes in administrative personnel, climate, and workload. In addition, assignment of the fire departments to one of the three implementation methods based on resource availability represents an experimental design limitation; however, our approach was aimed at evaluating reach and effectiveness and was reflective of how future SHPs will be implemented.
It is challenging to schedule all firefighters in any department for training, as they may be called away due to operational duties and the different methodologies required different policies. During the educational training in EL and TT departments, firefighters were on-duty but off-service to ensure they could complete the in-person training. Firefighters participating in OL departments were typically on-service, and could thus be interrupted by a fire call while viewing the education or completing the screening. Those firefighters could return to the viewing or screening, but the extra step or potential difficulties logging back into the system likely led to lower cooperation rates.
In addition, despite our discouragement, some departments allowed for group viewing of the online educational video, which made tracking of individual participation more difficult. Group viewing of the educational training prevented firefighters from having the chance to immediately complete the screening survey, as individuals were not in front of an individual computer. This restriction may have reduced the participation in the screening survey in OL departments. Group viewing of the educational video may have affected attention of the firefighter, either positively or negatively.25 Only eight simple questions on the before and after knowledge assessments were used in the analysis of learning. To achieve higher sensitivity and test the practical integration of the sleep health information, further assessments should use a greater number of questions and more difficult questions including firefighter-based scenario questions.
The benefits and challenges that we experienced in the implementation of a SHP in operational fire departments are similar to those seen in other industries. Nurses, who like firefighters work demanding schedules providing 24/7 coverage, identified greater awareness of sleep and fatigue issues and feeling more alert as benefits of an SHP.15 Following the implementation of the nursing SHP, sleep (duration and quality), alertness, and error prevention were improved.16 Confidentiality, changing the culture of the organization to embrace napping on-duty, and support of management for the program were challenges faced by hospital staff nurses.15
Screening for sleep disorders as part of an SHP introduces confidentiality concerns and potential anxiety, as untreated sleep disorders can impact employability in some occupations.26 However, screening for common sleep disorders has been successfully accomplished in a large group of North American police officers,5 and an occupational specific educational Web site with anonymous online sleep disorders screening was established for railroad workers.13 The goal of any screening program is to achieve a high rate of those at risk to seek evaluation, clinical diagnosis, and treatment of sleep disorders. That was not achieved in any of the implementation modes of the firefighter SHP. More research is needed to understand how best to remove barriers to evaluation, clinical diagnosis, and treatment such as utilizing home-based diagnostic tools.
The existence and persistence of untreated sleep disorders increases the risk of crashes and injuries, adversely affects mental and physical health, including an increased risk of chronic diseases, including cardiovascular disease, obesity and diabetes, and decreases workplace productivity and quality of life.3–8,27 It is therefore imperative to implement SHP in safety-sensitive occupations, especially in those with nonstandard work shifts, as chronic sleep disorders may affect 60 to 80% of all shift workers.11
CONCLUSION
The overall goal of this research was to compare the effectiveness of three different methods of implementation of a SHP. The program was well-received with three-quarters of firefighters participating. Firefighters in the EL departments gained the most knowledge and were most likely to seek further evaluation, diagnosis, and treatment if screened at a high risk for a sleep disorder. Focus groups identified communication, confidentiality, and firefighter culture as key challenges to implementation of SHPs and highlighted the fact that there is not a “one size fits all” program. Each department needs to decide which implementation method is best suited for conducting an SHP in order to successfully improve the health and safety of their employees.
Acknowledgments
The authors acknowledge contributions from KC Root, Amy Hallal, Sean Benedix, Michael Shreeve, and Rick Lilienthal (study coordination), Lauren Brogna, Shekeria Beale, Lily Anton and Tiffany Wu (research trainees), Kathleen Kennedy (senior finance manager) and Bonnie Wheeler (division administrator) from the Division of Sleep and Circadian Disorders, Departments of Medicine and Neurology, Brigham and Women's Hospital. The authors are grateful to all of the fire departments and firefighters who took part in this research program and are especially grateful to the firefighters who volunteered to be videoed as part of our training program. Thank you also to the International Association of Fire Chiefs Safety, Health and Survival Section for providing feedback on the baseline health screening survey questions and to David Whiting, Mike Wajda, Bill Ehrgood, Barbara Salas, and Paul Parrish, who served as members of the Harvard Firefighter Research Advisory Committee.
Footnotes
Laura K. Barger and Conor S. O’Brien are co–first authors.
This project was supported by the Federal Emergency Management Agency, Assistance to Firefighters Grant Program, EMW-2007-FP-02917 and EMW-2008-FP-02566. The contents of this manuscript are solely the responsibility of the authors and do not necessarily represent the official views of the Federal Emergency Management Agency or the Assistance to Firefighters Grant Program.
All authors report that they were supported in conducting this work by a research grant from the Assistance to Firefighters Grant Program through the Brigham and Women's Hospital and were supported in part by grant R01 OH 010300 from the National Institute of Occupational Safety and Health (LKB, CSO, SQ, JPS, WW, CAC, and SWL) and grant U01HL11478 from the National Heart Lung Blood Institute NHLBI (CAC and LKB).
Dr. Barger has previously received research support from Cephalon, NFL charities, Sysco, and San Francisco Bar Pilots. She has received consulting/lecture fees or served as a board member for Alertness Solution, Ceridian, Davis Joint Unified School Board, San Jose State University Foundation, Puget Sound Pilots, Sygma, and Torvec.
Mr. O’Brien reports no conflicts.
Dr Rajaratnam reports that he has served as a consultant through his institution to Vanda Pharmaceuticals, Philips Respironics, EdanSafe, National Transport Commission, Rail, Bus and Train Union, Australian Workers’ Union, Tontine Group, Transport Accident Commission, Media Consumer Healthcare, New South Wales Department of Education & Communities, and has through his institution received research grants and from Vanda Pharmaceuticals, Philips Respironics, and Rio Tinto, and reimbursements for conference travel expenses from Vanda Pharmaceuticals. He serves as a consultant to, and is a Program Leader for, the Cooperative Research Centre for Alertness, Safety and Productivity. His institution has received equipment donations or other support from Optalert, Compumedics, Philips Lighting, and Tyco Healthcare. He is a Director of the Sleep Health Foundation, and is a Past President of the Australasian Sleep Association. He has also served as an expert witness and/or consultant to shift work organizations and in legal cases involving sleep loss.
Mr Sullivan, Mr Qadri, Dr Wang report no conflicts.
Dr Czeisler has received consulting fees from or served as a paid member of scientific advisory boards for Amazon.com, Inc.; A2Z Development Center, Inc.; Bose Corporation; Boston Red Sox; Cleveland Browns; Institute of Digital Media and Child Development; Jazz Pharmaceuticals, Inc.; Merck Sharpe and Dohme; Purdue Pharma; Quest Diagnostics; Samsung Electronics; Teva Pharmaceutical Industries Ltd.; Koninklijke Philips Electronics, N.V.; Novartis; and Vanda Pharmaceuticals, Inc. He owns an equity interest in Somnus Therapeutics, Inc. and Vanda Pharmaceuticals, Inc., and has received research/education support from Mary Ann and Stanley Snider through Combined Jewish Philanthropies, National Football League Charities, Optum, ResMed, Philips Respironics, the San Francisco Bar Pilots, Simmons, Schneider, Inc., and Sysco. He has received royalties from McGraw Hill, Penguin Press/Houghton Mifflin Harcourt, and from Philips Respironics, Inc., for the Actiwatch 2 and Actiwatch Spectrum devices; his interests were reviewed and are managed by Brigham & Women's Hospital and Partners HealthCare in accordance with their conflict of interest policies. The Harvard Medical School Division of Sleep Medicine and the Brigham and Women's Hospital Division of Sleep and Circadian Disorders, which he directs, have received unrestricted research and educational gifts and endowment funds from Boehringer Ingelheim Pharmaceuticals, Inc.; Cephalon, Inc.; GlaxoSmithKline; Jazz Pharmaceuticals; Merck & Co., Inc.; Pfizer; ResMed; Philips Respironics, Inc.; Sanofi-Aventis, Inc.; Sealy, Inc.; Sepracor, Inc.; Simmons, Spring Aire; Takeda Pharmaceuticals; Tempur-Pedic; Walmart; Proctor and Gamble; Optum; Bose; Vanda Pharmaceuticals, Inc.; and has received Educational Grant funding from Apria Healthcare; Cephalon, Inc.; Jazz Pharmaceuticals; Philips Respironics; Takeda Pharmaceuticals; ResMed Foundation; Sanofi-Aventis, Inc.; Sepracor, Inc.; Teva Pharmaceutical Industries, Ltd.; and Wake Up Narcolepsy. He holds a number of process patent s in the field of sleep/circadian rhythms (eg, photic resetting of the human circadian pacemaker). Since 1985, he has also served as an expert witness on various legal cases related to sleep and circadian rhythms, including matters related to Bombardier, Citgo, HG Energy, Michael Jackson's mother and children, Purdue Pharma, StricLan LLC, Valero, and matters related to commercial drivers employed by Celadon, Crete Carrier Corporation, FedEx, United Parcel Service, and other commercial carriers.
In the past 3 years, Dr Lockley has received consulting fees from Blackrock; Carbon Limiting Technologies Ltd (on behalf of PhotonStar LED); Cowen & Co; Endurant Capital Management; Far West Capital Management; Fidelity; Frankel Group; Impax Laboratories; Kearney Venture Partners; Lazard Capital Markets; Naturebright; New Horizon Capital; Perceptive Advisors; Polar Capital; ResearchWorks Inc.; Serrado Capital; Wyvern Funds; and has current consulting contracts with Akili Interactive; Delos Living LLC; Environmental Light Sciences, LLC; Focal Point LLC; Headwaters Inc.; Hintsa Performance AG; OpTerra Energy Services Inc.; Pegasus Capital Advisors LP; PlanLED; Wyle Integrated Science and Engineering; has received unrestricted equipment gifts from Bioilluminations LLC; Bionetics Corporation; and Philips Lighting; advance author payment and royalties from Oxford University Press; payment for editing a textbook section from Elsevier; honoraria from the National Sleep Foundation; and for an article in the Wall Street Journal; honoraria plus travel, accommodation or meals for invited seminars, conference presentations or teaching from Brookline Adult Education; Brown University; Estee Lauder; Harvard University (CME); MediCom Worldwide, Inc.(CME); travel, accommodation and/or meals only (no honoraria) for invited seminars, conference presentations or teaching from 14th Annual Tennessee Perfusion Conference; American Society for Photobiology; Cantifix; Connecticut Business & Industry Association Health and Safety Conference; Emergency Services Steering Committee; FASEB; Harvard University; Hintsa Performance AG; Illuminating Engineering Society; Lightfair; Massachusetts General Hospital; Midwest Lighting Institute; New England College of Occupational and Environmental Medicine; Ontario Association of Fire Chiefs; Rio Tinto; UMass Memorial; University of Manchester; ongoing investigator-initiated research grants from Biological Illuminations LLC; and Respironics Inc.; completed service agreements with Rio Tinto Iron Ore and Vanda Pharmaceuticals Inc.; three completed sponsored initiated clinical research contracts with Vanda Pharmaceuticals Inc.; and one completed investigator-initiated research grant from Vanda Pharmaceuticals Inc. Dr Lockley also holds a process patent for the use of short-wavelength light for resetting the human circadian pacemaker and improving alertness and performance, which is assigned to the Brigham and Women's Hospital per Hospital policy. Dr Lockley has also served as a paid expert witness on behalf of four public bodies in cases related to sleep, circadian rhythms, and work hours. Dr Lockley also serves as a Program Leader in the Cooperative Research Centre for Alertness, Safety and Productivity.
REFERENCES
- 1.Elliot DL, Kuehl KS. The Effects of Sleep Deprivation on Fire fighters and EMS Responders. Final Report. Fairfax, VA: International Association of Fire Chiefs; 2007. [Google Scholar]
- 2.Jahnke S, Poston WS, Jitnarin N, Haddock CK. Health concerns of the U.S. fire service: perspectives from the firehouse. Am J Health Promot 2012; 27:111–118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Vargas de Barros V, Martins LF, Saitz R, Bastos RR, Ronzani TM. Mental health conditions, individual and job characteristics and sleep disturbances among firefighters. J Health Psychol 2013; 18:350–358. [DOI] [PubMed] [Google Scholar]
- 4.Barger LK, Rajaratnam SM, Wang W, et al. Harvard Work Hours Health and Safety Group. Common sleep disorders increase risk of motor vehicle crashes and adverse health outcomes in firefighters. J Clin Sleep Med 2015; 11:233–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rajaratnam SM, Barger LK, Lockley SW, et al. Harvard Work Hours, Health and Safety Group. Sleep disorders, health, and safety in police officers. JAMA 2011; 306:2567–2578. [DOI] [PubMed] [Google Scholar]
- 6.Czeisler CA. Duration, timing and quality of sleep are each vital for health, performance and safety. Sleep Health 2015; 1:5–8. [DOI] [PubMed] [Google Scholar]
- 7.Dobson M, Choi B, Schnall PL, et al. Exploring occupational and health behavioral causes of firefighter obesity: a qualitative study. Am J Ind Med 2013; 56:776–790. [DOI] [PubMed] [Google Scholar]
- 8.Shahly V, Berglund PA, Coulouvrat C, et al. The associations of insomnia with costly workplace accidents and errors: results from the America Insomnia Survey. Arch Gen Psychiatry 2012; 69:1054–1063. [DOI] [PubMed] [Google Scholar]
- 9.Centers for Disease Control and Prevention. Fatalities among volunteer and career firefighters–United States, 1994–2004. JAMA 2006; 295:2594–2596. [Google Scholar]
- 10.National Commission on Sleep Disorders Research. Wake up America: A National Sleep Alert. 1–18. Palo Alto, CA: Stanford University Sleep Disorders Clinic and Research Center; 1992. [Google Scholar]
- 11.Leger D. The cost of sleep-related accidents: a report for the national commission on sleep disorders research. Sleep 1994; 17:84–93. [DOI] [PubMed] [Google Scholar]
- 12.Federal Aviation Administration. Flightcrew Member Duty and Rest Requirements. Code of Federal Regulations. 14 CFR Part 117, 119 and 121.Vol. 77, No 2, January 4, 2012. [Google Scholar]
- 13.Barger L, Quan S. Brief review: sleep health and safety for transportation workers. Southwest J Pulmon Crit Care 2015; 10:130–139. [Google Scholar]
- 14.Meuleners L, Fraser ML, Govorko MH, Stevenson MR. Obstructive sleep apnea, health-related factors, and long distance heavy vehicle crashes in western australia: a case control study. J Clin Sleep Med 2015; 11:413–418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Scott LD, Hofmeister N, Rogness N, Rogers AE. Implementing a fatigue countermeasures program for nurses: a focus group analysis. J Nurs Admin 2010; 40:233–240. [DOI] [PubMed] [Google Scholar]
- 16.Scott LD, Hofmeister N, Rogness N, Rogers AE. An interventional approach for patient and nurse safety: a fatigue countermeasures feasibility study. Nurs Res 2010; 59:250–258. [DOI] [PubMed] [Google Scholar]
- 17.Czeisler CA. Role of sleep medicine and chronobiology for optimizing productivity, safety and health in the workplace. 86th Annual Meeting, Japan Society for Occupational Health; May 16, 2013; Matsuyama, Ehinme, Japan. [Google Scholar]
- 18.Netzer NC, Stoohs RA, Netzer CM, Clark K, Strohl KP. Using the Berlin Questionnaire to identify patients at risk for the sleep apnea syndrome. Ann Intern Med 1999; 131:485–491. [DOI] [PubMed] [Google Scholar]
- 19.Soldatos CR, Dikeos DG, Paparrigopoulos TJ. The diagnostic validity of the Athens Insomnia Scale. J Psychosom Res 2003; 55:263–267. [DOI] [PubMed] [Google Scholar]
- 20.Allen RP, Walters AS, Montplaisir J, et al. Restless legs syndrome prevalence and impact: REST general population study. Arch Intern Med 2005; 165:1286–1292. [DOI] [PubMed] [Google Scholar]
- 21.American Academy of Sleep Medicine. The International Classification of Sleep Disorders; Diagnostic and Coding Manual. Vol. Second Edition. Westchester, IL: American Academy of Sleep Medicine; 2005. [Google Scholar]
- 22.Straus A. Qualitative Analysis for Social Scientists. Cambridge, UK: Cambridge University Press; 1987. [Google Scholar]
- 23.Means B, Toyama Y, Murphy R, Bakia M, Jones K. Evaluation of Evidence-based Practices in Online Learning: a Meta-analysis and Review of Online Learning Studies. Washington, DC: Center for Technology in Learning US Department of Education; 2009. [Google Scholar]
- 24.Wener R, Panindre P, Kumar S, et al. Assessment of web-based interactive game system methodology for dissemination and diffusion to improve firefighter safety and wellness. Fire Saf J 2015; 72:59–67. [Google Scholar]
- 25.Cadiz J, Balachandran A, Sanocki E, Gupta A, Grudin J, Jancke G. Distance learning through distributed collaborative video viewing. CSCW ’00 Proceedings of the 2000 ACM Conference of Computer Supported Cooperative Work held in Philadelphia, PA, December 2–6, 2000. New York: Association of Computing Machinery; 2000. [Google Scholar]
- 26.Olszewski K, Wolf D. Obstructive sleep apnea among commercial motor vehicle drivers: using evidence-based practice to identify risk factors. Workplace Health Saf 2013; 61:479–485. [DOI] [PubMed] [Google Scholar]
- 27.Kessler RC, Berglund PA, Coulouvrat C, et al. Insomnia and the performance of US workers: results from the America insomnia survey. Sleep 2011; 34:1161–1171. [DOI] [PMC free article] [PubMed] [Google Scholar]