Abstract
Objectives
The COVID-19 pandemic placed health care personnel (HCP) at risk for stress, anxiety, burnout, and post-traumatic stress disorder (PTSD). To address this, hospitals developed programs to mitigate risk. The objectives of the current study were to measure the availability and use of these programs in a cohort of academic emergency departments (EDs) in the United States early in the pandemic and identify factors associated with program use.
Methods
Cross-sectional survey of ED HCP in 21 academic EDs in 15 states between June and September 2020. Site investigators provided data on the availability of 28 programs grouped into 9 categories. Individual support programs included: financial, workload mitigation, individual COVID-19 testing, emotional (e.g., mental health hotline), and instrumental (e.g., childcare) Clinical work support programs included: COVID-19 team communication (e.g., debriefing critical incident), patient-family communication facilitation, patient services (e.g., social work, ethics consultation), and system-level exposure reduction. Participants provided corresponding data on whether they used the programs. We used generalized linear mixed models clustered on site to measure the association between demographic and facility characteristics and program use.
Results
We received 1,541 survey responses (96% response rate) from emergency physicians or advanced practice providers, nurses, and nonclinical staff. Program availability in each of the 9 categories was high (>95% of hospitals). Program use was variable, with clinical work support programs used more frequently (28–50% of eligible HCP across categories) than individual employee support programs (6–13% of eligible HCP across categories). Fifty-seven percent of respondents reported that the COVID-19 pandemic had affected their stress and anxiety, and 12% were at elevated risk for PTSD. Program use did not significantly differ for HCP who reported symptoms of anxiety and/or stress compared to those who did not.
Conclusions
Early in the pandemic, support programs were widely available to ED HCP, but program use was low. Future work will focus on identifying barriers and facilitators to use and specific programs most likely to be effective during periods of highest occupational stress.
Introduction
The coronavirus disease 2019 (COVID-19) pandemic created challenging work conditions across the healthcare system. Healthcare personnel (HCP) have been faced with uncertainty about resources, staffing, risk of SARS-CoV-2 exposure, concern for the well-being of coworkers and family, and providing clinical care amid visitor restrictions. Many studies have documented increases in anxiety, depression, stress disorders, and burnout among HCP since the start of the COVID-19 pandemic [1–8]. In addition to causing significant individual distress, these symptoms are associated with a higher likelihood of leaving the field of medicine [9]. Data from a public health workforce survey in 2021 suggest that just under half (44%) of workers reported that they were considering leaving their jobs within the next 5 years for retirement or other reasons, with the majority (76%) having begun to think about leaving since the start of the COVID-19 pandemic [10]. This is an alarming trend for maintaining health system capacity in the US, and information about how to address stress and burnout is urgently needed. Frontline HCP, particularly those working in emergency departments (EDs) and critical care units, have long been recognized to be at increased risk for stress and burnout and this risk was further exacerbated by the pandemic [11–15]. We previously reported prevalent symptoms of anxiety and burnout across the spectrum of ED staff during the initial wave of the pandemic and here examine support program availability and use in this at-risk population [16, 17].
Support programs for frontline HCP can mitigate the negative effects of the pandemic on HCP well-being and preserve the ability of frontline HCP to provide medical care during this public health emergency [18–26]. Research suggests that programs targeting both individual (HCP-targeted) and structural (workflow-targeted) factors are important [27]. Several types of programs have been evaluated, including education, peer support, mental health treatment, and changes to clinical workflow, with most have being evaluated in single-site studies [28–32]. Data on how hospitals implemented support programs and how they were used in the context of the COVID-19 pandemic has been lacking, however [15]. Studying the COVID-19 experience is important to understand the long-term impact of COVID-19 on health care careers, evaluate the effectiveness of the COVID-19 health system response, and maintain readiness for future public health crises.
The goals of this study were to describe the availability and use of institutional support programs targeting both individual ED staff and clinical workflow during the early COVID-19 pandemic, and to identify HCP and facility characteristics associated with program use.
Materials and methods
Study design and population
We implemented this cross-sectional survey of ED personnel as a part of the COVID-19 Evaluation of Risks in Emergency Departments Project (COVERED), a prospective cohort study of emergency physicians, advanced practice providers (nurse practitioners and physician assistants), nurses and non-clinical ED personnel in 21 academic medical center EDs located in 15 states in the United States. The main study methods and results have been previously published [16, 33]. The parent study initially enrolled 1,606 participants (49.4% physicians or advanced practice providers, 25.5% nurses, and 25.0% nonclinical staff) between May 13 and August 26, 2020. For this survey, participants were from a group of HCP who had already consented to participate in the parent study. We used e-mail and text message reminders, participant incentives for completing surveys, and follow-up by local project staff as needed to ensure participation. The U.S. Centers for Disease Control and Prevention (CDC) and local institutional review boards reviewed this activity, which was conducted as public health surveillance consistent with applicable federal law and CDC policy [34]. All participants provided written informed consent, and study personnel had access to participant identifiers during the conduct of the project. This report is prepared in accordance with the Strengthening Observational Studies in Epidemiology (STROBE) statement [35].
Survey content and administration
We administered an electronic survey regarding support program availability and use as part of the items that each participant completed in Week 4 of their enrollment in COVID-19 surveillance between June 10 and September 23, 2020. The list of programs included in the survey was developed a priori by study investigators based on a literature review and information provided to the research team from participating institutions (see S1 File for survey items). All site investigators and participants were asked about the same list of programs. Each site investigator reported whether their institution offered specific individual employee support programs and/or clinical work support programs within their ED at the time of data collection. On their week 4 survey, each study participant was asked whether they had personally used each of the programs on the list. For analysis and interpretation, program types were grouped into those targeting HCP individually and those targeting clinical workflow. These program groupings were created a priori by members of the study team based on the likely primary intended target of the program.
Individual employee support programs
Individual support programs included 15 types of programs that we grouped into 5 categories: (1) financial support (i.e., disability, paid time off, and additional payments); (2) workload mitigation (i.e., flexible scheduling and staff surge plans); (3) HCP COVID-19 testing (i.e., test per request and asymptomatic testing program); (4) emotional support (e.g., stress reduction/resilience training, access to mental health hotline, and social media support programs); and (5) instrumental supports (i.e., laundry services, alternative living provided, transportation, eldercare, and childcare).
Clinical work support programs
Clinical work support programs included 13 types of programs that we grouped into 4 categories: (1) healthcare team communication (i.e., volume status board, personal protective equipment [36] status board, and debriefing after critical incidents); (2) family communication facilitation to mitigate social distancing visitor limits (i.e., audio-facilitated and video-facilitated); (3) patient services (i.e., interpreters, social worker, ethics consultation, and palliative care consultation); and (4) system-level HCP COVID-19 exposure reduction programs (i.e., team doffing of PPE, patient COVID-19 testing self-swabs, telehealth patient triage, telehealth patient care appointments) (S1 File).
Stress and anxiety measures
In the same survey, participants completed two measures rating their stress and anxiety, worded to specifically refer to the impact of the COVID-19 pandemic. First, participants responded to the following item: “In the past week, how much has the COVID-19 pandemic affected your stress or anxiety levels?” Elevated COVID-19 related stress and anxiety was defined as an answer of “somewhat” or more to this item (which corresponds to a score of ≥4; see S1 File). Second, participants completed the Primary Care Post-Traumatic Stress Disorder Screen for DSM-5 (PC-PTSD-5), a 5-item screening instrument for which a score of ≥3 signifies elevated risk for PTSD [37].
Data analysis
We calculated availability for each program type (e.g., elder support, transportation) and each category of programs (e.g., instrumental supports), reporting categorical measures as counts (percentages) and continuous measures as means (standard deviations) or medians (interquartile ranges, IQRs) as appropriate. We analyzed data using SAS v.9.4 (SAS Institute, Cary, NC) and R v.4.1.1 (R Foundation for Statistical Computing, Vienna, Austria).
Program availability and use
We summarized the availability and use for each type of program according to the following a priori–defined classification plan:
A program was considered available at a site if the site investigator indicated that a program was available or if 2 or more participants indicated that they had used it.
A program was considered used by a participant if he/she/they indicated personal use. All participants were included in calculations for individual support program use, but only clinical staff (physicians, advanced practice providers, and nurses) were included in calculations for clinical work support program use. Program use was calculated only among sites at which that program was available.
We used this flexible definition to account for cases in which a program was available but not known to the local COVERED site investigator.
HCP and facility characteristics associated with program use
We used generalized linear mixed modeling (GLMM) with a logit link to make bivariate comparisons between HCP/site characteristics (age, gender, race/ethnicity, professional role, HCP hours worked per week, HCP stress and anxiety measures, and facility COVID-19 case count the month of the survey) and support program use and included a random intercept in each model to adjust for site clustering. For each model, the GLMM provides p-values for the overall effect. Although we have multiple comparisons, we elected not to adjust our threshold for significance. We were interested in identifying hypothesis-generating factors that relate to availability and use with an emphasis on minimizing our type II rather than type I error [38].
Results
Participants
A total of 1,541 HCP completed the survey (response rate 96.0%), including 762 (49%) physicians or advanced practice providers, 396 (26%) nurses, and 383 (25%) non-clinical staff. Their median age was 36 years (IQR 30–45 years). Most were female (63%) and non-Hispanic White (81%). At the time of survey completion, the median ED COVID-19 patient volume was 107 patients (IQR 57–271) per facility per month (Table 1).
Table 1. Emergency Department (ED) Health Care Personnel (HCP; N = 1,541) and facility COVID-19 volume for 21 participating COVERED EDs during the COVID-19 pandemic, June–September 2020.
| HCP or Facility Characteristic | |
|---|---|
| Age (years), median (IQR) | 36 (30–45) |
| Hours worked per week, mean (SD) | 31.8 (10) |
| Facility COVID-19 patient volume in month before survey response, median (IQR) | 107 (57–271) |
| Gender, n (%) | |
| Female | 974 (63) |
| Male | 560 (36) |
| Transgender/Non-Conforming | 7 (1) |
| Race, n (%) | |
| White | 1214 (79) |
| Black | 110 (7) |
| Asian | 119 (8) |
| Multi/Other | 98 (6) |
| Hispanic/Latino, n (%) | 145/1499 (10) |
| Professional Role, n (%) | |
| Physician/APP | 762 (50) |
| Nurse | 396 (26) |
| Non-clinical staff | 383 (25) |
| HCP Stress Measures, n (%) | |
| Elevated COVID-19 related stress and anxiety rating (≥4 points) | 787/1393 (57) |
| Elevated risk for PTSD (PC-PTSD-5 ≥3 points) | 170/1398 (12) |
IQR, interquartile range; APP, advanced practice provider; PTSD, post-traumatic stress disorder; PC-PTSD-5, Primary Care Post-Traumatic Stress Disorder-5 Scale
Support program availability and use
The proportion of sites with each type of program and each program category available and HCP use are shown in Fig 1 and S1 Table. S2 Table presents information about which of the classification criteria resulted in coding each program as available. In 61% of cases the program was coded as available due to site investigator report and in 39% of cases the program was coded as available due to program use by ≥ 2 HCP. For HCP individual support programs, the proportion of sites that had at least one type of program available from each category was 95–100%. Individual support program category use ranged from 6% for emotional support to 13% for work demand mitigation. All sites had at least one type of program in each of the clinical work program categories in place in their ED (i.e., COVID-19 exposure reduction, patient care services, or patient–family communication facilitation and ED team communication). Clinical work support programs with the lowest availability were ethics consultations (47%) and self-administered swabs for COVID-19 patient testing (33%). Use of clinical work support programs ranged from 24% for patient–family communication facilitation to 43% for use of patient care services, which was driven by high social worker and interpreter involvement in care (24% and 36%, respectively) and low use of palliative care (7%) and ethics consultations (1%).
Fig 1. Availability and use of health care personnel support programs across 21 COVERED emergency departments during the COVID-19 pandemic, June–September 2020.
A. Individual Employee Support Programs among all HCP participants, B. Clinical Work Support Programs among clinical HCP. The black bars indicate the proportion of participants for whom that support program type and/or category of program was available during the study period. The colored bars indicate the proportion of participants who used that support program type or category. Category pooled totals are represented in all capital letters.
Emotional support programs, such as stress reduction and mental health hotlines, were reported by site investigators to be available to 95% of participants. Overall, 6% of HCP reported using these programs at some point during the COVID-19 pandemic prior to completing the survey. In our cohort, 57% (n = 787/1393) of HCP reported elevated COVID-19–related stress or anxiety and 12% were at elevated risk of PTSD assessed via the Primary Care PTSD Screen for DSM-5 [PC-PTSD-5]; n = 170/1398). Use of emotional support programs did not significantly differ between HCP who reported elevated stress or anxiety or were at risk of PTSD based on the PC-PTSD-5 and those who were not (see Table 2). Use of emotional support programs was 7.5% among HCP with elevated COVID-19–related stress or anxiety and 9.3% among those who were at elevated risk for PTSD based on the PC-PTSD-5.
Table 2. Demographic and facility characteristics by use of individual COVERED emergency department health care personnel support program categories during the COVID-19 pandemic, June–September 2020.
In this table, HCP use of each support program category is listed.
| Instrumental Support | Emotional Support | COVID-19 Testing for HCPs | Word Demand Mitigation | Financial Compensation | |
|---|---|---|---|---|---|
| Used (%) | Used (%) | Used (%) | Used (%) | Used (%) | |
| Age | |||||
| Less than 30 y | 30/359 (8.4%) | 18/336 (5.4%) | 48/359 (13.4%) | 40/359 (11.1%) | 35/359 (9.7%) |
| 30 y ≤ Age < 40 y | 83/602 (13.8%) | 43/586 (7.3%) | 82/602 (13.6%) | 89/602 (14.8%) | 66/602 (11.0%) |
| 40 y ≤ Age < 50 y | 31/308 (10.1%) | 14/297 (4.7%) | 33/308 (10.7%) | 46/308 (14.9%) | 30/308 (9.7%) |
| 50 y and Older | 17/272 (6.3%) | 11/255 (4.3) | 26/272 (9.6%) | 27/272 (9.9%) | 25/272 (9.2%) |
| Gender | |||||
| Male | 62/560 (11.1%) | 25/535 (4.7%) | 70/560 (12.5%) | 70/560 (12.5%) | 39/560 (7.0%) |
| Female | 98/974 (10.1%) | 61/932 (6.5%) | 117/974 (12.0%) | 131/974 (13.4%) | 114/974 (11.7%) |
| Nonbinary | 1/7 (14.3%) | 0/7 (0%) | 2/7 (28.6%) | 1/7 (14.3%) | 3/7 (42.9%) |
| Race1 | |||||
| White | 123/1214 (10.1%) | 62/1157 (5.4%) | 154/1214 (12.7%) | 158/1214 (13.0%) | 123/1214 (10.1%) |
| Black/AA | 10/110 (9.1%) | 4/107 (3.7%) | 7/110 (6.4%) | 15/110 (13.6%) | 10/110 (9.1%) |
| Asian | 15/119 (12.6%) | 12/114 (10.5%) | 13/119 (10.9%) | 18/119 (15.1%) | 8/119 (6.7%) |
| Multi/Other | 13/98 (13.3%) | 8/96 (8.3%) | 15/98 (15.3%) | 11/98 (11.2%) | 15/98 (15.3%) |
| Ethnicity | |||||
| Hispanic/Latino | 16/145 (11.0%) | 11/145 (7.6%) | 15/145 (10.3%) | 8/145 (5.5%) | 8/145 (5.5%) |
| Non-Hispanic/Latino | 143/1354 (10.6%) | 72/1289 (5.6%) | 166/1354 (12.3%) | 191/1354 (14.1%) | 147/1354 (10.9%) |
| Professional Role | |||||
| Physician/APP | 92/762 (12.1%) | 52/728 (7.1%) | 103/762 (13.5%) | 123/762 (16.1%) | 53/762 (7.0%) |
| Nurse | 43/396 (10.9%) | 20/376 (5.3%) | 52/396 (13.1%) | 38/396 (9.6%) | 65/396 (16.4%) |
| Non-Clinical Staff | 26/383 (6.8%) | 14/370 (3.8%) | 34/383 (8.9%) | 41/383 (10.7%) | 38/383 (9.9%) |
| Hours Worked per Week | |||||
| Less than 20 h | 16/188 (8.5%) | 8/177 (4.5%) | 27/188 (14.4%) | 34/188 (18.1%) | 19/188 (10.1%) |
| 20 h ≤ Hours < 30 h | 50/390 (12.8%) | 20/371 (5.4%) | 52/390 (13.3%) | 66/390 (16.9%) | 28/390 (7.2%) |
| 30 h ≤ Hours < 40 h | 66/595 (11.1%) | 46/575 (8.0%) | 71/595 (11.9%) | 62/595 (10.4%) | 77/595 (12.9%) |
| 40 h or More | 28/343 (8.2%) | 12/328 (3.7%) | 36/343 (10.5%) | 35/343 (10.2%) | 30/343 (8.7%) |
| Elevated COVID-19 Stress/Anxiety | |||||
| Low stress/anxiety (0–3) | 66/606 (10.9%) | 28/577 (4.9%) | 83/606 (13.7%) | 89/606 (14.7%) | 60/606 (9.9%) |
| Elevated stress/anxiety (≥ 4) | 94/787 (11.9%) | 57/759 (7.5%) | 105/787 (13.3%) | 112/787 (14.2%) | 95/787 (12.1%) |
| Elevated risk for PTSD (PC-PTSD-5) | |||||
| Low risk score (0–2) | 138/1228 (11.2%) | 71/1179 (6.0%) | 165/1228 (13.4%) | 172/1228 (14.0%) | 137/1228 (11.2%) |
| High risk score (≥ 3) | 23/170 (13.5%) | 15/162 (9.3%) | 24/170 (14.1%) | 30/170 (17.6%) | 19/170 (11.2%) |
| Facility COVID-19 patient volume (in last month) | |||||
| Volume < 50 pts/m | 33/292 (11.3%) | 21/225 (9.3%) | 43/292 (14.7%) | 43/292 (14.7%) | 60/292 (20.5%) |
| 50 pts/m ≤ Volume < 100 pts/m | 44/413 (10.7%) | 20/413 (4.8%) | 37/413 (9.0%) | 57/413 (13.8%) | 42/413 (10.2%) |
| 100 pts/m ≤ Volume < 250 pts/m | 49/444 (11.0%) | 25/444 (5.6%) | 48/444 (10.8%) | 55/444 (12.4%) | 25/444 (5.6%) |
| 250 pts/m or more | 35/392 (8.9%) | 20/392 (5.1%) | 61/392 (15.6%) | 47/392 (12.0%) | 29/392 (7.4%) |
y, years; APP, advanced practice provider; h, hours; pts, patients; m, month
1The number of HCP in American Indian/Alaskan Native, Native Hawaiian/Other Pacific Islander, Multiple, and Other race groups were too small to draw meaningful inferences about race/ethnicity and thus they were grouped together for comparison.
HCP and facility characteristics associated with program use
Several factors were associated with program use. Instrumental support program use differed by age, with a higher rate of use among HCPs ages 30–39 years than those younger than 30 years (13.8% vs. 8.4%; adjusted odds ratio [aOR] 1.61, 95% CI 1.01–2.56). Instrumental support programs were more likely to be used by physicians and advanced practice providers than nonclinical staff (12.1% vs. 6.8%; aOR 1.84, 95% CI 1.15–2.94). Work demand mitigation program use was most common among physicians/advanced practice providers (16.1% compared with 9.6% among nurses [aOR 1.81, 95% CI 1.23–2.67] and 10.7% among non-clinical staff [aOR 1.59, 95% CI 1.09–2.32]). Staff who worked fewer hours reported more use of work demand mitigation programs (18% for those working 20 hours per week vs. 10.2% working 40 or more hours per week; aOR 1.95, 95% CI 1.17–3.28). Physicians/advanced practice providers reported lower use of financial compensation programs than nurses (7.0% vs 16.4%; aOR 0.35, 95% CI 0.24–0.53) or non-clinical staff (9.9%; aOR 0.62, 95% CI 0.39–0.97). HCP who worked at facilities with the lowest COVID-19 patient volumes reported using financial compensation programs at the highest rate (20.5% in facilities with COVID-19 patient volume <50 in the past month vs. 10.2% in facilities with volume 50–99 [aOR 2.32, 95% CI 1.11–4.88], 5.6% in facilities with volume 100–250 [aOR 4.60, 95% CI 2.12–9.98)] and 7.4% in facilities with volume >250 [aOR 3.38, 95% CI 1.56–7.34]).
Clinical work program use was similar across HCP/facility characteristics, except that non-Hispanic White HCPs reported using programs in the patient care services category (47.8% vs. 30.6%; aOR 1.99, 95% CI 1.55–2.55) and patient-family communication category (25.7% vs. 18.7%; aOR 1.43, 95% CI 1.07–1.92) at a higher rate than HCPs from other racial/ethnic groups.
Discussion
In this study we describe the availability and use of institutional support programs for frontline ED HCP at 21 academic hospitals in the early COVID-19 pandemic before vaccine availability and identified HCP characteristics that were associated with program use. Nearly all sites had at least one program from each category that was included in our survey (95–100%), though there were differences in availability across program types. Programs to mitigate work demands via surge staffing, paid time off, and COVID-19 testing for staff were widely available (90–100%). In contrast, elder care, disability services after SARS-CoV-2 infection, and additional payments for frontline workers were less available. Our primary observation was that despite relatively high availability of many types of individual staff support programs, HCP reported low use (<15% for each program).
Several medical professional organizations have published recommendations to improve staff support [39]. A systematic review, which included data from multiple prior virus outbreaks worldwide (i.e., severe acute respiratory syndrome [SARS], influenza caused by H1N1, Middle East respiratory syndrome [MERS], and Ebola), demonstrated that staff support protocols, clear communication, psychosocial interventions, and adequate infection protection were all important components of programming to improve the psychological wellbeing of medical staff during virus outbreaks [40]. Digital learning packages have been developed to aid in resiliency and coping in some medical centers [28]. Data regarding effective interventions to promote resilience in military settings and/or during natural disasters are relevant to inform programming during pandemics. For example, peer-support interventions developed initially for the military have also been used in the context of the COVID-19 pandemic [29], as peer-support has the advantage of being rapidly deployable and promoting interpersonal relationships. Indeed, in a qualitative study of HCP participating in the COVID-19 Pandemic and Emotional Well-Being Study, participants highlighted personal strategies for maintaining social connectedness that were important in coping with uncertainty from their work [41].
Our primary analyses examined rates of program use across ED HCP, although different HCP are likely to have greater needs for certain interventions than others. For this reason, additional research to target support programs optimally for those that are most likely to benefit is essential. In our study, we examined support program use comparing HCP with and without elevated stress or anxiety. Among the 57% of ED HCP who reported elevated stress or anxiety in our sample, less than 10% had engaged with an institution-provided emotional support program since the beginning of the pandemic, a rate of engagement that was similar to HCP without elevated stress or anxiety (8% vs 5%, respectively). The rate of support program use <10% should be considered in the context of the societal turmoil and work-related demands placed on ED HCP during the early period of the COVID-19 pandemic, and it suggests that employer-provided emotional support services may not reach the population of HCP most likely to be affected. The low rate of support program use highlights the importance of work to better identify social and practical barriers to institutional program use to support frontline HCP during future widespread health crises. In addition to work demands, other potential barriers may include HCP concerns about the efficacy of programs and/or concern about stigmatization by coworkers for utilizing psychosocial assistance programs.
We observed some HCP and facility characteristics that were associated with differential rates of support program use, which should be viewed as hypothesis-generating given that HCP were not asked about their perceived needs or the applicability of each program to their individual circumstances. First, HCP working over 40 hours per week reported the lowest use of individual staff support programs. Full-time staff working extra hours may not have time to engage with institutional programming, which could be one significant barrier to participation. Institutional support programming may have required additional time that was not perceived as an effective countermeasure for stress, anxiety, burnout, and PTSD by all. It is also possible that HCP who worked more hours became more familiar with use of PPE and might not have experienced the same level of concern about risk. Second, facilities with the highest COVID-19 patient volumes had the lowest proportion of HCP receiving extra financial compensation. This observation raises the possibility that in summer 2020, those facilities that were most burdened with pandemic-related demands may have had less capacity to roll out new financial support programs. This period was during the first wave of COVID-19 in most US locations and travel continued to be limited.
This study has several limitations. First, participants were ED HCP from primarily large academic and urban centers in the US with demographic characteristics generally consistent with the US hospital workforce [42]; however, their experiences may not reflect those of community and rural ED HCP. Second, information about institutional program use was collected only at one time point, and thus the cross-sectional nature of the data precludes an assessment of changes in program availability and use over time. It is possible that HCP with anxiety, stress, burnout, and/or PTSD symptoms used program resources later in the pandemic. However, information regarding HCP needs early in the pandemic are important to prepare for the future, and the current data build on prior research from early responses during other virus outbreaks. Third, our survey asked only about availability and use of institution-based programs and did not gather information about other potential sources of support for HCP outside of work, HCP’s perception of unmet program needs, the perceived or actual effectiveness of these program types, or satisfaction with programming. Fourth, our definition of program availability relied on site investigators and HCP’s awareness of existing programing at their hospital and this approach likely missed some support programs that were available but unknown to either study investigators or HCP participants. HCP were not asked about their awareness of programs and instead reported only about their individual program use. We classified a program as available at a site if the site investigator indicated that it was available or if 2 or more HCP participants at the site had used it. In 39% of cases programs were coded as being available based on HCP use (i.e., instances where the site investigator had not identified the program as present). Using this flexible definition allowed us to utilize all information collected in the study regarding program availability, but the data suggest that lack of awareness of support programming is an important issue for facilities and ED departments to address. Fifth, the small sample prevented substantive analysis stratified by race/ethnicity. Future efforts can focus on improving data, including by race/ethnicity, to reflect diversity of the workforce. Finally, these data are from the initial wave of the COVID-19 pandemic from June to September 2020, a time of uncertainty about COVID-19 transmission, prior to availability of the COVID-19 vaccine, when personal protective equipment was limited at many sites in the U.S. Indeed, our data do not reflect changes in hospital programming, increasing HCP burnout, and changes in clinical practice as the pandemic has proceeded. Nonetheless, our data from frontline HCP can be used to inform our preparedness for future widespread health disasters and the impact of those events on a capable HCP workforce.
Conclusions
In conclusion, institutional support programs for frontline ED HCP were widely available early in the COVID-19 pandemic. Despite availability and the high prevalence of symptoms of stress, anxiety, and burnout, use of individual employee support programs was low, highlighting the importance of future work to identify and address barriers to accessing current services and the need to develop more effective interventions to protect and improve ED HCP well-being during times of crisis.
Supporting information
(PDF)
(PDF)
(PDF)
Acknowledgments
The authors thank the COVERED participants, the participating institutions/emergency departments, and individuals. The authors acknowledge the following participating Project COVERED emergency departments: Allegheny General Hospital, Pittsburgh, PA; Baystate Medical Center, Springfield, MA; Denver Health Medical Center, Denver, CO; Detroit Receiving Hospital/Sinai–Grace Hospital, Detroit, MI; Hennepin County Medical Center, Minneapolis, MN; Jackson Memorial Hospital, Miami, FL; Johns Hopkins Medical Institute, Baltimore, MD; University Medical Center, New Orleans, LA; Mount Sinai Hospital East/Elmhurst Hospital Center, New York, NY; Orlando Regional Medical Center, Orlando, FL; University of Alabama at Birmingham Hospital, Birmingham, AL; Ronald Reagan–UCLA Medical Center/Olive View–UCLA Medical Center, Los Angeles, CA; University of Iowa, Iowa City, IA; University of Massachusetts Memorial Medical Center, Worcester, MA; University of Mississippi Medical Center, Jackson, MS; UCSF Zuckerberg San Francisco General, San Francisco, CA; UT Southwestern Medical Center/Parkland Memorial Hospital, Dallas, TX; Truman Medical Center, Kansas City, MO; Thomas Jefferson University, Philadelphia, PA; and Washington University Barnes–Jewish Hospital, St. Louis, MO.
The authors acknowledge the following individuals for their help with the study: Aishat Adeyemi, BA, Lisa Allen, MPH, MPA, Gregory Almonte, Otuwe Anya, MS, Paula Arellano-Cruz, BS, Ruzana Aronov, Danielle Beckham, RN, MSN, CRC, Lauren Buck, BS, Samuel Ceckowski, BS, Maxime Centeno, BS, Virginia, Chan, BS, Anna Marie Chang, MD, MSCE, Melissa Connor, RN, Gabriella Dashler, BS, Jenna Davis, MSN, Cynthia Delgado, BS, Veronica Delgado PA, MS, Brianna DiFronzo, Radhika L Edpuganti, BS, Alyssa Espinera, MD, Fresa Estevez, BA, Shelly Ann Evans, MBA, RN, Cathy Fairfield, BSN, Phillip Fairweather, MD, Theodore Falcon, BS, Brian Fuller, MD, MSCI, David Gallegos, BBA, Samuel Ganier, MD, Stephanie Gravitz, MPH, Jeffrey Harrison, RN, Kyle Herbert MD, Judy Hermans, MPH, Emily Hopkins, MSPH, Alan Jones, MD, Kia M. Jones, DrPH, Momina Khan, BS, Laura Iavicoli, MD, Robin Kemball, MPH, Laurie Kemble, BHSRT, CCRC, Stuart Kessler, MD, Catherine Lind RN, NP, Karina Loayza, LCSW, Carol Lynn Lyle, PA, MPH, Virginia B. Mangolds, MS, RN, Hannah Makarevich-Manilla, MPH, CCRC, Thomas Mazzocco, RN, Sarah Meram, MS, Valerie H. Mika, MS, Reynaldo Padilla, BA, Giacomo Passaglia, BS, Rebekah Peacock, BSN, Danielle Perez, BS, Kye E. Poronsky, MA, Eric Raines, EMT-P, Monica N. Ramage, MSN, RN, Kavita Rampertaap, MSN, RN, Sarah Reineck, BS, Nicole Renzi, RN, Erin P. Ricketts, MSPH, Stephanie Rodriguez, MA, Justin Sabol, BS, Valeria Samame, BA, Katie Schneider, MSN, Robert Sellman, PA, Kristine Sernulka, BSN, CCRP, Edward Sheriff, PhD, MPH, Jennifer Siller, DNP, RN, Colleen Smith, MD, Timothy Smith, BS, Patricia Slev, MD, Kelly Szabo, MPH, CCRC, Meghan Tinetti, BSN, CCRP, Denise Tritt, CIM, CRCP, Julia Vargas, BS, Samuel Vargas, BS, Kavey Vidal, BS, Lori Wilkerson, RN, CRC, Darleen Williams, DNP, APRN-CNS, Sallie Anne Wright, MPH, BSMT, and Isaias Yin, LVN. Additionally, the authors acknowledge Paul Casella, MFA for his editorial assistance.
The Project COVERED Emergency Department Network includes the following: Monica Bahamon, MPH, Jestin N. Carlson MD, MSc, Makini Chisolm-Straker, MD, MPH, Brian Driver, MD, Brett Faine, Pharm D, MS, Brian M. Fuller, MD, James Galbraith, MD, Philip A. Giordano, MD, John P. Haran, MD, PhD, Elisabeth Hesse, MD, MTM&H, Amanda Higgins, MS, Jeremiah Hinson, MD, Stacey House, MD, PhD, Ahamed H. Idris, MD, Efrat Kean, MD, Elizabeth Krebs, MD, MSc, Michael C. Kurz, MD, MS, Preeta K. Kutty, MD, MPH, Lilly Lee SM, MD, Stephen Y. Liang, MD, MPHS, Stephen C. Lim, MD, Clifford McDonald, MD, Gregory Moran, MD, William Mower, MD, PhD; Utsav Nandi, MD, MSCI, Kavitha Pathmarajah, MPH, James H. Paxton MD, Yesenia Perez, BA, Lynne D. Richardson, MD, Richard Rothman, MD, PhD, Walter A. Schrading MD, Jessica Shuck, BA, Patricia Slev, MD, Howard A. Smithline, MD, Michelle St. Romain, MD, Kimberly Souffront, PhD, FNP-BC, RN, Mark T. Steele, MD, Amy Stubbs, MD, Morgan B. Swanson, Josh Tiao, MD, Jesus R. Torres, MD, MPH, Stacy A. Trent MD MPH, Lisandra Uribe, BS, Arvind Venkat, MD, Gregory Volturo, MD, Kurt D. Weber, MD, and James Willey, MD. The lead author of this authorship group is Nicholas Mohr, MD (nicholas-mohr@uiowa.edu).
The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
Data Availability
The deidentified data upon which this manuscript is based has been posted on Open Science Framework (osf.io) and can be accessed from the following DOI: https://osf.io/dr2kj/.
Funding Statement
This project was funded by a cooperative agreement from the Centers for Disease Control and Prevention (CDC) (U01CK000480, MPI: DAT, NMM, www.cdc.gov) and the Institute for Clinical and Translational Science at the University of Iowa through a grant from the National Center for Advancing Translational Sciences at the National Institutes of Health (UL1TR002537, https://icts.uiowa.edu/). Support was additionally provided by NIH grant UM1TR004403. The sponsors played no role in the study design, data collection or analysis, decision to publish, or preparation of the manuscript. The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
References
- 1.Shanafelt T, Ripp J, Trockel M. Understanding and Addressing Sources of Anxiety Among Health Care Professionals During the COVID-19 Pandemic. JAMA. 2020;323(21):2133–4. doi: 10.1001/jama.2020.5893 [DOI] [PubMed] [Google Scholar]
- 2.Rodriguez BO, Sanchez TL. The Psychosocial Impact of COVID-19 on health care workers. Int Braz J Urol. 2020;46(suppl.1):195–200. Epub 2020/07/04. doi: 10.1590/S1677-5538.IBJU.2020.S124 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Makino M, Kanie A, Nakajima A, Takebayashi Y. Mental health crisis of Japanese health care workers under COVID-19. Psychol Trauma. 2020;12(S1):S136–S7. Epub 2020/06/26. doi: 10.1037/tra0000819 . [DOI] [PubMed] [Google Scholar]
- 4.Shah AH, Becene IA, Nguyen K, Stuart JJ, West MG, Berrill JES, et al. A qualitative analysis of psychosocial stressors and health impacts of the COVID-19 pandemic on frontline healthcare personnel in the United States. SSM Qual Res Health. 2022;2:100130. Epub 2022/07/26. doi: 10.1016/j.ssmqr.2022.100130 ; PubMed Central PMCID: PMC9293380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pala AN, Chuang JC, Chien A, Krauth DM, Leitner SA, Okoye NM, et al. Depression, anxiety, and burnout among hospital workers during the COVID-19 pandemic: A cross-sectional study. PLoS One. 2022;17(12):e0276861. Epub 20221209. doi: 10.1371/journal.pone.0276861 ; PubMed Central PMCID: PMC9733879. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.La Torre G, Barletta VI, Marte M, Paludetti F, Faticoni A, Barone LC, et al. Assessment of Anxiety, Depression, Work-Related Stress, and Burnout in Health Care Workers (HCWs) Affected by COVID-19: Results of a Case-Control Study in Italy. J Clin Med. 2022;11(15). Epub 20220729. doi: 10.3390/jcm11154434 ; PubMed Central PMCID: PMC9369262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Conejero I, Petrier M, Fabbro Peray P, Voisin C, Courtet P, Potier H, et al. Post-traumatic stress disorder, anxiety, depression and burnout in nursing home staff in South France during the COVID-19 pandemic. Translational Psychiatry. 2023;13(1):205. doi: 10.1038/s41398-023-02488-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Novilla MLB, Moxley VBA, Hanson CL, Redelfs AH, Glenn J, Donoso Naranjo PG, et al. COVID-19 and Psychosocial Well-Being: Did COVID-19 Worsen U.S. Frontline Healthcare Workers’ Burnout, Anxiety, and Depression? International Journal of Environmental Research and Public Health. 2023;20(5):4414. doi: 10.3390/ijerph20054414 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.LeClaire M, Poplau S, Linzer M, Brown R, Sinsky C. Compromised Integrity, Burnout, and Intent to Leave the Job in Critical Care Nurses and Physicians. Crit Care Explor. 2022;4(2):e0629. Epub 2022/02/15. doi: 10.1097/CCE.0000000000000629 ; PubMed Central PMCID: PMC8824411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hare Bork R, Robins M, Schaffer K, Leider JP, Castrucci BC. Workplace Perceptions and Experiences Related to COVID-19 Response Efforts Among Public Health Workers—Public Health Workforce Interests and Needs Survey, United States, September 2021-January 2022. MMWR Morb Mortal Wkly Rep. 2022;71(29):920–4. Epub 2022/07/22. doi: 10.15585/mmwr.mm7129a3 ; PubMed Central PMCID: PMC9310630. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Moss M, Good VS, Gozal D, Kleinpell R, Sessler CN. An Official Critical Care Societies Collaborative Statement-Burnout Syndrome in Critical Care Health-care Professionals: A Call for Action. Chest. 2016;150(1):17–26. doi: 10.1016/j.chest.2016.02.649 . [DOI] [PubMed] [Google Scholar]
- 12.Costa DK, Moss M. The Cost of Caring: Emotion, Burnout, and Psychological Distress in Critical Care Clinicians. Ann Am Thorac Soc. 2018;15(7):787–90. Epub 2018/05/05. doi: 10.1513/AnnalsATS.201804-269PS ; PubMed Central PMCID: PMC6207112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Moukarzel A, Michelet P, Durand AC, Sebbane M, Bourgeois S, Markarian T, et al. Burnout Syndrome among Emergency Department Staff: Prevalence and Associated Factors. BioMed Research International. 2019;2019. doi: 10.1155/2019/6462472 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Yazıcı MU, Teksam O, Agın H, Erkek N, Arslankoylu AE, Akca H, et al. The Burden of Burnout Syndrome in Pediatric Intensive Care Unit and Pediatric Emergency Department: A Multicenter Evaluation. Pediatr Emerg Care. 2021;37(12):e955–e61. doi: 10.1097/PEC.0000000000001839 . [DOI] [PubMed] [Google Scholar]
- 15.Gualano MR, Sinigaglia T, Lo Moro G, Rousset S, Cremona A, Bert F, et al. The Burden of Burnout among Healthcare Professionals of Intensive Care Units and Emergency Departments during the COVID-19 Pandemic: A Systematic Review. Int J Environ Res Public Health. 2021;18(15). Epub 2021/08/08. doi: 10.3390/ijerph18158172 ; PubMed Central PMCID: PMC8346023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Mohr NM, Harland KK, Krishnadasan A, Eyck PT, Mower WR, Willey J, et al. Diagnosed and Undiagnosed COVID-19 in US Emergency Department Health Care Personnel: A Cross-sectional Analysis. Ann Emerg Med. 2021;78(1):27–34. Epub 20201217. doi: 10.1016/j.annemergmed.2020.12.007 ; PubMed Central PMCID: PMC7746085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rodriguez RM, Montoy JCC, Hoth KF, Talan DA, Harland KK, Eyck PT, et al. Symptoms of Anxiety, Burnout, and PTSD and the Mitigation Effect of Serologic Testing in Emergency Department Personnel During the COVID-19 Pandemic. Ann Emerg Med. 2021;78(1):35–43.e2. Epub 20210205. doi: 10.1016/j.annemergmed.2021.01.028 ; PubMed Central PMCID: PMC7862892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Yao H, Chen JH, Zhao M, Qiu JY, Koenen KC, Stewart R, et al. Mitigating mental health consequences during the COVID-19 outbreak: Lessons from China. Psychiatry Clin Neurosci. 2020;74(7):407–8. Epub 2020/05/05. doi: 10.1111/pcn.13018 ; PubMed Central PMCID: PMC7267425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wong AH, Pacella-LaBarbara ML, Ray JM, Ranney ML, Chang BP. Healing the Healer: Protecting Emergency Health Care Workers’ Mental Health During COVID-19. Ann Emerg Med. 2020;76(4):379–84. Epub 20200503. doi: 10.1016/j.annemergmed.2020.04.041 ; PubMed Central PMCID: PMC7196406. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Tan Z, Khoo DWS, Zeng LA, Tien JC, Lee AKY, Ong YY, et al. Protecting health care workers in the front line: Innovation in COVID-19 pandemic. J Glob Health. 2020;10(1):010357. Epub 2020/06/09. doi: 10.7189/jogh.10.010357 ; PubMed Central PMCID: PMC7242895. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Leszcz M, Maunder R, Hunter J. Psychological support for health care workers during the COVID-19 pandemic. Canadian Medical Association Journal. 2020;192(24):E660–E. doi: 10.1503/cmaj.75864 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Wu PE, Styra R, Gold WL. Mitigating the psychological effects of COVID-19 on health care workers. CMAJ. 2020;192(17):E459–E60. Epub 2020/04/17. doi: 10.1503/cmaj.200519 ; PubMed Central PMCID: PMC7207194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Griffin KM, Karas MG, Ivascu NS, Lief L. Hospital Preparedness for COVID-19: A Practical Guide from a Critical Care Perspective. Am J Respir Crit Care Med. 2020;201(11):1337–44. Epub 2020/04/17. doi: 10.1164/rccm.202004-1037CP ; PubMed Central PMCID: PMC7258631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Greenberg N, Brooks SK, Wessely S, Tracy DK. How might the NHS protect the mental health of health-care workers after the COVID-19 crisis? Lancet Psychiatry. 2020;7(9):733–4. Epub 2020/06/01. doi: 10.1016/S2215-0366(20)30224-8 ; PubMed Central PMCID: PMC7255732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.David E, DePierro JM, Marin DB, Sharma V, Charney DS, Katz CL. COVID-19 Pandemic Support Programs for Healthcare Workers and Implications for Occupational Mental Health: A Narrative Review. Psychiatr Q. 2022;93(1):227–47. Epub 20211004. doi: 10.1007/s11126-021-09952-5 ; PubMed Central PMCID: PMC8488324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Diver S, Buccheri N, Ohri C. The value of healthcare worker support strategies to enhance wellbeing and optimise patient care. Future Healthc J. 2021;8(1):e60–e6. doi: 10.7861/fhj.2020-0176 ; PubMed Central PMCID: PMC8004324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. The Lancet. 2016;388(10057):2272–81. doi: 10.1016/S0140-6736(16)31279-X [DOI] [PubMed] [Google Scholar]
- 28.Blake H, Bermingham F, Johnson G, Tabner A. Mitigating the Psychological Impact of COVID-19 on Healthcare Workers: A Digital Learning Package. Int J Environ Res Public Health. 2020;17(9). Epub 2020/05/03. doi: 10.3390/ijerph17092997 ; PubMed Central PMCID: PMC7246821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Albott CS, Wozniak JR, McGlinch BP, Wall MH, Gold BS, Vinogradov S. Battle Buddies: Rapid Deployment of a Psychological Resilience Intervention for Health Care Workers During the COVID-19 Pandemic. Anesth Analg. 2020;131(1):43–54. Epub 2020/04/30. doi: 10.1213/ANE.0000000000004912 ; PubMed Central PMCID: PMC7199769. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Miotto K, Sanford J, Brymer MJ, Bursch B, Pynoos RS. Implementing an emotional support and mental health response plan for healthcare workers during the COVID-19 pandemic. Psychol Trauma. 2020;12(S1):S165–S7. Epub 2020/06/12. doi: 10.1037/tra0000918 . [DOI] [PubMed] [Google Scholar]
- 31.Feinstein RE, Kotara S, Jones B, Shanor D, Nemeroff CB. A health care workers mental health crisis line in the age of COVID-19. Depress Anxiety. 2020;37(8):822–6. Epub 20200715. doi: 10.1002/da.23073 ; PubMed Central PMCID: PMC7405037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Augustin M, Schommers P, Suarez I, Koehler P, Gruell H, Klein F, et al. Rapid response infrastructure for pandemic preparedness in a tertiary care hospital: lessons learned from the COVID-19 outbreak in Cologne, Germany, February to March 2020. Euro Surveill. 2020;25(21). Epub 2020/06/04. doi: 10.2807/1560-7917.ES.2020.25.21.2000531 ; PubMed Central PMCID: PMC7268272. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Mohr NM, Krishnadasan A, Harland KK, Ten Eyck P, Mower WR, Schrading WA, et al. Emergency department personnel patient care-related COVID-19 risk. PLoS One. 2022;17(7):e0271597. Epub 20220722. doi: 10.1371/journal.pone.0271597 ; PubMed Central PMCID: PMC9307202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.45 C.F.R. part 46.102(l)(2), 21 C.F.R. part 56; 42 U.S.C. §241(d); 5 U.S.C. §552a; 44 U.S.C. §3501 et seq.
- 35.von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–8. doi: 10.1136/bmj.39335.541782.AD [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.d’Ettorre G, Ceccarelli G, Santinelli L, Vassalini P, Innocenti GP, Alessandri F, et al. Post-Traumatic Stress Symptoms in Healthcare Workers Dealing with the COVID-19 Pandemic: A Systematic Review. Int J Environ Res Public Health. 2021;18(2). Epub 20210112. doi: 10.3390/ijerph18020601 ; PubMed Central PMCID: PMC7828167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Prins A, Bovin MJ, Smolenski DJ, Marx BP, Kimerling R, Jenkins-Guarnieri MA, et al. The Primary Care PTSD Screen for DSM-5 (PC-PTSD-5): Development and Evaluation Within a Veteran Primary Care Sample. J Gen Intern Med. 2016;31(10):1206–11. Epub 20160511. doi: 10.1007/s11606-016-3703-5 ; PubMed Central PMCID: PMC5023594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology. 1990;1(1):43–6. [PubMed] [Google Scholar]
- 39.Ralph J, Freeman LA, Ménard AD, Soucie K. Practical strategies and the need for psychological support: recommendations from nurses working in hospitals during the COVID-19 pandemic. J Health Organ Manag. 2021;ahead-of-print(ahead-of-print). doi: 10.1108/JHOM-02-2021-0051 . [DOI] [PubMed] [Google Scholar]
- 40.Kisely S, Warren N, McMahon L, Dalais C, Henry I, Siskind D. Occurrence, prevention, and management of the psychological effects of emerging virus outbreaks on healthcare workers: rapid review and meta-analysis. Bmj. 2020;369:m1642. Epub 20200505. doi: 10.1136/bmj.m1642 ; PubMed Central PMCID: PMC7199468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bender AE, Berg KA, Miller EK, Evans KE, Holmes MR. "Making Sure We Are All Okay": Healthcare Workers’ Strategies for Emotional Connectedness During the COVID-19 Pandemic. Clin Soc Work J. 2021;49(4):445–55. Epub 20210111. doi: 10.1007/s10615-020-00781-w ; PubMed Central PMCID: PMC7798008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Bureau of Labor Statistics USDoL. Labor Force Statistics from the Current Population Survey https://www.bls.gov/cps/cpsaat18.htm2022 [cited 2023 09/22/2023].