Abstract
This narrative review aims to summarize initiatives developed during the COVID-19 pandemic to support healthcare workers’ emotional well-being within the context of a pre-existing framework of occupational mental health guidelines. This occupational mental health framework integrates principles from multiple disciplines to optimize prevention and management of mental health issues among employees. We conducted an online search on Medline/PubMed, Cochrane Library, and Embase for studies that reported on design or execution of medical institution-based interventions, aiming to support healthcare worker mental health during the COVID-19 pandemic. Inclusion criteria was intentionally broad in order to incorporate as many types of interventions at varying stages of development or evaluation. We included 31 studies in our review that reported on newly designed psychological support interventions for healthcare workers (HCW) during the COVID-19 pandemic. We found that most programs commonly supported HCW mental health through offering one or more of the following initiatives: expanded basic need resources/services, additional workplace training programs that bolstered professional preparedness while also indirectly boosting HCW emotional health, and/or expanded psychological support programs, such as peer support programs, psychoeducational or counseling services. Most programs, however, did not consider methods to ensure program longevity or sustainability. The COVID-19 pandemic has underscored the acuity of HCW mental health issues and is likely to leave long lasting mental health strains among HCW. This pandemic is a critical point in time to catalyze much needed progress in reducing stigma and expanding HCW mental health care access.
Keywords: Mental health, Psychological support programs, Healthcare worker mental health intervention, COVID-19
Introduction
The COVID-19 pandemic has generated an unprecedented global health crisis. In the face of volatility and uncertainty, health care workers (HCW) continue to provide care under physically and emotionally stressful conditions that continue to evolve in parallel with the changing landscape of the pandemic itself. HCW, facing sustained periods of acutely stressful conditions and workplace environments, are at higher risk of burnout, chronic psychological distress, and post-traumatic stress, along with other mental health complications of varying severity, persisting for many years. The mental health strain of the COVID-19 pandemic on HCW has emerged as a global concern. The growing body of data on the negative impacts of COVID-19 on HCW mental health emphasizes the need for the evaluation and development of effective, sustainable interventions that can address both acute and long-term concerns, during and beyond the pandemic [1, 2].
Recently published literature outlines the expansion of medical institution-based initiatives supporting HCW emotional well-being and resilience during the COVID-19 pandemic; however, qualitative data on the efficacy of these training programs and interventions remains relatively sparse [3]. This review attempts to consolidate the published literature on mental health interventions and emotional support programs developed for HCW during the COVID-19 pandemic, showcasing the global surge of creativity and commitment to addressing HCW mental health needs under unprecedented circumstances. Previous reviews have evaluated and/or compared the efficacy and utility of select programs developed during the COVID-19 pandemic and previous epidemics [4–7]; however, these reviews evaluate the quantitative data available, which are limited and based on small cohorts studied for short periods of time. This narrative review, however, to our knowledge, adopts a unique lens, which evaluates a broad body of literature published on HCW mental health interventions specifically designed in response to COVID-19.
Additionally, rather than attempt to compare limited quantitative data that measures different variables across studies, we seek to evaluate these initiatives under the umbrella of a pre-existing definition of mental health and occupational mental health guidelines proposed by LaMontagne et al. [8]. This occupational mental health framework unites principles from the domains of occupational health and safety, organizational or positive psychology, and medical/illness-focused techniques for mental health treatment. The framework suggests that the greatest mental health benefits for working populations is achieved through interventions that simultaneously “protect” mental health by alleviating work-related risk factors, “promote” mental health by encouraging positive aspects of work along with workers’ strengths, and “address” mental health problems independent of whether the cause is directly attributed to the workplace. Using this framework to categorize the reviewed initiatives, we aim to identify key components that promote sustainable and effective mental health initiatives that holistically address HCW mental health needs during the COVID-19 pandemic and beyond.
Methods
We conducted an electronic search in Medline/Pubmed, Cochrane Library, and Embase. Digital search keywords included “healthcare worker OR professional OR provider,” “mental health services OR program OR service,” “mental health,” “mental disorders,” “resilience,” and “COVID-19.” To broaden results, we also utilized the PubMed Mesh keyword tool with Mesh keywords including “COVID-19,” “health personnel,” “Betacoronavirus,” “coronavirus infections,” “mental health,” “mental health services,” “occupational stress,” “stress, psychological,” and “resilience, psychological.” We also found additional studies through the reference sections of identified papers. Our study inclusion criteria were intentionally broad. Identified studies met criteria if they reported on an intervention that aimed to improve HCW mental health during COVID-19 and were published between April 2020 to April 2021. We included studies even if they were still evaluating intervention efficacy or had ongoing trials. We excluded all mental health interventions that were not designed during the COVID-19 pandemic and studies that were not published in English. We included 31 studies of interest in the following review, which represented a diverse array of interventional designs. A geographical breakdown of the studies is found in Fig. 1. Following the framework presented by LaMontagne et al. we subsequently organized the studies under the following categories: harm prevention, illness-focused approach to mental health management, and promotion of positivity, which is further elaborated on in the following results section (See Table 1) [8].
Table 1.
Author | Month+ Year | Country of Origin | Target HCW | Study Type | Outcomes Measured | Results Characteristics | Occupational Mental Health Framework Category |
---|---|---|---|---|---|---|---|
Chen et al. | April 2020 | China | All HCW | Descriptive, Commentary | Qualitative interviews | Qualitative | Harm Prevention |
Shanafelt et al. | April 20020 | US | All HCW | Descriptive, Commentary | Qualitative interviews | Qualitative | Harm Prevention, Promote Positivity |
Brown-Johnson et al. | July 2020 | US | All HCW | Descriptive | N/A | Qualitative | Harm Prevention |
Orsini et al. | November 2020 | US | All HCW | Descriptive | N/A | Qualitative | Harm Prevention |
Hong et al. | May 2021 | China | All HCW working in fever clinic | Initial program evaluations | Impact of Even Scale-Revised (IES-R) [50] Source of Distress Questionnaire [51] General Self-Efficacy Scale, Semi-structured qualitative interviews | Quantitative, Qualitative | Harm Prevention, Illness management |
Gonzalez et al. | August 2020 | US | All HCW | Descriptive, Commentary | N/A | N/A | Harm Prevention, Illness Management |
Saqib et al. | August 2020 | UK | All HCW | Initial program evaluations, Quality assurance (QA)/ Quality improvement (QI) | Staff surveys, qualitative feedback, immediate/delayed feedback outlets | Qualitative, Quantitative | Harm Prevention |
Putrino et al. | November 2020 | US | All HCW | Initial program evaluations | Single- item Likert style measure of perceived stress + Qualitative feedback | Quantitative, Qualitative | Harm Prevention |
Khan and Kiani | June 2020 | Pakistan | All HCW | Survey-based descriptive study, uncontrolled pre/post trial | Qualitative feedback, questionnaire on risk perception and preparedness using 10-point Likert scale | Quantitative, Qualitative | Harm Prevention |
Cheung et al. | September 2020 | Hong Kong | All HCW | Cross sectional study | Validated questionnaire on 5 domains of personal strength on 5-point Likert scale: (assertiveness, mental preparedness, self-efficacy, internal locus of control and responsibility) | Quantitative | Harm Prevention |
Ripp et al. | August 2020 | US | All HCW | Descriptive, Commentary | N/A | N/A | Harm Prevention, Illness Management, Promote Positivity |
Gray et al. | May 2021 | US | All HCW | Descriptive, Commentary, Initial program evaluation | Quantitative operational output metrics of service utilization and outreach capacity, qualitative received feedback | Quantitative/ Qualitative | Harm Prevention, Illness Management |
Serrano-Ripoll et al. | June 2021 | Spain | All HCW | Randomized controlled trial | Depression Anxiety and Stress Scales [52], Davidson Trauma Scale [53], Maslach Burnout Inventory[54] Insomnia Severity Index [55], General Self-Efficacy Scale [56], System Usability Scale [57] | Quantitative (Pending) | Illness Management |
Blake et al. | April 2020 | UK | All HCW | Initial program evaluations, QA/QI | Stakeholder feedback/peer review and Digital package evaluation surveys (ie. fidelity and implementation qualities) [58], User demographics | Quantitative, Qualitative | Illness Management |
Weiner et al. | October 2020 | France | All HCW | Randomized controlled trial | Perceived Stress Scale[59], Patient Health Questionnaire[60], Short Form Posttraumatic Stress Disorder Checklist 5 [61], Connor-Davidson Resilience Scale [62], Insomnia Severity Index [55], Affective Rumination Questionnaire [63], Credibility and Expectancy Questionnaire [64], Client Satisfaction Questionnaire [65] | Quantitative (Pending) | Illness Management |
Lai et al. | July 2020 | Canada | All HCW | Open label feasibility randomized controlled trial | Feasibility outcomes (rate of participant recruitment, retention rate, data entry completeness, intervention cost, unexpected costs) Intervention outcomes (Athens Insomnia Scale, Generalized Anxiety Disorder [66], Patient Health Questionnaire [67, 68], Connor Davidson Resilience Scale [69]) | Quantitative (Pending) | Illness Management |
Mira et al. | October 2020 | Spain | All HCW | Descriptive | Web traffic, user preferences/usages, app discharges | Quantitative | Illness Management |
Zhang et al. | June 2020 | China | Anesthesiologists | Descriptive, Commentary | N/A | N/A | Harm Prevention, Illness Management |
Krystal et al. | January- February 2021 | US | All HCW | Descriptive, Commentary | Qualitative | Qualitative | Harm Prevention, Illness Management, Promote Positivity |
Golden et al. | May 2021 | US | All HCW | Descriptive, Initial program evaluations | “Key performance indicators” (numbers of downloads, baseline surveys, postbaseline surveys, app engagement metrics) | Qualitative, Quantitative | Illness Management, Harm Prevention |
Lewis et al. | March 2021 | Australia | All HCW | Initial program evaluations, QA/QI | Qualitative user interviews, app analytics (ie. participation rates, user engagement, user demographics), mental health measures (ie. Patient Health Questionnaire 4 [70]) | Quantitative | Illness Management, Promote Positivity |
Saniasiaya and Ramasamy | June 2020 | Malaysia | All HCW | Descriptive, Commentary | N/A | N/A | Promote Positivity |
Mellins et al. | November-December 2020 | US | All HCW | Descriptive | Qualitative peer supervision feedback, anonymous confidential/voluntary post-group brief Qualtrics survey | Quantitative, Qualitative | Illness Management, Promote Positivity |
DePierro et al. | November 2020 | US | All HCW | Descriptive, Commentary | N/A | N/A | Illness Management, Promote Positivity |
Cheng et al. | August 2020 | China | All HCW | Initial program evaluations | User questionnaire responses (ie. demographics, subjective unit of feeling, “Gain-and-Issue scale,”) subjective effectiveness rating of Balint group, most impressive topics in online themed chat | Quantitative | Promote Positivity |
Albott et al. | May 2020 | US | All HCW | Descriptive | N/A | N/A | Promote Positivity |
Kazerooni et al. | April 2020 | Iran | Medical students | Descriptive | User questionnaire [71] | Qualitative, Quantitative | Promote Positivity |
Cheng et al. | May 2020 | China, USA, Canada, Australia | All HCW | Descriptive | N/A (only limited anecdotal evidence) | N/A | Promote Positivity |
Hall et al. | December 2020 | US | All HCW | Descriptive, Commentary | N/A | N/A | Promote Positivity |
Shingler-Nace | June 2020 | US | All HCW | Descriptive, Commentary | N/A | N/A | Promote Positivity |
López-Atanes et al. | August 2020 | Spain | Female HCW | Descriptive, Commentary | N/A | N/A | Harm Prevention, Illness Management |
Results
Harm Prevention
The first arm of LaMontagne et al.’s framework, harm prevention, draws from the principles of public health and psychology. According to LaMontagne et al. effective forms of harm prevention aim to simultaneously reduce jobs stressors while also promoting workers’ capabilities to overcome work-related stressors. Programs that address HCW basic needs could be classified as “harm prevention” initiatives, since they seek to decrease the incidence of work-related mental health problems through modifying the workplace environment to reduce work-related risk factors, along with personal risk factors that extend beyond the workplace proper [8].
Feedback collected by Chen et al. in fact, shows that HCW are less willing to engage in other forms of psychological support resources if their basic needs are not met, consistent with Maslow’s hierarchy of needs [9, 10]. At a New York medical institution, listening sessions revealed that HCW concerns primarily related to basic professional and personal needs, including access to PPE, potential exposure to COVID-19, access to expanded childcare, and additional training if redeployed to new units [11]. Shanafelt et al. summarized these concerns in the form of 5 general HCW requests to their institution: “hear me, protect me, prepare me, support me, and care for me.” These findings underscore the notion that psychological support is most effective and well received when embedded within a larger network of resources that meet HCW basic, professional, and personal needs, coupled with active, ongoing dialogue between HCW and hospital leadership [11].
The reviewed institutional initiatives to address HCW basic needs included food and alternative housing provisions, assistance with transportation, childcare, and expanded workplace rest spaces. A tertiary hospital in China adopted some unique strategies, helping staff film their daily work routines to share with family members who were concerned with their relatives’ safety, along with training HCW on how to manage psychological problems experienced by patients [10]. One institution even piloted a “PPE portrait” program, whereby HCW received self-portrait picture stickers that could be placed over their PPE to lessen the obstacles PPE introduce in connecting and building rapport with patients [12]. In another case, one institution complemented expanded education on PPE with access to expedited COVID-19 testing [13]. Another hospital readjusted shift schedules to help HCW more easily adapt to new units and protocols [14].
Expanding HCW rest spaces was another attempt to meet HCW basic needs [e.g. 15]. Relaxation spaces within the workplace can cultivate connection and casual peer debrief. One institution equipped staff rest spaces with mood boards for anonymous emotional expression, soft, relaxed furnishing, inspirational messaging, green plants, and mindfulness resources [16]. Another institution, similarly, created multisensory “recharge rooms,” which HCW could reserve individually for 15 minutes. Researchers found that these rooms were able to produce significant reductions in short term perceived stress [17].
Training-based intervention also falls under the category of harm prevention, since training programs provide HCW with skills to better cope with work-related stressors in order to reduce workplace mental health risk factors [8]. In some cases, training focused on non-psychological skills. This form of intervention does not explicitly address HCW mental health, but enhances HCW skills, knowledge, self-efficacy, and emotional/mental preparedness, which may collectively serve as protective measures for individual and organizational mental health.
In Pakistan, following simulation-based training with mock patients and mannikins, HCW reported feeling safer, less fearful, and more willing to participate in the management of COVID-19 patients [18]. A separate report in China, found that similar training programs improved measures, such as HCW assertiveness, mental preparedness, self-efficacy, internal locus of control and internal locus of responsibility [19]. Other institutions provided more traditional training programs and informational sessions to educate HCW directly on proper PPE usage [14, 20].
Illness-focused Mental Health Management
In addition to “harm prevention” tactics, institutions also sought to expand access to mental health treatment. Improved access to mental health/psychiatric treatment falls under the second arm of LaMontagne’s integrated framework, namely medical/illness-focused approach to treat mental health disorders. As reflected in the reviewed literature, in order to effectively provide mental healthcare to HCW, institutions must take multiple steps, which include promoting mental health literacy, providing individuals with tools to self-identify common symptoms of mental health disorders, expanding mental health resources, and encouraging the use of these resources.
Psychoeducational training can be considered a technique to increase mental health literacy. Reviewed papers that reported on psychoeducational training programs integrated a broad spectrum of strategies, which ranged from comprehensive signage to online screening resources and access to in person counseling and on-site rounding by mental health experts [21]. These programs were complemented by virtual and remote resources, which broadened mental health care outreach. Comprehensive psychological support programs aimed to normalize psychological distress following crisis and improve HCW coping skills [15].
Remote mental health resources are another technique described in some papers included in the review to broaden access to psychological support in order to encourage greater utilization of available resources. Telephone hot line services were one type of remote mental health care service offered, allowing HCW to connect with mental health experts and receive immediate emotional assistance, especially critical for individuals facing acute distress during traumatic workplace-related events [21]. Remote virtual and online modalities also more easily facilitate dissemination of mental health education material and resources to staff. Although the strength of digital learning platforms relative to other modalities has not been rigorously investigated, these modalities may reduce logistical and financial barriers, as well as provide users more flexibility and access to immediate help through portable, anonymous mechanisms [22]. In the UK, Blake et al. designed a freely accessible online training package comprised of slides, which provide information on HCW moral injury, decision making, and links to external self-care and mindfulness resources. By giving HCW users the option to pick which information to review at their own pace, program creators sought to design a more personalized educational experience [23].
In France, experts also utilized the online program modality, but focused on teaching HCW about effective coping techniques. These investigators are evaluating the efficacy of an online cognitive behavioral therapy program, which is organized into 7 sessions, over the course of 8 weeks, and reviews topics like mindfulness, self-compassion, and emotional regulation [24]. Another group of researchers is evaluating the efficacy of an online Sudarshan Kriya Yoga program, consisting of self-paced online modules and interactive online sessions, through which HCW will learn breathing control techniques. The results to the aforementioned studies, to our knowledge, have not yet been published [24, 25].
In other countries, administrators utilized the digital modality in a distinct way, creating websites, which aimed to centralize resources on emotional well-being during the pandemic and update HCW on the latest information regarding COVID-19. Investigators in Spain gathered recommendations and support resources in a free website that was made accessible to HCW in a variety of languages. Resources ranged from daily news updates on COVID-19 to individual counseling outlets, and an assessment tool to self-monitor stress levels. The frequency with which users conducted self-assessments correlated with the intensity of the outbreak, particularly in territories where COVID-19 incidence and mortality were higher [26]. In China, the national anesthesiology society created an online platform through which anesthesiologists could get free mental health consultations and access an online forum to receive answers from experts on infection control [27]. In other cases, new websites were designed for the purpose of consolidating institution-wide communication and access to mental health services and resources [28].
Recently, mobile health applications have also gained popularity as a tool for promoting personal health. During the COVID-19 pandemic, this format has also been adopted to develop mental health-focused applications for HCW. In Spain, administrators are assessing the efficacy of an app for HCW, which focuses on CBT and mindfulness techniques and includes content on emotional skills, lifestyle behavior, work stress and burnout, social support, and practical tools. In addition to educational material, users also have access to mental health screening tools. The app also includes “temporal modules,” through which users will be prompted to answer questions, which are used to monitor mental health status and deliver personalized mental health resources based on user responses [22].
Another team of experts in New York City created a mental health-focused app, which included resilience-focused exercises that tracked user progress and provided personalized feedback. Users also had access to a private digital journal and various mental health surveys, which were intended as tools to self-monitor emotional and mental states [29]. Similarly, a group based in Australia created a mobile app that collected user feedback and utilized an experience-based co-design approach to more effectively meet HCW needs. App designers conducted phone interviews and co-design workshops with HCW, and asked HCW for photos on the frontlines to capture their emotional experience [30]. More data is necessary to evaluate the efficacy of these mental health focused mobile health apps. Nevertheless, these initiatives offer innovative tools that allow for broader dissemination of mental health support.
Collectively, these reviewed initiatives, align with LaMontagne et al.’s principles that illness focused mental health programs are most effective when they adopt a comprehensive approach that minimizes obstacles to receival of care as well as provide alternative platforms through which individuals can independently self-screen and/or seek support.
Promoting Positivity
In addition to the importance of providing resources that utilize an illness focused approach to treat mental health issues, LaMontagne et al. reinforces the importance of promoting positivity among workers and acknowledging HCW strengths. This can be achieved through ensuring that workers find meaning within the workplace setting and actively build positive work relationships. A common approach observed across institutional programs to promote positivity among HCW involved including HCW in the process of intervention design through town halls, frequent feedback sessions, or co-design workshops. In establishing HCW as important stakeholders in the design and execution of interventions, program administrators argue that HCW may also find more meaning and purpose in their professional responsibilities.
Peer support intervention is another method that promotes positivity and HCW strengths within the workplace. Rather than solely focus on mental health on an individual-level, peer support helps promote mental health on an organizational level through encouraging stronger social support circles. Data, in fact, suggests that even informal peer support lessens HCW need for other more formal mental healthcare interventions [31]. Strong camaraderie built around common goals makes peer interventions especially powerful in helping individuals emerge from shared traumatic experiences.
“Schwartz Rounds” may be considered a particular form of peer support, typically structured as a monthly 60-minute session with a panel of 3-4 staff members, who share personal experiences followed by a group-wide discussion facilitated by a trained expert. Schwartz Rounds have been previously proven effective in improving HCW patient compassion, teamwork, and psychosocial well-being. Experts have suggested that shorter interval Schwartz Round-like interventions could improve HCW mental health during the COVID-19 pandemic [32, 33]. A medical institution in NYC created a peer support program closely resembling the Schwartz Rounds structure. Trained facilitators led department-specific, virtual 30-minute sessions, whereby clinical and non-clinical staff reflected on workplace challenges and resilience-enhancing techniques based on principles adopted from cognitive behavioral and group-problem solving therapy. Larger virtual town hall sessions were followed by dedicated question and answer time focused on stress/anxiety management, trauma, loss, grief, self-care, and emotional well-being strategies. Researchers note that these open discussion formats also allowed facilitators to stay updated on HCW evolving concerns, and adjust their support programs in a timely manner [34]. Another New York City medical institution developed a peer co-led workshop, consisting of facilitated discussion around evidence-based resilience factors. These workshops were one cornerstone of a Center that was founded in order to sustainably provide resilience-focused education and mental health supports to its institutions’ employees [35]. In China, administrators offered HCW a Balint group session once a week, which, similar to Schwartz Rounds, provides structured peer support under the guidance of a trained facilitator [36].
Unlike the peer support initiatives above, whereby peer support discussions are facilitated by trained facilitators, other institutions created peer support programs that independently paired HCW based on personal and professional similarities. At one institution, experts adopted the “Battle Buddy” system from the United States Armed Forces. In this model, each HCW is provided with pocket cards containing resources on how to best support each other through potential stressors their peers may encounter. This framework has already been proven effective within the military setting where it was shown to reduce army suicide rates [37]. If battle buddies recognize their peers are experiencing more severe psychological symptoms or distress they can encourage one another to seek professional support through the additional institutional mental health resources available [38]. In all the aforementioned reviewed peer support initiatives, HCW also had access to formal mental health counseling.
Social media has also played an instrumental role in the development of HCW peer support groups by connecting the global healthcare community and providing broad access to mental health treatment. A medical institution in Iran used social media to connect trained senior medical students with junior medical students, in order to help the younger students better cope with the stress of the COVID-19 pandemic [39]. Moreover, a group of international Chinese-speaking mental health experts, established a peer-to-peer psychological support program and crisis intervention for Wuhan HCW through the popular social media application, WeChat. Volunteer mental health experts provided positive affirmation via the application and disseminated psychological support tools. HCW were encouraged to reach out for support through the application either by using the group chat or individually contacting one of the volunteers on duty [40]. The promotion of positive well-being through peer support may also promote a sense of self sufficiency and autonomy among HCW, leading to greater sense of job control, which is another important risk factor for mental health problems [8].
Discussion
The high prevalence of stress and burnout among HCW has existed long before COVID-19; however, COVID-19 has brought to light the acuity and prevalence of emotional distress complications among HCW. Although in this review we have sought to organize the reviewed interventions under the distinct categories presented by LaMontagne et al. as evidenced in Table 1, many of these interventions actually fall under more than one category. Evidently, effective interventions require comprehensive methodologies that integrate multiple aspects from each thread of LaMontagne et al.’s framework, which are in fact highly complementary and intertwined.
Although experts can learn from previous disasters and public health crises responses, the severity of the COVID-19 pandemic exposes HCW to unique, chronic stressors, which require medical institutions to respond and support their workers in new and innovative ways that may not have been necessary beforehand [41]. Moreover, to minimize physical gatherings, institutions are faced with the challenge of fostering community and connection while adhering to infection control/prevention guidelines. Even with the advent of vaccinations, this remains a persistent challenge as many individuals remain unvaccinated and scientists continue to evaluate the efficacy of vaccines in conferring long term immunity against SARS-COV-2 variants.
Administrators also need to ensure that HCW actively engage with the available resources. Generally, independent of the pandemic, physicians have been found to experience high rates of mental health problems yet show increased reluctance to seek assistance [42]. To carry on professional responsibilities amidst difficult workplace-related events like death and disease, HCW, oftentimes depersonalize the situation, which may be detrimental to their long-term emotional well-being [11, 42]. HCW reluctance to seek help has also been attributed to concerns over workplace stigma and negative professional consequences [42].
Similar behaviors by HCW have already been demonstrated during COVID-19, as reflected in the underutilization of psychological support resources by hospital staff [28]. Chen et al. recounts HCW’s reluctance to use telephone hotline services and online mental health courses, with HCW stating that they preferred hospital administration prioritize resources that would meet their more basic needs over their psychological well-being [10]. In another medical institution, among the network of support services, individual support programs were particularly underutilized, with many individuals who expressed interest in counseling ultimately not following through with treatment. HCW cited privacy concerns and fear of stigma as reasons for not completing treatment [28]. Other HCW also cited logistical challenges and feared their professional aptitude would be challenged if they sought care [21].
Data on HCW participation and efficacy of interventions may not be generalizable across countries, since cultural differences may impact attitude towards mental healthcare; however, the reports presented here span multiple countries and underscore the global challenge of engaging HCW in psychological support services. Programs, evidently, must proactively reach out and meet HCW’s where they are, recognizing that intervention cannot be a “one size fits all.” HCW who exhibit more consistent avoidant behaviors or those who are at greater risk of experiencing mental health complications, for example, may require tailored forms of care. On-site presence of mental health experts has also proven useful in promoting treatment seeking behaviors [21].
Engagement is also dependent on providing HCW with the ability and time to participate in the provided programs. Carving out this time may seem especially challenging during public health crises requiring longer shifts and additional HCW redeployments. Nevertheless, scheduling short sessions during HCW breaks or providing remote opportunities for mental health support are two potential solutions proposed in the literature.
Ultimately, the promotion of HCW engagement may rely on the promotion of “workplace mental health literacy,” a concept that is also explored by LaMontagne et al. [8]. Advocating for workplace mental health literacy, defined by LaMontagne et al. as the “knowledge, beliefs, and skills that aid in prevention of mental illness” and “recognition, treatment, rehabilitation, and return of workers affected by mental illness” in the workplace, is inextricably linked to the workplace culture, which is largely shaped by the leadership [8]. Visible leaders who reinforce the positive reasons for participation in mental health interventions and participate in such programs themselves, foster a “culture of health” within the workplace [43]. Leadership must also foster a workplace culture that is amenable to change. Consequently, we extrapolate that effective, transparent, trustworthy, and supportive leadership is at the foundation of individual, team, and organizational resilience. Surveys during the pandemic revealed that HCW required assurance from leadership that their organization supports their well-being [11]. Support from all levels of leadership is key in establishing a sense of cohesion, trust building, and respect across the organization [44]. Leaders must listen to the HCW concerns, provide assurance that concerns are heard and addressed through targeted support, acknowledge the moral distress, and encourage HCW to use available emotional support resources. Support is more effective when leadership makes themselves visible to employees by physically visiting and checking in on HCW where they work. Leadership is a critical factor in securing long standing mental health interventions, which will be continually developed and executed even when health systems return to pre-pandemic work schedules [11].
In addressing HCW mental health, it is also critical that administrators acknowledge the diverse experience of stressors in the workplace setting, which few articles considered. Pappa et al. shows that anxiety and depression is more prevalent among nurses compared to doctors, perhaps partially due to their higher workloads and distinct occupational responsibilities, which necessitate more sustained, direct contact with patients and longer time spent in patient wards [45, 46]. While many physicians were able to work remotely, most nurses could not perform their responsibilities virtually [28]. Multidisciplinary team activities can encourage HCW to exercise empathy for their colleagues’ unique challenges. Pappa et al. also reports that female HCW more frequently experience anxiety and depression relative to their male counterparts [45, 46]. Lai et al. reports similar findings [2]. Lopez-Atanes et al. assert that mental health interventions must also recognize and actively address the inequalities between male and female HCW and that initiatives must be sensitive to gender specific stressors. Experts recommend incorporating strategies that empower women and address gender bias and expectations or demands [47].
The COVID-19 pandemic has also deepened pre-existing disparities in healthcare access and quality, brought to light by the disproportionate impact of COVID-19 on minorities and lower socioeconomic status communities. Concurrent events of police brutality and racial injustices have exacerbated these inequities. COVID-19 has also induced a resurgence of xenophobia and stigmatization against Asian Americans [48]. HCW may also experience the emotional and psychological effects of xenophobia and racism as both perpetrators or victims [49]. Experts at a New Haven medical system recount that long standing racial issues within the institution were heightened during the pandemic and increased feelings of distress among HCW, particularly individuals of color. In response, administrators refocused support efforts to include topics on racism and health disparities [28]. As evidenced by the HCW experience at this New Haven medical institution, issues of racial and social inequality can be detrimental to HCW mental health and must also be addressed as part of institutional psychological support programs. Debriefing and peer discussion formats may also be a useful space to moderate discussions regarding diversity and inclusion within the workforce.
Lastly, another important theme which was not addressed by many of the papers was the importance of setting up mechanisms that guarantee intervention longevity and sustainability. LaMontagne et al. also alludes to the importance of assessing longer term outcomes of workplace interventions [8]. The lack of long-term evaluations or consideration in the literature we reviewed may be attributed to the rapidity with which administrators sought to share the latest information from their institutions during the pandemic. The additive impact of persistent stressors has led to many HCW experiencing chronic stress. Individuals will respond and recuperate in varying ways and administrations must have the infrastructure in place to identify people with persisting stress-related problems and individuals at increased risk. Leadership must extend psychological interventions and support beyond the pandemic. Long term resilience-building will be reinforced through communicating the importance of adhering to a healthy lifestyle, diet and exercise habits along with reengaging in activities that bring individuals a greater sense of personal meaning. Once the crisis is over, supervisors must ensure that time is granted to reflect and learn from the previous experience. Active monitoring of staff should continue, and mental health staff should look out for therapeutic avoidance.
Lack of attention to sustainability may also derive from the fact that, to date, the interventions have occurred within the paradigm of disaster mental health rather than occupational mental health. The much-needed focus on HCW mental health has been driven by the pandemic, but when the pandemic recedes along with the adrenalin, what will become of the impressive amount of programming we have reviewed? As reflected in Fig. 2, the programs do fulfill much of LaMontagne et al.’s integrated approach to occupational mental health; however, it is yet to be seen how these programs will evolve or persist to address worker mental health long-term regardless of cause.
Conclusion
So long as mental health programming is tied to COVID-19 in how it is conceived and even funded, we will fail to sustain and transform it for the new normal of a post-pandemic world. As the pandemic recedes, or at least experience and science has enabled society to begin to stabilize, attention should be directed towards the long-term future of the mental health interventions it has spawned. The related themes of involving employees in program design and of employee engagement may be the key to this, helping to ensure that programs are less pandemic-centered and more employee-centered.
Ultimately, what will be remembered historically as a somber time, may help catalyze positive, transformative change within the sphere of HCW mental health. Beyond fiscal support, for psychological interventions to be well received and utilized by HCW, institutions must integrate these programs into the employees’ experience, starting from the entry stage, all the way up to the leadership and management. Effective interventions must continue to address the different dimensions of occupational mental health and be easily adaptable, flexible, and aware of the unique HC setting needs. The COVID-19 pandemic presents a critical opportunity to improve the scale and cost-effectiveness of various HCW mental health interventions and rethink and expand of access to care both within and outside the HCW community.
Biographies
Eden David, BA
graduated from Columbia University with a major in Neuroscience and is currently a second-year medical student at Icahn School of Medicine at Mount Sinai.
Jonathan M. DePierro, PhD
is an Assistant Professor of Psychiatry at the Icahn School of Medicine at Mount Sinai and the Clinical and Research Director of Mount Sinai’s Center for Stress, Resilience and Personal Growth.
Deborah B. Marin, MD
is the George and Marion Sokolik Blumenthal Professor of Psychiatry at the Icahn School of Medicine at Mount Sinai. She is the Director of Mount Sinai’s Center for Stress, Resilience, and Personal Growth, the Center for Spirituality and Heath, and of the Ombuds Office.
Vanshdeep Sharma, MD
is the Director for Quality Assurance and Performance Improvement at Mount Sinai’s Center for Stress, Resilience and Personal Growth, Medical Director of the Center of Spirituality and Health, and the Chair of the committee for Physician Wellness for the Mount Sinai Health System. He is an Associate Professor of Psychiatry at the Icahn School of Medicine at Mount Sinai.
Dennis S. Charney, MD
is the Anne and Joel Ehrenkranz Dean of the Icahn School of Medicine at Mount Sinai and President for Academic Affairs for the Mount Sinai Health System.
Craig L. Katz, MD
is a Clinical Professor of Psychiatry, Medical Education, System Design & Global Health at the Icahn School of Medicine at Mount Sinai.
Declarations
Ethics Approval
Not applicable.
Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Informed Consent
Not applicable.
Research Involving Human Participants and/or Animals
Not applicable.
Conflicts of Interest/Competing Interests
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests. Dr. Dennis Charney is named co-inventor on an issued patent in the United States, and several issued patents outside the U.S. filed by the Icahn School of Medicine at Mount Sinai (ISMMS), for the use of ketamine as a therapy for PTSD. This intellectual property has not been licensed. In addition, Dr. Charney is named co-inventor on several issued U.S. patents, and several pending U.S. patent applications, filed by ISMMS for the use of ketamine as a therapy for treatment-resistant depression and suicidal ideation. ISMMS has entered into a licensing agreement with Janssen Pharmaceuticals, Inc. and it has and will receive payments from Janssen under the license agreement related to these patents. As a co-inventor, Dr. Charney is entitled to a portion of the payments received by the ISMMS. Since SPRAVATO (esketamine) has received regulatory approval for TRD, ISMMS and Dr. Charney as its employee and a co-inventor, will be entitled to additional payments, under the license agreement. Dr. Charney is named co-inventor on a patent application filed by the ISMMS for the use of intranasally administered neuropeptide Y for the treatment of mood and anxiety disorders. This intellectual property has not been licensed. Dr. Charney is a named co-inventor on several patents filed by ISMMS for a cognitive training intervention to treat depression and related psychiatric disorders. ISMMS has entered into a licensing agreement with Click Therapeutics, Inc., and has and will receive payments related to the use of this cognitive training intervention for the treatment of psychiatric disorders. In accordance with the ISMMS Faculty Handbook, Dr. Charney has received a portion of these payments and is entitled to a portion of any additional payments that the medical school might receive from this license with Click Therapeutics. Dr. Craig L. Katz is a paid consultant with Advanced Recovery Systems and the RANE Crisis Network. Eden David and Drs. Jonathan DePierro, Deborah B. Marin, and Vanshdeep Sharma report no financial relationships with commercial interests.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Eden David, Email: eden.david@icahn.mssm.edu.
Jonathan M. DePierro, Email: jonathan.depierro@mssm.edu
Deborah B. Marin, Email: deborah.marin@mssm.edu
Vanshdeep Sharma, Email: vansh.sharma@mountsinai.org.
Dennis S. Charney, Email: dennis.charney@mssm.edu
Craig L. Katz, Email: craig.katz@mssm.edu
References
- 1.Feingold JH, et al. Psychological Impact of the COVID-19 Pandemic on Frontline Health Care Workers During the Pandemic Surge in New York City. Chronic Stress (Thousand Oaks) 2021;5:2470547020977891. doi: 10.1177/2470547020977891. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lai J, et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA Netw Open. 2020;3(3):e203976. [DOI] [PMC free article] [PubMed]
- 3.Serrano-Ripoll MJ, et al. Impact of viral epidemic outbreaks on mental health of healthcare workers: a rapid systematic review and meta-analysis. J Affect Disord. 2020;277:347–357. doi: 10.1016/j.jad.2020.08.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Buselli R, et al. Mental health of health care workers (HCWs): a review of organizational interventions put in place by local institutions to cope with new psychosocial challenges resulting from COVID-19. Psychiatry Res. 2021;299:113847. [DOI] [PMC free article] [PubMed]
- 5.Pollock A, et al. Interventions to support the resilience and mental health of frontline health and social care professionals during and after a disease outbreak, epidemic or pandemic: a mixed methods systematic review. Cochrane Database Syst Rev. 2020;11:Cd013779. [DOI] [PMC free article] [PubMed]
- 6.Walton M, Murray E, Christian MD. Mental health care for medical staff and affiliated healthcare workers during the COVID-19 pandemic. Eur Heart J Acute Cardiovasc Care. 2020;9(3):241–247. doi: 10.1177/2048872620922795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Zaçe D, et al. Interventions to address mental health issues in healthcare workers during infectious disease outbreaks: A systematic review. J Psychiatr Res. 2021;136:319–333. doi: 10.1016/j.jpsychires.2021.02.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.LaMontagne AD, et al. Workplace mental health: developing an integrated intervention approach. BMC Psychiatry. 2014;14:131. doi: 10.1186/1471-244X-14-131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Maslow A. Motivation and personality. New York, NY 2nd Harper & Row. 1970.
- 10.Chen Q, et al. Mental health care for medical staff in China during the COVID-19 outbreak. Lancet Psychiatry. 2020;7(4):e15–e16. doi: 10.1016/S2215-0366(20)30078-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shanafelt T, Ripp J, Trockel M. Understanding and Addressing Sources of Anxiety Among Health Care Professionals During the COVID-19 Pandemic. JAMA. 2020;323(21):2133–2134. doi: 10.1001/jama.2020.5893. [DOI] [PubMed] [Google Scholar]
- 12.Brown-Johnson C, et al. PPE Portraits-a Way to Humanize Personal Protective Equipment. J Gen Intern Med. 2020;35(7):2240–2242. doi: 10.1007/s11606-020-05875-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Orsini E, et al. Lessons on Outbreak Preparedness From the Cleveland Clinic. Chest. 2020;158(5):2090–2096. doi: 10.1016/j.chest.2020.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hong X, et al. Stress and psychological impact of the COVID-19 outbreak on the healthcare staff at the fever clinic of a tertiary general hospital in Beijing: a cross-sectional study. BJPsych Open. 2021;7(3):e76. [DOI] [PMC free article] [PubMed]
- 15.Gonzalez A, et al. Supporting health care workers during the COVID-19 pandemic: Mental health support initiatives and lessons learned from an academic medical center. Psychol Trauma. 2020;12(S1):S168–s170. doi: 10.1037/tra0000893. [DOI] [PubMed] [Google Scholar]
- 16.Saqib A, Rampal T. Quality improvement report: setting up a staff well-being hub through continuous engagement. BMJ Open Qual. 2020;9(3). [DOI] [PMC free article] [PubMed]
- 17.Putrino D, et al. Multisensory, nature-inspired recharge rooms yield short-term reductions in perceived stress among frontline healthcare workers. Front Psychol. 2020;11:560833. [DOI] [PMC free article] [PubMed]
- 18.Khan JA, Kiani MRB. Impact of multi-professional simulation-based training on perceptions of safety and preparedness among health workers caring for coronavirus disease 2019 patients in Pakistan. J Educ Eval Health Prof. 2020;17:19. doi: 10.3352/jeehp.2020.17.19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Cheung VK, et al. Investigating effects of healthcare simulation on personal strengths and organizational impacts for healthcare workers during COVID-19 pandemic: A cross-sectional study. Integr Med Res. 2020;9(3):100476. [DOI] [PMC free article] [PubMed]
- 20.Ripp J, Peccoralo L, Charney D. Attending to the Emotional Well-Being of the Health Care Workforce in a New York City Health System During the COVID-19 Pandemic. Acad Med. 2020;95(8):1136–1139. doi: 10.1097/ACM.0000000000003414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Gray M, et al. A Mental Health PPE model of proactive mental health support for frontline health care workers during the COVID-19 pandemic. Psychiatry Res. 2021;299:113878. [DOI] [PMC free article] [PubMed]
- 22.Serrano-Ripoll MJ, et al. Effect of a mobile-based intervention on mental health in frontline healthcare workers against COVID-19: Protocol for a randomized controlled trial. J Adv Nurs. 2021. [DOI] [PMC free article] [PubMed]
- 23.Blake H, et al. Mitigating the psychological impact of COVID-19 on healthcare workers: a digital learning package. Int J Environ Res Public Health. 2020;17(9). [DOI] [PMC free article] [PubMed]
- 24.Weiner L, et al. Efficacy of an online cognitive behavioral therapy program developed for healthcare workers during the COVID-19 pandemic: the REduction of STress (REST) study protocol for a randomized controlled trial. Trials. 2020;21(1):870. doi: 10.1186/s13063-020-04772-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Lai KSP, et al. Breath Regulation and yogic Exercise An online Therapy for calm and Happiness (BREATH) for frontline hospital and long-term care home staff managing the COVID-19 pandemic: A structured summary of a study protocol for a feasibility study for a randomised controlled trial. Trials. 2020;21(1):648. doi: 10.1186/s13063-020-04583-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mira JJ, et al. Preventing and addressing the stress reactions of health care workers caring for patients with COVID-19: Development of a digital platform (Be + Against COVID). JMIR Mhealth Uhealth. 2020;8(10):e21692. [DOI] [PMC free article] [PubMed]
- 27.Zhang HF, et al. Response of Chinese Anesthesiologists to the COVID-19 Outbreak. Anesthesiology. 2020;132(6):1333–1338. doi: 10.1097/ALN.0000000000003300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Krystal JH, et al. Mobilizing an institutional supportive response for healthcare workers and other staff in the context of COVID-19: The Yale experience. Gen Hosp Psychiatry. 2021;68:12–18. doi: 10.1016/j.genhosppsych.2020.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Golden EA, et al. A resilience-building app to support the mental health of health care workers in the COVID-19 era: Design process, distribution, and evaluation. JMIR Form Res. 2021. [DOI] [PMC free article] [PubMed]
- 30.Lewis M, et al. Rapid design and delivery of an experience-based co-designed mobile app to support the mental health needs of health care workers affected by the COVID-19 pandemic: Impact evaluation protocol. JMIR Res Protoc. 2021;10(3):e26168. [DOI] [PMC free article] [PubMed]
- 31.Gerada C. Healing doctors through groups: creating time to reflect together. Br J Gen Pract. 2016;66(651):e776–8. doi: 10.3399/bjgp16X687469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Flanagan E, et al. Reflection for all healthcare staff: A national evaluation of Schwartz Rounds. J Interprof Care. 2020;34(1):140–142. doi: 10.1080/13561820.2019.1636008. [DOI] [PubMed] [Google Scholar]
- 33.Saniasiaya J, Ramasamy K. Schwartz rounds for healthcare personnel in coping with COVID-19 pandemic. Postgrad Med J. 2020;96(1137):425. doi: 10.1136/postgradmedj-2020-137806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Mellins CA, et al. Supporting the well-being of health care providers during the COVID-19 pandemic: The CopeColumbia response. Gen Hosp Psychiatry. 2020;67:62–69. doi: 10.1016/j.genhosppsych.2020.08.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.DePierro J, et al. Mount Sinai’s center for stress, resilience and personal growth as a model for responding to the impact of COVID-19 on health care workers. Psychiatry Res. 2020;293:113426. [DOI] [PMC free article] [PubMed]
- 36.Cheng W, Zhang F, Liu Z, Zhang H, Lyu Y, Xu H, Hua Y, Gu J, Yang Z, Liu J. A psychological health support scheme for medical teams in COVID-19 outbreak and its effectiveness. Gen Psychiatry. 2020;33. [DOI] [PMC free article] [PubMed]
- 37.Ramsberger PF, Mills L, Legree P. Evaluation of the buddy team assignment program. United States Army Research Institute for the Behavioral and Social Sciences. 2002.
- 38.Albott CS, et al. Battle Buddies: Rapid Deployment of a Psychological Resilience Intervention for Health Care Workers During the COVID-19 Pandemic. Anesth Analg. 2020;131(1):43–54. doi: 10.1213/ANE.0000000000004912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Rastegar Kazerooni A, et al. Peer mentoring for medical students during the COVID-19 pandemic via a social media platform. Medical Education. 2020;54(8):762–763. doi: 10.1111/medu.14206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Cheng P, et al. COVID-19 epidemic peer support and crisis intervention via social media. Community mental health journal. 2020;56(5):786–792. doi: 10.1007/s10597-020-00624-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Hall DL, et al. Responding to COVID-19 Stress: Disseminating Mind-Body Resiliency Approaches. Glob Adv Health Med. 2020;9:2164956120976554. doi: 10.1177/2164956120976554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Wessely A, Gerada C. When doctors need treatment: an anthropological approach to why doctors make bad patients. BMJ. 2013;347:f6644.
- 43.Gray P, et al. Workplace-based organizational interventions promoting mental health and happiness among healthcare workers: a realist review. Int J Environ Res Public Health. 2019;16(22). [DOI] [PMC free article] [PubMed]
- 44.Shingler-Nace A. COVID-19: When Leadership Calls. Nurse Leader. 2020;18(3):202–203. doi: 10.1016/j.mnl.2020.03.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Pappa S, et al. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: A systematic review and meta-analysis. Brain Behav Immun. 2020;88:901–907. doi: 10.1016/j.bbi.2020.05.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Liu Z, Han B, Jiang R. Mental health status of doctors and nurses during COVID-19 epidemic in China. SSRN Electron J. 2020.
- 47.López-Atanes M, Recio-Barbero M, Sáenz-Herrero M. Are women still “the other”? Gendered mental health interventions for health care workers in Spain during COVID-19. Psychological Trauma: Theory, Research, Practice, and Policy. 2020;12(S1):S243–S244. doi: 10.1037/tra0000751. [DOI] [PubMed] [Google Scholar]
- 48.Pan SW, et al. Coronavirus stigmatization and psychological distress among Asians in the United States. Ethn Health. 2021;26(1):110–125. doi: 10.1080/13557858.2020.1849570. [DOI] [PubMed] [Google Scholar]
- 49.Markey K, Zhang Y. Demystifying, recognising and combating racism during the pandemic. Br J Nurs. 2020;29(21):1266–1270. doi: 10.12968/bjon.2020.29.21.1266. [DOI] [PubMed] [Google Scholar]
- 50.Weiss DS. The Impact of Event Scale: Revised. In Cross-Cultural Assessment of Psychological Trauma and PTSD (eds Wilson JP and Tang CS): 219–38. Springer, 2007.
- 51.Wong TW, et al. The psychological impact of severe acute respiratory syndrome outbreak on healthcare workers in emergency departments and how they cope. European Journal of Emergency Medicine. 2005;12(1):13–18. doi: 10.1097/00063110-200502000-00005. [DOI] [PubMed] [Google Scholar]
- 52.Bados A, Solanas A, Andrés R. Psychometric properties of the Spanish version of depression, anxiety and stress scales (DASS). Psicothema. 2005;679-683.
- 53.García JB, et al. Banco de instrumentos básicos para la práctica de la psiquiatría clínica. Ars Médica Madrid. 2002.
- 54.Gil-Monte PR. Factorial validity of the Maslach Burnout Inventory (MBI-HSS) among Spanish professionals. Rev Saude Publica. 2005;39(1):1–8. doi: 10.1590/S0034-89102005000100001. [DOI] [PubMed] [Google Scholar]
- 55.Bastien CH, Vallières A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med. 2001;2(4):297–307. doi: 10.1016/S1389-9457(00)00065-4. [DOI] [PubMed] [Google Scholar]
- 56.Herrero R, et al. Psychometric properties of the General Self Efficacy-12 Scale in Spanish: general and clinical population samples. Compr Psychiatry. 2014;55(7):1738–43. doi: 10.1016/j.comppsych.2014.05.015. [DOI] [PubMed] [Google Scholar]
- 57.Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Intl. Journal of Human-Computer Interaction. 2008;24(6):574–594. doi: 10.1080/10447310802205776. [DOI] [Google Scholar]
- 58.Blake H, Somerset S, Evans C. Development and fidelity testing of the test@work digital toolkit for employers on workplace health checks and opt-in HIV testing. Int J Environ Res Public Health. 2020;17(1). [DOI] [PMC free article] [PubMed]
- 59.Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385–96. doi: 10.2307/2136404. [DOI] [PubMed] [Google Scholar]
- 60.Arroll B, et al. Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Ann Fam Med. 2010;8(4):348–53. doi: 10.1370/afm.1139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Zuromski KL, et al. Developing an optimal short-form of the PTSD Checklist for DSM-5 (PCL-5) Depress Anxiety. 2019;36(9):790–800. doi: 10.1002/da.22942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Vaishnavi S, Connor K, Davidson JR. An abbreviated version of the Connor-Davidson Resilience Scale (CD-RISC), the CD-RISC2: psychometric properties and applications in psychopharmacological trials. Psychiatry Res. 2007;152(2–3):293–7. doi: 10.1016/j.psychres.2007.01.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Cropley M, et al. The relation of post-work ruminative thinking with eating behaviour. Stress Health. 2012;28(1):23–30. doi: 10.1002/smi.1397. [DOI] [PubMed] [Google Scholar]
- 64.Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. 2000;31(2):73–86. doi: 10.1016/S0005-7916(00)00012-4. [DOI] [PubMed] [Google Scholar]
- 65.Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5(3):233–7. [DOI] [PubMed]
- 66.Soldatos CR, Dikeos DG, Paparrigopoulos TJ. Athens Insomnia Scale: validation of an instrument based on ICD-10 criteria. J Psychosom Res. 2000;48(6):555–60. doi: 10.1016/S0022-3999(00)00095-7. [DOI] [PubMed] [Google Scholar]
- 67.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Kroenke K, et al. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–59. doi: 10.1016/j.genhosppsych.2010.03.006. [DOI] [PubMed] [Google Scholar]
- 69.Connor KM, Davidson JR. Development of a new resilience scale: the Connor-Davidson Resilience Scale (CD-RISC) Depress Anxiety. 2003;18(2):76–82. doi: 10.1002/da.10113. [DOI] [PubMed] [Google Scholar]
- 70.Kroenke K, et al. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics. 2009;50(6):613–21. doi: 10.1176/appi.psy.50.6.613. [DOI] [PubMed] [Google Scholar]
- 71.Ghahramani S, et al. Mentoring medical students by their Peers, Three Years' experience at Shiraz Medical School. J Adv Med Educ Prof. 2019;7(3):156–157. doi: 10.30476/JAMP.2019.45018. [DOI] [PMC free article] [PubMed] [Google Scholar]