Validation of the Japanese version of Stress and Anxiety to Viral Epidemics-9 (SAVE-9) and relationship among stress, insomnia, anxiety, and depression in healthcare workers exposed to coronavirus disease 2019

Abstract

Objective

This study aimed to validate the Japanese version of the 9-item Stress and Anxiety to Viral Epidemics scale (SAVE-9) and the relationships among the stress related to viral epidemics, insomnia, anxiety, and depression.

Patients/methods

A cross-sectional questionnaire-based study was conducted online. In total, 1000 healthcare workers (579 men, 421 women; mean age: 43.11 ± 11.69 years) were asked to complete the SAVE-9, Athens Insomnia Scale, Generalized Anxiety Disorder-7 Scale, and Center for Epidemiological Studies Depression Scale. For the analysis, participants were divided into two groups: healthcare workers at a medical institution designated for COVID-19 (COVID institution) and those working at an institution not designated for COVID-19 (non-COVID institution).

Results

Item response theory analysis showed that the SAVE-9 and SAVE-6 (6-item version) had good structural validity and internal consistency (ω = 0.91 and 0.93). Correlation analysis for convergent validity showed a significant positive correlation between both the SAVE-9 and SAVE-6 and the other scales for insomnia, anxiety, and depression. In addition, both SAVE-9 and SAVE-6 scores were higher for workers in COVID institutions than for those in non-COVID institutions. Furthermore, stress related to viral epidemics was found to directly affect anxiety (β = 0.48) and depression (β = 0.25) and indirectly affect anxiety (β = 0.37) and depression (β = 0.54) via insomnia (β = 0.33).

Conclusions

This study confirmed that the reliability and validity of both the SAVE-9 and SAVE-6 and that insomnia mediated the effects of stress to viral epidemics on anxiety and depression symptoms.

1. Introduction

Since the World Health Organization declared the global coronavirus disease 2019 (COVID-19) outbreak to be a pandemic [1], humanity has been continuously threatened by COVID-19. In Japan, the Tokyo metropolitan government first issued a stay-at-home request on March 25, 2020, and the Japanese national government declared a state of emergency on April 7, 2020, and again on February 7, 2021. Further, as the number of individuals infected with COVID-19 continues to increase, people have also been required to change their lifestyles [2].

Numerous studies have examined the impact of lockdowns and lifestyle changes on both mental and physical health [[3], [4], [5], [6]]. In a web-based survey conducted during the lockdown in Italy [3], prevalence rates were 42% for insomnia, of which 17% were individuals with moderate to high symptoms, 23% for anxiety, and 25% for depression. An online survey of 1491 adults conducted in Australia reported decreased physical activity (49%), sleep quality (41%), increased alcohol consumption (27%), and smoking (7%) since the onset of the COVID-19 pandemic [6]. These factors have also been reported to exacerbate depression, anxiety, and stress [6].

Furthermore, a cross-sectional study of 1257 healthcare workers conducted in China revealed that the participants who worked in medical institutions not designated for COVID-19 treatment were at a lower risk of experiencing symptoms of distress compared with those in medical institutions that were designated for COVID-19 treatment [7]. In addition, frontline healthcare workers engaged in the direct diagnosis, treatment, and care of patients with COVID-19 have been shown to be at increased risk of exhibiting symptoms of insomnia (OR = 2.97), anxiety (OR = 1.57), depression (OR = 1.52), and distress (OR = 1.60) [7]. Moreover, a systematic review that explored insomnia, anxiety, and depression among healthcare workers during the COVID-19 pandemic reported the prevalence rates to be 22.8%, 23.2%, and 38.9%, respectively [8]. Thus, many people exposed to unknown viral infections such as COVID-19 have experienced fear, insomnia, anxiety, depression, and other psychological difficulties. In particular, it showed that insomnia is a significant predictor for the onset of depression (OR = 2.83) and anxiety (OR = 3.23) in the meta-analytic study [9]. However, there has been a lack of comprehensive measures in assessing fear of viral infection.

Recently, a scale was developed that measures anxiety related to viral epidemics [10]. The Stress and Anxiety to Viral Epidemics (SAVE-9) scale is a 9-item self-reporting questionnaire used to measure anxiety symptoms and work-related stress in response to a viral epidemic among frontline healthcare workers [10]. The SAVE-9 assesses two factors: “anxiety about the viral epidemic” (Items 1, 2, 3, 4, 5, and 8) and “work-related stress associated with the viral epidemic” (Items 6, 7, and 9). The scale has previously demonstrated high internal consistency (Cronbach's α = 0.80) and convergent validity for anxiety compared to the Generalized Anxiety Disorder-7 scale (GAD-7; r = 0.51) and depression compared to the Patient Health Questionnaire-9 (r = 0.41). Further, the “anxiety about the viral epidemic” factor, as a six-item version of the scale (SAVE-6), has been independently verified for reliability and validity in a community sample [11].

In Japan, there is currently no scale available to measure stress and anxiety related to viral epidemics and few studies exist on the mental health effects of COVID-19 [12]. In addition, there has been a lack of studies considering psychological responses among healthcare workers, especially, studies distinguishing between those who work directly with COVID-19 patients compared to those who do not. Therefore, this study aimed to develop a Japanese version of the SAVE-9 and to examine a relationship among stress related to viral epidemics, insomnia, anxiety, and depression in healthcare workers with and without direct contact with COVID-19 patients.

2. Material and methods

The study was approved by the Ethics Committee of Tokyo Kasei University (ID: Ita-2020-17, date: October 28, 2020). All study participants provided informed consent.

2.1. Participants

The data analyzed in this study were collected in December 2020. The study participants were recruited by Rakuten Research, Inc., an online marketing research company that holds the contact details of approximately 2.3 million Japanese survey respondents. An e-mail containing a link to an online questionnaire was sent to individuals selected at random and stratified by gender and age throughout Japan. The participants were 1000 healthcare workers (579 men, 421 women; mean age, 43.11 ± 11.69 years). Of them, 232 (124 men, 108 women, mean age: 47.19 ± 11.55 years) worked in medical institutions designated for COVID-19 (COVID institutions) and 768 (455 men, 313 women, mean age: 46.25 ± 11.71 years) worked in institutions not designated for COVID-19 (non-COVID institutions).

2.2. Measures

2.2.1. Demographic data

The participants were asked to provide their age, gender, occupation, and duration of employment at their current job. In addition, they were asked if they worked at a COVID institution or a non-COVID institution.

2.2.2. Japanese version of the SAVE-9

The SAVE-9 is a validated 9-item self-report questionnaire that assesses anxiety symptoms and work-related stress in response to viral epidemics [10,11]. In addition to the original version of SAVE-9, there is the 6-item (SAVE-6) previously used in community samples [11]. The scores for both the SAVE-9 and SAVE-6 are summed independent of each other, with higher scores indicating more anxiety and stress.

2.2.3. Athens Insomnia Scale

The Athens Insomnia Scale (AIS) is a validated 8-item self-report questionnaire that assesses insomnia severity [[13], [14], [15]]. The score for the AIS is summed, with higher scores indicating more severe insomnia. A cut-off score of 5.5 points for the AIS was previously determined; therefore, in the present study, respondents with AIS scores of 6 or higher were considered to have clinical insomnia [14].

2.2.4. Generalized anxiety Disorder-7 scale

The GAD-7 is a validated 7-item self-report questionnaire that assesses the severity of anxiety disorders [16,17]. The scale has been recommended to assess anxiety symptoms as listed in the DSM-5 [18]. The score for the GAD-7 is summed, with higher scores indicating more anxiety. A cut-off score of 5 has previously been determined for the GAD-7 [16]; therefore, in the present study, respondents with GAD-7 scores of 5 or higher were considered to have psychopathological anxiety.

2.2.5. Center for Epidemiological Studies Depression Scale

The Center for Epidemiological Studies Depression Scale (CES-D) is a validated 20-item self-report questionnaire that assesses depressive symptoms [19,20]. Scores for the CES-D are summed, with higher scores reflecting higher levels of depression. A cut-off score of 16 points has previously been determined for the CES-D [19]; therefore, in the present study, respondents with CES-D scores of 16 or higher were considered to have clinical depression.

2.3. Sample size

The sample size was based on a power analysis conducted for the correlation coefficients (r) between SAVE-9, GAD-7, and PHQ-9 in Chung's study [10]. Correlation coefficient between SAVE-9 and GAD-7 was 0.51 (p < 0.001) and that between SAVE-9 and PHQ-9 was 0.41 (p < 0.001) in 1019 participants. It was calculated that power (1-β) were both 1.00. Therefore, we recruited a thousand participants.

2.4. Procedure

We developed the SAVE-9-J after obtaining permission from the first and corresponding author of the original version of the SAVE-9 [10]. A back-translation procedure was used to ensure equivalence between the original English version and the translated Japanese version. First, the scale was front-translated from English into Japanese by a clinical psychologist with expertise in sleep research and sleep medicine, and a tentative version of the scale was completed. Second, the scale was back-translated from Japanese into English independently by two native speakers of both Japanese and English. The two back-translations were reviewed and confirmed to be acceptable by the original author.

2.5. Statistical analysis

Descriptive statistics were computed using R statistical software version 3.6.3 (R Project for Statistical Computing, Vienna, Austria). To confirm the prevalence of clinical anxiety, depression, and insomnia in participants working in either COVID or non-COVID institutions, we conducted an χ² test, residual analysis, and phi coefficient (φ) analysis. In general, an absolute g value of ≥0.1 indicates a small effect size, a value around 0.3 indicates a moderate effect size, and a value ≥ 0.5 indicates a large effect size [21].

We conducted item response theory (IRT) analysis using the “ltm” package [22]. An IRT analysis allows for more precise examinations of the characteristics of each question than that based on classical test theory (CTT). To clarify the availability of the IRT for the SAVE-9, the discrimination parameters (a), boundary characteristic values (b _i), and difficulty parameters ($b_i^{\prime }$) of each item were evaluated using a graded response model [23] in the present analysis.

One of the most important assumptions of the application of IRT analysis is that it is unidimensional. To confirm the unidimensionality of the SAVE-9, which is an ordinal scale, we described the shape of the scree plot and conducted a polychoric correlation analysis for SAVE-9 items and a categorical factor analysis utilizing a maximum likelihood solution method. The item response category characteristic curve (IRCCC), which relates the probability of an item response to the underlying attribute (θ) and the test information function (TIF) were described to confirm a response characteristic of each item. The internal consistency of the SAVE-9 was evaluated using the McDonald's ω coefficient, which is the best measure of internal consistency [24].

Convergent validity was evaluated by correlation analysis of the SAVE-9-J with the GAD-7, CES-D, and AIS. To evaluate concurrent validity, we compared differences in the SAVE-9-J scores between COVID and non-COVID institutions using an unpaired two-tailed Welch's t-test. We estimated the effect sizes of scales between the institutions using Hedges' g. In general, an absolute g value of ≥0.2 indicates a small effect size, a value around 0.5 indicates a moderate effect size, and a value ≥ 0.8 indicates a large effect size [21].

According to previous studies [25,26], stress related to viral epidemics affects the onset of insomnia, anxiety, and depression. In particular, insomnia has been consistently reported to be a risk factor which worsens symptoms of anxiety and depression [9]. To determine whether the stress related to viral epidemics affects anxiety and depression via insomnia, predictive validity was evaluated through structural equation modeling (SEM) using the “lavaan” package [27]. We evaluated the following fit indices: chi-square (χ2), comparative fit index (CFI), and root mean square error of approximation (RMSEA).

3. Results

3.1. Demographic characteristics

Demographic characteristics are presented in Table 1 . The most common occupation among respondents were nurses (38%), followed by doctors (35%), and others (27%), including a physical therapist, medical technologist, social worker, occupational therapist, speech-language-hearing therapist, and clinical therapist. The mean duration of employment at one's current job was 12 years. The prevalence of psychopathological insomnia in the present sample was 42.2% (COVID institutions vs. non-COVID institutions: 52% vs. 39%), anxiety was 47.1% (59% vs. 43%), and depression was 33.1% (40% vs. 31%; Table 1). The results of the χ² test and residual analysis showed that the proportions of participants with symptoms of psychopathological insomnia, anxiety, and depression working in COVID institutions were significantly larger than for those working in non-COVID institutions (all p < 0.05; Table 1).

Table 1.

Demographic data.

			Total (n = 1000)	COVID-institution (n = 232)	non-COVID-institution (n = 768)	χ2 values (df)	φ
Age, M (SD)			46.25 (11.71)	43.11 (11.69)	47.19 (1155)
Gender, n (%)	M		579 (58)	124 (53)	455 (59)
	F		421 (42)	108 (47)	313 (41)
Doctor, n (%)			349 (35)	94 (41)	255 (33)
Nurse, n (%)			377 (38)	93 (40)	284 (37)
Others, n (%)			274 (27)	45 (19)	229 (30)
Insomnia (AIS ≥ 6), n (%)	N		578 (58)	111 (48)	467 (61)	12.27 (1)¶	−0.11
	Y		422 (42)	121 (52)	301 (39)
Anxiety (GAD-7 ≥ 5), n (%)	N		529 (53)	94 (41)	435 (57)	18.59 (1)¶	−0.14
	Y		471 (47)	138 (59)	333 (43)
Depression (CES-D ≥16), n (%)	N		669 (67)	140 (60)	529 (69)	5.86 (1)∗	−0.08
	Y		331 (33)	92 (40)	239 (31)

Note. AIS = Athens Insomnia Scale. CES-D = Center for Epidemic Studies Depression Scale. GAD = Generalized Anxiety Disorder. F = Female. M = Male. N = No. Y = Yes.

∗p < 0.05.

¶p < 0.001.

3.2. Structural validity using IRT analysis

From the shape of a scree plot, the first eigenvalue (5.2) was found to be much greater than the others (<1.3), suggesting that a unidimensional model was reasonable for this example. The results of a polychoric correlation analysis showed high correlation coefficients among Items 1, 2, 3, 4, 5, and 8 (r = 0.54 to 0.87) and among Items 6, 7, and 9 (r = 0.50 to 0.66; Supplement 1). In addition, the results of the categorical factor analysis showed that the contribution ratio was 62.3% and factor loadings ranged from 0.53 to 0.95. The communality of Items 1, 2, 3, 4, 5, and 8 was high (range: 0.57 to 0.91); however, it was relatively low for Items 6,7, and 9 (range: 0.28 to 0.38; Supplement 2).

The parameters of discrimination (a), boundary characteristic values (b _i), and difficulty ($b_i^{\prime }$) are shown in Supplement 3. Although all discrimination parameter values were over 1.0, the values for Items 6 (a = 1.384), 7 (a = 1.068), and 9 (a = 1.173) were lower than those of the other items (range: 1.970 to 4.195). By comparing the IRCCCs in Fig. 1 , it was found that the IRCCCs were sharper for Items 1, 2, 3, 4, 5, and 8 and flatter for Items 6, 7, and 9. The total information was 69.12, and there was 68.41 (98.97%) information between the characteristic values θ of −4 and 4. Based on these results, the reliability and validity of the SAVE-9 and SAVE-6 were further examined.

Fig. 1.

Item response category characteristic curve for each item of the SAVE-9. Each of the five trace lines represents one of the five response options that were used to assess for SAVE-9: 1 = “never”, 2 = “rarely”, 3 = “sometimes”, 4 = “often”, and 5 = “always”. The first answering option labeled 1, represents a greater likelihood of having a lower degree of stress to viral epidemics, and this likelihood decreases with increasing stress levels. Trace line 3 peaks in the mid-range of stress to viral epidemics, and trace line 5 peaks in the high stress range. SAVE = Stress and Anxiety of Viral Epidemics.

3.3. Reliability

McDonald's ω values for SAVE-9 and SAVE-6 were high (ω [95% CI] = 0.91 [0.89, 0.92] and 0.93 [0.92, 0.94], respectively).

3.4. Convergent and concurrent validity

The correlation analysis for convergent validity showed a significant positive correlation (r) between the SAVE-9 and the AIS (r [95% CI] = 0.33 [0.28, 0.39]), GAD-7 (r = 0.60 [0.56, 0.64]), and CES-D (r = 0.43 [0.38, 0.48]; all p < 0.001), and between the SAVE-6 and the AIS (r = 0.30 [0.24, 0.35]), GAD-7 (r = 0.54 [0.49, 0.58]), and CES-D (r = 0.35 [0.29, 0.40]; all ps < 0.001; Table 2 ).

Table 2.

The result of correlation analysis between scales.

	SAVE-9	SAVE-6	AIS	GAD-7	CES-D
SAVE-9	1
SAVE-6	0.95 [0.94, 0.96]	1
AIS	0.33 [0.28, 0.39]	0.30 [0.24, 0.35]	1
GAD-7	0.60 [0.56, 0.64]	0.54 [0.49, 0.58]	0.53 [0.48, 0.57]	1
CES-D	0.43 [0.38, 0.48]	0.35 [0.29, 0.40]	0.62 [0.58, 0.66]	0.68 [0.64, 0.71]	1

Note. AIS = Athens Insomnia Scale. CES-D = Center for Epidemic Studies Depression Scale. GAD = Generalized Anxiety Disorder. SD = standardized deviation.

The values in parentheses represent the 95% confidential interval.

All p-values of coefficients are less than 0.001.

The means and standard deviations for scores in the COVID institutions (n = 232) and non-COVID institutions (n = 768) are presented in Table 3 . The results of Welch's t-tests for concurrent validity showed that total scores for the SAVE-9 (t ₃₅₅ = 3.69, p < 0.001), SAVE-6 (t ₃₆₈ = 3.65, p < 0.001), AIS (t ₃₇₄ = 3.15, p = 0.002), GAD-7 (t ₃₃₄ = 4.83, p < 0.001), and CES-D (t ₃₅₁ = 2.37, p = 0.002) were significantly higher in the COVID institutions than in the non-COVID institutions. Effect sizes were small for all scales (Hedges' g: 0.19 to 0.40; Table 3).

Table 3.

Descriptive statistics for all scales in each medical institution.

	Total		COVID-institution		Non-COVID-institution		Hedges' g [95% CI]
	Mean	SD	Mean	SD	Mean	SD
SAVE-9	18.07	8.17	19.88	8.69	17.53	7.93	0.29 [0.14, 0.44]
SAVE-6	13.77	5.92	15.03	6.08	13.38	5.82	0.28 [0.13, 0.43]
AIS	5.40	4.64	6.25	4.70	5.14	4.59	0.24 [0.09, 0.39]
GAD-7	5.50	5.84	7.26	6.56	4.97	5.50	0.40 [0.25, 0.55]
CES-D	14.60	9.58	15.97	10.34	14.18	9.31	0.19 [0.04, 0.33]

Note. AIS = Athens Insomnia Scale. CES-D = Center for Epidemic Studies Depression Scale. CI = confidential interval. GAD = Generalized Anxiety Disorder. SD = standardized deviation.

3.5. Predictive validity

The SEM results showed the hypothesized model (Fig. 2 ) had good fit for both the SAVE-9 ($X_6^{\mathrm{2}}$ = 1593.45, Hedges' p < 0.001, CFI = 1.000, and RMSEA = 0.000) and SAVE-6 ($X_6^{\mathrm{2}}$ = 1484.27, p < 0.001, CFI = 1.000, and RMSEA = 0.000). The results also showed that both the SAVE-9 and SAVE-6 positively affected AIS (β = 0.33 and 0.30, p < 0.001, respectively), and the AIS positively affected the GAD-7 (β = 0.37 and 0.41, p < 0.001, respectively) and CES-D (β = 0.54 and 0.57, p < 0.001, respectively), even after accounting for the direct effects of the SAVE-9 and SAVE-6 on the GAD-7 (β = 0.48 and 0.42, p < 0.001, respectively) and CES-D (β = 0.25 and 0.18, p < 0.001, respectively).

Fig. 2.

Impact of stress related to viral epidemics on insomnia, anxiety, and depression by using structural equation modeling. It indicates that the direct pathway through which stress to viral epidemics affects both symptoms of anxiety and depression and the epidemics indirect pathway through which the stress affects those symptoms via insomnia. The coefficients in parentheses represent the coefficients of the SAVE-6. AIS = Athens Insomnia Scale; CES-D = Center for Epidemiological Studies Depression Scale; GAD = generalized anxiety disorder; SAVE = Stress and Anxiety of Viral Epidemics.All p-values of coeffcients are lower than 0.001.

4. Discussion

The study aimed to develop a Japanese version of the SAVE-9 and assess its reliability and validity. The findings indicated that both the 9-item (SAVE-9) and 6-item (SAVE-6) scales were viable, as both were shown to have high reliability and validity.

In the present study, high proportions of respondents demonstrated clinical levels of insomnia (42%), anxiety (47%), and depression (33%), which was similar to the findings of previous studies [[3], [4], [5], [6]], and the proportions were higher for healthcare workers in medical institutions designated for COVID-19 treatment. These results suggest that the deterioration of mental health due to COVID-19 is likely to be serious and appropriate approaches are needed in Japan.

Based on the results of IRT analysis, the discrimination of all items of the SAVE-9 was more than 1. However, according to the results of the IRCCCs, the SAVE-6 is more discriminative than the SAVE-9. The original version of the SAVE-9 was constructed with two factors, while the SAVE-6 comprises the items of Factor 1 [10]. In addition, the reliability and validation of the SAVE-6 was previously confirmed in a community sample [11]. The fact that the same structure was used in Japan and Korea might indicate that this is a measure that transcends cultural differences. In the future, it is necessary to examine the factor structure in various countries for cross-cultural validation.

The correlation analysis showed a significant positive moderate correlation between the stress to viral epidemics (SAVE-9 and SAVE-6), insomnia (AIS), anxiety (GAD-7), and depression (CES-D). The original version of the SAVE-9 was also positively moderately associated with anxiety (r = 0.51) and depression (r = 0.41), except for insomnia [10]. Further, scores for both SAVE-9 and SAVE-6 were higher among healthcare workers in medical institutions designated for COVID-19 than in those not designated for COVID-19, which was consistent with the findings of a previous study [7]. However, all effect sizes were low in the present study. This could be because Lai et al. [7] conducted their study in the early stages of the COVID-19 pandemic, whereas the present study was conducted as the pandemic was becoming widespread. Therefore, it is highly likely that stress related to viral epidemics would have increased, regardless of whether respondents worked in medical institutions designated for COVID-19 treatment.

The results of the SEM further revealed that stress related to viral epidemics could affect anxiety and depression directly and indirectly via insomnia. In particular, this is the first study to find that insomnia mediated the effects of stress to viral epidemics on anxiety and depression symptoms. This result is consistent with previous studies which found that stress reactivity is a vulnerability factor for insomnia and insomnia is a risk factor for anxiety and depression [9,28]. Therefore, stress related to viral epidemics could be considered within the framework of the traditional diathesis-stress model [29]. Cognitive-behavioral therapy has proved to be effective for distress, insomnia, anxiety, and depression [30]. Moreover, it has been shown that progressive muscle relaxation therapy was effective in reducing anxiety and improving sleep quality in patients with COVID-19 [31]. Thus, the findings of the present study show that the Japanese versions of the SAVE-9 and SAVE-6 have good reliability and validity and might be useful measures in both epidemiological and clinical studies.

However, this study has some limitations. First, as the study was conducted as an online survey and participants were limited to healthcare workers, the findings might not be representative of the general population in Japan. In particular, future studies to examine the usefulness of the SAVE-6 should be conducted with community samples.

5. Conclusions

To our knowledge, this was the first study to develop the Japanese version of scales measuring stress related to viral epidemics and use this measure to reveal the prevalence of psychopathological insomnia, anxiety, and depression during the COVID-19 pandemic. As Items 6, 7, and 9 of the SAVE-9 are specific to work-related stress associated with viral epidemics, the reliability and validity of the SAVE-6 were confirmed, thus indicating its usefulness for examining the impact of COVID-19 on mental and physical health within the general population.

Conflict of interest

The following are the Supplementary data to this article:

Multimedia component 1

Appendix A. Supplementary data

The following are the Supplementary data to this article: