Skip to main content
PLOS One logoLink to PLOS One
. 2023 Feb 2;18(2):e0279319. doi: 10.1371/journal.pone.0279319

COVID-19 stress syndrome in the German general population: Validation of a German version of the COVID Stress Scales

Stefanie M Jungmann 1,*, Martina Piefke 2, Vincent Nin 2, Gordon J G Asmundson 3, Michael Witthöft 1
Editor: Chung-Ying Lin4
PMCID: PMC9894493  PMID: 36730324

Abstract

The COVID Stress Scales (CSS) are a new self-report instrument for multidimensional assessment of psychological stress in the context of the pandemic. The CSS have now been translated and validated in over 20 languages, but a validated German version has not yet been available. Therefore, the aim was to develop a German version of the CSS, to test its factor structure, reliability, and validity, and to compare it with international studies. In an online survey (08/2020–06/2021), N = 1774 individuals from the German general population (71.5% female; Mage = 41.2 years, SD = 14.2) completed the CSS as well as questionnaires on related constructs and psychopathology. After eight weeks, participants were asked to participate again for the purpose of calculating retest reliability (N = 806). For the German version, the 6-factor structure with good model fit (Root Mean Square Error of Approximation, RMSEA = 0.06) was confirmed, with the six subscales: Danger, Socio-Economic Consequences, Xenophobia, Contamination, Traumatic Stress, and Compulsive Checking. Internal consistencies ranged from ω = .82–.94 (except Compulsive Checking ω = .70), and retest reliability from rtt = .62–.82. Convergent and discriminant validity were confirmed for the German version. Related constructs such as health anxiety, general xenophobia, obsessive-compulsive behavior, and posttraumatic stress disorder symptoms correlated moderately with the respective subscale and lower with the other scales. With anxiety and depression, Traumatic Stress showed the strongest correlation. Overall, there was a high degree of agreement in an international comparison. The CSS can help to identify pandemic-related psychological stress and to derive appropriate interventions.

Introduction

The COVID-19 pandemic negatively affects the mental health of people worldwide. Pandemic-related stress is multidimensional, involving different domains of life as well as people’s emotions, cognitions, and behaviors. Previous studies found that the COVID-19 pandemic is associated with worry and anxiety, hopelessness, pessimism, sleep disturbances, compulsive behaviors, and increased Internet use [110]. Meta-analyses showed that there were significant increases in mental health problems in 2020, and some also found significant increases in anxiety and depressive disorders [11]. Studies in Germany also found increased levels of psychological stress, anxiety, depressive symptoms, and symptoms of post-traumatic stress disorder during the pandemic [1214].

Based on a multidimensional view, Taylor et al. [15, 16] proposed a multifactorial COVID Stress Syndrome that can be assessed using the COVID Stress Scales (CSS). The CSS, a 36 item self-report measure of pandemic-related stress symptoms over the past 7 days, was developed based on symptoms from previous pandemics and current observations and included six subfacets in the original version [16]: Danger (e.g., ‘I am worried about catching the virus.’), Socio-Economic Consequences (e.g., ‘I am worried about grocery stores running out of cleaning or disinfectant supplies.’), Xenophobia (e.g., ‘If I met a person from a foreign country, I’d be worried that they might have the virus.’), Contamination (e.g., ‘I am worried that if someone coughed or sneezed near me, I would catch the virus.’), Traumatic Stress (e.g., ‘I had trouble sleeping because I worried about the virus.’), and Compulsive Checking/Reassurance (e.g., ‘Searched the Internet for treatments for COVID-19.’). A parallel analysis identified five factors, and this 5-factor solution (Danger and Contamination together as one factor) showed good model fit in a second sample. The reliabilities of the CSS subfacets were in the good to very good range, with Cronbach’s alpha of .83 to .95, and convergent and discriminant validity were demonstrated [16].

A network analysis showed that worry about danger (Danger subscale) constituted the core characteristic, which was significantly associated with the subfacets Socio-Economic Consequences, Xenophobia, and Traumatic Stress [16]. In addition, Traumatic Stress and Compulsive/Reassurance behaviors were strongly associated. In terms of sociodemographic data, women, individuals with lower levels of education, and unemployed individuals showed higher CSS scores, and significant negative correlations of CSS with age and income were found. The higher health anxiety, anxiety sensitivity, and uncertainty intolerance were before the pandemic, the higher the CSS total score was during the COVID-19 pandemic [15]. In addition, positive associations were found between Covid Stress Syndrome (assessed by CSS) and anxiety symptoms, depressive symptoms, hygiene measures, and beliefs in conspiracy theories [15, 16].

There also seems to be a different vulnerability to the COVID Stress Syndrome depending on the pre-existing mental disorder. Individuals with a pre-existing anxiety disorder showed significantly higher levels on the total score and subscales than individuals with a pre-existing mood disorder and with no mental disorder [except no difference in checking/reassurance between anxiety disorder vs. no mental disorder, 17]. Asmundson et al. [18] examined the COVID Stress Syndrome over the course of the pandemic (early-mid 2020 and early-mid 2021) and associations with current anxiety and mood disorders. CSS scores were higher in the 2020 sample than in the 2021 sample, suggesting a dynamic course of pandemic-related stress. Within mental disorders, individuals with panic disorder showed the highest severity in CSS. Individuals with generalized anxiety disorder, social anxiety disorder, and post-traumatic stress disorder reported, in particular, higher levels of Traumatic Stress.

The CSS has now been translated and validated in over 20 languages, including Dutch, Arabic, Persian, Polish, Serbian, Spanish, and Swedish [1925]. These versions were found to have a 5- or 6-factor structure, whereby in direct comparisons the 6-factorial model often showed a slightly better model fit [19, 2325]. The translated versions were able to confirm the reliability and validity of the CSS. Positive correlations of the translated CSS with anxiety and depressive symptoms were equally evident, with the strongest correlations particularly between anxiety symptoms and the Traumatic Stress subscale [19, 24]. Using the Persian version, Khosravani et al. [22] found that individuals with generalized anxiety disorder, panic disorder, and obsessive-compulsive disorder showed higher levels of COVID Stress Syndrome than individuals with social anxiety disorder and specific phobia.

In Germany, there has also been a lot of research on pandemic-related stress since the beginning of the COVID-19 pandemic. Previous studies showed that the German population suffers from high levels of psychological stress during the pandemic [12, 26]. Stress was measured, for example, with a one-dimensional stress thermometer [27] or the stress module of the Patient Health Questionnaire [28]. However, no validated German version of the CSS is yet available, which can help to assess stress both specifically against the background of the pandemic and multidimensionally. Furthermore, it enables the investigation of the COVID Stress Syndrome in the German population. Therefore, the aim of this study was to translate and validate a German version of the CSS. We expected that the German version to show internal consistencies (ω > .80), retest reliabilities (rtt > 70), and validity (correlations with e.g., obsessive-compulsive symptoms, xenophobia) comparable to the English original and the other translations. Since the 5-factor and 6-factor solutions have performed well internationally, these two models were tested and compared for the German version of the CSS.

Methods

Data collection procedures and sample

Individuals from the German general population participated in the online study "Stress and Strain during the COVID-19 Pandemic" between August 2020 and June 2021 (recruited via social media, press releases, and flyers). Inclusion criteria were age at least 16 years and written Informed Consent. After eight weeks, participants were invited (via mail) to participate in a second shortened survey (for the purpose of calculating retest reliability of the CSS). For measurement time points 1 and 2, participants could take part in a lottery for a shopping voucher. The study complies with the recommendations of the World Medical Association published in the current version of the Declaration of Helsinki and was approved by the local ethics committee of the Psychological Institute.

Individuals who did not provide written Informed Consent (N = 118) and duplicates (N = 49) were excluded, resulting in a final sample of N = 1774 for measurement time point 1 (main study). The sample was on average M = 41.23 years old (SD = 14.15; range 16–85 years), 71.5% were female, 28.0% male, and 0.5% diverse. In terms of education, half had a college degree (51.1%). Regarding occupational status, 3.1% reported being a pupil/in vocational training, 15.2% a student, 49.3% employed, 14.8% civil servant/self-employed, 6.3% retired, 4.4% househusband/-wife/on parental leave, 1.7% looking for work, and 5.2% other.

The sample at measurement time point 2 (shortened survey eight weeks after the first survey) comprised N = 852 individuals (after exclusion of 59 individuals who did not provide written Informed Consent and 6 duplicates). In the case of 46 persons, the individual codes from the 1st and 2nd measurement time points could not be assigned, so that the pooled data set with repeated participation (test and retest after eight weeks) comprised an n = 806 (Mage = 42.10, SD = 14.44 years, 26.6% male, 73.4% female).

Measures

Development of the German version of the CSS

The translation-back-translation process followed the guidelines for translating foreign language self-report measures [29, 30]. The English items were translated into German by the first and last author and back-translated into English by a professional bilingual translator whose native language is English. The two English versions had only minor differences in wording (e.g., keep me safe–protect me, mail handlers–postman, professionals–experts), but these were retained because of equivalence in content. The German version also consisted of 36 items, six items per scale (see above), and answered on a 5-point Likert scale from 0 = not at all to 4 = extremely (except for the scales Traumatic Stress and Compulsive behavior/reassurance related to frequency: from 0 = never to 4 = almost always). For the final German version of the CSS, see https://coronaphobia.org/professional-resources/.

Patient health questionnaire-4 [PHQ-4; 31]

The PHQ-4 is an economic self-report measure of depression and anxiety. It was compiled from the two items of the Patient Health Questionnaire-2 [PHQ-2; 32], which inquire about the two core diagnostic criteria of depression, and the two items of the Generalized Anxiety Disorder Screener [GAD-2; 33], which measure the two core criteria of generalized anxiety disorder. Respondents determine symptom severity for the past two weeks on a 4-point Likert scale ranging from not at all (0) to almost every day (3). Reliability and validity have been demonstrated in nonclinical and in clinical samples [31, 34]. In this study, the internal consistency of the PHQ-4 was McDonald’s omega ω = .88.

Short health anxiety inventory [SHAI, 35, 36]

In the present study, the 14-item main scale ‘Health anxiety and the feared probability of becoming ill’ of the SHAI was used. There are four statements for each item depending on the severity level (0–3; e.g., 0 = ‘I do not worry about my health.’, …, 3 = ‘I spend most of my time worrying about my health.’). The SHAI-14 has shown good reliability and validity in previous studies [36, 37]. In this study, the internal consistency was ω = .87.

Obsessive compulsive inventory-revised [OCI-R, 38, 39]

The OCI-R is an 18-item questionnaire assessing the core symptoms of obsessive-compulsive disorder on six subscales: washing, checking, ordering, obsessing, hoarding, and neutralizing. Respondents indicate on a 5-point Likert scale ranging from not at all (0) to extremely (4) the individual impairment/stress related to the mentioned symptoms for the past month. The OCI-R has been shown to be reliable and valid in nonclinical and clinical samples [38, 40]. For this study, only the washing subscale (e.g., ‘I sometimes have to wash or clean myself simply because I feel contaminated.’) seems relevant for testing convergent validity. In this study, the internal consistency of the OCI-R was ω = .88.

Short screening scale for DSM-IV posttraumatic stress disorder [PTSD-Screening; 41, 42]

The PTSD-Screening uses seven items to assess two symptom cluster of PTSD: a) avoidance and numbing (5 items) and b) hyperarousal (2 items). The answers are given on a 4-point Likert scale (1 = not at all to 4 = four times a week/most of the time). The Short Screening-Scale for PTSD is a highly reliable instrument and showed high correlations (r = .90, p < .01) with symptom scores assessed by a clinical interview [43]. This screening is very efficient with 7 items, covers to a large extent also the DSM-5 criteria, and it was shown that PTSD screenings according to DSM-IV and DSM-5 are largely equivalent [44]. We found an internal consistency of ω = .80 for the PTSD Screening.

Xenophobia scale [45]

The Xenophobia Scale is a self-report measure for assessing fear-based reactions to strangers. The questionnaire includes nine items (e.g., ‘With increased immigration I fear that our way of life will change for the worse.’), which are answered on a 6-point Likert scale from 1 (strongly disagree) to 6 (strongly agree). Van der Veer et al. [45] found reliabilities ranging from α = .77 to α = .86 in a cross-cultural study with American, Norwegian, and Dutch students. In this study, we found an internal consistency of ω = .95.

All questionnaires were used at measurement time point 1. Measurement time point 2 was a shortened survey including the German version of the CSS and the PHQ-4 (the PHQ-4 at T2 was not evaluated because it was beyond the aim of calculating retest reliability).

Statistical analyses

The two internationally established models, the original 6- and the 5-factor model of Taylor et al. [16], were tested and compared using a Confirmatory Factor Analysis (CFA, including variance adjusted weighted least squares estimator WLSMV) in Mplus [46]. Regarding model goodness of fit, the Root Mean Square Error of Approximation (RMSEA) was used as the absolute fit index, and the Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) were used as incremental fit indices. According to Hu and Bentler [47], RMSEA values close to (≤) .06 and CFI and TLI values close to (≥) .95 indicate a good model fit. For the direct comparison of the two models (5- and 6 factor model), the DIFFTEST option in Mplus was used [46]. To estimate reliability, McDonald’s omega (ω) using SPSS macro by Hayes and Coutts [48] and test-retest correlations were calculated. Convergent validity was tested via Pearson correlations between the CSS subscales and comparable constructs/corresponding measurement instruments (i.e., Danger, Contamination, and Compulsive Checking subscales and SHAI-14; Xenophobia and Xenophobia Scale; Contamination and OCI-R; Traumatic Stress and PTSD screening). Discriminant validity was examined via comparatively lower correlations of the CSS subscales and substantively more divergent constructs (e.g., Xenophobia subscale and SHAI-14, PTSD, and OCI-R). To statistically compare two correlation coefficients, we used the interactive calculator on the website of Lee and Preacher [49], including Fisher’s z transformation. Because it was not possible to skip items in the online studies, the data set did not include missing values.

Results

Factor structure

The CFA with six factors indicated a good model fit: χ2(579) = 4689.40, p < .001, RMSEA = .063 [90% CI: 0.062–0.065], CFI = .94, TLI = .94. The 5-factor model showed a slightly lower model fit: χ2(584) = 6288.58, p < .001, RMSEA = .074 [90% CI: 0.073–0.076], CFI = .92, TLI = .91. The chi-square difference test (χ2(5) = 561.47, p < .001) showed a significant difference with superiority of the 6-factor model, so subsequent analyses refer to this 6-factor model (Danger, Socio-Economic Consequences, Xenophobia, Contamination, Traumatic Stress, and Compulsive Checking). Fig 1 shows the results of the CFA with the 6-factor solution.

Fig 1. Six-factor Confirmatory Factor Analysis (CFA) of the German COVID Stress Scales (CSS).

Fig 1

Factor loadings, inter-correlations, and error terms in the Confirmatory Factor Analysis (CFA) of the German version of the COVID Stress Scales (CSS). Subscales D = Danger, SE = Socio-Economic Consequences, X = Xenophobia, C = Contamination, T = Traumatic Stress, CH = Compulsive Checking. CSS1-36 = Items 1 to 36 of the CSS.

Descriptive statistics, inter-correlations, and reliability

Table 1 shows the means, standard deviations, normality (skewness and kurtosis), and internal consistencies (McDonald’s ω) for the total CSS and the six subscales. Skewness was between 0.27 and 2.03, and kurtosis between 0.50 and 4.54. The subscales and the total scale are right skewed and show a positive kurtosis, i.e., a peaked distribution (except subscale Danger with a negative kurtosis). Bonferroni corrected t-tests showed no difference between women and men in the subscales or total scale (corrected α = .007, p ≥ .009, d ≤ 0.14). With age, small positive correlations were found with CSS total (r = .07, p = .004), the subscales Socio-Economic Consequences (r = .13, p < .001), Traumatic Stress (r = .21, p < .001), and a small negative correlation with Compulsive Checking (r = -.12, p < .001).

Table 1. Mean scores, standard deviations, normality (skewness and kurtosis), internal consistencies (McDonald’s ω), and (inter)-correlations of the total COVID Stress Scales score (CSStotal) and the subscales Danger (D), Socio-Economic Consequences (SE), Xenophobia (X), Contamination (C), Traumatic Stress (T), and Compulsive Checking (CH).

Scale M SD Skewness Kurtosis ω 1 2 3 4 5 6
CSStotal 27.55 17.91 1.11 1.63 .94 .82 .58 .71 .80 .70 .61
1. D 8.54 5.40 0.27 -0.58 .90 .35 .48 .65 .45 .36
2. SE 2.32 3.21 2.03 4.54 .82 .35 .34 .33 .27
3. X 4.75 4.70 1.16 1.17 .91 .52 .32 .24
4. C 6.05 4.58 0.88 0.50 .88 .41 .36
5. T 2.94 3.79 1.84 3.62 .89 .56
6. CH 2.96 3.23 1.49 2.58 .70

all ps < .001.

Internal consistencies (McDonald’s omega) were ω = .94 (Total), ω = .90 (Danger), ω = .82 (Socio-Economic Consequences), ω = .91 (Xenophobia), ω = .88 (Contamination), ω = .89 (Traumatic Stress), and ω = .70 (Compulsive Checking). Retest reliabilities (test-retest correlations after eight weeks) were rtt = .82 (Total), rtt = .77 (Danger), rtt = .70 (Socio-Economic Consequences), rtt = .73 (Xenophobia), rtt = .77 (Contamination), rtt = .71 (Traumatic Stress), and rtt = .62 (Compulsive Checking).

Convergent and discriminant validity

To confirm convergent validity, moderately strong correlations of the CSS subscales should be found with comparable constructs (Danger, Contamination, and Compulsive Checking and SHAI-14; Xenophobia and Xenophobia Scale; Contamination and OCI-Rwash; Traumatic Stress and PTSD-Screening). Table 2, confirming convergent validity, shows that these correlations ranged from r = .37 (SHAI-14 and Contamination) to r = .56 (PTSD and Traumatic Stress). Here, the constructs each showed the strongest correlation with the respective subscale (except for the SHAI-14, with the highest correlation r = .45 with Traumatic Stress). There were significantly lower correlations between the constructs (SHAI-14, Xenophobia Scale, OCI-Rwash, PTSD Screening) and content delineated subscales of the CSS (Zs ≥ 2.59, ps ≤ .01), which can be seen as an indication of discriminant validity. Anxiety and depression were included to test discriminant validity. Lower correlations between anxiety and depression and the subscales Danger, Socio-Economic Consequences, Xenophobia, Contamination, and Compulsive Checking were found (r = .11–.33), which can be seen as an indication of discriminant validity. Noticeably, both anxiety (r = .47) and depression (r = .40) had moderate and significantly stronger correlations with Traumatic Stress than with the other subscales (Zs ≥ 5.36, ps ≤ .001).

Table 2. Pearson correlations of the total score of CSS and the subscales and health anxiety (SHAI-14), general xenophobia (XP Scale), PTSD symptoms (Short screening scale for DSM-IV posttraumatic stress disorder), compulsive washing (OCI-Rwash), anxiety (PHQ-4), and depression (PHQ-4).

CSStotal D SE X C T CH
SHAI-14 .48 .38 .23 .29 .37 .45 .36
XP Scale .28 .08 .29 .49 .09 .15 .09
OCI-Rwash .41 .26 .24 .25 .40 .32 .28
PTSD .49 .36 .23 .25 .32 .56 .41
Anxiety .36 .25 .18 .15 .23 .47 .33
Depression .30 .22 .13 .11 .19 .40 .29

D = Danger, SE = Socio-Economic Consequences, X = Xenophobia, C = Contamination, T = Traumatic Stress, CH = Compulsive Checking subscale of the CSS. All ps < .001.

Discussion

Our study aimed to translate and validate a German version of the CSS. Although the CSS has been validated in more than 20 languages, the German version has not yet been tested.

For the German version of the CSS, the 6-factor model showed a significantly better model fit than the 5-factor model. This is consistent with the originally developed first English version [16] and other translated versions [Arabic, Polish, Spanish, and Swedish versions, 19, 21, 24, 25] which illustrates the international comparability. Taylor et al. [16] found five factors in their parallel analysis and summarized the scales Danger and Contamination. In the German version, these subscales showed a high and the highest intercorrelation (r = .65), so that the German version also confirms their proximity to each other. Nevertheless, the two subscales seem to be sufficiently differentiated from each other [24] and international studies found in direct comparisons a superiority of the 6-factor solution over the 5-factor solution [19, 2325]. Possible explanations for this slight deviation in factor structure could be due to the dynamic events of the COVID-19 pandemic as well as country-specific incidences and measures (e.g., change in threat due to changes in incidence, increase in knowledge regarding the virus, changes in protective measures). For example, the survey in the general population by Taylor et al. [16] occurred earlier (03/04 2020) than ours and studies in other countries (08/2020-06/2021), with possibly country-specific education and protective measures in addition to divergent incidences.

The means of the scale sums are comparable to other translated versions used in the general population (when means are reported), they are descriptively between, for example, the Swedish version (descriptively slightly lower values) [24] and the Serbian version (mostly descriptively slightly higher values) [23]. In the English version [data collection 03-04/2020, 16], values were slightly higher descriptively, whereas it was also shown in a longitudinal design [18] that CSS values were higher in an earlier phase of the pandemic (03-04/2020) than in a later phase (03-05/2021). We found, similar to Carlander et al. [24] and Mahamid et al. [21], no sex differences in the CSS, whereas Taylor et al. [15] found a significantly higher total score for women. It is possible that this could also be due to the early measurement period in Taylor et al. [15] as previous studies found that women in particular were stressed at the beginning of the pandemic and stress decreased over time [50]. Similar to Carlander et al. [24] we found a very small positive correlation between age and CSS total, whereas at the subscale level we found in particular a significant positive correlation with Xenophobia. This is consistent with findings that older people tend to have more negative attitudes toward immigrants [although in cross-sectional studies age and cohort effects may be present, 51]. The small, but significant, negative correlation, between age and the Compulsive Checking subscale could be explained by the fact that half of the items involve Internet use (i.e., You Tube and posts on social media).

The inter-correlations with the three highest correlations between Danger and Contamination (r = .65), between Traumatic Stress and Compulsive Checking (r = .56), and between Danger and Xenophobia (r = .48) are consistent with the network analysis of Taylor et al. [15] who found the strongest interconnections between Traumatic Stress and Compulsive Checking, between Danger/Contamination (factor summarized here) and Xenophobia, as well as between Danger/Contamination and Socio-Economic Consequences.

Internal consistencies were in the good to very good range (ω = .82–.94), except for the Compulsive Checking scale, which had an acceptable internal consistency (ω = .70). Also in international studies, Compulsive Checking was found to have the lowest consistency of the subscales, e.g., Carlander et al. [24] also found a value of ω = .72. Possibly, this could be due to the fact that the scale includes both, items describing a more general search for information (e.g., ‘YouTube videos about COVID-19.’) and items with behaviors related to health concerns or with the intention to reduce them (e.g., ‘Seeking reassurance from friends or family about COVID-19.’). The source of the information search could also play a role. If someone does not use You Tube or social media, these two items would be negated per se. Using the German version of the CSS, we found quite high stability of COVID stress syndrome over eight weeks, with the behavioral scale Compulsive Checking showing the lowest stability (rtt = .62) and worries about getting infected and that measures are not enough (Danger) and Contamination being the most stable (rtt = .77). The retest reliability seems to be similar internationally, e.g., Mahamid [21] found a correlation of rtt = .81 (after three weeks) for the total scale of the Arabic version (German version rtt = .82).

The German version was also able to show its validity. The subscales each correlated significantly and moderately strongly with related constructs. As in previous studies [16, 24], relevant positive associations (r >.30) were found between health anxiety (SHAI-14) and Danger, Contamination, Compulsive Checking, and especially Traumatic Stress. Regarding the latter, the stronger the health concerns, the stronger the hyperarousal and intrusive experience related to the COVID-19 pandemic. This is also consistent with the finding of Asmundson et al. [18] that patients with panic disorder who have health concerns related to an acute physical incident are particularly vulnerable to COVID stress syndrome. The role of health anxiety and the associated trait of uncertainty intolerance in COVID stress syndrome (CSS) was also highlighted by the study of Taylor et al. [52]. They found that the association between negative affectivity and CSS is mediated by uncertainty intolerance and tendency toward health anxiety. Regarding xenophobia and compulsive washing, the strongest correlations were with the respective subscale of the CSS (Xenophobia and Contamination) and also comparable in strength to previous studies [16, 19]. The overlap of CSS (especially Traumatic Stress) with symptoms of PTSD has been hypothesized [53], but previous validations have rarely used a corresponding PTSD scale. With the German version we could confirm the very good validity of the subscale Traumatic Stress (with PTSD screening r = .56, with other subscales r ≤ .41). Generalized anxiety and depression (PHQ-4) were recorded primarily to test discriminate validity. Apart from medium-strong correlations with the Traumatic Stress subscale (r = .47/.40), there are lower correlations with the other subscales (r = .11–.33), which is also consistent with further translations of the CSS [19, 24]. The higher correlation between Traumatic Stress and, in particular, generalized anxiety can be explained by the similarity in content of the items dealing with anxiousness/tension (compare hyperarousal) and difficulties in controlling cognitions (compare intrusive experiencing).

Practical implications of the CSS are its use both, in the general population and in individuals with mental disorders, to assess the level of pandemic-related psychological burden and thus the need for interventions (e.g., to prevent or reduce the development or worsening of mental distress or disorders). In addition to the general need for intervention, the CSS can be used to derive the exact need for specific interventions either at the individual or at the population level. For example, mean scores were highest for the Danger and Contamination subscales. Here, political health campaigns to educate, for example, the transmission routes could be helpful, or in the case of patients with a mental disorder, corresponding interventions similar to those for the treatment of anxiety and obsessive-compulsive disorders [17, 18, 22]. In addition, more innovative approaches should also be encouraged, such as promoting a life span perspective (e.g., differentiated consideration of different age groups and the transition age), more targeted programs (e.g., families, health care workers), concrete formats such as promoting telehealth (i.e., audio/video), brief population-based/public prevention and interventions, and lay-provider systems [54].

Some limitations should be mentioned. Our sample consisted of more than 2/3 women, was comparatively young and educated, thus not representative. These sample characteristics (unequal distribution of gender and younger) could also (partly) explain the slight deviations from the study of Taylor et al. [15] regarding the correlations of CSS with sociodemographic variables. For a validation study and against the background of a dynamic pandemic, the time period of the survey was quite long (08/2020-06/2021). Although some study confirmed factorial invariance over time [18, 19], the dynamic happening and, for example, changes in mean values over time might have affected comparability with the English version and other translations.

Conclusions

In summary, the German version of the CSS is a reliable and valid method to assess psychological stress during pandemics multifactorially. The German version comprises six subfacets with the scales: Danger, Socio-Economic Consequences, Xenophobia, Contamination, Traumatic Stress, and Compulsive Checking. Internationally, there is a high degree of agreement regarding factor solution, psychometric quality, and correlations with related constructs as well as psychopathology. The CSS was developed for immediate application in the context of the current pandemic but, also, to be adaptable for application in future pandemics; as such, the German CSS adds another translation to the corpus of available scales and positions researchers to assess both current and future pandemic-stress using the current gold standard.

Acknowledgments

Acknowledgments We thank Steven Stelz for assistance with the literature review and description of some measurement instruments.

Data Availability

Dataset used in the research are publicly available on the Open Science Framework at https://osf.io/sm4f2/.

Funding Statement

The authors received no specific funding for this work.

References

  • 1.Rajabimajd N, Alimoradi Z, Griffiths M. Impact of COVID-19-related fear and anxiety on job attributes: A systematic review. Asian J Soc Health Behav. 2021;4:51. doi: 10.4103/shb.shb_24_21 [DOI] [Google Scholar]
  • 2.Olashore A, Akanni O, Fela-Thomas A, Khutsafalo K. The psychological impact of COVID-19 on health-care workers in African Countries: A systematic review. Asian J Soc Health Behav. 2021;4:85. doi: 10.4103/shb.shb_32_21 [DOI] [Google Scholar]
  • 3.Hasannia E, Mohammadzadeh F, Tavakolizadeh M, Davoudian N, Bay M. Assessment of the anxiety level and trust in information resources among iranian health-care workers during the pandemic of coronavirus disease. Asian J Soc Health Behav. 2021;4:163–8. [Google Scholar]
  • 4.Maraz A, Yi S. Compulsive buying gradually increased during the first six months of the Covid-19 outbreak. J Behav Addict. 2022;11:88–101. doi: 10.1556/2006.2022.00002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Morin CM, Bjorvatn B, Chung F, Holzinger B, Partinen M, Penzel T, et al. Insomnia, anxiety, and depression during the COVID-19 pandemic: an international collaborative study. Sleep Med. 2021;87:38–45. doi: 10.1016/j.sleep.2021.07.035 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Alimoradi Z, Broström A, Tsang HWH, Griffiths MD, Haghayegh S, Ohayon MM, et al. Sleep problems during COVID-19 pandemic and its’ association to psychological distress: A systematic review and meta-analysis. EClinicalMedicine. 2021;36:100916. doi: 10.1016/j.eclinm.2021.100916 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Hacimusalar Y, Kahve AC, Yasar AB, Aydin MS. Anxiety and hopelessness levels in COVID-19 pandemic: A comparative study of healthcare professionals and other community sample in Turkey. J Psychiatr Res. 2020;129:181–8. doi: 10.1016/j.jpsychires.2020.07.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Duan L, Shao X, Wang Y, Huang Y, Miao J, Yang X, et al. An investigation of mental health status of children and adolescents in china during the outbreak of COVID-19. J Affect Disord. 2020;275:112–8. doi: 10.1016/j.jad.2020.06.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Arslan G, Yıldırım M, Tanhan A, Buluş M, Allen K-A. Coronavirus Stress, Optimism-Pessimism, Psychological Inflexibility, and Psychological Health: Psychometric Properties of the Coronavirus Stress Measure. Int J Ment Health Addict. 2021;19:2423–39. doi: 10.1007/s11469-020-00337-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Manalang Vicerra P. Mental stress and well-being among low-income older adults during COVID-19 pandemic. Asian J Soc Health Behav. 2022;5:101. doi: 10.4103/shb.shb_110_22 [DOI] [Google Scholar]
  • 11.World Health Organization. Mental Health and COVID-19: Early evidence of the pandemic’s impact. 2022. www.who.int/publications/i/item/WHO-2019-nCoV-Sci_Brief-Mental_health-2022.1. Accessed 24 Aug 2022. [Google Scholar]
  • 12.Bäuerle A, Teufel M, Musche V, Weismüller B, Kohler H, Hetkamp M, et al. Increased generalized anxiety, depression and distress during the COVID-19 pandemic: a cross-sectional study in Germany. J Public Health (Oxf). 2020;42:672–8. doi: 10.1093/pubmed/fdaa106 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Jungmann SM, Witthöft M. Health anxiety, cyberchondria, and coping in the current COVID-19 pandemic: Which factors are related to coronavirus anxiety? J Anxiety Disord. 2020;73:102239. doi: 10.1016/j.janxdis.2020.102239 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Wechsler TF, Schmidmeier M, Biehl S, Gerczuk J, Guerrero-Cerda F-M, Mühlberger A. Individual changes in stress, depression, anxiety, pathological worry, posttraumatic stress, and health anxiety from before to during the COVID-19 pandemic in adults from Southeastern Germany. BMC Psychiatry. 2022;22:528. doi: 10.1186/s12888-022-04148-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Taylor S, Landry CA, Paluszek MM, Fergus TA, McKay D, Asmundson GJG. COVID stress syndrome: Concept, structure, and correlates. Depress Anxiety. 2020;37:706–14. doi: 10.1002/da.23071 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Taylor S, Landry CA, Paluszek MM, Fergus TA, McKay D, Asmundson GJG. Development and initial validation of the COVID Stress Scales. J Anxiety Disord. 2020;72:102232. doi: 10.1016/j.janxdis.2020.102232 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Asmundson GJG, Paluszek MM, Landry CA, Rachor GS, McKay D, Taylor S. Do pre-existing anxiety-related and mood disorders differentially impact COVID-19 stress responses and coping? J Anxiety Disord. 2020;74:102271. doi: 10.1016/j.janxdis.2020.102271 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Asmundson GJG, Rachor G, Drakes DH, Boehme BAE, Paluszek MM, Taylor S. How does COVID stress vary across the anxiety-related disorders? Assessing factorial invariance and changes in COVID Stress Scale scores during the pandemic. J Anxiety Disord. 2022;87:102554. doi: 10.1016/j.janxdis.2022.102554 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Adamczyk K, Clark DA, Pradelok J. The Polish COVID Stress Scales: Considerations of psychometric functioning, measurement invariance, and validity. PLoS One. 2021;16:e0260459. doi: 10.1371/journal.pone.0260459 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Abbady AS, El-Gilany A-H, El-Dabee FA, Elsadek AM, ElWasify M, Elwasify M. Psychometric characteristics of the of COVID Stress Scales-Arabic version (CSS-Arabic) in Egyptian and Saudi university students. Middle East Curr Psychiatry 2021. doi: 10.1186/s43045-021-00095-8 [DOI] [Google Scholar]
  • 21.Mahamid FA, Veronese G, Bdier D, Pancake R. Psychometric properties of the COVID stress scales (CSS) within Arabic language in a Palestinian context. Curr Psychol. 2021:1–10. doi: 10.1007/s12144-021-01794-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Khosravani V, Asmundson GJG, Taylor S, Sharifi Bastan F, Samimi Ardestani SM. The Persian COVID stress scales (Persian-CSS) and COVID-19-related stress reactions in patients with obsessive-compulsive and anxiety disorders. J Obsessive Compuls Relat Disord. 2021;28:100615. doi: 10.1016/j.jocrd.2020.100615 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Milic M, Dotlic J, Rachor GS, Asmundson GJG, Joksimovic B, Stevanovic J, et al. Validity and reliability of the Serbian COVID Stress Scales. PLoS One. 2021;16:e0259062. doi: 10.1371/journal.pone.0259062 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Carlander A, Lekander M, Asmundson GJG, Taylor S, Olofsson Bagge R, Lindqvist Bagge A-S. COVID-19 related distress in the Swedish population: Validation of the Swedish version of the COVID Stress Scales (CSS). PLoS One. 2022;17:e0263888. doi: 10.1371/journal.pone.0263888 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Noe-Grijalva M, Polo-Ambrocio A, Gómez-Bedia K, Caycho-Rodríguez T. Spanish Translation and Validation of the COVID Stress Scales in Peru. Front Psychol. 2022;13:840302. doi: 10.3389/fpsyg.2022.840302 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Benke C, Autenrieth LK, Asselmann E, Pané-Farré CA. One year after the COVID-19 outbreak in Germany: long-term changes in depression, anxiety, loneliness, distress and life satisfaction. Eur Arch Psychiatry Clin Neurosci 2022. doi: 10.1007/s00406-022-01400-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hinz A, Mitchell AJ, Dégi CL, Mehnert-Theuerkauf A. Normative values for the distress thermometer (DT) and the emotion thermometers (ET), derived from a German general population sample. Qual Life Res. 2019;28:277–82. doi: 10.1007/s11136-018-2014-1 [DOI] [PubMed] [Google Scholar]
  • 28.Löwe B, Spitzer RL, Zipfel S, Herzog W. Patient Health Questionnaire. Karlsruhe: Pfizer; 2002. [Google Scholar]
  • 29.Schmitt M, Eid M. Richtlinien für die Übersetzung fremdsprachlicher Messinstrumente [Guidelines for the translation of foreign language measurement instruments]. Diagnostica. 2007;53:1–2. doi: 10.1026/0012-1924.53.1.1 [DOI] [Google Scholar]
  • 30.Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25:3186–91. doi: 10.1097/00007632-200012150-00014 [DOI] [PubMed] [Google Scholar]
  • 31.Kroenke K, Spitzer RL, Williams JBW, Löwe B. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics. 2009;50:613–21. doi: 10.1176/appi.psy.50.6.613 [DOI] [PubMed] [Google Scholar]
  • 32.Kroenke K, Spitzer RL, Williams JBW. The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41:1284–92. doi: 10.1097/01.MLR.0000093487.78664.3C [DOI] [PubMed] [Google Scholar]
  • 33.Kroenke K, Spitzer RL, Williams JBW, Monahan PO, Löwe B. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med. 2007;146:317–25. doi: 10.7326/0003-4819-146-5-200703060-00004 [DOI] [PubMed] [Google Scholar]
  • 34.Löwe B, Wahl I, Rose M, Spitzer C, Glaesmer H, Wingenfeld K, et al. A 4-item measure of depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord. 2010;122:86–95. doi: 10.1016/j.jad.2009.06.019 [DOI] [PubMed] [Google Scholar]
  • 35.Bailer J, Rist F, Müller T, Mier D., Diener C., Ofer J, et al. German validation of the short health anxiety inventory (SHAI). 2013;34:378–98. [Google Scholar]
  • 36.Salkovskis PM, Rimes KA, Warwick HMC, Clark DM. The Health Anxiety Inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis. Psychol Med. 2002;32:843–53. doi: 10.1017/s0033291702005822 [DOI] [PubMed] [Google Scholar]
  • 37.Alberts NM, Hadjistavropoulos HD, Jones SL, Sharpe D. The Short Health Anxiety Inventory: a systematic review and meta-analysis. J Anxiety Disord. 2013;27:68–78. doi: 10.1016/j.janxdis.2012.10.009 [DOI] [PubMed] [Google Scholar]
  • 38.Foa EB, Huppert JD, Leiberg S, Langner R, Kichic R, Hajcak G, et al. The Obsessive-Compulsive Inventory: Development and validation of a short version. Psychological Assessment. 2002;14:485–96. doi: 10.1037/1040-3590.14.4.485 [DOI] [PubMed] [Google Scholar]
  • 39.Gönner S, Leonhart R, Ecker W. Das Zwangsinventar OCI-R—die deutsche Version des Obsessive-Compulsive Inventory-Revised—Ein kurzes Selbstbeurteilungsinstrument zur mehrdimensionalen Messung von Zwangssymptomen. [The German version of the obsessive-compulsive inventory-revised: a brief self-report measure for the multidimensional assessment of obsessive-compulsive symptoms]. Psychother Psychosom Med Psychol. 2007;57:395–404. doi: 10.1055/s-2007-970894 [DOI] [PubMed] [Google Scholar]
  • 40.Abramowitz JS, Deacon BJ. Psychometric properties and construct validity of the Obsessive-Compulsive Inventory-Revised: Replication and extension with a clinical sample. J Anxiety Disord. 2006;20:1016–35. doi: 10.1016/j.janxdis.2006.03.001 [DOI] [PubMed] [Google Scholar]
  • 41.Breslau N, Peterson EL, Kessler RC, Schultz LR. Short screening scale for DSM-IV posttraumatic stress disorder. Am J Psychiatry. 1999;156:908–11. doi: 10.1176/ajp.156.6.908 [DOI] [PubMed] [Google Scholar]
  • 42.Siegrist P, Maercker A. Deutsche Fassung der Short Screening Scale for DSM-IV Posttraumatic Stress Disorder [German version of the Short DSM-IV Screening Scale to detect posttraumatic stress disorder]. Trauma & Gewalt. 2010;4:208–13. [Google Scholar]
  • 43.Menning H, Renz A, Seifert J, Maercker A. Reduced mismatch negativity in posttraumatic stress disorder: a compensatory mechanism for chronic hyperarousal? Int J Psychophysiol. 2008;68:27–34. doi: 10.1016/j.ijpsycho.2007.12.003 [DOI] [PubMed] [Google Scholar]
  • 44.Hoge CW, Riviere LA, Wilk JE, Herrell RK, Weathers FW. The prevalence of post-traumatic stress disorder (PTSD) in US combat soldiers: a head-to-head comparison of DSM-5 versus DSM-IV-TR symptom criteria with the PTSD checklist. The Lancet Psychiatry. 2014;1:269–77. doi: 10.1016/S2215-0366(14)70235-4 [DOI] [PubMed] [Google Scholar]
  • 45.van der Veer K, Ommundsen R, Yakushko O, Higler L, Woelders S, Hagen KA. Psychometrically and qualitatively validating a cross-national cumulative measure of fear-based xenophobia. Qual Quant. 2013;47:1429–44. doi: 10.1007/s11135-011-9599-6 [DOI] [Google Scholar]
  • 46.Muthén LK, Muthén BO. Mplus user’s guide: Statistical analysis with latent variables. Los Angeles, CA: Muthén & Muthén; 2017. [Google Scholar]
  • 47.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. doi: 10.1080/10705519909540118 [DOI] [Google Scholar]
  • 48.Hayes AF, Coutts JJ. Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But… Communication Methods and Measures. 2020;14:1–24. doi: 10.1080/19312458.2020.1718629 [DOI] [Google Scholar]
  • 49.Lee IA, Preacher KJ. Calculation for the test of the difference between two dependent correlations with one variable in common; 2013. [Google Scholar]
  • 50.Vaterlaus JM, Spruance LA, Heiser KD, Patten EV. Women’s well-being and experiences during the COVID-19 pandemic in the United States: A longitudinal mixed-methods approach. The Social Science Journal. 2022:1–18. doi: 10.1080/03623319.2022.2028255 [DOI] [Google Scholar]
  • 51.Schotte S, Winkler H. Why Are the Elderly More Averse to Immigration When They Are More Likely to Benefit? Evidence across Countries. International Migration Review. 2018;52:1250–82. doi: 10.1177/0197918318767927 [DOI] [Google Scholar]
  • 52.Taylor S, Fong A, Asmundson GJG. Predicting the Severity of Symptoms of the COVID Stress Syndrome From Personality Traits: A Prospective Network Analysis. Front Psychol. 2021;12:632227. doi: 10.3389/fpsyg.2021.632227 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Taylor S. COVID Stress Syndrome: Clinical and Nosological Considerations. Curr Psychiatry Rep. 2021;23:19. doi: 10.1007/s11920-021-01226-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Gruber J, Clark LA, Abramowitz JS, Aldao A, Chung T, Forbes EE, et al. Mental Health and Clinical Psychological Science in the Time of COVID-19: Challenges, Opportunities, and a Call to Action. Am Psychol. 2020;76:409–26. doi: 10.1037/amp0000707 [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

Chung-Ying Lin

1 Dec 2022

PONE-D-22-28279COVID-19 Stress Syndrome in the German general population: Validation of a German version of the COVID Stress ScalesPLOS ONE

Dear Dr. Jungmann,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

One expert reviewer in the field of psychometrics has reviewed the work and provided some comments for revision. I agree with the opinions from the reviewer that the present contribution in general is good. However, it needs to be revised before publication. Apart from the reviewer's comments, the authors should consider the following of my concerns as well.1. I think that the authors did not provide a figure legend to explain their figure 1. Specifically, the authors have used some abbreviations in the figure 1 and their figure legend should explain these abbreviations because figure should stand alone.2. The authors mentioned the abbreviation of RMSEA in the Abstract without providing a full spell-out. Given that Abstract also stands alone to the main text, the full spell-out of the RMSEA should be given in the Abstract.3. This may be a stylish issue; however, I prefer the authors report the full spell-out of the abbreviations and put the abbreviations in the brackets. Instead of reporting the abbreviations and putting the full spell-outs in the brackets.4. The Introduction has used some systematic reviews to describe the mental health issues during COVID-19; however, I would encourage the authors further using the following systematic reviews and large-scale studies to strengthen the descriptions.Khankeh H, Pourebrahimi M, Karibozorg MF, Hosseinabadi-Farahani M, Ranjbar M, Ghods MJ, Saatchi M. Public trust, preparedness, and the influencing factors regarding COVID-19 pandemic situation in Iran: A population-based cross-sectional study. Asian J Soc Health Behav 2022;5:154-61Vicerra PM. Mental stress and well-being among low-income older adults during COVID-19 pandemic. Asian J Soc Health Behav 2022;5:101-7Hasannia E, Mohammadzadeh F, Tavakolizadeh M, Davoudian N, Bay M. Assessment of the anxiety level and trust in information resources among iranian health-care workers during the pandemic of coronavirus disease 2019. Asian J Soc Health Behav 2021;4:163-8Olashore AA, Akanni OO, Fela-Thomas AL, Khutsafalo K. The psychological impact of COVID-19 on health-care workers in African Countries: A systematic review. Asian J Soc Health Behav 2021;4:85-97Rajabimajd N, Alimoradi Z, Griffiths MD. Impact of COVID-19-related fear and anxiety on job attributes: A systematic review. Asian J Soc Health Behav 2021;4:51-55. This statement "prove to be as reliable (ω > .80)" should be revised. Although many people consider that internal consistency is a type of reliability; its nature is more like validity (i.e., all items are associated with each other). Therefore, I would use test-retest reliability to emphasize if the tested instrument (i.e., CSS) is reliable. The hypothesis of ω here is fine, but it would be much better if test-retest reliability is also hypothesized.6. I prefer the authors not using some strong words (e.g., prove) in the present submission.7. As the present sample included some participants aged below 18 years, I wonder if the authors have obtained their parents' informed consent. 8. The authors did a chi-square difference test; however, this is not mentioned in their Statistical analysis section. Although some people know that the DIFFTEST option in Mplus is used for chi-square difference test, not everybody knows this. Please clearly mention this in the Statistical analysis section.9. The authors have compared the correlation coefficients; however, this is not stated in the Statistical analysis section either. I think that the authors have used Lee & Preacher's website to make the correlation coefficient comparisons. However, they did not disclose this. Or, if I get it wrong, the authors should provide a proper citation to indicate how they compared the correlation coefficients. ​

==============================

Please submit your revised manuscript by Jan 15 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Chung-Ying Lin

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review this work. The authors attempted to validate the German version of COVID Stress Scales. The validation would be advantageous to the target population. However, there are a few small issues that need to be addressed.

1. Line 115, it is more appropriate to say “translate and validate” rather than “develop and validate”.

2. Lines 142-143, what is the meaning of "denied the inclusion criteria"?

3. Line 146, “26.6% m, 73.4% f”. Please use “male” and “female”.

4. Please provide the reliability of PHQ-4, SHAI, OCI-R(wash), PTSD-Screening, Xenophobia Scale in current study.

5. Line 203, What is the purpose of PHQ-4 in time point 2?

6. In statistical analyses, description about convergent and discriminant validity was missing.

7. In results, it would be better to report “descriptive statistics, inter-correlations, and reliability” before “Factor structure”.

8. Line 289, again, it is more appropriate to say “translate and validate” rather than “develop and validate”.

9. Lines 310–312, the use of square brackets may confuse readers. For example, [descriptively slightly lower values, 20], readers may think 20 is the value, but it is the citation. Use round bracket is fine, i.e. (descriptively slightly lower values) [20].

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Feb 2;18(2):e0279319. doi: 10.1371/journal.pone.0279319.r002

Author response to Decision Letter 0


2 Dec 2022

Dear Professor Lin,

Dear reviewer,

Thank you very much for your letter, and for the very helpful review of our manuscript titled "COVID-19 Stress Syndrome in the German general population: Validation of a German version of the COVID Stress Scales".

We thank you and the reviewer for the thoughtful and constructive feedback and suggestions for improving the manuscript. We have revised the manuscript according to your recommendations. We have carefully considered and responded in detail to each of the points made by you and the reviewer. Please find our actions detailed in the following:

Comments editor:

One expert reviewer in the field of psychometrics has reviewed the work and provided some comments for revision. I agree with the opinions from the reviewer that the present contribution in general is good. However, it needs to be revised before publication. Apart from the reviewer's comments, the authors should consider the following of my concerns as well.

1. I think that the authors did not provide a figure legend to explain their figure 1. Specifically, the authors have used some abbreviations in the figure 1 and their figure legend should explain these abbreviations because figure should stand alone.

[AU]: Thank you for the thoughtful comment. We have added a legend for Figure 1 on page 9 in the revised manuscript.

“Factor loadings, inter-correlations, and error terms in the Confirmatory Factor Analysis (CFA) of the German version of the COVID Stress Scales (CSS). Subscales D = Danger, SE = Socio-Economic Consequences, X = Xenophobia, C = Contamination, T = Traumatic Stress, CH = Compulsive Checking. CSS1-36 = Items 1 to 36 of the CSS.” (page 9)

2. The authors mentioned the abbreviation of RMSEA in the Abstract without providing a full spell-out. Given that Abstract also stands alone to the main text, the full spell-out of the RMSEA should be given in the Abstract.

[AU]: We have RMSEA spelled out in the abstract. Since the abbreviation is very common, we have left it additional.

“(Root Mean Square Error of Approximation, RMSEA = 0.06)”

3. This may be a stylish issue; however, I prefer the authors report the full spell-out of the abbreviations and put the abbreviations in the brackets. Instead of reporting the abbreviations and putting the full spell-outs in the brackets.

[AU]: Thank you for pointing this out. We have adopted the changes as recommended, first written out, then the abbreviation in brackets.

“Regarding model goodness of fit, the Root Mean Square Error of Approximation (RMSEA) was used as the absolute fit index, and the Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) were used as incremental fit indices.” (page 8)

4. The Introduction has used some systematic reviews to describe the mental health issues during COVID-19; however, I would encourage the authors further using the following systematic reviews and large-scale studies to strengthen the descriptions.

Khankeh H, Pourebrahimi M, Karibozorg MF, Hosseinabadi-Farahani M, Ranjbar M, Ghods MJ, Saatchi M. Public trust, preparedness, and the influencing factors regarding COVID-19 pandemic situation in Iran: A population-based cross-sectional study. Asian J Soc Health Behav 2022;5:154-61

Vicerra PM. Mental stress and well-being among low-income older adults during COVID-19 pandemic. Asian J Soc Health Behav 2022;5:101-7

Hasannia E, Mohammadzadeh F, Tavakolizadeh M, Davoudian N, Bay M. Assessment of the anxiety level and trust in information resources among iranian health-care workers during the pandemic of coronavirus disease 2019. Asian J Soc Health Behav 2021;4:163-8

Olashore AA, Akanni OO, Fela-Thomas AL, Khutsafalo K. The psychological impact of COVID-19 on health-care workers in African Countries: A systematic review. Asian J Soc Health Behav 2021;4:85-97

Rajabimajd N, Alimoradi Z, Griffiths MD. Impact of COVID-19-related fear and anxiety on job attributes: A systematic review. Asian J Soc Health Behav 2021;4:51-5

[AU]: Thank you for the recommendations regarding other interesting studies for our theoretical background. We have now cited 4 of these studies as suggested, which we think fit well with our theoretical background (Hasannia et al., 2021; Manalang Vicerra, 2022; Olashore et al., 2021; Rajabimajd et al., 2021). Regarding the sentence with the meta-analysis, we had specifically aimed at a before vs. during comparison (increase during the pandemic).

5. This statement "prove to be as reliable (ω > .80)" should be revised. Although many people consider that internal consistency is a type of reliability; its nature is more like validity (i.e., all items are associated with each other). Therefore, I would use test-retest reliability to emphasize if the tested instrument (i.e., CSS) is reliable. The hypothesis of ω here is fine, but it would be much better if test-retest reliability is also hypothesized.

[AU]: Thank you for this helpful hint, we have reworded this sentence and added a retest reliability hypothesis (on page 5)

“We expected that the German version to show internal consistencies (ω > .80), retest reliabilities (rtt > 70), and validity (correlations with e.g., obsessive-compulsive symptoms, xenophobia) comparable to the English original and the other translations.” (on page 5)

6. I prefer the authors not using some strong words (e.g., prove) in the present submission.

[AU]: As recommended, we have replaced strong words, especially "proven" in the complete manuscript.

7. As the present sample included some participants aged below 18 years, I wonder if the authors have obtained their parents' informed consent.

[AU]: Thank you for your inquiry regarding the participation of 16 and 17 year old participants. In the present study, participation was possible from the age of 16. Separate parental consent for 16 and 17 year olds was not required. The questions we asked are appropriate from the age of 16. Within the framework of the European Union's General Data Protection Regulation, processing and storage of personal data (anonymous survey in this case) is possible from the age of 16. This procedure has been reviewed and positively assessed by the ethics committee.

8. The authors did a chi-square difference test; however, this is not mentioned in their Statistical analysis section. Although some people know that the DIFFTEST option in Mplus is used for chi-square difference test, not everybody knows this. Please clearly mention this in the Statistical analysis section.

[AU]: Thank you for pointing this out. We had briefly mentioned the DIFF test in the statistical analyses, but have emphasized this more clearly in the now revised version.

“For the direct comparison of the two models (5- and 6 factor model), the DIFFTEST option in Mplus was used [46].” (page 9)

9. The authors have compared the correlation coefficients; however, this is not stated in the Statistical analysis section either. I think that the authors have used Lee & Preacher's website to make the correlation coefficient comparisons. However, they did not disclose this. Or, if I get it wrong, the authors should provide a proper citation to indicate how they compared the correlation coefficients.

[AU]: Thank you also for this comment. We agree that this is a relevant point. We have now added a supplementary description including the reference in the statistical analysis section (page 9)

“To statistically compare two correlation coefficients, we used the interactive calculator on the website of Lee and Preacher [49], including Fisher's z transformation.” (Page 9)

Comments external reviewer:

Reviewer #1: Thank you for the opportunity to review this work. The authors attempted to validate the German version of COVID Stress Scales. The validation would be advantageous to the target population. However, there are a few small issues that need to be addressed.

1. Line 115, it is more appropriate to say “translate and validate” rather than “develop and validate”.

[AU]: Thank you for pointing this out. As recommended, the revised version includes "translate and validate" (page 5).

2. Lines 142-143, what is the meaning of "denied the inclusion criteria"?

[AU]: These subjects had accessed the questionnaire but had not agreed to the consent form ("I hereby consent to participate voluntarily in the study....."). For better understanding, we have chosen a different wording here. (page 6)

“The sample at measurement time point 2 (shortened survey eight weeks after the first survey) comprised N = 852 individuals (after exclusion of 59 individuals who did not provide written Informed Consent and 6 duplicates).” (page 6)

3. Line 146, “26.6% m, 73.4% f”. Please use “male” and “female”.

[AU]: In the revised version of the manuscript, we have now used "male" and "female".

4. Please provide the reliability of PHQ-4, SHAI, OCI-R(wash), PTSD-Screening, Xenophobia Scale in current study.

[AU]: Thank you for this important advice. We have added the internal consistency in each case.

PHQ-4: “In this study, the internal consistency of the PHQ-4 was McDonald's omega ω = .88.” (Page 7)

SHAI-14: “In this study, the internal consistency was ω = .87.” (page 7)

OCI-R: “In this study, the internal consistency of the OCI-R was ω = .88.” (page 8)

PTSD-Screening: “We found an internal consistency of ω = .80 for the PTSD Screening.” (page 8)

Xenophobie Scale: “In this study, we found an internal consistency of ω = .95.” (page 8)

5. Line 203, What is the purpose of PHQ-4 in time point 2?

[AU]: Thank you for the comment, that is a very understandable query. Originally, we had the intention to look at correlations between the COVID Stress Scales and anxiety and depression at T2 as well. However, it was beyond the aim and scope of this validation study. In the revised version, we have explained this for better comprehensibility.

“Measurement time point 2 was a shortened survey including the German version of the CSS and the PHQ-4 (the PHQ-4 at T2 was not evaluated because it was beyond the aim of calculating retest reliability).” (page 8)

6. In statistical analyses, description about convergent and discriminant validity was missing.

[AU]: Thank you very much for this suggestion. We have now added the description of convergent and discriminant validity in the statistical methods section.

“Convergent validity was tested via Pearson correlations between the CSS subscales and comparable constructs/corresponding measurement instruments (i.e., Danger, Contamination, and Compulsive Checking subscales and SHAI-14; Xenophobia and Xenophobia Scale; Contamination and OCI-R; Traumatic Stress and PTSD screening). Discriminant validity was examined via comparatively lower correlations of the CSS subscales and substantively more divergent constructs (e.g., Xenophobia subscale and SHAI-14, PTSD, and OCI-R).” (page 9)

7. In results, it would be better to report “descriptive statistics, inter-correlations, and reliability” before “Factor structure”.

[AU]: When writing the results section, we had exactly the same thoughts. We then decided on this order for the following reason: The result of the factor structure (5 or 6 factors) influences the scale formation, which is the basis of the descriptive statistics, intercorrelation and reliability per subscale, i.e. whether 5 or 6 subscales are considered in more detail. We hope our explanation of the order is comprehensible and finds support.

8. Line 289, again, it is more appropriate to say “translate and validate” rather than “develop and validate”.

[AU]: We have also replaced "develop" with "translate" at this point (in the revised version of article line 306).

9. Lines 310–312, the use of square brackets may confuse readers. For example, [descriptively slightly lower values, 20], readers may think 20 is the value, but it is the citation. Use round bracket is fine, i.e. (descriptively slightly lower values) [20].

[AU]: Thank you for this recommendation to make the information in the brackets clearer. We have changed this as follows:

“The means of the scale sums are comparable to other translated versions used in the general population (when means are reported), they are descriptively between, for example, the Swedish version (descriptively slightly lower values) [24] and the Serbian version (mostly descriptively slightly higher values) [23].” (page 14)

Yours sincerely,

Stefanie Jungmann

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 1

Chung-Ying Lin

5 Dec 2022

COVID-19 Stress Syndrome in the German general population: Validation of a German version of the COVID Stress Scales

PONE-D-22-28279R1

Dear Dr. Jungmann,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Chung-Ying Lin

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

I have evaluated the authors' responses to the reviewer's comments and my comments. The authors have satisfactorily addressed all the comments and improve the manuscript satisfactorily. I applaud and thank for the authors' efforts in this contribution and am happy to accept this contribution. 

Reviewers' comments:

Acceptance letter

Chung-Ying Lin

24 Jan 2023

PONE-D-22-28279R1

COVID-19 Stress Syndrome in the German general population: Validation of a German version of the COVID Stress Scales

Dear Dr. Jungmann:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Chung-Ying Lin

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: Response to Reviewers.docx

    Data Availability Statement

    Dataset used in the research are publicly available on the Open Science Framework at https://osf.io/sm4f2/.


    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES