Summary
The coronavirus disease 2019 (COVID-19) pandemic may cause psychological distress in the general population and has the potential to cause anxiety regarding COVID-19. No validated questionnaires exist for the measurement of specific COVID-19 anxiety. We modified the DSM 5 – Severity Measure for Specific Phobia–Adult scale specifically for anxiety regarding COVID-19, and report the psychometric properties from an online study with 6262 participants from the general population in Germany. We analysed internal consistency as well as concordant validity. The scale showed good internal consistency (α = 0.86) and good concordant validity (rs = 0.60) with the 2-item Generalized Anxiety Disorder measure and rs = 0.61 with self-rating of limitations in daily life caused by COVID-19 anxiety.
Keywords: Corona, anxiety, measurement, questionnaire, assessment
Background
The current coronavirus disease 2019 (COVID-19) pandemic has the potential to cause severe psychological distress for the general population worldwide.1–3 The development of public health approaches to combat COVID-19 heavily depend on strategies for social and behavioural change. It is therefore of critical importance to assess and compare behavioural as well as psychological responses to COVID-19.
Although there are a large number of media reports on the psychological consequences of the COVID-19 pandemic substantial research on this topic is lacking. Several cross-sectional studies as well as one longitudinal study, have shown elevated levels of psychological distress, depression and anxiety.3–9 For example, in a longitudinal study with 1738 participants from the Chinese general population elevated levels of symptoms of post-traumatic stress disorder, depression and anxiety were found, which were in large part stable from the first assessment at the end of January to the second one at the end of February 2020.6 In addition to general symptoms of psychological distress, depression and anxiety, the development of symptoms of anxiety specifically focused on COVID-19 seems possible.2 With an increasing number of infections, the development of ‘coronaphobia’ in a growing number of individuals has to be expected.2
Aims
Although there are a large number of well-established and validated questionnaires to assess general anxiety as well as disorder-specific anxiety, to our knowledge there are no scales for the assessment of specific anxiety regarding COVID-19. Research undertaken during past outbreaks of infectious diseases show that psychometric questionnaires are able to identify individuals at high risk for the later development of mental health problems.10,11 Thus, screening questionnaires that enable the incorporation of screen-and-treat approaches in clinical settings, as well as their use in research focusing on anxiety in the context of the current pandemic, are urgently needed.2 The purpose of this study was to present a scale that aims to measure COVID-19 anxiety. Therefore, we present a slightly modified version of the DSM-5 – Severity Measure for Specific Phobia–Adult (SP-D)12,13 questionnaire that focuses on anxiety regarding COVID-19 and we report our results on the psychometric properties from a large online survey. Using our measure research about whether specific forms of anxiety relating to COVID-19 are present in the general populations worldwide should be possible. As we do not know whether such specific forms of anxiety exist, the scale is thought of as an addition to the existing established measures of anxiety rather than a replacement.
Method
Development of the questionnaire
Modifications to the original SP-D questionnaire were as follows. We modified the original instruction: ‘The following questions ask about thoughts, feelings, and behaviors that you may have had in a variety of situations’ to ‘The following questions ask about thoughts, feelings and behaviors that you may have had in connection to Corona’. Furthermore, we added ‘due to Corona’ at the end of item 1; ‘felt regarding Corona …’ at the beginning of item 2; modified ‘in these situations’ to ‘due to Corona’ in item 3; added ‘caused by anxiety regarding Corona…’ at the beginning of item 4; added ‘due to Corona…’ at the end of item 5; modified ‘these situations’ to ‘situations linked with Corona’ at the end of item 6 and 8; and replaced ‘these situations' with ‘Corona' at the end of item 9. Item 7 and 10 remained unchanged. Supplement 1, available at https://doi.org/10.1192/bjo.2020.82, contains a link to the original SP-D questionnaire by the American Psychiatric Association13 and also full details of our modifications to the SP-D.
We decided to use the non-scientific term ‘Corona’ as we thought it might be more commonally used in the general population. We named the resulting questionnaire the COVID-19-Anxiety Questionnaire (C-19-A).
Design, recruitment and eligibility criteria
To test its psychometric properties, we used data from a large-scale online survey on psychological distress, anxiety and depression in the current COVID-19 pandemic in Germany. This large trial was approved by the ethics commission of Charité Universitätsmedizin Berlin (Approval number: EA1/071/20) and registered on clinicaltrials.gov (NCT04331106). The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
The questionnaire was delivered online via SoSci Survey; participants were recruited mainly using the official social media channels (Twitter and Instagram) of the Charité University Medicine and the Charité University Medicine website. An invitation to participate in the study was posted on each channel once. We did not use paid advertising. No compensation was offered. Participants gave their informed consent. The recruitment period was from 27 March 2020 to 10 April 2020.
Measurement
Participants answered questions about demographics as well as completing the C-19-A. For assessment of concordant validity regarding general anxiety and depressive symptoms we used the four-item Patient Health Questionnaire 4 (PHQ-4)14 an ultra-brief screening tool for anxiety and depression, that can be divided into two subscales of depression (PHQ-2) and anxiety (Generalized Anxiety Disorder, GAD-2). For the assessment of concordant validity regarding specific anxiety symptoms that focus on COVID-19, and with the aim of measuring the extent of anxiety, in that specific anxiety can be problematic, we used two self-developed items: ‘My anxiety concerning Corona leads to limitations in my daily life’ as well as ‘My anxiety concerning Corona is exaggerated’ which could be answered on a six-point Likert scale from not true at all to totally true.
Data analysis
Data was analysed using SPSS 25.0. Internal consistency was calculated using Cronbach's alpha. Concordant validity was calculated using Spearman's Rho.
Results
Participants
Of the 6262 participants, 70.5% (n = 4412) reported gender as female, 28.7% (n = 1796) reported gender as male and 0.9% (n = 54) reported gender as diverse. Tables 1 and 2 provide details of the demographics of the participants and the descriptive statistics of the scales use, respectively.
Table 1.
Demographics of the sample (n = 6262)
| Characteristic | Value |
|---|---|
| Age, years: mean (s.d.) range | 36.43 (11.59) 18–99 |
| Gender, n (%) | |
| Female | 4412 (70.5) |
| Male | 1796 (28.7) |
| Diverse | 54 (0.9) |
| Education level, n (%) | |
| University degree | 3174 (50.7) |
| Higher education entrance qualification | 2009 (32.1) |
| Secondary degree | 942 (15.0) |
| Lower secondary degree | 119 (1.9) |
| No school degree | 18 (0.3) |
| Having children, n (%) | |
| Yes | 2347 (37.5) |
| No | 3915 (62.5) |
| Relationship status, n (%) | |
| Single, widowed, divorced | 2301 (36.7) |
| Stable relationship | 1976 (31.6) |
| Married | 1875 (29.9) |
| Other | 110 (1.8) |
| Working in medical context, n (%) | |
| Yes | 1056 (16.9) |
| No | 5206 (83.1) |
Table 2.
Descriptive statistics of the used scales
| Scale | Mean | s.d. | Range |
|---|---|---|---|
| COVID-19-Anxiety Questionnaire | 10.14 | 6.96 | 0–40 |
| Patient Health Questionnaire-4 | 4.14 | 3.18 | 0–12 |
| Patient Health Questionnaire-2 | 2.11 | 1.69 | 0–6 |
| Generalized Anxiety Disorder-2 | 2.03 | 1.75 | 0–6 |
| Item: Limitation in daily life | 2.41 | 1.53 | 1–6 |
| Item: Anxiety is exaggerated | 2.31 | 1.21 | 1–6 |
Internal consistency
The C-19-A scale showed an internal consistency of α = 0.86 (n = 6262). We did not find great differences in internal consistency when calculated for different subgroups (men: α = 0.86; women: α = 0.86; age ≤30: α = 0.86; age >30 and ≤60: α = 0.87, age >60: α = 0.89).
Concordant validity
The C-19-A showed correlations of rs = 0.58 (n = 6247, P < 0.001) with the PHQ-4; rs = 0.45 (n = 6248, P < 0.001) with the PHQ-2; and rs = 0.60 (n = 6247, P < 0.001) with the GAD-2. Figure 1 shows scatterplots for these associations.
Fig. 1.
Scatterplots for the associations of COVID-19-Anxiety Questionnaire (C-19-A) with the 4-item Patient Health Questionnaire (PHQ-4), PHQ-2 and Generalized Anxiety Disorder (GAD)-2.
Interpolation lines: PHQ-4: y = 4.77 + 1.3 × x, R2-linear, 0.35; PHQ-2: y = 6.09 + 1.92 × x, R2-linear, 0.22; GAD-2: y = 5.13 + 2.47 × x, R2-linear, 0.39.
Furthermore, the C-19-A showed a correlation of rs = 0.61 (n = 6262, P < 0.001) with the item ‘My anxiety concerning Corona leads to limitations in my daily life’ and a correlation of rs = 0.32 (n = 6262, P < 0.001) with the item ‘My anxiety concerning Corona is exaggerated’. When analysed for the different subgroups regarding age and gender, correlations did not show substantial differences between the groups.
Discussion
We found higher average values for C-19-A (mean 10.14) than those reported for the original scale (SP-D) in a sample of 102 patients seeking out-patient treatment for a mental disorder in Germany in 2010 and 2011 (mean 8.2).15 Even though the two measures are not directly comparable, the fact that we found higher average scores on the C-19-A in a sample of the general population than those in a study that assessed a population of patients seeking treatment for a mental disorder seems remarkable, and is in line with the hypothesis that the COVID-19 pandemic causes a large amount of psychological distress in the general population.1 However, these results should be interpreted with great caution as this study is only a cross-sectional assessment and longitudinal studies are needed to generate more information on this subject.
The C-19-A revealed good internal consistency (α = 0.86), which is somewhat lower but comparable with the results of the validation study of the SP-D where a version with only eight items showed an internal consistency of α = 0.91 in a non-clinical sample with 702 participants, and similar values in the 10-item version (α = 0.93 in a non-clinical sample with 57 participants and α = 0.93 in a clinical sample with 48 patients with anxiety disorders).12 This demonstrates that the C-19-A is a reliable method to assess specific anxiety about COVID-19. It is, however, important that test–retest reliability is assessed further; we were not able to analyse this in our study. The rapidly changing situation in the COVID-19 pandemic makes it very difficult to assess test–retest reliability.
Our results demonstrate that the C-19-A achieved good concordant validity with depressive and anxiety symptoms as well as with specific anxiety regarding COVID-19. In terms of general anxiety, our scale revealed a correlation of 0.60 with the GAD-2, whereas the original scale showed a correlation of 0.53 with the GAD-7.15 These results suggest that the anxiety regarding COVID-19 seem to be closely associated with general anxiety symptoms. The correlation of the C-19-A with the PHQ-2 of 0.45 shows that there is also a close relationship with depressive symptomatology, whereas the fact that the correlation is lower than that with general anxiety is in line with our aim to measure specific anxiety symptomatology (discriminant validity). The high correlation of our scale with the item that assessed the amount of limitations that anxiety regarding COVID-19 causes participants in their daily life shows that our scale might be able to assess clinically relevant anxiety regarding COVID-19. This is also supported by the moderate correlation with the item that asks for the extent to which the participants rated their anxiety regarding COVID-19 as exaggerated.
Strengths and limitations
This study presents the first self-rating scale to assess anxiety regarding COVID-19, which seems urgently needed for further research. We used a well-validated measure (SP-D) as the basis and made only slight modifications with the aim of not losing the good psychometric properties of the original instrument. We present data on the psychometric properties from a fairly large sample of over 6000 participants. Nevertheless, there are some limitations.
We were unable to assess test–retest-reliability, the measures used to test concordant validity were very short and it was not possible to assess further the differences between general anxiety and COVID-19 anxiety. Therefore, further studies with longitudinal data and structural equational modelling are needed. Another limitation comes from the fact that we used a convenience sample that might come with sample bias; those familiar with social media such as young participants and also people with higher levels of anxiety might have been more likely to participate in our study. As a result of this, levels of anxiety might have been overestimated in our study. Furthermore, we were not able to assess how many people received the invitation to participate in the study and therefore we were unable to calculate the response rate. The recruitment strategy used may have reduced the generalisability of our results as there are some differences between our sample and the general population in Germany (for example lower age and overrepresentation of women). However, in our sample, no substantial differences in internal validity and concurrent validity were found for different subgroups regarding age and gender.
Implications
In conclusion, we present the first self-rating scale for anxiety relating to COVID-19. The instrument is a modification of the SP-D scale. Our data show that the scale has good internal consistency as well as good concordant validity. This instrument might help researchers identify determinants of adaptive and mal-adaptive psychological responses to COVID-19, such as COVID-19-specific anxiety. Thereby it might help in identifying subgroups of the general population that show a higher risk for anxiety and enable the development of individually targeted treatment options for individuals at higher risk for psychological burden as a result of the COVID-19 pandemic.
Author contributions
M.B.P.: literature research, questionnaire construction, data collection, data analysis, data interpretation, writing, revision of the manuscript. A.B.: literature research, questionnaire construction, data collection, data. interpretation, writing, revision of the manuscript. J.P.: literature research, ethics committee communication, questionnaire construction, writing, revision of the manuscript. L.P.: literature research, data protection committee communication, questionnaire construction, data collection, writing, revision of the manuscript. L.M.M.: literature research, questionnaire construction, translation, writing, revision of the manuscript. F.B.: literature research, questionnaire construction, revision of the manuscript. J.R.: literature research, writing, revision of the manuscript. J.G.: literature research, questionnaire construction, data collection, writing, revision of the manuscript. A.S.: literature research, questionnaire construction, data interpretation, writing, revision of the manuscript.
Supplementary material
For supplementary material accompanying this paper visit http://dx.doi.org/10.1192/bjo.2020.82.
click here to view supplementary material
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Declaration of interest
None.
ICMJE forms are in the supplementary material, available online at https://doi.org/10.1192/bjo.2020.82
References
- 1.Petzold MB, Plag J, Ströhle A. COVID-19-Pandemie: psychische Belastungen können reduziert werden. [COVID-19-Pandemic: Psychological distress can be reduced.] Dtsch Arztebl 2020; 117: 648–54. [Google Scholar]
- 2.Asmundson GJG, Taylor S. Coronaphobia: fear and the 2019-nCoV outbreak. J Anxiety Disord 2020; 70: 102196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Torales J, O'Higgins M, Castaldelli-Maia JM, Ventriglio A. The outbreak of COVID-19 coronavirus and its impact on global mental health. Int J Soc Psychiatry 2020; 66: 317–20. [DOI] [PubMed] [Google Scholar]
- 4.Wang C, Pan R, Wan X, Tan Y, Xu L, Ho CS, et al. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int J Environ Res Public Health 2020; 17: 1729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Qiu J, Shen B, Zhao M, Wang Z, Xie B, Xu Y. A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: implications and policy recommendations. Gen Psychiatr 2020; 33: e100213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Wang C, Pan R, Wan X, Tan Y, Xu L, McIntyre RS, et al. A longitudinal study on the mental health of general population during the COVID-19 epidemic in China. Brain Behav Immun 2020; 87: 40–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.González-Sanguino C, Ausín B, Castellanos MÁ, Saiz J, López-Gómez A, Ugidos C, et al. Mental health consequences during the initial stage of the 2020 Coronavirus pandemic (COVID-19) in Spain. Brain Behav Immun 2020; 87: 172–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zhang Y, Ma ZF. Impact of the COVID-19 pandemic on mental health and quality of life among local residents in Liaoning Province, China: a cross-sectional study. Int J Environ Res Public Health 2020; 17: 2381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Mazza C, Ricci E, Biondi S, Colasanti M, Ferracuti S, Napoli C, et al. A nationwide survey of psychological distress among Italian People during the COVID-19 pandemic: immediate psychological responses and associated factors. Int J Environ Res Public Health 2020; 17: 3165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Shah K, Kamrai D, Mekala H, Mann B, Desai K, Patel RS. Focus on mental health during the coronavirus (COVID-19) pandemic: applying learnings from the past outbreaks. Cureus 2020; 12: e7405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lee SM, Kang WS, Cho AR, Kim T, Park JK. Psychological impact of the 2015 MERS outbreak on hospital workers and quarantined hemodialysis patients. Compr Psychiatry 2018; 87: 123–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lebeau RT, Glenn DE, Hanover LN, Beesdo-Baum K, Wittchen H-U, Craske MG. A dimensional approach to measuring anxiety for DSM-5. Int J Methods Psychiatr Res 2012; 21: 258–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Craske M, Wittchen U, Bogels S, Stein M, Andrews G, Lebeu R. Severity Measure for Specific Phobia—Adult. American Psychiatric Association, 2020. (https://www.psychiatry.org/File%20Library/Psychiatrists/Practice/DSM/APA_DSM5_Severity-Measure-For-Specific-Phobia-Adult.pdf). [Google Scholar]
- 14.Löwe B, Wahl I, Rose M, Spitzer C, Glaesmer H, Wingenfeld K, et al. A 4-item measure of depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord 2010; 122: 86–95. [DOI] [PubMed] [Google Scholar]
- 15.Beesdo-Baum K, Klotsche J, Knappe S, Craske MG, Lebeau RT, Hoyer J, et al. Psychometric properties of the dimensional anxiety scales for DSM-V in an unselected sample of German treatment seeking patients. Depress Anxiety 2012; 29: 1014–24. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
For supplementary material accompanying this paper visit http://dx.doi.org/10.1192/bjo.2020.82.
click here to view supplementary material
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.