Skip to main content
PMC home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2021 Sep 3;295:1131–1137. doi: 10.1016/j.jad.2021.08.104

Measuring COVID-19 related anxiety and obsession: Validation of the Coronavirus Anxiety Scale and the Obsession with COVID-19 Scale in a probability Chinese sample

Juliet Honglei Chen a,b, Kwok Kit Tong a, Xiaoyu Su a, Eilo Wing-yat Yu c, Anise MS Wu a,b,⁎
PMCID: PMC8414841  PMID: 34706425

Abstract

Background

With concern over the rise in mental health symptoms associated with the COVID-19 pandemic, the present study set out to address the absence of pandemic-specific screening tools for detecting those in Chinese societies who are at-risk for experiencing mental distress due to the pandemic; thus, its aim was to validate the Coronavirus Anxiety Scale (CAS) and Obsession with COVID-19 Scale (OCS) in Chinese adults.

Methods

With a two-stage cluster random sampling method, we surveyed 1011 Chinese community-dwelling adults (38.8% men; 41.2 years old on average with an SD of 15.8) in June and July of 2020.

Results

Our psychometric evaluation results showed that the Chinese version of CAS and OCS retained their original one-dimensional structure and demonstrated measurement invariance across genders. In line with validation studies of the CAS and OCS in other languages, subsequent analyses also provided support to our Chinese version with respect to their satisfactory internal consistency (α = .87 and .73, respectively), and good concurrent validity (i.e., positive associations with negative feelings, excessive time-consumption, subjective distress, and functioning impairment).

Limitations

Due to constraints of time and cross-sectional design, we only validated CAS and OCS among Chinese adults and did not evaluate their test-retest reliability nor predictive validity.

Conclusions

Considering the practical benefits of understanding the source of mental symptoms during the pandemic, we recommend the use of CAS and OCS in Chinese communities to facilitate early identification and intervention for those who require clinical attention due to their COVID-19 related anxiety and obsessive thoughts.

Keywords: COVID-19, Coronavirus, Pandemic, Anxiety, Obsession, Assessment, Mental health

1. Introduction

Since the end of 2019, the newly discovered coronavirus SARS-CoV-2, which can cause coronavirus disease (COVID-19), has quickly developed into one of the deadliest pandemics in the 21st century, with a global death toll of over 2.7 million as of March 2021 (World Health Organization WHO, 2021). The emergence of symptoms of anxiety and obsessive thoughts as epidemics and pandemics breakout and progress has been well-documented with respect to other public health crises (e.g., Blakey et al. 2015, Brand et al. 2013, Xie et al. 2011). In a manner similar to responses to other crises, such as severe acute respiratory syndrome (SARS; Wong et al., 2007) and the swine flu (Rubin et al., 2009), various groups have experienced heightened levels of anxiety and related symptoms during the COVID-19 pandemic (e.g., Choi et al. 2020a, 2020b, González-Sanguino et al. 2020, Huang and Zhao 2020). A reasonable level of anxiety and cognitive arousal evoked by the pandemic may enhance one's level of awareness concerning the need to protect oneself from the virus and help lower one's susceptibility to infection; however, excessive amounts of anxiety and obsessive thoughts tend to bring about psychosomatic symptoms and eventually lead to mental disorders (Taylor, 2019). Because the impact to mental wellbeing brought by pandemics tends to be pervasive, long-lasting, and most likely occurs in several waves (Taylor, 2019), regular screenings throughout the pandemic and even beyond for maladaptive anxiety and obsessive thoughts in communities are greatly needed.

Whereas extant findings (e.g., Choi et al. 2020a, 2020b, González-Sanguino et al. 2020, Huang and Zhao 2020) have substantiated the need to screen for anxiety and related symptoms during the COVID-19 pandemic, they have also exposed one shared drawback—the inability to differentiate anxiety and associated symptoms that are directly related to the pandemic from those that are not. For example, all of the recent studies on pandemic-related anxiety in Chinese societies have adopted generalized assessment tools, such as the Generalized Anxiety Disorder Scale (GAD-7; e.g., Choi et al. 2020a, 2020b, Huang and Zhao 2020), the State-Trait Anxiety Inventory (STAI; e.g., Lin et al. 2020, Wong et al., 2020) and the Depression, Anxiety, and Stress Scale (DASS-21; e.g., Chen et al. 2021, Wang et al. 2020), instead of pandemic-specific tools, to measure individuals’ anxiety level. Although these scales are common and valid instruments in evaluating anxiety levels in general, they are unable to identify the possible origins of anxiety-related symptoms: symptoms directly triggered by thinking or being exposed to coronavirus-related information (i.e., COVID-19 anxiety) versus symptoms elicited by lifestyle changes (e.g., social distancing) and indirectly associated with the COVID-19 pandemic (i.e., the general anxiety). A significant difference between the COVID-19 related anxiety and the general anxiety lies in that the former comprises broader but specific, excessive worries about coronavirus, which include but are not limited to a morbid preoccupation about getting infected with COVID-19 (i.e., COVID-19 health anxiety, Tyrer 2020) and a generalized, dysfunctional fear of COVID-19 (i.e., coronaphobia; Asmundson and Taylor, 2020); in contrast, the latter only entails generalized anticipation of a future threat (American Psychiatric Association [APA], 2013). The development of the general anxiety is more likely to be related to disorders under the DSM-5 anxiety disorders; while the COVID-19 anxiety may be associated with disorders not only under the DSM-5 anxiety disorders but also under somatic symptoms and related disorders (e.g., illness anxiety disorder) and trauma- and stress- related disorders. Failure to differentiate the sources of anxiety and related symptoms linked with COVID-19 might hinder an accurate diagnosis, delay the process of effective intervention, and eventually lead to individuals developing more serious mental disorders, such as posttraumatic stress disorder, obsessive and compulsive disorder, illness anxiety disorder, or generalized anxiety disorder.

Unlike those generalized scales, pandemic-specific screening tools can reveal a clearer picture of the direct impact on the mental health of the pandemic. For example, the specific scales developed for AIDS (Snell and Finney, 1998) and the swine flu (Wheaton et al., 2012) were able to provide more precise identification of potential causes of mental symptoms during each epidemic or pandemic wave and to inform health professionals and policy-makers of a more cost-effective approach to prevention and intervention planning that targeted the core source(s) of the problem. Furthermore, in screening symptoms with disease/situation-based scales, clinicians can attend to idiosyncrasies that might be at play under unusual circumstances and thus are able to overcome the limitations of generalized tools (for a discussion, see Sarason, 1978). Therefore, devising and validating specific measurement tools for COVID-19 related mental symptoms may facilitate the efficiency of health professionals in targeting the source of the distress and hence safeguard the public's mental health in general.

In response to the COVID-19 global outbreak, Lee (2020b, 2020c) developed two short, specialized assessment tools, the Coronavirus Anxiety Scale (CAS) and the Obsession with COVID-19 Scale (OCS), to screen for COVID-19 induced mental distress that may need clinical attention. The CAS is a 5-item self-report screen for identifying people who might suffer from an unhealthy degree of anxiety due to the COVID-19 pandemic, which mainly entails behavioral and physiological reactions of elevated fear and excessive worry about coronavirus and its related stimuli (Lee, 2020a, 2020c). It has an optimal cutoff score of ≥ 9, which corresponds to 90% sensitivity and 85% specificity (Lee, 2020c), and has demonstrated satisfactory reliability and validity initially in an American adult sample (Lee, 2020c) and subsequently in various other ethnic samples, such as in South Koreans (Choi et al., 2020), Portuguese (Magano et al., 2021), and Turks (Evren et al., 2020). To date, the CAS has been included in the World Psychiatry Association's COVID-19 Mental Health Care Toolkit for quick screening for COVID-19 related anxiety at the primary care level (Adiukwu et al., 2020). The 4-item OCS was developed for discerning maladaptive thinking about COVID-19, which approximately means spending at least three to seven days in a two-week time-frame on dreaming and repetitively thinking about the coronavirus and having disturbing thoughts that one has caught the coronavirus and that one saw particular people who may have the coronavirus (Lee, 2020b). Its optimal cutoff score was proposed as ≥ 7, indicating 81-93% sensitivity and 73-75% specificity. Its solid reliability and validity were found in the two American adult samples initially used for scale development (Lee, 2020b) and were later confirmed in several other samples with its different language versions (e.g., Andrade et al. 2021, Ashraf et al. 2020, Choi et al., 2020). Nevertheless, to date, a Chinese validation of Lee's CAS and OCS is not yet available.

The present study aimed to address the lack of specialized scales for assessing symptoms of mental distress during the COVID-19 pandemic in Chinese societies by validating a Chinese version of Lee's CAS and OCS (Lee, 2020b, Lee, 2020c)with a probability Chinese adult sample. The validated Chinese version of CAS and OCS can be used as quick, easy-to-administer screening tools for early identification of at-risk cases of COVID-19 related anxiety and obsession that may warrant clinical attention in both clinical and public settings. They may also inform the development of tailored mental interventions and support to those who are suffering more directly from mental distress due to situational factors surrounding, as well as thoughts and fears concerning, coronavirus-related anxiety and obsession, which may develop into more serious, intrusive symptoms that are part of the criteria of posttraumatic stress disorder or illness anxiety disorder (for a discussion, see American Psychiatric Association [APA], 2013).

2. Methods

2.1. Respondents and procedures

We designed a telephone survey with a two-stage cluster random sampling method to acquire a probability sample of local residents in Macao, China. The inclusion criteria were male and female local adults (18 years old or above), with the ability to understand and speak Cantonese or Mandarin Chinese. In the first-stage of sampling, we randomly selected local households from the recent Macao residential phonebook as clusters. For households with more than one eligible respondent, the second-stage of random sampling was performed using the last-birthday rule (Gaziano, 2008), in which the individual who most recently had his or her birthday was invited to participate in the survey voluntarily. The selected respondents received a briefing by trained research assistants on the nature of the study and their rights as research participants; there were no monetary incentives. The formal survey was administrated to only those who explicitly gave us oral consent to participate. Ethical approval of this study was acquired from the affiliated university of the corresponding author.

During late June and early July of 2020, we successfully interviewed 1011 respondents, which represented a cooperation rate of 80.6% (i.e., the percentage of all cases interviewed versus all eligible respondents ever contacted) according to the calculation method advised by the American Association for Public Opinion Research (2016). There were 38.8% men (95% CI [35.8%, 41.8%]) and 61.2% women (95% CI [58.2%, 64.2%]) whose average age was 41.2 years (SD = 15.8; range = 18–94). The majority of the respondents received education at tertiary (50.2%), senior high (28.2%), or junior high (12.5%) levels, while only a small portion were educated at the primary school (6.6%) or no formal education levels (1.7%). No respondents had a history of COVID-19 infection, and only 12 respondents (1.2%) reported knowing friend(s) or relative(s) who had been infected by COVID-19.

2.2. Measures

2.2.1. Coronavirus anxiety and obsession with COVID-19

The Chinese version of CAS and OCS was developed in three stages. First, Lee's CAS and OCS (2020b, 2020c) were translated to Chinese and then back-translated to English independently by two bilingual researchers. Second, minor inconsistencies between the translation and back-translation (e.g., whether to stress the “cold feeling” in addition to “being unable to move” for the translation of “frozen” for CAS Item C) were discussed and resolved with further consultation with a third bilingual psychologist. A preliminary Chinese version of these two scales was generated based on the three experts’ unanimous agreement. Third, two additional psychologists independently evaluated the consistency between the translated Chinese version and the original English version and gave their endorsement to the translated version, which was subsequently finalized for use in the survey.

Consistent with the original English version, the Chinese version of 5-item CAS and 4-item OCS both adopted a 5-point Likert scale, ranging from 0 = not at all to 4 = nearly every day. The CAS captured information regarding how frequently respondents experienced COVID-19 anxiety in the past two weeks regarding behavioral and physiological symptoms (e.g., “I felt dizzy, lightheaded, or faint, when I read or listened to news about the coronavirus”). The OCS measured the frequency of persistent and disturbing thinking about COVID-19 (e.g., “I had disturbing thoughts that I may have caught the coronavirus”) during the previous two weeks.

2.2.2. Negative feelings

The 6-item negative feelings subscale of the Chinese version of Diener et al.’s (2010) scale of positive and negative experience (Tong and Wang, 2017) was used to assess respondents’ negative feelings, such as unpleasant, sad, and afraid, during the past four weeks, on a 5-point Likert scale, which ranged from 1 = very rarely or never to 5 = very often or always. A sample question is: “How much have you experienced the feeling of being afraid in the past four weeks?” A summative score of the six items was computed to reflect the extent of one's experience of negative feelings in the recent month. Higher scores represented greater degrees of negative feeling. Cronbach's alpha of this subscale was .87 in the present study.

2.2.3. Symptom severity indicators

Three symptom severity indicators were included to evaluate potential adverse consequences associated with coronavirus anxiety and obsessive thinking: (a) excessive time-consumption (i.e., “thinking about coronavirus related things have taken too much of my time”), (b) subjective distress (i.e., “thinking about coronavirus related things have caused me psychological distress”), and (c) functioning impairment (i.e., “thinking about coronavirus has affected my daily life and/or work”). Respondents were prompted to rate the frequency they had experienced the condition described in each indicator in the past two weeks on a 5-point Likert scale, ranging from 0 = not at all to 4 = nearly every day. Each indicator was treated separately in the subsequent analyses. Higher indicator scores represented greater levels of symptom severity.

2.2.4. Demographics

Respondents were requested to provide information on gender, age, and educational attainment (i.e., six levels from no formal education to tertiary education with a bachelor's degree or above). Additionally, respondents were asked whether they and their family and friends had ever been infected by COVID-19 (1 = yes, 0 = no).

2.3. Statistical analysis

We first employed descriptive statistics to provide a profile of the sample's demographic and psychological characteristics. Psychometric evaluation of the two measures began with a series of item analyses on each item of CAS and OCS, including the endorsement rate on the response scale, corrected-total correlation, squared-multiple correlation, and Cronbach's alpha if the item was deleted. Subsequently, we assessed the model fit of the original one-factor structure of CAS and OCS to the data independently with confirmatory factor analysis (CFA). A model was considered to be acceptable if it met the following criteria: (a) a comparative fit index (CFI) ≥ .90 and a Tucker–Lewis fit index (TLI) ≥ .90 (Hu and Bentler, 1999), (b) a root mean square error of approximation (RMSEA) ≤ .08, and a standardized root mean square residual (SRMR) ≤ .08 (Browne and Cudeck, 1992; Hu and Bentler, 1999). Further CFA testing was performed to examine the model's invariance across both genders on configural, metric, and scalar levels for CAS and OCS, respectively. The corresponding level of measurement invariance was defined as an acceptable model fit with a nonsignificant χ2 difference test (p > .05; Vandenberg and Lance, 2000). Using bivariate correlation analysis, the concurrent validity of CAS and OCS was then evaluated by testing their hypothesized positive associations with negative feelings and three symptom severity indicators. Additional non-parametric analyses were performed to explore the potential differences between high and low COVID-19 anxiety/obsession groups to examine the screening efficacy of the cutoff scores of CAS and OCS, respectively. The CFA analyses were conducted in Mplus 7.3 (Muthén and Muthén, 2012) and utilized the robust Maximum Likelihood (MLR) as the model estimator because it does not assume multivariate normality and takes missing values into account (i.e., four missing cases [0.4%] for CAS items and three missing cases [0.3%] for OCS items). Other analyses were performed in SPSS 25.0 (IBM Corp, 2017).

3. Results

Table 1 shows the preliminary item performance of the CAS and OCS items in terms of their endorsement, descriptive statistics, and item metrics. Because these items were designed to screen for individuals who are at-risk for mental conditions, positively skewed responses of all the items in both CAS and OCS were expected and observed (i.e., Mdn = 0, M = 0.21 to 0.35, SD = 0.66 to 0.85 for CAS and Mdn = 0, M = 0.25 to 0.97, SD = 0.67 to 1.33 for OCS). The most high-level (i.e., reported scores of “3 = more than 7 days [in 2 weeks]” or above) items of CAS and OCS endorsed were “I felt dizzy, lightheaded, or faint, when I read or listened to news about the coronavirus” (5.1%) and “I cannot stop thinking about coronavirus” (15.7%), respectively. The corrected item-total correlation of the CAS and OCS items ranged from .68 to .73 and from .45 to .68, respectively, exceeding the minimum threshold of .30 (Boateng et al., 2018).

Table 1.

Item analysis and CFA factor loadings of the Chinese version of CAS and OCS items (N = 1011).

Scales/Items Endorsement % by Responsea
 Descriptive Statistics
 Item Metrics
 Factor Loadingb
0 1 2 3 4 M SD ITC SMC CID
1. CAS
a. I felt dizzy, lightheaded, or faint, when I read or listened to news about the coronavirus. 83.4 6.5 5 3.5 1.6 0.33 0.85 0.68 0.48 0.86 0.72⁎⁎⁎
b. I had trouble falling or staying asleep because I was thinking about the coronavirus. 80.8 8.6 6.7 2.8 1.1 0.35 0.81 0.71 0.52 0.85 0.75⁎⁎⁎
c. I felt paralyzed or frozen when I thought about or was exposed to information about the coronavirus. 87.4 6.6 3.3 1.7 0.9 0.22 0.67 0.71 0.51 0.85 0.77⁎⁎⁎
d. I lost interest in eating when I thought about or was exposed to information about the coronavirus. 84.0 7.4 5.3 2.1 1.1 0.28 0.74 0.73 0.57 0.84 0.79⁎⁎⁎
e. I felt nauseous or had stomach problems when I thought about or was exposed to information about the coronavirus. 87.4 6.9 2.8 1.8 0.9 0.21 0.66 0.71 0.55 0.85 0.78⁎⁎⁎
2. OCS
a. I had disturbing thoughts that I may have caught the coronavirus. 70.7 13.0 10.1 4.5 1.7 0.54 0.96 0.60 0.48 0.62 0.76⁎⁎⁎
b. I had disturbing thoughts that certain people I saw may have the coronavirus. 67.4 13.5 11.4 5.8 2.0 0.62 1.03 0.68 0.54 0.56 0.90⁎⁎⁎
c. I could not stop thinking about the coronavirus. 55.8 16.6 11.7 6.4 9.3 0.97 1.33 0.44 0.20 0.75 0.48⁎⁎⁎
d. I dreamed about the coronavirus. 85.1 8.6 3.3 2.6 0.4 0.25 0.67 0.45 0.22 0.72 0.50⁎⁎⁎
Open in a new tab

Note. CAS = coronavirus anxiety scale, OCS = obsession with COVID-19 scale, ITC = corrected item-total correlation, SMC = squared-multiple correlation, CID = Cronbach's alpha if item deleted. ⁎⁎⁎p < .001.

a

The five-point response of CAS and OCS correspondents to 0 = Not at all, 1 = Rare, less than a day or two, 2 = several days, 3 = more than seven days, 4 = nearly nevery day over the last two weeks.

b

The goodness-of-fit indices for CAS was χ2(5) = 36.03, p < .001, CFI = .95, TLI = .90, RMSEA = .078, 90% CI [.056, .103], SRMR = .031; The goodness-of-fit indices for OCS was χ2(2) = 4.46, p = .11, CFI = .99, TLI = .99, RMSEA = .035, 90% CI [<.001, .079], SRMR = .015.

Subsequently, the goodness-of-fit of the hypothesized unidimensional structure for CAS and OCS was evaluated with our data independently. The CFA results demonstrated a satisfactory model fit for both CAS and OCS (CAS: χ2(5) = 36.03, p < .001, CFI = .95, TLI = .90, RMSEA = .078, 90% CI [.056, .103], SRMR = .031; OCS: χ2(2) = 4.46, p = .11, CFI = .99, TLI = .99, RMSEA = .035, 90% CI [<.001, .079], SRMR = .0151). As shown in Table 1, the item factor loadings were statistically significant and ranged from .72 to .79 for CAS and from .48 to .90 for OCS. Most of these factor loadings were greater than the good rule of thumb of .50 or higher and ideally .70 or higher proposed by Hair et al. (2014). Although two OCS items (i.e., items c and d) only presented a relatively lower factor loading of .48 and .50, we followed Knekta et al.’s (2019) suggestions and retained these two items for their essential contribution to the OCS construct as core diagnostic criteria. Given the confirmed unidimensionality of the two scales, the internal consistency of the 5-item CAS and the 4-item OCS was also satisfactory, with Cronbach's alphas of .87 and .73, respectively.

Table 2 shows that the invariance model fit the data of male and female respondents well for both CAS and OCS at configural, metric, and scalar levels (CAS: CFI = .961 to .967, TLI = .914 to .951, RMSEA = .054 to .071, SRMR = .023 to .038; OCS: CFI = .983 to .995, TLI = .948 to .994, RMSEA = .023 to .068, SRMR = .023 to .025). By comparing the metric model with the configural model, we found scaled Δχ2(4) = 5.37, p = .25 for CAS and Δχ2(3) = 0.83, p = .84 for OCS, which indicated they both met the metric invariance criteria. The scalar invariance also held for both CAS (scaled Δχ2(4) = 3.61, p = .46) and OCS (scaled Δχ2(3) = 0.24, p = .97). To conclude, our findings supported that factor loadings and measurement intercepts were equal across genders for both CAS and OCS.

Table 2.

Test of measurement invariance of the Chinese version of CAS and OCS (N = 1011).

Model Model fit
 Model comparisona
χ2 df p CFI TLI RMSEA [90% CI] SRMR Δχ2 Δdf p ΔCFI ΔTLI
1. CAS
1a. Configural 28.47 8 <.001 .966 .914 .071 [.044, .100] .023
1b. Metric 31.85 12 .002 .967 .945 .057 [.033, 082] .036 5.37 4 .25 .001 .031
1c. Scalar 39.24 16 .001 .961 .951 .054 [.033, .075] .038 3.61 4 .46 .006 .006
2. OCS
2a. Configural 13.26 4 .010 .983 .948 .068 [.030, .110] .023
2b. Metric 10.85 7 .145 .993 .988 .033 [<.001, .069] .025 0.83 3 .84 .010 .040
2c. Scalar 12.66 10 .243 .995 .994 .023 [<.001, .056] .025 0.24 3 .97 .002 .006
Open in a new tab

Note. CAS = coronavirus anxiety scale, OCS = obsession with COVID-19 scale.

a

Chi-square difference testing (Δχ2) is carried out using scaling correction factor for MLR

As presented in Table 3 , the concurrent validity of CAS was supported by its significant, positive association with negative feelings, excessive time-consumption, subjective distress, and functioning impairment, with a mild-to-strong effect size (r = .27, .43, .52, .42, respectively; p < .001). Similarly, OCS had significant moderate-to-strong correlations with negative feelings, excessive time-consumption, subjective distress, and functioning impairment (r = .36, .56, .55, .47, respectively; p < .001). As expected, a strong, positive association between CAS and OCS (r = .55, p < .001) was also found.

Table 3.

Descriptive statistics and bivariate correlation among key variables (N = 1011).

Variables M (SD) 1 2 3 4 5
1. CAS 1.40 (3.05) 1
2. OCS 2.36 (3.04) .55⁎⁎⁎ 1
3. Negative feelings 2.03 (0.81) .27⁎⁎⁎ .36⁎⁎⁎ 1
4. Excessive time-consumption 0.60 (1.02) .43⁎⁎⁎ .56⁎⁎⁎ .23⁎⁎⁎ 1
5. Subjective distress 0.60 (1.06) .52⁎⁎⁎ .55⁎⁎⁎ .35⁎⁎⁎ .55⁎⁎⁎ 1
6. Functioning impairment 0.96 (1.38) .42⁎⁎⁎ .47⁎⁎⁎ .34⁎⁎⁎ .49⁎⁎⁎ .58⁎⁎⁎
Open in a new tab

Note. CAS = coronavirus anxiety scale, OCS = obsession with COVID-19 scale. ⁎⁎⁎p < .001.

When applying the criteria for an optimal cutoff score of ≥ 9 for CAS and ≥ 7 for OCS proposed by Lee (2020b, 2020c), we estimated 4.0% of our respondents reported scores in keeping with excessive levels of coronavirus anxiety (i.e., high COVID-19 anxiety group), whereas 10.5% reported excessive levels of obsession with COVID-19 (i.e., high COVID-19 obsession group). Furthermore, we explored potential differences between high and low COVID-19 anxiety/obsession groups (based on the cutoff scores of CAS and OCS), respectively, regarding demographic features and endorsement on negative feelings and three symptom severity indicators. By comparing high and low COVID-19 anxiety groups, we did not observe a significant difference in gender (χ2(1) = 0.25, p = .62), age (MRank: High = 512.48, Low = 490.09, Mann-Whitney U = 17961.00, p = .62), nor educational level (MRank: High = 469.50, Low = 503.35, Mann-Whitney U = 17960.00, p = .44). Respondents in the high COVID-19 anxiety group reported a significantly higher level of negative feelings (MRank: High = 777.74, Low = 494.81, Mann-Whitney U = 8550.50, p < .001), excessive time-consumption (MRank: High = 773.15, Low = 494.46, Mann-Whitney U = 8694.00, p < .001), subjective distress (MRank: High = 876.84, Low = 490.72, Mann-Whitney U = 4568.50, p < .001), and functioning impairment (MRank: High = 819.81, Low = 493.07, Mann-Whitney U = 6867.50, p < .001) than their counterparts. As for high and low COVID-19 obsession groups, we found a significant difference in age (MRank: High = 377.02, Low = 504.23, Mann-Whitney U = 33203.50, p < .001), but not in gender (χ2(1) = 0.20, p = .66) nor educational level (MRank: High = 539.06, Low = 497.62, Mann-Whitney U = 43613.00, p = .14). Respondents in the high COVID-19 obsession group also endorsed significantly more negative feelings (MRank: High = 732.33, Low = 479.49, Mann-Whitney U = 23973.50, p < .001), excessive time-consumption (MRank: High = 787.31, Low = 472.46, Mann-Whitney U = 18040.50, p < .001), subjective distress (MRank: High = 774.64, Low = 474.53, Mann-Whitney U = 19489.00, p < .001), and functioning impairment (MRank: High = 764.48, Low = 475.73, Mann-Whitney U = 20566.50, p < .001) than those in the low COVID-19 obsession group.

4. Discussion

The purpose of the present study was to validate a Chinese version of two coronavirus-specific mental health assessment tools, CAS and OCS, with a probability sample of Chinese adults; its two-stage cluster random sampling approach increased the generalizability of the findings to the target adult population. Our results supported the unidimensional structure of CAS and OCS initially proposed by Lee (2020b, 2020c), which indicates one latent construct for the six CAS items and the four OCS items, respectively. Consistent with previous findings from studies that validated these instruments in various languages, our findings lent extra support to the cross-cultural universality of the unidimensionality of the structure of CAS and OCS, in line with the Bangla (Ahmed et al., 2020), Korean (Choi et al., 2020), Portuguese (Magano et al., 2021), and Turkish (Evren et al., 2020) versions of CAS, as well as the Korean (Choi et al., 2020), Brazilian Portuguese (Andrade et al., 2021), and Urdu (Ashraf et al., 2020) versions of OCS. As for our finding of a relatively lower factor loading (i.e., .48 and .50) of two OCS items (i.e., Item c and d), the OCS's validation studies in other languages versions have also reported a similar pattern for these two items, such as .41 for Item c in the Urdu version (Ashraf et al., 2020), .45 for Item d in the Korean version (Choi et al., 2020), and .55 for Item d in the Brazilian Portuguese version (Andrade et al., 2021). Follow-up studies are warranted to explore further these two items’ contributions to the latent construct of obsession with COVID-19 across various regions and/or different courses as the COVID-19 pandemic develops. Furthermore, we found measurement invariance across genders for both CAS and OCS with our data at a scalar level. Not all researchers have tested for the gender invariance of CAS and OCS in the psychometric property evaluation of the two (e.g., Choi et al., 2020), while those who did have reported a robust gender invariance finding for CAS (Ahmed et al., 2020; Lee, 2020c) and OCS (Ashraf et al., 2020), similar to ours. Because previous studies have not yet specified at which level the gender invariance was held, our findings have revealed a more comprehensive picture of the measurement invariance across genders in Chinese.

Along the lines of previous validation studies (e.g., Choi et al., 2020; Evren et al., 2020), we confirmed good internal consistency of CAS and OCS in Chinese adults, with Cronbach's alphas of .87 and .73, respectively, with our data. Because previous studies have also reported an increase in psychological distress and other negative consequences with higher levels of disease-specific anxiety (e.g., Wheaton et al. 2012), the significant, positive association of the Chinese version of CAS and OCS assessed symptoms with negative feelings, excessive time-consumption, subjective distress, and functioning impairment provided empirical evidence to the concurrent validity of CAS and OCS. Our findings were also consistent with those of existing studies on CAS and OCS, which unanimously showed associations between CAS and OCS scores and adverse health outcomes, such as negative feelings or symptoms of psychological distress in general (Ashraf et al., 2020; Choi et al., 2020; Khosravani et al., 2020), worse quality of life (Andrade et al., 2021), and impairment in functioning (Choi et al., 2020; Lee, 2020b). These findings lend support to the CAS and OCS scores in their capacity to reliably indicate the level of maladaptive anxiety and compulsive thoughts regarding the COVID-19 epidemic.

Using the cutoff score proposed by Lee (2020b, 2020c), we identified 4.0% and 10.5% of the sampled respondents as cases that are at-risk for excessive mental distress due to coronavirus anxiety and obsession, respectively. As shown by our data, these at-risk cases reported significantly higher levels of negative feelings, excessive time-consumption, subjective distress, and functioning impairment than their counterparts, supporting the screening efficacy of this cutoff score for Chinese. Using the identical cutoff scores of CAS ≥ 9 and OCS ≥ 7, cases showing excessive distress due to coronavirus anxiety and thinking of coronavirus were more prevalent in our Chinese community adult sample when compared to similar general population studies conducted in Korea (N = 329, 3.3% for anxiety and 5.5% for obsession; Choi et al., 2020), and Turkey (N = 763, 3.80% for anxiety; Yurttas et al., 2021); while an Indian general population study (N = 2004; Srivastava et al., 2020) reported a lower rate of CAS screened anxiety (i.e., 3.29%), but a higher rate of OCS screened obsession (i.e., 13.47%) than ours. This difference in the prevalence of pandemic-specific anxiety may imply subtle stress response patterns across various ethnicities, pointing to a need for tailored prevention and intervention measures for people of different origins. By comparing at-risk case rates at general and pandemic-specific levels, we also detected a noticeable difference between the two in terms of a sizable higher rate of the former than the latter (e.g., 12.0% of probable generalized anxiety cases assessed by DASS-21 in Chen et al., 2020; versus 4.0% of at-risk for coronavirus anxiety cases in our sample). These differences in prevalence rates indicate the need to differentiate the source of mental symptoms reported during the pandemic period and discern the pandemic-specific disordered cases from the general ones for targeted interventions. For example, in order to implement more cost-effective interventions, one may consider focusing more on coronavirus anxiety in women given their higher reported CAS scores than their male counterparts.

5. Limitations

This study has three limitations. First, because of time restrictions of the phone survey, we did not measure the general anxiety and thus could not directly assess its association with the COVID-19 anxiety in our study. Admittedly, extant empirical studies have already established the distinctions between the general anxiety and the COVID-19 anxiety in terms of varying prevalence in the same sample (36.0% versus 25.4%, respectively; Lee et al., 2021) and a relatively high but not too high correlation coefficient (e.g., .75 among South Koreans and .57 among Brazilians; Choi et al., 2020; Padovan-Neto et al., 2021); however, it is still meaningful to explore how the two types of anxiety may interact with each other to shed light on tailed prevention and intervention strategies. Second, we only validated CAS and OCS among Chinese adults due to time constraints and limited resources. Given that pandemic-specific symptoms may also prevail in younger groups (e.g., Chen et al., 2020; Courtney et al., 2020), we recommend that subsequent studies further adapt and test adolescent and children's versions of pandemic-specific mental health screening tools. Third, because of our cross-sectional design, we were unable to evaluate the test-retest reliability nor the predictive validity of CAS and OCS. Due to the inevitable fluctuations of mental symptoms during the pandemic (Taylor, 2019; Wang, et al., 2020), longitudinal studies are warranted to track their changes over time.

6. Conclusions

To conclude, we validated CAS and OCS in a probability sample of Chinese community adults to address the need for pandemic-specific screening tools to assess for symptoms of psychological distress associated with the coronavirus in the Chinese context. Our findings provided empirical support for both CAS and OCS to be reliable and valid measures for screening for coronavirus anxiety and obsession. The resultant tools not only allow for a quick screening for at-risk cases for excessive levels of anxiety and obsessions due to the COVID-19 pandemic, but also in identifying the degree and saliency of symptoms of such anxiety and obsessive thoughts among community-dwelling Chinese adults. As the impact of the COVID-19 pandemic tends to be prolonged and fluctuating in its intensity, we recommend the regular use of CAS and OCS in the community's mass screening for mental symptoms associated with coronavirus for early identification and intervention among Chinese in need.

CRediT authorship contribution statement

Juliet Honglei Chen: Conceptualization, Methodology, Formal analysis, Writing – original draft, Writing – review & editing. Kwok Kit Tong: Conceptualization, Methodology, Writing – review & editing. Xiaoyu Su: Writing – original draft. Eilo Wing-yat Yu: Conceptualization, Funding acquisition, Resources. Anise M.S. Wu: Conceptualization, Methodology, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare no conflicts of interest.

Acknowledgments

Funding

The research was supported by the Science and Technology Development Fund (FDCT), Macao (grant number: 0022/2020/A).

Acknowledgment

None.

Appendix A

The Chinese version of Coronavirus Anxiety Scale

最近這兩個星期, 請問你有多經常有以下這些經歷呢 請用0-4評分, 0 = “完全沒有”, 1 = “很少, 少於1-2日”, 2 = “有幾日”, 3 = “多於一個星期”, 4 = “幾乎每日”。

  • a

    當我讀到或聽到有關新型冠狀病毒的新聞時, 我感到頭暈,頭昏眼花或昏厥

  • b

    當我想到新型冠狀病毒時, 我難以入睡或維持睡眠狀態

  • c

    當我想到或接觸到有關新型冠狀病毒的資訊時, 我身體感到麻痹或冰冷

  • d

    當我想到或接觸到有關新型冠狀病毒的資訊時, 我對吃東西失去了興致

  • e

    當我想到或接觸到有關新型冠狀病毒的資訊時, 我就感到噁心或胃部不適

The Chinese version of Obsession with COVID-19 Scale

最近這兩個星期, 請問你有多經常有以下這些經歷呢? 請用0-4評分, 0 = “完全沒有”, 1 = “很少, 少於1-2日”, 2 = “有幾日”, 3 = “多於一個星期”, 4 = “幾乎每日。

  • a

    我有不安地想過我可能已經感染了新型冠狀病毒

  • b

    我有不安地想過我見過的人裡面可能有人已經感染了新型冠狀病毒

  • c

    我沒有辦法不去想關於新型冠狀病毒的事情

  • d

    我曾經夢到過關於新型冠狀病毒的事情

References

  1. Adiukwu F., Orsolini L., Gashi Bytyçi D., El Hayek S., Gonzalez-Diaz J.M., Larnaout A., Grandinetti P., Nofal M., Pereira-Sanchez V., Pinto da Costa M., Ransing R., Schuh Teixeira A.L., Shalbafan M., Soler-Vidal J., Syarif Z., Kudva Kundadak G., Ramalho R. COVID-19 mental health care toolkit: an international collaborative effort by early career psychiatrists section. Gen. Psychiatry. 2020;33(5) doi: 10.1136/gpsych-2020-100270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ahmed O., Faisal R.A., Sharker T., Lee S.A., Jobe M.C. Adaptation of the Bangla version of the COVID-19 anxiety scale. Int. J. Ment. Health Addict. 2020 doi: 10.1007/s11469-020-00357-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. American Psychiatric Association [APA] 5th ed. American Psychiatric Association; 2013. Diagnostic and Statistical Manual of Mental Disorders. [Google Scholar]
  4. Andrade E.F., Pereira L.J., Orlando D.R., Peixoto M.F.D., Lee S.A., Castelo P.M. Validation of the Brazilian Portuguese version of the obsession with COVID-19 Scale (BP-OCS) using a a large University Sample in Brazil. Death Stud. 2021 doi: 10.1080/07481187.2021.1879319. [DOI] [PubMed] [Google Scholar]
  5. Ashraf F., Lee S.A., Elizabeth Crunk A. Factorial validity of the Urdu version of the obsession with COVID-19 Scale: preliminary investigation using a university sample in Pakistan. Death Stud. 2020:1–6. doi: 10.1080/07481187.2020.1779436. [DOI] [PubMed] [Google Scholar]
  6. Asmundson G.J.G., Taylor S. Coronaphobia: fear and the 2019-nCoV outbreak. J. Anxiety Disord. 2020;70 doi: 10.1016/j.janxdis.2020.102196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Blakey S.M., Reuman L., Jacoby R.J., Abramowitz J.S. Tracing “Fearbola”: psychological predictors of anxious responding to the threat of Ebola. Cogn. Ther. Res. 2015;39(6):816–825. doi: 10.1007/s10608-015-9701-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Boateng G.O., Neilands T.B., Frongillo E.A., Melgar-Quiñonez H.R., Young S.L. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health. 2018;6:149. doi: 10.3389/fpubh.2018.00149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Brand J., McKay D., Wheaton M.G., Abramowitz J.S. The relationship between obsessive compulsive beliefs and symptoms, anxiety and disgust sensitivity, and Swine Flu fears. J. Obs. Compuls. Relat. Disord. 2013;2(2):200–206. doi: 10.1016/j.jocrd.2013.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Browne M.W., Cudeck R. Alternative ways of assessing model fit. Sociol. Methods Res. 1992;21(2):230–258. doi: 10.1177/0049124192021002005. [DOI] [Google Scholar]
  11. Chen F., Zheng D., Liu J., Gong Y., Guan Z., Lou D. Depression and anxiety among adolescents during COVID-19: a cross-sectional study. Brain Behav. Immun. 2020;88:36–38. doi: 10.1016/j.bbi.2020.05.061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Chen J.H., Yu E.W., Su X., Tong K.K., Wu A.M.S. Mental distress during the COVID-19 pandemic: its association with public perceptions toward government's pandemic responses and lifestyle changes. Curr. Psychol. 2021 doi: 10.1007/s12144-021-01547-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Choi E., Lee J., Lee S.A. Validation of the Korean version of the obsession with COVID-19 scale and the Coronavirus anxiety scale. Death Stud. 2020:1–7. doi: 10.1080/07481187.2020.1833383. [DOI] [PubMed] [Google Scholar]
  14. Choi E.P.H., Hui B.P.H., Wan E.Y.F. Depression and anxiety in Hong Kong during COVID-19. Int. J. Environ. Res. Public Health. 2020;17(10):3740. doi: 10.3390/ijerph17103740. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Courtney D., Watson P., Battaglia M., Mulsant B.H., Szatmari P. COVID-19 impacts on child and youth anxiety and depression: challenges and opportunities. Int J Community Med Public Health. 2020;7(11):4570–4577. doi: 10.18203/2394-6040.ijcmph20204763. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Diener E., Wirtz D., Tov W., Kim-Prieto C., Choi D., Oishi S., Biswas-Diener R. New well-being measures: short scales to assess flourishing and positive and negative feelings. Soc. Indic. Res. 2010;97(2):143–156. doi: 10.1007/s11205-009-9493-y. [DOI] [Google Scholar]
  17. Evren C., Evren B., Dalbudak E., Topcu M., Kutlu N. Measuring anxiety related to COVID-19: a Turkish validation study of the Coronavirus anxiety scale. Death Studies. 2020:1–7. doi: 10.1080/07481187.2020.1774969. [DOI] [PubMed] [Google Scholar]
  18. Gaziano C. In: Encyclopedia of Survey Research Methods. Lavrakas P.J., editor. SAGE Publications; 2008. Last-birthday selection; pp. 417–418. [Google Scholar]
  19. González-Sanguino C., Ausín B., Castellanos M.Á., Saiz J., López-Gómez A., Ugidos C., Muñoz M. Mental health consequences during the initial stage of the 2020 Coronavirus pandemic (COVID-19) in Spain. Brain Behav. Immun. 2020;87:172–176. doi: 10.1016/j.bbi.2020.05.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hair J.F., Black W.C., Babin B.J., Anderson R.E. 7th ed. Pearson; 2014. Multivariate Data Analysis. [Google Scholar]
  21. Hu L., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 1999;6(1):1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
  22. Huang Y., Zhao N. Generalized anxiety disorder, depressive symptoms and sleep quality during COVID-19 outbreak in China: a web-based cross-sectional survey. Psychiatry Res. 2020;288 doi: 10.1016/j.psychres.2020.112954. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. IBM Corp . IBM Corp.; 2017. IBM SPSS Statistics. Version 25.0. [Google Scholar]
  24. Khosravani V., Abramowitz J.S., Samimi Ardestani S.M., Sharifi Bastan F., Kamali Z. The Persian version of the dimensional obsessive-compulsive scale (P-DOCS): a psychometric evaluation. J. Obs. Compuls. Relat. Disord. 2020;25 doi: 10.1016/j.jocrd.2020.100522. [DOI] [Google Scholar]
  25. Knekta E., Runyon C., Eddy S. One size doesn't fit all: using factor analysis to gather validity evidence when using surveys in your research. CBE Life Sci. Educ. 2019;18(1):rm1. doi: 10.1187/cbe.18-04-0064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lee S.A. Measuring coronaphobia: the psychological basis of the Coronavirus anxiety scale. Dusunen Adam J. Psychiatry Neurol. Sci. 2020 doi: 10.14744/DAJPNS.2020.00069. [DOI] [Google Scholar]
  27. Lee S.A. How much “Thinking” about COVID-19 is clinically dysfunctional? Brain Behav. Immun. 2020;87:97–98. doi: 10.1016/j.bbi.2020.04.067. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lee S.A. Coronavirus anxiety scale: a brief mental health screener for COVID-19 related anxiety. Death Stud. 2020;44(7):393–401. doi: 10.1080/07481187.2020.1748481. [DOI] [PubMed] [Google Scholar]
  29. Lee S.A., Jobe M.C., Mathis A.A. Mental health characteristics associated with dysfunctional coronavirus anxiety. Psychol. Med. 2021;51(8):1403–1404. doi: 10.1017/S003329172000121X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Lin Y., Hu Z., Alias H., Wong L.P. Knowledge, attitudes, impact, and anxiety regarding COVID-19 infection among the public in China. Front. Public Health. 2020;8:236. doi: 10.3389/fpubh.2020.00236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Magano J., Vidal D.G., Sousa H.F.P.E., Dinis M.A.P., Leite Â. Validation and psychometric properties of the Portuguese version of the Coronavirus anxiety scale (CAS) and Fear of COVID-19 scale (FCV-19S) and associations with travel, tourism and hospitality. Int. J. Environ. Res. Public Health. 2021;18(2):427. doi: 10.3390/ijerph18020427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Muthén L.K., Muthén B.O. 7th ed. Muthén & Muthén; 2012. Mplus User's Guide. [Google Scholar]
  33. Padovan-Neto F.E., Lee S.A., Guimarães R.P., Godoy L.D., Costa H.B., Zerbini F.L.S., Fukusima S.S. Brazilian adaptation of the Coronavirus anxiety scale: a psychometric investigation of a measure of Coronaphobia. OMEGA J. Death Dying. 2021 doi: 10.1177/0030222821991325. 003022282199132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Rubin G.J., Amlot R., Page L., Wessely S. Public perceptions, anxiety, and behaviour change in relation to the swine flu outbreak: cross sectional telephone survey. BMJ. 2009;339:b2651. doi: 10.1136/bmj.b2651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Sarason I.G., Spielberger C.D., Sarason I.G. In: Spielberger C.D., Sarason I.G., editors. Hemisphere; 1978. The test anxiety scale: concept and research; pp. 193–216. (Stress and Anxiety: V). Eds. Vol.pp. [Google Scholar]
  36. Snell W.J., Finney P. In: Handbook of Sexuality-Related Measures. Davis C., Yarber W., Bauserman R., editors. SAGE Publications; 1998. Multidimensional AIDS anxiety questionnaire; pp. 351–353. Eds. pp. [Google Scholar]
  37. Srivastava A., Bala R., Srivastava A.K., Mishra A., Shamim R., Sinha P. Anxiety, obsession and fear from coronavirus in Indian population: a web-based study using COVID-19 specific scales. Int. J. Community Med. Public Health. 2020;7(11):4570. doi: 10.18203/2394-6040.ijcmph20204763. [DOI] [Google Scholar]
  38. Taylor S. Cambridge Scholars Publishing; 2019. The Psychology of Pandemics: Preparing for the Next Global Outbreak of Infectious Disease. [Google Scholar]
  39. Tong K.K., Wang Y.Y. Validation of the flourishing scale and scale of positive and negative experience in a Chinese community sample. PLOS ONE. 2017;12(8) doi: 10.1371/journal.pone.0181616. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Tyrer P. COVID-19 health anxiety. World Psychiatry. 2020;19(3):307–308. doi: 10.1002/wps.20798. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Vandenberg R.J., Lance C.E. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods. 2000;3(1):4–70. doi: 10.1177/109442810031002. [DOI] [Google Scholar]
  42. Wang C., Pan R., Wan X., Tan Y., Xu L., Ho C.S., Ho R.C. Immediate psychological responses and associated factors during the initial stage of the 2019 Coronavirus disease (COVID-19) epidemic among the general population in China. Int. J. Environ. Res. Public Health. 2020;17(5):1729. doi: 10.3390/ijerph17051729. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Wang C., Pan R., Wan X., Tan Y., Xu L., McIntyre R.S., Choo F.N., Tran B., Ho R., Sharma V.K., Ho C. A longitudinal study on the mental health of general population during the COVID-19 epidemic in China. Brain Behav. Immun. 2020;87:40–48. doi: 10.1016/j.bbi.2020.04.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Wheaton M.G., Abramowitz J.S., Berman N.C., Fabricant L.E., Olatunji B.O. Psychological predictors of anxiety in response to the H1N1 (Swine Flu) Pandemic. Cogn. Ther. Res. 2012;36(3):210–218. doi: 10.1007/s10608-011-9353-3. [DOI] [Google Scholar]
  45. World Health Organization [WHO]. (2021). WHO Coronavirus (COVID-19) dashboard (as of 22 March 2021). https://covid19.who.int/table (accessed on 22 March 2021).
  46. Wong L.P., Hung C.C., Alias H., Lee T.S.H. Anxiety symptoms and preventive measures during the COVID-19 outbreak in Taiwan. BMC Psychiatry. 2020;20(1):376. doi: 10.1186/s12888-020-02786-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Wong T.W., Gao Y., Tam W.W.S. Anxiety among university students during the SARS epidemic in Hong Kong. Stress Health. 2007;23(1):31–35. doi: 10.1002/smi.1116. [DOI] [Google Scholar]
  48. Xie X.F., Stone E., Zheng R., Zhang R.G. The ‘Typhoon Eye Effect’: determinants of distress during the SARS epidemic. J. Risk Res. 2011;14(9):1091–1107. doi: 10.1080/13669877.2011.571790. [DOI] [Google Scholar]
  49. Yurttas B., Poyraz B.C., Sut N., Ozdede A., Oztas M., Uğurlu S., Tabak F., Hamuryudan V., Seyahi E. Willingness to get the COVID-19 vaccine among patients with rheumatic diseases, healthcare workers and general population in Turkey: a web-based survey. Rheumatol. Int. 2021;41(6):1105–1114. doi: 10.1007/s00296-021-04841-3. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Affective Disorders are provided here courtesy of Elsevier

RESOURCES