Psychological distress in the shadow of the COVID-19 pandemic: Preliminary development of an assessment scale

Highlights

• •The CORPD had good internal consistency reliability and split-half reliability, and also had good structure validity, content validity and criterion correlation validity.
• •The CORPD can be used in psychological distress evaluation for people in the COVID-19 affected countries and regions.
• •The general public may have a common COVID-19 related psychological distress, including: suspicion, anxiety and fear.
• •Our results also provide further insight for the future research for COVID-19 or other infectious diseases related mental health problems.

Abstract

COVID-19 is now spreading worldwide, and poses some public mental health problems which requires close attention. This study aims to develop a scale of COVID-19 related psychological distress in healthy public (CORPD) to assess the severity of psychological distress in uninfected healthy populations. We compiled a 14-item scale which contains two dimensions— Anxiety & fear and Suspicion —using the classical measurement theory. 652 Chinese citizens consented and completed a survey through an online questionnaire APP. The reliability test showed that the scale had good internal consistency reliability and Split-Half reliability, and the validity test showed that it had good structure validity, content validity and criterion correlation validity. This scale can be used to assess the psychological distress of people in China and in other COVID-19-hit regions and countries. It also provides a reference for future studies on COVID-19 or other respiratory infectious diseases related public mental health.

1. Introduction

Since the detection and report of the coronavirus disease 2019 (COVID-19) in December 2019 (Huang et al., 2020), more than 4 million cases have been confirmed in more than 200 countries and regions. Humans have experienced three coronavirus-involved pandemics in this century: SARS in 2003, MERS in 2012, and COVID-19 in 2019. Each pandemic not only poses serious threats to the lives of those infected in the affected areas, but also brings about different degrees of negative and profound mental and psychological problems to both the infected and uninfected healthy people (Lee et al., 2007; Yoon et al., 2016; Holmes et al., 2020). Psychological distress is an unpleasant emotional experience caused by a variety of factors, which can be manifested as tension, fear, anxiety, and psychological instability. Certain distress even leads to serious psychological problems such as depression. Infectious diseases can be an important cause of psychological distress (Brody et al., 2016; Tola et al., 2015). Previous studies have suggested a Dose-response association between psychological distress and the risk of death from infectious diseases, and a similar relationship was also found between viral infectious diseases and psychological distress (Hamer et al., 2019).

Since current outbreak, some studies have demonstrated an increase of mental health problems and psychological distress in uninfected healthy people caused by the increasing risk of COVID-19 infection, strict quarantine measures, mandatory home quarantine and other events (Zhu et al., 2020; Losada-Baltar et al., 2020). It is particularly important to find out more accurately whether uninfected healthy people are suffering from COVID-19 related psychological distress. Therefore, a reliable and effective assessment tool is in need. The existing scales, such as self-rating anxiety scale (SAS), self-rating depression scale (SDS), and general health questionnaire (GHQ-9), generalized anxiety disorder scale (GAD-7), are mostly used for the initial screening or clinical diagnosis of anxiety, depression and other mental symptoms. In our work, we found that the above scales are not particularly specified for the assessment of the psychological distress in the uninfected healthy public. Consequently, they may not be able to truthfully and deeply reveal the specific psychological distress level of the uninfected healthy public caused by this COVID-19 pandemic. We did literature review but have not found any targeted and specific reports on a COVID-19 related psychological distress scale. Therefore, we have aimed to preliminarily develop and compile a scale of COVID-19 related psychological distress in healthy public (CORPD) adopting the classical measurement theory. With its verified validity, this scale can be used in psychological distress evaluation for people in the COVID-19 affected countries and regions. Our results also provide further insight for the future research for COVID-19 or other infectious diseases related mental health problems.

2. Methods

2.1. Scale compilation

We used the classical measurement theory to compile the CORPD. 11 uninfected healthy people in Yunnan, China were invited in semi-structured interviews from March 20, to April 2, 2020. The sample size was determined by information saturation method. The interviews were terminated once no further effective information could be obtained from more participants (Bao and Pan, 2015). An entry pool was formed by extracting information from the semi-structured interview data using NVivo 11 software from April 3, to April 12, 2020. By Delphi method, 5 experts from Kunming Medical University, specializing in mental health, social psychology, public health management, nursing, and linguistics respectively, were invited to demonstrate and modify the entry pool from April 13 to April 14, 2020. Each expert was required to rate each item for good content validity (on a scale of 0 to 10, a higher score indicated that the item was more in line with the content range of psychological distress). Another 20 participants were invited to conduct a preliminary survey of the scale using cognitive interview technique from April 15, to April 17, 2020. Then, the items of CORPD were constructed and completed.

Likert five-point scoring method was adopted in the scale. There were five options: strongly disagree, disagree, not sure, agree and strongly agree, scored as 1, 2, 3, 4, 5 points respectively. Higher severity of psychological distress was reflected by higher score. The participants chose one as the answer to a specific statement according to their own perceptions and attitudes.

2.2. Data collection

The formal survey was conducted from April 18, to April 21, 2020. The questionnaire included three dimensions: demographic and sociological characteristics, CORPD, and SCL-90 anxiety assessment. This study was approved and reviewed by the Ethics Committee of the Sixth Affiliated Hospital of Kunming Medical University. As it was during the epidemic period, we complied to the Chinese government's epidemic prevention and control policies, and reduced the contact between people as much as possible. When collecting samples, we adopted convenient sampling and combined it with snowball sampling. We sent the survey to the general public through an online questionnaire link on WeChat APP, the most commonly used instant messaging software among Chinese people (convenient sampling). After the participants completed the questionnaire, they sent the link to more people to participate (snowball sampling). The questionnaire was completed voluntarily, anonymously and confidentially. The survey did not involve any commercial purpose. Before taking the survey, the participants were required to read an introduction of the research team, research objectives, research methods, ethical statements, and notes for completing the questionnaire. After the introduction, the participants were required to choose and declare whether they were willing to participate in the survey. If they chose "YES", the system would automatically load the survey page. If they chose "NO", the system would exit automatically. The participants were free to leave the survey any time they preferred. The participants' name, contact number, detailed address and other personal information were not obtainable for the questionnaire and system managers.

2.3. Statistical analysis

After the data collection, the reliability and validity of the CORPD were tested. A structural equation model was constructed and the structural validity of the CORPD was tested by confirmatory factor analysis. GFI, AGFI, RMR, RMSEA, CFI, NFI, TLI, PNFI and PGFI were used as the test indexes of the fitting degree. Pearson correlation analysis and experts’ consultation in I-CVI value were used to evaluate content validity. The SCL-90 anxiety dimension was used as a reference to evaluate the validity of criterion. Cronbach α was used to evaluate the internal consistency reliability. Guttman Split-Half Coefficient were used to evaluate the Split-Half reliability. Independent Sample T test or One-way ANOVA was used to evaluate the differences in scores of populations with different demographic and sociological characteristics. SPSS Statistics 19.0 and SPSS AMOS 22.0 were used for statistical analysis. The test level was α=0.05, and all P values represented bilateral probability.

3. Results

3.1. Samples

Between April 18 and 21, 656 healthy public clicked the link to the survey, and 652 people chose to participate and complete the questionnaire, with an effective rate of 99.39%. Among them, 439 (67.3%) were women, 113 (17.3%) were over 45 years old, 536 (82.2%) were Han nationality, 540 (82.8%) had college degree or above, 388 (59.5%) were married, 368 (56.4%) had a monthly income of more than 4000 yuan,539 people (82.7%) were from Yunnan Province, 387 people (59.4%) were urban residents, 375 (57.5%) had children to support and 502(77.0%) had dependent seniors in the family.

3.2. Validity test

We believed anxiety & fear and suspicion were the two major aspects of psychological distress in uninfected healthy public. Therefore, we divided the 14 items we extracted from the interview data using NVivo 11 software into the above-mentioned two dimensions, Anxiety & fear and Suspicion, with seven items in each (see Table 1 ).

Table 1.

The item pool of scale of COVID-19 related psychological distress in healthy public (CORPD).

Item	Strongly disagree	Disagree	Not sure	Agree	Strongly agree	Factor
						Anxiety & fear	Suspicion
1. If I were infected with COVID-19, I might not be able to recovery from it.	1	2	3	4	5	√
2. When talking to a stranger, I would suspect that s/he might be infected with COVID-19.	1	2	3	4	5		√
3. I'm afraid to travel to places hard-hit by COVID-19.	1	2	3	4	5	√
4. When I see an increase in the number of COVID-19 patients on the news, I feel anxious.	1	2	3	4	5	√
5. When I see someone sneeze, I suspect s/he might be infected with COVID-19.	1	2	3	4	5		√
6. I think frequent hospital visits would make it easier to be infected with COVID-19.	1	2	3	4	5	√
7. I fear to see the doctors and nurses who had worked in COVID-19 isolation wards.	1	2	3	4	5	√
8. I think frequent use of air, train, bus and other public transport would make it easier to be infected with COVID-19.	1	2	3	4	5	√
9. When I notice someone running a fever, I suspect s/he might be infected with COVID-19.	1	2	3	4	5		√
10. When I see someone vomiting, I suspect s/he might be infected with COVID-19.	1	2	3	4	5		√
11. I fear to live nearby a COVID-19 isolation hospital.	1	2	3	4	5	√
12. When I see someone coughing, I suspect s/he might be infected with COVID-19.	1	2	3	4	5		√
13. When I see someone without a mask, I suspect s/he might be infected with COVID-19.	1	2	3	4	5		√
14. I suspect there were novel coronavirus in the air when there were people around.	1	2	3	4	5		√

The KMO and Bartlett test of sphericity were used to determine the appropriateness of factor analysis. The result (KMO=0.912, Bartlett significance P < 0.001) indicated perfect appropriateness to further conduct the confirmatory factor analysis (Cheng et al., 2020). In the confirmatory factor analysis, the structural equation model showed that the two-factor model was consistent with the designed dimension. In addition, we found that by increasing the correlation between different latent variables in the same factor, the model's fitting degree was improved. This suggested that there might be potential interactions between some CORPD items. Therefore, the fitting degree of the corrected two-factor model was better than that of the uncorrected two-factor model (see Table 2 and Fig. 1 ). Anxiety & fear dimension mainly described the fear and anxiety resulted from fearing of being infected by COVID-19 among the general public, their deliberate avoidance of going to the affected areas, their avoidance of taking public transportation or going to the hospital for medical treatment, and their rejection of the medical staff who had worked in isolation wards. Suspicion dimension mainly described the uninfected healthy public's suspicion that others were infected with COVID-19 when they noticed some typical respiratory tract infection symptoms (such as fever, cough), non-respiratory symptoms (such as vomiting), or even general physiological reactions (such as sneezing) in other people. It also described the public's suspicion that others might be infected with COVID-19 when they saw others not wearing a mask or when they talked to a stranger, or even the suspicion that the air in the street may contain COVID-19. Our findings showed that the scale had good structural validity.

Table 2.

Fitting degree of structural equation model of the CORPD.

	GFI	AGFI	RMR	RMSEA	CFI	NFI	TLI	PNFI	PGFI
Uncorrected two factor model	0.904	0.868	0.071	0.088	0.887	0.868	0.865	0.725	0.655
Corrected two factor model	0.936	0.904	0.060	0.072	0.930	0.912	0.909	0.701	0.624
Threshold	>0.90	>0.90	<0.05	<0.06	>0.90	>0.90	>0.90	>0.50	>0.50

Fig. 1.

Structural equation model of CORPD.

In Delphi expert consultation, the I-CVI values of all 14 items in the CORPD were greater than 0.80 (range from 0.82 to 0.98). The ratings of the five experts showed good inter-rater reliability (the Kendall concordance coefficient was 0.787, P<0.001). Pearson's correlation analysis showed that there was a high correlation between the score of the CORPD and the dimension of Anxiety & fear (γ =0.909, P < 0.05), and a high correlation between the score of the CORPD and the dimension of Suspicion (γ =0.933, P < 0.05). Good correlations were found between the score of each item and the total score of the CORPD (γ range= 0.387–0.757, P < 0.05). These suggested that the CORPD have a good content validity.

In the criterion validity test, Pearson correlation analysis showed that there was a good positive correlation between the total score of the CORPD and the total score of SCL-90 anxiety factors (γ =0.314, P < 0.05). According to the score standard of SCL-90, when the score of anxiety factor was < 2 points, it was defined as negative in the primary screening, and a score which was ≥2 points was defined as positive in the primary screening. The 652 participants were divided into a negative (N = 570) and a positive group (N = 82) according to the primary screening. Independent Sample T test showed that there were statistically significant differences (40.11±10.18 vs. 46.71±9.45, P < 0.001) in the scores of CORPD between the two groups. The above results suggested that the CORPD have a good criterion validity.

3.3. Reliability test

The internal consistency reliability test showed that Cronbach's α of the CORPD was 0.885, Cronbach's α on the Anxiety & fear dimension was 0.742, and Cronbach's α on the Suspicion dimension was 0.869. The Split-Half reliability test showed that the Guttman Split-Half coefficient of the CORPD was 0.907, the Guttman Split-Half coefficient of the Anxiety & fear dimension was 0.705, and 0.858 for the Suspicion dimension. It was safe to draw the conclusion that the CORPD and its two dimensions were all reliable and had good internal consistency reliability and Split-Half reliability.

3.4. Subgroup analysis

Through Independent Sample T test or one-way ANOVA, we also found that there were statistically significant differences in subgroups of different gender, age, education level, marital status, monthly income level, and supporting children or not (P < 0.05) (see Table 3 ). There were no statistically significant differences among the subgroups in nationality, residential area, and supporting seniors or not (P > 0.05).

Table 3.

Scores of the CORPD in populations with different sociodemographic characteristics (x ± s).

	Characteristics	N	CORPD Scores	P
Gender	Male	213	39.65±10.80	0.026
	Female	439	41.57±10.03
Age	<45	539	40.51±10.36	0.020
	≥45	113	42.99±9.91
Educational Level	≤high school	112	44.00±10.00	0.001
	≥college	540	40.31±10.28
Marital status	Married	388	41.79±10.74	0.010
	Not in a marriagea	264	39.70±9.57
Monthly income	≤4000	284	42.51±10.04	0.001
	>4000	368	39.74±10.39
Supporting children or not	Yes	375	41.82±10.19	0.011
	No	277	39.75±10.40

^aNot in a marriage includes: single, divorced, widowed.

4. Discussion

We found that the general public may have a common COVID-19 related psychological distress. To our knowledge, this is the first scale specially developed to evaluate the negative impact of COVID-19 on the psychological distress of the uninfected healthy public. After statistical tests, we believe that this scale has good reliability and validity and worth of promotion in future research practice.

The COVID-19 pandemic is undoubtedly a stressful event for the general public, which can cause mental health problems among the public, and existing studies have shown that COVID-19 crisis can lead to dysfunctional anxiety among the general public. Sherman developed a Coronavirus Anxiety Scale (CAS) with good reliability and validity. By evaluating the frequency of 5 items—dizziness, sleep disturbance, tonic immobility, appetite loss, and nausea or abdominal distress, CAS was used to screen and assess the severity of COVID-19-related dysfunctional anxiety (Lee, 2020). The scale was proved with good reliability and validity in a reassessment of subway workers in the United States and found that COVID-19 related anxiety is positively correlated with coronavirus fear, functional impairment, hopelessness, suicidal ideation and many other factors (Lee et al., 2020). Other studies showed that COVID-19 has caused serious fear and concern among the general public. Ahorsu et al. compiled and developed "the Fear of COVID-19 Scale (FCV-19S)" in Iran specifically to measure the public's fear of COVID-19 (Ahorsu et al., 2020). This FCV-19S has proved with good reliability and validity in the evaluation of the Turkish public and further found that COVID-19 related fear has a positive predictive effect on anxiety, depression and depersonalization, and an indirectly correlation to life satisfaction (Satici et al., 2020). It is safe to draw the conclusion that there is a potential link between anxiety and fear caused by COVID-19, which are the exact important manifestations of COVID-19-related psychological distress. However, when we conducted the semi-structured interviews, we found that COVID-19-related psychological distress includes not only anxiety and fear, but also suspicion.

As a result, we tried to divide this mental sub-health state into three aspects: anxiety, fear and suspicion through inductive analysis. However, among them, the relation between anxiety and fear is closer, and the two are often accompanied and inseparable. Therefore, we integrated them into the same dimension for evaluation. Anxiety and fear of the risk of virus infection in the general public may seriously impair their functions in daily life and social interaction, generate social avoidance, and induce suspicion of others. We believe that suspicion may be a psychological defense mechanism subconsciously developed by the general public in the face of infectious diseases. This study found that the general public may not only suspect COVID-19 in people with fever and cough, but also in people who vomit (although vomiting is not a common and typical symptom of COVID-19), and may suspect that people who do not wear masks may be infected with COVID-19, and even bring about stigmatization and discrimination against COVID-19. In some worse cases, the uninfected healthy public may even suspect that the air in public places such as streets and markets contain COVID-19, which is obviously an overstating of the infectivity of the virus. Serious suspicion often leads to more serious anxiety and fear symptoms, and leads to problems such as anxiety disorder and depressive disorder in certain individuals.

In the early phase of the outbreak, some studies reported the psychological distress caused by COVID-19 in different populations may be related to gender, social support level, COVID-19 infection experience, isolation period and media exposure (Li et al., 2020). At present, the domestic epidemic in China has been effectively controlled, and the whole country is in an orderly resumption of work. However, with the rapid increase of the global infection/death toll, the global situation of this pandemic prevention and control is not optimistic (Kandel et al., 2020). Many regions in China have also seen imported cases and related infection cases, that's why the Chinese central government is taking strong measures to prevent the imported cases and prevent the rebound of the domestic cases. Although the domestic epidemic situation was much better than before, we still found COVID-19 related anxiety, fear and suspicion in uninfected healthy people, suggesting that the mental health problems caused by COVID-19 may last for a long time. Therefore, long-term psychological interventions should be developed and implemented. Subgroup analysis showed that women, middle-aged and elderly people, people with lower educational background, married people, those with lower income, and those with dependent children may face more serious COVID-19 related mental health problems, suggesting that mental health problems of these people should arouse more consideration and attention during the prevalence of COVID-19 and other respiratory infectious diseases. Targeted, specific and timely psychological counselling and psychological intervention should be taken in these subgroups (Zhang et al., 2020).

In response to government regulations on epidemic prevention and control, this study did not adopt strict random sampling method, but adopted the method of convenient sampling combined with snowball sampling. This may cause partial bias in the samples. We carried out necessary publicity through newspapers, medias and posters to recruit as many participants as we could. In this study, the participants were general public from 22 provinces, municipalities or autonomous regions in China, while mostly they were from Yunnan province, so there may be some regional bias in the research results. Due to the relatively small sample size in this study, we did not carry out a multi-factor analysis on the influencing factors of COVID-19 related psychological distress. At present, our team is recruiting more uninfected healthy public to participate in other surveys of COVID-19 related mental health, so that we can reduce the bias, make up for the limits, and further verify the reliability and effectiveness of the scale especially in different regions and different populations. In this way, we can obtain a more comprehensive understanding of the negative impacts of mental health to the public during and after the COVID-19 pandemic. These ongoing works shall provide more sufficient evidence for the psychological interventions in the general public. If the scale is of help to a researcher, s/he has our permission to translate it into other languages, to revise it as necessary, to re-examine the reliability and validity of the scale, and to apply it to COVID-19 or other infection-related studies, on the premise that this literature is cited and commercial interests are not involved.

Author statement

Lin-sen Feng: Conceptualization, Investigation, Methodology, Statistic analysis, Validation, Writing - original draft, Writing - review & editing.

Zheng-jiao Dong: Conceptualization, Investigation, Methodology, Statistic analysis, Validation, Writing - original draft, Writing - review & editing.

Ruo-yu Yan: Investigation, Validation, Writing - review & editing.

Xiao-qian Wu: Investigation, Validation, Writing - review & editing.

Li Zhang: Investigation, Validation, Writing - review & editing.

Jun Ma: Investigation, Validation, Writing - review & editing.

Yong Zeng: Conceptualization, Investigation, Methodology, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no competing interests.