Validation of the Pandemic Emotional Impact Scale

Abstract

Objective

The COVID-19 pandemic represents the most universal shared stressor for the general United States (U.S) population in many decades. Due to the unprecedented circumstances of COVID-19, no existing questionnaires can comprehensively measure the multi-faceted psychological effects attributable to this health crisis. This study aimed to validate a measure for that purpose.

Methods

A 16-item questionnaire, the Pandemic Emotional Impact Scale (PEIS), was designed and subjected to initial validation in an internet survey completed by a nationally representative sample of 1500 adults living in the U.S. This survey was completed between May 18 and May 30, 2020, during the height of the pandemic’s impact on society.

Results

The PEIS demonstrated excellent internal consistency (Cronbach’s α ​= ​0.94) and Guttman split-half reliability (0.95). Exploratory factor analysis suggested two sub-scales -- emotional impact and pragmatic worries -- but these were highly correlated with the overall scale score suggesting that the total score can be used in most cases. The PEIS demonstrated good concurrent validity via robust positive correlations with anxiety, depression and stress, and negative correlations with quality of life and happiness. Criterion validity was supported by the finding that individuals who reported employment loss or loss of income due to the pandemic, had experienced COVID-19 infection in their household, or knew somebody personally who died from the pandemic, had elevated scores on the PEIS.

Conclusions

The PEIS questionnaire is a reliable and valid instrument that addresses a significant unmet need for a research instrument that can comprehensively measure pandemic-related effects on the emotional wellbeing of individuals in the U.S population.

Highlights

• •This study demonstrates excellent validity and reliability of the Pandemic Emotional Impact Scale.
• •Data were from a nationally representative sample of 1500 adults living in the US.
• •The PEIS can be used to assess the emotional effects of this and future pandemics.

1. Introduction

The COVID-19 pandemic of 2020 has caused an unprecedented upheaval in the lives of the general population of the United States (U.S) and around the world. The multi-faceted changes in society and everyday life that have directly and indirectly resulted from the pandemic are, without doubt, the most universally shared major stressor for adults in the U.S in many decades. Worldwide, it has created an omnipresent threat to people’s physical, financial, and emotional well-being (Brooks et al., 2020). The effects of these unique circumstances on the mental wellbeing of the U.S. population are presently poorly understood, and they need to be investigated and documented, as they may have long-term mental health consequences as well as implications for future national crises. Data from the U.S. Centers for Disease Control and Prevention show that rates of symptoms of both depressive and anxiety disorders have more than tripled in the U.S. as of June of 2020 compared to the previous year, and this is likely to be largely related to the pandemic’s effect on the population (Czeisler et al., 2020). Published research from international studies has similarly suggested that anxiety, depression, sleep disturbance, stress, and even vicarious traumatization are common psychological responses to COVID-19, causing researchers to call for greater inclusion of mental health resources in the public health response to the pandemic (Rajkumar, 2020; Bao et al., 2019; Li et al., 2020)–(Rajkumar, 2020; Bao et al., 2019; Li et al., 2020).

Due to the highly unique and largely unprecedented nature of this pandemic, there are, of course, no previously validated measures to evaluate the emotional impact of the pandemic. As mentioned above, several recent studies have used validated measures of trauma, anxiety, and depression (Li et al., 2020; Wang et al., 2020; Xiao et al., 2020; Choi et al., 2020) to identify the mental health impact of COVID-19. However, these scales fail to capture the nuances of a global pandemic. For example, a global pandemic introduces unprecedented fear/worry about one’s own safety as well as the health and safety of all; fears about local and global economic/political instability; and frustration/disappointment regarding the complete disruption of daily activities. Such fears cannot be captured on existing measures of anxiety, depression, and trauma. Unfortunately, it is possible that we may face future pandemics, with many of the same disruptions and threats to citizens as the current one. Having a validated scale to measure a pandemic’s emotional impact in the U.S would therefore be useful not only for assessing and treating the emotional effects of the population in the remaining phases of the COVID-19 pandemic, but also for mental health research and interventions in the event of a future pandemic.

The current article presents validity and reliability data of a recently developed scale designed to measure the emotional impact of the present pandemic in a broad manner. The study data were collected via a nationally representative survey of individuals in the United States during the height of the effects of the COVID-19 pandemic on the population; at a time when the great majority of Americans were sheltering at home, most non-essential business and services had shut down, school had been suspended in many states, and nearly all group events and social gatherings had been cancelled around the country. At the time of this survey, more than 20,000 people were being diagnosed with COVID-19 daily in the U.S and it was causing more than 1000 deaths each day nationwide (Johns Hopkins Coronavirus, 2020). The current scale adds to the existing literature by providing a validated measure of pandemic-specific distress. This scale can be used with other validated mental health scales to provide a more complete understanding of the mental health consequences of the current pandemic or future pandemics.

2. Materials and methods

The survey was conducted from the University of North Carolina at Chapel Hill (UNC-Chapel Hill), using Qualtrics XM survey software. Participants were recruited by Qualtrics, Inc. from panels of pre-registered survey-takers across the U.S. The data were collected between May 18, 2020 and May 30, 2020, using quota-based sampling designed to achieve a balanced representation of demographic sub-groups to make it nationally representative in regard to key U.S. population demographics. The nationwide sample of 1500 U.S. adults who completed the survey was identical to the U.S. population in regard to sex ratio, age group representation, representation of Black and Hispanic/Latinx participants, education level (% with college degree), and regional distribution.

The survey included a number of built-in quality assurance measures, including attention and speed checks and automated validation of completed questions. The study was reviewed before data collection started by the UNC-Chapel Hill Institutional Review Board, and was deemed IRB-exempt due to the anonymous survey method. The survey was an unfunded research project and was not sponsored or paid for by any organization or agency.

2.1. Experimental measure

The Pandemic Emotional Impact Scale (PEIS) was designed to measure the emotional impact of a worldwide pandemic. Due to the rapid evolution of the COVID-19 crisis and nationwide shutdown of many activities starting in March 2020, including on-site psychological research at our institutions, we did not have the opportunity to generate these items using a formal process including focus groups, qualitative interviews, and/or the Delphi technique, as it was imperative to use the new questionnaire measure while the impact of the pandemic was widespread. Therefore, the items of the questionnaire were generated based on expert input, qualitative review of a selection of news stories about personal emotional effects of the published in April 2020 to identify recurrent themes, as well as research pertaining to stress and trauma after acute respiratory illness (Hosey et al., 2019; Pfoh et al., 2016). For general reference on assessment of emotional impact of disease epidemics, we also reviewed empirical work on mental health effects of past disease epidemics with similarities to the COVID-19 pandemic, including Severe Acute Respiratory Syndrome (SARS) and H1N1 Influenza.

The resulting PEIS consists of 16 items that were selected to constitute a broad range of emotional effects that can be attributed directly to pandemic-related experiences. Respondents are asked to rate how each emotional domain is different in the past 4 weeks compared to prior to the beginning of the pandemic, rated on a 5-point scale (0 ​= ​“not at all”; 1 ​= ​“a little bit”; 2 ​= ​“moderately”; 3 ​= ​“A lot”; 4 ​= ​“Extremely”). The overall score is a sum of these items, ranging from 0 to 64. Higher scores indicate greater emotional impact of the pandemic. The final scale is provided in the Appendix.

2.2. Study measures

Demographics and other COVID-19 questions. Survey respondents were asked to provide their age, sex, race, ethnicity, and education level. Respondents were also asked whether they or anyone in their household had been diagnosed with COVID-19; whether they or anyone in their household had experienced job loss or reduction in income due to COVID-19; and whether they personally knew anyone who had died from COVID-19.

Patient Health Questionnaire – 4 (PHQ-4). The PHQ-4 is a brief, validated measure of anxiety and depression symptoms (Löwe et al., 2010; Kroenke et al., 2009). This scale consists of two subscales (anxiety and depression), consisting of two items each, and also produces an overall psychological distress sum score. A score of ≥3 on either subscale is considered the cutoff point for identifying possible symptoms of clinical anxiety or depression.

Quality of life (QOL). Subjective QOL was measured using a single item from PROMIS Global-10 scale (Hays et al., 2009; Cella et al., 2010), which assesses overall physical and mental health. The single item used to evaluate QOL asks “In general, how would you rate your quality of life?” with answers ranging from 1 (“poor”) to 5 (“excellent”). This single item correlates highly with the mental health summary score of the PROMIS Global-10 (Hays et al., 2009; Cella et al., 2019).

Happiness. Overall happiness was measured using a single item numeric rating scale asking “Overall, how happy has your life felt to you over the past month”. This was rated by respondents on a scale from 0 to 10, with higher scores indicating more happiness. Such single-item numeric happiness scale ratings are the most common way to measure happiness in the domain of happiness research (Kalmijn, 2013). The exact form of the question we used has not been validated in published research, but is similar to a single-item 11-point happiness rating found to be valid in prior work (Abdel-Khalek, 2006).

Stress. This survey included two questions about stress:

Overall life stress in the past month was rated on a 0–10 overall scale, with higher scores indicating more stress. This measure of life stress has been commonly used in previous studies (Lesage et al., 2012).

Stress related to the pandemic was asked with the following question “Compared to the month of January, how has your personal level of life stress been over the past month (that is, the last 4 weeks)?”. Response options to this questions were: “1”, Much less stress now than in January; “2”, A little less stress now than in January; “3” About the same amount of stress as in January; “4” A little more stress now than in January; “5” Much greater stress now than in January.

2.3. Statistical analysis

Data from the online survey were exported into IBS SPSS Statistics 26 software (SPSS Inc, Chicago, IL). Descriptive statistics are expressed in frequencies and percentages with 95% confidence intervals for proportions, and as means and standard deviations for continuous variables. Internal consistency and reliability of the PEIS were calculated using Cronbach’s α and Guttman split-half reliability statistics. Tests for normal distribution (skewness and kurtosis between −2.0 and ​+ ​2.0) and mean plus standard deviation were conducted to identify potential ceiling and floor effects for each item of the scale. Exploratory factor analysis with varimax rotation was used to evaluate the factor structure of the PEIS, with eigen values set to >1.0 to identify potential subscales. Concurrent validity of the PEIS was verified via Pearson correlations between the measure and other measures of stress, quality of life, anxiety, and depression, which the authors expected to be theoretically related to the emotional impact of a pandemic. Criterion-related validity was assessed by examining whether scores on the questionnaires could statistically differentiate subgroups reporting major personal pandemic-related adverse experiences, such as pandemic-related loss of employment or COVID-19 infection in the household, from other subjects. Independent sample t-tests, ANOVA, and Fisher’s exact tests were performed to evaluate differences in continuous and categorical data between demographic or validation test subgroups.

3. Results

A total of 1500 respondents completed the survey. As mentioned above, the survey was designed to be representative of U.S. population demographics. Demographic distributions are shown in Table 1.

Table 1.

Demographic characteristics of the study sample.

Demographic factor	% (n)
Sex Male Female	50.0% (750) 50.0% (750)
Age group 18–34 35–49 50–64 65+	30.0% (450) 24.5% (367) 25.0% (375) 20.5% (308)
Race Black White Asian Other, mixed race or not disclosed	13.0% (195) 74.5% (1118) 3.2% (48) 9.3% (139)
Hispanic Ethnicity	18.0% (270)
Education High school or less Some college or technical school Undergraduate college degree Graduate degree	28.5% (427) 38.9% (583) 12.2% (183) 20.5% (307)
Community size City (>50,000 inhabitants) Town (2500 to 50,000 inhabitants) Village (<2500 inhabitants) Rural setting, no community)	52.7% (790) 31.6% (474) 133 (8.9%) 103 (6.9%)
Regional representation Northeast South Midwest West	17.2% (258) 38.5% (577) 20.9% (314) 23.4% (351)

3.1. Factor structure of the PEIS

Exploratory factor analysis was conducted to assess whether the scale structure of the PEIS contained meaningfully different content subscales. The Kaiser-Meyer-Olkin (KMO) index score of 0.95 and the results of Bartlett’s test of sphericity (χ² ​= ​15396.89, p ​< ​0.0001) indicated that the PEIS scale data from our survey sample were suitable for factor analysis.

Principal components analysis revealed two strong factors with eigenvalues greater than 1, collectively explaining 61.6% of the total variance (53.7% contributed by Factor 1 and 7.9% by Factor 2). Visual inspection of the scree plot and eigenvalues table indicated that there was likely little benefit from including additional factors in the final solution. All of the 16 PEIS items loaded robustly on one or the other of the two identified factors rotated with Varimax method, with no overlap in content when loadings less than 0.5 were eliminated (see Table 2). After reviewing the emerging content division based on this two-factor structure, the themes of “emotional effects” and “pragmatic concerns” were selected to summarize and distinguish the apparent meaning of the subscales.

Table 2.

Component loading values of two factor structure of the PEIS^a.

	Factor 1 “Emotional Effects”	Factor 2 “Pragmatic Worries”
1. More worried about your finances		0.63
2. More anxious or ill at ease		0.67
3. Having more difficulty concentrating	0.63
4. Being less productive	0.66
5. More worried about your personal health or safety		0.78
6. Being more bored	0.75
7. More difficulty sleeping	0.74
8. Feeling more lonely or isolated	0.80
9. Feeling more down or depressed	0.80
10. More worried about getting necessities like groceries or medications		0.72
11. More worried about the health and safety of family members or friends		0.79
12. Feeling more frustrated about not being able to do what you usually enjoy doing	0.57
13. More worried about possible breakdown of society		0.68
14. Feeling more angry or irritated	0.68
15. Feeling that the future seems darker or scarier than before		0.66
16. Feeling more grief or sense of loss	0.63

^aExtraction Method: Principal Component Analysis. Rotation method: Varimax with Kaiser normalization.

Analysis of correlations between the full PEIS scale and the two sub-scales revealed that the two EFA-derived subscales correlated very highly with the full parent scale (Spearman’s Rho of 0.95 for Factor 1 and 0.92 for Factor 2), and also substantially with each other (Rho ​= ​0.76). These findings indicate that the full scale and the subscales will probably in most cases produce similar results in evaluation of individuals.

3.2. Reliability of the PEIS

The PEIS demonstrated excellent internal consistency (Cronbach α ​= ​0.94) and split-half reliability (Guttman’s Lambda 6 ​= ​0.95), both above the standard cutoff of 0.70 for scale development. No significant ceiling or floor effects of scale items were noted. Spearman’s Rho correlations of individual scale items with the PEIS sum score ranged from 0.64 to 0.83. See Table 3 for the means and standard deviations of all items and item-total score correlations for the sample. As seen in the table, the questionnaire item that received the highest average emotional impact rating by subjects in the survey sample was being more worried about the health and safety of family members or friends, followed by increased worry about own health or safety and increased frustration about being unable to do what they usually enjoy.

Table 3.

Mean and standard deviation for each item and Spearman Rho correlations with the total score.

How much has your wellbeing and functioning been different in the following ways in the past 4 weeks, compared to the way it was before the beginning of the COVID-19 pandemic in the U.S.?	Mean (SD)	Item-to-total score correlation
Item
1. More worried about your finances	1.76 (1.33)	0.67
2. More anxious or ill at ease	1.65 (1.26)	0.78
3. Having more difficulty concentrating	1.28 (1.25)	0.78
4. Being less productive	1.38 (1.29)	0.72
5. More worried about your personal health or safety	1.88 (1.35)	0.70
6. Being more bored	1.76 (1.43)	0.64
7. More difficulty sleeping	1.47 (1.41)	0.72
8. Feeling more lonely or isolated	1.50 (1.42)	0.74
9. Feeling more down or depressed	1.45 (1.39)	0.83
10. More worried about getting necessities like groceries or medications	1.48 (1.30)	0.68
11. More worried about the health and safety of family members or friends	2.15 (1.32)	0.67
12. Feeling more frustrated about not being able to do what you usually enjoy doing	1.88 (1.34)	0.69
13. More worried about possible breakdown of society	1.82 (1.33)	0.71
14. Feeling more angry or irritated	1.36 (1.32)	0.79
15. Feeling that the future seems darker or scarier than before	1.71 (1.36)	0.77
16. Feeling more grief or sense of loss	1.25 (1.30)	0.75

∗p values indicate results of t-tests or ANOVAsuperscripts indicate results of Bonferroni-adjusted post-hoc tests. Results with the same superscript are not significantly different from each other. Results with different superscripts were significantly different on post-hoc testing.

Several group differences by demographic variables were observed that are worth noting. First, the PEIS sum score was highest among young adults and decreased steadily across the adult age spectrum on ANOVA test (Bonferroni-adjusted significance p ​< ​0.05 for all comparisons, except p ​= ​0.06 for ages 18–34 vs. 35–49).

Women had higher average PEIS scores compared to men, p ​= ​0.027. ANOVA also revealed significant differences among racial groups. Post-hoc tests showed that white respondents scored significantly lower than black respondents (p ​= ​0.001) as well as respondents identifying as “other, or mixed race” (p ​< ​0.001). Those identifying as being of Hispanic/LatinX ethnicity also reported higher mean PEIS scores compared to those who did not identify as such (p ​< ​0.001). There were no differences in average PEIS scores among education levels. Finally, ANOVA revealed that respondents who were not working during COVID-19 (including those who were students, home-makers, unemployed, and retired) reported significantly lower PEIS compared to those who were working part time (p ​= ​0.004) and those who were working full-time (p ​= ​0.032), Table 4.

Table 4.

Reliability of the PEIS: demographic differences.

	PEIS mean (SD)
Age 18-34 35-49 50-64 65+	31.44 (14.46)a 28.77 (15.64)b 22.72 (15.47)c 17.68 (12.92)d	p ​< ​0.001
Gender Female Male	24.85 (15.21) 26.66 (15.96)	p ​= ​0.027
Race White Black Other/mixed	24.49 (15.61)a 29.14 (15.74)b 30.51 (14.65)b	p ​< ​0.001
Ethnicity Hispanic Latino	32.25 (15.09) 24.30 (15.36)	p ​< ​0.001
Education level High school or less Some college or technical school Undergraduate college degree Graduate degree	26.35 (16.34) 25.15 (15.23) 25.93 (15.66) 26.09 (15.33)	p ​= ​0.641
Employment Full time Part time Not working	26.71 (15.67)a 28.26 (15.46)a 24.44 (15.52)b	p ​= ​0.001

3.3. Validity of the PEIS

The PEIS demonstrated excellent concurrent validity as exhibited by moderate to strong correlations with other emotion- and wellbeing-related outcome measures included in the survey (Table 5). As would be expected for a valid measure of the emotional impact of a stressful life situation, PEIS scores were found to be positively and significantly correlated with anxiety, depression, and stress and, conversely, negatively and significantly correlated with QOL and self-reported happiness.

Table 5.

Pearson correlations between the PEIS and outcome variables.

Concurrent validity measure:	r:
Anxiety symptoms (PHQ-4)	0.748a
Depression symptoms (PHQ-4)	0.713a
QOL rating	−0.307a
General life stress in the past month	0.611a
Stress compared to January	0.384a
Happiness in the past month	−0.422a

^ap ​< ​0.001.

To measure criterion validity, we compared PEIS scores between those who reported and did not report direct personal experiences produced by COVID-19 that could be anticipated to result in increased pandemic-related emotional impact. We found that PEIS scores were higher among the 414 respondents who reported having lost a job or having experienced reduced household income (mean ​= ​32.69) compared to those whose employment/income was not affected by COVID-19 (n ​= ​1086, mean ​= ​23.15, p ​< ​0.001). PEIS score was also higher among respondents who reported that either they themselves or someone else in their household had been diagnosed with COVID-19 (n ​= ​49, mean PEIS ​= ​37.94) compared to those who had not experienced COVID-19 in their household (n ​= ​1451, PEIS mean ​= ​25.37, p ​< ​0.001) and was higher among those who knew someone personally who had died from COVID-19 (n ​= ​181, mean PEIS ​= ​29.36) compared to those who did not (n ​= ​1319, mean PEIS mean ​= ​25.29, p ​= ​0.001).

4. Conclusions

In this study, we demonstrate validity and reliability of the Pandemic Emotional Impact Scale using a nationally representative sample of 1500 adults living in the United States. This scale was created in response to the global COVID-19 pandemic, which has had a widespread impact on the daily lives of adults living in the US and around the world. The purpose was to construct a questionnaire measure that enables quantification of the emotional impact that is directly related to the present COVID-19 pandemic, or other similar pandemics, in a broad and coherent way. We believe that this was achieved, based on the results of our validation analyses.

We found two distinct subscale factors of the PEIS; “emotional effects” (e.g.,. feeling more anxious, depressed, bored or frustrated as a result of the pandemic) and “pragmatic concerns” (e.g., worry about finances, access to groceries or medicines, or the state of society in general). These subscales were highly correlated with the total scale score and with each other, however, and we would therefore recommend using the full scale for most analyses unless there is a specific research question about emotional or pragmatic concerns of a pandemic.

The PEIS demonstrated excellent internal consistency and concurrent validity. It correlated very substantially with higher levels on general measures of anxiety, depression, and stress. Similarly, the higher scores on the PEIS were correlated with worse quality of life and lower reported happiness. As we expected, we also found that individuals who were more personally impacted by COVID-19 (e.g. reported loss of job or income, reported knowing someone who had died of COVID-19, or had experienced COVID-19 infection in their own households) had significantly higher scores on the PEIS compared to those who had not been directly personally affected by the pandemic in these ways.

We also found demographic differences in PEIS scores that indicate that the questionnaire can be of use in assessing disparities in how the pandemic has impacted different demographic groups. For example, Women, Hispanic/latinx and racial minorities reported higher PEIS scores compared men, non-Hispanic, and white respondents, respectively. The PEIS scores in our sample also revealed a steady gradient from higher to lower pandemic-related emotional impact with increasing age across the adult age span that may be a novel finding and warrants further research. In contrast, it was interesting that there were no differences in PEIS scores among different education levels. These demographic differences are consistent with data reported by the CDC showing that younger individuals and ethnic minorities are experiencing greater levels of depression and anxiety than other groups (Czeisler et al., 2020), as well as an Austrian survey, in which younger individuals and women reported more stress during the pandemic compared to older respondents and men (Pieh et al., 2020).

Our findings are complementary to previous studies that have measured similar emotional and behavioral effects of the pandemic on individuals. Analysis of survey data collected in March of 2020 from the Pew Research Center’s American Trends Panel found that emotional distress of survey participants was associated with the COVID-19 pandemic causing major changes to their personal lives, as well as their perception of threats of the pandemic to the US economy and their personal health or finances (Holingue et al., 2020). A study of 1210 respondents in China surveyed between January 31 and February 2, 2020 (Wang et al., 2020)used the Impact Event Scale-Revised, a measure of subjective distress in response to traumatic events, and reported that more than half of the surveyed sample reported the psychological impact of the COVID-19 pandemic to be “moderate or severe” and one third reported moderate-to-severe anxiety. In their sample, 75.2% reported worry that a family member or loved one would contract COVID-19, and 85% were spending 20–24 ​h per day in their homes at that time. Another study of 717 Iranian participants validated a 7-item measure of fear of contracting COVID-19 and reported that this scale was significantly correlated with anxiety, depression, and perceived vulnerability to disease (Ahorsu et al., 2020).

We believe that our new PEIS measure shows promise to offer enhancement over the methodology of these and other studies on the psychological effects related to COVID-19 published so far, which have mostly been using pre-existing general mental health measures for assessment of the emotional effects of this health crisis and did not assess specific pandemic-related effects. The PEIS provides broader and pandemic-specific assessment of individuals, quantifying a range of emotional effects and pragmatic concerns directly attributable to the pandemic.

This validation study had several strengths and limitations that should be noted. The two major strengths of this survey were the close similarity in demographic composition between the survey sample and the adult population, and the timing of the surveying at the apex of the pandemic experience for the national population. The national demographic representativeness in regard to sex, age group composition, and proportional participation of the major race and ethnic groups means that the scale validation is likely to reflect well how this instrument would perform in future research when applied to the general population. The fielding of this study at the end of May 2020, when the vast majority of people had experienced months of marked pandemic-related life changes (83% of the subjects had been sheltering at home for several weeks and 38% of the households represented had suffered job or income loss), means that we were able to sample enough breadth and intensity of emotional impact in the population to test the scale’s ability to characterize the pandemic impact.

Despite these strengths, there were also several limitations. Although we were able to achieve a highly comparable sample to the U.S. adult population, we were not able to control all demographic features of the sample through our quota-based sampling. For example, Asians and other less-populous U.S. race/ethnic groups such as Native Americans or Hawaiian or Pacific Islanders, could not be selectively included in the sample to the extent that matched their national prevalence figures and, thus, we could not analyze their data separately. Second, no test-retest assessment was conducted in this survey, so we cannot provide figures for test-retest reliability of the PEIS when applied at different time points. Third, this study applies only to the US population and the PEIS will need to be validated (and possibly modified) in other countries. It is likely that items on this scale will be applicable in many other cultures, although it is certainly possible that not all items will apply to all cultures. Fourth, it is possible that the use of a survey software and Internet surveying methodology in our study may have introduced some bias. For example, surveying a national population via the Internet means that certain small subgroups in society, such as individuals who are illiterate, cognitively compromised, or do not have access to the Internet, are excluded from participation. However, the use of the Qualtrics XM software is common in research settings and increasingly used for nationwide epidemiological research studies, and allowed for a demographically diverse (and nationally representative sample). Finally, no formal Delphi methods or focus groups were used in the design of the instrument. This was due to the need to develop and disseminate this questionnaire quickly during an unprecedented and rapidly evolving situation, as well as practical constraints such as the temporary suspension of in-person human subjects research at our institutions during the time of the design of the survey. It was foreseeable that population lockdowns would only be implemented widely for a couple of months across the nation, and this constituted the optimal window for assessing the emotional impact on the U.S. population. By using a wide range of news articles describing and quoting personal experiences of the impact of the U.S. pandemic situation for reference, and combining this with health psychologist expert opinion in developing the items on the PEIS, we were able to quickly create a scale that we believe captures substantially the direct emotional effects of a pandemic, as reflected by the evidence of content validity and internal reliability presented above. Most importantly, this approach enabled us to implement data collection with the new instrument while the great majority of adults were sheltering at home by order or recommendation from local and national authorities; more than seventy percent of our study sample reported in the survey that they were still doing so at the time of the survey.

In conclusion, this study provides initial validation of a novel Pandemic Emotional Impact Scale evaluating the impact of the COVID-19 pandemic in the United States. The PEIS demonstrated excellent validity and reliability in a nationally representative sample in the U.S, and is a practical research tool (provided in full in the Appendix) that investigators can use to assess the emotional effects of this and similar future pandemics. Further research efforts should continue to evaluate the demographic disparities in pandemic-related emotional impact found in this study, assess the relationship between PEIS scores and clinical mental health problems, and examine longitudinal changes in the PEIS scores in the population during this, or future, public health crises.

Funding

None.

Declaration of competing interest

None.

Appendix 1

THE PANDEMIC EMOTIONAL IMPACT SCALE.

How much has your wellbeing and functioning been different in the following ways in the past 4 weeks, compared to the way it was before the beginning of the COVID-19 pandemic in the U.S.?

	Not at all	A little bit	Moderately	A lot	Extremely
More worried about your finances
More anxious or ill at ease
More difficulty concentrating
Being less productive
More worried about your personal health and safety
Being more bored
More difficulty sleeping
Feeling more lonely or isolated
Feeling more down or depressed
More worried about getting necessities like groceries or medications
More worried about the health and safety of family members or friends
Feeling more frustrated about not being able to do what you usually enjoy doing
More worried about possible breakdown of society
Feeling more angry or irritated
Feeling that the future seems darker or scarier than before
Feeling more grief or sense of loss

Appendix A. Supplementary data

The following is the Supplementary data to this article: