(Mis)information and anxiety: Evidence from a randomized Covid-19 information campaign

Abstract

Dispelling misinformation during crises is critical to public health. But information can also induce distress. We ask whether the mode of information delivery affects mental health during the Covid-19 pandemic. We randomized Indian migrant workers to receive Covid-19-related information through text messages, a pre-recorded audio message, or live phone calls. Phone calls increased knowledge among individuals without smartphones and reduced depression and anxiety overall. The amount of information delivered explains gains in knowledge but not improvements in mental health. Governments should consider broadcasts through live phone calls given their mental health benefits.

1. I Introduction

Information during crises can be a double-edged sword. On one hand, it is absolutely essential to dispel misinformation. Bursztyn et al. (2020) document that misinformation about the Covid-19 pandemic from television programs in the US correlated with high rates of Covid-19 infections and deaths. On the other hand, information can induce anxiety. Fetzer et al.’s (2020) experiment demonstrated that information conveying a higher degree of threat from Covid-19 led to higher anxiety about economic conditions. Holman et al. (2020) also report that acute stress and depressive symptoms increased during the Covid-19 pandemic, particularly among those more exposed to ambiguous information. In fact, individuals can go out of their way to avoid bad news. In one study, Ganguly and Tasoff (2016) found that their participants were willing to pay to avoid testing for a treatable sexually transmitted disease. If information— whether factual or false— leads to a perception of excessive risk, it can lead to fatalistic attitudes. When Akesson et al. (2020) experimentally changed individuals' beliefs about the infectiousness of Covid-19, those who believed the disease to be more infectious became less willing to adopt precautionary measures.

Anxiety is also tied to cognitive functions. Moran's (2016) meta-analysis shows that anxiety is consistently associated with poor working memory. To the extent anxiety impedes cognitive functions, it can lead to poor decisions. Xie et al. (2020) learned that individuals with lower working memory were less likely to comply with physical distancing measures during the Covid-19 pandemic. Mental illness can entrap the poor in a vicious cycle. Ridley et al. (2020) show that depression and anxiety caused poverty, which in turn exacerbates mental health. Haushofer and Fehr (2014) have made a similar argument.

Against this backdrop, we ask whether certain modes of broadcast are better at delivering information without negative consequences to mental health. We randomly assigned garment industry workers in India to receive information through text messages, a pre-recorded audio message, or phone calls. We then measured their knowledge of Covid-19 and screened them for depression and anxiety.

Phone calls led to the most engagement. Compared to participants receiving pre-recorded audio messages, those who received phone calls stayed on the phone longer, were more likely to listen to the full message, and opted to have the message repeated at a higher rate. Phone calls were only minimally better at improving knowledge, though the effect was prominent for individuals without smartphones. Surprisingly, phone calls reduced depression and anxiety by 16 percent.

These results prompt us to inquire to what extent information alone explains the effects of phone calls. Controlling for the amount of information delivered almost entirely eliminates the effect of phone calls on knowledge, but hardly changes the effect on mental health. This result is noteworthy because, by protocol, we restricted the caller's interaction with participants to reading from a script. In other words, phone calls improved mental health not because they relayed more information, but because a live person did so. A live interpersonal interaction was central to the intervention.

2. Data

Between June and August of 2020, we recruited internal migrants employed in the Indian garment industry to participate in the study. Internal migrants are an important population because they could be at a higher risk of contracting Covid-19 during return migration, and could infect their families. Mobarak (2020) has found that households in Bangladesh that had migrants return during the Covid-19 pandemic were more likely to report symptoms of the disease. This population is also at risk of severe economic and mental health consequences (Ridley et al., 2020). Furthermore, it is crucial for manufacturing sector workers to be informed about the disease in order for production to safely continue in the midst of a pandemic. Manufacturing environments are of particular concern given the potential of fast and large-scale spread. One factory in Sri Lanka, for instance, found 1,000 employees to be Covid-19 positive within three days of detecting the first case (Agence-France Presse 2020).

We selected individuals from an administrative dataset of 23 factories in Karnataka. The dataset contained information on gender, age, education, and whether individuals had left the job. All contact with participants occurred over the phone. While factory workers were predominantly women, we approached roughly the same number of men and women for participation. Since some individuals declined to participate, we were left with 914 individuals at baseline, 57 percent of whom were female. The sample is young with an average age of 24 years. Half of the participants had left their jobs at the time of baseline— also a feature of the recruitment strategy. A third of our sample had an education above 10th grade, and three fourths had smartphones. Just over a quarter lived in hostels with other workers. The average individual was able to recall 5-digit numbers in a test of short-term memory. The sample could solve roughly two out of three arithmetic problem, culminating into an average numeracy score of 2.19 (See Table IV in Appendix.).

We have two outcomes— knowledge and mental health. We measured knowledge using an index that sums scores each participant received on a set of questions measuring their knowledge about various aspects of Covid-19— the symptoms, potential remedies, and spread— standardized with mean zero and standard deviation one. We measured mental health with the four-item Patient Health Questionnaire (PHQ-4) (Kroenke et al., 2009). The PHQ-4 is a combination of two-item Patient Health Questionnaire (PHQ-2), which screens for depression, and the two-item General Anxiety Disorder (GAD-2) questionnaire, which screens for anxiety. (See Appendix for precise definitions.)

Baseline data confirm that there was substantial misinformation among participants (see Table IV in Appendix). About a third did not mention cough to be a symptom of Covid-19, and about half did not mention fever. Fewer than half were certain that non-symptomatic individuals can spread the disease. Participants also seemed to hold inconsistent beliefs. While only eight percent were certain that Covid-19 had remedies, 26 percent said they would recommend symptomatic individuals to take antibiotics, and 13 percent said they would recommend drinking cow's urine. A third felt that consuming turmeric regularly protects from Covid-19 infections, and 21 percent said that people of certain religions are more likely to spread the disease. About 18 percent of the sample attrited between baseline and endline (Attrition is further discussed in Section III.).

3. Research design and execution

We randomly assigned study participants to receive information on Covid-19 via one of three modes: text messages, a pre-recorded audio message, or live phone calls. We decided to not have a pure control group, one that does not receive any Covid-19 information, for two reasons. First, we wanted to take any opportunity to push back Covid-19 misinformation. When we designed the study, evidence was already emerging that misinformation led to risky health behavior and worse health outcomes (Bursztyn et al., 2020). Preliminary inquiries also led us to believe that misinformation was particularly high among our study population, and that they lacked reliable information in their native language. We viewed a pure control group as potentially unethical. Second, governments around the world were already launching information campaigns. Text and voice-based information were the policy status quo (World Health Organization 2020). We believed that our study would be more policy relevant if we were able to measure the efficacy of phone calls vis-a-vis the policy default. In our judgment, this was the best use of the study's resources and respondents' time.

We stratified the sample for randomization. Each strata was defined as a unique combination of the following four variables: 1. whether the individual was female, 2. whether the individual had education above 10th grade, 3. whether the individual had left their job as of February 2020 when the pandemic reached India, and 4. the factory where the individual was employed. About 20 percent of participants were assigned to receive text messages, and the rest were split between pre-recorded audio and phone calls. Surveyors too were assigned to participants at random at both baseline and endline. Table 1 shows that baseline characteristics and outcome measures were balanced across all three intervention groups.

Table 1.

Randomization balance.

Variable	TM	VR	PC	TM - VR	VR - PC	TM - PC
Female	0.58	0.56	0.56	0.02	0.00	0.02
				(0.04)	(0.04)	(0.04)
Age	23.86	24.23	24.00	−0.37	0.23	−0.14
				(0.49)	(0.45)	(0.50)
Educated above grade 10	0.36	0.33	0.32	0.03	0.01	0.04
				(0.04)	(0.04)	(0.04)
Left job	0.54	0.50	0.50	0.04	0.00	0.04
				(0.04)	(0.04)	(0.04)
Smartphone	0.75	0.75	0.75	−0.00	0.01	0.01
				(0.04)	(0.03)	(0.04)
Lives in hostel	0.23	0.30	0.30	−0.06	0.00	−0.06
				(0.04)	(0.03)	(0.04)
Numeracy Score	2.21	2.18	2.20	0.03	−0.02	0.01
				(0.09)	(0.08)	(0.09)
Short-Term Memory	5.24	5.34	5.15	−0.10	0.19	0.09
				(0.17)	(0.14)	(0.16)
Knowledge	−0.02	−0.00	0.02	−0.02	−0.02	−0.04
				(0.09)	(0.07)	(0.09)
PHQ-4	3.07	2.92	3.04	0.14	−0.11	0.03
				(0.24)	(0.21)	(0.24)

The table reports statistics from t-tests comparing differences in means between the three intervention groups: Text Message (TM), Voice Recordings (VM), and Phone Calls (PC). Standard errors are in parentheses.

The content of the message remained unchanged for all three modes (see Appendix). Moreover, we ensured that the pre-recorded audio message was in the voice of the same person who made the phone calls, and that the speed at which the caller read the message script matched the speed of speech in the pre-recorded audio. The caller used a tablet-based software that showed them paragraphs to read. The software did not allow the caller to move onto the next paragraph unless they had spent a minimum number of seconds on a given paragraph. We also trained the caller to decline requests for additional information and to refrain from consoling participants.

As a protocol, participants from about five strata were surveyed each day, then randomized and administered interventions the following day. Text messages were predominantly delivered in one attempt. On the contrary, we made up to eight attempts to deliver the treatment through pre-recorded audio or phone calls. Roughly 93 percent of text messages were delivered.1 Likewise, 94 percent of those sent pre-recorded audio answered their phones, but only 86 percent answered the phone when called (see Table 2 ).

Table 2.

Information delivery.

Variable	Phone Call	Pre-Recorded Audio	Difference
Answered phone	0.86	0.94	−0.08
			(0.02)
Information Content	4.89	3.64	1.25
			(0.22)
All information delivered	0.66	0.40	0.26
			(0.04)
Opted to repeat information	0.14	0.01	0.12
			(0.02)

The table reports statistics from t-tests comparing differences in means between groups assigned to phone call and pre-recorded audio. Standard errors are in parentheses.

We do not observe whether the text messages were read. However, conditional on answering the phone, we observe how long participants in the pre-recorded audio and phone call interventions stayed on the phone. Since we ensured that the phone calls and the pre-recorded audio delivered information at the same speed, we are able to determine how much of the message content these interventions delivered before the participant disconnected. Fig. 3 shows that participants who received phone calls stayed on the phone longer compared to those sent pre-recorded audio. Table 2 shows that two thirds of the participants assigned to phone calls were delivered the entire message in contrast to 40 percent of those assigned to pre-recorded audio. Participants had the option to repeat the message at the end of the pre-recorded audio message as well as phone calls. Those assigned to phone calls were 12 percentage points more likely to make this choice. Essentially, phone calls engaged participants more than pre-recorded audio.

We administered the median endline survey 18 days after intervention. Despite various survey protocols set up to minimize attrition, we were unable to contact 18 percent of our baseline sample for endline. As Table V shows, attrition is imbalanced along baseline characteristics and treatment assignment, but not along outcome measures. To correct for this imbalance, we estimate treatment effects by weighing observations with the inverse of the probability of remaining in the study at endline. We also estimate Lee (2009) bounds to check if zero effects can be ruled out.

4. Methods

While we present analysis in keeping with our pre-registered design in the appendix, certain features of our data deviate from assumptions under which we pre-registered our study. Knowledge and PHQ-4 were not as strongly correlated between baseline and endline as our power calculations anticipated. In order to draw meaningful insights, we present results where we pool two of the three interventions. Our views on the similarity of interventions have also evolved since pre-registration. Pre-recorded audio messages and phone calls are similar in that they convey information through audio rather than text. From a different viewpoint, text messages and pre-recorded audio messages are both impersonal compared to phone calls. The latter demarcation was predominantly favored by the group of India-based researchers that we consulted. Our interpretation of Fig. III and Table 2 also suggest that the latter view is the most pertinent to the analysis.

We estimate variants of the following econometric model using Least Squares regression:

$$outcom{e}_i^1=\beta \cdot 1\left[cal{l}_i\right]+\gamma \cdot outcom{e}_i^0+s_i+r_i+{\in }_i$$ (1)

where i denotes each individual, outcome ¹ _i is the outcome variable after treatment, 1[call _i] indicates assignment to phone calls, outcome ¹ _i is the outcome variable before treatment, s_i indicates surveyor fixed effects, r _i indicates trial round fixed effects, and _ϵi is the unobserved error. The parameter β represents the effect of assignment to phone calls relative to assignment to either pre-recorded audio messages or text messages.

We also test whether short-term memory, numeracy, and smartphone ownership moderate the effects of phone calls. We estimate variants of the following model using Least Squares regression:

$$outcom{e}_i^1=\phi \cdot 1\left[cal{l}_i\right]+\delta \cdot M_i\cdot 1\left[cal{l}_i\right]+\lambda \cdot M_i+\gamma \cdot outcom{e}_i^0+s_i+r_i+{\in }_i$$ (2)

where M_i is an indicator for either high (at or above median) short-term memory, high (at or above median) numeracy, or smartphone ownership.

Fig. 1 presents a comparison at endline of the treatment groups and Table 3 presents estimates of Model (1). They show that phone calls were only minimally better at improving knowledge. Individuals assigned to phone calls were more knowledgeable by about 10 percent of standard deviation when compared to those assigned to either text messages or pre-recorded audio messages. The estimate is smaller and less precise when we use inverse probability weights and control for participant characteristics. Lee (2009) bounds do not rule out a null effect. Contrarily, phone calls seem to reduce depression and anxiety. PHQ-4 scores were 16 percent lower for those assigned to receive phone calls. The effect on anxiety and depression is stable when we use inverse probability weights. Lee (2009) bounds rule out a null effect. Table VI in the Appendix shows that the effect is larger on anxiety (GAD-2) than on depression (PHQ-2). Phone calls reduced moderate to severe anxiety (GAD-2≥3) by 28 percent. The magnitude of the effects would likely be larger if our phone calls had reached participants at the same rate as pre-recorded audio messages.

Fig. 1.

Comparison of knowledge and PHQ-4 at endline.

Table 3.

Effects of phone calls on knowledge and PHQ-4.

	Knowledge				PHQ-4
Variables	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
Phone Call	0.12 (0.06)	0.11 (0.07)	0.10 (0.07)	0.06 (0.07)	−0.46 (0.16)	−0.49 (0.17)	-0.49 (0.16)	−0.48 (0.20)
Knowledge (Bl)	0.50 (0.03)	0.50 (0.04)	0.47 (0.04)	0.49 (0.04)
PHQ-4 (Bl)					0.53 (0.04)	0.53 (0.04)	0.51 (0.04)	0.49 (0.04)
Surveyor (Bl) FE	✓	✓	✓	✓	✓	✓	✓	✓
Surveyor (El) FE	✓	✓	✓	✓	✓	✓	✓	✓
Trial Round FE	✓	✓	✓	✓	✓	✓	✓	✓
IPW		✓	✓	✓		✓	✓	✓
Control Variables			✓	✓			✓	✓
Information Delivery				✓				✓
Observations	739	735	735	573	737	733	733	573
Adjusted R2	0.33	0.34	0.36	0.37	0.37	0.36	0.38	0.37
Mean of Outcome (Bl)		−0.00				3.00

The table reports results of Least Squares regressions estimating variants of Model (1). Bl means baseline, El means endline, FE means fixed effects, and IPW means inverse probability weights. Control variables include gender, age, employment status, indicator for hostel residence, numeracy, short-term memory, smartphone ownership, and indicator for having education above grade 10. Information delivery refers to two variables defined only for phone calls and pre-recorded audio: one that counts the paragraphs of information delivered before the participant disconnected and another that indicates whether the participant opted to have the information repeated. Regressions controlling for information delivery exclude individuals assigned to text messages. Standard errors are robust to heteroskedasticity.

We observe the information content delivered through phone calls and pre-recorded audio. We also know if individuals receiving these interventions opted to have the information repeated. Controlling for these variables explains almost all of the effect of phone calls on knowledge relative to the pre-recorded audio, but not the effect on PHQ-4. In other words, information delivery does not on its own account for the effect on depression and anxiety.

Fig. 2 presents a comparison at endline of the treatment groups by smartphone ownership and Table VIII in the Appendix shows estimates of Model (2). Phone calls were more effective on those without smartphones. Among participants assigned to phone calls, those without smartphones became more knowledgeable by 18 percent of standard deviation compared to those with smartphones. Numeracy and short-term memory do not moderate the effect of phone calls on knowledge, however. The effect on depression and anxiety also did not change with numeracy, short-term memory, or smartphone possession.

Fig. 2.

Comparison of knowledge and PHQ-4 at endline by smartphone ownership.

5. Conclusions

Previous research shows that exposure to discussions of mass violence (Holman et al., 2013; Thompson et al., 2019b), natural disasters (Thompson et al., 2019a), as well as outbreak of infectious diseases such as Ebola (Thompson et al., 2017) and Covid-19 (Gao et al., 2020; Holman et al., 2020) are associated with anxiety and distress. Given the rise in stress, anxiety and depression during the Covid-19 pandemic (Salari et al., 2020), the mental health consequences of risk communication demand attention. Holmes et al. (2020) have called for urgent research on health messaging that reins in distress. Our study responds to this call. Whereas text messages and pre-recorded audio dominate Covid-19 information campaigns targeting individuals without access to the internet (International Telecommunication Union 2020; World Health Organization 2020), we demonstrate that phone calls, which cost under $0.25 per message in our context, can broadcast information at least as effectively as conventional methods, yet with better consequences for mental health. These results suggest that governments reconsider the dependence on automated information broadcasts during crises.

Our results should be viewed in light of certain considerations. Since we do not have a pure control group, we are unable to comment on the effects of text messages, audio messages, or phone calls relative to no information intervention. Moreover, the study participants were low-income internal migrants in India who were under considerable distress during the early stages of the Covid-19 pandemic. While they represent an important segment of the population, extrapolating the results to other segments of the population requires care. We also reiterate the nature of the phone call intervention. It was designed strictly for the purpose of information delivery. Phone calls designed specifically to task-shift mental health— provide mental health services through minimally trained persons— could be even more effective, consistent with existing evidence (Javadi et al., 2017; Kakuma et al., 2011).

Contributorship statement

D Sadish, Achyuta Adhvaryu, and Anant Nyshadham designed the study, performed the analysis, and wrote the manuscript. D Sadish executed the data collection and experiment.

Pre-registration

The trial was pre-registered with the American Economic Association's Trial Registry (https://doi.org/10.1257/rct.5947).

Declaration of competing interest

Achyuta Adhvaryu and Anant Nyshadham disclose that they are members of the Board of Directors and serve as Chief Development Officer and Chief Strategy Officer, respectively, at Good Business Lab. None of the authors (Sadish, Adhvaryu, nor Nyshadham) have any financial interest in Good Business Lab.

The study received IRB approval at the University of Michigan (HUM00180576).

Ethics

This study was approved by Good Business Lab's Ethics Committee in India (Identifier: GBL0520) and University of Michigan Human Subjects Research Committee in the United States (Office of Human Research Protections Registration Number: IRB00000246). All study participants provided informed consent.

Funding

Good Business Lab funded the data collection and interventions.

Data and code availability

Anonymized data, a data dictionary defining each field, and software code used in the analysis are freely available at https://doi.org/10.7910/DVN/WKDC9I.

Appendix A. Supplementary data

The following is the Supplementary data to this article:

Data availability

Data are published with DOI: 10.7910/DVN/WKDC9I.