Increasing the use of telemedicine: A field experiment

Abstract

Patients are reluctant to use telemedicine health services, compared to its substitute in-person visits. One reason is that telemedicine can be accurately evaluated and compared to its substitute only after the product has been adopted and experienced. As such, an intervention that increases the probability of a first experience can have lasting effects. This article reports the results of a randomized field experiment conducted in collaboration with a health insurance company. During the intervention, half of the households out of 3,469 in the sample received periodic e-mails with information about the available services. It effectively increased the take-up and demand for telemedicine. Within the first 8 months of the experiment, patients assigned to the treatment group were 6 percentage points more likely to have used the service at least once (and had about five times the odds of using telemedicine compared to those in the control group). Eight months after the start of the intervention, the number of virtual consultations by the treatment group was six times larger than that of the control group. These results, even if limited by the sample and context in which the intervention took place, provide additional evidence about how information interventions can increase technological take-up within the health sector and could serve as the stepping stone for evaluating the impact of telemedicine on health outcomes causally.

Significance Statement.

Our study demonstrates the effectiveness of information interventions in overcoming patients' reluctance to adopt telemedicine. In a randomized experiment with a health insurance company, we sent periodic e-mails to about 2,000 households informing them about available telemedicine services. This approach significantly increased the uptake and demand for telemedicine, with treated patients being 6 percentage points more likely to use the service at least once and had about five times the odds of using telemedicine compared to those in the control group. Over 8 months, virtual consultations in the treatment group were 6-fold that in the control group. These findings underscore the role of targeted information in reducing behavioral barriers to technology adoption in health care, highlighting a scalable strategy to enhance patient engagement with telemedicine services and a procedure to generate the conditions to study the impact of telemedicine causally.

Introduction

High and increasing costs and inequality of access to health care are pressing issues for developed and developing countries alike. Telemedicine can potentially lower costs and increase the convenience of health services by shifting care from hospitals and clinics to homes and mobile devices (1–3). Telemedicine can provide primary and specialized care to the geographically disconnected (4, 5), during times of crisis in response to natural disasters (6) and humanitarian responses (7), and when mobility is restricted by a pandemic (8–10) or war (11, 12).

During the COVID-19 health crisis, telemedicine proved to be a viable substitute for in-person consultations when mobility restrictions limited them (13–15). Despite its growth and potential, telemedicine's place in the healthcare system is still small.¹ One healthcare insurer in Argentina saw that by the end of 2020, more than 80% of their affiliates had no experience with telemedicine, even when it was free and readily available to them.² There are several reasons why people may be reluctant and resist the use of new technologies, such as telemedicine, that go from inconvenience (17–21) and mistrust (22) to behavioral biases (19–21, 23–30). Because in-person consultations have been the default method of contact between patients and medical doctors for centuries, experiencing the product can have multiplicative effects in its use (13).

In this paper, we evaluate the impact of an informational campaign that attempts to reduce the behavioral barriers to adoption. For the intervention, we partnered with APSOT, a health insurance company in Argentina, and Llamando al Doctor, the telemedicine company that provides telemedicine services to APSOT members and many others.³ During the intervention, about 3,500 households that had access but had never used the system were randomly allocated to a treatment and a control group. Households in the treatment arm received a series of messages that presented simple and actionable information about the benefits of telemedicine and how to use the service. The e-mails were sent from 2021 July 6 to August 24. After 8 months, the households assigned to treatment were about 6 percentage points (pp) more likely to have used the system at least once than those in the control group (3.5 times more likely than those in the control group). Considering the households that opened at least one of the e-mails, the difference doubles. Importantly, this first use led to a larger cumulative difference in use over time. After 8 months, the number of virtual consultations by the treatment group was six times larger than that of the control group.

The results provide evidence that information interventions can effectively change patients' behaviors and the status quo. They are even more encouraging, considering that the environment for this intervention is more complex than for other take-up interventions. Compared to interventions where the individual can take immediate action after receiving the message, in this case, households are receiving information about a service they do not necessarily need to use immediately but only once they have to conduct a medical consultation. As such, even if the messages change beliefs and intentions, they may not affect actual measurable behavior in the short-to-medium run.

This paper contributes to the literature using reminders, messages, and other behaviorally informed treatments to increase compliance in the health sector. Previous studies have found that reminders can decrease no-show rates for clinics (31) and increase prenatal doctor visits (32), the frequency of dental check-ups (33), and the screening rates for breast and cervical cancer (34–36). They can also increase the demand for vaccination (37–41). Messages that employ digital tools are proven to benefit meaningful public health decisions and reduce the cost of these interventions, and they can even change the behavior of doctors and nurses (42–45).

Still, plenty of message-based interventions have no effects. For example, message interventions did not improve willingness to vaccinate or parents' attitudes toward vaccines (46, 47). A systematic review of the literature shows that some types of messages work, while others do not, including vaccine myths debunking or employing scare tactics (48). More generally, Ruggeri et al. (49) found null effects in 141 out of 737 pandemic-related research studies for messages that emphasize benefits to the recipient and messages that focus on protecting others. The results presented in this paper constitute evidence that information interventions are also effective in promoting telemedicine use.

This paper also adds to the literature on “experience goods.” The demand for these goods changes significantly after individuals try them (50), which leads to an inefficient equilibrium ex ante. A simple way to characterize these goods is as follows. Let U_b = f (C_b, E_b) be the utility (or perceived value) of telemedicine before trying it, where C stands for perceived convenience and E stands for perceived effectiveness (we mention two attributes for simplicity, but it could be generalized to N). Let U_a = f (C_a, E_a) be the utility of telemedicine after trying it. Assuming that the experience of using telemedicine impacts perceptions of its convenience and effectiveness, we would expect C_a > C_b and E_a > E_b. Hence, U_a(.) > U_b(.), and the consumption of telemedicine after experiencing it is higher than before doing it.⁴

Because people underestimate the value of the good, dynamic pricing (lowering the price originally and then gradually increasing it) has usually been the market solution (51–53).⁵ Another way of dealing with such an inefficient equilibrium has involved regulation. The case of rear-view cameras in cars is a good example of this approach. Once they became mandatory and people experienced them, they were willing to pay more and ask for compensation if the car did not have it (50).

In the case of certain products, however, consumption may be low even if the price is zero, and making it the default option may have economic and political costs. This is the case with telemedicine. Most health insurance plans provide the service for free to avoid charging a price. However, making telemedicine mandatory at the health insurance level risks exit to other insurers. Making it mandatory at the national level could generate political backlash. Moreover, because for certain medical conditions, it may be preferable to attend to the patient in person, mandatory use could carry additional health and welfare costs. Behaviorally informed interventions, such as those described in this paper, could complement the set of policy tools for this type of goods.

Methods

All methods were carried out in accordance with relevant guidelines and regulations.

Intervention

Background

Argentina is one of the first adopters of telemedicine in the region (54). It is no surprise, considering the country's vast territorial extension and the increasing healthcare costs.⁶ In 2019, 26% of hospitals nationwide reported offering telemedicine services to their patients (54).⁷ During the pandemic, the Argentinian telemedicine network experienced a rapid expansion within the private and public sectors, and regulations adapted to allow electronic or digital prescriptions, enabling their use in telecare platforms.

The public platform TELESALUD hosted 120,000 consultations in 2020, a 600% increase from the previous year (55). Private health providers also experienced a sharp expansion of demand for telemedicine. Telemedicine demand for these providers increased by 200%, with a persisting trend for the rest of the year (13). Despite this unprecedented increase, telemedicine's place in the healthcare system is still small. In the case of APSOT, which is similar to all the other insurance companies Llamando al Doctor works with, by the end of 2020, more than 80% of their affiliates had no experience with telemedicine, even when it was readily and freely available to them. If the goal is to achieve widespread use of telemedicine, the challenge lies in understanding the elements that fuel resistance to adoption.

The telemedicine service Llamando al Doctor is provided at no cost by the health insurance APSOT. The services provided by Llamando al Doctor include: Clinic, Pediatrics, Gynecology and Obstetrics, Cardiology, Ophthalmology, Dermatology, Otorhinolaryngology (Ear, Nose, and Throat), and Neurology.

Llamando al Doctor has contracted physicians from the top hospitals in the country to avoid trust issues regarding the quality of the service. In some cases, health insurers could use their own physicians or have a mix. As described in the e-mail sent on weeks 2 to 4, APSOT uses doctors from Llamando al Doctor, but patients also have the ability to reach the doctors they usually visit in person (but in that case, with limited hours.) As such, doctors are not necessarily the same as the ones the individual regularly attends, particularly if the individual lives outside the City of Buenos Aires, where most of the physicians reside.

The way the service works is as follows. Patients access the service primarily through a mobile phone application, which asks a series of screening questions about the medical specialty required, the reason for the consultation, and any existing health conditions the caller may have. Following the screening, the caller proceeds to the online consultation with a physician through a video call. Each video call can result in one of three outcomes: The first and most common outcome is that a doctor resolves the patient's issue during the online consultation. This was the case for 67% of the calls that were made in 2019 (13). Doctors who resolved the issues presented in a call sometimes prescribed medicine to the patient (as was the case in 11.5% of the overall calls). A second outcome is that a patient receives a recommendation to participate in a follow-up call (as was the case for 8.3% of the overall calls). A third outcome is that a doctor refers the patient to an in-person visit (as occurred in 10.8% of the calls).⁸

Each call produces a log that registers the patient's gender and age, the medical specialty requested, a description of the reason for the call, and the diagnosis that resulted from the call. Descriptive statistics for 2019 based on the (anonymized) administrative data indicate that patients who use telemedicine are relatively young (30 years old on average) and more likely to be women (57%) (13). Figure S11 shows the distribution of age for patients following our intervention. The majority of consultations relate to general medicine and pediatrics both in 2019 and 2022, as can be observed in Table S2 in the Supplementary material.

Physicians have access to the medical history of telemedicine consultations but do not have access to the patient's overall medical history unless they also treat them in person.

Treatment design

Why do people resist adoption? Most people have experience with in-person medical consultations, which have long been the standard. As switching from the status quo is not easy, people need to experience telemedicine before adopting it as a regular practice. In Argentina, and in line with pricing practices for experience goods, telemedicine is free (either publicly provided by those attended by the public system or by private health insurance companies), so reducing the price is not an option.⁹ Still, dealing with the reasons that may delay adoption can reduce the cost of switching.

First, patients may lack information or mistrust telemedicine's effectiveness. Individuals do not know how telemedicine works, and they may be worried that the quality of the physicians and the experience could be subpar (56, 57). Second, switching to telemedicine has some real—even if small—inconvenience factors, such as downloading and setting up the technology, which could discourage its use or trigger procrastination (24). Third, a number of behavioral biases may limit the use of telemedicine. Individuals may not download the application because of present bias, which makes people undervalue the future gains of having the service ready to use should they become sick (19, 20, 25, 26, 29). Once they are sick, moreover, the cognitive burden could be too high for individuals to use the service. This can be compounded by optimism bias, which leads people to underestimate the probability of negative events (e.g. “Why would I download the app and register if I never get sick?”) or by loss aversion, which can lead people to worry that using telemedicine could jeopardize access to in-person visits later on (24, 58). These biases build on consumers' reticence to move from a known status quo to newer alternatives (21, 23, 24, 27, 28, 30).

The messages employed in the campaign and sent to the treatment group are designed to address these barriers. First, the messages present simplified access to information about the benefits of Llamando al Doctor and easy-to-follow guides for using the service (20, 59). Second, they include actionable steps and calls to action (60). Third, the messages rely on innovative and interactive ways to present information about how to use the service, giving more salience to the ease of use and utility (60). These messages also contain priming questions designed for the reader to actively consider their choice of using telemedicine (61). Finally, these e-mails also highlight that the service is provided by the same doctors the patients see in their in-person visits to reduce mistrust. By providing quotes and the doctors' personal information, the messages generate personalization and familiarity (62). The eight e-mails are attached in Section S.1. in the supplementary material. An explanation of each behavioral insight used in constructing the message is also provided.

Sample and data collection

To increase telemedicine take-up using e-mail reminders and behavioral messaging, we conducted a communication campaign in partnership with APSOT, a private health insurance company that serves the managerial staff of the Techint Group in Argentina.¹⁰ At the time of the intervention, APSOT had 10,936 individual members (which includes policyholders working for Techint and their relatives or dependents).¹¹ Several criteria were considered for inclusion in the intervention: (i) the primary beneficiary of the insurance account should be at least 18 years old, (ii) all of the members under one account had no prior experience with telemedicine, and (iii) the primary beneficiary has a registered e-mail address with APSOT. For the purposes of the study, the primary beneficiary is considered to be the head of the household. Because APSOT has contact information for the primary account holders and not each individual in the household, the unit of observation for this study is a household comprised of all members sharing one account. Column 1 in Table S1 in the Supplementary material provides descriptive statistics of the primary beneficiaries and household composition.¹²

A sample of 3,548 households was randomized into a control and a treatment group (1,774 in each group). Two strata were used during the randomization: households that live in the capital city of Argentina (29%) and those that live in other cities (71%).¹³ The sample is balanced between treatment and control in all the available observable covariates (Table S1 in the Supplementary material).

The intervention consisted of eight e-mails sent by APSOT between 2021 June 6 and August 24. Table 1 presents the dates and subject lines of these e-mails. The content of each individual e-mail is included in Section S.1. in the supplementary material. We only use administrative data. APSOT provided household data, and Llamando al Doctor provided information about the virtual medical consultations for each APSOT member. Llamando al Doctor also provided group-level information on the number of downloads of the app (Llamando al Doctor has access to the total number of downloads coming from the members of APSOT, but that information cannot be linked to the individual household until the app is used).

Table 1.

E-mail campaign timeline.

E-mail	Date sent	E-mail subject translation	Behavioral insights
S1	2021 July 6	Winter is coming! Download Call the Doctor	Actionable steps, Simplification, Prominence, Call to action, Making it easy
S2	2021 July 14	Winter is here. Download Call the Doctor	Simplification and prominence, Call to action
S3	2021 July 20	A quick and safe way to visit your doctor. Download Call the Doctor	Simplification and prominence, Call to action
S4	2021 July 27	Does someone in your family need to see the doctor? Download Call the Doctor for them.	Attractive and interactive, Simplification and prominence, Call to action.
S5	2021 August 3	Your doctor from the comfort of your home. Download and use Call the Doctor	Priming question, Personalization and familiarity, Actionable steps, Call to action.
S6	2021 August 10	Call the Doctor: A doctor without leaving your home	Simplification, Priming questions.
S7	2021 August 17	Did you already use our app Call the Doctor?	Priming questions, Personalization and familiarity, Actionable steps, Call to action.
S8	2021 August 24	Do you need a consultation? Download Call the Doctor	Priming question, Actionable steps, Call to action

The table presents the dates and subject lines of the e-mails. The last column shows the behavioral insights used in the design of the e-mail content. All the e-mails are included in Section S.1. in the Supplementary material.

The number of e-mails read weekly ranged from 435 to 613, with a decreasing trend over time. Throughout the first week, 35% of the treatment sample opened the e-mail. The opening rate gradually decreased; the last week, the opening rate was only 25%. Figure 1 shows the cumulative opening rates (bars) and the additional number of households that opened an e-mail for the first time (boxes). While the number of households that opened an e-mail for the first time in week 1 was 613, the number dropped to 17 by week 8. By the end of the campaign, 901 households, or about 51% of the treatment sample, had opened at least one of the e-mails.

Fig. 1.

E-mails opening rate. This figure shows information about the opening rate of the e-mails during the campaign. The dark blue columns (lower section of each column) represent the accumulated number of households that opened an e-mail. The left scale represents the number of households, while the right scale represents the proportion of households in this category. The label above each column represents the additional number of households that opened an e-mail for the corresponding week.

Empirical strategy

The main specification is:

$$Y_i={\beta }_0+{\beta }_1{T}_i+X_i\mathrm{\Delta }+{ϵ}_i$$ (1)

where Y_i is an outcome variable for individual i—whether the household used telemedicine, T_i is an indicator variable that records assignment to treatment, β₁ is the intent-to-treat effect of the communication campaign, and X_i is a vector of characteristics of household i: sex of head of household, location—capital city or elsewhere, age of the head of household (18–24, 25–39, 40–54, 55–64, and 65 or older), number of family members (≤2, 3–5, and 5–9), and household age composition (it has at least one member younger than 18, it has at least one member aged 65 or older, it has both, and it has none of those).

We estimate the model using a linear probability model and a logistic regression. Given that many households may not actually receive the e-mails or open them, we also compute the LATE estimates using an instrumental variables estimator of actual treatment (opening at least one of the eight e-mails) instrumented by assignment to treatment.

Results

Was the campaign successful? Before showing the effect on use, we look at the number of households downloading the mobile application following the e-mail campaign, which is a prerequisite to being able to use telemedicine services when needed. Figure 2 shows the time series of downloads, with dotted lines indicating the dates of each e-mail. As can be observed, the pattern of downloads was quite stable during the 6 months prior to the campaign, with an average of six downloads per day. In order to see whether this pattern changes statistically after the intervention, we conducted an event study approach to evaluate the differences in downloads of the application during each week of 2021. Figure 3 shows the event study estimation of new app users each week relative to the immediate week before our first intervention (week 26). As can be observed, the download pattern prior to the intervention is consistent and does not statistically deviate from our baseline. The pattern, however, drastically changes once the intervention begins in week 27. On July 7, the number of mobile application downloads jumped to 107. The application download growth relative to the baseline is statistically significant for the first 3 weeks after the first intervention, with user growth also significant at week 5. A similar pattern of out-of-trend peaks occurs the days the rest of the messages are sent. The decreasing size of the peaks is aligned with the decreasing rate of new e-mail openings shown in Figure 1. After the last e-mail sent in week 34, the number of downloads stabilized again. This evidence helps to link the e-mail campaign and the use of the service, which we explore next.

Fig. 2.

Time-series application downloads for 2021. This figure shows time-series data about the number of new users of the application for the year 2021. These data pertain only to patients who use APSOT as their insurer. The vertical dashed lines indicate the dates when the e-mails were sent. Information about the subject of the e-mails is found in Table 1.

Fig. 3.

Impact of treatment on application downloads per week during 2021. This figure shows the event study estimation of the number of new application users each week from 2021 January to 2021 December. The relative baseline of downloads is week 26, the immediate week before our first intervention. Each week, between the 27th and the 34th, included a day treated by the intervention. The data pertain only to patients who use APSOT as their insurer. The vertical dashed lines indicate the dates when the e-mails were sent. The first e-mail was sent in week 26 (2021 July 6), and the last e-mail was sent in week 34 (2021 August 24).

Table 2 presents the regression results for the estimation of Eq. 1 (Intention to Treat estimates [ITT]) using ordinary least squares (OLS). Odd columns present the regression results with no controls and even columns with controls for the use of the app by three different cutoff dates: 2021 November 30, 2021 December 31, and 2022 February 28. The coefficient of interest estimates the probability that a household in the treatment group would have used the service compared to one in the control group. Five months after the beginning of the campaign, the treatment group was 4.1 pp more likely to have used telemedicine compared to the control group (column 2). The difference between the treatment and control group grew to 4.5 pp after 6 months and 6 pp after the eighth month.

Table 2.

Regression results (ITT)—OLS.

Dependent variable: 1{Telemedicine Use}
	2021 November		2021 December		2022 February
	(1)	(2)	(3)	(4)	(5)	(6)
Treatment	0.041***	0.041***	0.045***	0.045***	0.060***	0.059***
	(0.006)	(0.006)	(0.006)	(0.006)	(0.006)	(0.007)
Constant	0.013***	0.006**	0.014	0.022***	0.017	0.017***
	(0.004)	(0.003)	(0.022)	(0.003)	(0.024)	(0.003)
Control variable	No	Yes	No	Yes	No	Yes
Observations	3,469	3,469	3,469	3,469	3,469	3,469
Adjusted R2	0.013	0.017	0.014	0.020	0.020	0.029

The table presents the results from model 1 using a linear probability model. The first two columns present results using the dataset with a cutoff date of2021 November 30 with the first column presenting OLS results and the second column controlling for covariates described in Table S1. Columns 3 and 4 present these results for the dataset with a cutoff date of 2021 December 31, and columns 5 and 6 present results for the dataset with a 2022 February 28 cutoff date. **indicates significance at the 5%, and ***indicates significance at 1% level.

Table 3 presents the results using a logistic regression. Columns present the odds ratio. Again, the effect is large and highly significant and grows over time from an odds ratio of 4.2 to approximately 4.9. An odds ratio of 4.9 means that those who received the treatment have 4.9 times the odds of using telemedicine compared to those in the control group.

Table 3.

Regression results (ITT)—logit.

Dependent variable: 1{Telemedicine Use}
	2021 November		2021 December		2022 February
	(1)	(2)	(3)	(4)	(5)	(6)
Treatment	4.251***	4.245***	4.486***	4.463***	4.869***	4.865***
Constant	(1.000) 0.013***	(1.002) 0.004***	(1.029) 0.014***	(1.027) 0.010***	(1.012) 0.017***	(1.016) 0.008***
	(0.003)	(0.005)	(0.003)	(0.010)	(0.003)	(0.007)
Control variable	No	Yes	No	Yes	No	Yes
Observations	3,469	3,469	3,469	3,469	3,469	3,469
Pseudo-R2	0.05	0.07	0.05	0.08	0.06	0.09

The table presents the results from model 1 using logistic regressions. Coefficients correspond to odds ratios. The first two columns present results using the dataset with a cutoff date of 2021 November 30, with the first column presenting logit results and the second column controlling for covariates described in Table S1. Columns 3 and 4 present these results for the dataset with a cutoff date of 2021 December 31, and columns 5 and 6 present results for the dataset with a 2022 February 28 cutoff date. ***indicates significance at the 1% level.

While the coefficients and absolute numbers may seem small, only about 1 in 13 people who received the e-mail used telemedicine, it is important to notice two things. First, these estimates are computing the active use of the app, which requires not only downloading it but also having a medical need to use it. As described in Figure 2, the number of application downloads is several times higher than the use. Calculating the share of treated individuals that would have needed the system is more difficult. For example, data available for Argentina indicate that in 2021, 58% of people did not attend a physician's office, and 41% did not do it in 2022 (divided by sex, about one-third of women did not consult a physician, and almost one-half for men) (63).¹⁴ Only a share of those consultations can occur using telemedicine services (for example, most treatments for accidents and open wounds cannot occur online). Consequently, if we assume that about half of the people have a medical consultation in a year and only one-fourth have a consultation during a 6-month period, a back-of-the-envelope calculation would indicate that the significance of the results could quadruple.

Additionally, because of the way the randomization was performed—at the individual level instead of across locations—there could be spillovers and noncompliance of the control group. In the case of spillovers and noncompliance, individuals in the control group would receive information about the telemedicine service that they were not supposed to receive. As such, they would bias our results downward as individuals in the control group would adjust their behavior like those in the treatment group. As shown in Table 2, the effect at the end of the period was 6 pp. If we assume that half of the control group households who used telemedicine in the period were because of spillovers or noncompliance, the effect would be 7 pp instead. If we assume that all of the control group response is explained by spillovers or noncompliance, the effect would be 8 pp.

Finally, only about half of the households in the treatment opened the e-mail. To compute the e-mails' effect on those who opened them, we run an instrumental variable model of the variable Read, which indicates which households opened at least one e-mail using assignment to treatment T as the instrument.

Table S4 in the Supplementary material presents the results (the first stage is available in Table S3 in the Supplementary material). As expected, given the opening rate of e-mails, the results are now twice as large. For the individuals who opened at least one e-mail, the probability of using the service was 8.1, 9, and 11.9 pp larger than that of the control group at each data cutoff.¹⁵

Results in Table 2 report the likelihood that a household assigned to treatment used telemedicine at least once. Figure 4 shows the cumulative number of telemedicine calls and first-time users from the start of our treatment until the last cutoff date in February 2022. Because neither control nor treatment individuals had any experience with telemedicine at the beginning of the intervention, the lines start at zero. Soon after the intervention started, the trends started to diverge. By the end of 2021, the treatment group had 103 new first-time users compared to 24 in the control group. By the eighth month, the difference was 133 to 29, more than four-and-a-half times larger. As expected, the number of total calls has grown even more than the first use for both groups, which is compatible with users being able to evaluate better telemedicine after first use. Evidence on the use of telemedicine during the pandemic is also consistent with this pattern (13). First callers increased at the onset of the pandemic and fell afterward. Still, the number of calls remained stable, fueled by the repeated use of those who tried it. By the last cutoff date, the average user in the treatment group had made more than two calls and six times more calls than those in the control group.

Fig. 4.

Cumulative calls and first-time telemedicine users. This figure shows the number of cumulative calls and new mobile application users by control and treatment groups. The yellow lines (upper two lines) represent the demand growth of the treatment group, with the dashed line indicating the number of calls and the solid line indicating the number of first-time users. Similarly, the gray lines (lower two lines) represent the demand of the control group, with the dashed line indicating the number of calls and the solid line indicating the cumulative number of first-time users. The light blue panel (section highlighed on the left side of the figure) indicates the period of the communication campaign.

Given that the benefits of a telemedicine consultation outweigh the costs significantly, even a small increase in telemedicine use could have large welfare effects (see Table S6 in the Supplementary material for the cost–benefit analysis for the health system and Table S7 for the cost–benefit analysis for the individual). The case of the individual is very straightforward. For a medical consultation that can take place online effectively [about 67% of telemedicine medical consultations are resolved online and do not require in-person follow-up (13)], the in-person consultation implies travel costs and opportunity costs of traveling to the location and waiting. In terms of the costs for the health system, according to the health insurance company, they are also substantially lower using a telemedicine consultation. Plenty of costs are slightly higher in-person, such as medical consultation fees and administrative expenses, and several that are relevant for in-person and do not take place when the consultation is online. For example, a major cost highlighted by the health insurance company is the cost of no-shows and missed medical appointments. In the case of the United Kingdom, for which the data have been collected, missed appointments cost the National Health Service (NHS) more than 216 million pounds a year (65).

Given that the intervention took place in 2021, one potential question is whether the results would be similar during a nonpandemic period. A cursory look at the evidence seems to indicate that they are not pandemic specific. First, the sample is comprised of individuals who had access to telemedicine before the pandemic but, by July 2021, had not tried it, even after the e-mail campaigns designed by APSOT in 2020 took place. Second, the number of downloads following the e-mail campaign was much larger than the subsequent use in the near term, which seems to indicate that the e-mails were not sent during a peak pandemic period. Of course, this question remains open for future empirical investigation.

Conclusion

Telemedicine can increase access to health care, reduce healthcare costs, and expand service, particularly to geographically remote and underserved populations (66). The pandemic, the war in Ukraine, and recurring natural disasters have heightened its role to provide relief to strained healthcare systems, help meet increasing demand, and provide basic medical care when mobility or access to medical centers is restricted. The use of telemedicine, however, faces demand as well as technological restrictions. For instance, offering the service for free is not enough for widespread adoption. There are many reasons why demand is lower than it could be, but once households try the service, its use augments. Behavioral tools can help lower barriers to the service and nudge people into using it.

Even though we partnered with a health insurance company, we cannot estimate the impact of using telemedicine on in-person visits and health outcomes. For example, is telemedicine a substitute or complement to in-person visits? Does it improve health outcomes by providing more and easier access to doctors? These are still open questions that have not been answered by the literature or this paper. One of the main reasons those questions are hard to answer is that they cannot be answered using observational data. They require an experimental setting that manipulates access to telemedicine. Given that excluding individuals from a health service that has a very low marginal cost should be discouraged because of ethical concerns, the only internally valid option is to perform an exercise like the one we attempt here: In the context of a service that is available to everybody, encourage take-up of the service randomly. Only by having this “first stage” can the researcher focus on evaluating telemedicine's impact on in-person visits and health outcomes.

While we cannot proceed beyond the first stage because of power limitations (our sample size is relatively small and even more so the group of adopters to make relevant inferences about in-person visits), we show that a first stage is possible and draw attention to how it can be achieved. As such, this small (but first result on using messages for telemedicine adoption) contribution may provide the stepping stone upon which other researchers can build.

This paper shows that information campaigns can be successful in increasing telemedicine use, but not any campaign can. Some of the key aspects that telemedicine providers should take into account are the following: First, message campaigns should be tailored to the diagnostic of why people do not use the service. For example, people who have never used the system expressed concerns about the quality of the doctors and the service. Given that Llamando al Doctor has retained medical professionals from the best hospitals in the country and APSOT added their regular doctors to the rosters of telemedicine ones, highlighting this was important (and probably effective, at least according to anecdotal evidence). Second, the timing of the campaign is important to deal with some of the behavioral biases identified during the diagnostic, such as present and optimism biases. For example, timing the campaign to the seasonality of medical visits would be more effective. Third, there is plenty of evidence that the method of communication matters for increasing take-up (67). While we use e-mails in our intervention, some alternatives may be more effective. Finally, attaching telemedicine to regular services, such as starting medical attention with a triage done through telemedicine, could be useful for increasing the first usage. Of course, it should be done in a way that does not increase the cost of medical services and discourages seeking medical attention.

Ethics

The intervention was approved by the Universidad del Rosario's IRB (# 21420). A preanalysis plan was drafted at the time of submission to the IRB and is available upon request. All participants provided informed consent to APSOT to receive e-mails and other types of marketing communications. All individuals whose faces appear in the e-mail communications gave informed consent to APSOT or Llamando al Doctor for the use of the images or recording of the video.

Supplementary Material

Supplementary material is available at PNAS Nexus online.

Funding

Project costs were funded by the Inter-American Development Bank.

Author Contributions

M.P.G and C.S. conceived and conducted the experiments, M.P.G and C.S. analyzed the results. C.S. wrote and reviewed the manuscript.

Data Availability

All codes for data analysis to generate the results and all data generated for or used in this manuscript have been deposited and are publicly available here: https://doi.org/10.7910/DVN/QUEUEN.