A learning health system approach to COVID‐19 exposure notification system rollout

Abstract

Introduction

Digital exposure notification (EN) approaches may offer considerable advantages over traditional contact tracing in speed, scale, efficacy, and confidentiality in pandemic control. We applied the science of learning health systems to test the effect of framing and digital means, email vs Short Message Service (SMS), on EN adoption among patients of an academic health center.

Methods

We tested three communication approaches of the Apple and Google EN system in a rapid learning cycle involving 15 000 patients pseudorandomly assigned to three groups. The patients in the first group received a 284‐word email that presented EN as a tool that can help slow the spread. The patients in the second group received a 32‐word SMS that described EN as a new tool to help slow the spread (SlowTheSpreadSMS). Patients in the third group received a 47‐word SMS that depicted the system as a new digital tool that can empower them to protect their family and friends (EmpowerSMS). A brief four‐question anonymous survey of adoption was included in a reminder message sent 2 days after the initial outreach.

Results

One hundred and sixty people responded to the survey within 1 week: 2.33% from EmpowerSMS, 0.97% from SlowTheSpreadSMS, and 0.53% from emails; 29 (41.43%), 24 (41.38%), and 11 (34.38%) reported having adopted EN from each group, respectively. Patient reported barriers to adoption included iOS version incompatibility, privacy concerns, and low trust of government agencies or companies like Apple and Google. Patients recommended that healthcare systems play an active role in disseminating information about this tool. Patients also recommended advertising on social media and providing reassurance about privacy.

Conclusions

The EmpowerSMS resulted in relatively more survey responses. Both SMS groups had slightly higher, but not statistically significant EN adoption rates compared to email. Findings from the pilot not only informed operational decision‐making in our health system but also contributed to EN rollout planning in our State.

1. INTRODUCTION

Numerous efforts aim at using smartphones emitting randomized Bluetooth codes to notify people exposed to COVID‐19‐positive individuals. These digital exposure notification (EN) approaches may offer considerable advantages over traditional contact tracing in speed, scale, efficacy, and confidentiality in pandemic control. ¹ With privacy concerns being a significant barrier to the adoption of traditional contact tracing and EN, ² it is essential to effectively communicate with the public the purpose of contact tracing and why a digital EN tool is being provided. While government and technology companies have made EN tools available to the public, academic health centers, as hubs of innovation and learning health systems, ³ , ⁴ , ⁵ have an important role to play in improving the adoption of these tools. For this reason, UC San Diego Health, an academic health center with a strong history of digital innovation, ⁶ , ⁷ helped to lead the pilot of the EN tool for the State of California. This brief study reports a rapid‐cycle learning project testing the wording of messages, and the digital means with which they were sent, email vs Short Message Service (SMS), aimed at encouraging adoption among UC San Diego Health patients.

2. QUESTION OF INTEREST

Behavioral economics literature provides evidence that the framing of messages can influence behavior. ⁸ , ⁹ Based on evidence that many people feel powerless in the pandemic ¹⁰ , ¹¹ and that gain framing messages could promote risk‐aversion and preventive behaviors, ¹² we developed a message about EN as a new tool that empowers people to protect their family and friends. This empowerment‐framing message was expected to promote pro‐social and prevention behavior. ¹¹ , ¹³ , ¹⁴ We were interested in testing the effectiveness of the empowerment‐framing message in comparison with two other messages conveying EN as a tool for slowing the spread of COVID‐19, but without mentioning, it as an empowerment tool. We hypothesized that the empowerment‐framing SMS message would result in a higher rate of adoption of a digital EN tool among patients. We had a week to conduct the pilot project before contacting all 350 000 of our patients prior to the Thanksgiving Holiday when people were at higher risk of being exposed to COVID. Our intention was to apply the lessons learned from the pilot to facilitate the rollout of EN to our entire patient population and then facilitate EN adoption in California.

3. METHODS

Commonly known as A/B testing, ¹⁵ the pilot project took place at the UC San Diego Health (UCSDH) System. We tested three communication approaches of the Apple and Google EN tool (CA Notify) in our patient population in a rapid learning cycle. ³ The Institutional Review Board certified the project as not qualifying as human subject research.

3.1. Patient populations and quasi‐experimental assignments in pilot

We contacted 15 000 UCSDH patients for the initial EN pilot on November 17, 2020, followed by a reminder and a survey on November 19, 2020.

Patients were eligible and automatically included on the active patient registry if they had a face‐to‐face encounter with a UC San Diego Health provider in the past 36 months, including televideo visits, age 18 years or older. The 15 000 patients in the pilot were initially informed of the EN tool in three separate approaches after their medical records number (MRN) were pseudorandomly ³ assigned to three groups. The first group included the first 6000 patients with an even MRN. The second group included the first 6000 patients with an odd MRN. A third group included the next 1500 patients with even MRNs and the next 1500 patients with odd MRNs, after the first two groups had been formed. Patients in the first group received an email that contained 284 words and presented CA Notify as a tool that can help slow the spread. The patients in the second group received a 32‐word Short Message Service (SMS) that described the EN system as a new tool to help slow the spread (SlowTheSpreadSMS). The patients in the third group received an SMS with 47 words that described the system as a new digital tool that can empower them to protect their family and friends (EmpowerSMS) (see Appendix S1 for each message.)

3.2. Patient survey

A four‐question anonymous survey was sent to all 15 000 patients in the pilot as part of a reminder message 2 days later, on November 19, 2020. The same communication approach as the initial outreach was used, that is, email reminder to the email group and SMS reminders to the two SMS groups. The survey asked whether they had adopted the EN tool and recommended the tool to other people, reasons for non‐adoption if they had not adopted it, and their recommendations for how UCSDH could improve its communication about the tool so more people would adopt CA Notify.

3.3. Analysis

Rates of adoption were tabulated from survey responses in each group and compared across the three groups using logistic regression analysis. Free text responses to reasons for non‐adoption and recommendations for improvements were coded and analyzed iteratively and thematically. ¹⁶

4. RESULTS

4.1. Adoption rates

From 15 000 reminders with invitation to survey, 160 people responded within 1 week, 2.33% from EmpowerSMS, 0.97% from SlowTheSpreadSMS, and 0.53% from the email group. Among the respondents, 29 (41.43%), 24 (41.38%), and 11 (34.38%) reported having adopted CA Notify from each group, respectively (Table 1). These rates were not statistically different from each other. On average, 34% had recommended the tool to others (not tabulated).

TABLE 1.

Adoption rates among pilot participants

	N of messages Sent	N of survey responses	Response rate	Reported EN adoption
				Yes: N (%)	No: N (%)
EmpowerSMS	3000	70	2.33%	29 (41.43%)	41 (58.57%)
SlowTheSpreadSMS	6000	58	0.97%	24 (41.38%)	34 (58.62%)
Email	6000	32	0.55%	11 (33.33%)	21 (66.67%)
Total	15 000	160		64 (40.00%)	96 (60.00%)

Data suggested that 32 patient respondents reported getting an error message that the tool was not available in California after they tried to activate the tool (see Section 4.2 for more descriptions.) This finding implied that these patients had attempted to activate the tool but were not able to complete the task because their iPhones had older iOS versions. Therefore, the potential adoption rates would have been considerably higher than the observed adoption rates had the EN tool been accessible to iPhones running on older iOS versions.

4.2. Appreciations and barriers to adoption

Some patients expressed appreciations for being informed of the EN tool by us. For example, one patient wrote, “You found and influenced me. I appreciate that.” As mentioned above, answers to the question on reasons for non‐adoption suggested that many patients had tried to turn on CA Notify but got an error message that the program was not available in California. We learned that the error was because their iPhones did not have iOS14.2, which was required at the time of the pilot. Besides the iOS version incompatibility, reasons for non‐adoption included privacy concerns, low trust of government or Apple and Google, lack of a smartphone, help need, not receiving initial outreach, not knowing it was available, and misunderstandings about having their location tracked by EN (Table 2).

TABLE 2.

Reasons for non‐adoption

Reasons	Patient response examples
Technical challenge	“I WANT to participate! But the technology is not working for me. Very frustrating.”
Privacy concern	“Too invasive of my privacy.” “Serious privacy and big brother concerns. This is nose under the tent for our future hope of privacy.”
Lack smartphone	“I don't have a smartphone.”
Unaware that it existed	“Did not know about it.” “Never heard of it.”
Need help	“How to do it??”
Misunderstanding	“I am not going to have my whereabouts traced for silent reporting.”
Distrust	“I don't trust Cal government.” “I have no reason to TRUST the government.”

4.3. Communication improvement recommendations

Fifty‐three patients provided communication improvement recommendations. It is noteworthy that eight patients recommended that healthcare systems and healthcare providers play an active role in disseminating information about the EN tool and encouraging their patients directly. Table 3 summarizes three themes that have emerged from the recommendations: build on existing relationship between patients and healthcare providers, use multimedia outreach, and facilitate.

TABLE 3.

Communication improvement ideas from patients

1. Build on existing relationship between patients and healthcare providers:
a. All health systems/hospitals in California reach out to their patients and invite them to adopt	“Have all hospitals contact patients.”
b. Reassure patients about privacy protection	“I think the text message suggestion needs to include reassurance about privacy. I think you could simply add this line from your website: ‘Your identity will never be revealed, and your location will never be tracked.’”
c. Health systems include adoption messaging in checkout papers or online portals (MyChart) after visits in clinic	“Put announcement about this app on the checkout papers that patients received after seeing their doctors and so forth.”
2. Use multimedia outreach:
d. Apple and Google send automated messages	“Would Apple & Google cooperate in sending automated messages to all phone owners?”
e. Use traditional and social media: Facebook, Instagram, TV, and newspaper	“Remind and remind again: Keep publicizing it, gamify, post adoption stats online” “Just keep sending people texts and advertise on social media.” “Expand access to the tool beyond UCSD patients. Perhaps social media campaign to get the word out?”
3. Facilitate
f. Highlight positive outcomes	“Highlighting positive outcomes would help.”
g. Enable people to go online to add their phone number to CANotify	“Work on the site for recording phone numbers.”
h. Collaborate with local public health agencies	“Work with local health authorities?”

5. DISCUSSION

The EmpowerSMS group had a slightly higher adoption rate than the SlowTheSpreadSMS group and the email group. The differences were not statistically significant, however. A relatively higher proportion of patients in the EmpowerSMS group responded to our survey, compared to those in the other groups. It is plausible that the empowerment gain frame had nudged more patients to interact with our survey.

5.1. Encouraging adoption rate

The results from 160 survey respondents suggested that 41% had adopted EN after receiving an empowerment‐framing SMS, with more people intended to adopt but had encountered difficulties due to iOS versions. Furthermore, 34% of those who had adopted the tool recommended it to others. Because EN protects user privacy while serving a critical public health purpose, promising messaging and operational approaches to mitigate privacy concerns and technical barriers could help slow the spread of COVID‐19.

If a significant proportion of the population adopt digital exposure notification, it can be a powerful additional tool among other public health practices such as facial masking, handwashing, physical distancing, and regular decontamination. ¹ Moreover, EN could approach instantaneous contact tracing if a user who tested positive immediately sends out an anonymous notification to other users through the EN tool. Therefore, it could significantly reduce the number of days to isolation and contact quarantine as compared to traditional contact tracing. ¹

5.2. Rapid‐cycle learning and application of strategies for full rollout among UCSDH patients

Results and lessons learned from this pilot were considered by UCSDH's patient communications team for the full rollout of CA Notify to over 350 000 active patients. Patients were sent the email message in the morning and the EmpowerSMS in the afternoon on November 23, 2020, before the Thanksgiving Holiday. One week later, an EmpowerSMS reminder to these 350 000 patients and the 15 000 patients in the pilot test was sent on November 30, 2020. The last patient suggestion noted in Section 4.2 resonated so well with us that our communications team adopted it in subsequent communications about EN to the entire UCSDH patient population.

We have taken a continuous and ongoing learning and improvement approach. To reduce the probability that potential users' iPhones had an older version of the iOS, we added iOS detection to the website on November 20, 2020. If an earlier iOS was detected, an alert was displayed at the top of the web page to encourage the user to upgrade to iOS 14.2. On December 14, 2020, Apple relaxed its requirement to iOS 12.5 to enable more people to adopt CA Notify.

5.3. Application of strategies for state‐wide rollout in California

Patient responses to the opportunity to adopt EN have helped us understand facilitators and barriers to this important public health tool. The findings of the quasi‐experiment pilot directly informed operational decisions not only for the rollout of EN to all our patients but also for the state‐wide CA Notify rollout. For example, we were mindful about the findings that mistrust in government and large private companies could result in resistance to adoption whereas trust in healthcare providers could facilitate adoption. ¹⁷ Development of our state‐wide rollout communication strategies (http://CANotify.ca.gov) took into account the lessons learned from our pilot. Our project team provided a toolkit to healthcare providers via the California Department of Public Health, California Medical Association, and California Hospital Association.

5.4. Limitations

A low number of responses from patients were received making the findings preliminary. The decision to keep the survey anonymous and brief confined the learning to what is available from the survey alone. Furthermore, the contracted digital health firm was not able to provide us with data on message open rates, which limited our ability to measure the more immediate indicator of the effectiveness of our approaches, that is, getting the recipient to read our message and go to our website. Future outreach efforts should strengthen use data capturing while preserving user privacy. We further acknowledge that the reliance on smartphones excludes those who do not own smartphones and those who own smartphones but do not have the technical ability to opt in. A technical support phone line provides assistance to callers in English and Spanish. For users whose smartphones were too old to access the EN system at the time of the pilot, they were able to access EN after December 14, 2020, if their phones use iOS12.5 or later. Relatedly, EN is offered in 13 languages so that language barrier to EN is less likely. Lastly, issues related to civil liberties and corporate or government power associated with digital EN are important but out of scope for this brief research article.

5.5. Conclusion

This pilot was designed to allow us to rapidly incorporate real patient feedback into the way we communicated the use and benefits of the digital EN tool. Applying the science of learning health systems ⁵ through theory‐guided rapid‐cycle testing and dissemination of promising communication approaches, the pilot project team not only informed fast‐paced operational decision‐making in our health system but also contributed to this important public health effort in California.

CONFLICT OF INTEREST

Christopher Longhurst and Nicole May had a contract with the California Department of Public Health to support exposure notification for California.

Supporting information

Appendix S1. Supporting information.

Tai‐Seale M, May N, Sitapati A, Longhurst CA. A learning health system approach to COVID‐19 exposure notification system rollout. Learn Health Sys. 2022;6(2):e10290. doi: 10.1002/lrh2.10290