Abstract
Text messages used by healthcare organizations to communicate with patients have known limitations for certain populations, especially older adults. This study analyzed text message interactions with over 17 000 patients aged 65 and older during the initial phase of the COVID-19 vaccination campaign. We coded the responses of 4247 patients who responded to this outreach to understand issues they experienced with the text message system. Our analysis highlighted areas for technology improvement and the need for more robust strategies to effectively reach older populations.
Keywords: text messaging, COVID-19, community outreach, aged, vaccine
INTRODUCTION
Communicating health information to patients by text message has been shown to be effective.1,2 During the COVID-19 vaccine rollout, healthcare facilities, pharmacies, and other organizations used text messages to inform patients of appointment availability. While this method may be convenient for many of the millions of mobile phone users in the United States, it may not work well for older populations who may have less experience receiving scripted text message-based communications.3,4 In a time when older adults are at highest risk for poor outcomes from COVID-19, the inability to communicate desired intent for a vaccine can pose serious health risks.5 The barriers that limit older populations from using digital healthcare resources have been characterized in prior studies, such as insufficient technological skills or concerns for the amount of time it will take to effectively use it.6 The specific technical issues faced by older populations while interacting with automated text messaging systems are not currently well characterized in the literature. We analyzed thousands of text messages sent to patients at our health system aged 65 and older to understand issues they experienced with text message outreach and provide information to improve future outreach initiatives.
METHODS
A large East coast healthcare system sent 17 736 patients who were residents of Washington, DC and aged 65 and older invitations to schedule a COVID-19 vaccine through an automated text message service during March–April of 2021. The text messaging service used was a large-scale cloud-based system that offered an online portal for customized patient engagement messages over SMS. The vendor product was used for a previous patient outreach at a smaller scale.
The invitation message read: “[Health System] has COVID-19 vaccines for eligible patients. To schedule online, reply [A]. To receive a call, reply [G]. If you are not interested, reply [D]. Please respond within 12 hours.” (The response option letters in the message were chosen from a set of letter groups and assigned at the time of delivery based on the text message service’s automated batch-processing system, see Figure 1.) For the automated service to recognize a response and trigger follow-up messages, it must have been received within 8 days of the invitation and one of the designated response letters must have been either the only character in the return message or the first character followed by a space or punctuation. Patient responses that did not follow this structure, called non-conforming messages, required manual review to interpret.
Figure 1.
The message logic of the automated text message service. Text message responses containing the appropriate terms triggered subsequent messages as shown in the flow chart.
In this study, we analyzed the content of each patient’s responses to the vaccine invitation in entirety. First, each set of responses (termed “conversation”) was analyzed to determine the patient’s “vaccine intent.” They were then grouped into having “interest” in the vaccine, “non-interest,” or expressing both interest and non-interest (termed “mixed”) at some point in the conversation. Next, patients’ conversations were analyzed to determine if they sent all conforming messages, all non-conforming messages, or a combination of conforming and non-conforming messages. Conversations including non-conforming messages were further analyzed to determine if the patient was ultimately able to communicate their vaccine intent with the text message system. Patients whose intent was accurately recognized by the system were coded as having a “recognized intent” and those whose intent was not recognized were coded as having an “unrecognized intent.” Lastly, conversations including non-conforming messages were analyzed to determine which technology issues each patient faced while interacting with the text message system (see Table 1). A scheme of technology issues and criteria were jointly developed by 3 human factors researchers (KG, RR, and RD). KG coded the conversations with the 4247 patients (consisting of 16 323 patient-generated text messages) that replied to the vaccine invitation, with group discussion to resolve any ambiguous messages. Patients who responded in a language other than English were excluded from the study as the research team was not able to interpret these messages. Similarly, patients with uninterpretable vaccine interest were excluded from detailed analysis.
Table 1.
Types of text message issues
| Non-conforming messages | Definition | Example |
|---|---|---|
| Late response | Message not received in the 8-day response window. | Not applicable |
| Typo | Message included a misspelling or wrong character(s). | “G” followed by “Sorry I sent wrong response D is the response” |
| Seeking assistance | Message included a request for help, information, or confirmation about the vaccine appointment. | “Please show me what website I should go to” |
| Extra information | Message included extra information about their vaccination status, contact information, or individual needs. | “Already got the 1st shot” |
| Wrong format | Message did not specify their response to the message prompt using a letter or did not use the letter in the correct way. | “Reply G” or “Yes I’m interested please send link” |
| Miscellaneous | Message was either unrelated to the vaccine prompt or unintelligible. | “Never told them a cover app.” |
| Unsupported message function | Message involved a text message feature not supported by the system, such as auto-reply or a message “reaction.” | “[Auto-Reply] I’m driving right now—I’ll get back to you later.” or “Liked ‘Thank you. We will update our records’.” |
Two separate analyses were performed in this study. The first analysis focused on patients who expressed interest in the vaccine at any point in the conversation (including the interested and mixed vaccine intent) to determine the fraction of these patients with unrecognized intent and which technology issues they faced. For each patient with an unrecognized intent, we extracted the specific technology issue which led to their intent being unrecognized. The second analysis focused on all patients who faced technology issues, regardless of their intent and whether it was recognized, to determine the most frequent technology issues they experienced.
The study was approved by MedStar Health Research Institute’s Institutional Review Board and patient consent was waived as the data involved de-identified secondary analysis of existing data with no risk to subjects.
RESULTS
In our dataset of text message interactions with patients of a large East coast healthcare system, we identified 17 736 patients over age 65 who were invited to schedule a COVID-19 vaccine through an automated text message service during March–April 2021. Of the 17 736 patients that were sent an invitation message, 4247 (23.9%) patients responded. Forty-four patients (33 with non-English responses and 11 with uninterpretable responses) were excluded from detailed analysis. A total of 4203 patients had interpretable vaccine intent and were included in the analysis.
Vaccine intent issues
Of the 4203 patients who had an interpretable vaccine intent, 1768 (42.1%) were interested, 2252 (53.6%) were not interested, and 183 (4.3%) expressed both interest and non-interest (termed “mixed”). Of 1951 patients that expressed interest in the vaccine at any point in the conversation (including “mixed” patients), 1805 (92.5%) of these patients had their intent recognized by the system while 146 (7.5%) did not. The issues that led to unrecognized responses included late response (76 of 146 patients, 52.0%), wrong format (61 of 146, 41.8%), and seeking assistance (9 of 146, 6.2%) (see Table 1 for types of text message issues). These results are summarized in Figure 2.
Figure 2.
Breakdown of patients’ vaccine intent and the issues that they faced in communicating their interest in the vaccine.
General technology issues
Of the 4203 patients who expressed a vaccine intent, 1175 patients (28.0%) had at least one message that did not conform to the message protocol. The most frequent issue types were wrong format of the message (693 patients, 60.0%) and extra information in the message (683 patients, 58.1%). Of these 1175 patients who experienced issues, 767 (65.3%) had their intent recognized by the system and 408 (34.7%) did not. The most prevalent issues for patients who did not have their intent recognized were wrong format of their message (298 of 408, 73.0%) and late response (104 of 408, 25.5%). The most common issue for the patients that were able to express their intent was extra information in the message (683 of 767, 89.0%). Finally, while 96 patients with recognized intent (12.5% of 767) attempted to seek additional assistance, only 9 patients with unrecognized intent (2.2% of 408) did so (Supplementary Table S1).
DISCUSSION
The text message intervention in this study showed clear gaps in outreach to older patients, with 28.0% of all patients who responded having a technical issue. Further, 7.5% of patients interested in the vaccine were unable to express this interest due to sending a late response, responding in the wrong format, or seeking assistance. These issues may have been due to a variety of reasons that warrant further research including system design, patient preferences, and technology use behavior, and other cultural or demographic factors, such as language.
A previous literature review found that older individuals commonly experienced technical issues while interacting with mobile health applications.6 Additionally, older individuals cited prior technical issues and concerns about their ability to use the technology as barriers to using mobile health applications. These results align with the prevalence of technical issues in our study, such as responding in the wrong format or seeking assistance while using the automated text messaging service. Another prior study found that 75.9% of older adults owned a cell phone, while only 59.6% of individuals ages 65–69 and 45% of those 85 and older used email or text messaging on most days.4 Infrequent use of text messaging was also displayed in our study with common late responses. Our study expanded upon prior research by identifying the specific issues faced by older populations while interacting with an automated text messaging system, highlighting areas for improvement to the broader vaccine outreach strategy for older adults.
Limitations to this study include inaccuracies in patient records that may have led to text messages sent to people that they were not intended for, analysis only of issues for patients who responded to the invitation prompt, and the interpretation of vaccine intent based solely on the messages received from the patient.
In conclusion, older individuals frequently faced technical issues while interacting with the automated text message systems in this study, most commonly by sending a late response, responding in the wrong format, adding extra information, or seeking assistance with the technology. Using automated text message systems that can process natural language and support patient responses across a variety of formats and timeframes may address some of the issues experienced by older patients. Alternatively, systems could provide feedback to users based on non-conforming message types to support their understanding of outreach interventions. Outreach efforts should also include a tiered approach in which those that do not respond to the text messages receive a phone call or conversation at their next health appointment encounter. As telehealth visits, mobile health applications, and patient portals are becoming commonplace, it is pertinent that these technologies are usable for older populations to narrow the digital divide.
Supplementary Material
ACKNOWLEDGEMENTS
The authors wish to acknowledge David Bauer and Christian Boxley for their support in processing the text message data.
Contributor Information
Kaelyn Gwynne, Georgetown University School of Medicine, Washington, District of Columbia, USA.
Raj Ratwani, Georgetown University School of Medicine, Washington, District of Columbia, USA; MedStar Health National Center for Human Factors in Healthcare, Hyattsville, Maryland, USA.
Ram Dixit, MedStar Health National Center for Human Factors in Healthcare, Hyattsville, Maryland, USA.
FUNDING
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
AUTHOR CONTRIBUTIONS
All authors (KG, RR, and RD) contributed to the study conceived and designed the analysis. KG and RD collected and coded the data. All authors jointly performed the analysis and wrote the paper.
SUPPLEMENTARY MATERIAL
Supplementary material is available at JAMIA Open online.
CONFLICT OF INTEREST STATEMENT
The authors declare no competing financial or non-financial interests.
DATA AVAILABILITY
The data underlying this article are not publicly available due to them containing information that could compromise patient privacy. The data will be shared on reasonable request to the corresponding author.
REFERENCES
- 1. Krishna S, Boren SA, Balas EA. Healthcare via cell phones: a systematic review. Telemed J E Health 2009; 15 (3): 231–40. [DOI] [PubMed] [Google Scholar]
- 2. Henrikson NB, Zhu W, Baba L, et al. Outreach and reminders to improve human papillomavirus vaccination in an integrated primary care system. Clin Pediatr (Phila) 2018; 57 (13): 1523–31. [DOI] [PubMed] [Google Scholar]
- 3. Pew Research Center Internet and Technology- Mobile Fact Sheet. 2021. https://www.pewresearch.org/internet/fact-sheet/mobile/. Accessed July 2, 2021
- 4. Gell NM, Rosenberg DE, Demiris G, et al. Patterns of technology use among older adults with and without disabilities. Gerontologist 2015; 55 (3): 412–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Shahid Z, Kalayanamitra R, McClafferty B, et al. COVID‐19 and older adults: what we know. J Am Geriatr Soc 2020; 68 (5): 926–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ahmad NA, Mat Ludin AF, Shahar S, et al. Willingness, perceived barriers and motivators in adopting mobile applications for health-related interventions among older adults: a scoping review. BMJ Open 2022; 12 (3): e054561. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data underlying this article are not publicly available due to them containing information that could compromise patient privacy. The data will be shared on reasonable request to the corresponding author.