Retaining Homeless Veterans in Outpatient Care: A Pilot Study of Mobile Phone Text Message Appointment Reminders

Abstract

Objectives. We examined the feasibility of using mobile phone text messaging with homeless veterans to increase their engagement in care and reduce appointment no-shows.

Methods. We sent 2 text message reminders to participants (n = 20) before each of their outpatient appointments at an urban Veterans Affairs medical center. Evaluation included pre- and postsurvey questionnaires, open-ended questions, and review of medical records. We estimated costs and savings of large-scale implementation.

Results. Participants were satisfied with the text-messaging intervention, had very few technical difficulties, and were interested in continuing. Patient-cancelled visits and no-shows trended downward from 53 to 37 and from 31 to 25, respectively. Participants also experienced a statistically significant reduction in emergency department visits, from 15 to 5 (difference of 10; 95% confidence interval [CI]  = 2.2, 17.8; P = .01), and a borderline significant reduction in hospitalizations, from 3 to 0 (difference of 3; 95% CI = −0.4, 6.4; P = .08).

Conclusions. Text message reminders are a feasible means of reaching homeless veterans, and users consider it acceptable and useful. Implementation may reduce missed visits and emergency department use, and thus produce substantial cost savings.

Homeless persons are sicker and die younger than those with stable housing.^1–5 Despite poor health, homeless persons underutilize primary care. They use emergency department (ED) care 3 times more often⁶ and are hospitalized 4 times more often than are those in the general US population.^7,8 Homeless veterans, who number 62 000,⁹ are in similarly poor health.¹⁰ Despite substantial outreach efforts,¹¹ rates of missed medical appointments (“no-shows”) among homeless persons are high^12,13; for example, a study of homeless patients with tuberculosis reported 47% missed a first follow-up appointment.¹⁴ Studies of causes of appointment no-shows indicate that forgetting about the appointment is the most commonly cited reason (19%) followed by being unaware there was an appointment (15%).¹⁵

Outside the Veterans Affairs (VA) homeless population, a promising means of improving patient engagement with outpatient care has been through mobile phone text messaging, for example for appointment keeping¹⁶ and getting vaccinated.¹⁷ It is not known, however, how well text messaging would work with homeless populations. To our knowledge, this pilot study represents the first step in addressing this knowledge gap. We assessed the feasibility of using mobile phone text messaging to remind homeless veterans of their outpatient appointments. There is growing evidence that homeless persons increasingly have mobile phones. Even among a sample of street homeless, 44% had mobile phones in 2009.¹⁸ A review of the literature on technology use by homeless persons found that mobile phone ownership ranged from 44% to 62% and computer access and use ranged from 47% to 55%,¹⁹ and a recent study of young adult homeless found that 93% used Internet technology (e-mail, Internet, computer, or social networking) at least weekly.²⁰

We developed a text-messaging intervention called Texting Intervention for Linkage and Engagement and piloted it in an urban VA homeless-oriented primary care clinic in preparation for a larger-scale trial.²¹ Pilot testing in this population is especially important, as their lives, compared with those of most study populations, include more complexities, such as unpredictable daily schedules, many competing needs, access to fewer communication channels, and high prevalence of mental health conditions and substance use disorders. We conducted the pilot study to evaluate whether text messaging is a feasible, effective, and acceptable means of reaching homeless veterans and to assess the suitability and ease of collection of our process and outcome measures. We investigated whether the intervention merits a larger study.^22–24 If text messaging is found to be feasible and effective, it would provide a low-cost, efficient means of reaching many veterans who are homeless—increasing their engagement in outpatient care and potentially improving their health and reducing their utilization of expensive ED and hospital care.

METHODS

To be eligible, veterans had to be receiving care at the homeless primary care clinic at the Providence VA Medical Center, Rhode Island; own a mobile phone; have the capability and knowledge to use text messaging; and be able to speak and read English. We recruited patients between February and May 2013 at the medical center’s homeless-oriented clinic.

After we completed the informed consent process, a research assistant administered a baseline survey questionnaire to the participants, which drew items from a questionnaire developed for use with homeless populations.¹⁸ Questions covered sociodemographic characteristics, mobile phone practices (e.g., frequency of use, cost), reasons for making mobile phone calls and for sending text messages, barriers to mobile phone use (e.g., running out of minutes), and attitudes toward health care providers reaching out to homeless veterans, via mobile phone calls or texts, if they had not been seen in the clinic in a long time (e.g., 1–2 years).

After the 8-week intervention period, the research assistant administered a follow-up survey questionnaire and conducted a qualitative semistructured interview, which was audio recorded. We designed the semistructured interview to understand the experience of the participants in receiving the Texting Intervention for Linkage and Engagement and to evaluate usability and usefulness. Usability questions, as determined by Nielsen’s heuristic evaluation,²⁵ assessed the intervention for use of simple and natural dialogue, speaking the user’s language, and minimizing user memory load. In addition we asked respondents about privacy and confidentiality concerns and their attitudes toward expanding the intervention to include text message reminders about medication taking. Participants received a $15 store voucher for completing the baseline survey and a $25 store voucher for completing the follow-up survey and semistructured interview.

Intervention

To make our intervention as user centered as possible, we followed principles Fogg outlined in his book Persuasive Technologies.^25,26 First, we used simple tools with brief messages. The high attrition rates with Web-based interventions have often been attributed to their complexity.^27,28 Second we selected tools that fit into daily routines because they are more likely to be used. Text messages are an unobtrusive means to connect with patients, and mobile phones are one of the few communication tools that homeless persons can safely and easily keep.¹⁹ Third, we selected tools that simplify tasks because they are more likely to be used. Getting a reminder about an appointment and storing it on a phone makes it easier to get to the appointment. Forgetting appointments is the main reason for no-shows in the VA.²⁹

We administered the intervention through Web-based text-messaging software called MessageMedia (http://www.message-media.com), which enables the creation, delivery, and management of text messages sent to mobile phone numbers from a computer terminal. We created a registry of participating patients with their mobile phone numbers and created a template appointment reminder message. Participants were sent 2 text message appointment reminders on a schedule of 5 days and 2 days before their appointment. Daily, the research assistant reviewed participants’ appointment schedules in the VA electronic health record to identify appointments that needed either the 5-day or 2-day reminder, modified the message template for the appropriate date and time of the appointment, and sent participants the text messages. An example text message reads as follows, “Remember: Friday May 24 at 8:30 AM you have an appointment at Providence VA. If you have questions or to cancel call 401-273-xxxx. Thanks.” The appointments for which reminders were sent included primary care, specialty care, and scheduled laboratory visits and procedures (e.g., magnetic resonance imaging). We excluded telephone appointments, home care visits, walk-in visits, and visits that occurred daily at the same time (e.g., for methadone treatments).

Measures

We assessed user experience with the intervention through a qualitative interview, following a semistructured interview guide with 15 questions (data available as a supplement to the online version of this article at http://www.ajph.org). The guide also included additional probes that the interviewer could use, as needed, for clarification or elaboration.

Three items asked broadly about participant experience with the intervention (e.g., What was it like receiving text message reminders?). Three items addressed issues of simple or natural language and employing the user’s language (e.g., What did you think of the way the text messages were worded?). One item addressed minimizing user’s memory load (e.g., Did the text message help you remember to go to an appointment?).

We assessed usefulness with 5 items (e.g., Did the texts make any difference in how you planned for getting to appointments? If yes, how?). We assessed privacy and confidentiality with a single item in the interview guide (e.g., Did you worry about invasion of privacy or confidentiality with the texts?). We assessed overall satisfaction with the intervention with 1 item and a probe (e.g., If you could continue to receive text message reminders, would you want them? Why or why not?). Two items asked about adding features, including patients being able to respond to text messages and the VA using text messages to remind patients to take their medications. We confirmed delivery of text messages to recipients through examination of a MessageMedia log file accessible to the research team online.

We abstracted medical record data from the VA medical center electronic health record. For each participant, we abstracted outpatient appointments attended, outpatient appointments cancelled by patient, missed outpatient appointments (patient “no-show”), ED use, and hospitalizations.

Data Analysis

We downloaded interview data into NVivo version 10 (Qualitative Solutions and Research, Doncaster, Australia) qualitative data analysis software for coding and analysis. Two investigators with qualitative research experience conducted a thematic analysis of the semistructured interview transcripts. We conducted a close reading of transcripts for coding purposes guided by themes of usability, usefulness, privacy and confidentiality, and satisfaction that had informed the development of the interview guide. We then reviewed the coded data and discussed any areas of disagreement or lack of clarity until we reached consensus on the coding.

We entered survey questionnaire data into an Excel version 14.0 (Microsoft, Redmond, WA) spreadsheet and imported data into SAS version 9.3 (SAS Institute, Cary, NC). We stored utilization data abstracted from the electronic health record on a spreadsheet to enable analysis of attendance, no-shows, ED use, and hospitalizations. For each participant we defined the preperiod as the 8 weeks preceding the date of enrollment in the study. The postperiod was the 8 weeks in which participants were receiving text message appointment reminders, starting with the study enrollment date. We obtained summary measures (e.g., counts, frequencies, means) on the participant characteristics. We assessed statistical significance of differences in the outcome measures using 2-sided t tests, with P < .05 considered significant. We also estimated confidence intervals (CIs) around the differences.

Cost Analysis

We used an average cost approach³⁰ to estimate cost savings owing to the intervention if it were expanded to target homeless veterans throughout the national Veterans Health Administration (VHA) health care system. We inflation-adjusted costs to 2013 US dollars. Our estimate of the cost of an unused appointment (for estimating costs of patient cancellations and no-shows), $198, comes from the Department of Veterans Affairs Office of the Inspector General.²⁹ We adjusted our estimate of total number of patient cancellations downward by multiplying it by 0.4, because the VHA estimates that 60% of patient-cancelled visits are filled by another patient.²⁹ For ED visits we used the average cost of an ED visit in the VHA, of $791, as reported by VanKirk et al.³¹ We did not include hospital stays in the cost analyses because there were so few and they would likely lead to skewed cost estimates. We grounded the estimated cost of the VHA sending text messaging on a conservative estimate of 12 cents per text sent to patients, drawn from a study of a text-messaging system implemented in a single hospital department.³²

Although a research assistant sent text messages in this pilot study because of the small scale, we did not include research assistant costs because if text messaging were implemented hospital- or health system–wide an automated system would be used. The costs to veterans of receiving text messages are on the basis of 10 cents per message received³³—a conservative estimate because many homeless and low-income persons use free phones supported through US government programs that include up to 1000 free text messages per month.³⁴ To estimate the reach, and thus the total savings and total costs, we assumed 15 500 homeless veteran participants. This is on the basis of the count of 62 000 homeless veterans on a single January night in 2013, reduced by half (to 31 000) on the assumption that many of these veterans are not connected to VHA care or would decline to participate and then reduced by half again (to 15 500) because of published estimates that approximately half of street homeless persons have mobile phones.^18,35

Because of our small sample size, we calculated a range of possible cost savings to reflect the wide CIs around our point estimates for reductions in utilization. The CIs around the estimates for reductions in patient cancellations and no-shows included negative numbers; however, we set the negative CI bound for no-shows to zero because it was unlikely that the intervention would lead to more no-shows.

RESULTS

We enrolled 21 homeless veterans who used the Providence VA homeless primary care clinic. One participant dropped out before receiving any text messages; thus we have demographic data on 21 participants but outcome results on only 20. Most participants were male (81%) and White (62%), with a mean age of 55 years (range = 25–68; Table 1). Most (85%) had 1 or more chronic medical conditions, 80% had a mental health condition, and 55% had a substance use disorder. The most common medical, mental health, and substance use problems were, respectively, arthritis or degenerative joint disease (55%), depression (75%), and problem alcohol use (40%; data not shown).

TABLE 1—

Veteran Characteristics (n = 21): Providence, RI, 2013

Characteristic	No. (%)
Gender
Male	17 (81)
Female	3 (14)
Transgender	1 (5)
Age, y
< 40	3 (14)
40–49	2 (10)
50–59	12 (57)
60–69	4 (19)
Hispanic
Yes	2 (10)
No	19 (91)
Racea
Asian, American Indian, or Pacific Islander	4 (19)
Black or African American	4 (19)
White	13 (62)
Marital status
Married or living together	3 (14)
Separated	2 (10)
Widowed	3 (14)
Divorced	8 (38)
Single or never married	5 (24)
Education, highest level
≤ high school	1 (5)
High school degree or GED	8 (38)
2 y of college	11 (52)
4 y college graduate	1 (5)
Annual household income, $
< 6000	4 (19)
6000–11 999	9 (43)
12 000–24 999	6 (29)
≥ 25 000	2 (10)
Health status, self-report
Excellent	2 (10)
Very good	0 (0)
Good	6 (29)
Fair	10 (48)
Poor	3 (14)
Chronic conditionsb
Medical	17 (85)
Mental health	16 (80)
Substance use disorder	11 (55)
Military service
Post-Korean	1 (5)
Vietnam era (1964–75)	4 (19)
Post-Vietnam era (1975–1990)	12 (57)
Persian Gulf era (1990–2001)	1 (5)
Post–September 11, 2001 era	3 (14)
Where living
Own house or apartment	7 (33)
Friend or relative	9 (43)
Transitional housing	2 (10)
Motel	1 (5)
Emergency shelter	1 (5)
Car or street	1 (5)

Note. GED = general equivalency diploma.

^aPercentages do not add to 100 because of rounding.

^bPercentages do not add to 100 because respondents could report more than 1 chronic condition.

Usability and Usefulness

Participants found it easy to use the texting intervention. They found the text message wording clear and did not mind that the specific clinic or doctor to be seen was not mentioned in the text. Only 1 respondent reported occasionally having difficulty finding the messages again in his mobile phone in-box when he wanted to refer back to them. Participants perceived text messages as a valuable aid to fallible memories.

Participants liked having the appointment information in their mobile phone. Not only were messages helpful reminders, but the text message in-box also served as a recordkeeping system. “You know if I didn’t have this I’d be walking around with a lot of papers in my pocket,” reported 1 respondent. They felt the text messages helped them keep appointments and reduce the number that they forgot about or were not even aware that they had scheduled. There was strong interest in having the text-messaging intervention continue.

Participants noted they had no privacy concerns because the information sent via text (date and time of appointment and a phone number to call if needed) did not contain sensitive information. There appeared to be high satisfaction with the text-messaging intervention, with all but 1 indicating they would like to continue receiving text messages. Only 1 participant was unable, for a 3-week period, to receive text messages. This was because he had not renewed his monthly mobile phone plan; his service was restored when he paid the monthly fee, after which he continued to receive the study text messages.

Appointment Attendance and Utilization

Comparing pre- and postintervention periods, there were trends in the hypothesized direction for patient-cancelled appointments that reduced from 53 to 37, a 30% change, and for no-shows, which reduced from 31 to 25, a 19% change (Figure 1). The results of paired t tests indicated there was a statistically significant change in ED visits, which were reduced from 15 to 5, a difference of 10 (95% CI = 2.2, 17.8; P = .01), and a borderline significant reduction in hospitalizations, from 3 to 0, a difference of 3 (95% CI = −0.4, 6.4; P = .08).

FIGURE 1—

Utilization changes pre- versus postintervention: Providence, RI, 2013.

Note. Appt. = appointment; ER = emergency room.

Potential Cost Savings

We estimated the potential cost savings from preventing cancelled visits, no-shows, and ED visits. Converting observed utilization from the 8-week intervention period to an annual rate produced 2.08 cancellations (reduction of 16 cancellations × 6.5 8-week periods per year × 0.05 for a per-person basis × 0.40 because of 60% of cancellations filled), 1.95 no-shows, and 3.25 ED visits avoided per person per year (Table 2). To show the uncertainty around the estimates, we incorporated the 95% CIs around the point estimates of reduced utilization, that is patient cancellations (−1.95, 6.11), no-shows (−4.75, 8.65), and ED visits (0.72, 5.79).

TABLE 2—

Estimated Potential Annual Net Cost Savings if Text Message Reminders Were Implemented Throughout VHA With 15 500 Homeless Veterans: Providence, RI, 2013

	Savings From Avoiding Unused Appointments and Preventing Visits			Costs of Text-Messaging System, From Health System and Patient Perspectives
Variable	Saved Utilization per Person per Yeara	Average Unit Cost of Service, $	Savings for 15 500 Veterans,a in Millions of $	Texts Sent or Received per Patient per Year	Average Cost per Text Message Sent or Received, $	Cost for 15 500 Veterans, $
Savings factor
Patient-cancelled appointments (and not refilled) avoidedb	2.08 (−1.95, 6.11)	198	6.38 (−5.98, 18.75)
Patient no-shows avoidedc	1.95 (0.00, 8.65)	198	5.98 (0.00, 26.53)
Patient ED visits avoided	3.25 (0.72, 5.79)	791	39.85 (8.77, 70.93)
Total savings			52.21 (2.78, 116.21)
Costs factor
Cost to health system of text messaging				137	0.12	254 820
Cost to veterans of receiving text messages				137	0.10	212 350
Total costs						467 170
Net savings, $, in millions			51.75 (2.31, 115.74)

Note. ED = emergency department; VHA = Veterans Health Administration.

^aRanges reflect the 95% confidence intervals (CIs) around the point estimates for mean reduction in service.

^bPatient cancellations only count the 40% that are estimated not to be refilled with another patient.

^cRange originally included a negative number (−4.75). Because it was highly unlikely the intervention would increase no-shows, the negative bound was set to zero.

We truncated the negative portion of the no-show bounds at zero, as shown in Table 2. Multiplying these utilizations by estimated costs for service^29,31 showed that cost savings would be $2.8 to $116.2 million per year. Table 2 shows costs incurred because of the text-messaging system. They are derived from 137 text messages sent per patient annually (on the basis of the 420 texts sent during the pilot, for 20 persons, over 8 weeks) and unit cost estimates from published literature for sending and receiving text messages.^32,33 Cost to the health care system of implementing text message reminders for homeless and unstably housed veterans would be $254 820, whereas the cost to veterans for receiving these text messages would be $212 350, for a combined cost of $467 170. Subtracting the costs (sending and receiving text messages) from the savings because of changes in utilization resulted in a potential total net annual VHA system savings of $2.3 to $115.7 million.

DISCUSSION

In a pilot study of mobile phone text-messaging appointment reminders with homeless persons, the first such study to our knowledge, participants found the text-messaging system easy to use and useful. Nearly all wanted to continue receiving text message reminders. They reported that it helped keep them organized, facilitated planning for their health care visits, and thus reduced the number of missed visits. Medical record data suggested that the text messages may have led to reductions in patient-cancelled visits, appointment no-shows, and ED visits. Our cost analysis, premised on national implementation of the text-messaging system with homeless and unstably housed veterans, indicated there could be substantial annual cost savings for the VA medical system, motivating future studies and wider implementation.

Although our findings must be interpreted with great caution because of the small sample size and lack of control group, they concur with a growing literature on ways text messaging contributes to a variety of process and outcome improvements, especially in appointment attendance and medication adherence. Guy et al., in their meta-analysis of appointment reminder systems, found that text message reminders were associated with a 50% increase in the likelihood of appointments attended compared with no reminders,¹⁶ whereas a Cochrane review found evidence for text messaging contributing to improved medication adherence.³⁶ We were encouraged by the findings that the intervention appeared to be feasible and acceptable and participants were enthusiastic about the benefits and indicated a desire to continue receiving text message reminders.

Difficulties engaging homeless populations in health care have been well documented—many do not have a regular doctor, ED use and hospitalization rates are high, and few receive preventive services.³⁷ Only recently have attempts been made to use information technologies to engage homeless persons. Burda et al. piloted an intervention of automated cell phone calls to 10 homeless persons to monitor medication adherence.³⁸ They reported, similar to our findings, that use of a mobile phone intervention was feasible, attrition from the study was not a problem, and few participants reported technical difficulties in using the mobile phone to complete the intervention activities.

An important objective of our pilot study was to demonstrate feasibility in this population. From this pilot we learned that (1) participants using their own phones is a viable option—although it may prevent the neediest from participating; (2) simple, brief text messages appear to be well accepted by this population; and (3) the amount of drop-out and loss to follow-up is acceptable. An improvement, before a larger study, would include additional screening to inquire about planned periods in which mobile phone contact would be interrupted, such as jail time, moving temporarily or permanently out of the area, or hospitalizations.

Automated phone call reminders tested in 3 VA medical centers with HIV-positive veterans were recently shown to be ineffective, with a homeless subgroup, at reducing HIV-primary care clinic no-shows.³⁹ Text messages, compared with automated mobile phone calls or live phone calls, represent a viable and preferred means of contacting homeless populations for several reasons: they are less expensive, they are more easily stored and retrieved from the phone, they cost the recipient less money, there are fewer comprehension issues such as poor acoustics of some live and automated calls, and, because of different technology, they have higher transmission rates. Appointment reminders are greatly needed for this population because they frequently lack the tools that nonhomeless take for granted: reliable mailing address, landline phones, wall or computerized calendars, and social supports to remind them of appointments (e.g., spouse, relative).⁴⁰

Limitations

Our participants represented a small, nonrandom sample. Our findings, although they show trends in the anticipated direction for all outcomes, were statistically significant only for reduction in ED use. Thus our findings may be owing to random variation. Additionally we did not have an external control group, and thus, changes unrelated to the intervention, such as changes in clinic practices, may have accounted for the observed reductions in patient cancellations and no-shows. In our discussion with clinic staff about this, however, no known changes in practices, policies, or staff surfaced. Our participants were all users of a homeless-oriented clinic at a VA medical center and, as such, were relatively frequent users of VA services. Thus when expanded to the wider homeless and unstably housed population who make fewer visits, the cost saving may not be as dramatic as our estimates indicated.

Also, our cost projections included estimates derived from reductions in patient-cancelled appointments and patient no-shows, neither of which was statistically significant. Additionally our findings may not generalize to non-English speakers, to veterans who use VA general primary care clinics, or to nonveteran homeless persons. We recruited participants who had their own phone and already used texting. Programs that plan to provide mobile phones to homeless persons who do not have phones may not have the same results and would have to provide training in the use of the phone and the phone’s text-messaging system. A recent literature review, however, indicates that mobile phones are common among homeless persons and are likely to increase as phone prices continue to drop,¹⁹ and a survey of homeless veterans found that 89% had mobile phones.⁴¹

Conclusions

In this pilot study, we demonstrated the feasibility of a mobile phone text message appointment reminder intervention with homeless or unstably housed veterans and indicated it could lead to health system cost reductions. It was well liked by participants and was associated with substantial reductions in no-shows, patient-cancelled appointments, ED visits, and hospitalizations. These reductions, extrapolated to the larger population of homeless and unstably housed veterans could produce substantial cost savings.

These findings suggest the importance of conducting additional studies, with other homeless populations, in different care sites. This would allow assessment of statistical and clinical significance and greater confidence that the changes in utilization of services were owing to the text message reminder intervention. Continued research into, and implementation of, new communication approaches with homeless and unstably housed persons may lead not only to improved engagement in and receipt of health care services but also to improved delivery of other services for homeless populations, such as housing assistance, case management, and job training programs.