Use of mobile applications and mental health services among veterans who received internet-connected tablets

Abstract

Objectives

Among Veterans Health Administration (VHA) patients receiving internet-connected tablets, we examined (1) app usage, (2) clusters of commonly used apps and patient characteristics across clusters, and (3) the relationship between app clusters and mental health services.

Materials and Methods

A survey was distributed to a national sample of VHA patients (n = 5449) within 6 months of tablet receipt. Self-reported app use was clustered with agglomerative hierarchical clustering. Survey data were combined with electronic health record data that included sociodemographics and mental health care utilization during the 6 months following survey completion. Differences in patient characteristics by cluster were examined using chi-square tests. A multivariable model assessed prediction of mental health services across app clusters.

Results

Among 1664 patients with complete survey and health care utilization data, the majority (74%) reported using at least 1 app. The most used apps were for video visits (60%) and VA’s patient portal (53%). Cluster analysis revealed 5 distinct groups reflecting app usage: Patient Portal (n = 685, 41%), Video Connection (n = 319, 19%), Treatment Companion (n = 166, 10%), Self-Management (n = 85, 5%), and Non-App Group (n = 409, 25%). Cluster characteristics varied, and those in the Patient Portal Group (adjusted odds ratio  = 1.64; 95% CI = 1.15-2.32) and Video Connection Group (adjusted odds ratio = 1.52; 95% CI = 1.02-2.26) were more likely to have a mental health visit in the 6 months after survey completion when compared to the Non-App Group (P < .05).

Discussion

Tablet provision supports opportunities to use apps for multiple health management tasks, and app use clusters vary across patient characteristics and health care utilization.

Conclusion

Patients most commonly used apps to connect with their care team, and use of some apps and app combinations may improve engagement with mental health services.

Background and significance

The health application (app) ecosystem is vast and a growing component of health care delivery. Patient apps encompass tools to support treatment and self-management for specific health conditions, video visits with clinicians, and access to patient portals with health information and services. Estimates indicate that over 250 000 health-related apps are available in the Apple iOS and Google Play app stores.¹ Because few of these apps have been rigorously evaluated and tested, some health care systems and institutions have assembled suites of evidence-based apps reviewed by clinical experts for patients to choose from.^2–4 The U.S. Department of Veterans Affairs (VA) has a suite of more than 30 evidence-informed apps that are freely available via the VA Mobile App store⁵ and can be downloaded from the Apple and Google Play app stores. Many of these apps provide support for mental health needs for veterans living with symptoms such as posttraumatic stress disorder (PTSD). Nearly all of the VA apps have been evaluated and optimized for usability, acceptance, and effectiveness.⁶ For example, several of these apps have demonstrated reduced symptom severity, improved psychosocial functioning, and better treatment engagement among veterans with mental health needs.⁷

Despite veterans’ high interest in using health apps, many are not familiar with how to find or use VA’s apps, and they experience limitations that prevent usage.^8,9 In a 2016 survey conducted among 149 veterans, the majority (73%) expressed interest in using apps for mental health, but only 11% had actually done so.⁹ Half (51%) reported not knowing where to find an app, and nearly a quarter did not own a smartphone.⁹ National surveys have indicated lower smartphone ownership and internet use with age, although these rates have increased across all age groups over time.^10,11 Veterans have also reported financial limitations and connectivity issues as barriers to app use, which are more common among those living in rural areas compared to urban locations.¹²

To promote access to virtual care opportunities, the VA has shipped over 180 000 internet-connected tablets with VA mobile and web-based app capabilities to veterans with limited access to technology and barriers to in-person care (ie, geographic, social, clinical barriers).¹³ Prior evaluations of this tablet program among veterans with mental health conditions demonstrated improved continuity of care,¹⁴ reduced suicide behavior,¹⁵ and fewer emergency department visits.¹⁵ Little is known about how veterans who receive a VA-issued tablet engage with the assortment of apps available through the tablet, including commonly used combinations of apps (ie, clusters) and how app use may influence health care engagement. Thus, we sought to understand (1) which apps and combinations of apps are used on VA tablets, (2) patient characteristics related to app use, and (3) the association between app use and health care utilization. Having a better understanding of how patients use apps on health system-issued tablets and their health care utilization is valuable to ensure the tablets are meeting patients’ needs and can be used to inform other health care systems’ efforts to engage patients in virtual care.

Objectives

Among patients receiving a VA-issued tablet who completed a survey, we aimed to (1) examine which apps are used, (2) assess clusters of commonly used apps and patient-level characteristics associated with app clusters, and (3) examine the relationship between app use and mental health care utilization.

Methods

Setting

The VA’s Offices of Rural Health and Connected Care tablet initiative provides tablets (Apple iPads with T-Mobile or Verizon data plans) to veterans who lack a device or internet connection for video visits, experience a barrier to accessing in-person care, and have an upcoming appointment that could be conducted over video within the next 90 days.¹³ Tablet eligibility is assessed through a digital divide consult template in the VA’s electronic health record (EHR) by clinicians and social workers.¹⁶ The VA’s Connected Device Support Team calls each tablet recipient to assist with setup and to conduct a test video visit call. There are no specific setup instructions for other apps, but patients have the option to download any VA apps of their choosing through a VA App Catalogue icon on the tablet home screen that routes them to the VA Mobile App store.⁵ Default apps on the home screen include: VA Video Connect, My HealtheVet, Rx Refill, Annie, and PTSD Coach (see Table 1). Patients can contact the Office of Connected Care Help Desk or local telehealth team to receive technical support on any of the apps. This evaluation was conducted as part of the Enhancing Veterans’ Access to Care through Video Telehealth Tablets Quality Enhancement Research Initiative. The evaluation was considered a quality improvement evaluation in partnership with VA’s Office of Rural Health and Connected Care, and designated as nonresearch by the local IRB committee.

Table 1.

Veterans Affairs (VA) mobile and web apps included in the survey.

App	Description	Platform
VA Video Connect	VA Video Connect allows veterans and their caregivers to quickly and easily meet with VA health care providers through live video on any computer, tablet, or mobile device with an internet connection	iOS, Android, Web
My HealtheVeta	My HealtheVet is the VA’s personal health record where veterans can view their health information, securely message their care team, refill perceptions, view labs, view and schedule appointments, and download portions of their health record	Web
Rx Refilla	Rx Refill allows veterans to request refills of VA-dispensed prescriptions, scan VA prescriptions to access prescription information, track VA prescription deliveries, and view VA prescription history. The app provides a native mobile interface for the Rx Refill feature found within My HealtheVet	iOS, Android
VA Online Schedulinga	VA Online Scheduling offers a secure way to request and self-schedule appointments at VA facilities and Community Care facilities. Veterans can also view details about upcoming appointments and download appointment data to their personal calendar	Web
My VA Imagesb	My VA Images enables veterans to send secure photos and videos to their VA providers	Web
Anniea,b	Annie is a VA messaging service that empowers veterans to take an active role in their care. Messages can include reminders to take medication, tips for reducing stress, instructions to prepare for health care appointments, and more	Web, SMS
Mental Health Checkupa,b	Mental Health Checkup allows veterans to monitor, assess, and access information for mental health conditions	Web
ACT Coach	Acceptance and Commitment Therapy (ACT) Coach was developed for veterans, service members, and others who are in ACT with a therapist. It offers exercises, tools, information, and tracking logs to help veterans practice skills in daily life	iOS, Android
CBT-i Coach	CBT-i Coach is for people engaged in Cognitive Behavioral Therapy for Insomnia with a health provider, or who have experienced symptoms of insomnia and would like to improve sleep habits on their own	iOS, Android
PTSD Coach	PTSD Coach was designed for those who have, or may have, posttraumatic stress disorder (PTSD). The app provides education about PTSD, information about professional care, and tools that can help manage the stresses of daily life with PTSD. The app is available in English and Spanish	iOS, Android
Mindfulness Coach	Mindfulness Coach was developed to help veterans, service members, and others learn how to practice mindfulness. The app provides a gradual, self-guided training program to help users understand and adopt a simple mindfulness practice.	iOS, Android
MOVE Coach	MOVE! Coach is a weight management app for veterans, service members, families, and others who want to lose weight. The 16-week program guides participants to achieve success with weight loss and management through education and use of tools	iOS, Android
COVID Coach	COVID Coach was created for everyone, including veterans and service members, to support self-care and mental health during the COVID-19 pandemic. The app is available in English and Spanish	iOS, Android

VA app icon source: https://mobile.va.gov/appstore

^aUsername and password authentication required.

^bApp use is initiated by a VA provider.

Survey sample and sampling

A survey was sent to a national sample of patients (n = 5449) who received a VA tablet within 6 months prior to survey administration (September 13, 2021-January 21, 2022). EHR data from the VA Corporate Data Warehouse¹⁷ were used for sampling, and the sampling frame was defined based on tablet receipt and EHR-recorded health care utilization. The sample was selected via stratified random sampling proportional to the population in each Veterans Integrated Services Network to represent VA’s 18 geographically defined regions. Patients were eligible for the survey if they had a tablet for ≥3 months and ≥2 mental health or primary care visits in the 12 months before the survey. Survey exclusion criteria were patients with incomplete address information or located in Manila or San Juan. Survey invitation letters were sent by email or mail including information on the study’s purpose and instructions for completing the survey through the web, phone, or mail. In order to reach adequate numbers of patients with and without video visit experience, phone surveys were conducted on a subset of nonresponders. All participants were offered a $10 incentive for survey completion.

A total of 1697 patients completed the survey (31% response rate, 1697/5449). In this analysis, we excluded survey respondents who did not answer at least one of the three health app questions as defined below (n = 4) and those who received care at an Oracle Cerner Millennium EHR site given incomplete outcome data (n = 29) (Figure 1). Thus, the final analytic sample for the app cluster analysis comprised 1664 patients, of which 686 completed the survey through a website, 649 by phone, and 329 on paper.

Figure 1.

Survey sampling and analytic samples.

Survey and EHR measures

Use of mobile and web-based health apps was assessed with 3 questions: (1) “Have you or someone in your home used your VA tablet for a video visit?” (“Yes” or “No”), (2) “In the last 6 months, have you used the VA’s online portal, My HealtheVet, to do any of the following tasks?” Responses included a list of portal tasks (ie, manage appointments, send secure message), (3) “VA offers several mobile apps that can be used on VA tablets and other devices. Please circle any of the apps that you have used in the last 6 months.” Responses included a list of 13 app names with their icons focusing on nationally rolled out apps by the VA Office of Connected Care or their partner program offices with the highest number of downloads at the time of survey development (Table 1).

Survey respondents were also queried about their gender identity, race, ethnicity, education, financial hardship, and social support. Financial hardship was assessed through a 3-item question on “How hard has it been for you to pay for the basics like food, housing, medical care, and heating?” (“Not hard at all,” Somewhat hard,” “Very hard”). Social support was measured with 4 items related to tangible support from the Medical Outcomes Study Social Support Survey Instrument.¹⁸ Likert-scale responses ranged from “None of the time” to “All of the time,” and we considered adequate social support as those who “Mostly agree” or “Completely agree.”

We combined survey data with EHR data that included age, rurality, housing instability, physical health conditions, mental health conditions, and mental health care utilization during the 6 months following survey completion. Rurality was based on each patient’s ZIP code using the Census tract Rural Urban Commuting Area codes.¹⁹ Housing instability was defined using stop codes related to use of homeless services and diagnosis codes.²⁰ Chronic physical and mental health conditions were defined using International Classification of Diseases, tenth revision (ICD-10) code groupings from the Agency for Healthcare Research and Quality Clinical Classification Software and the VA’s Women’s Health Evaluation Initiative.²¹ The number of physical health conditions (out of 37) and mental health conditions (out of 10) were calculated and binned into quartiles.

Analysis

Descriptive analyses were first performed to characterize app use. To examine the binary association between each of the 13 apps, phi (φ) coefficients were calculated and visualized in a correlation matrix to facilitate the cluster analysis. App use data were then clustered with agglomerative hierarchical clustering—a bottom-up approach where each data point (use of an app: yes/no) starts in a separate cluster and are paired together based on similarities with other data points.²² Similarity was based on the Jaccard distance measure, which measures the relationship between binary data.²² To be included in the cluster analysis, patients had to complete at least one of the three app questions assessing use of the 13 apps. Approximately 10% (168/1664) of patients did not complete all 3 questions, but our sensitivity analysis removing patients with missing app data showed no meaningful impact on the mental health utilization outcome in the multivariable model (Appendix S1). Thus, we included all 1664 patients in the cluster analysis as the majority answered the video visit and patient portal questions.

Iterative clustering was conducted, and the clustering model with the highest silhouette coefficient, which measures the amount of cohesion and separation of the clusters, was selected.²² Initially, clustering was conducted with all 13 apps, but several apps are also available as features within the patient portal (eg, Rx Refill, VA Online Scheduling, My VA Images). Many apps also had low self-reported usage (<7% use for 9/13 apps). Thus, we categorized apps according to function, and app use was clustered by category: video connect app, patient portal with subfunctions app, treatment companion apps, and self-management apps. These categories were informed by the app correlation matrix and discussions with clinicians and VA mobile app researchers. In our final clustering model, the average silhouette coefficient was 0.75, indicating moderate to high cohesion within each cluster and separation from other clusters.

Once the clusters were established, we examined the cluster compositions with radial bar charts to determine a name that sufficiently represented each cluster.²³ We then assessed differences across clinical and sociodemographic characteristics among clusters using chi-squared tests. Lastly, among patients with a mental health diagnosis (n = 1307), we regressed the dependent variable of any mental health visit during the 6 months after survey completion on independent variables of app clusters, controlling for age, gender, race/ethnicity, housing instability, and the number of physical and mental health conditions. Analyses were conducted in Python 3.7 (Scikit-learn AgglomerativeClustering package)²⁴ and Stata 17 to account for survey design in SE estimation.

Results

Survey participant characteristics

Among 1664 patients who completed the survey, the mean age was 63 (SD = 13); a majority were men (84%), White non-Hispanic (54%), and lived in an urban location (70%)—see Table 2. Approximately 22% had housing instability, 19% found it very difficult paying for the basics, and 33% earned a high school/GED degree or less. One quarter had 8 or more physical health conditions, and one-third had 3 or more mental health conditions.

Table 2.

Participant characteristics and app use^a,b (n = 1664).

Characteristics	% (95% CI)
Age (mean: 63, SD: 13)
18-44	11.1 (9.8, 12.6)
45-64	40.7 (38.5, 43.0)
65+	48.2 (45.9, 50.4)
Gender
Man	83.6 (81.8, 85.2)
Woman	14.1 (12.6, 15.7)
Nonbinary or other	0.8 (0.5, 1.4)
Race and ethnicity
White non-Hispanic	53.7 (51.5, 55.9)
Black non-Hispanic	26.8 (24.9, 28.8)
Hispanic	7.5 (6.4, 8.7)
Mixed race or other	5.0 (4.1, 6.1)
Housing instability
Yes	21.8 (20.0, 23.7)
No	78.2 (76.3, 80.0)
Rurality
Rural or highly rural area	30.2 (28.2, 32.3)
Urban area	69.8 (67.7, 71.8)
Education
High school, GED, or less	32.9 (30.8, 35.0)
Some college or 2-year degree	45.4 (43.2, 47.7)
4-year college graduate or more	20.6 (18.8, 22.4)
Difficulty paying for basics
Not hard at all	34.6 (32.4, 36.7)
Somewhat hard	44.3 (42.0, 46.6)
Very hard	19.2 (17.5, 21.0)
Physical health conditions (mean: 6, SD: 3)
0-3	29.2 (27.3, 31.4)
4-5	25.0 (23.1, 27.0)
6-7	21.2 (19.4, 23.1)
8+	24.6 (22.7, 26.6)
Mental health conditions (mean: 2, SD: 2)
0	21.5 (19.7, 23.4)
1	20.8 (19.1, 22.8)
2	25.2 (23.3, 27.2)
3+	32.5 (30.4, 34.7)
Adequate social support
If confined to bed	54.7 (52.5, 57.0)
To take to the doctor’s office	62.3 (60.0, 64.4)
To prepare meals	58.1 (55.9, 60.3)
To help with daily chores	56.4 (54.2, 58.7)

^aMissing data: gender (n = 25), race ethnicity (n = 117), education (n = 19), difficulty paying for basics (n = 33), social support (n = 27).

^bPatient-reported survey items include gender, race ethnicity, education, financial hardship, and social support. Electronic health record-derived items include age, housing instability, rurality, physical health conditions, and mental health conditions. Stata’s survey commands were used to account for survey design in SE estimation.

App use

Six months after completing the survey, the majority of the sample reported using at least 1 app (74%, 1225/1664), with the most common being the VA video visit app (60%) and My HealtheVet (53%) patient portal (Figure 2). Apps for specific conditions and self-management purposes were used less commonly, a result that is not unexpected given the lower proportion of potential users (eg, 43% of the study population had a diagnosis of PTSD, 26% had a diagnosis of obesity). Apps whose use was most highly correlated were PTSD Coach and Mindfulness Coach (φ = 0.50), followed by Mental Health Check Up and Mindfulness Coach (φ = 0.39), and Mental Health Check Up and PTSD Coach (φ = 0.36). Patient portal and subfunction apps also had moderate correlations, including Rx Refill and VA Online Scheduling (φ = 0.37) as well as Rx Refill and My HealtheVet (φ = 0.36). Participants inputted a total of 75 “Other” free-text apps, and the most common included non-VA mobile and web apps (n = 19) such as Google, Safari, Gmail, Spotify, Pinterest, Youtube, Facebook, VA websites (n = 16), other VA apps (n = 11), and VA E-benefits app (n = 6).

Figure 2.

App correlation matrix. Missing data: VA Video Connect (n = 10), My HealtheVet (n = 60), remaining apps (n = 163).

App clusters

Cluster analysis revealed 5 app clusters such that patients could be categorized into 5 groups: Patient Portal Group, Video Connection Group, Treatment Companion Group, Self-Management Group, and Non-App Group (Figure 3). Out of the 1664 patients include in the cluster analysis, the largest cluster was the Patient Portal Group (n = 685, 41%), in which all patients used the My HealtheVet portal and/or related subfunctions to refill prescriptions, schedule appointments, or send images to their care team. Most in the Patient Portal Group (68%) also used the video visit app to connect with their provider. The Video Connection Group (n = 319, 19%) comprised patients who only used the video visit app. The smallest clusters included the Self-Management Group (n = 85, 5%) and Treatment Companion Group (n = 166, 10%). Most patients in these clusters also commonly used the patient portal and video visit apps. Nearly half of patients who used apps to supplement their mental health treatment also used self-management apps. Patients in the Non-App Group (n = 409, 25%) reported not using any of the 13 apps.

Figure 3.

App cluster compositions. Percentages within each radial bar chart are based on the proportion of patients within each cluster who reported using each app.

Patient characteristics varied across clusters. Compared with patients who used apps, those in the Non-App Group were older (P < .01) and had lower levels of social support (P < .01) (Table 3). The Self-Management and Treatment Companion Groups had higher proportions of patients who were younger, had unstable housing, higher educational attainment, more financial hardship, and more mental health conditions (P < .01). The Self-Management Group was the most diverse in terms of having a larger proportion of patients who identified as women and racial/ethnic representation compared to the other clusters (P < .01). There were no meaningful differences in rurality or physical health conditions across the clusters.

Table 3.

Characteristics among app clusters^a,b (n = 1664).

Characteristics, % (95% CI)	Patient Portal Group (n = 685)	Video Connection Group (n = 319)	Treatment Companion Group (n = 166)	Self-Management Group (n = 85)	Non-App Group (n = 409)	P
Age
18-44	14.6 (12.3, 17.3)	7.2 (5.0, 10.4)	21.7 (16.3, 28.2)	21.2 (14.1, 30.5)	2.0 (1.0, 3.7)
45-64	41.2 (37.7, 44.7)	37.0 (32.2, 42.1)	55.4 (48.2, 62.4)	55.3 (45.2, 64.9)	34.0 (29.8, 38.5)	<.01
65+	44.2 (40.8, 47.8)	55.8 (50.6, 60.9)	22.9 (17.4, 29.5)	23.5 (16.1, 33.1)	64.0 (59.5, 68.3)
Gender
Man	82.9 (80.1, 85.4)	86.5 (82.6, 89.7)	82.5 (76.4, 87.4)	74.1 (64.4, 81.9)	84.8 (81.2, 87.8)
Woman	15.0 (12.7, 17.8)	12.2 (9.2, 16.0)	14.5 (10.1, 20.3)	24.7 (17.1, 34.4)	11.5 (8.9, 14.8)	<.01
Nonbinary or other	1.6 (0.9, 2.8)	0	1.2 (0.3, 4.3)	1.2 (0.2, 7.2)	0
Race and ethnicity
White non-Hispanic	53.1 (49.6, 56.6)	55.8 (50.6, 60.8)	46.4 (39.3, 53.6)	43.5 (34.0, 53.6)	58.2 (53.7, 62.6)
Black non-Hispanic	28.8 (25.7, 32.0)	25.1 (20.9, 29.8)	25.3 (19.6, 32.0)	31.8 (23.2, 41.7)	24.4 (20.8, 28.5)	<.01
Hispanic	7.7 (6.1, 9.8)	9.1 (6.6, 12.5)	7.8 (4.7, 12.7)	16.5 (10.3, 25.2)	3.7 (2.3, 5.8)
Mixed race or other	4.8 (3.5, 6.6)	2.2 (1.1, 4.3)	9.0 (5.7, 14.1)	8.2 (4.1, 15.7)	5.1 (3.5, 7.6)
Housing instability
Yes	19.6 (16.9, 22.5)	20.7 (16.8, 25.2)	31.9 (25.6, 39.0)	27.1 (19.1, 36.8)	21.3 (17.8, 25.3)	<.01
No	80.4 (77.5, 83.1)	79.3 (74.8, 83.2)	68.1 (61.0, 74.4)	72.9 (63.2, 80.9)	78.7 (74.7, 82.2)
Rurality
Rural or highly rural area	30.8 (27.7, 34.1)	31.3 (26.8, 36.3)	26.5 (20.7, 33.3)	28.2 (20.1, 38.1)	30.3 (26.3, 34.7)	.79
Urban area	69.2 (65.9, 72.3)	68.7 (63.7, 73.2)	73.5 (66.7, 79.3)	71.8 (61.9, 79.9)	69.7 (65.3, 73.7)
Education
High school, GED, or less	26.9 (23.9, 30.1)	42.6 (37.6, 47.8)	24.7 (19.0, 31.4)	20.0 (13.1, 29.3)	41.3 (36.9, 45.9)
Some college	52.3 (48.7, 55.8)	38.6 (33.7, 43.7)	44.0 (37.0, 51.2)	48.2 (38.4, 58.2)	39.4 (35.0, 43.9)	<.01
4-year degree or more	20.6 (17.9, 23.6)	18.2 (14.5, 22.5)	30.7 (24.5, 37.7)	31.8 (23.2, 41.8)	15.9 (12.8, 19.5)
Difficulty paying for basics
Not hard at all	35.6 (32.3, 39.1)	38.9 (33.9, 44.0)	22.9 (17.4, 29.5)	30.6 (22.2, 40.5)	35.0 (30.7, 39.4)
Somewhat hard	46.9 (43.4, 50.4)	41.1 (36.1, 46.2)	45.8 (38.7, 53.0)	42.3 (32.8, 52.5)	42.3 (37.8, 46.9)	<.01
Very hard	16.8 (14.3, 19.6)	17.9 (14.2, 22.2)	28.9 (22.9, 35.8)	27.1 (19.1, 36.8)	18.6 (15.3, 22.4)
Physical health conditions
0-3	27.4 (24.4, 30.7)	31.1 (26.5, 36.0)	34.9 (28.4, 42.1)	31.8 (23.2, 41.8)	28.1 (24.2, 32.4)
4-5	24.4 (21.5, 27.6)	25.7 (21.4, 30.5)	24.1 (18.5, 30.8)	29.4 (21.1, 39.3)	24.9 (21.2, 29.1)	.39
6-7	22.5 (19.7, 25.6)	21.3 (17.4, 25.9)	19.3 (14.2, 25.6)	22.4 (15.1, 31.8)	19.3 (15.9, 23.2)
8+	25.7 (22.7, 28.9)	21.9 (18.0, 26.5)	21.7 (16.3, 28.2)	16.4 (10.3, 25.3)	27.7 (23.7, 31.9)
Mental health conditions
0	21.2 (18.4, 24.2)	21.3 (17.4, 25.9)	8.4 (5.2, 13.4)	16.5 (10.3, 25.3)	28.4 (24.4, 32.6)
1	20.7 (18.0, 23.8)	21.3 (17.4, 25.9)	13.3 (9.1, 18.9)	11.8 (6.7, 19.9)	25.6 (21.9, 29.9)	<.01
2	25.4 (22.4, 28.6)	25.4 (21.1, 30.2)	27.7 (21.8, 34.6)	22.4 (15.1, 31.8)	24.2 (20.5, 28.3)
3+	32.7 (29.5, 36.1)	32.0 (27.4, 37.0)	50.6 (43.5, 57.7)	49.3 (39.5, 59.4)	21.8 (18.2, 25.8)
Adequate social support
If confined to bed	59.1 (55.6, 62.6)	55.8 (50.6, 60.9)	54.8 (47.6, 61.8)	52.9 (42.9, 62.7)	46.9 (42.4, 51.5)	<.01
To take to doctor’s office	65.3 (61.8, 68.5)	64.9 (59.8, 69.7)	56.6 (49.4, 63.6)	58.8 (48.7, 68.2)	58.2 (53.6, 62.6)	<.01
To prepare meals	60.4 (56.9, 63.8)	62.1 (56.9, 67.0)	58.4 (51.2, 65.3)	55.3 (45.2, 64.9)	51.6 (47.0, 56.1)	<.01
To help with daily chores	59.3 (55.7, 62.7)	61.1 (56.0, 66.0)	56.0 (48.8, 63.0)	55.3 (45.3, 64.9)	48.4 (43.9, 53.0)	<.01

^aMissing data: gender (n = 25), race ethnicity (n = 117), education (n = 19), difficulty paying for basics (n = 33), social support (n = 27).

^bPatient-reported survey items include gender, race ethnicity, education, financial hardship, and social support. Electronic health record-derived items include age, housing instability, rurality, physical health conditions, and mental health conditions. Stata’s survey commands were used to account for survey design in SE estimation.

App use patterns and mental health engagement

When we examined mental health utilization among patients with a mental health condition, those in the Patient Portal Group (adjusted odds ratio = 1.64; 95% CI = 1.15, 2.32) and Video Connection Group (adjusted odds ratio = 1.52; 95% CI = 1.02, 2.26) were more likely to have a mental health visit in the 6 months after survey completion when compared to the Non-App Group (P < .05). We found no meaningful differences in mental health utilization after receiving the tablet in the Treatment Companion Group or Self-Management Group compared to the Non-App Group (Figure 4).

Figure 4.

Adjusted odds ratios of having a mental health visit during the 6 months post-tablet period relative to app cluster among patients with a mental health condition (n = 1307). The model was adjusted for age, gender, race/ethnicity, housing instability, number of physical health conditions, and number of mental health conditions. Stata’s survey commands were used to account for survey design in SE estimation.

Discussion

Main findings

Among patients with limited technology and barriers to in-person care who received a tablet, approximately three quarters reported using at least 1 VA web or mobile app on the tablet. Patients most often used apps to connect to their care team and manage their health information through video visits and the patient portal. A smaller proportion (15%) took the opportunity to engage in the self-management and treatment companion apps, and most of these patients also used the patient portal and video connect apps. Patient-level characteristics varied across app clusters, with those who did not use an app being older and reporting less social support. Among patients with a mental health condition, those in the Patient Portal Group and Video Connection Group were more likely than those in the Non-App Group to have a mental health visit in the 6 months after completing the survey. Altogether, these findings suggest that tablets support access to multiple health management opportunities, and the use of some apps may improve downstream engagement with the health care system. As health care systems seek to continually engage patients with mental health needs, health apps may be a potential tool to increase and sustain such engagement.

Implications for health care and research

This study has several implications for the combined use of apps to assist patients in connecting to their care team and completing health management tasks for both veteran and civilian populations. Prior studies among civilians suggest that apps encompassing a broad set of virtual care tools may help expand access to care and narrow the digital divide.^25,26 Our findings build on previous work in several ways. First, we found that apps that foster interactions, including video connection apps and patient portals, are more widely used. These types of tools incorporate synchronous (ie, audio and/or video visits) and asynchronous (ie, secure messaging) communication. Patient portals can serve as one means to facilitate video visits through appointment scheduling, reminders, and video visit links. This is aligned with our finding that patients in the Patient Portal Group, which comprised a majority of patients using both the patient portal and video connect app, were more likely to have a mental health visit after receiving their tablet. Second, apps that do not have direct interactions with care providers do not foster as much engagement. This suggests that tablet provision alone may not be sufficient to consistently engage patients in self-management or supplemental treatment opportunities. As health care systems consider ways to interact with and engage patients, providing pathways to utilize self-management and treatment companion apps through common tools, such as video-based apps and patient portals, would be valuable.

Our findings also have implications for digital health policies and health system-wide initiatives that promote access to health information and technologies. Several recent national initiatives in the United States have been implemented to facilitate access to health information through patient portals and apps as part of the 21st Century Cures Act and Centers for Medicare and Medicaid Services Interoperability and Patient Access regulations.^27,28 Key provisions from these final rules implement standards and policies that prevent electronic health information blocking and provide patients with added convenience through health apps of their choice. While federal efforts increase the availability of health information and technologies, they benefit more well-resourced patient subgroups who are already equipped to use them.^29,30 Health care systems have often struggled to provide equitable access to health technologies, but the VA’s tablet distribution program has shown that such programs can engage digitally vulnerable patients and improve access to care.^14,15 Our findings provide further support for use of additional apps and combinations of apps available through the tablets that may influence participation in health care services.

Health care systems should also consider providing evidence-based app catalogues to patients to facilitate engagement in their care and self-management. The VA-developed apps are unique as they are developed for noncommercial purposes, evidence-informed, contain similar interface components to enhance user comfort, and many are mental health-related with content on managing psychological distress for traumatic experiences.⁷ Most of these apps are also freely available to civilians, and some apps have been tested among civilian populations. Notably, PTSD Coach has demonstrated acceptability, feasibility, and effectiveness among community trauma survivors,^31,32 cancer survivors,³³ and acutely injured patients in the emergency department.³⁴ PTSD Coach has also been tested among traumatized populations in Sweden,^35,36 South Africa,^37,38 and the Netherlands.³⁹ Little is known about how the VA apps compare to the large number of apps available through the Apple iOS and Google Play app stores, indicating a future area of work to compare common app user metrics (eg, retention, churn) and effectiveness to facilitate identification of high-quality apps for all patients.

Given the known structural inequities in digital access, additional strategies to promote digital inclusion for at-risk individuals in veteran and civilian populations should be prioritized. For example, we found that among those who did not report using any apps (25%), most tended to be older with lower levels of social support—known characteristics among digitally vulnerable individuals in the general population.^10,11,40 Importantly, use of technology has been shown to positively impact social connectedness and social isolation among older adults.⁴¹ Prior studies have also found that living with family or friends has been positively associated with virtual care use, especially for older adults who may also be receiving technical assistance from family or friends.⁴² With 90% of older adults in the United States owning a smartphone¹⁰ many have reported satisfaction with health technologies and prefer to use them in combination with in-person care.^43,44 Being mindful of preferences and abilities to adopt and use technologies is particularly important for older adults given the variety of attitudes and challenges that may decrease comfortability in navigating these tools.⁴⁵

The VA has several initiatives to support patients in gaining confidence in technology setup and ongoing usage. For video visits, VA providers can leverage a built-in EHR consult template to order a test video visit call to ensure patient comfort ahead of virtual visits with their care team. Patient portal coordinators are also available at VA medical facilities to provide educational material and training to both patients and staff. The VA Virtual Health Resource Centers also provide virtual and on-site technology assistance for patients and staff at certain medical facilities.⁴⁶ While such resources do exist, additional investigation is needed to examine the effectiveness and experiences with these technology training initiatives.

Limitations

Our study had several limitations. First, this study was conducted in the VA health care system—a large integrated health care system that has resources and a mobile app store that other systems or institutions may not have. Thus, our results may not be generalizable beyond this population. Second, the sample was limited to tablet recipients who completed a survey, and a disproportionate number of respondents were female and had stable housing compared to nonrespondents (Appendix S2). As the survey could only be distributed to patients with a valid mailing address, email, or phone number, our findings may not be generalizable to all patients in the VA health care system or those receiving tablets. Third, we were unable to directly track app use from the tablets and relied upon self-reported use of apps, which may be influenced by recall bias. We were also unable to track how many patients used apps that were initiated by a provider. These factors may impact estimates of app use. Fourth, only 5 of the 13 apps from the survey were available on the tablet home screen, and the remaining apps were accessed through the VA App Catalogue icon on the home screen. Discovering apps through the Catalogue may require individuals to have more motivation and potentially influence app use. Fifth, our analysis does not examine the frequency of app use or how patients use the apps which could impact health care engagement. Sixth, the cross-sectional nature of the survey precludes assessment of causal associations, including the relationship between app use and mental health care utilization. Lastly, tablet receipt and survey administration occurred during the COVID-19 pandemic which may influence app usage, as prior studies have shown increased use of video visits,^47,48 patient portals,⁴⁹ and other apps^50,51 during the pandemic.

Conclusion

Providing patients who experience barriers to care and limited technology access with an app-enabled device facilitated use of health apps. The majority of tablet recipients engaged with one or more apps within 6 months of receiving the tablet, most commonly for video visits. Many used apps for both video visits and to access the patient portal, suggesting that patients most often use the tablet to connect with their care team either synchronously or asynchronously. A smaller proportion of patients utilized the self-management and treatment companion apps, and the majority of these patients also used the video connect app and patient portal. This indicates that the tablets support access to multiple communication modes but may not be adequate for engaging patients in self-care or supplemental treatment opportunities. In addition, non-app users had lower participation in mental health care services when compared to other app clusters, and patient-level differences across clusters suggest that non-app users may benefit from additional training and social support to promote more equitable adoption and use of health apps.

Author contributions

All authors contributed to the study conception, design, manuscript writing, and revision. Ashley C. Griffin and Liberty Greene were primarily responsible for the analyses. Ashley C. Griffin developed the initial draft of the manuscript. All authors reviewed, edited, and approved the final version.

Supplementary material

Supplementary material is available at JAMIA Open online.

Funding

This work was supported by a Veterans Affairs (VA) Quality Enhancement Research Initiative (PEI 18-205). This work was also supported using resources and facilities at the VA Informatics and Computing Infrastructure, VA HSR RES 13-457. A.C.G. was supported by a VA Advanced Fellowship in Medical Informatics. D.B. was supported by a VA Research Career Scientist Award (RCS 23-080).

Conflicts of interest

None declared.

Data availability

The data underlying this article cannot be shared publicly due to Veterans Health Affairs policy on sharing patient data (eg, for the privacy of individuals who participated in the study).