Automated Remote Monitoring of Depression: Acceptance Among Low-Income Patients in Diabetes Disease Management

Abstract

Background

Remote patient monitoring is increasingly integrated into health care delivery to expand access and increase effectiveness. Automation can add efficiency to remote monitoring, but patient acceptance of automated tools is critical for success. From 2010 to 2013, the Diabetes-Depression Care-management Adoption Trial (DCAT)–a quasi-experimental comparative effectiveness research trial aimed at accelerating the adoption of collaborative depression care in a safety-net health care system–tested a fully automated telephonic assessment (ATA) depression monitoring system serving low-income patients with diabetes.

Objective

The aim of this study was to determine patient acceptance of ATA calls over time, and to identify factors predicting long-term patient acceptance of ATA calls.

Methods

We conducted two analyses using data from the DCAT technology-facilitated care arm, in which for 12 months the ATA system periodically assessed depression symptoms, monitored treatment adherence, prompted self-care behaviors, and inquired about patients’ needs for provider contact. Patients received assessments at 6, 12, and 18 months using Likert-scale measures of willingness to use ATA calls, preferred mode of reach, perceived ease of use, usefulness, nonintrusiveness, privacy/security, and long-term usefulness. For the first analysis (patient acceptance over time), we computed descriptive statistics of these measures. In the second analysis (predictive factors), we collapsed patients into two groups: those reporting “high” versus “low” willingness to use ATA calls. To compare them, we used independent t tests for continuous variables and Pearson chi-square tests for categorical variables. Next, we jointly entered independent factors found to be significantly associated with 18-month willingness to use ATA calls at the univariate level into a logistic regression model with backward selection to identify predictive factors. We performed a final logistic regression model with the identified significant predictive factors and reported the odds ratio estimates and 95% confidence intervals.

Results

At 6 and 12 months, respectively, 89.6% (69/77) and 63.7% (49/77) of patients “agreed” or “strongly agreed” that they would be willing to use ATA calls in the future. At 18 months, 51.0% (64/125) of patients perceived ATA calls as useful and 59.7% (46/77) were willing to use the technology. Moreover, in the first 6 months, most patients reported that ATA calls felt private/secure (75.9%, 82/108) and were easy to use (86.2%, 94/109), useful (65.1%, 71/109), and nonintrusive (87.2%, 95/109). Perceived usefulness, however, decreased to 54.1% (59/109) in the second 6 months of the trial. Factors predicting willingness to use ATA calls at the 18-month follow-up were perceived privacy/security and long-term perceived usefulness of ATA calls. No patient characteristics were significant predictors of long-term acceptance.

Conclusions

In the short term, patients are generally accepting of ATA calls for depression monitoring, with ATA call design and the care management intervention being primary factors influencing patient acceptance. Acceptance over the long term requires that the system be perceived as private/secure, and that it be constantly useful for patients’ needs of awareness of feelings, self-care reminders, and connectivity with health care providers.

Trial Registration

ClinicalTrials.gov NCT01781013; https://clinicaltrials.gov/ct2/show/NCT01781013 (Archived by WebCite at http://www.webcitation.org/6e7NGku56)

Introduction

In late 2014, the Centers for Medicare and Medicaid Services (CMS) issued new rules that expanded provider reimbursements beginning in 2015 for remote monitoring of Medicare beneficiaries [1]. Telemedicine and telehealth—or, what Kvedar and colleagues refer to collectively as “connected health”—capitalize on advances in health information technology (HIT) to remotely provide health care services, information, health education, and self-management support [2]. A number of studies have demonstrated the potential of these technologies to increase access and quality of care while decreasing health care costs [3-8].

Given the mounting evidence for the clinical and cost effectiveness of connected health, the CMS ruling is likely to boost interest in its adoption. Attempts to improve patient care with connected health, however, will be futile unless patients accept these technologies. Prior studies suggest that individuals who do not accept technologies simply will not use them [9,10]. Indeed, so critical is user acceptance that it has been regarded as “the pivotal factor in determining the success or failure of an information system” [11]. In an editorial review of connected health technologies to support behavior changes for self-management, Piette [12] remarks that patients’ discontinued use, which results from a lack of acceptance, has largely hindered large-scale implementation. Therefore, it is clear that patient acceptance has important implications for the broader domain of connected health, since patients who do not accept (and thus do not use) these technologies will not realize the full benefits of them, resulting in a loss for both patients and payers.

This study investigates patient acceptance of an automated telecommunications system designed to facilitate depression care management of low-income patients with diabetes in a safety-net care system [13,14]. There is evidence of significant disparities in receipt of depression treatment in low-income, uninsured, and minority populations. These groups are less likely to receive depression care [15-19], show greater treatment discontinuation [20], and experience higher rates of clinically significant depression. Patient barriers to depression care influence detection and treatment processes. For example, minority patients are less likely to voluntarily report depressive symptoms, may view depression as a moral weakness or character flaw instead of an illness, may be more likely to ascribe symptoms of depression to a physical illness [21,22], and may refuse or discontinue treatment due to stigma [23]. Nonadherence to depression treatment in minority groups with diabetes is common, due in part to side effects of diabetes medications [24-26]. Further exacerbating the challenges are cost and complex patient-provider interactions inherent in caring for patients with comorbid chronic illnesses. For instance, prioritizing among competing demands may negatively affect initiation and long-term follow-up of depression management in primary care [27-34].

To address these issues in order to accelerate the adoption of evidence-based depression care [24,35], we designed an advanced automated telephonic assessment (ATA) system. It had the capability to inquire—via periodic telephone calls to patients—about important aspects of depression care using a combination of the following six modules: monitoring for depression, assessing pain, assessing adherence to antidepressant medications, assessing psychotherapy practice, prompting depression self-care activities, and allowing patients to request contact from a clinician [14]. The ATA system was fully integrated into an existing disease management registry (DMR), which allowed it to automatically select these modules and the frequency of calls depending on individual patient clinical data: results from previous ATA calls or clinical assessments (depressed patients were called monthly, nondepressed patients quarterly), whether patients had an active antidepressant medication prescription, and whether patients were receiving psychotherapy. It also allowed patients to indicate their preferences for language (English or Spanish), call days and times, and receiving human calls instead of machine calls. If a call was not answered, the ATA system attempted again three times per day for 7 days (morning, afternoon, and evening). As a whole, the design allowed the ATA system to individually customize calls to focus on patients’ specific needs and preferences rather than having patients adapt to standard comprehensive assessments, in essence illustrating the philosophy of patient-centered care.

Moreover, the ATA system facilitated timely, proactive follow-up by clinicians and staff. Data captured on the ATA calls were automatically assessed and the results sent to the DMR for clinician and staff review [14]. Notifications, tasks, and alerts were triggered in response to specific issues identified from the ATA calls: patient requests for contact, high depression scores, nonadherence to antidepressant medications, or suicidal ideation.

The ATA system was tested in the Diabetes-Depression Care-management Adoption Trial (DCAT) [13]. DCAT was a 12-month, quasi-experimental comparative effectiveness research trial conducted in collaboration with the Los Angeles County Department of Health Services (LACDHS) with the aim of comparing different approaches for accelerating the adoption of collaborative team depression care in routine safety-net primary care practice. The study was conducted in its ambulatory care clinics serving low-income, racially/ethnically diverse (but primarily Hispanic/Latino) patients. It tested three depression care delivery models: usual care (UC), supported care (SC), and technology-facilitated care (TC). UC represented the status quo, whereby primary care providers (PCPs) and their staff initiate the translation and adoption of depression care evidence. Both SC and TC included care teams of the LACDHS disease management program (DMP) for the first 6 months of the trial to support diabetes care as well as depression care using evidence-based protocols [36]. After 6 months, patients returned to their PCPs for care. The difference between SC and TC was that the latter utilized the ATA system for 12 months to facilitate automated depression screening and monitoring, and timely follow-up by clinicians and staff. The provider notifications, tasks, and alerts generated by the ATA system were sent to DMP teams during the first 6 months of DCAT and to PCPs and their staff during the second 6 months.

If such automated remote screening and monitoring of depression—and more broadly, connected health—is to be integrated into mainstream health care delivery to help reach the important policy goal of expanding access to high-quality, effective, and efficient care, an understanding of patient technology acceptance is urgently needed. Studies on remote assessment and monitoring via connected health, however, continue to overlook this important research area [35,37]. Those that do touch upon elements of patient acceptance tend to be cross-sectional and operationalize the construct using measures of patient satisfaction with care, which in itself reveals little about technology acceptance or how to design the system to improve patient acceptance.

The present study echoes the information technology literature [38-40] by measuring technology acceptance as patients’ willingness to use ATA calls as part of their depression care. Moreover, this study is longitudinal, which allows for an understanding of how patient acceptance may change over time. Finally, to inform future design choices for automated remote depression monitoring technology, the evaluation includes several system characteristics that may explain why patients accept or reject the technology. Thus, in sum, the study was undertaken to (1) determine patient acceptance of ATA calls for remote depression screening and monitoring over time, and (2) identify what factors predict long-term patient acceptance of ATA calls.

Methods

Study Design and Participants

To answer the research questions, we analyzed data collected from patients in the TC arm of DCAT. English-Spanish bilingual interviewers administered assessments of technology acceptance at 6, 12, and 18 months. Thus, patients received two assessments during the study and one assessment 6 months after the study had ended.

Survey-Based Measures of ATA Call Acceptance

DCAT defined technology acceptance as patients’ reported willingness to use ATA calls in the future as part of their depression care. The measurement was administered at 6, 12, and 18 months. DCAT also assessed additional measures of ATA call design characteristics: perceived ease of use (7 items), perceived usefulness (6 items), perceived nonintrusiveness (3 items), and perceived privacy/security (1 item). DCAT administered these assessments at 6 and 12 months. Moreover, patients’ preference for mode of reach (1 item) was assessed at 6, 12, and 18 months. Finally, at 18 months, patients were asked about their long-term perceived usefulness of ATA calls (3 items). All measures were assessed on a 5-point Likert scale. Table 1 provides the exact wording used in the DCAT assessments.

Table 1.

Measures of patient ATA call acceptance.

Domain of measurement	Items	Administration
Willingness to use ATA callsa	To what extent do you agree or disagree with the following statement?	6, 12, and 18 months
	You would not mind receiving automated calls as part of your depression care in the future.
Perceived ease of useb,c	“How often would you say...”	6 and 12 months
	The language used by Amyc in the calls was easy for you to understand?
	Amy’s voice on the call was loud enough to hear without straining?
	Amy was speaking too fast on the automated call?
	You were clear on how to respond to Amy’s questions?
	You had difficulty answering the questions when asked to press buttons on your phone?
	Giving answers to a real person would have been easier than giving answers to the automated operator Amy?
	Amy had difficulty understanding you when you responded verbally?
Perceived usefulnessb,c	“How often would you say...”	6 and 12 months
	The call made you feel confident that your nurse or social worker knew how you were doing?
	The calls made you feel like your nurse of social worker was more accessible?
	The calls by Amy were just as effectiveness in reporting your feelings as an in-person visit with your care provider?
	The antidepressant medication questions asked by Amy reminded you to take your medications?
	The problem-solving skills questions asked by Amy reminded you to use these skills?
	The calls reminded you to do things like a physical activity or a fun activity?
Perceived nonintrusivenessb	“How often would you say...”	6 and 12 months
	You enjoyed receiving the calls?
	You felt the calls were a bother?
	The length of the calls seemed about right?
Perceived privacy/securitya	To what extent do you agree or disagree with the following statement?	6 and 12 months
	You feel automated calls are private and/or secure.
Preferred mode of reach	To what extent do you agree or disagree with the following statement?	6, 12, and 18 months
	Instead of receiving automated calls, you would prefer to call the automated service at your convenience.
Long-term perceived usefulnessa	To what extent do you agree or disagree with the following statements?	18 months
	The automated calls helped you be more aware of how you are feeling.
	The automated calls reminded you to take care of your health, such as doing exercise.
	The automated calls helped you stay better connected with your doctors, nurses or social worker.

^aPatients responded using a 5-point Likert scale of agreement (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, and 5=strongly agree).

^bPatients responded using a 5-point Likert scale of frequency (1=never, 2=rarely, 3=about half the time, 4=usually, and 5=always).

^c“Amy” was the persona of the ATA calls

ATA Call Completion Rates

We assessed the rate of completed ATA calls for three periods: 0-6 months, 7-12 months, and 0-12 months. An ATA call was defined as complete if it reached the patient and recorded answers to the depression assessment questions: PHQ-2 or PHQ-9, whichever was asked.

Statistical Analysis

We conducted two analyses: one to determine patient acceptance of ATA calls for remote depression screening and monitoring over time, and the other to identify what factors predict long-term patient acceptance of ATA calls. Sample characteristics and sample sizes for each analysis are shown in the Results section (Table 2).

Table 2.

Patient characteristics for samples in the two analyses (no statistically significant difference between the two samples).

Characteristic	Sample for first analysis		Sample for second analysis
	N	Statisticsa	N	Statisticsa
Female	109	72 (66.1%)	125	80 (64.0%)
Age	109	51.94 (9.01)	125	51.31 (8.81)
Hispanic/Latino	109	105 (96.3%)	125	116 (92.8%)
Spanish as preferred language	109	93 (85.0%)	125	104 (83.2%)
Married	109	49 (45.0%)	125	55 (44.0%)
PHQ-9 (range 0-27, higher=more severe depression)b,c	109	5.73 (4.93)	125	5.65 (4.60)
Total number of socioeconomic stressorsc	109	2.28 (1.56)	125	2.37 (1.46)
SCL-20, mean scorec,d	109	0.54 (0.53)	125	0.51 (0.48)
SF-12 mental (general population=50, higher=better)c,e	109	50.54 (9.15)	125	51.08 (9.03)
Time with diabetes in years	107	10.15 (7.42)	124	9.98 (7.05)
On insulin treatmentc	109	82 (75.2%)	125	89 (71.2%)
BMIc,f	109	32.93 (6.55)	125	32.75 (6.16)
A1C valuec,g	108	8.87 (1.39)	124	8.72 (1.39)
Low-density lipoprotein cholesterolc	108	167.08 (36.20)	124	168.44 (36.60)
Whitty-9 diabetes symptoms (range 1-5, 1=none to 5=every day)c	109	1.64 (0.54)	125	1.62 (0.49)
Number of diabetes complicationsc	109	1.26 (0.89)	125	1.22 (0.79)
Toolbert diabetes self-care in the past 7 days (range 0-7)c	109	4.63 (0.98)	125	4.65 (1.01)
Diabetes emotional burden (range 1-5, 1=not a problem to 5=very burdensome)c	109	2.53 (1.35)	125	2.48 (1.37)
Diabetes regime distress (range 1-5, 1=not a problem to 5=very burdensome)c	109	2.19 (1.14)	125	2.13 (1.17)
Self-rated health (range 1-5, 1=poor to 5=excellent)c	109	2.29 (0.60)	125	2.34 (0.60)
Chronic painc	109	17 (15.6%)	125	24 (19.2%)
SF-12 physical (general population=50, higher=better health)c,e	109	43.18 (9.62)	125	43.17 (9.49)
Sheehan disability scale (range 0-10, 0=none to 10=extremely)c	109	2.21 (2.34)	125	2.14 (2.26)
Number of ICD-9 diagnosisc,h	108	8.60 (4.50)	124	8.46 (4.46)
Number of clinic visitsc	107	10.44 (5.61)	124	10.56 (5.64)
Number of emergency room visitsc	41	1.33 (0.61)	44	1.33 (0.60)
Number of hospitalizationsc	15	1.47 (0.83)	18	1.39 (0.78)
Willingness to usec	109	4.02 (0.93)	125	4.00 (1.08)
Perceived ease of usec	109	4.05 (0.56)	125	4.12 (0.50)
Perceived usefulnessc	109	3.63 (0.89)	125	3.69 (0.90)
Perceived nonintrusivenessc	109	4.20 (0.87)	125	4.29 (0.84)
Perceived privacy/securityc	109	4.10 (1.11)	125	4.17 (1.08)
Preference of ATA call modec	109	3.82 (1.06)	125	3.58 (1.32)
Long-term perceived usefulness	76	3.71 (0.92)	125	3.74 (0.99)
ATA call completion ratec	108	0.70 (0.26)	123	0.74 (0.24)

^aValues are numbers (column percentages) for categorical variables and mean (SD) for continuous variables.

^bPatient Health Questionnaire, 9 items

^cAssessment at 6 or 12 months. If both were available, then the average was taken.

^dSymptoms CheckList, 20 items

^eShort-Form Health Survey, 12 items

^fBody mass index

^gGlycated hemoglobin test

^hInternational Classification of Diseases, 9th revision

For the first analysis (patient acceptance over time), we included the DCAT TC arm patients who provided responses for a given survey-based measure at each of the measurement periods. By excluding patients who did not meet this criterion, we were able to estimate changes more accurately for each measure over time. We computed descriptive statistics of all measures. For those measures consisting of multiple items, we computed the average points across items and rounded the average to the nearest integer. Furthermore, we conducted a paired t test to determine if there was a significant difference between the ATA call completion rates from 0 to 6 months and from 6 to 12 months. We also used Spearman rank correlation to test the association between the ATA call completion rate of months 0 to 12 and the survey-based measures of ATA call acceptance.

In the second analysis (predictive factors), we used a different criterion to select patients from among the pool of TC arm patients: patients who responded to the question of willingness to use ATA calls at 18 months and at least once at 6 or 12 months or both. If patients answered the question at both 6 and 12 months, we computed the average for use in the analysis. The 125 patients in this sample were collapsed into two groups: those reporting “high” willingness to use ATA calls at 18 months (Likert scale response of 4 or higher) and those reporting “low” willingness to use ATA calls at 18 months (all other response categories). We compared the descriptive statistics for the two groups: patient sociodemographic characteristics, health conditions, health care utilization, and ATA call completion rate. We also compared their responses for perceived ease of use, perceived usefulness, perceived nonintrusiveness, perceived privacy/security, preference of ATA call mode, and long-term perceived usefulness. If patients completed assessments of these measures at both 6 and 12 months, we computed the average of the two for use in the analysis. To compare the two groups of patients, we used independent t tests for continuous variables and Pearson chi-square tests for categorical variables. Next, we jointly entered independent factors found to be significantly associated with 18-month willingness to use ATA calls at the univariate level into a logistic regression model with backward selection to identify predictive factors. Then, we performed a final logistic regression model with the identified significant predictive factors and reported the odds ratio estimates and 95% confidence intervals. All analyses were conducted at 0.05 significance level (2-tailed) using IBM SPSS software, version 22.0.

Results

Sample Characteristics

Table 2 provides the characteristics of patients in the two samples used in the two analyses. The majority of patients were female, Hispanic/Latino, and preferred Spanish as their primary language. The characteristics of the two samples were not significantly different from one another. A comparison of these samples with the rest of the patients in DCAT TC excluded from the analyses did reveal significant differences in characteristics (see Tables A-1 and A-2 in Multimedia Appendix 1). Compared to the rest of DCAT TC, the two samples had a greater proportion of Hispanics/Latinos, reported a higher willingness to use ATA calls at 6 and 12 months, and had a higher ATA call completion rate. The sample for the second analysis also had lower blood sugar values, better diabetes self-care, and reported higher perceived ease of use and perceived nonintrusiveness at 6 and 12 months compared to the rest of patients in the TC arm of DCAT.

First Analysis: Patient Acceptance of ATA Calls Over Time

Figure 1 illustrates patient acceptance of ATA calls over time. In the first 6 months of the trial, 90% (69/77) of patients reported a high willingness to use ATA calls. At 12 and 18 months, however, the proportion of patients reporting a high willingness to use ATA calls decreased to 64% (49/77) and 60% (46/77), respectively. After 6 months in the trial, 83% (62/75) of patients agreed or strongly agreed that they would prefer to receive automated calls rather than calling the ATA system at their convenience. The proportion of patients reporting this decreased to 40% (30/75) and 29% (22/75) at 12 and 18 months, respectively. Throughout the trial, most patients agreed or strongly agreed that ATA calls felt private/secure (82/108 at 6 months, 89/108 at 12 months). At 6 months, 86.2% (94/109) of patients reported that ATA calls were usually or always easy to use. This number decreased to 78.0% (85/109) at 12 months. The proportion of patients reporting that ATA calls were usually or always useful decreased from 65.1% (71/109) at 6 months to 54.1% (59/109) at 12 months. At the 18-month follow-up, 51.0% (64/125) of patients agreed or strongly agreed that the ATA calls were useful. At 6 months, most patients 87% (95/109) perceived that ATA calls were usually or always nonintrusive. More patients perceived the calls to be intrusive after 12 months in the trial as is evident from a decrease in the proportion of patients who reported otherwise 76% (83/109).

Figure 1.

Patient acceptance of ATA calls over time.

The ATA average call completion rate was 72.6% and 67.8% at 6 and 12 months, respectively—the difference between the two was not statistically significant (P=.10). In investigating the associations between the ATA call completion rate of months 0 to 12 (70.2%) and the various survey-based acceptance measures, only two measures were statistically significant: (1) perceived ease of use (Spearman correlation coefficient=0.25, P=.008) and (2) perceived nonintrusiveness (Spearman correlation coefficient=0.27, P=.004).

Second Analysis: Factors Predicting Patient Acceptance of ATA Calls

We compared patients who reported, at 18 months, a high willingness to use ATA with patients reporting low willingness to use ATA calls to determine how the two groups differed in terms of the various sociodemographic characteristics, health conditions, health care utilization, and ATA-related measures listed in Table 2. Table 3 provides results for characteristics where there was a statistically significant difference between the two groups. See Table A-3 in Appendix for full results.

Table 3.

Characteristics of patients reporting high versus low willingness to use ATA calls at 18 months.

Characteristic	High willingness to use ATA calls at 18 months		Low willingness to use ATA calls at 18 months		P b
	N	Statisticsa	N	Statisticsa
Toolbert diabetes self-care in the past 7 days (range 0-7)c	74	4.81 (0.95)	51	4.43 (1.05)	.03
Willingness to usec	74	4.17 (1.00)	51	3.75 (1.16)	.04
Perceived usefulnessc	74	3.84 (0.82)	51	3.49 (0.97)	.03
Perceived nonintrusivenessc	74	4.42 (0.65)	51	4.09 (1.03)	.05
Perceived privacy/securityc	74	4.42 (0.91)	51	3.81 (1.22)	.003
Long-term perceived usefulness	74	4.07 (0.91)	51	3.25 (0.91)	<.001

^aValues are numbers (column percentages) for categorical variables and mean (SD) for continuous variables.

^bTwo-sample t test

^cPatients’ response at 6 or 12 months. If patients provided responses at 6 and 12 months, then the average of these was used.

When we compared patients who reported a high versus a low willingness to use ATA calls at 18 months, we found six factors to be significantly associated with patients’ reported willingness to use ATA calls. Patients with a high willingness to use ATA calls at 18 months (1) had better diabetes self-care (P=.03) and (2) reported a higher willingness to use ATA calls while in the study (P=.04); they also reported (3) higher perceived usefulness (P=.03), (4) nonintrusiveness (P=.05), and (5) privacy/security (P=.003) while in the study. Moreover, patients who reported a high willingness to use ATA calls at 18 months also reported (6) higher long-term perceived usefulness (P<.001). We jointly entered the six factors into a logistic regression model with backward selection to identify predictive factors. The results revealed that two factors jointly predicted willingness to use ATA calls at 18 months: perceived privacy/security (odds ratio OR=1.59, P=.017, 95% CI [1.09, 2.33]) and long-term perceived usefulness (OR=2.77, P<.001, 95% CI [1.65, 4.63]).

Discussion

Principal Findings

The promises of connected health to efficiently improve access and quality of care [2], rest upon the assumption that patients will readily accept the technologies. Our study on safety-net patient acceptance of automated depression screening and monitoring using ATA calls has important findings suggesting that assumption may be questionable. In the first 6 months of the trial, most patients were accepting of ATA calls and perceived the calls to be private/secure, easy to use, useful, and nonintrusive. Over time, however, patients’ acceptance and their positive perception of ATA call characteristics decreased—although call completion rates remained high and steady. One explanation may be that since ATA call results and prompts for follow-up were sent to DMP care teams during the first 6 months of the trial and to PCPs thereafter, timely follow-up by the latter might have been challenging due to their busy practice loads. Thus, although patients continued to complete ATA calls in the second half of the trial, their PCPs may not have responded to their needs in a timely manner thereby leading them to doubt the value of ATA calls. Furthermore, patients’ acceptance and their perception of ATA call characteristics may also reflect an improvement in their depressive symptoms over time. That is, patients with improved depressive symptoms—due, possibly, to the intervention itself—may no longer perceive the benefits of the ATA calls. We investigated this hypothesis and found that there was no statistically significant difference in the percentage of patients reporting high usefulness and high willingness to use ATA calls among those with improved symptoms, no change in symptoms, or worse symptoms. It may be, however, that our sample size was not large enough to detect these differences.

Another important finding in our study was the identification of two factors that significantly predicted patients’ long-term acceptance of ATA calls: the perception that ATA calls are private/secure and the long-term perceived usefulness of ATA calls. These two factors could be potentially modified to improve patients’ willingness to use ATA calls as part of their depression care.

Limitations

This study has limitations worth noting. First, we used two different samples for the analyses. For the sample used to determine patient acceptance of ATA calls over time, we included only those patients in the DCAT TC arm who responded to ATA-related measures at each of the corresponding measurement periods. For the sample used to identify factors that predict long-term patient acceptance of ATA calls, we included only those patients in the TC arm who answered the question on willingness to use ATA calls at 18 months and at least once at 6 or 12 months. We chose to accommodate two sample sizes for our study in order to maximize the sample sizes for both analyses, although this may have introduced additional bias.

Second, although the two different samples for the analyses were not significantly different from each other, they were both somewhat different from the rest of TC patients who were excluded from the analyses because they did not answer any of the ATA-related questions. Samples used in the analyses reported a slightly higher willingness to use ATA calls at 6 and 12 months than TC patients excluded from the analyses. However, it is not likely that this limitation affected our findings because only a small percentage (about 10%) of patients excluded from the analyses refused to engage with ATA calls. Nearly 90% of them reported that they could not answer the ATA-related questions because they did not receive or did not remember receiving ATA calls, or they received calls but did not answer because they were unavailable.

Furthermore, the small sample size of 125 patients reporting on willingness to use ATA calls limits the robustness of our findings of factors predicting long-term patient acceptance of ATA calls. Future studies should validate the generalizability of our findings.

A final limitation is that in the analysis of factors predicting long-term acceptance, we defined acceptance as patients’ self-reported willingness to use ATA calls at 18 months instead of using a more objective measure such as ATA call completion rate. This may seem to be a better indicator of patient acceptance, but since we were interested in learning about patients’ long-term acceptance, we did not have the ATA call completion rate at 18 months (the intervention was only for 12 months). Moreover, in our qualitative study of DCAT TC patients with incomplete ATA calls, we discovered that patients were actually willing to take the ATA calls, but were unable to do so mainly because of nonintervention related reasons, including not being available, the ATA system having the wrong phone number, or experiencing connection issues [41]. For this reason, we assumed that if patients did not complete ATA calls, it was not due to a lack of acceptance. Therefore, given the DCAT study design and the practical reasons for patients not answering ATA calls, we chose to follow the Patient Technology Acceptance Model (PTAM) [39] and define acceptance as self-reported willingness (ie, intention) to use the technology.

Comparison With Prior Work

The finding that patients are generally accepting of ATA calls, albeit in the short term, is a promising start to our understanding of patients’ perception of such technologies. Because automated depression screening and monitoring technology is emerging, little is known about patients’ acceptance of it. Related studies of connected health technologies [42], including those focused on depression care [37,43-54], uncritically regard acceptance as patient satisfaction with care, which tells us little about why patients accept or reject the technology or how system design features affect patient acceptance. This study is significant in the connected health literature for depression care in that it utilizes measures from the literature of user acceptance of new technologies [11,55,56]. These user acceptance measures allow us to derive new knowledge that helps not only to explain why the ATA system is acceptable or not to patients, but also to understand how we may improve patient acceptance through the design of the system.

Numerous studies on connected health applications have reported a drop in technology usage over time [57-67]. Unlike these studies, we found that patients’ completion of ATA calls was high and constant throughout the trial. As mentioned above, the main reasons patients reported for incomplete ATA calls were not related to the intervention [41]. In fact, we found in an analysis not included in this paper that the survey-based measures of acceptance were not statistically significant predictors of ATA call completion rates. Nonetheless, as reported in the Results section, the ATA call completion rate was positively correlated with perceived ease of use and perceived nonintrusiveness. The significance of the former factor is in agreement with the PTAM. However, the finding that patients continued to complete ATA calls over time despite a general decrease in acceptance is surprising. Future research is needed to determine whether it was the special characteristics of the study population (ie, predominantly urban, low-income Hispanics/Latinos) or the technology design (ie, outbound calls to patients) that resulted in this finding.

The PTAM sheds light on factors that increase the likelihood that patients will be willing to use connected health technologies. Among a myriad of potential factors, the main ones predicting patient acceptance are perceived usefulness, perceived ease of use, subjective norm, and health care knowledge. Others have similarly reported that perceived usefulness and perceived ease of use are the main driving forces of patient technology acceptance [11,68,69]. Likewise, we found that long-term perceived usefulness of ATA calls significantly predicted patient acceptance of automated depression screening and monitoring. A new predictor of acceptance suggested in our analysis was patients’ perception that calls were private/secure. Future patient technology acceptance research should consider this factor in the technology design and should validate the finding.

Conclusions

In the short term, safety-net ambulatory care patients with diabetes are generally accepting of ATA calls for depression screening and monitoring. How patient acceptance can be sustained over time is an important topic for future investigation. In order to increase the odds that patients will accept ATA calls over the long term, especially for sensitive mental health conditions, the system should gauge patient perception of its privacy/security. Moreover, it is critically important that the technology not only be aligned with patients’ needs, but also be perceived as useful for them over the long term. Based on the items measuring long-term usefulness, future research should focus on designing and testing technologies that help patients be more aware of how they are feeling, remind them to take care of their health, and help them stay better connected with their health care providers.

Abbreviations

ATA

automated telephonic assessment

BMI

body mass index

CMS

Centers for Medicare and Medicaid Services

DCAT

Diabetes-Depression Care-management Adoption Trial

DMP

disease management program

DMR

disease management registry

HIT

health information technology

ICD-9

International Classification of Diseases, 9th Revision

LACDHS

Los Angeles County Department of Health Services

PCP

primary care provider

PHQ

Patient Health Questionnaire

SC

supported care

SCL-20

Symptom Checklist, 20 items

SF-12

Short-form Health Survey, 12 items

TC

technology-facilitated care

UC

usual care