This randomized clinical trial assesses the effect of a comprehensive telehealth intervention vs a simpler telehealth approach for patients with persistently poorly controlled type 2 diabetes.
Key Points
Question
Compared with a simpler telehealth approach (telemonitoring and care coordination), can a practical, comprehensive telehealth intervention improve outcomes among patients whose type 2 diabetes remains persistently poorly controlled despite clinic-based care?
Findings
In this randomized clinical trial of 200 adults with persistently poorly controlled type 2 diabetes, hemoglobin A1c level improved by 1.59% at 12 months among those randomized to receive the comprehensive telehealth intervention, compared with 0.98% for the telemonitoring/care coordination group.
Meaning
A comprehensive telehealth intervention improved outcomes in persistently poorly controlled type 2 diabetes compared with a simpler telehealth intervention; because it was explicitly designed for feasible use in clinical practice, this approach may warrant implementation in systems that need to improve diabetes control in which the requisite infrastructure is available.
Abstract
Importance
Persistently poorly controlled type 2 diabetes (PPDM) is common and causes poor outcomes. Comprehensive telehealth interventions could help address PPDM, but effectiveness is uncertain, and barriers impede use in clinical practice.
Objective
To address evidence gaps preventing use of comprehensive telehealth for PPDM by comparing a practical, comprehensive telehealth intervention to a simpler telehealth approach.
Design, Setting, and Participants
This active-comparator, parallel-arm, randomized clinical trial was conducted in 2 Veterans Affairs health care systems. From December 2018 to January 2020, 1128 outpatients with PPDM were assessed for eligibility and 200 were randomized; PPDM was defined as maintenance of hemoglobin A1c (HbA1c) level of 8.5% or higher for 1 year or longer despite engagement with clinic-based primary care and/or diabetes specialty care. Data analyses were preformed between March 2021 and May 2022.
Interventions
Each 12-month intervention was nurse-delivered and used only clinical staffing/resources. The comprehensive telehealth group (n = 101) received telemonitoring, self-management support, diet/activity support, medication management, and depression support. Patients assigned to the simpler intervention (n = 99) received telemonitoring and care coordination.
Main Outcomes and Measures
Primary (HbA1c) and secondary outcomes (diabetes distress, diabetes self-care, self-efficacy, body mass index, depression symptoms) were analyzed over 12 months using intent-to-treat linear mixed longitudinal models. Sensitivity analyses with multiple imputation and inclusion of clinical data examined the impact of missing HbA1c measurements. Adverse events and intervention costs were examined.
Results
The population (n = 200) had a mean (SD) age of 57.8 (8.2) years; 45 (22.5%) were women, 144 (72.0%) were of Black race, and 11 (5.5%) were of Hispanic/Latinx ethnicity. From baseline to 12 months, HbA1c change was −1.59% (10.17% to 8.58%) in the comprehensive telehealth group and −0.98% (10.17% to 9.19%) in the telemonitoring/care coordination group, for an estimated mean difference of −0.61% (95% CI, −1.12% to −0.11%; P = .02). Sensitivity analyses showed similar results. At 12 months, patients receiving comprehensive telehealth had significantly greater improvements in diabetes distress, diabetes self-care, and self-efficacy; no differences in body mass index or depression were seen. Adverse events were similar between groups. Comprehensive telehealth cost an additional $1519 per patient per year to deliver.
Conclusions and Relevance
This randomized clinical trial found that compared with telemonitoring/care coordination, comprehensive telehealth improved multiple outcomes in patients with PPDM at a reasonable additional cost. This study supports consideration of comprehensive telehealth implementation for PPDM in systems with appropriate infrastructure and may enhance the value of telehealth during the COVID-19 pandemic and beyond.
Trial Registration
ClinicalTrials.gov Identifier: NCT03520413
Introduction
Patients with persistently poor type 2 diabetes (T2D) control disproportionately experience negative outcomes.1,2,3 We have defined persistently poorly controlled diabetes (PPDM) as maintenance of hemoglobin A1c (HbA1c) level of 8.5% or greater for more than 1 year despite receiving clinic-based diabetes care; 10% to 15% of all patients with T2D meet PPDM criteria.4,5 Given their high risk for complications and costs,1,2,3 patients with PPDM represent a compelling population for care delivery redesign.
Drivers of PPDM, including unavailable blood glucose data, medication nonadherence, suboptimal diet/activity, complex medications, and depression,6,7,8,9,10,11,12 can be difficult to address in the clinic setting.13,14 By facilitating contact outside of clinic, telehealth could improve outcomes in PPDM. Telehealth strategies targeting individual factors underlying poor T2D control reduce HbA1c level vs clinic-based care by 0.3% to 0.6%.15,16,17,18,19 Although such HbA1c changes may not suffice for patients with PPDM, combining multiple strategies into comprehensive telehealth interventions could produce greater improvement. However, multicomponent T2D interventions have achieved variable results.20,21,22,23,24,25
Beyond this uncertain effectiveness, other barriers have hindered implementation of comprehensive telehealth for PPDM in practice. Implementation barriers include intervention designs reliant on research-funded staff and resources, insufficient electronic health record (EHR) integration of patient data, and uncertain reimbursement.26,27,28,29 Before comprehensive telehealth can become a real-world solution for PPDM, approaches are needed that are unambiguously effective and also explicitly designed for feasible implementation. The upsurge in telehealth use during the COVID-19 pandemic has only strengthened the case for considering comprehensive telehealth as a means to address PPDM.30
We sought to address barriers to practical use of comprehensive telehealth for PPDM by evaluating a comprehensive telehealth intervention in a randomized clinical trial (RCT). This intervention combined 5 strategies targeting contributors to PPDM: telemonitoring, self-management support, diet/activity support, medication management, and depression support. To facilitate eventual implementation, we explicitly designed the intervention for delivery by clinical Veterans Health Administration (VHA) Home Telehealth (HT) nurses using existing clinical resources.
Methods
The protocol for this active-comparator, parallel-arm RCT (NCT03520413) has been published5 and appears in Supplement 1. We compared a practical, comprehensive telehealth intervention with a simpler telehealth approach, consisting of telemonitoring and care coordination, which is already available in VHA practice for T2D. A usual-care comparator was deemed inappropriate because clinic-based care is by definition insufficient for PPDM.31
This RCT was conducted at 2 VHA sites (Durham, North Carolina, and Richmond, Virginia). Institutional review boards at both sites approved the study. This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.
Population
We recruited patients with PPDM, defined as the following: diagnosed T2D (International Statistical Classification of Diseases and Related Health Problems, Tenth Revision E11); 2 or more HbA1c values of 8.5% or greater during the prior year with none less than 8.5%; and at least 1 appointment during the prior year with a primary care clinician or diabetes specialist (endocrinologist or other diabetes clinician). Exclusion criteria included the following: refusal to enroll in VHA HT (because both study interventions were delivered by HT nurses); factors making HbA1c reduction potentially inadvisable (age >70 years, metastatic cancer/life expectancy <5 years, recent cardiovascular disease complications, or prior hypoglycemic seizure/coma); lack of telephone access; dementia, psychosis, or substance use disorder; pregnancy; receiving dialysis or skilled nursing care; insulin pump use; or continuous glucose monitor use (unless also willing to submit self-monitored blood glucose [SMBG] data per HT protocol).
Race was determined by self-report and categorized as Asian, American Indian or Alaska Native, Black or African American, Native Hawaiian or Other Pacific Islander, White, other, or unknown. Because of low numbers in the other categories, race was ultimately presented as Black or African American, White, or other race. Self-reported ethnicity was assessed using a single question: “Are you of Latino/a or Hispanic origin or descent?” Race and ethnicity data were collected to facilitate generalizability assessment and to determine whether intervention effectiveness varied by these factors.
Recruitment and Enrollment
After EHR screening, a research assistant mailed opt-out letters to potential participants, then conducted phone screening. Eligible participants provided informed consent and underwent in-person baseline assessment; consented patients with a baseline HbA1c level of less than 8.5% were excluded before randomization. Given the high proportion of men in the VHA population, we oversampled women, aiming to achieve greater than 20% in the randomized population.
Randomization and Blinding
Randomization was stratified with blocks of 2; stratification variables were site, prior VHA HT use, and preenrollment diabetes specialty care (endocrinologist or other diabetes clinician). The computer-generated randomization sequence was accessible only to study statisticians. Patients received randomization assignments by phone from the project coordinator within 1 week of consent. Because participants received information about both interventions during consent, they were not blinded to randomization arm. Research assistants collecting outcome data were blinded to participant randomization status.
Interventions
Both 12-month interventions were delivered by clinical HT nurses rather than research staff. Although experienced with telehealth-based disease care, these nurses had no specialized diabetes training. Individual nurses delivered only 1 study intervention, with no crossover. Each intervention was delivered by 1 nurse in Durham, while in Richmond, 2 nurses delivered comprehensive telehealth and 5 delivered telemonitoring/care coordination.
Following randomization, all participants enrolled in VHA HT. Patients enrolled in HT used a telehealth device (Medtronic), blood glucose meter (Abbott), and connector cable; once connected to the blood glucose meter, the telehealth device automatically transmitted SMBG data to HT. After HT enrollment, participants began their assigned intervention; both groups also continued care with existing clinicians. Patients’ HbA1c goals were individualized per American Diabetes Association guidelines.32
Comprehensive Telehealth Intervention
This intervention comprised 5 nurse-delivered components (Figure 1)5: telemonitoring, self-management support, diet/activity support (together with a study dietitian), medication management (together with a study medication manager), and depression support (together with a study psychiatrist). The study dietitian, medication managers, and psychiatrists were clinicians, not research staff. Intervention nurses completed a single training session and received a manual. Nurses delivered the intervention to participants during 26 every-2-week telephone encounters, which the nurses scheduled directly with participants; nurses could additionally be reached for acute issues. Clinical information was tracked using templated EHR notes.
For the telemonitoring component, participants transmitted SMBG data up to 4 times daily based on their medication regimens but could monitor less frequently per nurse discretion. During each of the 26 scheduled encounters, the nurse reviewed SMBG data, reconciled medications, and assessed self-reported medication adherence.
For the self-management support component, intervention nurses delivered module-based self-management education during 16 of the 26 scheduled encounters. Each module covered a unique topic addressing knowledge and/or self-efficacy.5
For the diet/activity support component, a dietitian called participants with a body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) of 25 or greater to develop an individualized diet plan postenrollment. Plans were tailored to patient preferences and targeted to greater than 5% weight loss via a deficit of 500 to 750 calories per day.33 Patients were also encouraged to maintain 150 minutes or more of moderate to vigorous activity weekly.34 During each of the 26 scheduled encounters, the nurse reviewed progress. An additional dietitian phone follow-up could be arranged for patients not meeting goals.
For the medication management component, each site used 2 to 3 diabetes specialists (physicians, clinical pharmacists, or nurse practitioners). After each of the 26 scheduled encounters, the intervention nurse forwarded an EHR-based summary note to the medication manager. The medication manager considered treatment changes with guidance from a medication protocol, which targeted a fasting glucose level of 90 to 150 mg/dL and preprandial glucose level of 140 to 180 mg/dL (tailoring permitted based on HbA1c goal/hypoglycemia; to convert glucose level to mmol/L, multiply by 0.0555). The medication manager conveyed recommendations via an EHR note addendum, which the nurse implemented; medication managers did not routinely contact participants. Primary clinicians were alerted to changes via the EHR.
For the depression support component, participants with a Personal Health Questionnaire-8 (PHQ-8) score of 10 or higher at baseline or subsequent screening entered the depression protocol, which was supported by 1 psychiatrist at each site and provided pharmacologic and nonpharmacologic options.35 All patients receiving depression support had PHQ-8 follow-up every 8 weeks, with treatment changes as needed.
Telemonitoring/Care Coordination Intervention
Participants transmitted SMBG data and received automated self-management information daily by phone. Participants received nurse calls for alert SMBG values and could reach nurses as needed for acute issues but did not complete scheduled encounter calls. Participants also received care coordination, including communication about upcoming appointments, notification of primary clinicians regarding acute needs, and preappointment compilation of SMBG data for review by primary clinicians. Diabetes medication management was not formally integrated into this intervention; instead, medication adjustments were at the discretion of existing clinicians during or outside of scheduled encounters. Of note, because telemonitoring/care coordination is considered routine HT practice, no training was required for nurses delivering this intervention.
Outcomes
Outcome assessments were performed by blinded research assistants at 3-month intervals for 12 months. Full assessments were competed at 0, 6, and 12 months, with additional HbA1c-only assessments at 3 and 9 months.
The primary outcome was HbA1c level. Secondary outcomes were diabetes distress,36 diabetes self-care,37 self-efficacy,38 BMI, and depression symptoms.39 Adverse events were assessed by structured self-report40; incidence of blood glucose level less than 70 mg/dL was also examined using SMBG data transmitted to HT in both arms.
Intervention Costs
Intervention costs were examined in both arms. Labor costs included all intervention nurse, dietitian, medication manager, and psychiatrist time spent delivering the intervention; capital costs included HT equipment, telephone service costs, overhead, and supplies.
Fidelity Assessment
Fidelity assessment for the comprehensive telehealth intervention included nurse tracking of encounters and time using online software. The principal investigator and project coordinator conducted periodic shadowing of nurses and biannual case review meetings with the medication managers.5 Updated medications were tracked at outcome visits.
Influence of COVID-19 Pandemic on Outcome Ascertainment
In March 2020, VHA announced a pandemic-related restriction on in-person research interactions. Survey data collection continued by phone when possible, but study measurement of HbA1c level and BMI was interrupted. In May 2020, we received permission to resume collection of HbA1c level given its importance to diabetes management; however, BMI could not be measured after March 2020, which affected the 12-month time point for 169 participants. Despite the pandemic, intervention delivery continued uninterrupted at both sites.
Statistical Analyses
Sample Size
Per previous data,31 we used an α level of .05, 80% power, 20% dropout, within-patient correlation 0.55, SD 1.6, and baseline HbA1c level of 10.3% to estimate that 100 participants per arm would detect a clinically significant HbA1c difference of 0.6% at 12 months.41 Power estimates were derived by generating 1000 stimulated data sets with these assumptions and fitting linear mixed models to assess the effect difference at 12 months.
Analytic Approach
All analyses were intention-to-treat and performed using SAS, version 9.4 (SAS Institute).42 Linear mixed longitudinal models were used for all primary and secondary outcomes.43 The primary outcome model included fixed effects for linear, quadratic and cubic time, time-by-arm interaction terms, and randomization stratification variables, and random effects for intercept and linear time (see eMethods in Supplement 2). The covariance structure was determined using Akaike information criteria.44 Our primary inference was on the estimated between-arm 12-month HbA1c difference. As a post hoc sensitivity analysis, we included baseline covariates with between-arm differences in the primary model. Also, to explore a dose-response effect of the comprehensive telehealth intervention, we conducted a descriptive post hoc analysis examining HbA1c change among participants completing more than 20 vs 20 or fewer encounters.
For secondary outcomes, fixed effects included dummy-coded time effects for each time point and time-by-arm interaction terms. Given the pandemic’s hindrance of BMI ascertainment, BMI was analyzed only at 0 and 6 months. To account for within-participant repeated measures, we fit an unstructured covariance. We descriptively analyzed intervention engagement (encounter completion, SMBG transmission), adverse events, and costs.
Missing Data
Our analyses implicitly accommodated missingness when related to prior outcome data or other baseline model covariates defined as missing at random (MAR). As a sensitivity analysis for the primary model, we also multiplied imputed missing HbA1c data using a Markov chain Monte Carlo algorithm incorporating additional variables to strengthen the MAR assumption (see eMethods in Supplement 2). With the pandemic’s influence on outcome ascertainment, we conducted another sensitivity analysis fitting our primary model with inclusion of additional clinical HbA1c measurements obtained from the EHR.
Data Monitoring Committee
Our data monitoring committee comprised the study statisticians (A.S.J. and C.J.C.) and 3 independent experts. This committee met at 6-month intervals during the study and examined recruitment, retention, randomization, adverse events, and outcomes.
Results
Participants, Retention, and Fidelity
Participants were enrolled from December 2018 through January 2020; participant contact concluded in January 2021. Of 1128 individuals assessed, 257 were consented, and 200 were randomized (Figure 2); most of those consented but not randomized were excluded for baseline HbA1c level of less than 8.5%. Randomized participants were similar to those declining participation (eTable 1 in Supplement 2). Of those randomized, 101 were allocated to comprehensive telehealth and 99 to telemonitoring/care coordination; all were analyzed in their randomized group, and there was no between-arm crossover.
Per Table 1, participants had baseline mean (SD) age of 57.8 (8.2) years, HbA1c level of 10.2% (1.3%), and BMI of 34.8 (6.7). A total of 45 (22.5%) participants were women; 144 (72.0%) were Black, 11 (5.5%) were Hispanic/Latinx, 42 (21.0%) were White, and 14 (7.0%) were of other race. Prior to enrollment, 136 (68.0%) participants had received diabetes specialty care, and 12 (6.0%) had received HT care. Baseline characteristics were generally balanced across arms; moderate between-arm imbalances were noted in race, medication use, BMI, and social support.
Table 1. Baseline Characteristics, Overall and Stratified by Study Arm.
Baseline characteristics | No. (%) | ||
---|---|---|---|
Overall (n = 200) | Comprehensive telehealth (n = 101) | Telemonitoring/care coordination (n = 99) | |
Demographic characteristics | |||
Age, mean (SD), y | 57.8 (8.2) | 57.7 (8.3) | 57.8 (8.0) |
Sex | |||
Female | 45 (22.5) | 24 (23.8) | 21 (21.2) |
Male | 155 (77.5) | 77 (76.2) | 78 (78.8) |
Race | |||
Black or African American | 144 (72.0) | 68 (67.3) | 76 (76.8) |
White | 42 (21.0) | 25 (24.8) | 17 (17.2) |
Other racea | 14 (7.0) | 8 (7.9) | 6 (6.0) |
Hispanic/Latino ethnicityb | 11 (5.5) | 6 (5.9) | 5 (5.1) |
Did not graduate high school | 57 (28.5) | 29 (28.7) | 28 (28.3) |
Currently married | 91 (45.5) | 45 (46.5) | 46 (45.5) |
Social supportc,d | 191 (95.5) | 98 (97.0) | 93 (93.9) |
Employed (full/part-time/self) | 90 (45.0) | 48 (47.5) | 42 (42.4) |
Study site | |||
Durham | 115 (57.5) | 57 (56.4) | 58 (58.6) |
Richmond | 85 (42.5) | 44 (43.6) | 41 (41.4) |
Years with diabetes, mean (SD) | 12.1 (7.7) | 12.1 (8.0) | 12.0 (7.5) |
Prior diabetes specialty care | 136 (68.0) | 69 (68.3) | 67 (67.7) |
Prior Home Telehealth enrollment | 12 (6.0) | 6 (5.9) | 6 (6.1) |
Clinical measures | |||
Baseline HbA1c, mean (SD), % | 10.2 (1.3) | 10.1 (1.2) | 10.2 (1.4) |
BMI, mean (SD) | 34.8 (6.7) | 34.5 (6.4) | 35.2 (7.0) |
Hypertension | 166 (83.0) | 81 (80.2) | 85 (85.9) |
Hyperlipidemiad | 171 (85.5) | 84 (83.2) | 87 (87.9) |
Tobacco use in past 6 mo | 30 (25.4) | 16 (30.2) | 14 (21.5) |
Metformin | 160 (80.0) | 78 (77.2) | 82 (82.8) |
Sulfonylurea | 83 (41.5) | 35 (34.7) | 48 (48.9) |
Thiazolidinedione | 14 (7.0) | 7 (6.9) | 7 (7.1) |
SGLT-2 inhibitor | 22 (11.0) | 15 (14.9) | 7 (7.1) |
GLP-1 receptor agonist | 25 (12.5) | 11 (10.9) | 14 (14.1) |
DPP-4 inhibitor | 4 (2.0) | 0 | 4 (4.0) |
Insulin use | 142 (71.0) | 78 (77.2) | 64 (64.6) |
Psychosocial measures | |||
Diabetes distress (DDS), mean (SD) | 1.9 (0.8) | 1.9 (0.7) | 1.9 (0.9) |
Diabetes self-care (DSMQ), mean (SD) | 6.7 (1.6) | 6.9 (1.5) | 6.5 (1.7) |
Self-efficacy (PCS), mean (SD) | 5.2 (1.5) | 5.2 (1.5) | 5.2 (1.4) |
Depression (PHQ-8) score, mean (SD)e | 7.3 (5.7) | 7.0 (5.2) | 7.6 (6.1) |
Abbreviations: BMI, body mass index, calculated as weight in kilograms divided by height in meters squared; DDS, Diabetes Distress Scale; DPP-4, dipeptidyl peptidase-4; DSMQ, Diabetes Self-Management Questionnaire; GLP-1, glucagon-like peptide-1; HbA1c, hemoglobin A1c; PCS, Perceived Competence Scale; PHQ-8, Patient Health Questionnaire-8; SGLT-2, sodium-glucose cotransporter-2.
Because of low numbers in the Asian, American Indian or Alaska Native, Native Hawaiian or Other Pacific Islander, other, and unknown categories, these were combined into a single category, “other race.”
One patient in the telemonitoring/care coordination group responded “Don’t know” to the Hispanic/Latino ethnicity question.
Social support was assessed by asking, “Do you have someone you feel close to, someone you can trust and confide in?”
One patient in the comprehensive telehealth group responded “Don’t know” to having high cholesterol and to having social support.
One patient in the comprehensive telehealth group and 1 patient in the telemonitoring/care coordination group were missing the PHQ-8 score.
Overall, 150 of 200 (75%) participants completed the 12-month assessment for HbA1c level, and 137 (68.5%) for survey-based outcomes (Figure 2). Participants in the comprehensive telehealth arm completed an average of 19.6 of 26 possible encounters; 33 participants completed 20 or fewer encounters, and 14 completed 10 or fewer. Mean (SD) encounter time was 17.0 (11.2) minutes. Telemonitoring, self-management support, diet/activity support, and medication management were delivered during all completed comprehensive telehealth encounters; 30 of 101 (29.7%) participants initiated depression support based on elevated PHQ-8 score at baseline, and 23 of 90 (25.6%) at 6 months. Because the telemonitoring/care coordination intervention did not involve scheduled encounters, encounter metrics were not tracked. Both groups experienced changes in medication use during the study (eTable 2 in Supplement 2); descriptively, the comprehensive telehealth group had greater increases in use of glucagon-like peptide-1 receptor agonists (+15% vs +8%) and dipeptidyl peptidase-4 inhibitors (+5% vs +1%) from baseline to 12 months.
Primary Outcome
Between baseline and 12 months, estimated HbA1c change was −1.59% (10.17% to 8.58%) in the comprehensive telehealth group and −0.98% (10.17% to 9.19%) in the telemonitoring/care coordination group; the estimated difference of −0.61% (95% CI, −1.12% to −0.11%; P = .02) favored comprehensive telehealth (Table 2, Figure 3). Three-way interactions between intervention arm, time, and key stratification variables were not statistically significant, indicating no evidence for differential HbA1c effects over time based on preenrollment diabetes specialty care or study site.
Table 2. Estimated Outcome Means and Mean Differences for Comprehensive Telehealth (n = 101) and Telemonitoring/Care Coordination (n = 99) Arms by Time Pointa.
Outcome | Estimated mean | Estimated mean difference (95% CI) | P value | |
---|---|---|---|---|
Comprehensive telehealth | Telemonitoring/care coordination | |||
HbA1c, % | ||||
Baseline | 10.17 | 10.17 | NA | NA |
3 mo | 8.90 | 9.18 | −0.29 (−0.48 to −0.09) | NA |
6 mo | 8.54 | 9.03 | −0.48 (−0.78 to −0.18) | NA |
9 mo | 8.61 | 9.20 | −0.60 (−0.96 to −0.22) | NA |
12 mo | 8.58 | 9.19 | −0.61 (−1.12 to −0.11) | .02 |
Diabetes distress (DDS)b | ||||
Baseline | 1.93 | 1.93 | NA | NA |
6 mo | 1.53 | 1.57 | −0.04 (−0.18 to 0.09) | NA |
12 mo | 1.43 | 1.67 | −0.25 (−0.42 to −0.07) | .007 |
Diabetes self-care (DSMQ)c | ||||
Baseline | 6.67 | 6.67 | NA | NA |
6 mo | 8.15 | 7.92 | 0.22 (−0.07 to 0.51) | NA |
12 mo | 8.34 | 7.83 | 0.51 (0.25 to 0.78) | <.001 |
Self-efficacy (PCS)d | ||||
Baseline | 5.20 | 5.20 | NA | NA |
6 mo | 6.09 | 5.84 | 0.24 (−0.06 to 0.54) | NA |
12 mo | 6.31 | 5.92 | 0.39 (0.07 to 0.71) | .02 |
BMI | ||||
Baseline | 34.81 | 34.81 | NA | NA |
6 mo | 35.05 | 34.86 | 0.19 (−0.24 to 0.62) | .39 |
Depression symptoms (PHQ-8)e | ||||
Baseline | 7.32 | 7.32 | NA | NA |
6 mo | 6.54 | 6.06 | 0.48 (−0.72 to 1.69) | NA |
12 mo | 4.64 | 5.80 | −1.16 (−2.53 to 0.21) | .10 |
Abbreviations: BMI, body mass index, calculated as weight in kilograms divided by height in meters squared; DDS, Diabetes Distress Scale; DSMQ, Diabetes Self-Management Questionnaire; HbA1c, hemoglobin A1c; NA, not applicable; PCS, Perceived Competence Scale; PHQ-8, Patient Health Questionnaire-8.
Missing data by time point for the comprehensive telehealth group were as follows: HbA1c: 3 months n = 14, 6 months n = 19, 9 months n = 23, 12 months n = 22; survey measures: 6 months n = 12, 12 months n = 30; BMI: 6 months n = 34. Missing data by time point for the telemonitoring/care coordination group were as follows: HbA1c: 3 months n = 14, 6 months n = 12, 9 months n = 24, 12 months n = 26; survey measures: 6 months n = 10, 12 months n = 35; BMI: 6 months n = 33. No data points were missing at baseline.
A lower score on the DDS indicates lower levels of diabetes distress, so is preferred.
A higher score on the DSMQ indicates better diabetes self-care, so is preferred.
A higher score on the PCS indicates higher self-efficacy, so is preferred.
A lower score on the PHQ-8 indicates fewer depressive symptoms, so is preferred.
Similar results were found on sensitivity analyses with MAR imputation using multiply imputed data sets (mean 12-month difference, −0.63%; 95% CI, −0.95% to −0.35%; P = .03) and inclusion of additional clinical HbA1c measures from the study period (n = 191 from 62 comprehensive telehealth and 63 telemonitoring/care coordination participants; mean 12-month difference, −0.50%; 95% CI, −0.99% to −0.01%; P = .04). Findings with baseline covariate adjustment (race; insulin, metformin, sulfonylurea, sodium-glucose cotransporter-2 inhibitor use; BMI; social support) were also similar (mean 12-month difference, −0.66%; 95% CI, −1.17% to −0.14%; P = .01).
On exploratory descriptive analyses (eTable 3 in Supplement 2), comprehensive telehealth patients who completed more than 20 encounters (n = 68) experienced greater HbA1c reduction (1.84%) than those who completed 20 or fewer (n = 33, 0.79%).
Secondary Outcomes
Per Table 2, the comprehensive telehealth group had a greater improvement than the telemonitoring/care coordination group at 12 months for diabetes distress (mean difference, −0.25; 95% CI, −0.42 to −0.07), diabetes self-care (mean difference, 0.51; 95% CI, 0.25 to 0.78), and self-efficacy (mean difference, 0.39; 95% CI, 0.07 to 0.71). There were no statistically significant between-group differences in depressive symptoms (12 months) or BMI (6 months).
Adverse Events
Adverse events were similar between arms. Comprehensive telehealth participants had 26 serious events (16 hospitalizations), 3 possibly study-related (episodes of ketoacidosis, hyperglycemia, and possible medication-related urinary infection). In the telemonitoring/care coordination group, there were 19 serious events (17 hospitalizations, 1 death), none deemed study-related. Among the 97 comprehensive telehealth participants with available SMBG data, 70 (72.1%) reported 1 or more SMBG values less than 70 mg/dL over 12 months, with a per-patient mean (SD) of 7.3 (9.8). Among the 89 telemonitoring/care coordination participants with available SMBG data, 61 (68.5%) reported 1 or more SMBG values less than 70 mg/dL over 12 months, with a per-patient mean (SD) of 6.9 (11.1).
Intervention Costs
Per-patient intervention costs were $2465 for comprehensive telehealth and $946 for telemonitoring/care coordination, for a between-arm difference of $1519 over 12 months.
Discussion
We sought to promote practical use of telehealth for PPDM by comparing 2 approaches designed for feasible clinical implementation: a comprehensive telehealth intervention and telemonitoring/care coordination. While both approaches improved HbA1c level, the comprehensive telehealth intervention produced a greater 12-month improvement in HbA1c level and multiple secondary outcomes, without excess hypoglycemia.
These findings demonstrate that practically designed telehealth can be effective for patients whose T2D remains persistently poorly controlled despite clinic-based care. Moreover, combining telehealth strategies to target multiple barriers to improvement lowers HbA1c level more than simpler approaches like telemonitoring/care coordination. Given our active-comparator design, we cannot exclude that the within-arm HbA1c effects (−1.59% for comprehensive telehealth and −0.98% for telemonitoring/care coordination) may partly reflect regression to the mean; for example, the −0.98% HbA1c improvement with telemonitoring/care coordination exceeds the effect reported for telemonitoring in systematic reviews (−0.4% to 0.5% vs usual care).15,45,46,47 However, the relative HbA1c benefit seen with comprehensive telehealth in this study (−0.61%) was not subject to regression to the mean. Importantly, the HbA1c reduction with comprehensive telehealth was durably retained through 12 months.
While the comprehensive telehealth intervention was more expensive (additional $1519 over 12 months), this incremental cost is less than most branded glucose-lowering medications, and the comprehensive approach came with added benefits for diabetes distress, self-care, and self-efficacy. Given the high complication rates characteristic of PPDM and the long-term cost benefits of HbA1c reduction,1,2 implementing comprehensive telehealth in practice may represent an appropriate investment for health care systems in which the requisite infrastructure is or can be made available.
When we developed this study, our focus was on leveraging the VHA’s telehealth infrastructure to generate solutions for PPDM. Since then, the COVID-19 pandemic has driven a dramatic upsurge in telehealth use worldwide. Currently, telehealth, as used by most systems, implies intermittent video or phone appointments,30,48,49,50 with limited interim patient–clinician contact or transfer of patient-generated data into the EHR—in essence, “clinic-like” care, only delivered remotely. Just as clinic-based care does not fully address the factors underlying PPDM, “clinic-like” telehealth is also likely inadequate. There has always been a sound argument for using comprehensive telehealth when clinic-based chronic disease care falls short; now that telehealth has gained wider acceptance, systems have a clear mandate to maximize its value for those high-risk patients who respond insufficiently to clinic care. The present study provides evidence supporting comprehensive telehealth for PPDM within the VHA, but also presents a template for how other systems might use existing resources to improve the management of PPDM and other hard-to-treat conditions.
While our focus on examining practical, comprehensive telehealth specifically for PPDM is novel, the findings also add to the broader telehealth literature in T2D. The between-arm HbA1c effect we observed (−0.61%) is notable given our active-comparator design and the relatively modest effects reported in recent systematic reviews of telehealth interventions (mean HbA1c benefit vs usual care, −0.4%).51,52 In particular, prior RCTs examining comprehensive interventions for T2D have not lowered HbA1c level20,21,22,23,24,25; these include studies that specifically sought to examine multicomponent interventions vs usual care in pragmatic settings (non-VHA), with neutral results.20,21,22
Limitations
Despite efforts to oversample women, the population demographics may limit generalizability. However, the cohort’s high proportion of Black participants (72.0%) is a strength and may suggest that the pandemic-induced shift to telehealth need not exacerbate health care inequities.53,54 The studied interventions were designed for practical delivery within the VHA, which may limit generalizability to systems lacking capacity for nurse-delivered telehealth, integrated mental health, and dietitian services; nevertheless, the idea of designing telehealth interventions to leverage available resources is broadly applicable. The VHA costs may not fully generalize, but the between-arm cost difference may be more translatable.
While clinical intervention delivery continued unimpeded during the pandemic, the missing data frequency was higher than expected at 12 months; however, the sensitivity analyses with multiple imputation and inclusion of clinical HbA1c data support the validity of the findings. Of note, participants could not be blinded to randomization status, which leaves potential for bias, especially pertaining to subjective survey measures.
This study was not designed to evaluate the effectiveness of each individual component of the comprehensive intervention. Future mediator analyses will examine how each component contributes to the overall intervention effect. Finally, the comprehensive telehealth intervention does not account for all contributors to PPDM, including social determinants of health.
Conclusions
Findings from this randomized clinical trial showed that compared with a simpler telehealth approach, a comprehensive telehealth intervention improved HbA1c level and other outcomes in patients with PPDM. Because this comprehensive telehealth intervention was delivered by clinical staff using existing resources, it may warrant clinical implementation in systems with appropriate infrastructure. More broadly, this study provides valuable comparative evidence that may help systems maximize the value of telehealth during the COVID-19 pandemic and beyond.
References
- 1.Gilmer TP, O’Connor PJ, Rush WA, et al. Predictors of health care costs in adults with diabetes. Diabetes Care. 2005;28(1):59-64. doi: 10.2337/diacare.28.1.59 [DOI] [PubMed] [Google Scholar]
- 2.McBrien KA, Manns BJ, Chui B, et al. Health care costs in people with diabetes and their association with glycemic control and kidney function. Diabetes Care. 2013;36(5):1172-1180. doi: 10.2337/dc12-0862 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Stratton IM, Adler AI, Neil HA, et al. Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observational study. BMJ. 2000;321(7258):405-412. doi: 10.1136/bmj.321.7258.405 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Alexopoulos AS, Jackson GL, Edelman D, et al. Clinical factors associated with persistently poor diabetes control in the Veterans Health Administration: a nationwide cohort study. PLoS One. 2019;14(3):e0214679. doi: 10.1371/journal.pone.0214679 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kobe EA, Edelman D, Tarkington PE, et al. Practical telehealth to improve control and engagement for patients with clinic-refractory diabetes mellitus (PRACTICE-DM): protocol and baseline data for a randomized trial. Contemp Clin Trials. 2020;98:106157. doi: 10.1016/j.cct.2020.106157 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Davidson J. Strategies for improving glycemic control: effective use of glucose monitoring. Am J Med. 2005;118(suppl 9A):27S-32S. doi: 10.1016/j.amjmed.2005.07.054 [DOI] [PubMed] [Google Scholar]
- 7.Donnelly LA, Morris AD, Evans JM; DARTS/MEMO collaboration . Adherence to insulin and its association with glycaemic control in patients with type 2 diabetes. QJM. 2007;100(6):345-350. doi: 10.1093/qjmed/hcm031 [DOI] [PubMed] [Google Scholar]
- 8.Hill-Briggs F, Gary TL, Bone LR, Hill MN, Levine DM, Brancati FL. Medication adherence and diabetes control in urban African Americans with type 2 diabetes. Health Psychol. 2005;24(4):349-357. doi: 10.1037/0278-6133.24.4.349 [DOI] [PubMed] [Google Scholar]
- 9.American Diabetes Association . 7. Obesity management for the treatment of type 2 diabetes. Diabetes Care. 2017;40(suppl 1):S57-S63. doi: 10.2337/dc17-S010 [DOI] [PubMed] [Google Scholar]
- 10.Chiu CJ, Wray LA. Factors predicting glycemic control in middle-aged and older adults with type 2 diabetes. Prev Chronic Dis. 2010;7(1):A08. [PMC free article] [PubMed] [Google Scholar]
- 11.Hartz A, Kent S, James P, Xu Y, Kelly M, Daly J. Factors that influence improvement for patients with poorly controlled type 2 diabetes. Diabetes Res Clin Pract. 2006;74(3):227-232. doi: 10.1016/j.diabres.2006.03.023 [DOI] [PubMed] [Google Scholar]
- 12.de Groot M, Anderson R, Freedland KE, Clouse RE, Lustman PJ. Association of depression and diabetes complications: a meta-analysis. Psychosom Med. 2001;63(4):619-630. doi: 10.1097/00006842-200107000-00015 [DOI] [PubMed] [Google Scholar]
- 13.Morrison F, Shubina M, Turchin A. Encounter frequency and serum glucose level, blood pressure, and cholesterol level control in patients with diabetes mellitus. Arch Intern Med. 2011;171(17):1542-1550. doi: 10.1001/archinternmed.2011.400 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rosenthal ES, Bashan E, Herman WH, Hodish I. The effort required to achieve and maintain optimal glycemic control. J Diabetes Complications. 2011;25(5):283-288. doi: 10.1016/j.jdiacomp.2011.01.002 [DOI] [PubMed] [Google Scholar]
- 15.Medical Advisory Secretariat . Home telemonitoring for type 2 diabetes: an evidence-based analysis. Ont Health Technol Assess Ser. 2009;9(24):1-38. [PMC free article] [PubMed] [Google Scholar]
- 16.Medical Advisory Secretariat . Behavioural interventions for type 2 diabetes: an evidence-based analysis. Ont Health Technol Assess Ser. 2009;9(21):1-45. [PMC free article] [PubMed] [Google Scholar]
- 17.Kempf K, Altpeter B, Berger J, et al. Efficacy of the telemedical lifestyle intervention program TeLiPro in advanced stages of type 2 diabetes: a randomized controlled trial. Diabetes Care. 2017;40(7):863-871. doi: 10.2337/dc17-0303 [DOI] [PubMed] [Google Scholar]
- 18.Pimouguet C, Le Goff M, Thiébaut R, Dartigues JF, Helmer C. Effectiveness of disease-management programs for improving diabetes care: a meta-analysis. CMAJ. 2011;183(2):E115-E127. doi: 10.1503/cmaj.091786 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Atlantis E, Fahey P, Foster J. Collaborative care for comorbid depression and diabetes: a systematic review and meta-analysis. BMJ Open. 2014;4(4):e004706. doi: 10.1136/bmjopen-2013-004706 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lauffenburger JC, Ghazinouri R, Jan S, et al. Impact of a novel pharmacist-delivered behavioral intervention for patients with poorly-controlled diabetes: the ENhancing outcomes through Goal Assessment and Generating Engagement in Diabetes Mellitus (ENGAGE-DM) pragmatic randomized trial. PLoS One. 2019;14(4):e0214754. doi: 10.1371/journal.pone.0214754 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Sarayani A, Mashayekhi M, Nosrati M, et al. Efficacy of a telephone-based intervention among patients with type-2 diabetes: a randomized controlled trial in pharmacy practice. Int J Clin Pharm. 2018;40(2):345-353. doi: 10.1007/s11096-018-0593-0 [DOI] [PubMed] [Google Scholar]
- 22.Edelman D, Dolor RJ, Coffman CJ, et al. Nurse-led behavioral management of diabetes and hypertension in community practices: a randomized trial. J Gen Intern Med. 2015;30(5):626-633. doi: 10.1007/s11606-014-3154-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lee JY, Chan CKY, Chua SS, et al. Telemonitoring and team-based management of glycemic control on people with type 2 diabetes: a cluster-randomized controlled trial. J Gen Intern Med. 2020;35(1):87-94. doi: 10.1007/s11606-019-05316-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Crowley MJ, Powers BJ, Olsen MK, et al. The Cholesterol, Hypertension, And Glucose Education (CHANGE) study: results from a randomized controlled trial in African Americans with diabetes. Am Heart J. 2013;166(1):179-186. doi: 10.1016/j.ahj.2013.04.004 [DOI] [PubMed] [Google Scholar]
- 25.Tang PC, Overhage JM, Chan AS, et al. Online disease management of diabetes: engaging and motivating patients online with enhanced resources-diabetes (EMPOWER-D), a randomized controlled trial. J Am Med Inform Assoc. 2013;20(3):526-534. doi: 10.1136/amiajnl-2012-001263 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood). 2014;33(2):207-215. doi: 10.1377/hlthaff.2013.1054 [DOI] [PubMed] [Google Scholar]
- 27.Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93(8):1261-1267. doi: 10.2105/AJPH.93.8.1261 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare. 2018;24(1):4-12. doi: 10.1177/1357633X16674087 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016;11(1):146. doi: 10.1186/s13012-016-0510-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Wosik J, Fudim M, Cameron B, et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc. 2020;27(6):957-962. doi: 10.1093/jamia/ocaa067 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Crowley MJ, Edelman D, McAndrew AT, et al. Practical telemedicine for veterans with persistently poor diabetes control: a randomized pilot trial. Telemed J E Health. 2016;22(5):376-384. doi: 10.1089/tmj.2015.0145 [DOI] [PubMed] [Google Scholar]
- 32.Draznin B, Aroda VR, Bakris G, et al. ; American Diabetes Association Professional Practice Committee . 6. Glycemic targets: Standards of Medical Care in Diabetes—2022. Diabetes Care. 2022;45(S1)(suppl 1):S83-S96. [DOI] [PubMed] [Google Scholar]
- 33.Draznin B, Aroda VR, Bakris G, et al. ; American Diabetes Association Professional Practice Committee . 8. Obesity and weight management for the prevention and treatment of type 2 diabetes: Standards of Medical Care in Diabetes—2022. Diabetes Care. 2022;45(S1)(suppl 1):S113-S124. [DOI] [PubMed] [Google Scholar]
- 34.Draznin B, Aroda VR, Bakris G, et al. ; American Diabetes Association Professional Practice Committee; American Diabetes Association Professional Practice Committee . 5. Facilitating behavior change and well-being to improve health outcomes: Standards of Medical Care in Diabetes—2022. Diabetes Care. 2022;45(S1)(suppl 1):S60-S82. [DOI] [PubMed] [Google Scholar]
- 35.US Department of Veterans Affairs . VA/DoD clinical practice guideline for the management of major depressive disorder. Accessed May 4, 2022. https://www.healthquality.va.gov/guidelines/MH/mdd/VADoDMDDCPGFINAL82916.pdf [DOI] [PubMed]
- 36.Polonsky WH, Fisher L, Earles J, et al. Assessing psychosocial distress in diabetes: development of the diabetes distress scale. Diabetes Care. 2005;28(3):626-631. doi: 10.2337/diacare.28.3.626 [DOI] [PubMed] [Google Scholar]
- 37.Schmitt A, Gahr A, Hermanns N, Kulzer B, Huber J, Haak T. The Diabetes Self-Management Questionnaire (DSMQ): development and evaluation of an instrument to assess diabetes self-care activities associated with glycaemic control. Health Qual Life Outcomes. 2013;11:138. doi: 10.1186/1477-7525-11-138 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Williams GC, Freedman ZR, Deci EL. Supporting autonomy to motivate patients with diabetes for glucose control. Diabetes Care. 1998;21(10):1644-1651. doi: 10.2337/diacare.21.10.1644 [DOI] [PubMed] [Google Scholar]
- 39.Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009;114(1-3):163-173. doi: 10.1016/j.jad.2008.06.026 [DOI] [PubMed] [Google Scholar]
- 40.Bent S, Padula A, Avins AL. Brief communication: Better ways to question patients about adverse medical events: a randomized, controlled trial. Ann Intern Med. 2006;144(4):257-261. doi: 10.7326/0003-4819-144-4-200602210-00007 [DOI] [PubMed] [Google Scholar]
- 41.Sacks DB, Arnold M, Bakris GL, et al. ; National Academy of Clinical Biochemistry; Evidence-Based Laboratory Medicine Committee of the American Association for Clinical Chemistry . Guidelines and recommendations for laboratory analysis in the diagnosis and management of diabetes mellitus. Diabetes Care. 2011;34(6):e61-e99. doi: 10.2337/dc11-9998 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Little R, Kang S. Intention-to-treat analysis with treatment discontinuation and missing data in clinical trials. Stat Med. 2015;34(16):2381-2390. doi: 10.1002/sim.6352 [DOI] [PubMed] [Google Scholar]
- 43.Fitzmaurice GM, Laird NM, Ware JH. Applied Longitudinal Analysis. 2nd ed. Wiley; 2011. doi: 10.1002/9781119513469 [DOI] [Google Scholar]
- 44.Akaike H. A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19(6):716-723. doi: 10.1109/TAC.1974.1100705 [DOI] [Google Scholar]
- 45.Zhu X, Williams M, Finuf K, et al. Home telemonitoring of patients with type 2 diabetes: a meta-analysis and systematic review. Diabetes Spectr. 2022;35(1):118-128. doi: 10.2337/ds21-0023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Kim Y, Park JE, Lee BW, Jung CH, Park DA. Comparative effectiveness of telemonitoring versus usual care for type 2 diabetes: a systematic review and meta-analysis. J Telemed Telecare. 2019;25(10):587-601. doi: 10.1177/1357633X18782599 [DOI] [PubMed] [Google Scholar]
- 47.Lee PA, Greenfield G, Pappas Y. The impact of telehealth remote patient monitoring on glycemic control in type 2 diabetes: a systematic review and meta-analysis of systematic reviews of randomised controlled trials. BMC Health Serv Res. 2018;18(1):495. doi: 10.1186/s12913-018-3274-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Baum A, Kaboli PJ, Schwartz MD. Reduced in-person and increased telehealth outpatient visits during the COVID-19 pandemic. Ann Intern Med. 2021;174(1):129-131. doi: 10.7326/M20-3026 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Heyworth L, Kirsh S, Zulman D, Ferguson JM, Kizer KW. Expanding access through virtual care: the VA’s early experience with COVID-19. NEJM Catalyst. July 1, 2020. Accessed January 18, 2022. https://catalyst.nejm.org/doi/full/10.1056/cat.20.0327
- 50.Artandi M, Thomas S, Shah NR, Srinivasan M. Rapid system transformation to more than 75% primary care video visits within three weeks at Stanford: response to public safety crisis during a pandemic. NEJM Catalyst. April 21, 2020. Accessed January 18, 2022. https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0100
- 51.Hangaard S, Laursen SH, Andersen JD, et al. The effectiveness of telemedicine solutions for the management of type 2 diabetes: a systematic review, meta-analysis, and meta-regression. J Diabetes Sci Technol. Published online December 26, 2021. doi: 10.1177/19322968211064633 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Faruque LI, Wiebe N, Ehteshami-Afshar A, et al. ; Alberta Kidney Disease Network . Effect of telemedicine on glycated hemoglobin in diabetes: a systematic review and meta-analysis of randomized trials. CMAJ. 2017;189(9):E341-E364. doi: 10.1503/cmaj.150885 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Litchfield I, Shukla D, Greenfield S. Impact of COVID-19 on the digital divide: a rapid review. BMJ Open. 2021;11(10):e053440. doi: 10.1136/bmjopen-2021-053440 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Eruchalu CN, Pichardo MS, Bharadwaj M, et al. The expanding digital divide: digital health access inequities during the COVID-19 pandemic in New York City. J Urban Health. 2021;98(2):183-186. doi: 10.1007/s11524-020-00508-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.