Abstract
Context
Telemedicine is a promising but largely unproven technology for providing case management services to patients with chronic conditions and lower access to care.
Objectives
To examine the effectiveness of a telemedicine intervention to achieve clinical management goals in older, ethnically diverse, medically underserved patients with diabetes.
Design, Setting, and Patients
A randomized controlled trial was conducted, comparing telemedicine case management to usual care, with blinded outcome evaluation, in 1,665 Medicare recipients with diabetes, aged ≥ 55 years, residing in federally designated medically underserved areas of New York State.
Interventions
Home telemedicine unit with nurse case management versus usual care.
Main Outcome Measures
The primary endpoints assessed over 5 years of follow-up were hemoglobin A1c (HgbA1c), low density lipoprotein (LDL) cholesterol, and blood pressure levels.
Results
Intention-to-treat mixed models showed that telemedicine achieved net overall reductions over five years of follow-up in the primary endpoints (HgbA1c, p = 0.001; LDL, p < 0.001; systolic and diastolic blood pressure, p = 0.024; p < 0.001). Estimated differences (95% CI) in year 5 were 0.29 (0.12, 0.46)% for HgbA1c, 3.84 (−0.08, 7.77) mg/dL for LDL cholesterol, and 4.32 (1.93, 6.72) mm Hg for systolic and 2.64 (1.53, 3.74) mm Hg for diastolic blood pressure. There were 176 deaths in the intervention group and 169 in the usual care group (hazard ratio 1.01 [0.82, 1.24]).
Conclusions
Telemedicine case management resulted in net improvements in HgbA1c, LDL-cholesterol and blood pressure levels over 5 years in medically underserved Medicare beneficiaries. Mortality was not different between the groups, although power was limited.
Trial Registration
http://clinicaltrials.gov Identifier: NCT00271739.
Introduction
Telemedicine holds great promise to improve health care delivery to underserved populations. 1 However, despite its use in several clinical settings and the publication of several evaluation studies, the clinical effectiveness of telemedicine remains largely unknown. 2 Telemedicine has many potential applications, but it may be particularly useful to improve health care delivery in chronic diseases, such as diabetes mellitus, and in assisting to overcome socio-economic, geographic, and weather-related barriers. The rationale for focusing on diabetes in the context of telemedicine arises from the high prevalence of the disease, its substantial morbidity and mortality, and its enormous cost. 3
There is widespread consensus regarding the need to improve the quality of diabetes care, but the goals are still far from being met. 4 This need is a particular concern in the uninsured 5 but extends to people with health insurance coverage. Medicare provides coverage to an estimated 14% of the United States population, and enables access to medical care for at risk populations. 6 Several studies have shown that the quality of diabetes care is not optimal among Medicare beneficiaries, and that quality indicators are poorer for women, racial minorities, rural residents, and patients with a lower socioeconomic status. 7–9 Therefore, there is a growing interest in case management interventions, including telemedicine, that may improve the quality of care provided to Medicare beneficiaries with diabetes. 10
The Informatics for Diabetes Education and Telemedicine (IDEATel) study was a randomized trial comparing telemedicine case management with usual care in older, ethnically diverse, medically underserved Medicare beneficiaries with diabetes mellitus residing in medically underserved areas of New York State. 11 At 1 year of follow up we observed differences of moderate magnitude favoring the intervention group in the three main outcomes of the trial, namely hemoglobin A1c (HgbA1c), low density lipoprotein (LDL) cholesterol, and blood pressure levels. 12 The IDEATel study was therefore extended to obtain longer term results. We report here our findings over 5 years of follow-up.
Methods
Design
The IDEATel study was conducted as a randomized controlled trial with blinded assessment of the outcomes. As previously described, 11,12 subjects were enrolled through primary care practices in New York City (NYC), with the enrollment hub at Columbia University Medical Center, and in upstate New York, where the enrollment hub was at State University of New York (SUNY) Upstate Medical University at Syracuse. The Institutional Review Boards at all participating institutions approved the study protocol. All participants provided informed consent. An independent Data and Safety Monitoring Board monitored the study to ensure participant safety and adherence to the protocol.
Sample
The sample consisted of 1,665 subjects residing in New York State, recruited and randomized between December 2000 and October 2002 (▶). Inclusion criteria were age 55 or older, being a current Medicare beneficiary, having diabetes as defined by a physician's diagnosis and being on treatment with diet, an oral hypoglycemic agent, or insulin, residence in a federally designated medically underserved area (either of two federal designations, Medically Underserved Area [MUA] or Health Professional Shortage Area [HPSA]), and fluency in either English or Spanish. Exclusion criteria were moderate or severe cognitive impairment, severe visual, mobility, or motor coordination impairment, severe comorbid condition, severe expressive or receptive communication impairment, lack of free electrical outlet for home telemedicine unit, and spending more than 3 months a year at a location different from their New York State residence.
Participants were recruited through their primary care provider (PCP) practice. The number of subjects per PCP ranged from 1 to 35 (mean, 3.27; SD, 4.40). Randomization to telemedicine case management or to usual care was assigned within primary care provider strata immediately upon completion of the baseline examination.
Intervention
The telemedicine intervention in IDEATel has been described in detail elsewhere. 11,13 Participants randomized to the intervention group received a home telemedicine unit (American Telecare, Inc, Eden Prairie, MN) consisting of a web-enabled computer with modem connection to an existing telephone line. The home telemedicine unit had the following components: (i) a web camera that allowed video conferencing with nurse case managers at the Berrie Diabetes Center at Columbia University or the Joslin Diabetes Center at SUNY Upstate Medical University; (ii) home glucose meter (One Touch Sure Step; Lifescan, Inc, Milpitas, CA) and blood pressure cuff (UA-767-PC Blood Pressure Monitor; A&D Medical; San Jose, CA) connected to the home telemedicine unit through an RS-232 serial port, so that home fingerstick glucose and blood pressure readings could be uploaded into a clinical database; (iii) access to patients' own clinical data; and (iv) access to a special educational web page created for the project by the American Diabetes Association in English and Spanish and in regular and low-literacy versions in each language.
Nurse case managers were trained in diabetes management and in the use of computer-based case management tools to facilitate interactions through videoconferencing with patients. The goals for the intervention were initially based on version 2.2 b (updated May 2000) of the Veterans Health Administration Clinical Practice Guidelines for the Management of Diabetes Mellitus in the Primary Care Setting. 14 Intervention management goals were subsequently updated to reflect the available authoritative guidelines, namely the Adult Treatment Panel (ATP) III guidelines of the National Cholesterol Education Program (NCEP), 15 the American Diabetes Association (ADA) Clinical Practice Guidelines, and the recommendations of the Joint National Commission (JNC) VII on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. 16 For the entire duration of the study the target HgbA1c was ≤7%, except for participants with significantly reduced life expectancy and/or severe hypoglycemic unawareness, for whom that target was ≤8%. 14 At study onset the blood pressure goal was <130/85 mm Hg, or <125/75 mm Hg in the presence of proteinuria (>1 g/24 h) or renal insufficiency. In January of 2003 the general blood pressure goal was adjusted to <130/80 mm Hg, maintaining a goal of <125/75 mm Hg in case of proteinuria or renal insufficiency. The goals for LDL-cholesterol at the beginning of the study were <130 mg/dL for primary prevention, and <100 mg/dL for those with known cardiovascular disease. In 2001, after the ATP III guidelines were published, the goal for LDL-cholesterol was changed to <100 mg/dL for all subjects. 15 In 2004, this goal was amended to reflect that an LDL-cholesterol goal of <70 mg/dL was a reasonable clinical strategy for patients with very high cardiovascular risk, according to an NCEP report. 17 The intervention was delivered by approximately four full-time equivalent nurse case managers, with supervision by endocrinologists daily.
The primary care physicians of intervention patients retained full responsibility and control over their patients' care. When a case manager determined that a change in management was indicated, he or she contacted the primary care physician (by e-mail, fax, or phone). The study sought to avoid disruption of established relationships and patterns of care and to insure continuity of care for intervention patients at the end of the study.
Usual Care
Participating primary care providers cared for patients in both the intervention and usual care groups, following the design whereby randomization was clustered within provider panel. The primary care providers received periodic mailings with current guidelines for the care of patients with diabetes. Patients in the usual care group received clinical care from their primary care providers, without other guidance or direction from study personnel.
Clinical End Points
Pre-specified clinical endpoints were HgbA1c, LDL cholesterol, and blood pressure levels. Follow-up examinations were conducted at 1-year intervals after randomization, with study time beginning at the baseline examination. Personnel conducting these examinations were blinded to intervention status and were not involved in supporting the clinical or technical aspects of the intervention. Data collected annually, starting at the baseline visit and including five follow-up visits through Feb 28, 2007 (when the intervention ended), were used in the primary analyses. Thus, the primary analyses consisted of six waves of data.
Subjects were instructed to come to the baseline and follow-up examinations fasting and having held their glycemic control medications, but taking their blood pressure medications. Specimens were processed and frozen at the data collection sites and analyzed at Medstar Laboratory (Washington, DC). HgbA1c was analyzed by boronate affinity chromatography with the Primus CLC 385 (Primus, Kansas City, MO). Total cholesterol, triglyceride, and high-density lipoprotein (HDL) cholesterol were measured using enzymatic colorimetric methods (Vitros, Johnson and Johnson, New Brunswick, NJ). LDL cholesterol was calculated using the Friedewald equation 18 for subjects with triglyceride level ≤ 300 mg/dL and measured directly using a homogeneous assay (Polymedco, Cortlandt Manor, NY) for those with triglyceride level > 300 mg/dL or total cholesterol > 240 mg/dL or HDL < 35 mg/dL. Resting blood pressure was measured using a Dinamap Monitor Pro 100 (Critikon, Tampa, FL) automated oscillometric device. Three measurements were obtained following five minutes of rest using a standardized protocol. 19 The average of the second and third measurements was recorded as the resting blood pressure. In both the intervention and usual care groups blood pressure values were communicated to participants at the time of the examinations and HgbA1c and lipid levels were communicated by mail to participants and their primary care providers.
Vital Status Ascertainment
Vital status of participants was determined through queries of the Center for Medicare and Medicaid Services (CMS) database, which contains data updated weekly by automated cross-referencing with the Social Security Administration database. We queried the CMS vital status database for this cohort to obtain data through Feb 28, 2007.
Statistical Methods
The primary analysis was performed on an intention to treat basis and adjusted for clustering of subjects within PCP. The statistical tests were based on trajectories (slopes) estimated from observed data points. Significance tests were two-tailed. The end points entered into the models were treated as continuous variables and did not require prior transformation, based on graphical inspection of the distribution of the outcome and of the residuals from the models. Adjusted means (SE) of the clinical end points during follow-up were estimated as follows. Power terms were added if a non-linear model provided a better fit. A linear model was appropriate for the blood pressure end points, whereas the best-fit models for HbgA1c and LDL-cholesterol were non-linear and included quadratic (group × time2) and exponential (group × e −time) terms. A significant quadratic term indicates differences between treatment groups, with one of the groups exhibiting a U-shaped distribution of the outcome over time. A significant exponential term suggests that the treatment groups experienced different rates of decline over time. Group heterogeneity in cluster and residual variances also required modeling. The best covariance structure for modeling the repeated outcome variables was selected, based on examination of the Akaike Information Criterion 20 and Schwarz's Bayesian Information Criterion. 21 Compound symmetry (equal correlations between waves over time) was the best fitting covariance structure for cholesterol and blood pressure, whereas HgbA1c was best modeled by a first order auto-regressive structure (decreasing correlations over time).
Sensitivity analyses were performed with respect to specification of the covariance structure, distributional assumptions, shape of the trajectories of the outcome over time, assumptions about missing data, and time of exposure to the intervention. Both maximum likelihood estimation, based on generalized linear mixed effects models, and generalized estimating equation approaches were used. Several non-linear models were tested, including the adjacent change model and change since baseline, which can include random effects for heterogeneity of change since baseline. 22 Models were evaluated based on fit statistics.
In the primary intention to treat analyses, no imputation of missing data was performed, under the assumption that data were missing at random and not conditional on treatment group, prior values of the outcome, or other subject characteristics. Alternative approaches were investigated to assess the robustness of that assumption including imputation algorithms. The models were also replicated in the subgroup with complete data for at least two follow-up examinations; approximately 1,431–1,445 out of 1,665 subjects (or 86–87%) were included in these analyses. The size of the analytic sample varied slightly for each outcome due to missing data for that endpoint at baseline and follow-up in a small number of subjects. The analytic numbers were N = 1,658 for HgbA1c, N = 1,656 for LDL-cholesterol, and N = 1,663 for blood pressure levels.
A Cox proportional hazards model was used in an intention-to-treat analysis of all-cause mortality, adjusting for clustered randomization within PCP practices. The hazard ratio and its 95% confidence interval (CI) are reported.
Statistical analyses were performed using SPSS, version 15.0 (SPSS, Chicago, IL) and SAS, version 9.1 (SAS Institute, Inc, Cary, NC).
Results
The two study groups were equivalent on key baseline variables (▶), as reported earlier. 12 Approximately 15% of the participants were African-American, 35% were Hispanic, and 63% were women.
Table 1.
Characteristic | Telemedicine Case Management n = 844 | Usual Care n = 821 |
---|---|---|
Age at randomization, years | 70.8 (6.5) | 70.9 (6.8) |
Female, % | 63.5 | 62.1 |
Race/ethnicity, % | ||
African-American (non-Hispanic) | 15.3 | 14.5 |
Hispanic | 35.8 | 34.6 |
White (non-Hispanic) | 48.2 | 50.6 |
other | 0.7 | 0.2 |
Marital status, % | ||
married/living with significant other | 41.4 | 40.9 |
single, never married | 13.0 | 10.1 |
separated/divorced | 16.4 | 18.1 |
widowed | 29.1 | 30.7 |
data missing | 0.1 | 0.1 |
Lives alone, % | 38.1 | 37.1 |
Education, years | 9.7 (4.1) | 9.9 (4.1) |
Eligible for Medicaid, % | 39.0 | 39.2 |
Duration of diabetes, years | 11.2 (9.6) | 10.99 (9.2) |
Participant “knows how to use a computer” | ||
yes | 18.8 | 21.2 |
no | 79.9 | 78.1 |
data missing | 1.3 | 0.7 |
Variables are summarized as percentages, and mean (SD), as appropriate.
HgbA1c
Overall, over the five years of follow-up, the intervention group had net improvement in HgbA1c relative to usual care (p = 0.001 [exponential × group interaction term in non-linear model]; ▶, Panel A). Statistically significant differences in HgbA1c level favoring the intervention group were found in years 4 and 5, with a net adjusted difference (95% CI) favoring telemedicine of 0.29 (0.12, 0.46)% at year 5 (▶). The difference between baseline and 5 year adjusted means was 0.34% in the telemedicine group, compared to 0.07% in the usual care group (▶).
Table 2.
Outcome |
Examination |
Unadjusted |
Adjusted ∗ |
Treatment Effect † (95% CI) |
||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Usual Care |
Telemedicine |
Usual Care |
Telemedicine |
|||||||||
n | Mean | SD | n | Mean | SD | Mean | SE | Mean | SE | |||
Hemoglobin A1c (%) | Baseline | 802 | 7.40 | (1.60) | 829 | 7.36 | (1.48) | 7.45 | (0.06) | 7.43 | (0.05) | 0.02 (−0.13, 0.17) |
One year | 701 | 7.16 | (1.40) | 681 | 6.96 | (1.12) | 7.20 | (0.05) | 7.08 | (0.05) | 0.12 (−0.02, 0.25) | |
Two year | 633 | 7.15 | (1.25) | 618 | 7.05 | (1.12) | 7.17 | (0.05) | 7.06 | (0.05) | 0.11 (−0.03, 0.25) | |
Three year | 531 | 7.15 | (1.26) | 466 | 7.04 | (1.17) | 7.21 | (0.05) | 7.10 | (0.05) | 0.11 (−0.03, 0.25) | |
Four year | 494 | 7.24 | (1.39) | 436 | 7.06 | (1.17) | 7.29 | (0.05) | 7.13 | (0.05) | 0.16 (0.02, 0.31) | |
Five year | 372 | 7.34 | (1.54) | 355 | 7.05 | (1.17) | 7.38 | (0.06) | 7.09 | (0.06) | 0.29 (0.12, 0.46) | |
LDL cholesterol (mg/dl) | Baseline | 791 | 108.21 | (35.75) | 819 | 106.82 | (34.96) | 108.11 | (1.21) | 106.64 | (1.22) | 1.47 (−1.90, 4.85) |
One year | 700 | 105.65 | (39.41) | 679 | 95.80 | (31.68) | 105.49 | (1.10) | 97.84 | (1.11) | 7.65 (4.59, 10.71) | |
Two year | 633 | 101.33 | (35.70) | 616 | 95.03 | (33.85) | 102.86 | (1.06) | 94.29 | (1.12) | 8.57 (5.54, 11.60) | |
Three year | 528 | 99.26 | (35.78) | 466 | 92.02 | (32.99) | 100.23 | (1.11) | 92.68 | (1.08) | 7.55 (4.53, 10.58) | |
Four year | 492 | 97.06 | (40.65) | 434 | 86.55 | (33.52) | 97.60 | (1.23) | 91.77 | (1.17) | 5.83 (2.51, 9.15) | |
Five year | 369 | 94.39 | (34.61) | 354 | 92.00 | (34.57) | 94.97 | (1.40) | 91.13 | (1.43) | 3.84 (−0.08, 7.77) | |
Systolic blood pressure (mm Hg) | Baseline | 815 | 142.47 | (23.62) | 842 | 142.79 | (24.21) | 141.85 | (0.74) | 140.34 | (0.73) | 1.51 (−0.53, 3.56) |
One year | 714 | 140.56 | (22.92) | 698 | 137.38 | (21.23) | 141.51 | (0.68) | 139.43 | (0.68) | 2.07 (0.19, 3.96) | |
Two year | 636 | 140.95 | (22.75) | 620 | 137.28 | (21.88) | 141.17 | (0.66) | 138.53 | (0.67) | 2.64 (0.80, 4.48) | |
Three year | 535 | 139.15 | (22.75) | 468 | 136.27 | (20.96) | 140.83 | (0.69) | 137.63 | (0.70) | 3.20 (1.27, 5.12) | |
Four year | 493 | 141.45 | (22.46) | 437 | 138.44 | (21.24) | 140.49 | (0.76) | 136.73 | (0.77) | 3.76 (1.64, 5.88) | |
Five year | 373 | 139.48 | (22.22) | 362 | 136.12 | (20.38) | 140.15 | (0.86) | 135.83 | (0.87) | 4.32 (1.93, 6.72) | |
Diastolic blood pressure (mm Hg) | Baseline | 815 | 71.01 | (10.42) | 842 | 71.59 | (11.35) | 70.35 | (0.34) | 69.88 | (0.33) | 0.47 (−0.47, 1.41) |
One year | 714 | 70.01 | (11.06) | 698 | 68.45 | (9.91) | 69.94 | (0.32) | 69.04 | (0.31) | 0.90 (0.04, 1.77) | |
Two year | 636 | 69.42 | (10.87) | 620 | 68.52 | (10.54) | 69.53 | (0.31) | 68.19 | (0.30) | 1.34 (0.49, 2.18) | |
Three year | 535 | 69.22 | (10.64) | 468 | 67.56 | (9.57) | 69.12 | (0.32) | 67.35 | (0.32) | 1.77 (0.88, 2.66) | |
Four year | 493 | 69.79 | (10.86) | 437 | 67.94 | (9.95) | 68.70 | (0.35) | 66.50 | (0.35) | 2.20 (1.23, 3.18) | |
Five year | 373 | 68.46 | (11.12) | 362 | 67.30 | (10.16) | 68.29 | (0.40) | 65.66 | (0.40) | 2.64 (1.53, 3.74) |
∗ Non-linear models with quadratic (group*[time]2) (time centered) and/or exponential (e−time) terms to model non-linearity with first order auto-regressive covariance structure was used for hemoglobin A1c and Compound Symmetry covariance structure was used for total and LDL cholesterol analyses. Linear models with the compound symmetry covariance structure was used for the systolic and diastolic blood pressure analyses. The adjusted means, standard errors and treatment effects (differences) were estimated from longitudinal linear and non-linear regression models, adjusting for clustering, and heterogeneous variances. The analytic samples were N = 1,658 for hemoglobin A1c, N = 1,656 for LDL-cholesterol, and N = 1663 for blood pressure levels.
† Treatment effects are differences in adjusted means, scored in the direction of improvement associated with the intervention.
LDL Cholesterol
Overall, over the five years of follow-up, the intervention group experienced net improvement in LDL cholesterol level relative to usual care (p < 0.001 [exponential × group interaction term in non-linear model]; ▶, Panel B). Statistically significant differences in LDL cholesterol level favoring the intervention group were found in years 1–4 (▶), reflecting progressive narrowing of the relatively greater improvement in the telemedicine group in the earlier years of follow-up. The difference between baseline and 5 year adjusted mean LDL-cholesterol was 15.51 mg/dL in the telemedicine group, compared to 13.14 mg/dL in the usual care group (▶).
Blood Pressure
Reductions in systolic and diastolic blood pressure were also significantly greater in the telemedicine group. As seen in ▶, Panel C, overall the intervention group had greater reductions in systolic and diastolic blood pressure relative to the usual care group (group × time interaction terms in linear models, p = 0.024 and p < 0.001, respectively). Statistically significant differences favoring the intervention group were found in all five years for both systolic and diastolic blood pressure (▶). Adjusted differences at 5 years were 4.32 (1.93, 6.72) mm Hg for systolic blood pressure, and 2.64 (1.53, 3.74) mm Hg for diastolic blood pressure. The difference between baseline and 5 year adjusted mean systolic and diastolic blood pressure in the telemedicine group was 4.51 and 4.22 mm Hg, respectively, as compared to 1.70 and 2.06 mm Hg in the usual care group (▶).
Sensitivity Analyses
Sensitivity analyses for all three pre-specified outcomes rendered essentially similar results (available from the authors upon request).
All Cause Mortality and Adverse Effects
There were 176 deaths in the intervention group and 169 in the usual care group, which corresponded to a hazard ratio = 1.01; 95% CI, 0.82–1.24. There were no serious adverse events related to the intervention.
Discussion
The telemedicine case management intervention achieved sustained reductions in HgbA1c, LDL-cholesterol, and systolic and diastolic blood pressure levels over 5 years of follow-up net of changes in the usual care group. Differences (95% CI) in Year 5 were 0.29 (0.12, 0.46)% for HgbA1c, 3.84 (−0.08, 7.77) mg/dL for LDL cholesterol, 4.32 (1.93, 6.72) mm Hg for systolic blood pressure, and 2.64 (1.53, 3.74) mm Hg for diastolic blood pressure. Differences were present at 1 year of follow up and were sustained over five years for the three main endpoints. Multifactorial improvement has greater clinical significance than improvement in single risk factors. All-cause mortality did not differ over the 5 years of follow-up between the intervention and usual care groups.
The improvement in glycemic control achieved by telemedicine was similar to that seen in randomized controlled trials assessing other case management interventions. 23 A meta-regression analysis by Shojania, et al. estimated that the mean additional reduction in HgbA1c by case management interventions was 0.20% in studies whose participants had a mean HgbA1c less than 8%. 23 Larger reductions were observed in studies whose participants had higher baseline HgbA1c. Study eligibility in IDEATel was restricted to Medicare beneficiaries residing in federally designated medically underserved areas of New York State. As a result, the number of potentially eligible patients with diabetes was limited and all eligible subjects were therefore enrolled, regardless of baseline HgbA1c level. Many studies of diabetes case management, including some smaller telemedicine studies, have focused on patients with elevated HgbA1c levels, above 8.0% or even above 9.0%. We included all eligible patients regardless of baseline HgbA1c level, even if below 7.0%. Thus, the IDEATel subjects had a mean baseline HgbA1c level of approximately 7.4%. While we observed a larger intervention effect in the subgroup with mean HgbA1c level > 7.0% at baseline, this was not a pre-specified analysis and inferences about intervention effect size to groups with higher or lower HgbA1c level at baseline should be made with caution.
Temporal trends in the usual care group differed among the clinical outcomes over the 5 years of follow-up. Glycemic control in the usual care group improved during the first two years, and then reverted toward baseline. In contrast, lipid and blood pressure levels continued to decrease in the usual care group over the 5 years of follow-up. Blood pressure levels remained lower in the intervention than the usual care group in year 5, while LDL cholesterol levels no longer differed significantly between groups. The reductions in LDL-cholesterol level in the usual care group are consistent with secular changes in care for people with diabetes in the United States, which have included lower cholesterol levels. 24 In addition, participants were randomized within physician practices, and there may have been “Hawthorne” or spillover effects from treatment recommendations for the intervention subjects given to primary care providers by the telemedicine diabetes team.
Most diabetes-related mortality is due to macrovascular complications, specifically coronary, peripheral, and cerebral arterial disease. 25 Treatment of hypertension and dyslipidemia decreases the risk of these complications in a cost-effective manner. 26–32 The role of tight glycemic control is less clear—it lowers the risk of microvascular complications but may increase mortality risk in certain populations. 33,34 The Steno-2 trial showed a reduction in mortality and progression to end-stage renal disease through a multifactorial intervention similar to ours. 31 However, the intervention in Steno-2 was conducted in the 1990s, and when it ended, the mean LDL-cholesterol, blood pressure, and glycosylated hemoglobin in the conventional therapy group were higher than recommended goals. 31 The glycemic control arm of the ACCORD study 34 was stopped due to higher mortality in the intensive treatment arm (target HgbA1c < 6%), which used multiple medications, frequently including rosiglitazone, to reduce HgbA1c rapidly. By contrast, in the tighter glycemic control arm of the ADVANCE study (target HgbA1c < 6.5%) there was a reduction in the risk of nephropathy, but without a reduction in mortality. 33 Long-term follow-up in the UKPDS showed significant reductions in microvascular and macrovascular disease as well as death with better glycemic control. 35 The current consensus is that prudent glycemic control remains desirable for most people with diabetes, with a target HgbA1c ≤ 7%, which should be adjusted upwardly depending on individual patient characteristics. 36,37 The glycemic goal in the telemedicine arm of IDEATel reflected this latter, more cautious approach, which we deemed appropriate in elderly people, who are at higher risk of hypoglycemic complications. 38
IDEATel was not designed to detect either a reduction or an increase in mortality. Given the concerns raised by the results of the ACCORD trial, 34 we compared all-cause mortality between study groups and found no difference, but statistical power for all cause mortality was limited and the confidence interval did not exclude a clinically important increase. The study was also not designed or powered to detect differences in clinical events such as macrovascular complications.
Several unique or special features of the IDEATel study may be noted. IDEATel is one of the few randomized trials of telemedicine to have been conducted with concurrent controls. IDEATel is one of the largest evaluation projects of telemedicine to have been conducted, of any design, the largest (to our knowledge) to have been conducted in a civilian population, and the largest randomized controlled trial to date to evaluate telemedicine as a means to provide home-based care to people with diabetes mellitus. The 5 year follow-up was substantially longer than that of previous studies, which lasted between 3 and 12 months. 39,40 All evaluation data were collected at separate, in-person evaluation visits by personnel who were separated from both the clinical and technical aspects of the intervention and blinded insofar as possible to the group assignment of the participants. The data collection visits were scheduled at 1-year intervals rather than coinciding with visits to the participants' clinical care providers. Visits to clinical care providers may be triggered by changes in clinical status (e.g., poor home monitoring results) or inter-current illness and therefore may not represent the time period of follow-up in an unbiased fashion. All clinical outcome measurements were made following strictly defined data collection protocols, and laboratory assays were all conducted in a single central laboratory. The study sample was composed of elderly, ethnically diverse Medicare beneficiaries residing in rural and inner-city medically underserved areas, with an adequate representation of women. This is a population whose access to medical care is impaired by multiple barriers. 7,41 In rural areas those barriers include long distances to medical offices, episodic isolation during winter, and low access to specialized diabetes care. 42 In New York City most study participants belonged to an ethnic/racial minority, had lower socioeconomic and educational levels, and almost three-quarters spoke primarily Spanish. 12 Thus, we recruited and enrolled patients whose level of technical literacy was generally representative of their (medically underserved) communities at large. The project was conducted shortly after passage of the Health Insurance Portability and Accountability Act (HIPAA) and therefore incorporated HIPAA compliance into its technical design and implementation. Finally, the study data were analyzed on the basis of intention-to-treat; that is, all randomized patients were included in the analysis. Intention-to-treat analysis of main trial results is widely viewed as essential to protect against potential biases that would arise from on-treatment or completer analysis.
The intervention strategy technical platform, which was developed in 2000, was designed to support three key capabilities that were thought to anticipate future functions that would characterize telemedicine-supported chronic disease management: the ability to upload self-monitoring data for fingerstick glucose and blood pressure, Internet access to support information gathering by participants, and face-to-face synchronous video teleconferencing between the patient and the nurse case manager. 13 In a field such as telemedicine, where the technology is rapidly evolving, it is extremely difficult to carry out a longitudinal study of any significant duration since the state of the art towards the end will differ greatly from what was in place at the start. One option is to continue with the original technology throughout the study. This approach simplifies the study but fails to take advantage of technologic advances that would increase benefits—including ease of use, reliability, and improved human-computer interactions—and runs the risk that the results of the study will be considered irrelevant, since the technology used was outmoded. We chose the alternative approach of changing to a better technological platform and user interface after the first four years of the project, based on early experience and technologic advances. We believe this decision makes the study results more relevant for those who would seek to implement the technology or develop the next generation for further study. Earlier reports described the IDEATel study design, 11 technical implementation, 13 and 1 year results. 12 This paper adds information indicating that telemedicine case management can be sustained over a longer period in a medically underserved, ethnically diverse, older population, that benefits can also be sustained despite a study design that could not eliminate spill over effects or secular trends, and that many patients can and will use computer technology and advanced medical informatics capabilities to improve the management of their chronic medical conditions.
Several limitations may also be noted. Substantial attrition occurred in this study. Attrition by competing risks is commonly seen in elderly people, 43 and attrition is a pervasive problem for studies in medically underserved populations. 44 To address potential biases arising from attrition we performed sensitivity analyses that modeled the possible effects of missing data and reached materially equivalent results. There were 514 subjects who dropped out in the intervention arm and 358 in the usual care group (▶). The numbers who died or became too sick to continue were similar in the two groups. A larger number of randomized subjects in the intervention group dropped out early in the study when they refused delivery of the home telemedicine unit or refused to continue because of difficulty or frustration with the computer. The assignment of reasons for discontinuation in the study was based on telephone interviews with subjects who dropped out. It is possible that some gave reasons, such as “change in living arrangement”, for reasons of social acceptability, when in fact the decision to drop out may have been motivated in part or wholly by other factors, such as frustration with the home telemedicine unit. Our interactions with participants, both those who dropped out and those who remained in the study, indicated that many participants found learning how to use the computer challenging and that a substantial amount of training and support was needed in many cases. 45–47 These observations are consistent with the very low levels of computer literacy in the randomized study population (▶) and the relatively low educational status of the population living in study-eligible medically underserved areas.
While speculative, it may nonetheless be worth considering possible scenarios in which attrition had been lower. More intervention than usual care patients dropped out, and many of the intervention group drop outs were very early in the study. Thus, some intervention patients received little or none of the intervention but did receive usual care. Had attrition been lower, the observed intervention effects might have been greater.
The statistical methods used to analyze the main pre-specified clinical outcomes were complex for several reasons. First, the statistical models needed to take into account the design feature whereby participants were randomized within primary care provider panels. Second, the study observed participants annually for five years, and, to use all the data, the models needed to take into account the repeat observations of the same individuals. Finally, as shown in the figures, the trajectories over time for HgbA1c and LDL cholesterol were not linear. Linear models could therefore not be applied, if all the data and the observed shapes of the trajectories were to be considered. Because non-linear models were used for HgbA1c and LDL cholesterol, and because complex models were used for analysis of all outcomes, no summary estimate of intervention effects over the five years could be derived. Instead we reported intervention effects at five years (rather than over five years). We were able to perform (and did report) statistical tests on the appropriate interaction terms in the non-linear models that provided p values for the intervention effects over five years.
We did not perform a formal cost-effectiveness analysis. The relative costs and benefits of telemedicine as a strategy for chronic disease case management will depend on several factors, including the baseline levels of risk factors, the magnitude of risk factor reductions, and the costs of case management personnel, telemedicine devices, and the underlying communications and data management infrastructures. Handheld or laptop devices that are owned and maintained by patients and that combine many of the capabilities of personal computers and telephones have the potential to support remote patient-provider interactions, including data uploads from home monitoring devices, at much lower cost than the special purpose devices that we used because of the technical environment at the time IDEATel was conducted.
Conclusions
Telemedicine case management intervention achieved reductions in HgbA1c, LDL-cholesterol, and systolic and diastolic blood pressure levels net of changes over time in the group receiving usual care. No differences were seen between the groups in all-cause mortality. The study population was a large, ethnically diverse sample of elderly Medicare beneficiaries residing in medically underserved areas. This study is unique in the current medical literature due to its duration, large sample size, the inclusion of both rural and urban participants, and the multiple barriers to medical care the participants faced. The IDEATel study provides much needed evidence supporting the effectiveness of telemedicine as a means to deliver long term diabetes care to at risk populations.
The IDEATel Research Consortium includes the following: Columbia University and NewYork-Presbyterian Hospital (New York, NY): Steven Shea, MD, MS, Justin Starren, MD, PhD (current address: Marshfield Clinic Research Foundation (Marshfield, WI), Walter Palmas, MD, MS, Lesley Field, RN, MSN, Robin Goland, MD, Catherine Tuck, MD (deceased), James Cimino, MD, Albert Lai, PhD (current address: the Ohio State University, Columbus, OH), Susana Ebner, MD, Rafael Lantigua, MD, David Kaufman, PhD, Charlyn Hilliman, MPA, PhD, David Liss, Sean Mota, Soumitra Sengupta, PhD, Vimla Patel, PhD, Mohammed Ashraf, Patricia Kringas, RN, Jessica Rivera, RN, Rene Bachmann, RN, CDN, CDE, Vilma Moreno, RN, Christine McGuinness, RN, Iris Fernandez, RN, David Baorto, MD, PhD, Dimitri Borovtsov, Yao Huang, Rajhunathan Kamasmudram, PhD, Khalid Qureshi, Yuri Vayman; Harlem Hospital Center (New York, NY): Linnea Capps, MD; St Luke's-Roosevelt Hospital Center (New York, NY): Alejandro Prigollini, MD; Research Division of the Hebrew Home for the Aged at Riverdale (Bronx, NY): Jeanne A. Teresi, EdD, PhD, Douglas Holmes, PhD (deceased), Stephanie Silver, MPH, Jian Kong, MS, Eva Petkova, PhD, Joseph P. Eimicke, MS, Onie Manzano, BS, Mildred Ramirez, PhD; SUNY Upstate Medical University at Syracuse (Syracuse, NY): Ruth S. Weinstock, MD, PhD, Philip C. Morin, MS, CCRP, Roberto Izquierdo, MD, L. Thomas Wolff, MD, Suzanne Meyer, RN, CDE, Suzan M. Carusone, RN, CCRC, Paul E. Knudson, MD (current address: Medical College of Wisconsin, Milwaukee, WI), Fraser G. Tudiver, MD (current address: E. Tennessee State University, Johnson City, TN), Teresa J. Wagner, MS, Carina Lagua, MA, RD, CDE, Susan Fox, RN, Susan West, RN, MSN, CDE, Laura Ferri, RN, RD, CDCE, JoAnn Shupe, AOS, Donald R. Roller, BS, Joseph Ziemba, MBA, David P. Thomson, MD, Kelly Duncan BSW, CCRC, Sarah Doolittle, BS, CCRC, Joanne Kearns, MA, RD, CDE, Deborah Bowerman, RN, CDE, Pamela Suddaby, RN, BS, Kristine M. Brindak, RN, Paula Trief, PhD, Julia Morina, RD; A. L. Lee Memorial Hospital, Inc (Fulton, NY): Shirley Seabury, MHA; Arnot Ogden Medical Center (Elmira, NY): John F. Carroll, MD, MBA, Frederic P. Skinner, MD, PhD, Terri L. Ennis, RN; Bassett Healthcare (Cooperstown, NY): Kathleen Sellers, RN, PhD, Barbara Leo, BA, BS, RN, Elsie Haus, MEd, FNP, Megan Wood, MS, RN, Barbara Gibbons, MS, RN, Catherine Cooney, BS, MA, RN; Delaware Valley Hospital (Walton, NY): Michael J. Freeman, DO, MPH; Family Health Network of Central New York, Inc (Cortland, NY): Susan Ulrich, MS, RN, CPNP; Guthrie Healthcare System (Sayre, PA): Brian D. Cassetta, MD; Hudson Headwaters Health Network (Glens Falls, NY): Paul Bachman, MD, Patricia Auer, MSN, FNP, W. Emily Hammond, AAS, RN, Cairenn L. Young, MSW, Mary M. Beadnell, LPN; Lourdes Hospital (Binghamton, NY): Timothy C. Howland, MD; Olean General Hospital (Olean, NY): Robert Catalano, MD, Galvin Anderson, MD, Terri N. Kolasinski, RN; Samaritan Medical Center (Watertown, NY): Anil Shah, MD, Kate R. Fraczek, RN, Terry Flynt, BSN, RN, Julie Stone, RN; State University of New York at Buffalo (Buffalo, NY): John Taylor, MBA, Lurene R. Tumiel, PhD, Marc Shilling, MA, MPA, Chester Fox, MD; United Health Services (Binghamton, NY): David Kwiatkowski, MD; American Telecare, Inc (Eden Prairie, MN): Randall S. Moore, MD, MBA, C.R. Abbruscato, MSEE, Karen Boril.
Dr. Jeanne A. Teresi had access to all the study data and takes responsibility for the statistical analyses.
Role of the Funding Source
The funding agency had the opportunity to comment on the paper before submission but did not participate in the analysis of the data or the writing of the paper. The inferences, conclusions, and opinions expressed in the paper are those of the authors and do not represent the position of CMS.
Funding and Research Support in Previous 5 Years
Steven Shea, MD, MS: Centers for Medicare and Medicaid Services (CMS), National Heart, Lung, and Blood Institute (NHLBI), National Institute on Aging (NIA), National Library of Medicine (NLM), National Center on Minority Health and Health Disparities (NCMHD), Health Resources and Services Administration (HRSA)
Ruth S. Weinstock, MD, PhD: CMS, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), CDC/ATSDR, New York State Department of Health (NYSDOH), Onondaga County, Merck, Eli Lilly, Pfizer, Novartis, GlaxoSmithKline, Bristol-Myers Squibb, Biodel, Novo-Nordisk, Sanofi-Aventis
Jeanne A. Teresi, EdD, PhD: CMS, NIA, NHLBI, NCMHD, National Cancer Institute (NCI), Agency for Healthcare Research and Quality (AHRQ), NYSDOH
Walter Palmas, MD, MS: CMS, NHLBI
Justin Starren, MD, PhD: CMS, NLM, National Science Foundation (NSF), Centers for Disease Control, New York State Center for Advanced Technology, NHGRI, NCI, NCRR, NIOSH, Lifescan, Delta Dental
James J. Cimino, MD: CMS, NLM, NIH Clinical Center
Albert M. Lai, PhD: CMS, NLM, NSF
Lesley Field, RN, MSN: CMS
Philip Morin, MS: CMS, NYSDOH
Robin Goland, MD: CMS, NHLBI, NIDDK
Roberto E. Izquierdo, MD: CMS, NIDDK, NYSDOH, Abbott Labs, Macrogenics, Inc.
Susanna Ebner, MD: CMS, Sanofi-Aventis
Stephanie Silver, MPH: CMS, NIA, NHLBI, NCMHD, AHRQ, NYSDOH
Eva Petkova, PhD: NIA, NHLBI, National Institute on Drug Abuse (NIDA), National Institute on Alcohol Abuse and Alcoholism (NIAAA), National Institute of Mental Health (NIMH)
Jian Kong, MS: CMS, NIA, NHLBI, NCMHD, AHRQ, NYSDOH
Joseph P. Eimicke, MS: CMS, NIA, NHLBI, NCMHD, AHRQ, NYSDOH
References
- 1.Wootton R, Jebamani LS, Dow SA. E-health and the Universitas21 Organization. 2. Telemedicine and underserved populations. J Telemed Telecare 2005;11:221-224. [DOI] [PubMed] [Google Scholar]
- 2.Farmer A, Gibson OJ, Tarassenko L, Neil A. A systematic review of telemedicine interventions to support blood glucose self-monitoring in diabetes Diabet Med 2005;22:1372-1378. [DOI] [PubMed] [Google Scholar]
- 3.American Diabetes Association Economic costs of diabetes in the U.S. in 2007 Diab Care 2008;31:596-615. [DOI] [PubMed] [Google Scholar]
- 4.Saydah SH, Fradkin J, Cowie CC. Poor control of risk factors for vascular disease among adults with previously diagnosed diabetes J Am Med Assoc 2004;291:335-342. [DOI] [PubMed] [Google Scholar]
- 5.Nelson KM, Chapko MK, Reiber G, Boyko EJ. The association between health insurance coverage and diabetes care; data from the 2000 behavioral risk factor surveillance system Health Serv Res 2005;40:361-372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.McWilliams JM, Meara E, Zaslavsky AM, Ayanian JZ. Use of health services by previously uninsured Medicare beneficiaries N Engl J Med 2007;357:143-153. [DOI] [PubMed] [Google Scholar]
- 7.Chou AF, Brown AF, Jensen RE, et al. Gender and racial disparities in the management of diabetes mellitus among Medicare patients Womens Health Issues 2007;17:150-161. [DOI] [PubMed] [Google Scholar]
- 8.Weingarten JP, Brittman S, Hu W, et al. The state of diabetes care provided to Medicare beneficiaries living in rural America J Rural Health 2006;22:351-358. [DOI] [PubMed] [Google Scholar]
- 9.Scott TL, Gazmararian JA, Williams MV, Baker DW. Health literacy and preventive health care use among Medicare enrollees in a managed care organization Med Care 2002;40:395-404. [DOI] [PubMed] [Google Scholar]
- 10.Hersh WR, Hickam DH, Severance SM, et al. Telemedicine for the Medicare population: Update Evid Rep Technol Assess (Full rep) 2006:1-41. [PMC free article] [PubMed]
- 11.Shea S, Starren J, Weinstock RS, et al. Informatics for Diabetes Education and Telemedicine (IDEATel) project: Rationale and design J Am Med Inform Assoc 2002;9:49-62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Shea S, Weinstock RS, Starren J, et al. A randomized trial comparing telemedicine case management with usual care in older, ethnically diverse, medically underserved patients with diabetes mellitus J Am Med Inform Assoc 2006;13:40-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Starren J, Hripcsak G, Sengupta S, et al. Columbia University's Informatics for Diabetes Education and Telemedicine (IDEATel) project: Technical implementation J Am Med Inform Assoc 2002;9:25-36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Department of Veteran Affairs. Veterans Health Administration Diabetes Program. VHA Clinical Practice Guidelines for the Management of Diabetes Mellitus in the Primary Care Setting, updated May 2000.
- 15.Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) Executive summary of the third report of the National Cholesterol Education Program (NCEP) J Am Med Assoc 2001;285:2486-2497. [DOI] [PubMed] [Google Scholar]
- 16.Chobanian AV, Bakris GL, Black HR, et al. The seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: The JNC 7 report J Am Med Assoc 2003;289:2560-2572. [DOI] [PubMed] [Google Scholar]
- 17.Grundy SM, Cleeman JI, Merz CN, et al. Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines Circulation 2004;110:227-239. [DOI] [PubMed] [Google Scholar]
- 18.Friedewald WT, Levy RI, Fredrickson DS. Estimation of the concentration of low-density lipoprotein cholesterol in plasma, without use of the preparative ultracentrifuge Clin Chem 1972;18:499-502. [PubMed] [Google Scholar]
- 19.Perloff D, Grim C, Flack J, et al. Human blood pressure determination by sphygmomanometry Circulation 1993;88:2460-2470. [DOI] [PubMed] [Google Scholar]
- 20.Sullivan LM. Repeated measures Circulation 2008;117:1238-1243. [DOI] [PubMed] [Google Scholar]
- 21.Schwarz G. Estimating the dimension of a model Ann Stat 1978;6:461-464. [Google Scholar]
- 22.Liu X, Teresi JA, Waternaux C. Modelling the decline pattern in functional measures from a prevalent cohort study Stat Med 2000;19:1593-1606. [DOI] [PubMed] [Google Scholar]
- 23.Shojania KG, Ranji SR, McDonald KM, et al. Effects of quality improvement strategies for type 2 diabetes on glycemic control: A meta-regression analysis J Am Med Assoc 2006;296:427-440. [DOI] [PubMed] [Google Scholar]
- 24.Saaddine JB, Cadwell B, Gregg EW, et al. Improvements in diabetes processes of care and intermediate outcomes: United States, 1988–2002 Ann Intern Med 2006;144:465-474. [DOI] [PubMed] [Google Scholar]
- 25.Beckman JA, Creager MA, Libby P. Diabetes and atherosclerosis: Epidemiology, pathophysiology, and management J Am Med Assoc 2002;287:2570-2581. [DOI] [PubMed] [Google Scholar]
- 26.UK Prospective Diabetes Study Group Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes: UKPDS 38 Br Med J 1998;317:703-713. [PMC free article] [PubMed] [Google Scholar]
- 27.UK Prospective Diabetes Study Group Cost effectiveness analysis of improved blood pressure control in hypertensive patients with type 2 diabetes: UKPDS 40 Br Med J 1998;317:720-726. [PMC free article] [PubMed] [Google Scholar]
- 28.Pyorala K, Pedersen T, Kjekshus J, Faergeman O. Cholesterol lowering with simvastatin improves prognosis of diabetic patients with coronary heart disease: A subgroup analysis of the Scandinavian simvastatin survival study (4S) Diab Care 1997;20:614-620. [DOI] [PubMed] [Google Scholar]
- 29.Grover SA, Coupal L, Zowall H, et al. How cost-effective is the treatment of dyslipidemia in patients with diabetes but without cardiovascular disease? Diab Care 2001;24:45-50. [DOI] [PubMed] [Google Scholar]
- 30.The CDC Diabetes Cost-effectiveness Group Cost-effectiveness of intensive glycemic control, intensified hypertension control, and serum cholesterol level reduction for type 2 diabetes J Am Med Assoc 2002;287:2542-2551. [DOI] [PubMed] [Google Scholar]
- 31.Gaede P, Vedel P, Larsen N, et al. Multifactorial intervention and cardiovascular disease in patients with type 2 diabetes N Engl J Med 2003;348:383-393. [DOI] [PubMed] [Google Scholar]
- 32.Stratton IM, Adler AI, Neil HA, et al. Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): Prospective observational study Br Med J 2000;321:405-412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Patel A, MacMahon S, Chalmers J, et al. Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes N Engl J Med 2008;358:2560-2572. [DOI] [PubMed] [Google Scholar]
- 34.Gerstein HC, Miller ME, Byington RP, et al. Effects of intensive glucose lowering in type 2 diabetes N Engl J Med 2008;358:2545-2559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Holman RR, Paul SK, Bethel MA, Matthews DR, Neil HA. 10-Year follow-up of intensive glucose control in type 2 diabetes N Engl J Med 2008;359:1577-1589. [DOI] [PubMed] [Google Scholar]
- 36.Dluhy RG, McMahon GT. Intensive glycemic control in the ACCORD and ADVANCE trials N Engl J Med 2008;358:2630-2633. [DOI] [PubMed] [Google Scholar]
- 37.Cefalu WT. Glycemic targets and cardiovascular disease N Engl J Med 2008;358:2633-2635. [DOI] [PubMed] [Google Scholar]
- 38.Meneilly GS, Cheung E, Tuokko H. Altered responses to hypoglycemia of healthy elderly people J Clin Endocrinol Metab 1994;78:1341-1348. [DOI] [PubMed] [Google Scholar]
- 39.Starren J, Weinstock RS, Palmas W, et al. DiabetesIn: Wootton R, Dimmick SL, Kvedar JC, editors. Home Telehealth: Connecting care within the community. London: Royal Society of Medicine Press; 2006. pp. 170-172.
- 40.Jaana M, Pare G. Home telemonitoring of patients with diabetes: A systematic assessment of observed effects J Eval Clin Pract 2007;13:242-253. [DOI] [PubMed] [Google Scholar]
- 41.Chan L, Hart LG, Goodman DC. Geographic access to health care for rural Medicare beneficiaries J Rural Health 2006;22:140-146. [DOI] [PubMed] [Google Scholar]
- 42.Calman NS, Hauser D, Forte G, Continelli T. New York State physicians: Characteristics and distribution in health professional shortage areas J Urban Health 2007;84:307-309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Bowsher J, Bramlett M, Burnside IM, Gueldner SH. Methodological considerations in the study of frail elderly people J Adv Nurs 1993;18:873-879. [DOI] [PubMed] [Google Scholar]
- 44.Lindenberg CS, Solorzano RM, Vilaro FM, Westbrook LO. Challenges and strategies for conducting intervention research with culturally diverse populations J Transcult Nurs 2001;12:132-139. [DOI] [PubMed] [Google Scholar]
- 45.Kaufman DR, Starren J, Patel VL, et al. A cognitive framework for understanding barriers to the productive use of a diabetes home telemedicine system AMIA Annu Symp Proc 2003:356-360. [PMC free article] [PubMed]
- 46.Lai AM, Kaufman DR, Starren J. Training digital divide seniors to use a telehealth system: A remote training approach AMIA Annu Symp Proc 2006:459-463. [PMC free article] [PubMed]
- 47.Kaufman DR, Patel VL, Hilliman C, et al. Usability in the real world: Assessing medical information technologies in patients' homes J Biomed Inform 2003;36:45-60. [DOI] [PubMed] [Google Scholar]