Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial

Abstract

Objective To assess the effect of home based telehealth interventions on the use of secondary healthcare and mortality.

Design Pragmatic, multisite, cluster randomised trial comparing telehealth with usual care, using data from routine administrative datasets. General practice was the unit of randomisation. We allocated practices using a minimisation algorithm, and did analyses by intention to treat.

Setting 179 general practices in three areas in England.

Participants 3230 people with diabetes, chronic obstructive pulmonary disease, or heart failure recruited from practices between May 2008 and November 2009.

Interventions Telehealth involved remote exchange of data between patients and healthcare professionals as part of patients’ diagnosis and management. Usual care reflected the range of services available in the trial sites, excluding telehealth.

Main outcome measure Proportion of patients admitted to hospital during 12 month trial period.

Results Patient characteristics were similar at baseline. Compared with controls, the intervention group had a lower admission proportion within 12 month follow-up (odds ratio 0.82, 95% confidence interval 0.70 to 0.97, P=0.017). Mortality at 12 months was also lower for intervention patients than for controls (4.6% v 8.3%; odds ratio 0.54, 0.39 to 0.75, P<0.001). These differences in admissions and mortality remained significant after adjustment. The mean number of emergency admissions per head also differed between groups (crude rates, intervention 0.54 v control 0.68); these changes were significant in unadjusted comparisons (incidence rate ratio 0.81, 0.65 to 1.00, P=0.046) and after adjusting for a predictive risk score, but not after adjusting for baseline characteristics. Length of hospital stay was shorter for intervention patients than for controls (mean bed days per head 4.87 v 5.68; geometric mean difference −0.64 days, −1.14 to −0.10, P=0.023, which remained significant after adjustment). Observed differences in other forms of hospital use, including notional costs, were not significant in general. Differences in emergency admissions were greatest at the beginning of the trial, during which we observed a particularly large increase for the control group.

Conclusions Telehealth is associated with lower mortality and emergency admission rates. The reasons for the short term increases in admissions for the control group are not clear, but the trial recruitment processes could have had an effect.

Trial registration number International Standard Randomised Controlled Trial Number Register ISRCTN43002091.

Introduction

Efforts worldwide are dealing with the increasing prevalence of chronic disease among an ageing population. The past decade has seen the growing use of telehealth as one possible approach to this problem. Telehealth involves the remote exchange of data between a patient and healthcare professionals as part of the patient’s diagnosis and healthcare management.^{1 2} Examples include the monitoring of blood pressure and blood glucose. Telehealth may help patients to better understand their health conditions by providing tools for self monitoring, encourage better self management of health problems, and alert professional support if devices signal a problem. As a consequence, telehealth promises better quality and more appropriate care for each patient, as well as more efficient use of healthcare resources by reducing the need for expensive hospital care.

Some research suggests that telehealth can have a positive effect on patients with chronic disease, such as improved patient experiences, clinical indicators, and quality of life, and reduced use of secondary healthcare (including emergency hospital admissions).^{1 2} Yet, other studies have found either no effect or a negative effect.^{3 4} Furthermore, such evidence is usually based on assimilating findings from a number of small trials, which could be difficult to generalise,³ and with many of these trials not meeting robust evaluation standards.^{4 5} A recent review of self monitoring of blood glucose for people with diabetes concluded that there was a need for large controlled trials.⁶

Investment in telehealth has often been justified partly on the basis that its cost can be recovered by reductions in the use of secondary healthcare.⁷ However, assessing the scale of such an effect is complicated. Simple study designs comparing stages before and after an intervention can produce misleading results by not having a control group to compare with, particularly if the patients selected for intervention have a history of emergency care. Such patients have a tendency to show reductions in use of emergency care over time (that is, regression to the mean).⁸ Therefore, in the absence of a control group, whether observed reductions are the effect of the intervention is unclear.

Analyses of hospital use are further complicated by the fact that the distribution of admissions across patients can be highly skewed. Some high risk patients account for a very high proportion of admissions.⁹ Therefore, small differences in the risk profile of patients receiving the intervention can greatly affect observed outcomes in terms of hospital admission. Several predictive risk models have been developed that use information from a person’s health history to predict future hospital use,^{10 11} and can offer an opportunity for case mix adjustment. A further limitation on the size of previous evaluation studies has been the costs of obtaining information from patients, but it is now possible to extract information from operational administrative systems and use secure data linkage procedures to track resource use.

In 2006, the Department of Health in England published a white paper that included a focus on health and social care for people with long term needs.¹² The strategy proposed a series of demonstrator pilots to drive whole systems redesign, supported by advanced assistive technologies. These technologies included telehealth, along with a system of remote, automatic, and passive monitoring for patients with social care needs (known as telecare). The result was the Whole System Demonstrator project, funded by the Department of Health, which tested the benefits of integrated care supported by telehealth and telecare in three sites in England (Cornwall, Kent, and Newham).

One of the project’s aims was to test the effect of telehealth if delivered at a larger scale than existing pilot schemes, which were often limited to fewer than 100 patients. The resulting trial was pragmatic in design, to recruit and randomise suitably large numbers of patients and assess the effect of a broad class of telehealth and telecare technologies in the context of routine delivery of care provided by the United Kingdom’s health service.^{7 8} The telehealth part of the study included people diagnosed with chronic obstructive pulmonary disease, heart failure, or diabetes. These conditions have high prevalence and associated healthcare costs. The evaluation covered several different dimensions.¹³ This article is one of five analyses, and reports on how telehealth affected the use of secondary healthcare and mortality. The other analyses will assess how telehealth affected quality of life and cost effectiveness, and explores the patient, professional, and organisation factors related to implementation.

Methods

The trial protocol has been described by Bower and colleagues.¹³ Initial discussions with sites indicated that individual randomisation of patients would probably not be acceptable to stakeholders. Therefore, we used a pragmatic approach to randomise general practices. Participants in practices allocated to the control group were given usual care, which reflected the range of services available in the trial sites, excluding telehealth, and were offered telehealth or telecare at the end of the trial, if they were still eligible at that point.

Choices of telehealth devices and monitoring systems varied among the three trial sites, and there was no attempt to standardise these technologies across sites. We included a broad class of technologies, and the study was not designed or powered to examine differences between specific devices or monitoring systems. Although sites used different protocols for allocating peripheral devices, they all used a pulse oximeter for chronic obstructive pulmonary disease, a glucometer for diabetes, and weighing scales for heart failure. Sites asked participants to take clinical readings at the same time each day for up to five days per week, although the frequency was adjusted according to their individual history. For example, a participant with diabetes and well controlled blood glucose would be asked to take readings less frequently than another participant with poorly controlled blood glucose.

In addition to the telemonitoring aspect of the intervention, symptom questions and educational messages were sent to participants either via the telehealth base unit or via a set top box connected to a television. At the end of each session, data from clinical readings and symptom questions were transmitted to monitoring centres via a secure server. Monitoring centres were staffed by specialist nurses and community matrons from local health organisations, who used protocols to respond to the information from patients.

All practices in the geographical areas covered by three study sites (Cornwall, Kent, and Newham in England) were eligible to participate in the trial. Practices that accepted the invitation to participate were allocated to an intervention or control group via a centrally administered, minimisation algorithm that aimed to ensure that the groups of practices were similar in terms of practice size; deprivation index; proportion of non-white patients; prevalence of diabetes, chronic obstructive pulmonary disease, and heart failure; and site. Within each practice, patients aged 18 years or over were deemed eligible on the basis of a diagnosis in primary or secondary care of chronic obstructive pulmonary disease, diabetes, or heart failure. We did not confer eligibility on the basis of formal clinical assessment of disease severity. Instead, patients were deemed eligible on the basis of their inclusion on the relevant Quality Outcomes Framework register in primary care; a confirmed medical diagnosis in primary or secondary care medical records, as indicated by general practice Read codes or ICD-10 (international classification of diseases, 10th revision) codes; or confirmation of disease status by a local clinician (a general practitioner or community matron) or the patient’s hospital consultant. We did not exclude patients on the basis of additional physical comorbidities.

To meet ethical obligations, sites asked patients to complete and return a data sharing letter if they consented to their data being shared with the research team. Once this letter had been returned, patients received a “light touch” visit from members of the site project team, sometimes including clinical staff. These visits aimed to assess the suitability of the patient’s home for telehealth, provide information regarding the trial, and provide consent forms for participation. The study design required that patients were blinded at the point of consent. However, owing to the large sample size needed for the trial and the extended period of recruitment, it was not possible to guarantee the blinding of recruiters to the allocations of general practices throughout the process.

We assessed the effect of telehealth at the patient level. The primary endpoint was the proportion of people with an inpatient admission to hospital within the 12 month trial period. The study was powered on the basis of detecting a relative change of 17.5% from a baseline of 25% (from a priori site estimates), at 80% power, and a two sided value of P<0.05. Previous studies in the older population suggested that the intracluster correlation coefficient would be about 0.001.¹⁴ We did sample size calculations using the appropriate formulas,¹⁵ and found that 3000 patients would need to be recruited (25 patients from each of 120 general practices). We examined mortality over 12 months and prespecified secondary endpoints (including the number of inpatient bed days, emergency admissions, elective admissions, outpatient attendances, and emergency department visits, as well as the notional cost of hospital activity to commissioners of care based on national tariff costs).

Participants were linked at the person level to data for inpatient and outpatient secondary activity sourced from Hospital Episode Statistics, a national data warehouse for England.¹⁶ Participants were linked by the NHS Information Centre for health and social care, a trusted third party that was the only organisation to have access to both patient identifiers and data for secondary care activity. A linked mortality file provided data for all deaths occurring in and out of hospital. In addition, participants were linked to local commissioning datasets on visits to emergency departments, which included all visits to emergency department and not just those that resulted in an admission, and to general practice datasets. In these data, patient identifiable fields were removed before transfer and NHS numbers encrypted. We used this encrypted NHS number to link participants to the emergency and general practice data.

We restricted analysis of inpatient activity to ordinary admissions, and excluded transfers, regular ward attendances, and maternity events (leaving patient classifications 1 and 2 only). Admissions were classified by defined admission methods into emergency activity (codes 21-28) and elective activity (all other codes excluding transfers). Bed days included stays after emergency and elective admissions; same day admissions and discharges were assigned a stay length of 1 bed day. We restricted outpatient activity to appointments that were attended (codes 5 and 6).

A separate paper is planned to detail the effect of telehealth on costs. Here, we included notional costs of hospital care to summarise overall levels of hospital use in the intervention and treatment groups across the inpatient and outpatient categories. We estimated notional costs of care, from Hospital Episode Statistics data, by applying the set of mandatory and indicative tariffs used in England for the reimbursement of inpatient and outpatient care (2008-09 payment by results tariffs).¹⁷ These tariffs assume a stay of a certain number of days (the “trim point”), and allow hospitals to charge a prespecified amount for each additional excess bed day. Costs were not adjusted for the regional costs of providing care, and thus were effectively a weighted activity measure that allowed robust comparison of the magnitude of care received for control and intervention participants. We did not include costs for activity not covered by the tariffs, such as mental health, critical care, cystic fibrosis, high cost drugs, and outpatient physiotherapy.

The current study was restricted to those patients linked to administrative data who began the trial before 30 September 2009. The trial start date was taken as the date of telehealth installation for intervention patients, and as the date of the “light touch” visit for control patients. Analysis was based on comparing activity over 12 months after this date, at the person level.

Analysis of participants was on the basis of the intended treatment allocations, and regardless of subsequent withdrawal from the trial. For randomised trials, formal statistical tests on the similarity of intervention and control patients have been thought to be inappropriate, since allocations are known to have been random.¹⁸ However, in cluster randomised trials, selection bias is theoretically possible, either through systematic differences between practices in the control and intervention groups, or because of similar differences at the individual level.¹⁹ We presented standardised differences as a summary measure of differences between groups; we calculated a standardised difference as the difference between the sample means (or proportions), divided by the pooled standard deviation.²⁰

Although various aspects of the trial design mitigated against the risk of selection bias, differences between groups could still have occurred by chance. We applied case mix adjustment using three models to account for the effect of any differences between groups (box).

Box: Models used for case mix adjustment

Unadjusted model

The simplest models, although accounting for the effect of clustering, used no additional covariate adjustment.

Adjusted model

A more complex model additionally controlled for residual imbalances in a set of characteristics predictive of future hospital use. These characteristics included age, sex, ethnicity, site, number of chronic health conditions, principal long term condition (diabetes, chronic obstructive pulmonary disease, or heart failure), an area based deprivation score (national quartiles of the index of multiple deprivation 2007), and a metric corresponding to the endpoint calculated over several periods within the two years before recruitment. The number of chronic health conditions was a count of diagnoses recorded on inpatient data over the three years before starting the trial. We assigned principal long term conditions using a pragmatic approach according to published criteria.¹³

Combined model

More complex case mix adjustment was conducted using the combined model.¹¹ This model is a standard instrument designed to estimate the probability that an individual would be admitted to an emergency hospital department within a 12 month period. The combined model score accounts for 72 variables related to age, sex, recorded health conditions, previous hospital use, and prescriptions. These variables are sourced using administrative data from general practices as well as from local hospital commissioning datasets.

The combined model was originally derived using data for 2002-05;¹¹ we revalidated its performance on more recent data covering the period from April 2007 to March 2009. Revalidation used data extracted for the trial sites excluding trial participants. For the case mix adjustment, we calculated the combined model score for each participant at the end of the month before the start date. If a general practice did not grant approval for us to extract data, we imputed scores for patients on the basis of the available information, which included age, sex, and hospital data. We used single imputation on the basis of linear regression on the logit scale.

In a cluster randomised trial, hospital use for individuals in the same general practice will tend to be correlated. We accounted for this degree of clustering by constructing multilevel models that included random effects at the practice level. Logistic regression was used for the admission proportion and mortality, with the exponent of the coefficients used to calculate odds ratios. For emergency admissions, elective admissions, outpatient attendances, and emergency department visits, we used Poisson regression and exponentiated the coefficients to produce incidence rate ratios. Distribution of healthcare costs and hospital bed days are typically skewed, with some very large values and a considerable proportion of the population at zero. Although opinions differ on how to analyse such data,²¹ we incremented and log transformed notional costs and bed days to meet the assumptions needed for subsequent ordinary least squares modelling. Model coefficients were exponentiated to calculate geometric mean differences. We did all analyses in Stata 11.²²

The primary analysis assumed a 12 month follow-up for all patients, regardless of whether they died or not. This tested for differences between the groups in overall levels of hospital activity after the introduction of telehealth. However, clinicians and other healthcare professionals might also be interested in how telehealth would affect patients’ experiences of admission to hospital, which would depend on whether they were alive at that point. Such effects may be different if telehealth affected the mortality rate. Therefore, we did secondary analyses to assess group differences in admission rates at any point in time, on the condition that participants were alive just before that point in time and had not already been admitted. This analysis treated death as a form of statistical censoring and used the Kaplan-Meier curve.²³ Although the Kaplan-Meier curve did not take into account differences between intervention and control groups at baseline, we also estimated the corresponding adjusted hazard ratio. We calculated hazard ratios using a Cox proportional hazards model,²⁴ which included covariate adjustment according to the set of baseline variables (box) and random frailties to allow for homogeneity within practices.²⁵

Results

Data extraction, linkage, and processing

We allocated 238 practices to control or intervention groups. Although 59 practices eventually did not supply participants for the trial, sites assessed 15 171 patients for eligibility and sent data sharing consent forms; 5279 (34.8%) of these patients agreed to a “light touch” visit. Some patients did not consent to take part in the trial after this visit. Sites recruited 1625 control patients and 1605 intervention patients from 179 general practices (fig 1), with each practice recruiting an average of 18 patients. Recruitment started in May 2008 and was planned to finish in September 2009; we excluded seven patients who were recruited after this finish date. In addition, 69 patients could not be linked to administrative data on secondary care use. Overall, we included 1584 control patients and 1570 intervention patients in the analyses (98% of those recruited).

Fig 1 CONSORT diagram

We calculated full combined model scores for 1397 control and 1365 intervention patients (88% of those included in the analyses), and imputed scores for the remainder of patients. Most inpatient and outpatient hospital activity for the selected patients could be assigned unit costs using our methods. We assigned unit costs to 3189 (96.3%) of 3310 inpatient spells experienced by participants during the 12 months before the start of the trial, and to 13 670 (86.7%) of 15 766 outpatient attendances.

Baseline characteristics and trends in hospital activity

Tables 1 and 2 show that intervention and control patients were similar at baseline (all but one standardised difference <10%). The largest difference between intervention and control patients related to diabetes as an index condition (25.9% v 21.6%), followed by mean age (69.7 v 70.9 years). Intervention patients also had less costly hospital activity than controls in the 90 days before the start of the trial (£427 (€529; $662) v £506).

Table 1.

 Baseline characteristics of study patients. Data are no (%) of patients unless stated otherwise

	Control group (n=1584)	Intervention group (n=1570)	Standardised difference (%)
No of practices	85	92	—
No of patients per practice (median (range))	12 (1-98)	8 (1-77)	—
Index long term condition
Chronic obstructive pulmonary disease	786 (49.6)	739 (47.1)	−5.1
Diabetes	342 (21.6)	406 (25.9)	10.0
Heart failure	456 (28.8)	425 (27.1)	−3.8
No of chronic health conditions per patient (mean (SD))	1.8 (1.8)	1.8 (1.8)	−3.0
Site
Cornwall	614 (38.8)	558 (35.5)	−6.7
Kent	576 (36.4)	563 (35.9)	−1.0
Newham	394 (24.9)	449 (28.6)	8.4
Age
Mean (SD) age in years	70.9 (11.7)	69.7 (11.6)	−9.7
<65 years	446 (28.2)	463 (29.5)	2.9
65-74 years	500 (31.6)	548 (34.9)	7.1
75-84 years	500 (31.6)	446 (28.4)	−6.9
≥85 years	138 (8.7)	113 (7.2)	−5.6
Female	643 (40.6)	647 (41.2)	1.3
Ethnicity
White	1168 (73.7)	1127 (71.8)	−4.4
Non-white	173 (10.9)	182 (11.6)	2.1
Unknown	243 (15.3)	261 (16.6)	3.5
Area level deprivation*
Mean (SD)	27.9 (13.5)	28.4 (14.8)	2.9
First quartile	105 (6.6)	137 (8.8)	8.0
Second quartile	272 (17.2)	258 (16.5)	−1.8
Third quartile	571 (36.1)	522 (33.4)	−5.6
Fourth quartile	633 (40.0)	644 (41.3)	2.5
Combined model score†
Mean (SD)	0.261 (0.200)	0.260 (0.202)	−0.3
Low risk category	226 (16.2)	221 (16.2)	0.0
Moderate risk category	431 (30.9)	440 (32.2)	3.0
High risk category	591 (42.3)	559 (41.0)	−2.7
Very high risk category	149 (10.7)	145 (10.6)	−0.1

SD=standard deviation.

*N=1581 for control group, n=1561 for intervention group. First quartile is least deprived, fourth quartile is most deprived.

†N=1397 for control group, n=1365 for intervention group. Risk categories denote top proportions of site population: very high risk (0.5%), high risk (0.5-5%), moderate risk (5-20%), and low risk (20-100%).

Table 2.

 Baseline characteristics of study patients, in relation to previous hospital use before start of trial. Data are mean (standard deviation) unless stated otherwise

Characteristic and period before start of trial	Control group (n=1584)	Intervention group (n=1570)	Standardised difference (%)
Admission proportion (%)
1-90 days	17.5 (n=277)	16.3 (n=256)	−3.2
91-365 days	39.3 (n=622)	39.3 (n=617)	0.1
366-730 days	49.6 (n=785)	47.1 (n=740)	−4.9
Emergency admissions per head
1-90 days	0.13 (0.45)	0.11 (0.44)	−3.3
91-365 days	0.45 (1.01)	0.42 (0.94)	−3.8
366-730 days	0.60 (1.19)	0.54 (1.07)	−5.5
Bed days per head
1-90 days	0.94 (4.69)	0.84 (3.81)	−2.3
91-365 days	3.54 (11.25)	3.00 (8.96)	−5.3
366-730 days	4.65 (11.45)	4.57 (12.77)	−0.6
Elective admissions per head
1-90 days	0.11 (0.40)	0.11 (0.48)	0.5
91-365 days	0.36 (0.95)	0.36 (1.10)	0.6
366-730 days	0.52 (1.29)	0.49 (1.30)	−2.3
Outpatient attendances per head
1-90 days	1.23 (1.97)	1.14 (1.97)	−4.7
91-365 days	3.87 (5.24)	3.75 (4.98)	−2.4
366-730 days	4.75 (6.35)	4.63 (6.12)	−1.9
Emergency department visits per head
1-90 days	0.13 (0.44)	0.14 (0.48)	1.1
91-365 days	0.41 (1.05)	0.42 (1.10)	0.9
366-730 days	0.47 (1.05)	0.46 (1.21)	−1.1
Tariff cost per head (£ per head)
1-90 days	506 (1311)	427 (1177)	−6.3
91-365 days	1879 (3712)	1706 (3116)	−5.1
366-730 days	2411 (4342)	2296 (4143)	−2.7

Figure 2 shows trends in hospital activity without adjusting for clustering or baseline covariates (that is, as crude data). The break in the chart corresponds to the trial start date, and the chart summarises activity over a series of quarters before and after this date. Rates of emergency hospital admission had peaked for both intervention and control groups at around six quarters before the start of the trial. After the trial began, emergency admissions increased for the control group, from 0.13 per head in the quarter immediately before to 0.18 per head in the quarter immediately after. Emergency department visits also increased over this period, and the rate of conversion of emergency department visits into admissions rose slightly from 53% to 58%. After the initial increase in activity for the control group, rates of emergency admission for the two groups began to converge, although a difference in favour of the intervention group seemed to persist for the entire follow-up period. Table 3 shows summary figures for the crude rates of hospital activity during the follow-up period.

Fig 2 Crude trends in secondary care activity for patients recruited into the study

Table 3.

 Hospital use and mortality during trial (unadjusted for clustering and covariates). Data are mean (standard deviation) unless stated otherwise

	Control group (n=1584)	Intervention group (n=1570)	Absolute difference (95% CI)	Percentage difference (95% CI)
Admission proportion (%)	48.2 (n=763)	42.9 (n=674)	−5.2 (−8.7 to −1.8)	−10.8% (−18.1% to −3.7%)
Mortality (%)	8.3 (n=131)	4.6 (n=72)	−3.7 (−5.4 to −2.0)	−44.5% (−65.3% to −23.8%)
Emergency admissions per head	0.68 (1.41)	0.54 (1.16)	−0.14 (−0.23 to −0.05)	−20.6% (−33.8% to −7.4%)
Elective admissions per head	0.49 (1.31)	0.42 (0.99)	−0.07 (−0.15 to 0.01)	−14.3% (−30.6% to 2.0%)
Outpatient attendances per head	4.68 (6.81)	4.76 (6.74)	0.08 (−0.39 to 0.55)	1.7% (−8.3% to 11.8%)
Emergency department visits per head	0.75 (1.58)	0.64 (1.26)	−0.11 (−0.21 to −0.01)	−14.7% (−28.0% to −1.3%)
Bed days per head	5.68 (15.10)	4.87 (14.35)	−0.81 (−1.84 to 0.22)	−14.3% (−32.4% to 3.9%)
Tariff costs per head (£)	2448 (4099)	2260 (4117)	188 (−474.9 to 98.8)	−7.7% (−19.4% to 4.0%)

Analysis of primary and secondary endpoints

Of the intervention participants, 42.9% were admitted to hospital during the 12 months of the trial, compared with 48.2% of controls (table 3). These proportions corresponded to an unadjusted odds ratio of 0.82 (95% confidence interval 0.70 to 0.97, P=0.017; table 4). The odds ratio takes into account clustering at the general practice level. The intracluster correlation coefficient (0.017) was higher than assumed in the original power calculations.

Table 4.

 Results of mixed models (including case mix adjustment)

Endpoint (interpretation)	Model	Estimate (95% CI)	P
Admission proportion (odds ratio)	Unadjusted	0.82 (0.70 to 0.97)	0.017
	Adjusted	0.82 (0.69 to 0.98)	0.026
	Combined model	0.82 (0.69 to 0.96)	0.016
Mortality (odds ratio)*	Unadjusted	0.54 (0.39 to 0.75)	<0.001
	Combined model	0.53 (0.39 to 0.72)	<0.001
Emergency admissions (incidence rate ratio)	Unadjusted	0.81 (0.65 to 1.00)	0.046
	Adjusted	0.85 (0.72 to 1.00)	0.056
	Combined model	0.81 (0.69 to 0.95)	0.011
Elective admissions (incidence rate ratio)	Unadjusted	0.89 (0.75 to 1.07)	0.219
	Adjusted	0.87 (0.74 to 1.02)	0.078
	Combined model	0.90 (0.76 to 1.07)	0.241
Outpatient attendances (incidence rate ratio)	Unadjusted	0.96 (0.81 to 1.13)	0.621
	Adjusted	1.01 (0.92 to 1.12)	0.814
	Combined model	0.95 (0.81 to 1.13)	0.575
Emergency department visits (incidence rate ratio)	Unadjusted	0.85 (0.70 to 1.05)	0.135
	Adjusted	0.85 (0.73 to 1.00)	0.044
	Combined model	0.86 (0.72 to 1.02)	0.091
Bed days (difference in geometric means)	Unadjusted	−0.64 (−1.14 to −0.10)	0.023
	Adjusted	−0.44 (−0.85 to −0.01)	0.047
	Combined model	−0.58 (−1.00 to −0.13)	0.013
Tariff costs (difference in geometric means (£))	Unadjusted	−449 (−964 to 243)	0.181
	Adjusted	−242 (−629 to 228)	0.290
	Combined model	−382 (−840 to 206)	0.184

*One of the models did not converge for mortality.

The odds ratio for the admission proportion at 12 months remained significant after we adjusted for baseline characteristics and also after we adjusted for the combined model score. We assessed the performance of the combined model as a discriminatory tool to predict emergency hospital admissions by using data from the trial sites (excluding trial participants). We estimated the area under the receiver operating characteristics curve to be 0.746 (n=1 523 038).

During the trial, fewer participants died in the intervention group than in the control group (4.6% v 8.3%; unadjusted odds ratio 0.54, 95% confidence interval 0.39 to 0.75, P<0.001). Differences remained significant after adjustment for the combined model score, although we could not adjust for the set of baseline characteristics because models did not converge.

Of the secondary endpoints, emergency admissions, emergency department visits, bed days, and mortality showed significant findings in some or all of the models. Intervention participants underwent 0.54 emergency admissions per head, compared with 0.68 for controls (crude rates), corresponding to an unadjusted incidence rate ratio of 0.81 (95% confidence interval 0.65 to 1.00, P=0.046). However, after we adjusted for baseline characteristics, the upper end of the confidence interval for emergency admissions reached 1.

On average, intervention participants attended emergency departments 0.64 times per head during the trial, compared with 0.75 for controls. This difference was significant in the adjusted estimates only (incidence rate ratio 0.85, 0.73 to 1.00, P=0.044). The intervention and control groups spent an average of 4.87 and 5.68 days in hospital, respectively (unadjusted geometric mean difference −0.64 days, −1.14 to −0.10, P=0.023); this difference remained significant after adjustment.

Notional costs of hospital activity to commissioners of care were £188 per head lower for intervention participants than for controls (crude rates). Confidence intervals for the geometric mean were very wide and differences were not significant in any of the models (adjusted geometric mean difference −£242, 95% confidence interval −629 to 228, P=0.290).

Secondary analysis of the Kaplan-Meier curves and Cox regression (fig 3) confirmed that differences in the admission proportion remained significant after censoring observations at death (hazard ratio 0.87, 95% confidence interval 0.78 to 0.98, implying fewer admissions for the telehealth than control group). Graphical methods indicated that the underlying proportional hazards assumption was reasonable. Where Schoenfeld residual tests were significant,²⁶ results remained robust to alternative model specification.

Fig 3 Kaplan-Meier survival analysis for admission proportion

Discussion

Principal findings of the study

Among a set of patients with chronic obstructive pulmonary disease, diabetes, or heart failure, this study has shown that a smaller proportion of telehealth users than controls were admitted to hospital during a 12 month follow-up. This effect remained significant after adjusting for baseline characteristics and for a predictive risk score. However, the magnitude of the group difference in admission proportion was relatively small (10.8%, 95% confidence interval 3.7% to 18.1%), and smaller than the size that the planned study design was able to detect (17.5%), raising questions about the clinical relevance of the results. The significance of some of the effects reflected the increased power of the study, owing to the higher than assumed baseline level of admissions and to the larger number of small practices (even though the intracluster correlation coefficient was higher than assumed).

Intervention patients were significantly less likely to die within 12 months than controls. We also observed small differences in the mean number of emergency admissions per head between the intervention and control groups (crude rate 0.54 v 0.68; difference 0.14). These changes were significant in the unadjusted comparisons and when we adjusted for a predictive risk score, but not when we adjusted for baseline characteristics. Hospital bed days were significantly lower among intervention patients than controls, which reflected the reduced admission proportion overall.

For the other measures of hospital use (including the number of elective admissions, outpatient attendances, and emergency department visits), group differences were not significant in general. Crude differences in notional hospital costs to commissioners of care were also not significant and were relatively small (£188 per head over 12 months), especially compared with the potentially high costs of these types of telehealth intervention^{27 28 29}, which we did not take into account. In view of our results showing confidence intervals crossing the line of no difference, we cannot conclude that telehealth reduces secondary care costs over 12 months. A formal cost effectiveness analysis of the Whole System Demonstrator intervention has been undertaken on a subset of participants, using self reported data for hospital use and other services and taking into account the intervention cost, compared with health related quality of life and other outcomes.

Differences in hospital use were at their most marked at the start of the trial, when we observed a distinct increase in admissions for the control group. If we excluded activity from the first three months of the trial, differences in the admission proportion would not have been significant under any of the models. Therefore, this increase has implications for the interpretation. Trial recruitment processes may have led indirectly to changes in service use for control patients; however, the same processes might also have affected intervention patients in the absence of telehealth. In this case, differences in admissions can be attributed to telehealth, but with the limitation that the trial could have affected the context for the delivery of care for both trial groups.

One explanation for the increase in emergency admissions observed for the control group is that professionals may have identified additional health problems and unmet needs during the recruitment process and could have decided to intervene with control patients not allocated to receive telehealth. This explanation assumes that, if professionals found additional relevant health problems among the intervention group, they were content to manage these in a community setting, with the support of the remote monitoring.

The trial recruitment process might have also raised patients’ awareness of their health conditions. Anxiety could have increased as a result of being allocated to the control group, to the extent that these patients were more likely to present at emergency departments and be admitted than intervention patients. The decision to offer telehealth to control patients at the end of the 12 month period, while designed to reduce attrition rates, could have increased anxiety if it encouraged a stronger sense that they were being denied access to support that could be beneficial. A final possible explanation for the increase is that biases could have resulted during patient selection, in which only 35% of patients agreed to the initial “light touch” visit. There could have been a propensity to select controls with a higher risk of short term admission and intervention patients with a lower risk. However, observed differences were limited between intervention and control groups, and case mix adjustment was applied.

Strengths and weaknesses of the study

This particular analysis is one of a series planned by the Whole System Demonstrator Evaluation Team, and was limited to comparisons of inpatient, outpatient, and emergency department hospital use and mortality. We did not consider the full range of health and social care services, and intervention and control groups could have had differences in the use of primary care, community services, or social care. Telehealth could also have had different effects according to long term condition or other patient characteristics. Although we assigned unit costs to more than 95% of inpatient activity, we did not consider costs for some elements of hospital care, including mental health and critical care, which had no national tariff. Use of national tariffs meant that the analysis was relevant to decisions made by commissioners of care, who align with hospital reimbursement guidance, but the economic costs of providing care will differ from the notional costs shown here, and there are regional differences in the costs of providing care.

Although service use could have resource implications, it does not necessarily correlate with health status. Assessment of the effect of interventions should be multidimensional,³⁰ and important differences could also exist in health outcomes, cost effectiveness, and patient perceptions. These outcomes are explored in the related theme analyses. Telehealth could also have had knock-on effects in non-study patient groups, by freeing up clinical time and resources to care for non-study patients, or by diverting the attention of community teams towards those patients on the trial.

The study used administrative datasets. As a result, person level data were available for 98% of participants. Although these datasets avoided problems of non-response, the quality of data was not directly under the research team’s control. Patients tend to underestimate resource use compared with healthcare providers,³¹ but several studies have pointed out potential problems with using administrative data, such as limited insight into the quality and appropriateness of care.^{32 33}

Selection bias is recognised as a risk in cluster randomised trials, in which systematic differences can occur between intervention and control groups at both the cluster and individual level.¹⁹ At the individual level, if the trial recruiters had foreknowledge of the allocation group (as was often the case here), bias can result through the recruitment of different types of participant into the two groups. We designed this trial to minimise the possibility of bias within the context of a complex community based intervention. An independent team randomised allocations of practices and a minimisation algorithm aimed to ensure that intervention and control practices were similar in terms of practice size, disease prevalence, and other characteristics. At the individual level, we found no large differences in the characteristics of control and intervention participants at baseline. However, we saw group differences in the median number of participants per general practice (8.5 for telehealth v 12 for controls). Case mix adjustment controlled for observed differences between intervention and control groups.

We based this analysis on an intention to treat method, which compares patients according to their assigned intervention or control group. Although some patients did not receive their allocated interventions, these numbers were small. A substantial proportion of the intervention group could have stopped using telehealth before the end of the 12 months. This study had conservative estimates because, in other telehealth applications, equipment might be removed from patients who stop using it.³⁴ The trial design aimed to minimise differential rates of attrition between intervention and control groups, by ensuring that all practices were allocated to receive a telemonitoring intervention (telehealth or telecare), and that control participants were offered a telemonitoring intervention at the end of the trial period, if they were still eligible.

The effect of telehealth should be considered as just one element within the health system in which it was used. All participating practices and patients in the study could have benefited from the wider service redesign associated with these trials, including those assigned to usual care. Therefore, the study assessed the added value of telehealth over and above the effects of this wider service redesign.

The study aimed to review a broad class of telehealth devices and not to compare specific devices and monitoring systems. Our results reflect specific models of the introduction of telehealth, based on a series of decisions by local teams. There were differences in the interventions offered by the three sites. Although this plurality might be seen as problematic for the purposes of replicating and linking specific aspects of the interventions to likely changes in hospital use, in some ways it is the merit of a pragmatic trial. Other sites introducing telehealth will make choices driven by local contexts in the same way as did the sites in the present study, so the ability to reflect real life applications of telehealth will add generalisability to the study findings. However, the sites were specifically chosen for their innovations in these areas of care, and conclusions about the effectiveness of telehealth might depend on the environment in which it is used.

Strengths and weaknesses in relation to other studies

Assessment of the effectiveness of telehealth is usually based on assimilating evidence from several small trials, which could make findings difficult to generalise. By contrast, with over 3100 participants, this study is one of the largest randomised trials of telehealth done so far. The focus on combining three disease groups is novel and allowed us to examine the overall effect of telehealth across populations with chronic disease. However, telehealth could have varying effects in different groups. Other studies have typically examined condition groups separately; thus, comparisons between studies are not straightforward.

Paré and colleagues reviewed 65 empirical studies across four conditions and distinguished the effects of telehealth between different conditions. They suggested that effects on a range of measures (for example, reduced visits to emergency departments, hospital admissions, and average length of hospital stay) were more consistent in pulmonary and cardiac disorders than in diabetes and hypertension.³⁵

A systematic review of studies for heart failure reported that telemonitoring reduced all cause mortality, whereas both telemonitoring and structured telephone support reduced admissions for heart failure.³⁶ However, findings were based on generalising a large number of studies with a mean sample size of 330. A study showed that a telehealth intervention that included portable devices significantly reduced costs and admissions for people with heart failure. This was based on a sample size of 460, although the study was adequately powered for the larger effect size it assumed.²⁹ A more recent study of 1653 patients with heart failure found no significant effect on hospital use or mortality.³⁷ In relation to the management of chronic obstructive pulmonary disease, several studies showed lower rates of emergency admissions for patients receiving home monitoring plus telephone support,^{38 39 40} although one review noted that mortality rates were greater in patients receiving telephone support than those receiving usual care.³⁹ Evaluations of telehealth interventions for people with diabetes have focused on the achievement of a clinical outcome in terms of glycaemic control,^{41 42} with some reported success.

There is also a question about the effect of interventions that combine telemonitoring with educational and motivational tools, such as those we studied, compared with interventions consisting of telemonitoring only. Our study was not designed to answer this question. In a study by Domingo and colleagues of patients with heart failure receiving multidisciplinary care, those who used motivational support tools combined with telemonitoring of weight, blood pressure, and heart rate, thought that their quality of life had improved. These patients also spent less time in hospital after the intervention than before, although before and after comparisons can be vulnerable to regression to the mean. Researchers randomised patients to groups with and without the telemonitoring element and found no significant group differences.⁴³

Possible explanations and implications for clinicians and policymakers and other researchers

Our results suggest that telehealth helped patients to avoid the need for emergency hospital care. The mechanism for this is not yet clear. Telehealth could help patients manage their conditions better and therefore reduce the incidence of acute exacerbations that need emergency admissions. Telehealth could also change people’s perception of when they need to seek additional support, as well as professionals’ decisions about whether to refer or admit patients. Further analyses will provide insights into the mechanisms by which telehealth can lead to reductions in admission rates.¹³

The reduced mortality observed in the intervention group will be an important motivator to invest in these interventions and similar technologies. Although the observed difference in emergency admissions associated with the intervention indicates some potential to reduce use of secondary care, the findings need to be tempered by the estimated scale of the difference in notional hospital cost savings for commissioners of care and the cost of the intervention. Furthermore, the increases seen in emergency admissions among control participants suggest that the trial recruitment processes had an impact. The effect on quality of life must also be considered as part of a broader cost effectiveness analysis. For commissioners of care services, there are questions about whether any reduction in hospital use for patients receiving telehealth translates to an overall change at the organisational level. Any bed days released as a result could be filled with non-study patients rather than released as cash savings. In turn, this could have meant that health benefits accrued to non-study patients, which were not taken into account here.

The observation of a group effect between intervention and controls could mask differences by subgroups. For local practitioners, it is important to assess whether benefits of telehealth are greater in particular patient types, to inform decisions about prioritising the intervention in specific patient groups. For example, Maclean and colleagues observed that telehealth interventions probably did not result in clinically relevant improvements in health outcomes in patients with relatively mild asthma,⁴⁴ but could have a role in patients with more severe disease who are at high risk of admission. The current study was not designed to answer these specific questions.

The effect of telehealth could be intricately linked to wider issues about how health systems operate. It is unclear whether effects are attributable to the technology itself or attributable to how it is implemented,³⁴ and telehealth could be disruptive because it requires some professional groups to work in different ways.

This analysis is one part of the complete evaluation, and the Whole System Demonstrator trial in its entirety will allow a wider discussion of issues around the effects on cost effectiveness, quality of life, and patients’ and carers’ experiences as well as changes at the organisational level.

What is already known on this topic

• The known effect of telehealth on secondary healthcare use for patients with chronic diseases has been based on assimilating findings from several small trials, often with conflicting results

What this study adds

• Among people with chronic obstructive pulmonary disease, heart failure, or diabetes, a broad class of telehealth technologies could be associated with reduced rates of mortality and emergency hospital admission

• This effect, however, could be linked to short term increases in hospital use observed in the control group that may have been affected by recruitment processes during the trial

• The estimated scale of hospital cost savings for commissioners of care is modest, and the cost of the telehealth intervention should also be taken into account

Cite this as: BMJ 2012;344:e3874