Skip to main content
The BMJ logoLink to The BMJ
. 2002 Sep 14;325(7364):598.

Systematic review of cost effectiveness in telemedicine

Quality of cost effectiveness studies in systematic reviews is problematic

Paul Scuffham 1
PMCID: PMC1124113  PMID: 12228145

Editor—Witten et al reviewed the cost effectiveness of telemedicine.1 In their response on bmj.com they confuse the evaluation of quality with levels of evidence and study design.2 Quality should not be judged on levels (strength) of evidence or whether health consequences were measured but on how well these were undertaken and reported.3

Additionally, studies were rated on the “presence of a clear hypothesis.” Although essential for assessing efficacy, this is less important for economic evaluations. Hypotheses require statistical testing to acceptance or rejection of the null hypothesis; economic evaluations typically use point estimates and, without variance, cannot be subjected to statistical testing. Hence, sensitivity analyses are required for testing the robustness of results. A clear statement of aims is required, and as noted by Witten et al, most economic evaluations do clearly state the aims.

Economic evaluations of telemedicine often concentrate on costs and cost minimisation, assuming health outcomes are at least as good as those from conventional services. For example, in teleradiology a reasonable assumption is that patient outcomes will be equivalent to or better (such as in terms of quicker results and avoiding travel) than conventional reporting by a radiologist. This is the conservative approach; if benefits were measured and valued the cost effectiveness of telemedicine might only appear better. Cost neutrality, used in cost analyses and cost minimisation analyses, suggests that telemedicine provides the same health outcomes without additional costs. Cost effectiveness is concerned with additional costs for additional units of benefit; it requires a value judgment about whether additional spending to obtain additional benefits is worth while.

I was disappointed that the authors did not report circumstances where telemedicine was and was not cost effective. For example, live consultations are resource and cost intensive compared with asynchronous consultations, but occasionally—for example, the highlands and islands teledentistry project4—might be cost effective. Likewise, telemedicine might be cost effective for some disciplines such as dermatology and radiology but not for others.

In the critique of the generalisability of results, equity issues of access to healthcare services were overlooked. Many communities in the United Kingdom have barriers in accessing common healthcare services. For example, the referral rate for restorative dental consultations from the Shetland Islands (requiring travel to Aberdeen) is a 10th of that of the Orkney Islands, where a consultant visits for 1-2 days each year.4 Teledentistry and other telemedicine interventions, even with additional costs, help address inequities in access to healthcare services.

References

  • 1.Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ. 2002;324:1434–1437. doi: 10.1136/bmj.324.7351.1434. . (15 June.) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Haycox AR. Re: Economic evaluation is a broad church. bmj.com 2002. bmj.com/cgi/eletters/324/7351/1434#23142 (accessed 19 August 2002).
  • 3.NHS Centre for Reviews and Dissemination. Undertaking systematic reviews of research on effectiveness. 2nd ed. York: University of York; 2001. . (CRD report No 4.) [Google Scholar]
  • 4.Scuffham P, Steed M. An economic evaluation of the highlands and islands teledentistry project. J Telemed Telecare. 2002;8:165–177. doi: 10.1177/1357633X0200800307. [DOI] [PubMed] [Google Scholar]
BMJ. 2002 Sep 14;325(7364):598.

Authors' reply

A R Haycox 1, F S Mair 1

Editor—We did not try to introduce new evidence on telemedicine in our review but simply considered evidence already in existence. We emphasise that a lack of evidence for the cost effectiveness of telemedicine does not imply evidence of a lack of cost effectiveness for telemedicine. Our review identified the former without contributing towards the debate on the latter.

One weakness of the evidence base was its inability to identify circumstances where telemedicine was or was not cost effective. Perhaps more than in any other area of new technology, the cost effectiveness of telemedicine is location specific. This is not to say that results obtained in one location (the highlands of Scotland) cannot be generalised to other locations (the centre of Liverpool) but that such generalisation must be undertaken with the utmost care.

Specifically, an impact model should be developed to distinguish between elements that contribute to the success of a telemedicine service and can be generalised throughout the NHS—for example, technological factors—and those that are location specific—for example, geographical and environmental factors. Often, the success of a telemedicine service evaluated in inaccessible locations has been automatically assumed to be generalisable elsewhere, with no attempt being made to test this assumption. Our review presents a challenge to the telemedicine research community to strengthen its analyses by identifying the extent to which results obtained in one research location are likely to be replicable elsewhere.

Our review is also an early overview of the quality of research currently available in telemedicine. Although it is timely, given the growing interest in this branch of medicine, we also acknowledge that we reviewed a technology in its early stage of development. We would thus welcome the opportunity to revisit our review, hopefully to reflect an evidence base that has been enhanced in response to the challenges we have presented. We are generally persuaded of the potential value of telemedicine, but our role in undertaking the review was not to be for or against telemedicine but simply to reflect the quality of the current evidence base. We hope we have highlighted some of the areas in which the quality of this evidence base can be improved.


Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES