Systematic review of studies of patient satisfaction with telemedicine

Abstract

Objective

To review research into patient satisfaction with teleconsultation, specifically clinical consultations between healthcare providers and patients involving real time interactive video.

Design

Systematic review of telemedicine satisfaction studies. Electronic databases searched include Medline, Embase, Science Citation Index, Social Sciences Citation Index, Arts and Humanities Citation Index, and the TIE (Telemedicine Information Exchange) database.

Subjects

Studies conducted worldwide and published between 1966 and 1998.

Main outcome measures

Quality of evidence about patient satisfaction.

Results

32 studies were identified. Study methods used were simple survey instruments (26 studies), exact methods not specified (5), and qualitative methods (1). Study designs were randomised controlled trial (1 trial); random patient selection (2); case-control (1); and selection criteria not specified or participants represented consecutive referrals, convenience samples, or volunteers (28). Sample sizes were ≤20 (10 trials), ≤100 (14), >100 (7), and not specified (1). All studies reported good levels of patient satisfaction. Qualitative analysis revealed methodological problems with all the published work. Even so, important issues were highlighted that merit further investigation. There is a paucity of data examining patients' perceptions or the effects of this mode of healthcare delivery on the interaction between providers and clients.

Conclusions

Methodological deficiencies (low sample sizes, context, and study designs) of the published research limit the generalisability of the findings. The studies suggest that teleconsultation is acceptable to patients in a variety of circumstances, but issues relating to patient satisfaction require further exploration from the perspective of both clients and providers.

Introduction

Telemedicine can be defined as the use of telecommunications technologies to provide medical information and services.¹ There is increasing interest in the use of telemedicine as a means of healthcare delivery. This is partly because technological advances have made the equipment less expensive and simpler to use and partly because increasing healthcare costs and patient expectations have increased the need to find alternative modes of healthcare delivery.

A wide variety of studies concerning telemedicine, interactive video consultations, have been performed in different settings throughout the world. Commentators on telemedicine frequently highlight the need for research into safety, efficacy, and cost effectiveness. Telemedicine literature abounds with publications about patient satisfaction, which are generally positive, and as a result there is a tendency to assume that the need for further research into this is less of a priority.

We argue in this paper that (a) the available research fails both to provide satisfactory explanations of the underlying reasons for patient satisfaction or dissatisfaction with telemedicine and to explore communication issues in any depth and (b) generalisations about satisfaction with telemedical care are difficult because of methodological deficiencies of the current evidence. To support this perspective, we provide the results of a systematic literature review of research into telemedicine satisfaction, in the context of interactive video.

Methods

Search strategy

To identify telemedicine satisfaction studies the following electronic databases were searched: Medline 1966 to 1998, Embase 1988-98, Science Citation Index 1981-98, Social Sciences Citation Index 1981-98, Arts and Humanities Citation Index 1981-98, and the TIE (Telemedicine Information Exchange) database. Searches were restricted to English language papers, and the keywords used were: “patient satisfaction,” “consumer satisfaction,” “telecommunications,” and “telemedicine.” The reference lists of papers identified were hand searched for other relevant references.

We included only clinical trials that explored patient satisfaction with teleconsultation, specifically those clinical consultations between healthcare providers from any discipline and patients that involve the use of real time interactive video. We excluded review or discussion papers, studies in which the use of telecommunications technologies was primarily for educational or administrative purposes and not linked to direct patient care, and studies in which the patient was not physically present at either point of care. In addition, if any single study resulted in multiple publications, we reviewed only the principal paper focusing on patient satisfaction. The studies we reviewed had evaluation of patient satisfaction either as the main outcome measure or at least as a prominent feature of their overall assessment of the project. We did not include telemedicine projects that did not directly measure patient satisfaction but reported “unsolicited feedback” that suggests a reasonable degree of satisfaction with telemedicine services.²

Selection criteria

It is acknowledged that well designed and executed trials, particularly randomised controlled trials, provide the most reliable evidence for inclusion in any systematic review.³ However, in view of the limited number of patient satisfaction studies that met the search criteria outlined above, we analysed data from all clinical trials identified irrespective of sample size or methodologies used. Titles and abstracts of the studies identified by the outlined search strategy were read to determine their potential eligibility for the review. The full articles were then assessed for relevance.

Outcome measures and data extraction

The outcome measures we examined included patients' satisfaction (principally overall satisfaction with the telemedicine service but also including levels of satisfaction with communication via this medium, telemedicine consultations compared with traditional face to face consultations, and technical performance) and patients' willingness to use telemedicine in the future.

We recorded the studies' bibliographic details; descriptions of study setting and study population; subject selection criteria; details of form and delivery of the intervention; and outcome measures. We also noted patient numbers, response rates, study methodologies, and other factors affecting the validity of results, including effect modifiers.

Qualitative analysis

In view of the heterogeneous nature of the studies identified, the dearth of randomised controlled trials, and the preponderance of demonstration and feasibility studies, the data available did not permit the use of formal statistical techniques such as meta-analysis. Instead, we conducted a broad qualitative overview of the data, including a critical review of the strength of the findings. We judged the reliability and validity of data by the methodologies used in each study and judged their generalisability from the study context. We did not use a formal scoring method as no well validated instrument for qualitative review yet exists.³,⁴ However, as the basis for our critical appraisal of the studies, we used a checklist designed for assessing the methodological quality of both randomised and non-randomised studies of healthcare interventions.⁴

Results

Thirty two studies met our selection criteria. The studies examined the use of interactive video in diverse contexts ranging from specialist consultations to home nursing. Many of these represented demonstration and feasibility studies rather than full scale trials. This is reflected in their sample sizes often being small and selection criteria for study participants rarely being random in nature. Only seven studies had more than 100 participants,^5–11 14 were small pilot studies with less than 100 patients,^12–25 and 10 were simple feasibility studies with 20 or fewer patients.^26–35 One paper, which presented an overview of an Australian regional telepsychiatry project, did not provide patient numbers.³⁶

The table lists the studies by type of consultation. (An extra table on the BMJ 's website provides further detail of studies in which patient numbers were over 20 and methods of measuring patient satisfaction were explicitly described. None of these studies declared any conflicts of interest.)

In terms of methodologies used, 26 studies used simple survey instruments, five did not specify the exact methods, and one used qualitative methods. Only one study was a randomised controlled trial,¹⁰ in two others patients were randomly selected,¹⁹,²³ and one was a case-control study.²⁰ In the remaining 28 studies selection criteria were not specified or participants represented consecutive referrals, convenience samples, or volunteers.

Measures of patient satisfaction

The studies mainly used simple survey instruments to ascertain patient satisfaction. Firm conclusions are limited by methodological difficulties, but it would seem that the patients found teleconsultations acceptable; noted definite advantages, particularly increased accessibility of specialist expertise, less travel required, and reduced waiting times; but also had some disquiet about this mode of healthcare delivery, particularly relating to communication between provider and client via this medium.

Shortcomings of studies

We identified several problems with the studies that affect their reliability and validity. Many studies had small sample sizes, almost a third having 20 or fewer participants, and low response rates, as low as 50%.²² Patient selection criteria were often not clearly specified, or there were no formal selection criteria. Most of the studies (28) used volunteers or physician referrals and provided no information about refusal rates at point of initial referral. Thus, it is not possible to discount selection bias in favour of those likely to be positive about teleconsultation.

Methodologies used for assessing satisfaction were not clearly specified in many studies, making interpretation and comparison of results problematic. Most studies sought to measure whether patients would use the systems again or were “satisfied” with the service. However, few studies defined what satisfaction meant. Therefore, we are unable to discern whether the participants said they were satisfied because telemedicine didn't kill them, or that it was “OK,” or that it was a wonderful experience. The available evidence does not help us to understand the reasons underlying satisfaction or dissatisfaction. In addition, most of the studies presented only initial impressions and failed to explore what happened to patient satisfaction over time, thereby making it possible that the novelty value of the technology resulted in a positive bias.

The cost of teleconsultations compared with routine consultations was not addressed. This is particularly pertinent to the US studies, which account for over 45% of the studies found. The US system of healthcare delivery is a fee for service system, yet the US studies do not mention whether patients attending for teleconsultation paid for the service in the usual way or whether they received this service free of charge. As many US telemedicine projects are primarily grant funded, it is possible that in some studies participants received free teleconsultations, which could affect their satisfaction with the service provided.

Because of the survey nature of most of the studies, there are often inconsistencies in responses that remain unexplained. One possible explanation lies in the survey design. Many surveys have questions with multiple constructs (such as: “I felt the physician was easy to talk to and understood everything I said”). When a single question contains two constructs it is not possible to know which actual construct the participant is responding to, making the data difficult to interpret.

The effects, if any, of telemedicine on communicative behaviours and the interaction between provider and patient during the consultation remained virtually unexplored. There was a lack of data examining patients' perceptions.

Generalisability of results

The generalisability of much of the published research is limited because of effect modifiers such as study setting. One of the largest studies examined teleconsultation in a prison in the United States.⁵ Clearly, there are several reasons why satisfaction in prisoners may be different from that in the general population. Thus, the peculiarities of the setting mean that this study's results cannot be applied reliably to the general population of that country or more widely.

Furthermore, the delivery of health care was somewhat artificial in many studies. Participants often received a teleconsultation in addition to a routine consultation, and so were really being asked to make a hypothetical judgment as to its value. In many studies participants also received “special” treatment, with every effort being made to minimise inconvenience. Satisfaction in these somewhat artificial contexts may not be readily translatable to satisfaction with telemedicine when it is being used in routine practice.

Discussion

The published research suggests that healthcare delivery via telemedicine is acceptable to patients in a variety of circumstances, but, by addressing this issue in a rather superficial manner, most studies have produced more questions than answers. Thus far, most telemedicine research has had a technological focus. We know a great deal about bandwidths and resolution, but little about the human dimensions that make the practice possible. Pragmatic information that can benefit future delivery of health care via telemedicine is needed.

The following issues need to be addressed:

• What types of consultation are suitable for teleconsulting? Is it suitable for initial consultations, or do patients find it more acceptable to use telemedicine technology just for follow up appointments?

• What are the effects of this mode of healthcare delivery on the doctor-patient relationship? Examining patient perceptions would help to address the reasons why patients liked or disliked a service and help healthcare providers to better understand patients' subjective definitions of acceptability and utility.

• How do communicative issues affect the delivery of health care via telemedicine? We need to better understand the effects of telemedicine on consultations in order to improve the services we provide through this medium.

• What are the possible limitations of telemedicine in clinical practice?

In addition, we need to use research tools that have been shown to be reliable and valid. Questionnaires have advantages and disadvantages, but if they are to be used in future research we need to use instruments that have undergone rigorous testing and have been shown to produce repeatable results and to measure what they are intended to measure. Future evaluations need to start with a set of clear hypotheses and objectives and to use clearly defined methodologies that will increase the likelihood of meeting the initial aims. Although randomised controlled trials may not always be practical, representative patient samples are necessary in order to improve the usefulness of results obtained.

This review serves to highlight methodological deficiencies in the published research. Although there are practical obstacles to evaluating telemedicine,³⁷ there remains a need for further exploration of this field in order to facilitate an evidence based approach to the wider introduction of this new technology. It is an oversimplification to suggest that this aspect of telemedicine has undergone sufficient scrutiny.

What is already known on this topic

Telemedicine is currently advocated as a mode of healthcare delivery because of its potential to diminish inequalities in service provision and to improve access to care

Studies of interactive teleconsultations have been performed in a diversity of settings throughout the world, and most suggest that patients are satisfied with this mode of healthcare delivery

However, preliminary review of this literature indicates there are still many gaps in knowledge in relation to patient satisfaction with telemedicine

What this study adds

This systematic review of the telemedicine literature demonstrates that methodological deficiencies in the published research affect the validity and generalisability of the results and that communication issues, the quality of interpersonal relationships with this medium, and subsequent effects, if any, on the outcome of consultations have yet to be fully explored

Future research in this subject needs to be more scientifically robust in order to assist policymakers in reaching informed decisions about the appropriate use of this technology

Table.

Studies of patient satisfaction with teleconsultations

Study	Type of teleconsultation	No of participants	Location
Callahan et al13	Psychiatry	93	USA
Blackmon et al16	Psychiatry (child)	43	USA
Baigent et al19	Psychiatry	63	Australia
Dongier et al20	Psychiatry (adult and child)	50	Canada
Clarke22	Psychiatry	32	Australia
Graham25	Psychiatry	39	USA
Baer et al30	Psychiatry (obsessive compulsive disorder)	10	USA
Ball et al33	Psychiatry	6	UK
McLaren et al35	Psychiatry	3	UK
Trott36	Psychiatry	Not specified	Australia
Loane et al6*	Dermatology	334	UK
Lowitt et al7	Dermatology	139	USA
Gilmour et al8*	Dermatology	126	UK
Oakley et al9	Dermatology	104	Australia
Jones et al18	Dermatology	51	UK
Brecht et al5	Multispecialty consultations	585	USA
Huston et al12	Multispecialty consultation	96	USA
Harrison et al17	Multispecialty consultation	54	UK
Brennan et al10	Emergency medicine	104	USA
Allen et al21	Oncology	39	USA
Kunkler et al32	Oncology	6	UK
Doolittle et al31	Hospice	6	USA
Conrath et al14	Family practice consultations	32	Canada
Itzak et al29	Primary care consultations	11	Israel
Pedersen et al23	Otolaryngology	26	Norway
Blakeslee et al15	Otolaryngology	36	USA
Duffy et al11	Diagnosis of speech and language disorders	150 in group A, 8 in group B	USA
Takano et al27	Home health care (including medical consultations, physiotherapy, health and welfare services)	20	Japan
Whitten et al24	Home nursing	22	USA
Allen et al34	Home nursing	3	USA
Couturier et al28	Orthopaedic consultation	15	France
Hubble et al26	Patients with Parkinson's disease	9	USA

^*Study by Gilmour et al⁸ used some of same subjects as study by Loane et al.⁶