How Satisfied Are Patients and Surgeons with Telemedicine in Orthopaedic Care During the COVID-19 Pandemic? A Systematic Review and Meta-analysis

Abstract

Background

The coronavirus disease 2019 pandemic has resulted in a rapid pivot toward telemedicine owing to closure of in-person elective clinics and sustained efforts at physical distancing worldwide. Throughout this period, there has been revived enthusiasm for delivering and receiving orthopaedic care remotely. Unfortunately, rapidly published editorials and commentaries during the pandemic have not adequately conveyed findings of published randomized trials on this topic.

Questions/purposes

In this systematic review and meta-analysis of randomized trials, we asked: (1) What are the levels of patient and surgeon satisfaction with the use of telemedicine as a tool for orthopaedic care delivery? (2) Are there differences in patient-reported outcomes between telemedicine visits and in-person visits? (3) What is the difference in time commitment between telemedicine and in-person visits?

Methods

In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic review with the primary objective to determine patient and surgeon satisfaction with telemedicine, and secondary objectives to determine differences in patient-reported outcomes and time commitment. We used combinations of search keywords and medical subject headings around the terms “telemedicine”, “telehealth”, and “virtual care” combined with “orthopaedic”, “orthopaedic surgery” and “randomized.” We searched three medical databases (MEDLINE, Embase, and the Cochrane Library) in duplicate and performed manual searches to identify randomized controlled trials evaluating the outcomes of telemedicine and in-person orthopaedic assessments. Trials that studied an intervention that was considered to be telemedicine (that is, any form of remote or virtual care including, but not limited to, video, telephone, or internet-based care), had a control group that comprised in-person assessments performed by orthopaedic surgeons, and were reports of Level I original evidence were included in this study. Studies evaluating physiotherapy or rehabilitation interventions were excluded. Data was extracted by two reviewers and quantitative and qualitive summaries of results were generated. Methodological quality of included trials was assessed using the Cochrane Risk of Bias tool, which uniformly rated the trials at high risk of bias within the blinding categories (blinding of providers, patients, and outcome assessors). We screened 133 published articles; 12 articles (representing eight randomized controlled trials) met the inclusion criteria. There were 1008 patients randomized (511 to telemedicine groups and 497 to control groups). Subspecialties represented were hip and knee arthroplasty (two trials), upper extremity (two trials), pediatric trauma (one trial), adult trauma (one trial), and general orthopaedics (two trials).

Results

There was no difference in the odds of satisfaction between patients receiving telemedicine care and those receiving in-person care (pooled odds ratio 0.89 [95% CI 0.40 to 1.99]; p = 0.79). There were also no differences in surgeon satisfaction (pooled OR 0.38 [95% CI 0.07 to 2.19]; p = 0.28) or among multiple patient-reported outcome measures that evaluated pain and function. Patients reported time savings, both when travel time was excluded (17 minutes shorter [95% CI 2 to 32]; p = 0.03) and when it was included (180 minutes shorter [95% CI 78 to 281]; p < 0.001).

Conclusion

Evidence from heterogeneous randomized studies demonstrates that the use of telemedicine for orthopaedic assessments does not result in identifiable differences in patient or surgeon satisfaction compared with in-person assessments. Importantly, the source studies in this review did not adequately capture or report safety endpoints, such as complications or missed diagnoses. Future studies must be adequately powered to detect these differences to ensure patient safety is not compromised with the use of telemedicine. Although telemedicine may lead to a similar patient experience, surgeons should maintain a low threshold for follow-up with in-person assessments whenever possible in the absence of further safety data.

Level of Evidence

Level I, therapeutic study.

Introduction

Telemedicine refers to the use of telecommunication technology to assess and treat patients [40]. This technology typically takes the form of video, telephone, or an interactive webpage, among other media. Much of the outcomes data on telemedicine come from other medical fields, such as internal medicine, psychiatry, and preventative health [7]. In orthopaedics, the provision of ancillary services such as physiotherapy and rehabilitation has been studied more extensively than surgeon-provided assessments [27,38]. The coronavirus disease 2019 (COVID-19) pandemic has resulted in the worldwide closure of elective medical and surgical clinics and has been followed by sustained efforts at physical distancing [33,34]. Throughout this period of global practice reconfiguration, there has been revived enthusiasm for telemedicine, leading to practice changes which have the potential to continue long-term, well-beyond the acute stages of the pandemic.

Recently, case reports and expert opinions have pointed to the potential benefits of integrating telemedicine into orthopaedic practice [19, 33]. However, the adoption of telemedicine technology in the long-term must be done with care and continual reflection on the evidence. Past adoption of technologies based on surgeon-perceived superiority or even comparative observational data has been invalidated in the face of high-quality evidence from randomized controlled trials (RCTs) [13, 14, 26, 32]. In the context of telemedicine, this could mean compromising patients’ perceptions of care, poorer overall outcomes, or, at worst, increased complications because of treatment. Unfortunately, rapidly published editorials and commentaries during the pandemic have not adequately conveyed findings of published randomized trials on this topic [19, 28, 31, 33]. One study, which systematically reviewed the evidence on telemedicine to date, did not identify any orthopaedic publications [25]. We therefore sought to better summarize what is known by systematically reviewing and quantitatively pooling randomized trial data.

We therefore performed a systematic review and meta-analysis of randomized trials, in which we asked: (1) What are the levels of patient and surgeon satisfaction with the use of telemedicine as a tool for orthopaedic care delivery? (2) Are there differences in patient-reported outcomes between telemedicine visits and in-person visits? (3) What is the difference in time commitment between telemedicine and in-person visits?

Materials and Methods

We conducted a systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [18]. This review was not pre-registered. We searched three medical databases (MEDLINE, Embase, and the Cochrane library) from inception to April 2020 using a combination of medical subject headings and keywords (see Appendix 1; Supplemental Digital Content 1, http://links.lww.com/CORR/A428). We also manually searched Google Scholar, PubMed, and bibliographies of included studies. We did not include conference proceedings or unpublished abstracts. Pairs of reviewers screened abstracts and titles for potentially relevant studies.

Inclusion and Exclusion Criteria

We included studies published in any language that studied an intervention that was considered to be telemedicine (that is, any form of remote or virtual care including, but not limited to, video, telephone, or internet-based care), had a control (comparator) group that comprised in-person assessments performed by orthopaedic surgeons (regardless of subspecialty), and were reports of Level I original evidence (that is, randomized controlled trials). Studies evaluating physiotherapy or rehabilitation interventions were excluded.

Disagreements about which studies should be included were resolved by reviewing the full text, and further disagreements were resolved by consensus between the senior authors (HC, RM).

Relevant data, including trial details, participant demographics, nature of the intervention and control, patient satisfaction outcomes, appointment length, and disease-specific outcomes were extracted from included trials using standardized data collection forms. All studies were also evaluated for the risk of bias using the Cochrane Risk of Bias assessment tool. This involved an assessment of random sequence generation, allocation concealment, blinding (of participants, personnel, and/or outcome assessors), completeness of outcome data, and selective reporting bias. For trials that resulted in multiple publications, the cumulative reported data from all publications were considered.

Primary and Secondary Study Outcomes

Our primary study outcome was patient satisfaction. We evaluated this by comparing differences in the likelihood of satisfaction between groups, with ordinal or continuous outcome measures converted into binary outcomes using previously described methodology.

Our secondary study outcomes were surgeon satisfaction (differences in the likelihood of satisfaction), patient-reported outcome measures (standardized mean differences), and appointment length (with and without travel time, in minutes).

Search Results

We screened 133 published articles, ultimately including 12 studies representing eight RCTs (that is, eight separate populations) in this review (Fig. 1) [1, 2, 3, 6, 9, 12, 20, 23, 29, 30, 36, 39]. There was almost-perfect agreement between reviewers for inclusion and exclusion of studies (kappa for agreement = 0.94 [95% CI 0.85 to 1.0]). There were 1008 patients randomized (511 to telemedicine groups and 497 to control groups). Subspecialties represented were hip and knee arthroplasty (two trials), upper extremity (two trials), pediatric trauma (one trial), adult trauma (one trial), and general orthopaedics (two trials).

Fig. 1.

This flowchart shows the studies that were included in and excluded from our systematic review.

Among these RCTs, seven were English-language articles and one was published in German. Five were conducted in North America and three were conducted in Europe. Six RCTs evaluated video-based interventions and two evaluated internet-based interventions. Seven studies were conventional head-to-head RCTs, and one was a randomized crossover trial. Sample sizes ranged from 23 to 402 participants. Interventions varied from completely telemedicine-based follow-up to combinations of in-person and telemedicine visits (Table 1).

Table 1.

Details of included trials

Study	Country	Speciality (condition)	Sample size	Intervention description	Control description	Mean patient age (SD or range)	Follow-up period	Selection of reported outcome measures
Buvik et al. [1-3]	Norway	General orthopaedics (multiple orthopaedic conditions)	402	Live video-based first consult or follow-up visit conducted in a local health facility	In-person first consult or follow-up visit at an academic medical center	I: 48.0 (SD 24.0) C: 46.7 (SD 24.9)	1 year	PROM Treatment decision Complications Satisfaction (surgeon/patient) Time Cost
Eberl et al. [6]	Germany	Upper extremity (elbow arthrolysis)	23	Live video-based postoperative follow-up visit conducted from home supplemented with interval photographs	In-person postoperative follow-up visit only	42.6 (17 to 76)	6 months	Outcome rating scale Assessment rating scale Costs
Haukipuro et al. [9, 23, 36]	Finland	General orthopaedics (multiple orthopaedic conditions)	145	Live video-based consultation or follow-up visit conducted in a local health facility	In-person consult or follow-up visit	I: 58.3 (SD 17.7) C: 55.1 (SD 17.3)	1 year	Treatment decision Satisfaction (surgeon/patient) Time Cost
Kane et al. [12]	USA	Upper extremity (arthroscopic rotator cuff repair)	66	Live video-based postoperative follow-up at 2 weeks, 6 weeks, and 12 weeks	In-person postoperative follow-up at 2 weeks, 6 weeks, and 12 weeks	I: 60.6 (39 to 73) C: 59.8 (50 to 70)	12 weeks	PROM Complications Satisfaction (surgeon/patient) Time
Marsh et al. [20]	Canada	Arthroplasty (THA/TKA)	256	Web-based (written) annual (> 12 months) postoperative follow-up	In-person annual postoperative (> 12 months) follow-up	I: 68.8 (SD 10.0) C: 66.4 (SD 11.5)	Mean I: 5.0 (SD 3.4 C: 5.0 (SD 3.2, C)	PROM Complications Satisfaction (patient)
Sathiyakumar et al. [29]	USA	Orthopaedic trauma (upper and lower extremity fractures)	24	Live video-based follow-up visits at 6 weeks and 6 months. In-person visits at 2 weeks and 3 months	All in-person follow-up visits	I: 36.8 (SD 14.7) C: 27.7 (SD 9.4)	6 months	Complications Satisfaction Time
Silva et al. [30]	USA	Pediatrics (supracondylar distal humerus fractures)	52	Live video-based 4-week follow-up visit with self-cast removal. 1-week and final follow-up visits in-person	All in-person follow-up visits	I: 5.0 (2.6 to 9.4) C: 5.0 (1.9 to 10.8)	8 weeks	PROM Satisfaction (patient) ROM Radiographic parameters Time Costs
Wood et al. [39]	Canada	Arthroplasty (THA/TKA)	40	Web-based (written) annual (> 12 months) postoperative follow-up	In-person annual postoperative (> 12 months) follow-up	Age 42.6 (17 to 72)	Mean 38 months	PROM Time Data completeness

I = intervention; C = control; PROM = patient-reported outcome measures.

Six trials reported measures of patient satisfaction. Five reported ordinal or categorical variables and one reported continuous variables (VAS scores). Similarity between scales and conceptual measures warranted quantitative pooling. When multiple domains of satisfaction were reported, we pooled the overall satisfaction score; if this was not reported, we pooled the mean satisfaction score. Odds ratios of patients being satisfied or more than satisfied (that is, very or extremely) were calculated. For continuous measures, the standardized mean difference was calculated and converted to log-odds using previously described statistical methods [10].

Study Methodological Quality

We assessed methodological quality of included studies using the Cochrane Risk of Bias (RoB1) tool. The assessments demonstrated that the inability to blind patients or surgeons and the lack of objective outcomes for blinded outcome assessors presented the greatest risk of bias among trials (Fig. 2). Random sequence generation, allocation concealment, and outcome reporting and follow-up were generally well-reported.

Fig. 2.

This figure summarizes the study quality of the included trials. Among the trials studied, the greatest risk of bias was due to the inability to blind patients and surgeons and the lack of objective outcomes for blinded outcome assessors.

Quantitative and Qualitative Analysis

We assessed outcomes data to determine their appropriateness for quantitative pooling (that is, a meta-analysis) or qualitative summarization. Owing to the clinically heterogeneous nature of the interventions reported in the included studies (phone, video, and internet-based), we decided a priori that we would use a random-effects model for all analyses. We reported statistical heterogeneity with the use of the I² statistic alongside forest plots. Funnel plots were used to screen for publication bias for the main outcome; overall the plots were symmetric except for a single outlier small study [29] suggesting the possibility of small study bias. A post-hoc sensitivity analysis was undertaken to confirm that exclusion of this study did not change the results. For missing variance data, SDs were imputed using well-accepted methods [11]. In studies in which there was varied reporting of binary and continuous outcomes data, standardized mean differences were converted to natural log-odds and subsequently pooled using the generic-inverse variance method. The data analysis was performed using Review Manager Version 5.3 (Copenhagen, Denmark). Pooled means and 95% confidence intervals are reported. A p value of less than 0.05 was considered statistically significant.

Outcomes data that were deemed conceptually heterogeneous by the senior authors (HC, RM) were not pooled; this information was summarized qualitatively.

Results

Patient and Surgeon Satisfaction with Care Provided by Telemedicine

Patient Satisfaction

There was no difference in the odds of patient satisfaction between patients receiving telemedicine care and those receiving in-person care (pooled odds ratio 0.89 [95% CI 0.40 to 1.99]; p = 0.79) (Fig. 3). Overall, 92% (537 of 581) of patients were satisfied or more than satisfied (that is, very or extremely satisfied) after telemedicine visits and 93% (489 of 528) were satisfied after in-person visits. In an evaluation of only responses of “more than satisfied,” we found no between-group differences (pooled OR 0.82 [95% CI 0.41 to 1.61]; p = 0.56); 63% (366 of 581) of participants with telemedicine visits and 65% (342 of 528) of control participants were more than satisfied.

Fig. 3.

This forest plot summarizes the individual and pooled odds of satisfaction between patients receiving telemedicine care and those receiving in-person care.

Surgeon Satisfaction

There was no difference in the pooled odds of surgeon satisfaction between telemedicine and in-person visits (OR 0.38 [95% CI 0.07 to 2.19]; p = 0.28). Overall, 91% (417 of 457) of surgeon responses were rated as good or very good with the use of telemedicine compared with 94% (384 of 408) of responses with the control visits.

Patient-reported Outcomes

There were no reported differences in any measures of generic function (SF-12, EQ-5D), disease-specific function (WOMAC, Morrey Outcome Scale), or pain (VAS, Revised Faces Pain Scale) between the telemedicine and control visit groups (Table 2). The outcome measures used were heterogeneous across studies; therefore, quantitative pooling was not considered appropriate.

Table 2.

Summary of reported outcome measures

Study	Patient-reported treatment measure	Results
Buvik et al. [1-3]	EQ-QoL-VAS EQ-QoL-5D	No differences
Eberl et al. [6]	Morrey Outcome scale	No differences
Haukipuro et al. [9, 23, 36]
Kane et al. [12]	VAS Pain	No differences
Marsh et al. [20]	WOMAC SF-12	No differences
Sathiyakumar et al. [29]
Silva et al. [30]	Pain (Revised Faces Pain Scale)	No differences
Wood et al. [39]	WOMAC SF-12	No differences

QoL = quality of life.

Difference in Time Commitment between Telemedicine and In-person Visits

Time commitment for appointments was shorter for telemedicine visits in comparison to in-person visits. The mean visit length for telemedicine appointments was 19 ± 8) minutes when travel time was excluded and 27 ± 24) minutes when travel time was included. The mean visit length for in-person appointments was 79 ± 85) minutes when travel time was excluded and 313 ± 126) minutes when travel time was included. Further breakdown of allocated time within each appointment was not provided in the studies. Telemedicine visit length was shorter both when travel time was excluded (17 minutes shorter [95% CI 2 to 32]; p = 0.03) and when it was included (180 minutes shorter [95% CI 78 to 281]; p < 0.001).

Discussion

There has been renewed enthusiasm for telemedicine and its possibilities, with the COVID-19 pandemic functioning as a catalyst to accelerate widespread implementation. Although there is great enthusiasm of late for this topic [19, 28, 31, 33], we have seen little summative research on it [25], and there is none in our specialty of which we are aware. In this meta-analysis, we found that there were no differences in patient satisfaction scores, surgeon satisfaction scores, or patient-reported outcome measures between telemedicine and in-person appointments; however, patient time commitment was notably shorter than in-person assessments, both with and without travel time accounted for. Importantly, safety endpoints were not well measured by the source studies, so we could not evaluate them here.

Limitations

Our review has several limitations. The selection of satisfaction as our main outcome (both patient and surgeon) has well-described issues and limitations [4]. First, the instruments used to measure satisfaction were not uniform across studies, and in general they were not validated. Further the use of satisfaction as an outcome is known to demonstrate ceiling effects [22]—whereby at a certain high threshold, the measurement instruments are no longer able to detect differences in scores. Ceiling effects could partially account for the similar scores between telemedicine and in-person assessments in our analysis. In other words, it is possible that while patients and surgeons were highly satisfied with both telemedicine and in-person assessments, there may be differences in satisfaction that were not detectable. Future research is needed to explore the various domains of satisfaction—that is, which specific elements of the telemedicine (and in-person) visit patients and surgeons find satisfactory and unsatisfactory, and how these can be improved. Finally, satisfaction scores are more likely to measure satisfaction with the process of receiving care rather than the efficacy of the care itself [4]. Therefore, these scores are not necessarily reflective of objective treatment outcomes or patient-reported outcome measures.

The second most-important limitation was that safety-related outcomes were not consistently collected or reported across the included studies. Even a few severe complications or missed diagnoses can cause sufficient harm so as to negate the positive benefits of telemedicine (as it is currently practiced). As a nascent technology, it is plausible that many elements of the physical examination cannot be sufficiently replicated even with the integration of video-based technology. This lack of safety data is an issue that patients, surgeons, and policymakers must bear in mind when considering how to successfully integrate virtual medicine into clinical practice in the long term. Determining whether there are differences in safety endpoints and how to address these potential differences must be a priority in future research in telemedicine.

Heterogeneity of interventions, populations, and outcome measurements was also a major limitation in extrapolating our findings. As with any systematic review conducting a pooled analysis, the conclusions of our analysis are ultimately limited by the quality and heterogeneity of the data measured and reported in published trials. Not all trials reported each outcome, and even when the outcomes were reported, not all were similar enough to warrant a pooled analysis. Even when outcomes were similar enough to warrant pooling, there remained residual measurement heterogeneity. With appointment length, for instance, the studies were not specific in exactly what was included in appointment length (for example, x-ray time, or nurse or trainee assessment), and this could have varied between studies.

Many of our studies were also at a high risk of bias in several domains. Blinding was particularly difficult for the intervention being evaluated. We therefore cannot be certain that the outcomes are not unduly influenced by enthusiasm for a novel intervention among providers and patients. Incomplete outcome reporting was also an issue in some of the trials. Sathiyakumar et al. [29] had the highest proportion lost to follow-up, but it was also a small study which did not weigh heavily in the pooled quantitative analysis. A sensitivity analysis from which this study was excluded in the analysis did not alter the results. Each of the remaining studies reported no more than 10% missing data and this missing data was evenly distributed between treatment and control groups. For this reason, we did not feel missing data compromised the results of this meta-analysis.

Finally, not all subspecialties of orthopaedic surgery were represented—indeed, for an exhaustive review such as ours, we had hoped for a larger number of trials to improve the external validity of our findings. Regardless, we feel that this is an important first step in identifying gaps in knowledge and facilitating further trials.

Patient and Surgeon Satisfaction with Care Provided by Telemedicine

We found no differences in satisfaction between patients treated via telemedicine and those treated in person, and no differences in satisfaction among surgeons using the two approaches; in the dataset we reviewed, satisfaction generally was high across the board. This is consistent with a systematic review assessing telemedicine use across all medical studies [25]. Based on a non-comparative qualitative analysis, the authors concluded that satisfaction among patients, as measured in different domains, was rated highly. However, that review did not identify any clinical trials or observational cohort studies in orthopaedics specifically or in any other surgical field.

Unique aspects of the orthopaedic visit differ from other specialties and demands focused study, which is why we undertook this review. Specifically, surgeons need to assess the patient’s symptom status, clinically examine the limb or joint, assess any incisions or lesions, and determine ROM (and its interval progress), among other factors. Patients often have questions for their surgeons about sequelae of the injury or treatment. Further, unspoken elements of an in-person visit may be associated with the patient’s care experience [21]. There is evidence to suggest that similar satisfaction scores between interventions—as demonstrated in our analysis—translates to a similar patient experience with the clinical interaction [8], consolidating support for the use of telemedicine in routine orthopaedic practice. Further research is required to explore the comparative differences in various domains of patient experience, specifically with validated patient-reported experience measures (PREMs), in a well-controlled (ideally) randomized manner.

Patient-reported Outcomes

Although we could not pool the results, we found no obvious differences in patient-reported outcomes between patients treated using telemedicine and those cared for in person. This finding is consistent with other similar studies of orthopaedic conditions (albeit focused on rehabilitation interventions) and other medical specialties, such as those in internal medicine and psychiatry [7]. A review of 15 studies of telerehabilitation (that is, remote technology-based rehabilitation) after orthopaedic surgery and found strong evidence to support its use after THA and TKA and moderate or weak evidence to support its use after upper-limb surgery [27]. A more focused systematic review of telerehabilitation in the setting of THA and TKA only demonstrated that there was moderate- and low-quality evidence in terms of pain and functional improvement, respectively, with telerehabilitation after TKA only, but this was not necessarily clinically important [37]. Focused trials in different orthopaedic subspecialties employing uniform patient-reported outcomes measures would facilitate meaningful meta-analyses in the future to better inform clinical practice and policy.

Difference in Time Commitment between Telemedicine and In-person Visits

Unsurprisingly, telemedicine visits took less time, a difference that was especially prominent when travel time was factored in. Time and cost savings for patients seen remotely is a benefit of telemedicine, even in the absence of requirements for physical distancing. Indeed, one of the original impetuses for telemedicine was to assess patients in remote settings [5, 15, 16, 17, 24, 35]. Our findings corroborate that reduced patient time commitment is a major advantage of telemedicine. Driving time, time off work, and appointment length can be reduced with the use of telemedicine technology. This also results in substantial cost savings for patients because of less time off work, less gas consumption or additional travel costs, and fewer lost opportunity costs [1]. The influence of time commitment for orthopaedic surgeons was not reported; whether telemedicine requires more, less, or the same time commitment from orthopaedic surgeons needs to be assessed.

Conclusion

Evidence from heterogeneous randomized studies demonstrates that the use of telemedicine for orthopaedic assessments does not result in identifiable differences in patient or surgeon satisfaction compared to in-person assessments. However, the possibility of measurement ceiling effects is high, which could have failed to discern real differences among the high satisfaction scores in both groups. Time commitment for patients was lower with the use of telemedicine than in-person appointments. Importantly, the source studies in this review did not adequately capture or report safety endpoints (such as complications or missed diagnoses). It is imperative that future comparative studies are adequately powered to detect these differences in complications to ensure patient safety is not compromised with the use of telemedicine. Although telemedicine may lead to a similar patient experience, orthopaedic surgeons should maintain a low threshold for follow-up with in-person assessments whenever possible in the absence of further safety data. Moreover, identification and study of specific elements of the telemedicine visit is needed to facilitate standardization toward high quality evidence-based implementation of telemedicine protocols.