Application of the Quadruple Aim to evaluate the operational impact of a telemedicine program

Abstract

Background

In response to the COVID-19 pandemic, telemedicine utilization has increased dramatically, yet most institutions lack a standardized approach to determine how much to invest in these programs.

Methods

We used the Quadruple Aim to evaluate the operational impact of CardioClick, a program replacing in-person follow-up visits with video visits in a preventive cardiology clinic. We examined data for 134 patients enrolled in CardioClick with 181 video follow-up visits and 276 patients enrolled in the clinic's traditional prevention program with 694 in-person follow-up visits.

Results

Patients in CardioClick and the cohort receiving in-person care were similar in terms of age (43 vs 45 years), gender balance (74% vs 79% male), and baseline clinical characteristics. Video follow-up visits were shorter than in-person visits in terms of clinician time (median 22 vs 30 min) and total clinic time (median 22 vs 68 min). Video visits were more likely to end on time than in-person visits (71 vs 11%, p < .001). Physicians more often completed video visit documentation on the day of the visit (56 vs 42%, p = .002).

Conclusions

Implementation of video follow-up visits in a preventive cardiology clinic was associated with operational improvements in the areas of efficiency, patient experience, and clinician experience. These benefits in three domains of the Quadruple Aim justify expanded use of telemedicine at our institution.

Implications

The Quadruple Aim provides a framework to evaluate telemedicine programs recently implemented in many health systems.

Level of evidence

Level III (retrospective comparative study).

1. Introduction

The COVID-19 pandemic has resulted in the rapid adoption of telemedicine services in outpatient clinics.^{1 , 2} Telemedicine has allowed primary care and specialty clinics to continue delivering services while minimizing risk to patients and clinicians.2, 3, 4, 5 It is predicted that this shift to telemedicine during the COVID-19 pandemic will lead to the permanent inclusion of virtual care delivery models in health systems.^{6 , 7} As such, it is critical for institutions to assess the impact of telemedicine programs on care delivery to determine the optimal level of investment in these services.6, 7, 8

Previous studies have demonstrated that telemedicine can be used to effectively deliver preventive care and manage chronic diseases, such as diabetes and cardiovascular disease.9, 10, 11, 12, 13, 14 Telemedicine implementation has also been associated with equal or higher patient and clinician satisfaction when compared to traditional in-person care in diverse practice settings.^{5 ,} 15, 16, 17 Importantly, there is limited evidence on the impact of video visits and other virtual encounters on clinic operations.^{10 , 15 , 18} Replacing in-person visits with video visits may have significant effects on clinic efficiency, patient flow, and clinician workload. Institutions must systematically assess these operational impacts to fully evaluate telemedicine programs and guide future investments.

Most institutions lack a standardized approach to evaluate telemedicine programs. The Triple Aim is a widely accepted, standardized framework of three goals that clinical programs should strive to achieve: improving population health, reducing costs, and enhancing the patient experience.¹⁹ The Quadruple Aim builds on the Triple Aim by adding the goal of enhancing the clinician experience in order to address the challenge of clinician burnout and the central role of clinicians in the successful adoption of care delivery interventions.^{19 , 20} In this study, we use the Quadruple Aim as a standardized approach to evaluate the operational impact of CardioClick, a telemedicine program introduced in a preventive cardiology clinic at an academic medical center.

2. Methods

2.1. Intervention design

This study was conducted in a preventive cardiology clinic focused on primary cardiovascular disease prevention in high-risk South Asian adults. The clinic care team includes cardiologists, registered dieticians, and an insulin resistance specialist. Patients enrolled in the prevention clinic complete an initial visit and at least two follow-up visits with a physician and a registered dietician over a six-month period. Patients undergo a comprehensive risk assessment, including an advanced cardiometabolic laboratory panel. Patients then receive personalized treatment focused on intensive risk reduction through lifestyle interventions and pharmacotherapy as indicated.

CardioClick, a telemedicine program replacing in-person follow-up visits with video visits, was fully implemented in this prevention clinic in 2018. Eligibility for CardioClick was limited to patients aged 18 to 63 to restrict the program to those with private insurance as Medicare did not reimburse for video visits at the time. New patients were enrolled in CardioClick by default. Those who did not meet age criteria or did not wish to enroll continued usual care in the traditional in-person prevention program. Patients were consented for CardioClick during their first in-person visit or were virtually e-consented.

Clinicians were trained to use the video visit platform, which was integrated with the electronic health record (EHR). Video visits were conducted from a dedicated clinic or conference room. Patient access to video visits was enabled through the health system's EHR patient portal application. Patients were provided with instructions and assistance in installation of this application at the time of consent. Patients could complete video visits from a smartphone, tablet, or computer workstation. An automated reminder was sent to patients 24 h prior to the video visit instructing them to test their device in preparation. Technological problems encountered by clinicians and patients were handled by a dedicated technical support team.

This study was deemed exempt from review as a quality improvement initiative by our university's institutional review board.

2.2. Data

The video visit cohort consisted of 134 CardioClick patients with video follow-up visits with physicians from June 14, 2018 to April 21, 2020. The in-person cohort consisted of 276 patients enrolled in the traditional prevention program with in-person follow-up visits with physicians from September 11, 2014 to March 6, 2020. March 6, 2020 was selected as the final date for in-person visits as it was one week before a local COVID-19 shelter-in-place order was enacted.

Baseline demographic and clinical characteristics were extracted from the EHR for all CardioClick patients and 140 patients in the in-person cohort. Visit-level data, including appointment date, appointment status, timestamps, and timing of documentation completion, were provided by our institution's Clinical & Business Analytics team for all patients in both cohorts. Timestamps for in-person visits included check-in and check-out times, which were logged by clinic front desk staff, and the times when patients and physicians entered and exited the examination room, which were recorded by clinic medical assistants. For video visits, these times were based on when patients and physicians logged onto and off of the video visit platform. Financial data, including total reimbursement, were provided for these visits by our institution's Financial Planning and Decision Support Services team.

2.3. Analysis

Differences in baseline continuous variables between groups were assessed for statistical significance using unpaired two-sided t-tests. Differences in categorical variables were assessed using Fisher's exact test. P-values less than .05 were deemed statistically significant. To minimize the effect of outliers, visit time parameters, such as appointment duration, are reported as medians with interquartile ranges (IQR).

3. Results

3.1. Patient populations

Patients in the video visit cohort had lower baseline systolic and diastolic blood pressures and higher low-density lipoprotein cholesterol (LDL-C) as compared to patients in the in-person cohort. Patients in the video visit cohort were less likely to be on statin therapy at the time of enrollment. Patients in the two cohorts were otherwise similar in terms of average age (43 vs 45 years), gender balance (74% vs 79% male), and baseline clinical characteristics, including lipid profiles, hemoglobin A1c, body mass index, and smoking status (Table 1 ).

Table 1.

Baseline patient demographic and clinical characteristics.

Characteristic	Video Visit Cohort (n=134)	In-Person Cohort (n=140)	P-Value
Age, years	42.8 ± 8.9	45.0 ± 12.1	.10
Male (%)	97 (74)	111 (79)	.26
Systolic BP, mmHg	123 ± 13	128 ± 21	.014
Diastolic BP, mmHg	78 ± 11	81 ± 9	.016
Lipid panel
Total cholesterol, mg/dL	196 ± 43	184 ± 49	.053
LDL-C, mg/dL	130 ± 39	114 ± 43	.003
HDL-C, mg/dL	50 ± 14	50 ± 19	.88
Triglycerides, mg/dL	125 ± 68	129 ± 80	.65
Statin use (%)	30 (22)	54 (39)	.004
HbA1c, %	5.9 ± 1.1	5.7 ± 0.7	.27
Body mass index, kg/m2	26.3 ± 4.6	26.8 ± 3.8	.36
Smoking status (%)
Non-smoker	101 (75)	114 (81)	.99
Former smoker	19 (14)	20 (14)	.86
Current smoker	3 (2.2)	6 (4.3)	.50

Mean ± standard deviation is reported for continuous variables. Data were available for >90% of patients except for lipid panel and HbA1c. BP denotes blood pressure, LDL-C low-density lipoprotein cholesterol, HDL-C high-density lipoprotein cholesterol, HbA1c hemoglobin A1c.

3.2. Efficiency

Physician video and in-person follow-up visits were equally likely to be completed (49 vs 48%, p = .82), and the two cohorts had similar rates of visit cancellation/rescheduling (45 vs 49%, p = .18) and no-shows (5 vs 3%, p = .081) (Fig. 1 ). There were 181 completed video follow-up visits and 694 completed in-person follow-up visits.

Fig. 1.

Cancellation/rescheduling and no-show rates for video and in-person follow-up visits. (color online only).

The median video follow-up visit duration was 22 min (IQR 16, 29), while the median time spent by physicians with patients for in-person visits was 30 min (IQR 12, 59 min). The distribution of video visit durations was narrower than for in-person visits (Fig. 2 ).

Fig. 2.

Box-and-whisker plots of visit duration and the time by which visits ended late for video and in-person follow-up visits. (color online only)

Values 1.5 times below or above the interquartile range were excluded as outliers.

In 2019, the average reimbursement for physician video follow-up visits in this clinic was 29% lower than the average reimbursement for in-person visits. In 2020, however, this gap in reimbursement was no longer seen with rates at parity for both types of visits.

3.3. Patient time

Patients spent a median of 68 min (IQR 49, 102) in clinic from check-in to check-out for in-person follow-up visits, significantly more time than the median 22 min required for video visits (Fig. 2). The time spent by patients in clinic for in-person visits included a median of 8 min (IQR 5, 13) in the waiting room, 22 min (IQR 10, 35) for intake and awaiting the physician in the examination room, and 30 min with the physician.

Video follow-up visits were much more likely to end on time as compared to in-person visits (71% vs 11%, p < .001). The median video visit ended on time while the median in-person visit ended 32 min (IQR 13, 70) late (Fig. 2).

3.4. Clinician time

Physicians were more likely to complete EHR documentation for video visits on the day of the visit as compared to EHR documentation for in-person visits (56 vs 42%, p = .0016). The proportion of documentation completed at 30 days was similar for both types of visits (83%, p = .91) (Fig. 3 ).

Fig. 3.

Timing of clinician documentation completion for video and in-person follow-up visits. (color online only)

* indicates statistical significance.

4. Discussion

In this study, we used the Quadruple Aim as a standardized approach to examine the operational impact of CardioClick, a program replacing in-person follow-up visits with video visits in a preventive medicine clinic. It has been demonstrated previously that patient health outcomes from CardioClick matched or exceeded those achieved with in-person care.²¹ In this analysis, we found that the Quadruple Aim provides a framework to quantify the operational benefits of CardioClick in comparison to traditional in-person care in the domains of efficiency (cost), patient experience, and clinician experience.

We found that CardioClick increased the efficiency of care. Video follow-up visits were shorter in duration than in-person visits, both in terms of the total clinic and the physician time required. Importantly, video and in-person visits included in this study were conducted with the same group of physicians for the same clinical indications with similar groups of patients. It is possible that when patients invest less time in traveling to and waiting for a visit, patients and clinicians are more willing to conduct succinct visits. Clinicians may also be able to complete EHR documentation more seamlessly during a video visit as compared to an in-person visit. These results suggest that clinicians using video visits could see more patients in less time while maintaining a similar degree of visit complexity.

A theoretical inefficiency of telemedicine is that it may result in increased visit cancellations and no-shows due to technological barriers or increased patient willingness to cancel or skip virtual appointments. In this clinic, we found that video visits were equally likely to be completed as in-person visits with similar no-show and cancellation/rescheduling rates.

We also found that CardioClick was associated with operational improvements, which enhance the patient experience. CardioClick patients spent significantly less time completing follow-up visits with physicians. Video visits eliminate time spent by patients in the waiting room and in the examination room, awaiting the physician, as well as travel time. Video visits were also much more likely to end on time as compared to in-person visits.

Finally, we found that CardioClick was associated with changes in clinic workflow, which may improve the physician experience. Physicians were more likely to complete video visit EHR documentation on the day of the visit as compared to EHR documentation for in-person visits. Previous studies have suggested that the transition from paper records to EHR systems has exacerbated clinician burnout.²² Video visits may therefore reduce burnout by decreasing the post-clinic clerical burden for clinicians.

In the studied clinic, reimbursement rates for physician follow-up visits were at parity for video and in-person visits in 2020, likely due in part to changes in telemedicine payment policy enacted during the COVID-19 pandemic. The observed operational improvements in the domains of efficiency, patient experience, and clinician experience enhance the business case for telemedicine adoption in other clinics at our institution and motivate its continued use after the COVID-19 pandemic. Even if reimbursement rates do not remain at parity, these operational benefits may justify further investments in telemedicine.

This study should be interpreted in the context of several limitations. All visits analyzed occurred within a single clinic, limiting generalizability to other practice settings and health systems. A small number of video visits analyzed occurred during the COVID-19 pandemic and may have had different characteristics as compared to pre-pandemic video visits. Patients included were majority male, middle-aged, and South Asian. Further research is needed to determine if similar benefits are observed in clinics serving diverse patient populations. Though patients were enrolled in CardioClick by default, there may be differences between the patient populations that received video and in-person visits, which accounted for the observed differences in outcomes. Finally, we did not have data on the content of visits, so we could not determine whether video visits were shorter than in-person visits because the same clinical content was covered in less time or because fewer topics were addressed.

Future research should focus on identifying the characteristics of outpatient clinics and patient populations that derive the greatest benefit from telemedicine interventions within the framework of the Quadruple Aim. Health systems should also determine the impact of telemedicine implementation on access to care for diverse patient populations. Virtual care interventions should be designed explicitly to reduce disparities and enhance equity in care delivery.

5. Conclusion

In this study, we used the Quadruple Aim as a standardized approach to assess the operational impact of a telemedicine program in a preventive cardiology clinic. We found that implementation of video follow-up visits as the default care pathway was associated with shorter visit durations, increased on-time visits, and decreased post-clinic clerical burden for clinicians. These improvements in three domains of the Quadruple Aim, efficiency, patient experience, and clinician experience, justify expanded use of telemedicine at our institution. Further research is needed to identify clinics and patient populations most likely to benefit from telemedicine within the framework of the Quadruple Aim and to ensure that access to virtual care is equitable.

Funding

This work was funded by the United States Agency for Healthcare Research and Quality [T32 HS026128], the United States National Heart, Lung, and Blood Institute [K01 HL144607 01], the American Heart Association/Robert Wood Johnson Foundation, and the Kothari Family Fund. These sponsors had no involvement in the design or execution of this study or the drafting of this manuscript.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: