Abstract
Objectives
Missed appointments can lead to treatment delays and adverse outcomes. Telemedicine may improve appointment completion because it addresses barriers to in-person visits, such as childcare and transportation. This study compared appointment completion for appointments using telemedicine versus in-person care in a large cohort of patients at an urban academic health sciences center.
Materials and Methods
We conducted a retrospective cohort study of electronic health record data to determine whether telemedicine appointments have higher odds of completion compared to in-person care appointments, January 1, 2021, and April 30, 2023. The data were obtained from the University of South Florida (USF), a large academic health sciences center serving Tampa, FL, and surrounding communities. We implemented 1:1 propensity score matching based on age, gender, race, visit type, and Charlson Comorbidity Index (CCI).
Results
The matched cohort included 87 376 appointments, with diverse patient demographics. The percentage of completed telemedicine appointments exceeded that of completed in-person care appointments by 9.2 points (73.4% vs 64.2%, P < .001). The adjusted odds ratio for telemedicine versus in-person care in relation to appointment completion was 1.64 (95% CI, 1.59-1.69, P < .001), indicating that telemedicine appointments are associated with 64% higher odds of completion than in-person care appointments when controlling for other factors.
Discussion
This cohort study indicated that telemedicine appointments are more likely to be completed than in-person care appointments, regardless of demographics, comorbidity, payment type, or distance.
Conclusion
Telemedicine appointments are more likely to be completed than in-person healthcare appointments.
Keywords: telemedicine, missed appointments, appointments and schedules, digital health, data science, informatics
Introduction
Missed appointments are a longstanding and common problem in healthcare delivery. When appointments are scheduled but not attended, quality of care is adversely affected. Missed appointments can lead to downstream events and outcomes, including treatment delays, costly emergency department visits, poorer outcomes, and a higher risk of death.1 In England, the rate of missed mental health appointments has been estimated at 19.1%, nearly 1 out of every 5 appointments, and is associated with subsequent treatment drop-out and other adverse outcomes that include relapse, re-admission, and completed suicide.2 Additionally, the wasted space, equipment, and personnel resources adversely affect the healthcare system.3–5 Researchers have estimated the cost of missed appointments in diverse settings, with annual clinic-level costs of millions of U.S. dollars.3–5
Patients miss appointments for a variety of reasons. Missed appointments have been attributed to socioeconomic factors, emotional factors such as fear and avoidance, demographics, family stability, a lack of awareness about the reason for an appointment, and incompatibility of appointment time.6,7 Among urban, low-income patients, patients may need more affordable and reliable transportation and miss appointments due to family and work commitments or poor health.8 In effect, economically disadvantaged patients may feel unwell to travel to an appointment, unable to pay for the appointment, constrained by childcare responsibilities, unable to adjust their work schedule, or fear the appointment. They may also dislike the provider or not feel like going. Healthcare systems and providers also miss appointments due to scheduling mistakes, emergencies, staffing, or other reasons.9 The reasons for missed appointments are complex and multi-factorial. Reminder systems such as phone calls and short message service (SMS) messages that address appointment awareness, some augmented with artificial intelligence, show some marginal benefit in mitigating missed appointments.10,11 However, missed appointments largely persist. This suggests that factors other than appointment awareness more strongly influence appointment completion.
One potentially important strategy for improving appointment completion is telemedicine, which appears to address common barriers to obtaining healthcare, including lack of transportation or childcare and conflicts with work or personal commitments.12–14 Telemedicine also addresses barriers posed by a lack of access to local healthcare providers, disasters, and environmental hazards such as extreme heat or poor air quality that are linked to climate change.15–19 However, we need to better understand the effect of telemedicine on appointment completion to optimize its use within care delivery models and to inform policy governing its use and reimbursement.20,21
Objective
This study aimed to compare overall rates and characteristics of appointment completion between appointments conducted using telemedicine versus in-person care in a large, cohort of patients at an urban academic health sciences center.
Methods
In this retrospective cohort study of electronic health record (EHR) data, we examined whether telemedicine appointments had a higher completion rate than in-person care appointments between January 1, 2021, and April 30, 2023. Furthermore, we examined the crude and adjusted odds of appointment completion using multivariate Generalized Linear Models (GLM). The University of South Florida (USF) Institutional Review Board reviewed and approved the study (USF IRB STUDY005197).
Setting, data, and sample
Data were obtained from EHRs of USF patients who received medical care between January 1, 2021, and April 30, 2023. USF is a large academic health sciences center serving Tampa, FL, and surrounding communities. Tampa, FL, has a racially and ethnically diverse population.22 USF offers inpatient and outpatient services across numerous medical specialties, serving approximately 1.2 million patients.
Through preliminary analyses, we identified specialties and visit types offered using both in-person care and telemedicine. We used the visit type field in the EHR to identify telemedicine visits. In the study setting, visits with “telemedicine” or “telehealth” in the visit type description are telemedicine visits. We limited encounters to USF visit types with equivalent labels for both in-person care and telemedicine. Furthermore, to mitigate potential bias related to the multiplicate presence of individual patients, we limited the sample to one appointment per patient, including the first occurrence associated with a unique patient during the study dates. Then, we categorized visit types as new patient visits (NPV) or established patient visit types (EPV). A summary of inclusion and exclusion criteria for appointments follows.
The inclusion criteria for appointments were as follows: (1) appointment scheduled between January 1, 2021, and April 30, 2023; (2) appointment with visit type actively used for both telemedicine and standard (in-person) care; and (3) appointment is the first appointment associated with a unique patient during the study time period. The exclusion criteria for the study were as follows: (1) appointment visit type not actively used for both telemedicine or standard (in-person) care, and (2) telephone-only appointments. We focused the analysis on each patient’s first visit during the study window to reduce imbalances related to the heterogeneity in the number of visits per patient in both cohorts and to keep the cohorts as similar as possible.
We defined missed appointments as appointments that were canceled, no show, or left without being seen. “Cancellation” indicated an appointment was canceled in advance of the appointment date/time, without re-scheduling. “No show” indicated appointments that were not canceled in advance, and not attended by the patient. “Left without being seen” appointments indicated that the patient checked in, connected to the call, or arrived at a previously scheduled appointment, and exited before being seen by a provider. We calculated the Charlson Comorbidity Index (CCI) for each patient as a general indicator of patient health, which could potentially influence appointment completion.23 Travel distance was the distance in miles between a patient’s ZIP code and the ZIP code of the clinic for which the appointment was made.
Analysis
All statistical analyses were performed using R version 4.3.0. Differences between the telemedicine and in-person care groups were assessed using independent sample t-tests, for continuous variables, and Chi-square tests, for categorical variables. Because the in-person care group was significantly larger than the telemedicine group, we matched the 2 groups based on 1:1 propensity score matching using the K-nearest neighbor method, implemented using the MatchIt package.24 We matched on age, gender, race, visit type, and CCI. Travel distance was calculated using the zipcodeR package.25 To calculate the adjusted odds ratios, we developed a GLM model26 of visit completion with the following variables as inputs: appointment modality (ie, telemedicine vs in-person care), age, gender, race, CCI, and the variables that remained unbalanced after matching. Visit completion was constructed as a binary variable where 1 indicated a visit with a “Complete” status and 0 indicated a visit with a “Canceled,” “Left without seen,” or “No show” status. We used maximum likelihood as the estimation method and assumed a binomial probability distribution.
Results
A total of 236 502 appointments corresponded to visit types available as both telemedicine and in-person care at the USF between January 1, 2021, and April 30, 2023. The characteristics of the baseline cohort are described in Tables 1 and 2. Overall, 62.6% (148 054) appointments were completed and 37.4% (86 439) were missed appointments (cancelled, no-show, or left without being seen). The specialties with the highest number and percentage of telemedicine visits included Internal Medicine (18 881, 8%), Neurology (18 656, 7.9%), Neurosurgery (19 180, 8.1%), and Obstetrics & Gynecology (20 574, 8.7%). After applying inclusion and exclusion criteria, then implementing propensity score matching, 87 376 appointments remained eligible for the study (see matched cohort, Table 1). After matching, statistically significant differences were identified in patient ethnicity, insurance type, and travel distance between the telemedicine and in-person care groups.
Table 1.
Patient and appointment characteristics, baseline, and matched cohorts.
| Variable | Baseline cohort |
Matched cohort |
||||||
|---|---|---|---|---|---|---|---|---|
| In-person care (N = 182 665) | Telemedicine (N = 53 837) | All (N = 236 502) | P | In-person care (N = 43 688) | Telemedicine (N = 43 688) | All (N = 87 376) | P | |
| Age | ||||||||
| Mean (SD) | 52.0 (19.3) | 44.4 (20.1) | 50.3 (19.7) | <.001 | 45.9 (19.9) | 45.9 (19.9) | 45.9 (19.9) | .60 |
| Median [min, max] | 54.0 [18.0, 122] | 42.0 [18.0, 121] | 52.0 [18.0, 122] | 44.0 [18.0, 101] | 44.0 [18.0, 104] | 44.0 [18.0, 104] | ||
| Gender | ||||||||
| Female | 107 308 (58.8%) | 33 650 (62.5%) | 140 958 (59.6%) | <.001 | 27 515 (63.0%) | 27 652 (63.3%) | 55 167 (63.1%) | .33 |
| Male | 75 326 (41.2%) | 20 172 (37.5%) | 95 498 (40.4%) | 16 173 (37.0%) | 16 036 (36.7%) | 32 209 (36.9%) | ||
| Missing | 31 (0.0%) | 15 (0.0%) | 46 (0.0%) | – | – | |||
| Race | ||||||||
| Black or African American | 22 374 (12.5%) | 5886 (11.0%) | 28 260 (12.2%) | <.001 | 4828 (11.1%) | 4881 (11.2%) | 9709 (11.1%) | .85 |
| Other/Unknown | 37 248 (20.8%) | 10 646 (20.0%) | 47 894 (20.6%) | 6828 (15.6%) | 6827 (15.6%) | 13 655 (15.6%) | ||
| White or Caucasian | 119 151 (66.6%) | 36 814 (69.0%) | 155 965 (67.2%) | 32 032 (73.3%) | 31 980 (73.2%) | 64 012 (73.3%) | ||
| Missing | 3892 (2.1%) | 491 (0.9%) | 4383 (1.9%) | – | – | |||
| Ethnicity | ||||||||
| Hispanic or Latino | 30 954 (18.4%) | 8547 (17.0%) | 39 501 (18.1%) | <.001 | 7900 (18.1%) | 7157 (16.4%) | 15 057 (17.2%) | <.001 |
| Not Hispanic or Latino | 136 957 (81.6%) | 41 670 (83.0%) | 178 627 (81.9%) | 35 788 (81.9%) | 36 531 (83.6%) | 72 319 (82.8%) | ||
| Missing | 14 754 (8.1%) | 3620 (6.7%) | 18 374 (7.8%) | – | – | |||
| Visit type | ||||||||
| Established patient | 66 850 (36.6%) | 30 931 (57.5%) | 97 781 (41.3%) | <.001 | 26 571 (60.8%) | 26 621 (60.9%) | 53 192 (60.9%) | .72 |
| New patient | 115 815 (63.4%) | 22 906 (42.5%) | 138 721 (58.7%) | 17 117 (39.2%) | 17 067 (39.1%) | 34 184 (39.1%) | ||
| Insurance type | ||||||||
| Medicaid | 14 869 (9.0%) | 5046 (10.7%) | 19 915 (9.4%) | <.001 | 4560 (10.4%) | 4618 (10.6%) | 9178 (10.5%) | <.001 |
| Medicare | 54 417 (33.0%) | 12 382 (26.2%) | 66 799 (31.5%) | 10 588 (24.2%) | 11 466 (26.2%) | 22 054 (25.2%) | ||
| Other | 4377 (2.7%) | 1550 (3.3%) | 5927 (2.8%) | 1377 (3.2%) | 1450 (3.3%) | 2827 (3.2%) | ||
| Private | 91 284 (55.3%) | 28 255 (59.8%) | 119 539 (56.3%) | 27 163 (62.2%) | 26 154 (59.9%) | 53 317 (61.0%) | ||
| Missing | 17 718 (9.7%) | 6604 (12.3%) | 24 322 (10.3%) | – | – | |||
| Travel distance | ||||||||
| Mean (SD) | 36.1 (132) | 54.4 (176) | 40.2 (143) | <.001 | 39.3 (145) | 49.3 (163) | 44.3 (154) | <.001 |
| Median [min, max] | 14.9 [0, 4780] | 16.6 [0, 5240] | 15.1 [0, 5240] | 14.3 [0, 4780] | 15.7 [0, 5240] | 15.1 [0, 5240] | ||
| Missing | 2671 (1.5%) | 736 (1.4%) | 3407 (1.4%) | – | – | |||
| Cancellation types | ||||||||
| Canceled | 59 402 (32.5%) | 12 252 (22.8%) | 71 654 (30.3%) | <.001 | 12 777 (29.2%) | 9318 (21.3%) | 22 095 (25.3%) | <.001 |
| Left without being seen | 22 (0.0%) | 42 (0.1%) | 64 (0.0%) | 2 (0.0%) | 34 (0.1%) | 36 (0.0%) | ||
| No show | 13 610 (7.5%) | 3111 (5.8%) | 16 721 (7.1%) | 2874 (6.6%) | 2262 (5.2%) | 5136 (5.9%) | ||
Table 2.
Appointment specialty, baseline and matched cohorts.
| Specialty | In-person care (N = 182 665) | Telemedicine (N = 53 837) | All (N = 236 502) | P | In-person care (N = 43 688) | Telemedicine (N = 43 688) | All (N = 87 376) | P |
|---|---|---|---|---|---|---|---|---|
| Adolescent Medicine | 688 (0.4%) | 86 (0.2%) | 774 (0.3%) | <.001 | 115 (0.3%) | 77 (0.2%) | 192 (0.2%) | <.001 |
| Allergy | 770 (0.4%) | 95 (0.2%) | 865 (0.4%) | 230 (0.5%) | 89 (0.2%) | 319 (0.4%) | ||
| Audiology | 1273 (0.7%) | 28 (0.1%) | 1301 (0.6%) | 208 (0.5%) | 3 (0.0%) | 211 (0.2%) | ||
| Cardiology | 17 492 (9.6%) | 4587 (8.5%) | 22 079 (9.3%) | 3537 (8.1%) | 4098 (9.4%) | 7635 (8.7%) | ||
| Cardiothoracic Surgery | 405 (0.2%) | 54 (0.1%) | 459 (0.2%) | 119 (0.3%) | 47 (0.1%) | 166 (0.2%) | ||
| Colon & Rectal Surgery | 3338 (1.8%) | 209 (0.4%) | 3547 (1.5%) | 621 (1.4%) | 189 (0.4%) | 810 (0.9%) | ||
| Dermatology | 12 452 (6.8%) | 99 (0.2%) | 12 551 (5.3%) | 3728 (8.5%) | 82 (0.2%) | 3810 (4.4%) | ||
| Diabetes | 3262 (1.8%) | 1897 (3.5%) | 5159 (2.2%) | 1112 (2.5%) | 1677 (3.8%) | 2789 (3.2%) | ||
| Endocrinology, Diabetes & Metabolism | 976 (0.5%) | 155 (0.3%) | 1131 (0.5%) | 263 (0.6%) | 144 (0.3%) | 407 (0.5%) | ||
| Family Medicine | 6402 (3.5%) | 1813 (3.4%) | 8215 (3.5%) | 2232 (5.1%) | 1638 (3.7%) | 3870 (4.4%) | ||
| Gastroenterology | 7627 (4.2%) | 1613 (3.0%) | 9240 (3.9%) | 1545 (3.5%) | 1513 (3.5%) | 3058 (3.5%) | ||
| General Surgery | 4995 (2.7%) | 381 (0.7%) | 5376 (2.3%) | 1132 (2.6%) | 343 (0.8%) | 1475 (1.7%) | ||
| Gynecology | 1934 (1.1%) | 1076 (2.0%) | 3010 (1.3%) | 665 (1.5%) | 714 (1.6%) | 1379 (1.6%) | ||
| Hematology | 149 (0.1%) | 767 (1.4%) | 916 (0.4%) | 36 (0.1%) | 652 (1.5%) | 688 (0.8%) | ||
| Internal Medicine | 10 332 (5.7%) | 8549 (15.9%) | 18 881 (8.0%) | 3234 (7.4%) | 5508 (12.6%) | 8742 (10.0%) | ||
| Maternal & Fetal Medicine | 140 (0.1%) | 2157 (4.0%) | 2297 (1.0%) | 31 (0.1%) | 1886 (4.3%) | 1917 (2.2%) | ||
| Nephrology | 1051 (0.6%) | 459 (0.9%) | 1510 (0.6%) | 232 (0.5%) | 403 (0.9%) | 635 (0.7%) | ||
| Neurology | 6600 (3.6%) | 12 056 (22.4%) | 18 656 (7.9%) | 1674 (3.8%) | 10 255 (23.5%) | 11 929 (13.7%) | ||
| Neuropsychology | 6 (0.0%) | 402 (0.7%) | 408 (0.2%) | 3 (0.0%) | 354 (0.8%) | 357 (0.4%) | ||
| Neurosurgery | 17 043 (9.3%) | 2137 (4.0%) | 19 180 (8.1%) | 3211 (7.3%) | 1682 (3.9%) | 4893 (5.6%) | ||
| Obstetrics & Gynecology | 19 132 (10.5%) | 1442 (2.7%) | 20 574 (8.7%) | 5486 (12.6%) | 1317 (3.0%) | 6803 (7.8%) | ||
| Ophthalmology | 14 449 (7.9%) | 7 (0.0%) | 14 456 (6.1%) | 2665 (6.1%) | 5 (0.0%) | 2670 (3.1%) | ||
| Orthopedic Surgery | 2619 (1.4%) | 642 (1.2%) | 3261 (1.4%) | 666 (1.5%) | 439 (1.0%) | 1105 (1.3%) | ||
| Otorhinolaryngology | 11 899 (6.5%) | 663 (1.2%) | 12 562 (5.3%) | 2461 (5.6%) | 565 (1.3%) | 3026 (3.5%) | ||
| Pediatric Allergy & Immunology | 607 (0.3%) | 189 (0.4%) | 796 (0.3%) | 213 (0.5%) | 158 (0.4%) | 371 (0.4%) | ||
| Pediatric Gastroenterology | 69 (0.0%) | 4 (0.0%) | 73 (0.0%) | 26 (0.1%) | 4 (0.0%) | 30 (0.0%) | ||
| Pediatric Genetics | 190 (0.1%) | 369 (0.7%) | 559 (0.2%) | 57 (0.1%) | 273 (0.6%) | 330 (0.4%) | ||
| Pediatric Hematology-Oncology | 109 (0.1%) | 9 (0.0%) | 118 (0.0%) | 61 (0.1%) | 9 (0.0%) | 70 (0.1%) | ||
| Pediatric Infectious Diseases | 611 (0.3%) | 30 (0.1%) | 641 (0.3%) | 150 (0.3%) | 26 (0.1%) | 176 (0.2%) | ||
| Pediatric Nephrology | 612 (0.3%) | 21 (0.0%) | 633 (0.3%) | 346 (0.8%) | 19 (0.0%) | 365 (0.4%) | ||
| Pediatric Neurology | 108 (0.1%) | 94 (0.2%) | 202 (0.1%) | 38 (0.1%) | 81 (0.2%) | 119 (0.1%) | ||
| Pediatric Psychiatry | 2 (0.0%) | 71 (0.1%) | 73 (0.0%) | |||||
| Pediatric Pulmonology | 94 (0.1%) | 19 (0.0%) | 113 (0.0%) | 58 (0.1%) | 17 (0.0%) | 75 (0.1%) | ||
| Pediatrics | 3704 (2.0%) | 1572 (2.9%) | 5276 (2.2%) | 1238 (2.8%) | 1419 (3.2%) | 2657 (3.0%) | ||
| Physical Therapy | 1247 (0.7%) | 6 (0.0%) | 1253 (0.5%) | 488 (1.1%) | 5 (0.0%) | 493 (0.6%) | ||
| Plastic Surgery | 3836 (2.1%) | 146 (0.3%) | 3982 (1.7%) | 873 (2.0%) | 127 (0.3%) | 1000 (1.1%) | ||
| Psychiatry | 2269 (1.2%) | 4836 (9.0%) | 7105 (3.0%) | 388 (0.9%) | 3576 (8.2%) | 3964 (4.5%) | ||
| Psychology | 2 (0.0%) | 6 (0.0%) | 8 (0.0%) | 1 (0.0%) | 5 (0.0%) | 6 (0.0%) | ||
| Pulmonary Disease | 2741 (1.5%) | 345 (0.6%) | 3086 (1.3%) | 630 (1.4%) | 315 (0.7%) | 945 (1.1%) | ||
| Rheumatology | 4100 (2.2%) | 556 (1.0%) | 4656 (2.0%) | 886 (2.0%) | 520 (1.2%) | 1406 (1.6%) | ||
| Speech Pathology | 40 (0.0%) | 32 (0.1%) | 72 (0.0%) | 10 (0.0%) | 18 (0.0%) | 28 (0.0%) | ||
| Sports Medicine | 116 (0.1%) | 304 (0.6%) | 420 (0.2%) | 86 (0.2%) | 223 (0.5%) | 309 (0.4%) | ||
| Urgent Care | 656 (0.4%) | 117 (0.2%) | 773 (0.3%) | 254 (0.6%) | 65 (0.1%) | 319 (0.4%) | ||
| Urology | 11 916 (6.5%) | 3493 (6.5%) | 15 409 (6.5%) | 1855 (4.2%) | 2932 (6.7%) | 4787 (5.5%) | ||
| Vascular Surgery | 4602 (2.5%) | 244 (0.5%) | 4846 (2.0%) | 824 (1.9%) | 216 (0.5%) | 1040 (1.2%) |
Results for telemedicine versus in-person care
Figure 1 depicts the distribution of appointment status outcomes within the in-person care and telemedicine groups. In the matched cohort, the percentage of completed telemedicine appointments exceeded that of completed in-person care appointments by 9.2 points (73.4% vs 64.2%, P < .001). Within the not completed appointments, canceled appointments occurred less frequently in the telemedicine group than in the in-person care group (21.3% vs 29.2%, P < .001) and no-show appointments occurred less frequently in the telemedicine group than in the in-person care group (5.2% vs 6.6%).
Figure 1.
Distribution of appointment completion outcomes for telemedicine and standard (in-person) care (matched cohort).
The unadjusted odds ratio for appointment completion with telemedicine versus in-person care appointments was 1.54 (95% CI, 1.50-1.59, P < .001), as seen in Table 3. The multivariate model of appointment completion resulted in a Hosmer-Lemeshow Chi-squared 723, df = 8, P < .001. Multiple variables reached statistical significance as predictors of appointment completion including age, gender, race, ethnicity, insurance type, visit type, and comorbidities (see Table 4). Upon adjustment, the odds ratio for telemedicine versus in-person care in relation to appointment completion was 1.64 (95% CI, 1.59-1.69, P < .001), indicating that telemedicine appointments were associated with 64% higher odds of completion than in-person care appointments when controlling for other factors. We created an additional multivariate model incorporating specialty as a covariate, detailed in Supplementary Material, Appendix A. In this model, the adjusted odds ratio for telemedicine versus in-person care was 1.71 (95% CI, 1.65-1.77, P < .001).
Table 3.
Univariate model, effect of telemedicine on appointment completion (N = 104 698).
| Predictors | Odds ratio | 95% confidence intervals | P |
|---|---|---|---|
| (Intercept) | 1.79 | 1.76-1.83 | <.001 |
| Telemedicine | 1.54 | 1.50-1.59 | <.001 |
| Observations | 87 376 | ||
| R2 Tjura | 0.01 |
Tjur’s R2 coefficient of determination.
Table 4.
Multivariate model, effect of telemedicine on appointment completion (N = 104 698).
| Predictors | Odds ratio | 95% confidence intervals | P |
|---|---|---|---|
| (Intercept) | 9.74 | 9.06-10.48 | <.001 |
| Telemedicine | 1.64 | 1.59-1.69 | <.001 |
| Age | 0.95 | 0.94-0.95 | <.001 |
| Gender [Male] | 0.93 | 0.90-0.96 | <.001 |
| Race [ref: White/Caucasian] | |||
| Black or African American | 0.84 | 0.80-0.88 | <.001 |
| Other/Unknown | 0.95 | 0.91-1.00 | .041 |
| Ethnicity [Not Hispanic/Latino] | 1.11 | 1.07-1.16 | <.001 |
| Insurance Type [ref: Private] | |||
| Medicaid | 0.55 | 0.53-0.58 | <.001 |
| Medicare | 0.49 | 0.47-0.52 | <.001 |
| Other | 0.98 | 0.90-1.07 | .625 |
| Visit type [Established patient] | 0.94 | 0.91-0.97 | <.001 |
| Travel distance | 1 | 1.00-1.00 | .751 |
| Charlson Comorbidity Index | 2.47 | 2.42-2.53 | <.001 |
| Observations | 87 376 | ||
| R2 Tjura | 0.112 |
Tjur’s R2 coefficient of determination.
Discussion
This large cohort study compared the effect of telemedicine versus in-person care upon outpatient appointment completion across diverse specialties within an academic health sciences center. We found that telemedicine appointments are associated with a 64% increase in the odds of completion and are less likely to result in cancellations or no-shows after adjusting for demographics, payment type, comorbidity, and distance. These findings suggest that telemedicine appointments are more likely to be completed than in-person care appointments. Additional factors may also influence appointment completion and should be considered in efforts to implement telemedicine and optimize healthcare access for a community.
Completing scheduled healthcare appointments is fundamental to quality care delivery and continuity of care. Missed appointments, whether initial or follow-up visits, may diminish continuity of care for an individual, adversely affecting patient outcomes.27 For patients with a history of missed appointments and those receiving mental healthcare, particularly those at risk of suicide, the availability of telemedicine may be critical for accomplishing care and mitigating adverse outcomes.2 It may be equally important for any patient experiencing barriers such as transportation, childcare responsibilities, or conflicts with work. A break in care continuity may be especially consequential when patient outcomes depend on close monitoring or specific timing of treatments. Additionally, missed preventative care appointments could impact both individual and public health outcomes.28 Conversely, appointment completion may foster subsequent adherence. In a 2017 study, initial appointment completion for patients with HIV was associated with an increased rate of overall visits attended and fewer gaps in care.27
To deliver high-quality care, we must find solutions that address the underlying reasons for missed appointments. Currently, common strategies to prevent “no-shows” include telephone and SMS reminders, patient portal messages, and overbooking.29 While simple appointment reminders can counteract forgetfulness, there is a need to address other, more complex factors, including transportation and socioeconomic difficulties affecting an individual’s ability to adhere to an appointment.28 Our findings indicate that the availability of telemedicine services could be an important strategy for preventing “no-shows” and improving appointment completion in the post-pandemic era. However, current evidence suggests that telemedicine is more satisfying and accessible for some patients than others.15,30–32 In-person care appointments may be more appropriate for patients who indicate they do not have access to appropriate telemedicine technology, are uncomfortable with telemedicine, or simply prefer in-person care appointments.
We know that system-related factors, patient-related factors, socioeconomic characteristics, and the nature of medical care all influence patient preferences related to telemedicine versus in-person care.33 Brief assessment of these factors may enable better scheduling decisions where options exist for either telemedicine or in-person care appointments. If offered the option of telemedicine versus in-person care, patients can choose the modality that best supports their ability to attend the appointment. After initial scheduling, the capability to adapt the appointment type to the patient’s current circumstances may also support appointment completion. When in-person care appointments are necessary and preferred, planning to address barriers such as transportation and childcare may be helpful in promoting appointment completion.
Limitations
This study included a large and diverse population but also reflects the system, provider, and patient characteristics of a single academic healthcare system in the southeastern Unites States. Patterns of appointment completion with telemedicine could differ with local or regional variation in the quality and stability of telemedicine services, the travel distance of patients from healthcare facilities, policy governing reimbursement for telemedicine services, access to appropriate devices, internet connection quality, and preferences for obtaining care via telemedicine and in-person. We limited this study to first telemedicine visits and appointment completion may have differed for subsequent appointments, given increased telemedicine familiarity. We did not consider individual providers as a confounding variable. We note that this was a large-scale, EHR-based study, and individual provider and clinic documentation practices in classifying missed appointment outcomes may have varied. Additionally, we did not include telephone-only appointments in the analysis. However, telemedicine appointments may have taken place via telephone-only if patient or provider experience difficulties with their connection. Additionally, while telemedicine was associated with higher odds of appointment completion overall, there could be differences among population sub-groups with different perspectives and barriers related to healthcare services. We plan to explore these nuances in future mixed-methods research. While the completion of a scheduled healthcare appointment is desirable, and appointment completion has found to be associated with better overall health outcomes, there are numerous individual-, system-, and treatment-related factors that contribute to health commons. Certainly, when telemedicine is inappropriate given a patient’s healthcare needs, quality could be diminished, and additional healthcare resources may be needed for appropriate care.34
Conclusions
This retrospective cohort study conducted at a large academic health sciences center indicates that telemedicine appointments are more likely to be completed than in-person care appointments, regardless of demographics, comorbidity, payment type, or distance. Telemedicine appointments are more likely to be completed, potentially enhancing quality of care and avoiding adverse outcomes, including missed or delayed care, emergency department visits, poor clinical outcomes, and wasted resources. However, there is a need to consider patient preferences and barriers when selecting an appropriate modality for care delivery, whether in-person care or telemedicine, and these should be assessed to optimize scheduling. Given this finding, telemedicine services should be considered a key strategy in improving care quality and mitigating the adverse effects of missed appointments. Future research should more closely examine the experience and outcomes of telemedicine care delivery within specific clinical sub-populations and geographically diverse settings.
Supplementary Material
Contributor Information
Mollie R Cummins, Department of Biomedical Informatics, College of Nursing and Spencer Fox Eccles School of Medicine, University of Utah, Salt Lake City, UT 84112-5880, United States; Doxy.me Inc., Charleston, SC 29401, United States.
Athanasios Tsalatsanis, Office of Research, Morsani College of Medicine, University of South Florida, Tampa, FL 33612, United States.
Chaitanya Chaphalkar, Office of Research, Morsani College of Medicine, University of South Florida, Tampa, FL 33612, United States.
Julia Ivanova, Doxy.me Inc., Charleston, SC 29401, United States.
Triton Ong, Doxy.me Inc., Charleston, SC 29401, United States.
Hiral Soni, Doxy.me Inc., Charleston, SC 29401, United States.
Janelle F Barrera, Doxy.me Inc., Charleston, SC 29401, United States; Department of Psychiatry and Behavioral Neurosciences, Morsani College of Medicine, University of South Florida, Tampa, FL 33612, United States.
Hattie Wilczewski, Doxy.me Inc., Charleston, SC 29401, United States.
Brandon M Welch, Doxy.me Inc., Charleston, SC 29401, United States; Biomedical Informatics Center, Medical University of South Carolina, Charleston, SC 29425, United States.
Brian E Bunnell, Doxy.me Inc., Charleston, SC 29401, United States; Department of Psychiatry and Behavioral Neurosciences, Morsani College of Medicine, University of South Florida, Tampa, FL 33612, United States.
Author contributions
Mollie R. Cummins, Athanasios Tsalatsanis, and Brian E. Bunnell conceived and designed the study, developed the protocol, and obtained ethical approval. Athanasios Tsalatsanis and Chaitanya Chaphalkar conducted the statistical analysis. Julia Ivanova, Triton Ong, Hiral Soni, Janelle F. Barrera, Hattie Wilczewski, and Brandon M. Welch contributed to the literature search and interpretation of findings. Mollie R. Cummins wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Supplementary material
Supplementary material is available at JAMIA Open online.
Funding
This manuscript was partially supported by the National Institute of Mental Health (NIMH) of the National Institutes of Health under Award Numbers K23MH118482 and R42LM012547. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflicts of interest
B.M.W. is a shareholder, and M.R.C., J.I., T.O., H.S., J.F.B., H.W., and B.E.B. are employees of Doxy.me Inc., a commercial telemedicine company. The authors declare no other competing interests.
Data availability
This study entailed analysis of electronic health record data. Due to the nature of the research, and relevant ethical and legal considerations, supporting data are not available.
Ethical approval and consent to participate
The Institutional Review Board of the University of South Florida approved this study as exempt human subjects research (STUDY005197). Consistent with this determination, we did not obtain individual consent to participate.
References
- 1. McQueenie R, Ellis DA, McConnachie A, Wilson P, Williamson AE.. Morbidity, mortality and missed appointments in healthcare: a national retrospective data linkage study. BMC Med. 2019;17(1):2. 10.1186/s12916-018-1234-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Mitchell AJ, Selmes T.. Why don't patients attend their appointments? Maintaining engagement with psychiatric services. Adv Psychiatr Treat. 2007;13(6):423-434. 10.1192/apt.bp.106.003202 [DOI] [Google Scholar]
- 3. Davies ML, Goffman RM, May JH, et al. Large-scale no-show patterns and distributions for clinic operational research. Healthcare (Basel). 2016;4(1):15. 10.3390/healthcare4010015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Kheirkhah P, Feng Q, Travis LM, Tavakoli-Tabasi S, Sharafkhaneh A.. Prevalence, predictors and economic consequences of no-shows. BMC Health Serv Res. 2016;16:13. 10.1186/s12913-015-1243-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Triemstra JD, Lowery L.. Prevalence, predictors, and the financial impact of missed appointments in an academic adolescent clinic. Cureus. 2018;10(11):e3613. 10.7759/cureus.3613 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ellis DA, McQueenie R, McConnachie A, Wilson P, Williamson AE.. Demographic and practice factors predicting repeated non-attendance in primary care: a national retrospective cohort analysis. Lancet Public Health. 2017;2(12):e551-e559. 10.1016/S2468-2667(17)30217-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Ullah S, Rajan S, Liu T, et al. Why do patients miss their appointments at primary care clinics? J Fam Med Dis Prev. 2023;4:090. 10.23937/2469-5793/1510090 [DOI] [Google Scholar]
- 8. Brown EE, Schwartz M, Shi C, et al. Understanding why urban, low-income patients miss primary care appointments: insights from qualitative interviews of West Philadelphians. J Ambul Care Manage. 2020;43(1):30-40. 10.1097/JAC.0000000000000316 [DOI] [PubMed] [Google Scholar]
- 9. Abdalkareem ZA, Amir A, Al-Betar MA, Ekhan P, Hammouri AI.. Healthcare scheduling in optimization context: a review. Health Technol (Berl). 2021;11(3):445-469. 10.1007/s12553-021-00547-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Liu Q, Abba K, Alejandria MM, Sinclair D, Balanag VM, Lansang MAD.. Reminder systems to improve patient adherence to tuberculosis clinic appointments for diagnosis and treatment. Cochrane Database Syst Rev. 2014;2014(11):CD006594. 10.1002/14651858.CD006594.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Oikonomidi T, Norman G, McGarrigle L, Stokes J, van der Veer SN, Dowding D.. Predictive model-based interventions to reduce outpatient no-shows: a rapid systematic review. J Am Med Inform Assoc. 2023;30(3):559-569. 10.1093/jamia/ocac242 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Adepoju OE, Angelocci T, Matuk-Villazon O.. Increased revenue from averted missed appointments following telemedicine adoption at a large federally qualified health center. Health Serv Insights. 2022;15:11786329221125409. 10.1177/11786329221125409 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Bynum AB, Irwin CA, Cranford CO, Denny GS.. The impact of telemedicine on patients' cost savings: some preliminary findings. Telemed J E Health. 2003;9(4):361-367. 10.1089/153056203772744680 [DOI] [PubMed] [Google Scholar]
- 14. Marcin JP, Ellis J, Mawis R, Nagrampa E, Nesbitt TS, Dimand RJ.. Using telemedicine to provide pediatric subspecialty care to children with special health care needs in an underserved rural community. Pediatrics. 2004;113(1 Pt 1):1-6. 10.1542/peds.113.1.1 [DOI] [PubMed] [Google Scholar]
- 15. Nanda M, Sharma R.. A review of patient satisfaction and experience with telemedicine: a virtual solution during and beyond COVID-19 pandemic. Telemed J E Health. 2021;27(12):1325-1331. 10.1089/tmj.2020.0570 [DOI] [PubMed] [Google Scholar]
- 16. Barbosa W, Zhou K, Waddell E, Myers T, Dorsey ER.. Improving access to care: telemedicine across medical domains. Annu Rev Public Health. 2021;42:463-481. 10.1146/annurev-publhealth-090519-093711 [DOI] [PubMed] [Google Scholar]
- 17. Brunetti ND, Amoruso D, De Gennaro L, et al. Hot spot: impact of July 2011 heat-wave in Southern Italy (Apulia) on cardiovascular disease assessed by emergency medical service and tele-medicine support. Eur. Heart J. 2013;34(suppl 1):P2504. 10.1093/eurheartj/eht308.P2504 [DOI] [PubMed] [Google Scholar]
- 18. Grover JM, Smith B, Williams JG, Patel MD, Cabanas JG, Brice JH.. Novel use of telemedicine by hurricane evacuation shelters. Prehosp Emerg Care. 2020;24(6):804-812. 10.1080/10903127.2020.1723756 [DOI] [PubMed] [Google Scholar]
- 19. Haider Z, Aweid B, Subramanian P, Iranpour F.. Telemedicine in orthopaedics during COVID-19 and beyond: a systematic review. J Telemed Telecare. 2022;28(6):391-403. 10.1177/1357633X20938241 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. mHealthIntelligence. GAO tells Congress to wait on expanding telehealth coverage past the pandemic. 2023. Accessed June 24, 2024. https://mhealthintelligence.com/news/gao-tells-congress-to-wait-on-expanding-telehealth-coverage-past-the-pandemic
- 21. U.S. Government Accountability Office. Medicare and Medicaid: COVID-19 Program Flexibilities and Considerations for Their Continuation. U.S. Government Accountability Office; 2023. [Google Scholar]
- 22. United States Census Bureau. QuickFacts: Tampa City, Florida. United States Census Bureau; 2023. [Google Scholar]
- 23. Charlson ME, Pompei P, Ales KL, MacKenzie CR.. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-383. 10.1016/0021-9681(87)90171-8 [DOI] [PubMed] [Google Scholar]
- 24. Ho DE, Imai K, King G, Stuart EA.. MatchIt: nonparametric preprocessing for parametric causal inference. J Stat Soft. 2011;42(8):1-28. 10.18637/jss.v042.i08 [DOI] [Google Scholar]
- 25. Rozzi GC. zipcodeR: advancing the analysis of spatial data at the ZIP code level in R. Software Impacts. 2021;9:100099. [Google Scholar]
- 26. Nelder JA, Wedderburn RWM.. Generalized linear models. J R Stat Soc Ser A. 1972;135(3):370-384. 10.2307/2344614 [DOI] [Google Scholar]
- 27. Nijhawan AE, Liang Y, Vysyaraju K, et al. Missed initial medical visits: predictors, timing, and implications for retention in HIV care. AIDS Patient Care STDS. 2017;31(5):213-221. 10.1089/apc.2017.0030 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Wilson R, Winnard Y.. Causes, impacts and possible mitigation of non-attendance of appointments within the national health service: a literature review [published online ahead of print August 4, 2022]. J Health Organ Manag. 10.1108/JHOM-11-2021-0425 [DOI] [PubMed] [Google Scholar]
- 29. Berg BP, Murr M, Chermak D, et al. Estimating the cost of no-shows and evaluating the effects of mitigation strategies. Med Decis Making. 2013;33(8):976-985. 10.1177/0272989X13478194 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Bove R, Garcha P, Bevan CJ, Crabtree-Hartman E, Green AJ, Gelfand JM.. Clinic to in-home telemedicine reduces barriers to care for patients with MS or other neuroimmunologic conditions. Neurol Neuroimmunol Neuroinflamm. 2018;5(6):e505. 10.1212/NXI.0000000000000505 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Qian AS, Schiaffino MK, Nalawade V, et al. Disparities in telemedicine during COVID-19. Cancer Med. 2022;11(4):1192-1201. 10.1002/cam4.4518 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Scott Kruse C, Karem P, Shifflett K, Vegi L, Ravi K, Brooks M.. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare. 2018;24(1):4-12. 10.1177/1357633X16674087 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Alsabeeha NHM, Atieh MA, Balakrishnan MS.. Older adults' satisfaction with telemedicine during the COVID-19 pandemic: a systematic review. Telemed J E Health. 2023;29(1):38-49. 10.1089/tmj.2022.0045 [DOI] [PubMed] [Google Scholar]
- 34. Shah VV, Villaflores CW, Chuong LH, et al. Association between in-person vs telehealth follow-up and rates of repeated hospital visits among patients seen in the emergency department. JAMA Netw Open. 2022;5(10):e2237783. 10.1001/jamanetworkopen.2022.37783 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
This study entailed analysis of electronic health record data. Due to the nature of the research, and relevant ethical and legal considerations, supporting data are not available.