Application of a Machine Learning Algorithm to Develop and Validate a Prediction Model for Ambulatory Non-Arrivals

Abstract

Background

Non-arrivals to scheduled ambulatory visits are common and lead to a discontinuity of care, poor health outcomes, and increased subsequent healthcare utilization. Reducing non-arrivals is important given their association with poorer health outcomes and cost to health systems.

Objective

To develop and validate a prediction model for ambulatory non-arrivals.

Design

Retrospective cohort study.

Patients or Subjects

Patients at an integrated health system who had an outpatient visit scheduled from January 1, 2020, to February 28, 2022.

Main Measures

Non-arrivals to scheduled appointments.

Key Results

There were over 4.3 million ambulatory appointments from 1.2 million adult patients. Patients with appointment non-arrivals were more likely to be single, racial/ethnic minorities, and not having an established primary care provider compared to those who arrived at their appointments. A prediction model using the XGBoost machine learning algorithm had the highest AUC value (0.768 [0.767–0.770]). Using SHAP values, the most impactful features in the model include rescheduled appointments, lead time (number of days from scheduled to appointment date), appointment provider, number of days since last appointment with the same department, and a patient’s prior appointment status within the same department. Scheduling visits close to an appointment date is predicted to be less likely to result in a non-arrival. Overall, the prediction model calibrated well for each department, especially over the operationally relevant probability range of 0 to 40%. Departments with fewer observations and lower non-arrival rates generally had a worse calibration.

Conclusions

Using a machine learning algorithm, we developed a prediction model for non-arrivals to scheduled ambulatory appointments usable for all medical specialties. The proposed prediction model can be deployed within an electronic health system or integrated into other dashboards to reduce non-arrivals. Future work will focus on the implementation and application of the model to reduce non-arrivals.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11606-023-08065-y.

INTRODUCTION

Timely access to medical care is vital to improving health outcomes and reducing healthcare costs. Appropriate utilization of ambulatory care is cost-effective and is associated with a reduction in unnecessary emergency department visits and hospitalizations.^1–3 However, non-arrivals to scheduled ambulatory visits are common, ranging from 14% to as high as 50%,^4–6 and lead to a discontinuity of care, poor health outcomes, and increased subsequent healthcare utilization.^{1, 7–12} Appointment slots that go unfilled due to patient non-arrivals create inefficiencies and health system revenue loss, and can delay medical care for those in need. As non-arrivals are costly to health systems, there is increasing interest to reduce non-arrivals to improve clinical productivity, financial performance, and clinical quality and outcomes.^{13, 14}

Prior studies have identified patient and appointment characteristics associated with non-arrivals to scheduled appointments. Patients who are young, without health insurance, with mental health disorders, or with adverse social determinants of health often have high non-arrival rates.^15–23 A personal history of prior missed appointments has been strongly associated with increased odds of non-arrivals.^{19, 24, 25} More generally, the appointment time, day of the week, time of the day, and lead time to an appointment are associated with non-arrivals,^{17, 20, 26} as are unfavorable weather conditions, including heavy rain/snow and extreme temperature.^27–30 This latter factor is particularly important as extreme weather conditions have become more frequent in recent years and are expected to worsen.^{31, 32}

While several prediction models for non-arrivals exist, they have some limitations. Most non-arrival prediction models are limited to a single patient group, specialty, or department, resulting in limited generalizability and usability.^{30, 33–37} Development of a more narrowly focused prediction model may improve performance of a validation or test dataset, but generalizability and the ability to operationalize the model are limited. Lastly, the novel coronavirus disease 2019 (COVID-19) global pandemic led to significant changes in healthcare utilization.^38–40 Such changes could plausibly lead to degradation in model performance, when solely based on pre-COVID-19 data.

Using health system data that includes a diverse patient population, we have identified ways to overcome these limitations. To address variability in non-arrival prediction models among different specialties, the use of nonlinear, machine learning–based algorithms could potentially increase the predictive ability of the models.⁴¹ The development of a non-arrival prediction model applicable to all subspecialties can improve usability and ease of deployment. Lastly, due to significant changes in ambulatory care,^38–40 we examine model performance of our non-arrival prediction model in the post-COVID-19 period. To enhance predictability and applicability, we have sought to improve non-arrival risk prediction among patients scheduled to all major subspecialty outpatient clinics within a large integrated health system.

METHODS

Data Source

Northwell Health is the largest health system in New York with 23 hospitals and over 700 ambulatory facilities. Data for this study were obtained from the enterprise ambulatory electronic health record (EHR; TouchWorks, Altera [formerly Allscripts]), which is utilized at over 450 ambulatory locations, and the enterprise inpatient EHR (Sunrise Clinical Manager, Altera [formerly Allscripts]), capturing data from 13 inpatient hospitals.

Study Population

We first queried the ambulatory EHR database to obtain data on in-person ambulatory visits between January 1, 2020, and February 28, 2022. We only included visits of patients 18 years or older. We included ambulatory visits of the following types: return patient appointment, new patient appointment, annual appointment, complete physical examination, and hospital follow-up. We ranked ambulatory divisions by the number of appointments and chose to include the top twenty. We chose a large number (20) of ambulatory divisions to have representation from a diverse set of patients and create a generalizable prediction model. The divisions include General Cardiology, Dermatology, Endocrinology, Family Medicine, Gastroenterology, General Surgery, Heart and Vascular, Hematology Oncology, Infectious Disease, Internal Medicine, Nephrology, Neurology, Neurosurgery, Obstetrics and Gynecology, Ophthalmology, Orthopedic Surgery, Otolaryngology, Pulmonary Medicine, Rheumatology, and Urology. Data on patients’ inpatient hospital utilization, either inpatient or emergency department visits, were obtained from the inpatient EHR database. We identified patients’ hospital utilization up to 1 year prior to the ambulatory appointment. We used the health system Enterprise Master Patient Index to link patients’ inpatient admissions with their ambulatory visits.

Outcomes

The primary outcome was a “non-arrival” to a scheduled appointment. The EHR contains information about when appointments were scheduled, rescheduled, or canceled. We incorporated labeled no-shows and late cancelations, defined as a cancelation within 3 business days of an appointment, in our non-arrival definiton.⁴² We included late cancelations in the definition of non-arrivals because the clinical and operational impact of late cancelations and interventions to address late cancelations are functionally the same as those for no-shows.

Dataset Features

The feature set used to develop our non-arrival prediction model contains sociodemographic data on the patient, including age, sex, race/ethnicity, health insurance, English proficiency, marital status, having a primary care provider, patient zip code, and patient portal status (Table 1). The feature set also contains information about the scheduled appointments, including the appointment time, primary provider, day of the week, month of the year, number of days from scheduled to appointment date (lead time), whether the appointment was previously rescheduled, and whether the appointment is within 3 business days of a major American holiday. We included features that describe individual patients’ appointment history—whether the patient missed their previous appointment or not and the number of prior ambulatory appointments, as well as the arrival, cancelation, and no-show rates of those appointments—in the entire Northwell system, with specific departments, with specific providers, and with specific locations. The feature set contains information about the appointment type, the department, and the practice location. We also created features that described the non-arrival rates for each department, location, appointment type, patient zip code, appointment type, and day of week over the prior 3 months. Using the OpenWeatherMap application programming interface (API), we included local weather information, including the daily maximum and minimum temperature, average wind speed, and precipitation (coded as binary present or absent) in the feature set. Lastly, we added the inpatient and emergency department hospital utilization of the patient over the past 6 months and whether the patient was hospitalized within 3 business days of the appointment or not (Table S2). For all categorical features included in the dataset, we employed target encoding to provide the model with information on the past arrival rates for specific levels of each feature.⁴³ When target encoded features were missing data, we imputed the overall non-arrival rate in the training dataset.

Table 1.

Patient Characteristics by Non-arrival Status. Comparison of Appointment and Patient Characteristics by Study Outcome

	Arrived	Non-arrival
	(n = 3,607,324)	(n = 763,414)
Age (median [IQR])	61.00 [47.00, 72.00]	56.00 [41.00, 69.00]
Sex—female	2,067,121 (58.2)	398,945 (59.2)
Race
White	2,182,842 (60.5)	386,042 (50.6)
Black	387,944 (10.8)	110,408 (14.5)
Asian	186,169 (5.2)	38,493 (5.0)
Other	527,660 (14.6)	129,667 (17.0)
Declined	106,490 (3.0)	19,967 (2.6)
Unknown	216,219 (6.0)	78,837 (10.3)
Ethnicity
Non-Hispanic or Latino	2,670,985 (74.0)	483,551 (63.3)
Hispanic or Latino	390,932 (10.8)	92,618 (12.1)
Declined	168,286 (4.7)	28,034 (3.7)
Unknown	377,121 (10.5)	159,211 (20.9)
Marital status
Married	1,825,277 (50.6)	322,724 (43.4)
Single	984,085 (27.3)	249,291 (33.5)
Widowed	238,929 (6.6)	45,144 (6.1)
Divorce	229,359 (6.4)	49,465 (6.7)
Unknown	329,626 (9.1)	76,677 (10.3)
Language–English	3,304,710 (91.6)	683,610 (89.5)
Patient portal status
Info provided	1,551,246 (43.0)	328,704 (43.1)
Registered	1,636,102 (45.4)	271,498 (35.6)
Info not provided	404,029 (11.2)	160,047 (21.0)
Refused	15,947 (0.4)	3165 (0.4)
Insurance (%)
Private	2,252,027 (62.4)	446,057 (58.4)
Medicare	595,023 (16.5)	177,753 (23.3)
Medicaid	760,171 (21.1)	139,570 (18.3)
Other	55 (0.0)	26 (0.0)
Has Northwell PCP	2,615,942 (72.5)	465,220 (60.9)
Day of week
Monday	730,248 (20.2)	167,740 (22.0)
Tuesday	809,342 (22.4)	173,198 (22.7)
Wednesday	758,676 (21.0)	154,578 (20.2)
Thursday	740,429 (20.5)	152,786 (20.0)
Friday	568,629 (15.8)	115,112 (15.1)
Appointment within 3 days of holiday	149,786 (4.2)	32,579 (4.3)
Days from scheduled date (median [IQR])	14.89 [4.23, 42.01]	25.10 [7.12, 62.88]
Rescheduled appointment	947,220 (26.3)	83,181 (10.9)
Past arrival % (median [IQR])	90.00 [75.00, 100.00]	81.00 [64.00, 100.00]
Past canceled % (median [IQR])	0.00 [0.00, 17.00]	4.00 [0.00, 20.00]
Past no-show % (median [IQR])	0.00 [0.00, 3.00]	0.00 [0.00, 15.00]
Arrived to last appointment (%)	2,636,707 (84.2)	497,271 (76.3)
Number of inpatient visits (median [IQR])	1.00 [1.00, 1.00]	1.00 [1.00, 2.00]
Number of ED visits (median [IQR])	0.00 [0.00, 1.00]	0.00 [0.00, 1.00]
In hospital during appointment date (%)	94 (0.0)	7710 (1.0)
Appointment type (%)
Annual patient appointment	172,861 (4.8)	31,309 (4.1)
Complete physical exam	196,796 (5.5)	26,019 (3.4)
Hospital follow-up	38,019 (1.1)	9935 (1.3)
Internal referral appointment	40,227 (1.1)	8601 (1.1)
New GRN patient appointment	242 (0.0)	62 (0.0)
New patient appointment	805,375 (22.3)	197,748 (25.9)
Return GRN patient appointment	5415 (0.2)	1632 (0.2)
6-month return patient appointment	7389 (0.2)	1442 (0.2)
Return patient appointment	2,336,445 (64.8)	486,015 (63.7)
Study patient appointment	4076 (0.1)	578 (0.1)
Travel patient appointment	479 (0.0)	73 (0.0)
Medical specialties (%)
Cardiology	447,979 (12.4)	83,620 (11.0)
Dermatology	103,033 (2.9)	26,436 (3.5)
Endocrinology	144,423 (4.0)	35,743 (4.7)
Family medicine	293,601 (8.1)	51,364 (6.7)
Gastroenterology	127,402 (3.5)	33,955 (4.4)
General surgery	91,442 (2.5)	17,945 (2.4)
Heart and vascular	105,255 (2.9)	22,879 (3.0)
Hematology/oncology	155,120 (4.3)	24,864 (3.3)
Infectious disease	36,254 (1.0)	11,068 (1.4)
Internal medicine	551,678 (15.3)	95,283 (12.5)
Nephrology	40,541 (1.1)	8036 (1.1)
Neurology	112,822 (3.1)	33,717 (4.4)
Neurosurgery	39,224 (1.1)	8124 (1.1)
Obstetrics and gynecology	234,955 (6.5)	51,158 (6.7)
Ophthalmology	147,195 (4.1)	29,655 (3.9)
Orthopedic surgery	324,018 (9.0)	71,046 (9.3)
Otolaryngology	156,988 (4.4)	36,033 (4.7)
Pulmonary medicine	188,326 (5.2)	45,526 (6.0)
Rheumatology	91,766 (2.5)	20,798 (2.7)
Urology	215,302 (6.0)	56,164 (7.4)
Time slot (%)
Morning	1,290,535 (35.8)	262,969 (34.4)
Noon	754,224 (20.9)	156,804 (20.5)
Afternoon	1,265,400 (35.1)	274,919 (36.0)
Evening	297,165 (8.2)	68,722 (9.0)
Weather information
Precipitation (%)	951,153 (26.4)	206,268 (27.0)
Daily max temp (median [IQR])	17.78 [10.56, 25.56]	17.60 [10.00, 25.56]
Daily min temp (median [IQR])	9.13 [2.22, 17.22]	9.13 [2.22, 16.67]
Average wind speed (median [IQR])	8.28 [6.26, 10.07]	8.28 [6.26, 10.07]

Modeling

The dataset was split into training, validation, and testing sets based on the date of appointment (Fig. 1). We used appointments from January 1, 2020, through July 31, 2021, for model training, but excluded appointments during the peak of the first COVID-19 wave from March 1, 2020, to June 1, 2020, due to the Northwell ambulatory network being shut down and mislabeling of telehealth visits during this period. We used appointments from August 1, 2021, through November 30, 2021, to validate the model performance. We used appointments from December 1, 2021, through February 28, 2022, as a testing set to examine the model performance on data not previously seen by the model.

Fig. 1.

Description of training, validation, and testing cohort split. Appointments that took place during the peak of the first COVID-19 wave from March 1, 2020, to June 1, 2020, were excluded due to the Northwell ambulatory network being shut down and mislabeling of telehealth visits during this period

The primary algorithm used was XGBoost, a decision tree–based ensemble machine learning algorithm that uses a gradient boosting framework. As a reference model, we fit a logistic regression (LR) on the same data used to train the XGBoost model. We also compared the performance of the standard rule-based no-show identification method used at Northwell Health, which flags patients who have three or more no-shows in the past 6 months (NW Tool). We compared discrimination performance of the existing health system’s no-show model, a logistic regression, and an XGBoost model using the area under the receiver operating characteristic (ROC) curve, and used the Delong method when comparing the area under the ROC curve between the three models. Calibration was assessed using the Brier score, a commonly used score to evaluate accuracy of probabilistic predictions.^{44, 45} The Brier score ranges from 0 to 1 where 0 would represent no difference between predicted probabilities and the actual outcome (i.e., perfect model calibration). We compared the reference logistic regression and XGBoost models using the precision/recall curve and overall calibration. The XGBoost model was evaluated on the testing dataset for its calibration separately across the twenty ambulatory divisions. The model hyper-parameters were tuned though a random grid search optimized for the maximum ROC AUC for the validation dataset. Utilizing SHapley Additive exPlanations (SHAP) methodology, we identified features that increase or reduce the probability of predicted activity.^{46, 47} The SHAP approach is an extension of local interpretable model-agnostic explanations (LIME) according to which feature weights are represented as Shapley values from game theory.^{46, 47}

We used decision curve analysis to assess and compare the clinical utility of the logistic regression and XGBoost prediction models along with two default intervention strategies: intervention for all patients, and intervention for no patients. A decision curve analysis differs from statistical measures of model performance like calibration and discrimination and aims to describe the effectiveness of intervention strategies across a range of risk threshold probabilities.^{48, 49} The risk threshold probabilities are plotted on the x-axis of the decision curve and are traditionally defined by a clinician’s preference to either intervene liberally and capture a large amount of true positives at the cost of unnecessary interventions or intervene very specifically and avoid unnecessary interventions. For example, at a risk threshold of 0.1% (1/1000), 1 true positive will be identified with at most 1000 interventions. In our application, some potential intervention strategies include text message, personalized call reminders, conversion to telehealth (particularly if transportation is an issue), or possible assistance with transportation via rideshare services. We considered a range of 0.1% (more concerned about intervening on non-arrivals than with unnecessary interventions) to 0.8% (more concerned about unnecessary interventions on false positives) based on the general consensus at our health system. The net benefit of an intervention strategy or model is plotted on the y-axis and represents the net proportion of true positives.⁴⁸ The higher the net benefit of an intervention strategy across the range of applicable risk thresholds, the more clinically useful the intervention strategy is. The formula for net benefit is as follows:⁵⁰

$$\text{Net}\text{benefit}=\left(\frac{\text{True}\text{positives}}{\text{Sample}\text{size}}\right)-\left[\left(\frac{\text{False}\text{positives}}{\text{Sample}\text{size}}\right),\times ,\left(\frac{\text{Threshold}\text{probability}}{1-\text{Threshold}\text{Probability}}\right)\right]$$

All analyses were performed using the R programming language, version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria). This study was approved by the Feinstein Institutes for Medical Research at Northwell Health’s Institutional Review Board.

RESULTS

There were over 4.3 million ambulatory appointments from almost 1.2 million adult patients over the study period (January 1, 2020, to February 28, 2022) (Fig. 1). Table 1 shows the characteristics between patients with arrived appointments and those with non-arrived appointments, and Table S1 demonstrates patient characteristics by model development cohort (i.e., train, validation, holdout).

The XGBoost model has a significantly higher AUC value compared to the other two models (XGBoost AUC 0.768 [0.767–0.770]; LR AUC 0.714 [0.713–0.716]; NW Tool AUC 0.541 [0.540–0.542]; p-value: < 0.001) (Fig. 2A). The XGBoost and logistic regression models were both well calibrated, as seen in Figure 2B and as evident by the range of Brier scores from 0.118 to 0.153 (Table S3). As seen in Figure 2B, the calibration of the XGBoost model is nearly ideal over the densest parts of the probability range (as noted by the density plot beneath the calibration curves). The XGBoost significantly outperforms the logistic regression model in terms of the precision-recall balance across the range of confidence thresholds, but the difference is most dramatic at higher thresholds (Fig. 2C). The XGBoost model had the highest performance across all metrics, including discrimination (AUC), calibration (Brier), and on the PR curve.

Fig. 2.

Comparison of discrimination, calibration, and precision-recall between non-arrival models. A Comparison of area under the ROC curve for the XGBoost (orange), logistic regression (dark blue), and NW Tool (green) models. B Calibration and density plot for the XGBoost and logistic regression models. C Precision-recall curves for the XGBoost and logistic regression models

Performance was maintained when performing a sensitivity analysis by randomly splitting the training, validation, and testing sets and ensuring that each visit for a given patient is included in a single set (AUC 0.783 [0.782–0.785]) (Fig. S1). When comparing performance across racial and ethnic groups, the XGBoost model’s performance stayed consistent except for increased performance on patients with unknown race or unknown ethnicity (Table S4), likely due to the increased non-arrival percentage for patients with unknown race or ethnicity.

Using SHAP values to measure the impact of each feature in the XGBoost model (Fig. 3), the most impactful features in the model include rescheduled (coded as 0/1), lead time (number of days from scheduled date to appointment date), and the primary provider for the appointment (target encoded categorical variable). The SHAP values show that appointments scheduled shortly before the appointment date are predicted to be less likely to result in a non-arrival.

Fig. 3.

SHAP values describing the feature importance in the XGBoost model. Importance of the top 10 features in predicting non-arrivals using the SHAP values. Each point represents an individual prediction and shows the impact on the non-arrival probability for high feature values (purple) and low feature values (yellow). The value in bold next to the feature name shows the SHAP value across the testing dataset

Lastly, we examined the prediction model calibration for different specialties and departments (Fig. 4; Table S3). Overall, the prediction model calibrated well for each department, especially over the operationally relevant probability range of 0 to 40%. Supplementary Table S3 presents the AUC and Brier scores of each department and shows consistent performance across ambulatory departments. The Brier scores generally increase slightly for departments that are less well represented in the dataset.

Fig. 4.

Model calibration for individual medical specialties. Calibration plots for each individual ambulatory department. Departments with lower representation in the test set generally have worse calibration than those with more representation

The decision curve analysis shows that the XGBoost model is better for guiding interventions on non-arrivals at the relevant threshold probabilities, or cost-benefit ratios, compared to the logistic regression model and the default intervention strategies (Fig. 5). At the 40% risk threshold, the logistic regression model dips below the 0 net benefit mark and shows a lack of clinical utility while the XGBoost model has a positive net benefit of about 0.17.

Fig. 5.

Decision curve analysis comparing the net benefit of four intervention strategies for non-arrivals across risk thresholds. The strategies include intervention for all, intervention for none, intervention based on output of a logistic regression model, and intervention based on output of an XGBoost model. The benefit of an intervention strategy is that it correctly identifies non-arrivals. The decision curves show that the XGBoost model has the highest net benefit across the range of relevant risk thresholds

DISCUSSION

Using data from a large integrated health system, we developed a prediction model for non-arrivals to ambulatory appointments. The proposed prediction model is improved by the application of a nonlinear machine learning algorithm, the use of data from a large integrated health system, incorporation of weather information, and inclusion of multiple specialties. We identified several patient and clinic characteristics associated with non-arrivals, including a history of non-arrivals and whether the appointment was rescheduled in the past. We also internally validated the prediction model with post-COVID-19 data. While the COVID-19 pandemic changed healthcare utilization,³⁹ the proposed non-arrival model performed similarly in predicting non-arrivals during this time.

There is a tradeoff of accuracy and generalizability when developing a non-arrival model for one department versus for all major medical specialties. Non-arrivals for surgical specialties may be different from those for general internal medicine clinics. Therefore, a prediction model that is highly discriminatory for surgical subspecialties may not perform well for general medicine clinics.⁵¹ For implementation and utilization of the prediction model in a practical use case, a slight decrease in accuracy may be less important than the ability to use the model for all specialties throughout a health system. We further incorporated a decision curve analysis in our modeling, which demonstrated an overall benefit of using the XGBoost predictive model across the entire reasonable range of thresholds, demonstrating the utility of this model as compared with the logistic regression model, the current health system tool, or no model at all.

Previous studies have shown that racial and ethnic minorities have increased risk of non-arrivals.^{17, 19, 51–54} Our descriptive analysis showed that a higher percentage of racial and ethnic minorities have non-arrivals compared to non-Hispanic Whites. However, in terms of variables that predict non-arrival to ambulatory appointments, the race and ethnicity variables were not in the top ten variables. It is likely that the patient history of having non-arrivals correlates with the race and ethnicity variables. We further ran sensitivity analysis and found that model performance was well maintained across different racial and ethnic groups. This is particularly notable and important as a non-arrival predictive tool can be used to facilitate a decrease in non-arrivals thereby achieving health equity in marginalized populations.

We observed variations in model calibration by departments and specialties. For example, the prediction model did not perform well for oncology. We hypothesize that this may be due to oncology patients having a continuous need to see a specialist regarding their medical conditions, with highly specialized and idiosyncratic clinical workflows. These patients with a history of malignancy need to follow up regularly to monitor cancer progression or remission and manage their malignancy with chemotherapy treatment.

We also identified patient and appointment characteristics associated with non-arrivals. Non-arrival prediction models usually focus on patient characteristics, but these are innate characteristics that may be harder to modify. In contrast, appointment characteristics, such as time of the day, day of the week, or lead time to the appointments, can be modified for patients with a high risk of non-arrival. Numerous interventions can reduce the chances of an appointment slot going unfilled, including personal call reminders for those at high non-arrival risk or double-booking a given time slot. Also, for practices that maintain a patient waitlist, appointment slots that are rescheduled because of a personal call reminder can be easily filled to maximize the practice’s revenue, patient’s continuity of care, and the provider’s time.

Limitations

Although the non-arrival model has been validated and tested within the health system and for the COVID-19 pandemic, we have yet to externally validate the data at another hospital or health system. We also do not have information on whether appointments were canceled by patients or by providers (provider being sick or clinic being canceled for unforeseen reasons). Another limitation arises from the COVID-19 pandemic. Although we tested the model using data from the post-COVID-19 period, it is unclear what the new norm regarding ambulatory care will eventually look like. With the rapid uptake of telehealth visits,^{38, 39} it is possible that high risk for non-arrival visits can be converted to telehealth visits, reducing overall non-arrival rates. The study is additionally limited by a lack of patients’ socioeconomic status, which may be a primary driver of the differences in non-arrival rates between racial/ethnicity groups.

CONCLUSION

We developed a prediction model for non-arrival to scheduled ambulatory appointments that could be used for all medical specialties. The proposed prediction model can be deployed within an electronic health system to enable interventions that reduce non-arrivals, optimize healthcare utilization, avoid revenue loss and provide significant cost savings to health systems, and, most importantly, improve patients’ health outcomes. Future work will focus on the implementation and application of the model to reduce non-arrivals.

Declarations

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of Northwell Health, Feinstein Institutes for Medical Research, and Donald and Barbara Zucker School of Medicine at Hofstra/Northwell.