Using Machine Learning to Predict Adherence to Recommended Imaging Follow-Up

Completion of recommended imaging follow-up for abnormalities representing possible malignancy—detected at screening or during imaging performed for unrelated reasons—facilitates timely diagnosis in cases ultimately proven to be malignant. The timely completion of recommended follow-up is impacted by multilevel factors, including patient factors, social context, and healthcare delivery factors. Research efforts using robust analytic methods to identify the factors at each level that drive adherence to imaging recommendations may allow radiologists to better identify patients at greatest risk of not receiving recommended imaging follow-up. Prospective identification of such patients could ultimately lead to tailored interventions to increase timely imaging follow-up, reduce harm caused by delayed diagnosis, and improve cancer outcomes.

In this issue, Wang and Mortani Barbosa demonstrate the promise of incorporating sociodemographic and clinical factors in machine learning models to predict timely completion of recommended imaging follow-up [1]. The authors identified 1610 patients with a recommendation for follow-up imaging of incidental pulmonary nodules detected on chest CTs performed in 2016 at a large tertiary healthcare system. Several baseline patient characteristics—including race and ethnicity, health insurance status, employment status, and smoking status—along with neighborhood-level variables reflecting socioeconomic status, and factors such as care setting and nodule risk level were obtained from the electronic medical record. These multilevel factors were assessed for their association with completion of follow-up imaging in univariate and multivariate analyses. The authors then assessed and compared how well these factors predicted the completion of imaging follow-up using four different machine learning approaches.

In the multivariable logistic regression, clinical context was the strongest predictor of completing recommended imaging follow up. Patients with chest CTs obtained in the emergency setting had more than 8 times higher odds of not completing recommended imaging follow-up compared to patients obtaining CTs in the outpatient setting (OR [95% CI]: 8.56 [6.11, 12]). Nodule risk level was an additional clinical factor significantly associated with not completing recommended imaging. Patients with nodules described as high-risk had significantly lower odds of not completing recommended imaging follow-up compared to patients with low-risk nodules (OR [95% CI]: 0.25 [0.17, 0.37]). Patients with Medicaid insurance had nearly 2 times higher odds of not completing recommended imaging follow-up compared to patients insured by Medicare (OR [95% CI]: 1.99 [1.28, 3.1]).

Each of the 4 machine learning models predicting completion of recommended imaging follow-up achieved a mean testing area under the receiver operating characteristic curve (AUC) of > 0.75, with the multiple logistic regression model demonstrating the highest mean testing AUC of 0.802. Although this study was limited to a single institution and includes a relatively small sample of patients, the performance of machine learning models in predicting appropriate imaging follow-up highlights a potential first step toward for improving adherence to recommended imaging follow-up. If machine learning can reliably identify patients at greatest risk of delayed or incomplete care, radiology practices can better target these individuals with interventions designed to overcome barriers (e.g., patient navigators for identified individuals).

Future studies in larger cohorts are needed to optimize model performance and to increase generalizability across diverse populations and practices and address a wide range of clinical scenarios within radiology. While Wang and Mortani Barbosa focus on the follow-up of incidental pulmonary nodules, machine learning models could also play a key role in identifying populations that would benefit from opportunistic screening and timely completion of recommended additional diagnostic imaging for abnormalities. Just as Wang and Mortani Barbosa identified factors associated with timely follow-up of incidental pulmonary nodules, many multilevel factors are associated with imaging follow-up after abnormal mammography, abdominal CT, and neuroradiology CT/MR studies [2, 3] . Development of generalizable multilevel and multimodal machine learning prediction algorithms that aim to identify patients at risk of delayed follow-up in each of these contexts would be a natural extension of the work performed by Wang and Mortani Barbosa.

The potential of machine learning models incorporating socioeconomic indicators and clinical risk factors with imaging findings is not limited to optimizing care for just the patients with abnormalities. The success of breast and lung cancer screening programs, for instance, relies on patients returning for routine screening even after preceding examinations have been normal. However, many patients do not return for screening at regular intervals [4, 5], limiting the ability to detect cancers early when they are most amenable to treatment. Robust methods to identify patients who are less likely to adhere to regular screening intervals after normal screening results may allow for earlier identification of these patients and give radiologists an opportunity to meet targets set by the U.S. Department of Health and Human Services in Healthy People 2030 and improve population cancer outcomes [6].

While more work is required before machine learning models that incorporate both clinical factors and imaging findings will allow radiologists to routinely identify patient populations at risk of having delayed imaging follow-up, Wang and Mortani Barbosa’s study gives insight into a potential path towards this ultimate goal. Importantly, their work highlights that both clinical factors and social determinants of health will likely be key in any successful imaging exam-driven algorithm aimed at identifying patients at risk of suboptimal care delivery.