Predicting outcomes of nonsurgical retreatment with artificial intelligence: A multicenter retrospective study using logistic regression, random forest, and extreme gradient boosting

Abstract

Background:

Nonsurgical root canal retreatment is a complex procedure influenced by multiple clinical and procedural factors, making accurate prediction of healing outcomes difficult. Machine learning offers a potential approach to enhance prognostic precision and guide clinical decisions in endodontic retreatment.

Materials and Methods:

A retrospective multicenter dataset of 1100 patients treated between 2015 and 2023 was analyzed. Demographic, clinical, and treatment-related variables were collected. Logistic regression (LR), random forest (RF), and Extreme Gradient Boosting (XGBoost) models were developed using a 70:30 training–testing split with cross-validation. Model performance was assessed using accuracy, sensitivity, specificity, area under the curve-receiver operating characteristic (AUC-ROC), and Brier scores. Predictor importance was evaluated using Shapley Additive Explanations.

Results:

The overall healing rate of apical periodontitis was 82%. LR achieved moderate discrimination (AUC-ROC = 0.81), RF showed better performance (0.88), and XGBoost achieved the highest accuracy (0.91; Brier score = 0.12). Negative predictors included preoperative periapical lesions, molar teeth, poor restorations, posts/cores, and multiple prior treatments. Favorable healing was associated with the absence of lesions and good coronal restorations.

Conclusions:

Among the tested models, XGBoost demonstrated the highest prognostic accuracy for predicting healing following non-surgical root canal retreatment. Its strong predictive capability supports its potential application in clinical decision-making, warranting further prospective validation.

INTRODUCTION

Non-surgical root canal retreatment of previously treated teeth presents significant clinical challenges and remains a cornerstone of endodontic practice worldwide. These procedures are often technically demanding due to complex canal morphology, residual microbial infection, and restorative complications, requiring considerable expertise to achieve predictable success.[1,2] Despite improvements in endodontic techniques, instruments, and materials, the healing rates of apical periodontitis after retreatment remain variable. This inconsistency underscores the importance of developing reliable prediction models to guide clinicians in identifying high-risk cases and customizing treatment plans for optimal outcomes.[3]

Artificial intelligence (AI), encompassing machine learning (ML) and deep learning algorithms, has rapidly transformed several areas of healthcare, including dentistry. Over the past decade, these computational models have been increasingly used for diagnostic, predictive, and decision-support applications.[4,5] Their ability to analyze large, complex datasets and identify intricate, nonlinear relationships between variables allows for improved clinical predictions. In endodontics, AI has been used for applications such as diagnosis of periapical lesions, determination of working length, and detection of anatomical variations on radiographs and cone-beam computed tomography (CBCT) scans.[6]

For instance, convolutional neural networks have been shown to outperform human clinicians in identifying anatomical landmarks such as the apical foramen for working length determination, achieving accuracy levels between 93% and 96%.[7] AI algorithms have also demonstrated superior accuracy and efficiency in analyzing CBCT images for root fracture detection and segmentation of root canal systems compared to manual interpretation.[8] These advancements highlight AI’s potential to improve both diagnostic precision and procedural planning in endodontics.

However, limited research has explored AI applications in predicting treatment outcomes, particularly in non-surgical retreatment cases. Some early efforts, such as case-based reasoning systems, have achieved moderate accuracy (~84%) in predicting retreatment decisions.[9] Nevertheless, most prior studies have concentrated on primary endodontic treatments or imaging-based diagnostics, leaving a gap in evidence regarding structured, data-driven AI models designed to predict retreatment success.[10]

Given this gap, the present study aimed to evaluate the performance of three predictive models in forecasting non-surgical root canal retreatment outcomes. By integrating AI with real-world clinical data, this research aims to contribute toward personalized endodontic care and enhance clinician decision-making capabilities.

MATERIALS AND METHODS

Study design and setting

This retrospective, observational study utilized de-identified patient records from two dental colleges (Saveetha dental college and hospitals and mamata institute of dental sciences.) and one private hospital (The dental office by A and T) located in (India). Data were retrieved from electronic case notes and digital treatment histories of patients who underwent non-surgical root canal retreatment between January 2015 and December 2023. Ethical approval was obtained from the Institutional Review Board (SRB/SDC/ENDO-2302/25/194), and the study adhered to the Declaration of Helsinki and STROBE guidelines for observational research. The Retreatment outcomes (healing vs. nonhealing) were determined using European Society of Endodontology criteria.[11]

Sample size calculation

Using an events-per-variable ratio of 10, with 20 predictors and an estimated non-healing rate of 20%, a minimum of 1000 patients was required. Accounting for incomplete data (10% attrition), a final target sample of 1100 patients was established Assuming the expected retreatment success rate from previous literature.[12,13]

Study population

Eligible participants included adult patients (≥18 years) who had previously undergone conventional root canal therapy followed by nonsurgical retreatment. Inclusion criteria required the availability of complete electronic case notes, radiographic data, and follow-up records for a minimum of 12 months. Patients with incomplete records, extractions before follow-up, or systemic conditions impairing healing (e.g., uncontrolled diabetes, immunosuppression) were excluded.

To minimize attrition bias, only cases with complete outcome documentation were included. Teeth replaced by implants or subjected to surgical interventions (e.g., apicoectomy, hemisection) during follow-up were excluded to maintain uniformity in outcome assessment.

Data collection and variables

Data were systematically extracted using a structured proforma, including demographic variables (age, sex), tooth location, preoperative diagnosis, periapical status, previous treatment type, presence of posts/cores, coronal restoration quality, and retreatment techniques.

Outcome evaluation was independently performed by two calibrated examiners. A calibration exercise using 20% of randomly selected radiographs demonstrated high inter-examiner agreement (Cohen’s kappa = 0.84). Intraexaminer reliability was also strong (κ = 0.87 and 0.85). Final outcome classification was achieved through consensus review.

Data preprocessing and feature selection

All datasets were anonymized before analysis. Missing data were managed using multiple imputation with chained equations over ten iterations. Categorical variables were one-hot encoded, and continuous variables were normalized. Feature selection was performed using recursive feature elimination and Random Forest (RF)–based importance ranking. Only predictors retained after feature selection were included in model development.

Artificial intelligence model development

Three models were developed and compared: Logistic Regression (LR), RF, and Extreme Gradient Boosting (XGBoost).

Logistic regression

Used as the baseline model, LR estimated binary outcomes using a logit function. It provided interpretable coefficients (odds ratios [ORs]) to identify prognostic variables.

Random forest

An ensemble approach combining multiple decision trees trained on bootstrapped samples. It handles high-dimensional, heterogeneous data and reduces overfitting through aggregation.

Extreme gradient boosting

An optimized boosting algorithm that builds trees sequentially, correcting prior errors. It includes regularization terms to minimize overfitting and has demonstrated superior performance in structured healthcare datasets.

Nested cross-validation with a 10-fold outer loop and randomized hyperparameter tuning was employed. Models were trained on 70% of data and tested on the remaining 30%. Performance metrics such as: accuracy, sensitivity, specificity, area under the curve-receiver operating characteristic (AUC-ROC), and the Brier score were averaged across folds.

Statistical analysis

Descriptive statistics summarized demographics and clinical features. Model comparisons used the DeLong test for AUC differences. Confidence intervals were derived using 1,000 bootstrap replicates. Calibration was assessed with the Hosmer–Lemeshow test and Brier score. Shapley Additive Explanations (SHAP) analysis was employed to rank predictor contributions. Subgroup analyses by tooth type assessed discrimination consistency. Analyses were performed using Python (scikit-learn v1.3, SHAP) and R (caret package).

RESULTS

Inter- and intraexaminer reliability were high (Cohen’s kappa = 0.84). The dataset included 1100 patients, with an overall healing rate of 82% for apical periodontitis. The nonhealing rate was 18.2%, aligning with existing clinical evidence.

Model performance

• LR: AUC-ROC = 0.81 (moderate discrimination)

• RF: AUC-ROC = 0.88 (improved accuracy)

• XGBoost: AUC-ROC = 0.91; Brier score = 0.12 (highest predictive performance).

Predictors of nonhealing

Preoperative periapical lesions (OR = 2.8), molar teeth (OR = 2.1), presence of post/core (OR = 1.9), poor coronal restorations (OR = 1.7), and multiple previous treatments (OR = 1.5). Favourable healing was linked with the absence of lesions and well-sealed coronal restorations [Tables 1-5].

Table 1.

Demographic characteristics

Variable	Value
Age (years), mean±SD	41.3±12.6
Gender
Male	484
Female	616
Follow-up duration (months), mean±SD	18.2±6.5

SD: Standard deviation

Table 5.

Calibration performance of predictive models

Model	Hosmer-Lemeshow χ2 (df=8)	P	Brier score	Calibration intercept	Calibration slope
LR	7.4	0.49	0.19	−0.03	0.98
RF	6.1	0.64	0.15	−0.01	1.01
XGBoost	5.3	0.72	0.12	0.00	1.02

Lower brier score indicates better calibration. Ideal calibration is reflected by an intercept near 0 and slope near 1. LR: Logistic regression, RF: Random forest, XGBoost: Extreme gradient boosting

Table 2.

Distribution of tooth types and retreatment outcomes

Tooth type	Total cases	Success % (95% CI)	Failure % (95% CI)
Anterior	280	85.7 (81.3–89.2)	14.3 (10.8–18.7)
Premolar	310	83.9 (79.6–87.4)	16.1 (12.6–20.4)
Molar	510	78.4 (74.4–81.9)	21.6 (18.1–25.6)
Total	1100	81.8 (79.5–84.0)	18.2 (16.0–20.5)

CI: Confidence interval

Table 3.

Clinical predictors of retreatment success

Predictor	OR (95% CI)	P
Preoperative periapical lesion	2.8 (2.1–3.6)	<0.001
Tooth type (molar vs. others)	2.1 (1.6–2.7)	<0.001
Post/core present	1.9 (1.3–2.5)	0.002
Poor coronal restoration quality	1.7 (1.2–2.3)	0.004
Multiple previous treatments	1.5 (1.1–2.0)	0.01

CI: Confidence interval, OR: Odds ratio

Table 4.

Model performance metrics

Model	Accuracy (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	AUC-ROC (95% CI)	Brier score
LR	0.78 (0.75–0.81)	0.72 (0.68–0.76)	0.80 (0.76–0.83)	0.81 (0.78–0.84)	0.19
RF	0.85 (0.83–0.88)	0.83 (0.79–0.87)	0.86 (0.83–0.89)	0.88 (0.85–0.91)	0.15
XGBoost	0.88 (0.85–0.90)	0.87 (0.83–0.90)	0.89 (0.86–0.91)	0.91 (0.89–0.94)	0.12

CI: Confidence interval, ROC: Receiver operating characteristic, AUC: Area under curve, LR: Logistic regression, RF: Random forest, XGBoost: Extreme gradient boosting

Subgroup analyses confirmed consistent model performance across tooth types, with slightly lower accuracy in molars due to greater anatomical variability [Figure 1].

Figure 1.

(a) Receiver operating characteristic (ROC) curves. Illustrates the ROC curves for logistic regression (LR), Random forest (RF), and Extreme Gradient Boosting (XGBoost) models in predicting root canal retreatment outcomes. LR achieved an area under the curve (AUC) of 0.81, indicating good discriminative ability but lower performance compared to ensemble approaches. RF demonstrated stronger predictive capacity with an AUC of 0.88, reflecting its ability to capture nonlinear interactions among clinical variables. XGBoost outperformed both models with an AUC of 0.91, showing excellent discrimination between successful and failed retreatments. The curve for XGBoost lies furthest toward the upper left corner of the plot, signifying superior sensitivity and specificity trade-offs across thresholds. (b) Confusion matrices for model prediction. The confusion matrices for each model. LR exhibited higher rates of false positives and false negatives compared to ensemble methods, misclassifying both successful and failed retreatments. RF reduced misclassification, particularly improving true positive identification of successes. XGBoost performed best overall, yielding the lowest number of false negatives and false positives, which is consistent with its relatively higher sensitivity and specificity. (c) Forest plot of clinical predictors. Presents a forest plot of the main clinical predictors of retreatment success derived from multivariable analysis. Preoperative periapical status had the highest odds ratio (OR) (~2.8), indicating a strong association with retreatment failure. Tooth type, particularly molars, also carried elevated risk (OR = 2.1), underscoring anatomical complexity as a prognostic factor. Other significant predictors included presence of post/core, poor quality coronal restorations, and multiple previous treatments, each conferring incremental risk. The confidence intervals remained narrow and statistically significant, reinforcing the robustness of these predictors

DISCUSSION

This study demonstrated that AI-based predictive models, particularly XGBoost, can effectively forecast healing outcomes following non-surgical root canal retreatment. The XGBoost model achieved superior discrimination (AUC-ROC = 0.91) and calibration (Brier score = 0.12), outperforming both LR and RF.

These results align with the growing body of evidence supporting AI’s predictive capacity in endodontics. Lai et al.[14] reported that AI-guided models improved diagnostic precision and treatment planning. Similarly, Lu et al.[15] observed that ensemble models such as RF and Gradient Boosting Machines achieved AUCs above 0.93 in predicting regenerative endodontic outcomes. The present findings also agree with Karobari et al.,[16] who emphasized ML’s ability to manage complex canal morphologies more effectively than traditional methods. Recent literature[17] has also emphasised the importance of AI in accurately assessing canal anatomy compared with conventional methods.

Our observed healing rate (82%) aligns with contemporary literature. Sabeti et al.[12] reported success rates between 78% and 87%, while Bhatt et al.[18] documented 88% healing using modern ultrasonic irrigation protocols. Differences between studies may be attributable to variations in case selection, operator skill, and follow-up duration. Ponsri and Jirathanyanatt[19] found lower healing rates (65.7%–77.1%) over longer follow-ups, emphasizing the influence of time and case complexity.

Consistent with prior evidence,[20] preoperative periapical lesions were the strongest predictor of non-healing. Ponsri and Jirathanyanatt[19] and Kim[21] similarly reported markedly lower success in cases with existing lesions. The impact of molar tooth type, also significant here, reflects anatomical complexity and greater procedural difficulty. Poor restoration quality and post/core placement were additional negative predictors, echoing the findings of Gulabivala and Ng et al.,[22] who linked coronal integrity to long-term tooth survival.

An important strength of this study lies in its multicentre design and large dataset, which enhance generalizability. The integration of structured clinical data with AI-driven analysis[23,24] provides practical interpretability, particularly through SHAP, which explains each variable’s contribution to model predictions. This mitigates the black box issue of AI and enables clinicians to understand which clinical factors most influence prognosis.

However, limitations exist. Being retrospective, the study is subject to documentation and selection biases. Although missing data were handled using multiple imputation, residual confounding cannot be excluded. Follow-up averaged 18.2 months, adequate but shorter than long-term survival studies. Variables such as microbial species, solvent use, and operator experience were not included. Moreover, as data were drawn from specific institutions, external validation in broader clinical contexts remains necessary.

Future studies should incorporate prospective, multicentre designs and longer follow-ups to validate these findings. Integrating imaging data with structured datasets could enhance predictive accuracy. Furthermore, developing AI tools that interface directly with electronic dental records may facilitate real-time risk assessment and personalized treatment planning.

CONCLUSIONS

This multicentre retrospective study evaluated three AI-based models for predicting healing after non-surgical root canal retreatment. XGBoost demonstrated the highest accuracy, sensitivity, specificity, and calibration among the models tested. Key predictors of outcome included preoperative periapical lesions, molar tooth type, post/core presence, coronal restoration quality, and number of prior treatments. The interpretability achieved through SHAP analysis supports AI’s clinical applicability, offering transparency in decision-making.

Conflicts of interest

There are no conflicts of interest.

Funding Statement

Nil.