Prediction of medication-related osteonecrosis of the jaw (MRONJ) using automated machine learning in patients with osteoporosis associated with dental extraction and implantation: a retrospective study

Abstract

Objectives

This study aimed to develop and validate machine learning (ML) models using H2O-AutoML, an automated ML program, for predicting medication-related osteonecrosis of the jaw (MRONJ) in patients with osteoporosis undergoing tooth extraction or implantation.

Patients and Methods

We conducted a retrospective chart review of 340 patients who visited Dankook University Dental Hospital between January 2019 and June 2022 who met the following inclusion criteria female, age ≥55 years, osteoporosis treated with antiresorptive therapy, and recent dental extraction or implantation. We considered medication administration and duration, demographics, and systemic factors (age and medical history). Local factors, such as surgical method, number of operated teeth, and operation area, were also included. Six algorithms were used to generate the MRONJ prediction model.

Results

Gradient boosting demonstrated the best diagnostic accuracy, with an area under the receiver operating characteristic curve (AUC) of 0.8283. Validation with the test dataset yielded a stable AUC of 0.7526. Variable importance analysis identified duration of medication as the most important variable, followed by age, number of teeth operated, and operation site.

Conclusion

ML models can help predict MRONJ occurrence in patients with osteoporosis undergoing tooth extraction or implantation based on questionnaire data acquired at the first visit.

I. Introduction

Recently, the medical field has begun developing machine learning (ML) models that can aid doctors in their decision-making regarding diagnosis, prognosis, and management of multifactorial diseases^1-3. The dentistry field is using ML to improve patient care^1,3-5. For example, a deep-learning-based convolutional neural network algorithm was used to diagnose dental caries in periapical radiographs⁴, and neural network ML considering several cephalometric variables was used to diagnose the need for extractions in the field of orthodontics⁵. Additionally, various studies using artificial intelligence (AI) have been published in the fields of oncology and periodontics³.

As a subtype of AI, ML extrapolates results based on data and algorithms. The algorithms are trained using data with previously determined input-output relationships to infer output data with high predictability for new input data^6,7. As the demand for ML increases, automated ML programs that can be used effectively by non-experts are available for free or a nominal fee (e.g., DataRobot, H2O-AutoML, H2O-Driverless AI product, Darwin, Google Cloud AutoML, TPOT)⁸.

Medication-related osteonecrosis of the jaw (MRONJ) is a multifactorial disease. Although medication is one of the main risk factors, local, demographic, and systemic factors also affect its occurrence⁹. Medications that can influence MRONJ occurrence include antiresorptive therapies such as bisphosphonates (BPs), denosumab (DMB), and romosozumab, which are administered orally or parenterally to manage osteoporosis, cancer-related bone metastases, and metabolic bone diseases. MRONJ risk in patients with osteoporosis ranges from 0.02% to 0.3%, which is not higher than that in patients taking drugs for other diseases⁹. However, MRONJ can reduce quality of life, and it is even life-threatening in some cases¹⁰; hence, early diagnosis and treatment are critical. It is important to not only accurately diagnose MRONJ in a timely manner, but also to predict potential occurrence and provide appropriate follow-up during the perioperative surgical period.

The aim of this study was to develop and validate ML models that can predict MRONJ occurrence with information obtained from a questionnaire. We used an automated ML program, H2O-AutoML, to develop models for patients with osteoporosis who had undergone tooth extraction or implantation surgery.

II. Patients and Methods

1. Patients

This study included female patients aged 55 years or older who visited the Department of Oral and Maxillofacial Surgery at Dankook University Dental Hospital between January 2019 and June 2022. Medical and dental history surveys were conducted using a questionnaire presented in Table 1, with a focus on the duration and administration of antiresorptive medications for patients with osteoporosis. The inclusion criteria were as follows: female, age ≥55 years, osteoporosis currently or previously treated with antiresorptive therapy (e.g., BPs or DMB), and recent dental extraction or implantation (e.g., patients who were referred after surgery at the local center and patients who had undergone surgery at this hospital). The exclusion criteria were radiotherapy history for head and neck cancer and incomplete medical and dental history records. A total of 340 patients met the inclusion criteria and was included in the study. We considered medication, demographic, systemic, and dental factors. Medication factors included duration and administration of antiresorptive medications (oral [PO], intravenous [IV], or both). Demographic and systemic factors included age and history of hypertension, hyperlipidemia, diabetes mellitus, heart disease, thyroid disease, kidney disease, liver disease, rheumatoid arthritis, cancer, or dementia. Additionally, any history of anticoagulant medication was considered. We also included local factors, such as the method of operation (extraction or implantation), number of operated teeth, and operation area (maxilla, mandible, or both).

Table 1.

Questionnaire provided to patients

Questionnaire items	Yes	No	Comment
1. Have you ever been hospitalized or undergone surgery?			Site:
2. Are you currently being treated by a doctor?
3. What kind of medications are you currently taking?			Drug:
4. Have you ever had side effects from injections or medicines?
5. Are you bleeding excessively or are you taking anticoagulants?
6. Have you ever had tuberculosis or a sexually transmitted infection?
7. Do you have hypertension?			Blood pressure: /
8. Do you have hepatitis or jaundice?
9. Do you have heart disease?
10. Do you have kidney disease?
11. Do you have diabetes mellitus?			Blood sugar test
12. Are you taking osteoporosis medications?			Intravenous/ oral/ both Duration of medication:
13. Do you have trouble breathing or have stomach problems?
14. Are you pregnant?
15. Do you have dementia?
16. Do you have thyroid disease?

MRONJ occurrence was evaluated during post-surgical follow-up after at least six months and was diagnosed according to criteria defined in the position paper of the American Association of Oral and Maxillofacial Surgeons⁹:

(1) Current or previous treatment with antiresorptive therapy alone or in combination with immune modulators or antiangiogenic medications.

(2) Exposed bone or bone that can be probed through an intraoral or extraoral fistula(e) in the maxillofacial region that has persisted for more than eight weeks.

(3) No history of radiation therapy to the jaws or metastatic disease to the jaws.

Because this study aimed to predict only MRONJ occurrence, the staging system (stages 0-3), which is helpful for doctors in determining a patient’s prognosis and outcome, was not considered. Instead, the output value was dichotomized as presence or absence of MRONJ.

Participants were divided into a control group (n=210) and MRONJ group (n=130). A comparison of the parameters between the control and MRONJ groups is presented in Table 2.

Table 2.

Parameter comparison

	MRONJ		P-value
	No (control group, n=210)	Yes (MRONJ group, n=130)
Age (yr)	73.1±9.0	77.2±7.1	<0.0011
Medication administration
Oral	82 (39.0)	70 (53.8)	<0.0012
Intravenous	98 (46.7)	29 (22.3)
Both	30 (14.3)	31 (23.8)
Duration of medication (yr)	3.4±4.6	5.8±5.1	<0.0011
Medical history
Hypertension	104 (49.5)	85 (65.4)	0.0042
Hyperlipidemia	47 (22.4)	27 (20.8)	0.726
Anticoagulant	29 (13.8)	14 (10.8)	0.412
Diabetes mellitus	41 (19.5)	29 (22.3)	0.552
Heart disease	23 (11.0)	24 (18.5)	0.051
Thyroid disease	10 (4.8)	7 (5.4)	0.798
Liver disease	4 (1.9)	0 (0)	0.113
Kidney disease	10 (4.8)	4 (3.1)	0.447
Rheumatoid arthritis	4 (1.9)	7 (5.4)	0.078
Cancer	10 (4.8)	12 (9.2)	0.104
Dementia	11 (5.2)	2 (1.5)	0.084
Method of operation
Extraction	180 (85.7)	99 (76.2)	0.0262
Implantation	30 (14.3)	31 (23.8)
No. of operated teeth	2.3±1.6	1.7±1.0	<0.0011
Operation area
Maxilla	93 (44.3)	35 (26.9)	<0.0012
Mandible	87 (41.4)	90 (69.2)
Both	30 (14.3)	5 (3.8)

(MRONJ: medication-related osteonecrosis of the jaw)

Statistical analyses by ¹Student’s t-test and ²chi-square test.

Values are presented as mean±standard deviations or number (%).

2. Automated ML

The H2O-AutoML program was used to perform hyperparameter optimization and train the prediction models². The data were randomly divided into a training set (71.5%, n=243) and a testing set (28.5%, n=97). The training set was used to generate the prediction model, which was estimated through fivefold cross-validation. The testing set was used to estimate the accuracy of the models. The flow diagram of the proposed work is shown in Fig. 1.

Fig. 1.

Flow diagram of the proposed work.

ML can be classified into two categories: supervised and non-supervised. Supervised learning uses known labeled datasets to train an algorithm. Non-supervised learning uses unlabeled datasets¹¹. Among these, supervised learning, particularly classification, is suitable for disease prediction because it involves discrete output variables. Thus, many studies have used this method^1-3,11,12. We used the six algorithms of the generalized linear model (GLM), distributed random forest (DRF), gradient boosting machine (GBM), stacked ensemble, extreme gradient boosting (XGBoost), and deep learning to generate our prediction model.

The GLM is a regression model that estimates outcomes following exponential distributions. Although this model is easy to use and interpret, it cannot predict complex problems¹³. In contrast to using a single model like GLM or decision tree (DT) algorithms, the ensemble method uses numerous models to increase forecast accuracy¹². The three combination strategies of the ensemble method are bagging, boosting, and stacking.

Bagging creates multiple classifications, trains them with different training data (bootstrapping), and then votes on different predictions for the same test data (aggregating)^6,14. In this study, we used the DRF method to bag the DT algorithm.

While bagging learns several classifiers based on different algorithms in parallel, boosting trains classifiers of the same algorithm and learns from the errors of previous iterations by increasing the importance of incorrectly predicted training instances in future iterations⁶. GBM and XGBoost are examples of such boosting ensembles.

Stacking involves two phases. The first phase trains the dataset base using different algorithms, and the output of this phase is used as a new dataset for the second phase, called meta-learning, which provides the final output^6,14.

Deep learning is a subtype of ML that utilizes a representation-learning method to enable a machine to find the necessary representations required for detection or classification from raw, abstract data such as pictures. This process is not designed by human engineers but learns itself from data using a common process¹⁵.

3. Statistical analyses

Statistical analyses were conducted on continuous and categorical variables between the control and MRONJ groups using Student’s t-test and the chi-square test, respectively, as appropriate, following the Kolmogorov–Smirnov test. P-values <0.05 were considered statistically significant. The area under the receiver operating characteristic (ROC) curves (AUC) and F1 score were used for comparison between ML models. Statistical analyses were performed using IBM SPSS Statistics (ver. 29.0.0.0; IBM).

III. Results

Of the 29 models developed for predicting MRONJ, 20, including 14 GBM, 4 stacked ensemble models, 1 DRF model, and 1 XGBoost model, showed very good diagnostic accuracy with an AUC ≥0.8. The GBM model had the highest AUC value. The training and test dataset metrics for the model are presented in Tables 3 and 4, respectively. This model comprised 42 trees with a maximum depth of 6 and showed an AUC of 0.8283 on the training dataset.(Fig. 2. A) At the maximum F1 threshold, the sensitivity and specificity were 83.7% and 71.3%, respectively. For the validation dataset, the model showed a stable AUC of 0.7526.(Fig. 2. B) At the maximum F1 threshold, the sensitivity and specificity were 88.6% and 52.8%, respectively.

Table 3.

Training dataset metrics

MRONJ	Predicted
	Negative	Positive	Total
Actual
Negative	112	45	157
Positive	14	72	86
Total	126	117	243

(MRONJ: medication-related osteonecrosis of the jaw)

Table 4.

Test dataset metrics

MRONJ	Predicted
	Negative	Positive	Total
Actual
Negative	28	25	53
Positive	5	39	44
Total	33	64	97

(MRONJ: medication-related osteonecrosis of the jaw)

Fig. 2.

Area under the receiver operating characteristic curves (AUCs) of best performed gradient boosting machine model. A. Training dataset (AUC=0.8283). B. Test dataset (AUC=0.7526).

Variable importance analysis, shown in Fig. 3, revealed that the duration of medication was the most important variable, followed by age, number of operated teeth, operation area, hypertension, and administration of medication.

Fig. 3.

Variable importance.

IV. Discussion

Korea has a rapidly aging population; thus, a large proportion of the population has potential for osteoporosis¹⁶. BPs are frequently prescribed for female patients diagnosed with osteoporosis after menopause because they are non-invasive and easy to take orally. However, with age, the need for tooth extraction and implantation increases. Unfortunately, life-threatening MRONJ could occur after a seemingly simple dental treatment, and it cannot be easily predicted due to limited available information. Although biomarkers such as the C-terminal telopeptide of type-I collagen obtained through blood tests have been used in MRONJ clinical decision-making, their use remains controversial. While a drug holiday can be considered to reduce MRONJ risk in patients undergoing tooth extraction or implant surgery, it is not always feasible due to potential side effects such as skeletal-related events, fragility fractures, increased rebound bone resorption, or in emergency surgery situations. Recommendations for the duration of a drug holiday also remain controversial⁹. Recently, parathyroid hormone therapy, which stimulates new bone formation by osteoblasts, has been highlighted as an alternative drug. However, its use is limited to 24-month regimens due to potential side effects, and the subcutaneous daily injections required could be challenging for elderly patients¹⁷.

This study demonstrates the effectiveness of ML models for predicting MRONJ occurrence using only questionnaire-based information in patients with osteoporosis who have undergone tooth extractions or implant surgery. We objectively and consistently collected information from patients using BPs who visited our hospital in 2017 using the same questionnaire, which allowed for consistent and objective data collection. One advantage of our models is that they were developed based only on the questionnaire at the first visit, without the need for imaging or laboratory tests. These features enable surgeons to make clinical decisions and educate patients with more accurate information.

Most models in this study showed good diagnostic accuracy, with an AUC ≥0.8. For the validation dataset, the model showed an AUC of 0.7526, which is slightly lower but still a good predictive value. Additionally, sensitivity increased from 83.7% to 88.6%, and specificity decreased from 71.3% to 52.8%. With reference to our model, sensitivity can be seen as a more important indicator than specificity in that our model is intended to predict the possibility of MRONJ occurrence for educating patients and for follow-up.

We also compared variable importance.(Fig. 3) The most important variable was duration of medication use, followed by age. In this study, the mean ages of the control and MRONJ groups were 73.1 and 77.2 years, respectively, and the mean durations of medication use were 3.4 and 5.8 years. Both variables were significantly higher in the MRONJ group than in the control group. Many studies have also found significant increases in incidence rates, from 0.04% to 0.21%, in patients with osteoporosis who had been using BPs for more than four years^9,18,19. As osteoporosis in women generally occurs after menopause, age and duration of medication are closely related.

The second most significant variable was the number of teeth that underwent operation. Some studies have found that the MRONJ incidence rate was significantly higher in cases of multiple-tooth extractions than in single-tooth extractions²⁰. However, our analysis showed that the average number of teeth extracted in the MRONJ group was lower than that in the control group, which contradicts previously reported results. In response to these results, other studies have reported that tooth extraction does not act as a local factor alone, but it is associated with the presence or absence of inflammatory dental disease^9,19,20. Therefore, it is necessary to also consider the presence or absence of periodontal disease.

Several studies have reported on the surgical area associated with MRONJ, showing that MRONJ prevalence is two to three times higher in the mandible than in the maxilla^9,19,20. The results of our study are consistent with previous studies, in that we found 27% and 69% prevalence in the maxilla and mandible, respectively.

The most important variable in patient medical history was hypertension. Despite several studies showing that hypertension, diabetes, and anemia significantly enhance the risk of MRONJ incidence^9,21,22, in this study, hypertension was the most significant variable. Although the risk of overfitting in this algorithm cannot be completely ruled out because the variable distribution is not uniform, the association between MRONJ and hypertension has not been well studied, and further research is needed.

This study has several limitations. First, we collected the data from a single institution, and to include a large number of MRONJ cases (n=130, 38.2%) compared with the low incidence, we excluded drug holiday or steroid therapy from variables, which are also correlated with increased MRONJ risk⁹. Nevertheless, none of the variables showed a uniform distribution between the MRONJ and control groups. This may have caused overfitting or underfitting of the ML models. Second, the model was designed using only medical records based on patient questionnaires. Nonetheless, it is meaningful in that it created a model with a high AUC value. However, the accuracy of the model can be improved by considering variables from other records, including objective laboratory, genetics, or imaging data³. Finally, because the ML algorithm infers results by considering numerous factors at once, it is impossible to produce a single exact value for each variable.

A predictive model may help clinicians explain the possibility of MRONJ in greater depth to patients and help them actively cope with future complications before tooth extraction or implantation. Moreover, in the future, AI-based programs may support high-quality patient care and facilitate advanced decision tools. In future research, we anticipate developing a computerized system that can collect, computerize, and share big data from multiple institutions, including radiograph and laboratory data, in addition to medical history data.

V. Conclusion

We demonstrated that ML models can be used to predict the likelihood of MRONJ occurrence in patients with osteoporosis who have undergone tooth extraction or implantation surgery using information gathered from a questionnaire at their initial visit. Our findings suggest that the duration of medication use is the most significant variable associated with MRONJ risk, followed by age, number of operated teeth, and operation area.

Funding Statement

Funding No funding to declare.