Radiography-based AI decision support for further post-traumatic knee MRI referral in children

Abstract

Purpose

The aim of our systematic work was to investigate an artificial intelligence-based prediction of relevant internal injuries of the paediatric knee joint based on initial radiographs and to develop a corresponding AI model.

Methods

We queried the hospital information systems of two independent sites for pediatric and adolescent patients up to the age of 19 years with a history of recent trauma, who had undergone a radiograph at ap and lateral projections and Magnetic Resonance Imaging (MRI) of the same knee joint within 48 hours. After exclusion of patients due to postoperative situations, tumorous or infectious diseases, missing and invalid radiographs, 873 patients with 1746 total images were included. Each model was assessed for precision, recall, accuracy and the F1 score, revealing variation between model versions in performance metrics.

Results

The averaged performance across all EfficientNet models achieved a precision of 0.7340, recall of 0.7181, accuracy of 0.7131, and 0.7260. The best per- forming model, EfficientNet-B5, achieved an average precision of 0.7445, recall for 0.7890, and accuracy for 0.7450, respectively. The heat maps revealed significant concentrations of pathologies detected by AI primarily in the femoral and tibial condyles. AI also identified fractures and microtrabecular fractures, suggesting their effectiveness in identifying relevant injuries on (pediatric) knee radiograph.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12880-026-02185-7.

Introduction

Knee joint injuries are frequently observed in all age groups, encompassing children and adolescents [1–4]. The spectrum of these injuries varies significantly from minor sprains or contusions to severe and potentially irreversible damage, such as cartilage defects, ruptures of the cruciate ligaments, and meniscus lesions [5, 6]. In many circumstances, early detection of intraarticular knee injuries is essential to avoid long-term sequelae [7].

X-ray examinations remain the standard approach for the initial assessment of knee injuries [8]. They also play a critical role in follow-up evaluations, a practice that includes pediatric patients. As an advanced diagnostic tool, magnetic resonance imaging (MRI) holds an eminent position, demonstrating the capability to accurately identify nearly all types of soft tissue and bone injuries especially in children when the bone tissue is not fully ossified [9]. However, for the visualization of ossified structures in mature bone, computed tomography (CT) retains superiority over MRI, especially for the preoperative visualization of complex fractures [10].

Artificial intelligence (AI) has undergone substantial advancements, emerging as a highly effective approach to the analysis of distinct image attributes, particularly within the domain of medical radiology [11, 12]. Beyond applications such as bone age estimation [13, 14], which have been widely used for some time, sophisticated solutions capable of predicting or identifying various pathological conditions are increasingly gaining importance [15, 16]. AI enables the prediction of specific image attributes and provides the ability to locate and highlight image features, as well as to delineate individual image regions with pixel-level accuracy [15]. Pathological conditions are frequently identified by object recognition, which is often performed in the form of bounding boxes [16].

Our dual-site pilot study aimed to evaluate the feasibility of utilizing an AI-driven model to predict significant internal injuries in the pediatric knee joint. This was achieved by applying the model to initial radiographs alongside the corresponding MRI examination. The study was conducted to determine the algorithm’s effectiveness in predicting the presence of relevant internal pathology to strongly support the clinical necessity for additional MRI assessments. As far as we know, this is the first study of its kind carried out in both pediatric and adult populations.

Materials and methods

We searched the hospital information systems of two independent sites for pediatric and adolescent patients up to the age of 19 years with a history of recent trauma, who had undergone a radiograph and MRI of the same knee joint within 48 hours at

• The University Hospital of Graz, Austria (tertiary pediatric trauma center) and

• Klinik Diakonissen Schladming, Austria (tertiary trauma center).

The initial search yielded 969 relevant patients. All studies of these patients were recovered and manually interpreted for inclusion and exclusion criteria by a radiology resident (N.S.) with 4 years of experience in musculoskeletal imaging.

After exclusion of patients due to prior operations, tumors or infectious diseases, missing and invalid radiographs, 873 patients with a total of 1746 images in a.p. and lateral projections were included (compare Fig. 1).

Fig. 1.

Flow chart of the patient selection and dataset organization. The whole dataset was derived from 2 separate institutions. MRI findings were used to split x-ray examinations into relevant and non-relevant pathologies. K-fold cross validation was used to train, validate and test the data

We defined MRI as the non-invasive gold standard, because it showed high performance in non-invasive detection and diagnosis of acute knee injuries [17–26]. MRI vs. arthroscopy showed a sensitivity of 74.42%, specificity of 93.10%, accuracy of 84.21%, and negative predictive value of 88.04%. [25]. Phelan et al. showed in a meta-analysis that sensitivity and specificity of MRI were 87 and 93%, respectively, for ACL tears; 89 and 88%, respectively, for medial meniscal tears; and 78 and 95%, respectively, for lateral meniscal tears [22]. MRI is the gold standard for detecting occult fractures and stress fractures [24, 26]. The principal investigator developed and meticulously curated a comprehensive database including metadata and findings from MRI studies. Both direct imaging results and associated report texts were rigorously evaluated to enhance the database. Relevant pathologies in our study are defined as those typically requir- ing surgical intervention, immobilization, or specific follow-up, whereas “non-relevant” pathologies are generally managed conservatively without sequelae [27–31]. Table 1 provides a tabular representation delineating our criteria for categorizing pathologies as non-relevant or relevant.

Table 1.

Criteria for distinguishing studies that indicate either non- relevant or relevant pathology as a valid basis for recommending further MRI consultation

Category	Non-relevant pathology	Relevant pathology
Bone	Bone marrow edema Non-ossifying fibroma (NOF) Osteochondroma	Fractures
Cartilage		Cartilaginous tears Cartilaginous defects
Meniscus	Discoid meniscus	Meniscal tears
Ligaments	Ligament strains	Partial ligament tears Full substance ligament tears Avulsion injuries
Joint	Synovial plica syndrome Joint effusion
Soft tissue	Soft tissue edema

Patient characteristics

Out of the 873 included patients 760 patients originated from the database Graz and 113 patients from the database Schladming with a mean age of 13.7 years (range 1.3- 19.7 years). Forty-eight point three percent (422 patients) of the included patients were female and 51.7% (451 patients) were male.

The patients from the database Graz had a mean age of 13.5 years, a minimum age of one year and four months and a maximum age of 18.3 years with a gender distribution of 47.6% female and 52.4% male.

The patients from the smaller database Schladming were slightly older with a mean age of 15.7 years, a minimum age of eight years and a maximum age of 19.7 years with a gender distribution of 40.3% female and 59.7% male.

Image processing

X-ray studies were acquired from both hospitals in Digital Imaging and Communications in Medicine (DICOM) format. MRI examinations served solely as a foundation for the development of the findings catalogue and were, thus, not exported. The x- ray data were de-identified and subsequently converted into PNG files. Given that the inputs comprised either 12-bit or 16-bit grayscale values, normalization to 16-bit PNG files was conducted. The normalized images underwent preprocessing utilizing the scikit-image package version 0.19 [32]. A percentile crop was executed with 2% at the lower and 0.5% at the upper echelon of gray values. Subsequently, a local contrast enhancement (CLAHE) was implemented to ensure adequate image contrast. The PNG files were then converted into 8-bit RGB color images, which are typical inputs for most neural networks.

Model training

We employed the EfficientNet, including its variants B0 to B7 [33], and YOLOv8 binary classifiers due to its established efficacy in diverse medical imaging applications [34–36] and our prior experience. The primary objective was to differentiate normal x- rays from those exhibiting relevant pathological features. To address restrictions posed by a limited number of available x-rays, we implemented 16-fold cross-validation to optimize the utilization of the available samples.

The models were trained on a Linux workstation, which was configured with two Nvidia RTX 4090 graphics cards, each possessing 24 GB of video memory. The system was powered by an Intel Core i7 13700K processor and supported by 64 GB of RAM. The software environment included Python 3.10 running on Ubuntu 22.04 LTS. Specifically, FastAI Python version 2.7.15 [37] was employed for the training of the EfficientNet model variants.

AI performance metrics

Testing followed a two-step process, initially calculating average precision (AP), AP50, and AP50-95 [38] as standard performance metrics [39] for each test set.

Image classification and regression performance were measured by assessing

• True Positives (TP): The model predicted a label that correctly matches the ground truth.

• True Negatives (TN): The model correctly predicts the absence of a label not present

in the ground truth.

• False Positives (FP): The model incorrectly predicted a label that does not match the ground truth (Type I Error).

• False Negatives (FN): The model missed predicting a label that is present in the

ground truth (Type II Error).

• Precision (P): The ratio of TP to TP + FP, indicating the proportion of correct positive predictions out of all positive predictions made by the model.

• Recall (R): The ratio of TP to TP + FN, measuring the model’s ability to correctly

identify all relevant instances in the ground truth.

• Accuracy (A): The proportion of correct predictions (TP + TN) over the total cases examined, reflecting the model’s overall performance.

• F1 score (F1): The harmonic mean of precision and recall, balancing the effects of

both false positives and false negatives.

• Average Precision (AP): The area under the precision-recall curve, summarizing the precision-recall trade-off across different thresholds and providing a single measure of model performance.

For binary classification tasks with EfficientNet, “Precision”, “Recall”, “F1 score” and “Accuracy” were used as the primary performance metrices.

We assessed the calibration of all eight EfficientNet variants (B0–B7) using reliability diagrams, the Brier score, and the Expected Calibration Error (ECE). The Brier score quantified the mean squared difference between predicted probabilities and out- comes, while ECE measured the average absolute difference between confidence and accuracy across ten bins. Clinical utility was evaluated via Decision Curve Analysis (DCA), calculating net benefit across threshold probabilities from 0.0 to 0.99. Model performance was compared against”treat all” and”treat none” strategies.

Statistical analyses

Nonparametric Independent-Samples: Mann-Whitney U tests were employed to evaluate the differences between normal x-rays and images displaying relevant pathologies.

The distribution of parameters of interest across groups and sub-groups may have been imbalanced; therefore, precision-recall curves (PR AUC) were used for the graphical visualization of classification metrics. Participants were stratified into two distinct age cohorts: the first comprising individuals under the age of 13 years, and the second including those aged 13 years and older. Comparisons of the age groups and also between the ap and the lateral radiograph were conducted utilizing the Wilcoxon signed rank test. Statistical significance was determined by p values less than 0.05.

Ethical statement

All experiments were conducted in adherence to the Declaration of Helsinki. The assessments were executed subsequent to obtaining approval from the Ethics Commit- tee of the Medical University of Graz (Reference number 36–291 ex 23/24). Written informed consent was not necessary due to the retrospective nature of the study design.

Results

Of the 873 patients included, 531 patients showed relevant pathologies in the MRI images. The distribution of lesions among the 531 patients as shown in Table 2 highlighting a predominance of ligament tears, which accounted for 367 lesions, making them the most common pathology observed. Among these, the anterior cruciate ligament (115 lesions) and the medial patellofemoral ligament (157 lesions) were the structures most frequently affected. Cartilage defects and fractures were also prevalent, with the lateral femoral condyle being a particularly common site for both pathologies. Tears of the lateral and medial meniscus were almost equally distributed, with posterior horn tears being the most frequent subtype. Of the 80 total lesions in the lateral meniscus 50 lesions were located in the posterior horn. In the medial meniscus of the 82 lesions 67 were located in the posterior horn, with 24 of them being ramp lesions. Overall, the data underscore the high incidence of ligament and meniscus injuries compared to other types of pathology. The full list of relevant pathologies and their distribution are listed in a separate table in the supplementary material.

Table 2.

The table shows the number of relevant pathologies and their subsections. The section lateral ligaments included the lateral collateral ligament and the anterolateral ligament. The section medial ligaments included the medial collateral ligament and the posterior oblique ligament

Relevant Pathology	Structure	Lesions
N = 531 patients		N = 998 lesions total
Avulsions		N = 172 lesions
	Anterior cruciate ligament	50 total
	Lateral ligaments	123 total
	Ligamentum arcuatum	1 total
	Medial collateral ligament	9 total
	Medial patellofemoral ligament	80 total
	Posterior cruciate ligament	7 total
Cartilage defect		N = 121 lesions
	Femur lateral condyle	55 total
	Femur medial condyle	17 total
	Tibia lateral	1 total
	Patella medial	48 total
Fracture		N = 172 lesions
	Femur lateral condyle	64 total
	Femur medial condyle	14 total
	Proximal tibia	30 total
	Fibula	59 total
	Patella	5 total
Ligament tears		N = 367 lesions
	Anterior cruciate ligament	115 total
	Medial ligaments	11 total
	Lateral ligaments	43 total
	Ligamentum patellae	4 total
	Lateral patellofemoral ligament	2 total
	Medial patellofemoral ligament	157 total
	Posterior cruciate ligament tears	11 total
	Distal quadriceps tendon	1 partial tear
Soft tissue and muscle tears		N = 4 lesions
	Muscle tears	3 partial tears
	Hoffa fat pad	1 partial tear
Meniscus tears		N = 162 lesions
	Lateral meniscus	80 total
	Medial meniscus	82 total

The area under the curve (AUC) of the detection rate of true positives in EfficientNet increased from 74.9% in the B0 EfficientNet to 82.9% in the B4 EfficientNet shown in Fig. 2. From B5 to B7 the AUC for EfficientNet decreased again with an AUC of B7 EfficientNet of 81.6%.

Fig. 2.

PR AUC for all eight EfficientNet variants and the combined the graphs. The highest AUC was achieved for EffcienNet variant “B4” with 0.829. Grey areas in the subplots 1 to 8 refer to 95% confidence intervals

In general, the average performance of all EfficientNet models, shown in Table 3, demonstrated a trade-off between precision, recall, precision, and F1 score in different versions of the model. The best performing model, EfficientNet-B5, achieved balanced performance with a high F1 score of 0.7661, along with a precision of 0.7445, recall of 0.7890, and accuracy of 0.7450. EfficientNet-B6 and B7 also showed strong recall values (0.8110 and 0.8287, respectively), suggesting their effectiveness in identifying relevant internal injuries on radiographs, although with a decrease in precision. In Table 4 the total PRAUC values and the 95% confidence intervals are shown.

Table 3.

The table shows the overall performance results of the EfficientNet models

Table 4.

The table shows the PRAUC total values of the EfficientNet models and the 95% confidence intervals

EfficientNet-B4 exhibited superior calibration, achieving the lowest Brier score (0.216) and Expected Calibration Error (ECE = 0.194). EfficientNet-B5 and B6 followed with Brier scores of 0.228 and 0.235, respectively, while variants B0, B1, and B7 showed higher miscalibration (Brier >0.250; ECE >0.230) Fig. 3. Reliability diagrams confirmed EfficientNet-B4’s alignment with the ideal diagonal. In the Decision Curve Analysis, all models surpassed the “treat none” strategy Fig. 4. EfficientNet-B4 consistently yielded the highest net benefit in the majority of threshold probabilities, outperforming other variants, confirming its clinical utility, as it provided the highest net benefit in a wide range of thresholds, maximizing true positive identification while minimizing false positives.

Fig. 3.

Calibrationcurves for all eight EfficientNet variants, with EfficientNet-B4 achieving superior calibration with the lowest Brier score (0.216) and Expected calibration error (ECE = 0.194)

Fig. 4.

Decisioncurves for all eight EfficientNet variants, with EfficientNet-B4 yielding the highest net benefit in a wide range of thresholds

We also divided the patients into two groups, with one group of patients under 13 years and the second group 13 years or older. In the group of patients older than 13 years, an overall better performance in detecting true positives with higher AUC was found compared to the group of patients under 13 years. The highest area under the curve (AUC) of the detection rate of true positives was found in EfficientNet for patients older than 13 years in B4 EfficientNet (84. 6%). For patients younger than 13 years the highest values were achieved in B5 EfficientNet (77.9%), as presented in Fig. 5. Differences between age groups in terms of PR AUC were statistically significant (p = 0.008).

Fig. 5.

PR AUC for the eight EfficientNet variants compared between children (age below 13 years, blue line) and adolescents (age equal or higher than 13 years, red line). 95% confidence intervals are given as shaded areas

In Table 5 the PRAUC values and the 95% confidence intervals of the different age groups are shown. Age group 0 are the children below 13 years, age group 1 are the children over 13 years.

Table 5.

The table shows PRAUC values and the 95% confidence intervals of the different age groups. Age group 0 are the children below 13 years, age group 1 are the children 13 years or above

We separately evaluated the performance of EfficientNet in detecting pathologies in the two different planes, a.p. radiographs and lateral radiographs. The lateral projection showed overall better performance in the detection of true positives with higher AUC, compared to a.p. projections. The highest AUC of the detection rate of true positives in EfficientNet for lateral radiographs was found in EfficientNetB5 (82. 8%). For ap radiographs the highest values were achieved in EfficientNetB4 (81. 6%). AI performed significantly better on lateral radiographs in discriminating between non- relevant and relevant pathology in terms of PR AUC (Wilcoxon signed rank test p = 0.008, shown in Fig. 6).

Fig. 6.

PR AUC for the eight EfficientNet variants compared between a.P. (blue line) and lateral projection (orange line). 95% confidence intervals are given as shaded areas

In Table 6 and Table 7 the PRAUC values and the 95% confidence intervals of the different projections are shown.

Table 6.

The table shows PRAUC values and the 95% confidence intervals of the ap projection

Table 7.

The table shows PRAUC values and the 95% confidence intervals of the lateral projection

Discussion

Our dual-site pilot study systematically examined eight distinct variants of the EfficientNet model (EfficientNet-B0 through EfficientNet-B7) to predict clinically significant injuries of pediatric knee joints, by only using initial radiographic images to support the clinical necessity for potential referral to subsequent MRI imaging. Each model iteration was rigorously evaluated in terms of precision, recall, accuracy and F1 score, demonstrating variation in performance metrics between different versions.

Our pilot study offers several points of novelty and unique features that separate it from related research: The innovation of the present study lies in its methodical application of artificial intelligence to initial radiographic images to predict the need for additional MRIs, thus addressing a specific gap in the continuum between primary and advanced diagnostic procedures. Through the integration of datasets from two distinct institutions, the study augments both its robustness and generalizability. It distinctively amalgamates technical proficiency with clinical acumen in the domains of pediatric artificial intelligence and radiology.

Gillies et al. highlighted how radiomics transform imaging data into features that can be integrated with clinical data to enhance diagnostic, prognostic, and predictive accuracy in terms of decision support systems. Integrating these data into clinical models offers the potential for personalized patient care, although challenges like standardization and validation must be addressed before widespread use [40]. This study focuses on children and adolescents, a demographic often overlooked in AI research, which typically targets adults with conditions like osteoarthritis or fractures, filling a gap in smaller AI studies [41, 42].

We anticipate the development of an innovative automated system that possesses exceptional positive or negative predictive value, serving as an indispensable asset for treating physicians. This system is designed to transform both the precision and personalization of referrals for further MRI of the knee joint. Ideally, this sophisticated system would have the capability to authoritatively determine the necessity of additional imaging, thereby fostering unwavering confidence in its recommendations. Although an MRI examination typically represents only a minor inconvenience for the individual patient, the distressing experience of an unnecessary procedure in a confined space may induce considerable anxiety. In addition, unnecessary magnetic resonance imaging not only increases patient discomfort, but also exacerbates the escalating and preventable financial burdens within the healthcare system.

The distribution of lesions across the 531 patients highlights a predominance of anterior cruciate ligament tears and medial patellofemoral ligament tears and avulsions, making them the most common pathologies observed. In addition, the lateral condyle of the femur was more frequently affected by cartilage defects and fractures. Meniscus tears were common, with posterior horn tears being the most frequent sub- type. The distribution of the lesions can be explained by pivot shift injuries with anterior cruciate ligament tears and lateral patella luxations and their concomitant injuries, which are common injuries in children and adolescents [43, 44].

The heat maps revealed significant concentrations of pathologies detected by AI predominantly within the femoro-tibial and femoro-patellar joint areas, as well as in the femoral and tibial condyles, as shown in Fig. 7. In contrast, when the AI did not detect relevant pathologies, the heat maps were distributed more in the periphery, as shown in Fig. 8.

Fig. 7.

Example saliency maps of two true positive (TP) examples where the AI correctly classified and localized pathologies. On the left it identified a proximal avulsion fracture of the lateral collateral ligament and a concomitant bone marrow edema and incipient impression fracture of the medial femur condyle. On the right picture the AI correctly identified a multi-fragmentary fracture of the tibial plateau

Fig. 8.

Picture 1 and 2 show example heatmaps of true negative (TN) cases that show no substantial activation in the relevant section of the radiographs. Picture 3 and 4 show saliency maps of two false negative (FN) examples where the AI algorithms wrongfully did not identify relevant pathologies. On the left it did not properly identify a proximal fracture of the tibia and fibula. On the right picture the AI did not identify a multi-fragmentary fracture of the tibial plateau. Interestingly, fractures of the femur, tibia, and fibula located in the peripheral sections of the image escaped detection by the algorithm as shown in Fig. 8. The explanation might be that the number of training samples was insufficient to properly learn how to detect these injuries

In particular, AI also identified pathologies in the femoral and tibial condyles, which are likely associated with fractures and microtrabecular fractures. Furthermore, the intercondylar region was designated as pathological, a condition that could be attributed to avulsion fractures of the anterior cruciate ligament.

Limitations of this manuscript need to be discussed. The size of the dataset rep- resents one of the major issues, as the number of knee radiographs of pediatric and adolescent patients undergoing MR examinations within 48 hours is comparatively small. This inclusion criterion inherently selects for patients with a higher pre-test probability of severe injury compared to a general unselected emergency department population. Moreover, this investigation was exclusively conducted among children and adolescents; consequently, there is a scarcity of data regarding its applicability to older populations. Future research should particularly assess the impact of degenerative diseases, such as osteoarthritis, on the study’s outcomes.

Conclusions

AI showed promising approaches to correctly assess and classify initial X-ray images for the presence of relevant or non-relevant pathologies in our pilot study. In general, the average performance of all EfficientNet models achieved a precision of 0.734, a recall of 0.718, accuracy of 0.713, and an F1 score of 0.726. Provided a more comprehensive data set is available, these findings suggest that with continued refinement, AI models could evolve into valuable diagnostic adjuncts and effective triage tools to assist in the exclusion or detection of relevant injuries enhancing the ability to screen for relevant injuries on standard knee radiographs.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

MRI

Magnetic resonance imaging

CT

Computed tomography

AI

Artificial intelligence

NOF

Non-ossifying fibroma

DICOM

Digital Imaging and Communications in Medicine

CLAHE

Local contrast enhancement

AP

Average precision

TP

True positives

TN

True negatives

FP

False positives

FN

False negatives

P

Precision

R

Recall

A

Accuracy

F1

F1-Score

PR

Precision-recall curves

AUC

Area under the curve

Author contributions

All contributing authors read the paper, approved the contents, and meet the criteria for authorship as established.

Funding

We did not receive any funding for this study.

Data availability

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval, consent to participate

All experiments were conducted in adherence to the Declaration of Helsinki. The assessments were executed subsequent to obtaining approval from the Ethics Committee of the Medical University of Graz (Reference number 36–291 ex 23/24). Written informed consent was not necessary due to the retrospective nature of the study design. Institutional Review Board approval was obtained and the necessity for written informed consent was waived.

Consent for publication

We did neither publish study results (in print or electronically) nor sent the manuscript to other journals for review. There are no papers (published or under review), which duplicate parts of the work reported, or concern the same subjects as the present paper. Availability of data and materials: The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.