Determination of Modified Waldenström Staging in Legg–Calvé–Perthes Disease Using Deep Learning

Abstract

Background

Age and disease stage are critical factors in guiding management in Legg–Calvé–Perthes disease (LCPD). The modified Waldenström system is the most common staging system for LCPD but has only moderate inter-rater reliability. Deep learning (DL) has gained popularity in orthopaedics due to its potential to improve consistency/efficiency in determination of image-based classifications. The current study aimed to develop a DL model for determination of modified Waldenström staging from standard hip radiographs.

Methods

Paired anteroposterior (AP) and frog-lateral hip radiographs from patients with confirmed LCPD treated at 2 tertiary care children’s hospitals were included for model training/validation, hold-out testing, and external validation. Images of patients with skeletal dysplasias or other conditions were excluded. A modified Waldenström stage was assigned by the senior authors for each radiograph. A machine learning classification pipeline leveraging DL-automated extraction of clinically relevant and quantifiable parameters on AP/lateral radiographs was developed. Model performance was assessed on 2 separate LCPD classification schemes: early (Ia-IIa) vs late (IIb-IV) and the full classification.

Results

A total of 2,164 images were included from institution 1. The DL pipeline had excellent segmentation metrics (dice coefficient = 0.93) and was able to extract and calculate epiphyseal parameters on AP/lateral radiographs within 12 s. On the hold-out testing set (n = 229 radiograph pairs), the model had an area under the receiver operating characteristic curve (AUROC) of 0.82 in classifying LCPD as either early or late stage. The most predictive parameters driving model decision-making were AP epiphyseal sclerosis and epiphyseal area. On external validation at institution 2 (n = 533 radiograph pairs), the AUROC was 0.75 when categorizing early vs late stages.

Conclusions

The DL pipeline provided high accuracy in discerning early vs late stages of LCPD based on imaging parameters and moderate accuracy on the full classification, performing better at more advanced stages. Training the model on larger samples with more representation at earlier stages may improve accuracy. This has the potential to expedite and standardize staging for both clinical encounters and research.

Key Concepts

• (1)Deep learning can accurately classify early vs late-stage LCPD from standard hip radiographs.
• (2)Automated segmentation provides objective and quantifiable assessment of femoral morphology parameters pertinent to LCPD.
• (3)Artificial intelligence models could be applied to both clinical practice and research methodology as an automated tool for Waldenström staging.
• (4)Larger, multicenter datasets with greater representation of early stage LCPD imaging are needed to improve model performance and generalizability.

Level of Evidence

III

Introduction

Legg–Calvé–Perthes disease (LCPD) is an idiopathic avascular necrosis of the proximal femoral epiphysis typically seen in patients under the age of 12 years. Although controversy persists over the optimal treatment of LCPD, age and disease stage highly influence management decisions. The modified Waldenström (i.e. Elizabethtown) system is the most widely accepted staging paradigm for LCPD, assessing disease progression through the classic stages of 1) necrosis, 2) fragmentation, 3) reossification, and 4) healed, which were originally described by Waldenström. The modification of this staging system was described by Joseph et al., separating each of Waldenström’s original stages into early and late based on features of anteroposterior (AP) and frog-lateral hip radiographs (Table 1) [1]. Unfortunately, the modified Waldenström staging system has only moderate interrater reliability, limiting its utility both clinically and in research [2,3]. However, more simple differentiation of early vs late-stage disease has been consistently shown to be the most important differentiator for outcomes [1,4].

Table 1.

Modified Waldenström classification.

Stage	Radiographic findings
Ia	All/part of the epiphysis is sclerotic with no loss of epiphyseal height
Ib	Sclerosis of epiphysis with loss of epiphyseal height but no fragmentation of the epiphysis
IIa	Epiphysis has begun to fragment with evidence of 1 or 2 vertical fissures
IIb	Fragmentation is advanced. No evidence of new bone lateral to the fragmented epiphysis
IIIa	Early “porotic” new bone visible lateral to fragmented epiphysis, which covers <1/3 of epiphyseal width
IIIb	“Normal” new bone covers >1/3 of epiphyseal width
IV	Healing is complete with no radiographic evidence of avascular bone

Deep learning (DL) has gained popularity in orthopaedics due to its potential to improve consistency and efficiency for classification in similar image-based systems [5]. However, a prior study using DL for Waldenström staging achieved low accuracy owing to a small population and avoidance of combining multiple DL techniques such as segmentation, which involves automated identification of shapes and landmarks on imaging studies [6]. Other studies investigating the utility of machine learning (ML) and DL techniques in assessing LCPD staging for systems other than the modified Waldenström system relied largely on “black box” methods with questionable real-world utility [7,8]. No study has yet leveraged DL techniques based on image segmentation and segmentation analysis to identify LCPD and determine the modified Waldenström stage after the extraction of clinically relevant and quantifiable parameters on pelvis and hip radiographs.

The successful application of DL to staging of LCPD has the potential to improve efficiency and accuracy of routine radiographic assessment for this condition, which would benefit both clinicians and researchers. This study, therefore, aimed to develop a novel DL model pipeline for determination of an accurate Waldenström stage from routine hip radiographs for use by orthopaedic practitioners.

Materials and methods

Data

This was an institutional review board–approved retrospective study of consecutive patients ≤18 years of age diagnosed with LCPD from January 2008 to July 2024 at 2 tertiary care children’s hospitals. Individuals with a diagnosis of a systemic musculoskeletal condition such as a skeletal dysplasia (e.g. multiple epiphyseal dysplasia) were excluded. Radiographs were excluded if internal fixation crossed into the epiphysis, precluding assessment of epiphyseal fragmentation or reossification. A modified Waldenström stage was assigned to each radiograph by the senior authors who are extremely experienced in the evaluation and management of patients with LCPD.

Image sources

Paired anteroposterior (AP) and frog-lateral hip radiographs were included for model training. All imaging underwent preprocessing (cropping, normalization, resolution adjustment, and noise reduction) and augmentation (Supplemental Fig. 1). Manual segmentation was performed on 592ninety-two radiographs (296 AP and 296 frog-lateral) [9], outlining the epiphysis, metaphysis, greater trochanter apophysis, and hemipelvis.

Hip radiographic features: deep learning (U-Net) models

Manually annotated segmentation data and imaging were utilized to train and develop a DL U-Net model to automate segmentation of AP and lateral pelvis/hip images (Fig. 1). The dice coefficient was utilized to assess accuracy in segmentation of the trained model on a validation set created specifically for segmentation model training (Supplemental Fig. 2). Prior work has described the use of this method to automate clinically relevant parameters on pelvis radiographs [9]. In this study, 23 features related to the hip (epiphyseal area, height, curvature, and brightness) were derived on both the AP and lateral views (See Supplemental Methods) utilizing the automated segmentations (Supplemental Table 1, Supplemental Fig. 3).

Figure 1.

Deep learning extraction of epiphyseal parameters. Demonstration of workflow automating analysis of the epiphysis on hip radiographs on either AP or lateral radiographs. The analysis completed ∼11 s per radiograph depending on the processing unit. Upper images demonstrate segmentation analysis workflow. Lower images demonstrate extracted analyses from segmentation. The lower middle graph represents area and curvature analysis. The lower right graph represents brightness analysis as a proxy for sclerosis of the epiphysis. AP, anteroposterior.

Deep learning pipeline: machine learning models for LCPD classification

The derived radiographic features from the DL algorithm were then utilized as feature inputs in a separate tabular ML model to predict the modified Waldenström classification based on radiographs. Training/validation and hold-out testing cohorts were created from images at institution 1 (Children's Hospital of Philadelphia) in an 8:2 split by randomly splitting patients into either cohort. Patients, instead of images, were split to ensure there was no data leakage in between the sets given the patients had multiple sets of radiographs from different chronological timepoints. Two different models were created: early (I-IIa) vs late (IIb-IV) disease and the full modified Waldenström classification. (See Supplemental Methods for modeling specifics). The model was then tested externally on a set of radiographs from institution 2 (Gillette Children's Specialty Healthcare).

Statistical analysis

For binary classification models, the area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve were calculated. Metrics including sensitivity and specificity were calculated aiming to identify advanced stage cases. For multi-class models, a one-vs-rest analysis was utilized for each stage. The following scale was used to evaluate model performance for class discrimination: 0.5-0.6 = poor, 0.6-0.7 = fair, 0.7-0.8 = acceptable, 0.8-0.9 = excellent, and >0.9 = outstanding. Feature analysis was also performed for each model to determine which features were the most important in model prediction.

Results

Characteristics of the training/validation and hold-out testing set

A total of 2,164 radiographs from institution 1 were used for development of the DL pipeline. The training/validation set had 853 sets of AP and lateral radiographs (1,706 x-rays) from 157 unique patients. The hold-out testing set had 229 sets of AP and lateral radiographs (458 x-rays) from 46 patients. The distribution of modified Waldenstrom classification is shown in Table 2.

Table 2.

Classification breakdown of training/validation, testing, and external cohorts.

Stage	Training set: count (%) Total N = 853	Testing set: Count (%) Total N = 229	External dataset: count (%) Total N = 533
Ia	12 (1.41%)	1 (0.44%)	0 (0%)
Ib	52 (6.10%)	20 (8.73%)	41 (7.7%)
IIa	81 (9.50%)	20 (8.73%)	46 (8.6%)
IIb	110 (12.90%)	20 (8.73%)	85 (15.9%)
IIIa	176 (20.63%)	58 (25.33%)	146 (27.4%)
IIIb	221 (25.91%)	64 (27.95%)	194 (36.4%)
IV	201 (23.56%)	46 (20.09%)	21 (3.9%)

Characteristics of the external validation dataset

A total of 533 sets of AP and lateral radiographs (1,066 x-rays) from 70 unique patients were used for external validation of the DL pipeline. The distribution of the modified Waldenstrom classification is similarly shown in Table 2.

Deep learning segmentation performance

A subset of images was used to create a DL segmentation model for parameter extraction. One model was created to extract parameters from both AP and lateral pelvis/hip radiographs with a dice coefficient of 0.93, demonstrating segmentation performance on par with prior methods [9]. The parameters related to epiphyseal sclerosis, bone loss, growth, and curvature were extracted and calculated in less than 12 s per image view.

Deep learning pipeline testing: early (Ia-IIa) vs late (IIb-IV) stages

The model had an excellent AUROC of 0.82 on the hold-out testing set. The sensitivity was 0.78 in identifying late LCPD based on the AP and lateral radiographs. Specificity was 0.73, while positive predictive value (PPV) was 0.93 and negative predictive value (NPV) was 0.43 (Fig. 2). Feature analysis demonstrated that parameters related to the curvature of the epiphysis were the most predictive factor on the AP view. The brightness of the inferior epiphysis on the lateral view and the area of the epiphysis on both the AP and lateral also contributed as top features (Fig. 3). On the external dataset, the model had an acceptable AUROC of 0.75 for early vs late delineation (Supplemental Fig. 4). Specificity was 0.85, while PPV was 0.95 and NPV was 0.28.

Figure 2.

AUROC and classification matrix of early vs late model (Ia-IIa vs IIb-IV). AUROC, area under the receiver operating characteristic curve.

Figure 3.

Feature analysis of early vs late model (Ia-IIa vs IIb-IV).

Deep learning pipeline testing: full modified Waldenström classification

When tested on the full modified Waldenström classification, the model had an AUROC ranging from 0.60 to 0.81 on the testing set (Fig. 4). Grade IIA had the highest AUROC while grade IIIb had the lowest AUROC in a one-vs-rest analysis. Sensitivity ranged from 0.00 (stage Ia) to 0.70 (stage IV). Specificity ranged from 0.59 (stage IIIb) to 1.00 (stage Ia). On feature analysis, the most important features included the epiphyseal size on the AP and lateral views (Fig. 5). Additionally, the formation of new lateral epiphyseal bone on the AP view was important, and brightness of the inferior epiphysis on the lateral view also contributed. The AUROC values for the model on the external dataset ranged from 0.53 to 0.86 (Supplemental Fig. 5).

Figure 4.

AUROC and classification matrix of modified Waldenström classification model (all classes). AUROC, area under the receiver operating characteristic curve.

Figure 5.

Feature analysis of modified Waldenström model (all classes).

Discussion

The DL pipeline developed in this study demonstrated excellent model performance for determination of early vs late LCPD, specifically delineation of late-stage LCPD, based on both AP and frog-lateral radiographs. Lower performance was observed at earlier stages, which was attributed to the fact that fewer early stage radiographs were available for model training. Achievement of similar accuracy for modified Waldenström classification to prior manual reliability studies, particularly for later stage disease, has powerful implications for application to both clinical practice and research methodology as an automated tool for staging.

The Elizabethtown modification of the Waldenström classification is used for staging the predictable disease progression of LCPD, including sclerosis of the femoral epiphysis (stage I), fragmentation (stage II), re-ossification/revascularization (stage III), and healing/remodeling (stage 4) [10,11]. Waldenström staging dictates management, with patients in pre-fragmentation (stages IA/B) and early fragmentation (stage IIA) candidates for early containment procedures (e.g. proximal femoral varus osteotomy or shelf acetabuloplasty) to maintain femoral head sphericity and delay or prevent hip stiffness and premature osteoarthritis [12,13]. Past work has demonstrated that application of this classification system and defining the length of each stage allows for improved assessment of outcomes [14]. Oh et al. evaluated 49 patients with LCPD managed nonoperatively who presented at <6 years of age (mean age: 3.9 years) and were followed radiographically until skeletal maturity [15]. They observed that patients with better outcomes—defined as Stulberg I or II—had shorter stage I (4.1 vs 6.2 months) and stage II (5.9 vs 11.9 months) lengths. Accurate classification is also important for determining timing of intervention, particularly into early (Ia-IIa) and late (IIb-IV) stages given its utility in prognosticating outcomes. Sankar et al., corroborating prior work by Joseph [1,12], reported that 8 of 46 (17%) patients who underwent proximal varus osteotomy during Waldenström stage IA or IB were able to bypass advanced fragmentation (stage IIB) [16]. Therefore, a consistently accurate model for classification of LCPD into early and late disease as was accomplished here may be useful in dictating clinical management.

Numerous articles on the intra- and inter-rater reliability of the modified Waldenström classification have been published with marked variability in reported values. Hyman et al. published a reliability analysis on behalf of the International Perthes Study Group across 40 patients and reported the highest observed intra- (Kappa = 0.88) and inter- (Kappa = 0.81 to 0.82) values in the literature [2]. Reliability was lower (intra = 0.80, inter = 0.68 to 0.72) when disease was classified in a binary fashion into early (IA to IIA) or late (IIB to IV), and addition of temporal data (i.e. another radiograph at a separate time point) did not improve these results. However, other studies have failed to replicate these findings, likely owing to relatively consistent agreement across a high number of raters (n = 20), leading to elevated Fleiss’ kappa values. The more modest published intraobserver and interobserver values range from 0.32 to 0.89 and 0.23 to 0.76, respectively [3,[16], [17], [18], [19]]. Even the original study by Joseph et al. in 50 patients with LCPD only observed good intraobserver reliability of 0.71 and interobserver of 0.72 [11]. These data suggest that while the modified Waldenström classification is a useful tool for understanding disease progression, reliability is moderate at best. ML model performance is also limited by small sample size and the accuracy of graders. Therefore, accuracy and reliability improvements above the highest observed in publication are not feasible. However, we feel that given the results demonstrating high model performance for classification of LCPD into early or late stage, our work can be used nearly as reliably as a human rater in clinical and research settings.

Few other studies have applied ML to LCPD staging, classification, or outcomes [7]. Memis et al. evaluated an ML model for the assignment of Waldenström classification to hip magnetic resonance imaging rather than traditional radiographs [6]. They achieved a relatively low 62% accuracy owing to a small population (13 hips with LCPD) and failure to use additional DL techniques such as segmentation. Beyond the modified Waldenström classification, Soydan et al. developed a convolutional neural network (CNN) model for determination of Herring (lateral pillar) class and were able to achieve an intra-class correlation (ICC) of 0.87, though this necessitated modification of normal hip radiographs due to the low number of patients available with LCPD (81 hips) [8]. Zhang et al. applied an extreme gradient boosting (XGBoost) ML model incorporating clinical, radiomic, and DL features for prediction of early femoral head deformity in LCPD, offering a promising prognostic tool for guiding early treatment strategies [20]. Work in this space is otherwise limited, and no study has applied ML to Catterall or Stulberg classifications. In our study, segmentation allowed for the extraction of parameters such as epiphyseal curvature, epiphyseal growth, and epiphyseal sclerosis/brightness to serve as features for classification. Critically, the model was able to utilize and weigh parameters that were clinically relevant and in line with descriptors set out in the Waldenström classification (i.e. femoral head deformity and curvature, brightness and sclerosis, lateral epiphyseal area and lateral growth).

One oft-mentioned concern with DL models in orthopaedic surgery is their potential clinical applicability. Integration into a hospital’s picture archiving and communication systems may aid less experienced clinicians in making treatment decisions, similar to those already in place for other common pediatric hip conditions [[21], [22], [23]]. Additionally, for larger multicenter databases, rapid modified Waldenström classification has the potential to save time and resources for research staff while providing consistent, standardized readings across institutions. Additionally, the finite data obtained from the segmentation approach used here allow for more objective quantification of parameters pertinent to LCPD. It is unlikely that raters are consistently determining the texture/quality of new bone formation (stage IIIA) or measuring the new bone width covering the epiphysis (stage IIIB) in clinical practice. These new metrics may allow for improved inter-rater reliability in future studies.

Limitations

Several limitations must be noted. Model performance was lower for earlier stage LCPD, likely due to the relatively few radiographs that were available for inclusion as many children present in more advanced stages. This may limit the model’s generalizability/applicability to an earlier LCPD population, though distinguishing of IIA from IIB and IIIB from IV are often most important when using the modified Waldenström classification. Another frequently mentioned limitation of DL models is their generalizability to other hospitals as external validation is frequently not performed. As imaging protocols vary by institution including positioning and radiation dosage, substantial decreases in model performance can be observed [24]. Model performance was slightly decreased at institution 2 from the hold-out testing set but is still acceptable, suggesting its utility for delineation of early vs late stages. Testing on other patient populations with different racial and socioeconomic demographics would also allow for improved assessment of the model’s generalizability. Additionally, feature analysis was solely performed on the epiphysis based on the Waldenström classification. However, features from the metaphysis can contribute to delineation of disease severity with femoral neck widening and cyst formation, which could be included in future studies. Last, further work could also incorporate clinical and demographic variables such as age and gender to further delineate LCPD staging and disease progression.

Conclusion

The developed DL pipeline provided high accuracy in discerning early vs late stages of LCPD based on clinically relevant imaging parameters and moderate accuracy on the full classification. Training the model on larger samples with more representation at earlier stages may improve accuracy. Such a model has the potential to expedite, quantify, and ultimately improve both clinical encounters and research for LCPD by standardizing staging classifications.

Author contributions

Joshua T. Bram: Writing – review & editing, Writing – original draft, Supervision, Methodology, Investigation, Data curation, Conceptualization. Seong J. Jang: Writing – review & editing, Methodology, Formal analysis, Data curation, Conceptualization. Carter Hall: Supervision, Project administration, Methodology, Investigation, Formal analysis, Data curation. Martha Kebeh: Methodology, Investigation, Formal analysis, Data curation. Omar Shareef: Writing – review & editing, Writing – original draft, Software, Methodology, Formal analysis. Christopher J. DeFrancesco: Writing – review & editing, Project administration, Methodology, Investigation, Conceptualization. Jennifer C. Laine: Project administration, Methodology, Investigation, Conceptualization. Wudbhav N. Sankar: Writing – review & editing, Writing – original draft, Project administration, Methodology, Investigation, Data curation, Conceptualization.

Additional links

• •POSNAcademy: Treatment Strategies for Perthes
• •KSSTA: A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part I
• •KSSTA: A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part II

Ethics approval and consent

The author(s) declare that no patient consent was necessary as no images or identifying information are included in the article.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Supplementary material

The following is the supplementary data to this article: