The diagnostic value of artificial intelligence-assisted imaging for developmental dysplasia of the hip: a systematic review and meta-analysis

Abstract

Objective

To clarify the efficacy of artificial intelligence (AI)-assisted imaging in the diagnosis of developmental dysplasia of the hip (DDH) through a meta-analysis.

Methods

Relevant literature on AI for early DDH diagnosis was searched in PubMed, Web of Science, Embase, and The Cochrane Library databases until April 4, 2024. The Quality Assessment of Diagnostic Accuracy Studies tool was used to assess the quality of included studies. Revman5.4 and StataSE-64 software were used to calculate the combined sensitivity, specificity, AUC value, and DOC value of AI-assisted imaging for DDH diagnosis.

Results

The meta-analysis included 13 studies (6 prospective and 7 retrospective) with 28 AI models and a total of 10,673 samples. The summary sensitivity, specificity, AUC value, and DOC value were 99.0% (95% CI: 97.0-100.0%), 94.0% (95% CI: 89.0–96.0%), 99.0% (95% CI: 98.0-100.0%), and 1342 (95% CI: 469–3842), respectively.

Conclusion

AI-assisted imaging demonstrates high diagnostic efficacy for DDH detection, improving the accuracy of early DDH imaging examination. More prospective studies are needed to further confirm the value of AI-assisted imaging for early DDH diagnosis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13018-024-05003-4.

Introduction

Developmental dysplasia of the hip (DDH) is caused by an abnormal relationship between the femoral head and acetabulum of the immature hip joint [1], encompassing a spectrum of abnormalities from mild acetabular dysplasia to subluxation or complete dislocation of the femoral head. It is the most common pediatric musculoskeletal disorder, with an incidence of approximately 0.1–11%, more common in females and the left hip [2, 3]. Undetected or untreated DDH can lead to pain, limb shortening, gait abnormalities, reduced range of motion, and even severe disability. For DDH treatment, the Pavlik harness is the preferred method for infants under 6 months of age [4]. The main treatment methods for children over 6 months can be summarized as closed or open reduction, and the surgical efficacy is closely related to the timing of surgery. Studies have found that early diagnosis and treatment of DDH in infancy can achieve or approach normal pediatric hip joints [5, 6].

Currently, the diagnosis of pediatric DDH mainly includes X-ray, ultrasound, and MRI imaging. X-ray examination is the earliest diagnostic method but has certain limitations for use in younger infants [7]. Ultrasound is widely used in the diagnosis of hip joints in infants aged 0–6 months, but it has disadvantages such as a certain misdiagnosis rate, missed diagnosis rate, and strong dependence on the operator. In recent years, the rapid development of artificial intelligence (AI) technology and its combination with DDH gold standards have attracted widespread attention. Mohammad Fraiwan et al. [8] included pelvic anteroposterior X-ray images of 120 DDH patients and 234 normal individuals, classified them as DDH or normal, and developed and evaluated various performance indicators using multiple deep transfer learning models such as SqueezeNet. They found that the detection accuracy of DarkNet53 for DDH reached 96.3%, with an F1 score, precision, recall, and specificity of 95%, 90.6%, 100%, and 94.3%, respectively. He J et al. [9] explored the feasibility of using an automatic evaluation technology for hip ultrasound examination plane selection based on AI and three-dimensional ultrasound, including 216 infant hip joints. They found that the technology had an AUC = 0.938 (p = 0.00), with high sensitivity and specificity of 0.878 and 0.893, respectively, confirming the high feasibility and prospects of AI-based three-dimensional ultrasound.

To date, many AI-assisted models have been used to improve the diagnostic efficacy of DDH, but no related meta-analysis has been reported so far. The sensitivity and specificity of AI-assisted diagnosis are still unknown, and the diagnostic efficacy needs to be investigated, which is the purpose of this article. This study systematically searched and conducted a meta-analysis of all literature on AI-assisted diagnosis of DDH to obtain the latest and most comprehensive evidence-based medical evidence to confirm the value of AI in DDH diagnosis.

Methods

Protocol and registration

This systematic review and meta-analysis was conducted in accordance with the PRISMA 2020 statement and registered in the PROSPERO database (CAD42024558159).

Search strategy

A systematic literature search was conducted in PubMed, Web of Science, Embase, and The Cochrane Library databases using medical subject headings and free words, including “artificial intelligence,” “imaging,” and “developmental dysplasia of the hip.” The search formula for PubMed was as follows: (((“Artificial Intelligence“[Mesh]) OR (((((((((((((((((((Intelligence, Artificial) OR (Computer Reasoning)) OR (Reasoning, Computer)) OR (AI)) OR (Machine Intelligence)) OR (Intelligence, Machine)) OR (Computational Intelligence)) OR (Intelligence, Computational)) OR (Computer Vision Systems)) OR (Computer Vision System)) OR (System, Computer Vision)) OR (Systems, Computer Vision)) OR (Vision System, Computer)) OR (Vision Systems, Computer)) OR (Knowledge Acquisition)) OR (Acquisition, Knowledge)) OR (Knowledge Representation)) OR (Knowledge Representations)) OR (Representation, Knowledge))) AND ((“Diagnostic Imaging“[Mesh]) OR ((((((Imaging, Diagnostic) OR (Medical Imaging)) OR (Imaging, Medical)) OR (CT)) OR (MRI)) OR (UItrasound)))) AND ((“Developmental Dysplasia of the Hip“[Mesh]) OR ((((((((Developmental Hip Dysplasia) OR (Developmental Hip Dysplasias)) OR (Dysplasia, Developmental Hip)) OR (Hip Dysplasia, Developmental)) OR (Hip Dislocation, Developmental)) OR (Developmental Hip Dislocations)) OR (Dislocation, Developmental Hip)) OR (Developmental Hip Dislocation))). The search time was up to April 4, 2024. All retrieved literature was manually reviewed and verified through EndNote X9.

Inclusion and exclusion criteria

This study included prospective and retrospective studies on pediatric DDH that focused on the diagnostic efficacy of AI models assisting ultrasound or X-ray gold standards. Inclusion criteria: reports containing the number of DDH-positive patients, number of negative patients, sensitivity (Sen), specificity (Spe), true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), or articles from which these numbers could be calculated. Exclusion criteria: (1) duplicate publications; (2) articles unrelated to this study; (3) reviews, meta-analyses, conferences, etc.; (4) animal models, low sample size articles; (4) articles with no or incomplete data; (5) non-English articles.

Data extraction

Two independent reviewers extracted data from eligible studies, including first author, publication year, study period, study design, age of study subjects, AI-assisted diagnostic methods, gold standard, sample size of case group, sample size of control group, mean age, and number of males/females. If conflicts arose between the two extraction results, the opinion of a third party was sought. All data were summarized into a Microsoft Excel spreadsheet.

Quality assessment

The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool [10] was used to comprehensively assess the quality of included studies, completed independently by two reviewers and then verified by a third person, with final results determined through joint discussion for divergent parts. This assessment tool includes four aspects: case selection, index test, reference standard, and flow and timing. Each item is rated as “high risk,” “low risk,” or “unclear.” The quality assessment scoring table is provided in the supplementary materials.

Data analysis

Statistical analysis was mainly performed using Stata15.0 and RevMan5.4.1 software. A bivariate mixed-effects regression model was used after the random-effects model for the following indicators: combined sensitivity (Se), specificity (Sp), positive likelihood ratio (pLR), negative likelihood ratio (nLR), diagnostic odds ratio (DOR), and their 95% confidence intervals (95% CI). The summary receiver operating characteristic (SROC) curve was plotted, and the area under the curve (AUC) was calculated. The I² statistic was used to check for heterogeneity caused by non-threshold effects, and Deeks’ funnel plot was used to assess publication bias and assumed small-study effects. To evaluate the performance of AI models across different subgroups and compare them with the overall diagnostic efficacy of all models, five subgroups were created based on “region,” “study design,” “gold standard,” “mean age,” and “AI model,” and specificity, sensitivity, AUC value, and DOR value were compared to determine differences between subgroups and their overall differences.

Results

Literature screening process and results

A total of 425 articles were obtained through database searches. After removing duplicate literature using EndNote X9 and further excluding irrelevant literature by reading titles and abstracts, 34 articles were retained. After careful full-text reading by two researchers based on inclusion and exclusion criteria, 13 articles were finally included, comprising 6 prospective studies and 7 retrospective studies. The literature screening process is shown in Fig. 1, and the basic characteristics of included studies are shown in Table 1.

Fig. 1.

PRISMA flow-chart for the systematic review and meta-analysis

Table 1.

Characteristics and extracted data from included studies

Authors	Study period	Country	Study design	Population	AI-assisted diagnostic methods	Golden standard	Sample size		Mean/median Age	Gender
							Positive	Negative		Male	Female
Abhilash Rakkunedeth Hareendrananthan	2021–2022	Canada	prospective	0–6 m	CNN	ultrasound	1568	619	3 (0–6)m	NA	NA
Jia Sha	2014–2022	China	retrospective	0-12Y	CNN	X-rays	140	222	(3.93 ± 2.61)y	208	154
Bingxuan Huang	2021–2022	China	prospective	0–6 m	DDHnet	ultrasound	321	48	3(0–6)m	161	25
Hiroki Den	2009–2021	Japan.	retrospective	0–12 m	YOLOv5	X-rays	30	17	6(0–12)m	NA	NA
Bangming Gong	2017–2019	China	prospective	0–6 m	TML-DNN	ultrasound	379	379	3(0–6)m	NA	NA
Siyavash Ghasseminia	2012–2020	Canada	retrospective	0–6 m	3D ultrasound	ultrasound	165	2327	87(4-267)d	502	1061
Jingnan He	2019–2022	China	prospective	0–5 m	AI and 3D us	ultrasound	48	168	75(0-150)d	83	133
AYSUN SEZER	2011–2016	France	prospective	0–6 m	CNN	ultrasound	228	447	3(0–6)m	NA	NA
Hyoung Suk Park	2011–2018	Korea	retrospective	0–12 m	New Convolutional Neural Network Algorithm	X-rays	50	436	6(0–12)m	NA	NA
Maki Kinugasa	2016–2021	Japan.	retrospective	0–6 m	SqueezeNet、MobileNet、EfficientNet	ultrasound	64	262	3(0–6)m	NA	NA
S-C.Zhang	2014–2018	China	retrospective	10d-10y	FR-DDH	X-rays	289	1987	1.5y	242	896
Siyavash Ghasseminia	2012–2020	Canada	retrospective	4-267d	MEDO Hip	ultrasound	59	66	61d	70	170
Mohammad Fraiwan	2021–2022	Jordan	prospective	3.67–7 m	SqueezeNet、GoogLeNet、Inceptionv3、DenseNet-201、MobileNetv2、ShuffleNet、Resnet101 、Resnet50、Resnet18、Xception、Inception-ResNet-v2、ShuffleNet、DarkNet-53、EfficientNet-b0	X-rays	120	234	(4.5 ± 0.83)m	NA	NA

Quality assessment of included studies

The quality of the 13 included studies [8, 9, 11–21] was assessed using the QUADAS-2 scale. Detailed quality assessments are shown in Fig. 2.

Fig. 2.

Risk of bias and applicability concerns summary. indicates Low; and indicates Unclear

Meta-analysis results

The 13 included studies reported a total of 10,673 samples, with 3,461 samples in the DDH case group and a prevalence of 32.4%. The overall performance of AI-assisted imaging diagnosis showed an overall sensitivity, specificity, AUC, and DOR of 99.0% (95% CI: 97.0-100.0%), 94.0% (95% CI: 89.0–96.0%), 99.0% (95% CI: 98.0-100.0%), and 1342 (95% CI: 469–3842), respectively. The overall forest plot showed significant heterogeneity in both sensitivity and specificity, with p = 0.00 and I² = 95.05% (95% CI: 93.92–96.18%) for sensitivity and p = 0.00 and I² = 97.47% (95% CI: 97.01–97.94%) for specificity. For publication bias, Deeks’ funnel plot was used for assessment. In this study, the included studies were distributed on both sides of the regression line, with a P-value of 0.11, indicating no potential publication bias. Figure 3 shows the forest plot, Fig. 4 shows the DOR plot, and Fig. 5 represents the SROC for all datasets, with an AUC value of 0.99% (95% CI: 98.0-100.0%).

Fig. 3.

The forest map of the overall sensitivity, specificity, and CI confidence interval of artificial intelligence assisted imaging for DDH diagnosis in the published research

Fig. 4.

Funnel plot for sensitivity and specificity analysis. CI, confidence interval

Fig. 5.

Summarize the characteristic curves of artificial intelligence assisted imaging for DDH diagnosis

Subgroup analysis results showed that the diagnostic efficacy of AI-assisted imaging for DDH in Asia [AUC = 99.0, 95% CI: 98.0-100.0] was higher than in Europe [AUC = 98.0, 95% CI: 96.0–99.0]; the diagnostic efficacy of retrospective studies [AUC = 98.0, 95% CI: 97.0–99.0] was higher than prospective studies [AUC = 97.0, 95% CI: 95.0–98.0]; the diagnostic efficacy of AI for X-ray [AUC = 100.0, 95% CI: 98.0-100.0] was higher than ultrasound [AUC = 99.0, 95% CI: 98.0-100.0]; the diagnostic efficacy for children over 6 months of age [AUC = 98.0, 95% CI: 96.0–99.0] was higher than for children under 6 months [AUC = 99.0, 95% CI: 98.0-100.0]; among AI models, the diagnostic efficacy of CNN models [AUC = 99.0, 95% CI: 98.0-100.0] was higher than non-CNN models [AUC = 97.0, 95% CI: 96.0–98.0] (Table 2).

Table 2.

Sensitivity, specificity, AUC, and DOR values of subgroup analysis using artificial intelligence assisted imaging for DDH diagnosis

Subgroup	Diagnostic value of artificial intelligence-assisted imaging in developmental dysplasia of the hip
	Study	Sensitivity[95%CI]	Specifity[95%CI]	AUC[95%CI]	DOR[95%CI]
Total	28	0.99[0.97-1.00]	0.94[0.89–0.96]	0.99[0.98-1.00]	1342[469–3842]
Region
Asia	24	0.99[0.97-1.00]	0.95[0.91–0.97]	0.99[0.98-1.00]	1600[593–4317]
Europe	4	0.98[0.79-1.00]	0.82[0.42–0.97]	0.98[0.96–0.99]	287[8-10852]
Study design
Prospective	19	0.99[0.98-1.00]	0.90[0.87–0.92]	0.97[0.95–0.98]	1446[397–5262]
Retrospective	9	0.96[0.93–0.98]	0.99[0.89-1.00]	0.98[0.97–0.99]	1764[127-24555]
Golden standard
X-ray	18	0.99[0.98-1.00]	0.93[0.88–0.96]	1.00[0.98-1.00]	1514[613–3741]
US	10	0.98[0.91-1.00]	0.95[0.81–0.99]	0.99[0.98-1.00]	936[86-10185]
Mean/median age
≥ 6 m	4	0.96[0.92–0.98]	0.99[0.99-1.00]	1.00[0.99-1.00]	3881[972-15493]
<6 m	24	0.99[0.98-1.00]	0.91[0.86–0.95]	0.99[0.98-1.00]	1394[381–5108]
AI-model
CNN	18	1.00[0.99-1.00]	0.92[0.88–0.95]	0.99[0.98-1.00]	2321[785–6864]
Non-CNN	10	0.94[0.89–0.97]	0.97[0.84–0.99]	0.97[0.96–0.98]	447[57-3487]

Discussion

In recent years, artificial intelligence technology has developed rapidly, providing the possibility of its application in DDH diagnosis through image recognition and classification functions. Many studies have shown considerably high accuracy, equivalent to or better than humans [22–24]. Sezer et al.‘s [13] study showed that a fully automated computer-aided diagnosis system using convolutional neural networks (CNN) classified and automatically segmented hip ultrasound images captured in the Graf standard plane, classifying images based on image features with an accuracy as high as 97.9%, sensitivity of 96.17% (95% CI: 92.85-98.23%), and specificity of 98.02% (95% CI: 96.39-99.05%). Another study [17] applied the CNN-based AI model “SN-APR” to diagnose DDH in anteroposterior pelvic X-rays, finding that the model’s area under the ROC curve, accuracy, sensitivity, and specificity for quality assessment were 0.993, 99.4% (360/362), 98.6% (138/140), and 100% (222/222), respectively, considering the AI-assisted method to be more efficient and highly consistent with expert clinical opinions.

The image detection capabilities of artificial intelligence have been reported in multiple medical fields [25–27]. This article is the first to conduct a meta-analysis of the diagnostic efficacy of AI-assisted imaging applied to DDH. In this systematic review and meta-analysis, the efficacy of different AI models for early DDH diagnosis was synthesized, demonstrating the high value of AI-assisted imaging in early DDH diagnosis. The overall sensitivity, specificity, AUC value, and DOR value were 99.0% (95% CI: 97.0-100.0%), 94.0% (95% CI: 89.0–96.0%), 99.0% (95% CI: 98.0-100.0%), and 1342 (95% CI: 469–3842), respectively. Kinugasa M et al. [20] from Japan used the MATLAB deep learning toolbox (MathWorks, Natick, MA, US) in a retrospective study to establish SqueezeNet, MobileNet_v2, and EfficientNet, three pre-trained models for transfer learning, and assessed the accuracy of the models using confusion matrices. In each model, the highest scores for accuracy, precision, recall, and F-value were all 1.0, considering that deep learning ultrasound imaging could evaluate DDH with high precision. In S-C. Zhang et al.‘s [19] study, the area under the receiver operating characteristic curve, sensitivity, and specificity of the deep learning system for diagnosing DDH were 0.975, 95.5%, and 99.5%, respectively, highly consistent with clinician-led diagnosis in diagnosing DDH, and more convenient and effective. The overall sensitivity and AUC value in our study were higher than their study, while the specificity was lower than the above-mentioned studies.

In terms of subgroup analysis, the focus was on AUC values for diagnostic efficacy. The overall AUC was 99%, and the AUC values between subgroups in our study did not differ greatly, all above 97%, but there were slight differences within individual subgroups. For example, the AUC value was 99% in the Asian population but only 98% in the European population, indicating better diagnostic efficacy of AI-assisted diagnosis in the Asian population. The diagnostic efficacy of prospective studies (97%) was lower than retrospective studies (98%), considering that the quality of evidence from prospective studies is higher than retrospective studies, so this study may overestimate the diagnostic efficacy of AI assistance to some extent. CNN has recently been successfully applied in medical image processing [20], and some studies have pointed out that this technology is the purest and most advanced visual analysis method constructed by humans to date [13]. It consists of four types of layers: convolutional layers, activation layers, pooling layers, and fully connected layers, each with different roles in completing the intended task. Its data-driven approach has been proposed for automatic image segmentation and classification in hip examinations. Golan et al. [28] used CNN to generate two probability maps for the ilium and acetabular roof, respectively, for plotting and calculating the ɑ angle. Hareendranathan et al. [29] developed an automatic hip scanning system based on CNN using four landmarks in the Graf classification: iliac crest, acetabular labrum, ischium, and femoral head. These studies all demonstrate the feasibility and effectiveness of CNN for AI-assisted imaging in DDH ultrasound diagnosis. Another study [30] applied a CNN-based deep learning algorithm to automatically measure the Sharp angle in hip X-ray images based on X-ray diagnostic criteria. Currently, this model has become the main model in AI-assisted imaging, and its diagnostic efficacy is superior to other non-CNN model groups.

The surgical efficacy of DDH in children is closely related to the timing of surgery. Studies have found that early diagnosis and treatment of DDH in infancy can achieve or approach normal pediatric hip joints [5, 6]. The diagnostic methods differ according to the patient’s age [31]. Hip ultrasound examination (for infants under 6 months) and X-ray (for children over 6 months) are the gold standards for early DDH diagnosis in children in clinical practice [32, 33]. Both examination methods strongly depend on the operator’s experience and technique, and the variability of scanned images is high, which may produce considerable measurement errors, thus affecting the accuracy of diagnosis to a certain extent. In recent years, with the development of computer science, the performance advantages of artificial intelligence in image recognition, data analysis, and clinical decision-making have been demonstrated by numerous studies [34–36]. In terms of DDH-assisted diagnosis, AI has constructed diagnostic models that can accurately identify DDH ultrasound or X-ray image features by segmenting main anatomical structures or extracting quantitative features to measure relevant angles and classify them, automatically evaluating images during scanning to improve scan quality [37], effectively compensating for or overcoming the deficiencies of gold standards in diagnosis, significantly improving diagnostic accuracy [38], and reducing false positive and false negative rates in the diagnostic process, thereby improving the overall diagnostic efficacy of DDH.

This study still has some limitations: (1) Some of the included studies were retrospective, which may lead to potential uncontrollable bias risks. Subgroup analysis found that the diagnostic efficacy of prospective studies was weaker than retrospective studies, so the interpretation of the overall diagnostic efficacy in this study needs to consider the impact of retrospective studies. (2) Most studies were from Asia and Europe, with no data from the Americas, Africa, Oceania, and other regions to confirm the value of AI models in assisted diagnosis. Therefore, it is unclear whether AI-assisted imaging technology can be extended to other regions globally, requiring further studies for confirmation. (3) In our study, artificial intelligence was mainly applied to assist in the diagnosis of two gold standards: ultrasound imaging and X-ray, which also led to differences in the study population. In addition, there were many and scattered model types, all of which can cause high overall heterogeneity. Furthermore, apart from CNN models, other models could not be compared through subgroup analysis due to data limitations. It is hoped that more large-sample, prospective, multicenter clinical studies will be available in the future to further evaluate the value of AI in early DDH diagnosis.

Conclusion

This study demonstrates that artificial intelligence (AI)-assisted imaging technology has high accuracy for DDH diagnosis and can significantly improve the diagnostic efficacy of DDH diagnostic gold standards such as X-ray and ultrasound. Among them, the CNN model has particularly significant advantages in DDH-assisted diagnosis. More large-sample, prospective, multicenter clinical studies are needed in the future to further evaluate the value of AI in early DDH diagnosis.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author contributions

All authors contributed to the study conception and design. Min Chen: Conceptualization, Methodology, Software, Writing- Original draft, Data curation, Visualization were performed; Ruyi Cai and Aixia Zhang and Xia Chi: Investigation, Writing - Original Draft, Writing - Reviewing and Editing were performed; Jun Qian: Conceptualization, Supervision, Project administration, Funding acquisition were performed. All authors read and approved the final manuscript.

Funding

This study was supported by the research project fund of Jiangsu Maternal and Child Health Association [grant number FYX202212].

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethical approval

Not applicable.

Conflict of interest

The authors declare that there are no conflicts of interest.