Comparison of Transfer Learning Models in Pelvic Tilt and Rotation Measurement in Pediatric Anteroposterior Pelvic Radiographs

Abstract

The rotation and tilt of the pelvis during anteroposterior pelvic radiography can lead to misdiagnosis of developmental dysplasia of the hip (DDH) in children. At present, no method exists for accurately and conveniently measuring the precise rotation and tilt angles of pelvic on radiographs. The objective of this study was to develop several rotation and tilt measurement models using transfer learning and digital reconstructed radiographs (DRRs), and to compare their performances on pelvic radiographs. Based on the inclusion criteria, 30 of 92 children who underwent 3D hip CT scans at Xijing Hospital from 2015 to 2020 were included in the study. Using DRR techniques, radiographs were generated by rotating and tilting the pelvis in CT datasets at − 12 to 12° (projected every 3°) and were randomized to a 2:1:1 ratio of training dataset, validation dataset, and test dataset. Five pre-trained networks, including VGG16, Xception, VGG19, ResNet50 and InceptionV3 were used to develop pelvic rotation measurement models and tilt measurement models, and these models were trained with training dataset. The callback function was used during the training to slow down the learning rate when learning was stalled. Then, the validation set was used to optimize each model and compare their performances. At last, we tested the final performances of optimal rotation measurement model and optimal tilt measurement model on test dataset. The mean absolute error (MAE) was employed to assess the performance of the models. A total of 2430 pelvic DRRs were collected based on 30 CT datasets. Among 5 pre-trained transfer learning models, VGG16-Tilt achieved the best tilt prediction performance at the same BS and different LR. VGG16-Tilt model achieved its best performance on validation set at LR = 0.001 and BS = 4, and the final MAE on the test set was 0.5250°. In terms of rotation prediction, VGG16-Rotation also achieved the best performance, and it achieved its best performance on validation set at LR = 0.002 and BS = 8. The final MAE of VGG16-Rotation on the test set was 1.0731°. Pretrained transfer learning models worked well in predicting tilt and rotation angles of the pelvis on radiographs in children. Among them, VGG16-Tilt and VGG16-Rotation had the best effect in dealing with such problems despite their simple structures. These models deployed in devices can give orthopedic surgeons a powerful aid in DDH diagnosis.

Introduction

Developmental dysplasia of the hip (DDH) is a common bone and joint disease in children during their growth and development [1]. If not diagnosed in time, DDH is highly likely to cause disability, which may seriously affect the patient’s life quality [2, 3]. At present, the auxiliary diagnosis of DDH mainly relies on ultrasound, X-ray, and CT [4, 5]. Ultrasound is a commonly used modality in diagnosing DDH in infancy, and the most common analysis of ultrasound images is based on the Graf measurement method, but several studies have shown that it lacks reproducibility and leads to overdiagnosis of mild DDH [6–8]. DDH is in fact a 3D deformity of the acetabulum and CT offers the ability to more fully evaluate 3D hip deformity than 2D imaging [9]. CT is primarily used in operative planning to assess angles and lines for optimized 3D correction, and preoperative assessment includes evaluation of bony acetabular morphology and the ossified femoral epiphysis as well as the femoral head position relative to the acetabulum [10–12]. CT has excellent sensitivity and specificity in the diagnosis and treatment of DDH, but due to its high cost, low availability, and radiation dose involved, it is not a primary imaging modality in DDH management [11, 13]. Anteroposterior pelvic X-ray examination is of great significance in the diagnosis of DDH [14–16]. In order to measure the correct parameters on the pelvic radiographs to make the correct diagnosis, the pelvis should be placed in a neutral position at the time of radiography [17, 18]. However, due to the difficulty of children’s coordination during the radiography, once the shooting position is tilted or rotated, it may have a significant impact on the measurement of the acetabular index and other pelvic parameters [19]. Some previous studies have shown that the acceptability of an X-ray film can be roughly evaluated by measuring some indexes in the X-ray film, such as the ratio between the vertical diameter and horizontal diameter of the pelvic foramen, and the ratio between both horizontal diameters of the obturator foramen [20–22]. For now, however, there is no method to directly measure the exact rotation and tilt angle of the pelvis in a radiograph.

Benefit from the advances in artificial intelligence technology, many medical fields have made remarkable progress. Recently, some deep learning models have been developed to mark points to aid DDH diagnosis [23, 24]. However, due to the need for adequate pelvic radiographs with precise tilt and rotation angle labels, to develop an artificial intelligence model to predict tilt and rotation of the pelvis on a radiograph remains a challenge. Fortunately, the data shortage problem can be solved by digitally reconstructed radiographs (DRRs) technology [25]. These images generated by DRRs from CT data with accurate tilt and rotation labels have been proven to represent X-rays [17, 19, 26].

The aim of this study is to develop five transfer learning models that can measure pelvic tilt and rotation angles and compare their performance, and to help orthopedic surgeons make more accurate judgments in DDH screening.

Materials and Methods

Patients

We retrospectively collected the medical records of all children who underwent pelvic CT examination in Xijing hospital from January 2018 to December 2020 due to preoperative preparation or suspected DDH. According to the inclusion criteria, 30 of the 92 cases met the inclusion criteria, including 5 males and 25 females, with an average age of 4.9 years. The inclusion criteria were as follows: (1) Patients were between 0.5 and 12 years of age; (2) neuromuscular disease or chromosomal abnormalities were excluded; (3) there was no history of pelvic or hip surgery; (4) the children and their family agreed that their medical information was included in the study, and (5) CT data included a complete pelvis. The study design was evaluated by a panel of seven experts, including three pediatric orthopedic experts, a radiology expert, and an artificial intelligence expert. Each CT examination was evaluated and approved by at least three pediatric orthopedic specialists and one radiologist before being performed. The Ethics Committee of Xijing Hospital approved this study.

In the use of CT for young children, radiologists in our hospital use low-dose technology and adjust equipment parameters according to radiological protection requirements for X-ray computed tomography (GBZ 165–2012) and requirements for radiological protection in diagnostic radiology (GBZ 130–2020), so that the radiation dose meets the national requirements for CT dose for children under 1 year old, 1–5 years old, and 5–10 years old, respectively [27, 28].

Image Preparation

The image data of all medical records were obtained by CT scanning (United Imaging Healthcare, China), and some scanning parameters included slice thickness (1.0 mm), spacing between slices (0.8 mm), and pixel spacing (0.529 × 0.529 mm²). We first placed the pelvis into the standard neutral position and then simulated the CT data into anteroposterior pelvic radiographs through a 3D algorithm. Using DRR technique, each CT generated 81 radiographs at various tilt and rotation angles (from − 12 to 12°, projected every 3°) (Fig. 1), and a total of 2430 DRRs with exact pelvic rotation and tilt angle labels were collected. The standard neutral position refers to the following: (1) the line of the bilateral anterior superior iliac spines was perpendicular to the sagittal plane; (2) the line of the superior borders of bilateral obturator formamen was parallel to horizontal plane; and (3) the line between the midpoint of bilateral anterior superior iliac spine and the symphysis pubis is parallel to the coronal plane [17].

Fig. 1.

a Shows the DRR generated at neutral pelvis position. b–f Show DRRs generated at different pelvic positions (b posteriorly tilt to 3° and rotate to left 9°, c anteriorly tilt to 6° and rotate to right 12°, d anteriorly tilt to 3° and rotate to right 3°, e anteriorly tilt to 9° and rotate to left 9°, and f posteriorly tilt to 12° and rotate to right 6°). The key landmarks are as follows: (1) right tri-radiate cartilage center, (2) left tri-radiate cartilage center, (3) right acetabulum superolateral margin, (4) left acetabulum superolateral margin

Comparison of 5 Transfer Learning Models

We employed five classic deep learning models, including VGG16, Xception, VGG19, ResNet50, and Inception, which were pretrained on ImageNet [29–31]. Based on these, we constructed five pelvic tilt angle measurement models VGG16-Tilt, Xception-Tilt, VGG19-Tilt, ResNet-Tilt, and Inception-Tilt, and pelvic rotation measurement models VGG16-Rotation, Xception-Rotation, VGG19-Rotation (Fig. 2), ResNet-Rotation, and Inception-Rotation. The callback function was used during training to slow down the learning rate when learning was stalled.

Fig. 2.

This figure shows the architecture of VGG19-Rotation

We randomly divided the image set into training set (1216 images) (Fig. 3a), validation set (607 images) (Fig. 3b), and test set (607 images) (Fig. 3c) in a ratio of 2:1:1 (random seed: 1020) and trained these models with training set. First, we compared their performance on the validation set after training under the same batch size (BS) and different learning rate (LR). Then, we selected the best transfer learning model, trained it under different BS and LR, and optimized its hyperparameters on the validation set. Finally, the performance of optimized tilt angle prediction model and rotation prediction model was evaluated on the test set.

Fig. 3.

The data distribution of the training dataset, validation dataset and test dataset are shown in a–c respectively. The values on the X-axis in each image represent the rotation angle of the pelvis, and the values on the Y-axis in each image represent the tilt angle of the pelvis. The color of each cell represents the number of DRRs at different pelvic tilt and rotation angles in the training dataset (a), validation dataset (b), and test dataset (c)

Statistical Analysis

We used two parameters, mean square error (MSE) and mean absolute error (MAE), to train the models and evaluate the performance, respectively; MSE is the mean of the square of the difference between the predicted value and the true value, while MAE is the mean of the absolute difference between the predicted value and the true value. The smaller these two values, the better the performance of the models. All data operations were performed on TensorFlow 2.0.0 (Google Inc., USA).

Results

Comparison of Tilt Prediction Models

The performance of VGG16-Tilt, Xception-Tilt, VGG19-Tilt, ResNet-Tilt, and Inception-Tilt under different LR and the same BS on validation set is shown in Table 1, and we found that VGG16-Tilt performed best among the five models under the same LR and BS. Then, the hyperparameters of the VGG16-Tilt model was optimized (Table 2). And it can be observed that when LR was 0.001 and BS was 4, the VGG16-Tilt model achieved the best MAE of 0.52° on the validation set. Plots of training and validation loss and MAE for LR = 0.001 and BS = 4 are shown in Fig. 4a and b, respectively. Finally, the optimized VGG16-Tilt model was tested on the test set with MAE of 0.53°.

Table 1.

The performance (MAE) of VGG16-Tilt, Xception-Tilt, VGG19-Tilt, ResNet-Tilt and Inception-Tilt under different LR and the same BS

Model	Learning Rate
	0.0005	0.001	0.002	0.004	0.008	0.016	0.032
VGG16-Tilt	0.6708	0.7039	0.6168	0.5713	0.5411	0.8447	0.7860
VGG19-Tilt	1.0078	0.8482	0.6957	0.7699	0.7779	0.8811	0.8627
Xception-Tilt	4.8909	4.7638	4.5846	4.8207	4.9112	4.6655	4.7355
ResNet-Tilt	6.4643	6.6933	6.8011	6.5973	6.9104	6.7506	6.5426
Inception-Tilt	6.5922	7.0963	6.3183	6.1878	6.6489	7.0123	6.9215

MAE mean absolute error, LR learning rate, BS batch size

Table 2.

The performance (MAE) of VGG16-Tilt under different LR and BS

BS	LR
	0.0005	0.001	0.002	0.004	0.008	0.016	0.032
4	0.6692	0.5242	0.5413	0.5244	0.7121	0.8327	6.6804
8	0.7489	0.6699	0.6972	0.5811	0.5950	0.6295	0.7453
16	0.6708	0.7039	0.6168	0.5713	0.5411	0.8447	0.7860
32	0.7747	0.7969	0.7460	0.6813	0.6061	0.8125	0.9208

MAE mean absolute error, LR learning rate, BS batch size

Fig. 4.

The plot of training and validation loss value for LR = 0.001 and BS = 4 for VGG16-Tilt was shown in a. The plot of training and validation MAE for LR = 0.001 and BS = 4 for VGG16-Tilt was shown in b. The plot of training and validation loss value for LR = 0.002 and BS = 8 for VGG16-Rotation was shown in c. The plot of training and validation MAE for LR = 0.002 and BS = 8 for VGG16-Rotation was shown in d

Comparison of Rotation Prediction Models

The performance of VGG16-Rotation, Xception-Rotation, VGG19-Rotation, ResNet-Rotation,, and Inception-Rotation under different LR and the same BS on validation set is shown in Table 3, and we found that VGG16-Rotation performed best among the five models under the same LR and BS. Then, the hyperparameters of the VGG16-Rotation model was optimized (Table 4). And it can be observed that when LR was 0.002 and BS was 8, the VGG16-Rotation model achieved the best MAE of 1.07° on the validation set. Plots of training and validation loss and MAE for LR = 0.002 and BS = 8 are shown in Fig. 4c and d, respectively. Finally, the optimized VGG16-Tilt model was tested on the test set with MAE of 1.07°.

Table 3.

The performance (MAE) of VGG16-Rotation, Xception-Rotation, VGG19-Rotation, ResNet-Rotation and Inception-Rotation under different LR and the same BS

Model	LR
	0.0005	0.001	0.002	0.004	0.008	0.016	0.032
VGG16-Rotation	1.3670	1.1945	1.1409	1.0765	1.3318	1.5378	6.4778
VGG19- Rotation	1.4515	1.3845	1.4494	1.3398	1.6892	6.7619	6.7613
Xception- Rotation	7.8316	8.0849	7.7972	7.6336	7.6046	7.8234	7.7518
ResNet- Rotation	7.4168	7.5025	8.0625	8.1800	7.6673	7.5521	7.7662
Inception- Rotation	8.6304	8.4231	8.2010	8.5767	9.0607	8.0913	8.8230

MAE mean absolute error, LR learning rate, BS batch size

Table 4.

The performance (MAE) of VGG16-Rotation under different LR and BS

BS	LR
	0.0005	0.001	0.002	0.004	0.008	0.016	0.032
4	1.2634	1.1129	1.1010	1.1323	1.2315	6.4684	6.4674
8	1.3074	1.1552	1.0675	1.2206	1.5295	6.4660	6.4661
16	1.3670	1.7945	1.1409	1.0765	1.3318	1.5378	6.4778
32	2.2233	1.4547	1.3583	1.3757	1.4994	1.5117	1.3749

MAE mean absolute error, LR learning rate, BS batch size

Discussions

One of the key factors affecting the prognosis of DDH is diagnosis, and anteroposterior pelvic radiograph is one of the key tools for DDH diagnosis. During X-ray radiography, the patient is required to lie quietly on a platform and maintain a neutral pelvis position, which is difficult for most children in clinical practice. Therefore, the pelvis in most children’s radiographs is more or less rotated and tilted, and these changes will have a significant impact on the measurement of relevant parameters, such as acetabular index [19], thus affecting the accuracy of diagnosis.

This problem has been studied in several previous papers. Portinaro [32] found that pelvic tilt and rotation of more than 10° and 5°, respectively, affected acetabular index measurement in four cases. Yang [17] found that less than 6° was acceptable when considering just one condition of pelvic rotation or tilt. Van der Bom [19] suggested that a limit of 4° was acceptable when considering both pelvic tilt and rotation. But previous studies, limited by data-processing capabilities, only performed traditional analyses on a few medical records or a collection of 100 or so pelvic radiographs to come up with an acceptable range of pelvic tilt and rotation angles. What’s more, there is by far no method to measure the tilt and rotation angles of the pelvis directly on a radiograph, which can be used as a reference for orthopedic surgeons to make diagnosis. In our study, DRR technology and transfer learning models were adopted. Not only expanded the pelvic tilt and rotation radiograph set to more than 2000 pictures, we also constructed and compared five kinds of pelvic rotation or tilt angle measurement models. After optimization, the model with the best performance on the validation set finally achieved a tilt measurement accuracy of 0.53° and a rotation measurement accuracy of 1.07° on the test set.

In order to improve the training speed of the deep learning model, we used the transfer learning method. This technology allowed us to reduce the trainable parameters of the model and reduce the model training time by freezing a large number of pre-trained parameters based on the ImageNet dataset while maintaining the model size. In addition to this technology, in order to reduce overfitting, we also used a callback function to actively reduce LR when learning stagnated. Interestingly, VGG-16 and VGG-19 were simpler in structure than Xception, ResNet50, and Inception, but they performed surprisingly well. More importantly, VGG-16, which has three layers smaller than VGG-19, achieved the best performance under the same LR and BS. This may be explained by the fact that when transfer learning models were used to solve the problem of tilt and rotation measurement, the over-fitting of the model with too many layers is more likely to occur. Or the structure of a series of VGG models itself has a unique advantage in solving such problems.

Our study still has some limitations. First, due to strict inclusion criteria, although 92 cases were initially collected, only 30 cases were eventually included. Using multicenter studies to expand the data set may further improve accuracy and reduce overfitting. Second, the male to female ratio of patients included in the medical records was about 1:4, which seemed a little imbalance, but for DDH, it was consistent with the reality that male patients account for 20% [16]. Third, due to the limitation of computational power, we only compared the performance of five models under transfer learning conditions. We hope other groups with greater computational power may join this study and compare the performance of the transfer learning model and the ab initio model. Finally, we only used five classical transfer learning models, which did not mean that other transfer learning models would not achieve better performance on this problem.

Conclusions

The performance of five transfer learning models in measuring pelvic tilt and rotation angles in children’s pelvic radiographs were compared in this study. Although VGG16-Tilt and VGG19-Tilt had simple structures, they performed best in dealing this problem. These models, deployed in devices, can help orthopedic surgeons’ diagnosis DDH.

Abbreviations

BS

Batch size

CNN

Convolutional neural network

DDH

Developmental dysplasia of the hip

DRR

Digital reconstructed radiograph

LR

Learning rate

MAE

Mean absolute error

MSE

Mean square error

ReLU

Rectified linear unit

VGG

Visual geometry group

Author Contribution

HLY, CH, ZJ and LC contributed to the study design. LC, YYB, and XHF contributed to the data analysis and drafted the manuscript. HLY directed data collection and provided administrative support for the project. All authors contributed to interpretation of the data, commented on the manuscript, revised the manuscript, revised the manuscript, and approved the final version for publication.

Funding

This study was supported in part by China National Natural Science Foundation [81171735], Shaanxi Natural Science Foundation [2017JC2-04] and Fourth Military Medical University.

Data Availability

The data are available from the corresponding author upon reasonable request and with permission from the Xijing Hospital, Fourth Military Medical University.

Declarations

Ethics Approval

Approval was obtained from the Ethics Committee of the Xijing Hospital of Fourth Military Medical University. The procedures used in this study adhere to the tenets of the Declaration of Helsinki.

Consent to Participate

Informed consent was obtained from all individual participants and their legal guardians included in the study.

Competing Interests

The authors declare no competing interests.