The Comparison of Convolutional Neural Networks and the Manual Measurement of Cobb Angle in Adolescent Idiopathic Scoliosis

Abstract

Study Design

Comparative study

Objective

To compare manual and deep learning-based automated measurement of Cobb angle in adolescent idiopathic scoliosis.

Methods

We proposed a fully automated framework to measure the Cobb angle of AIS patients. Whole-spine images of 500 AIS individuals were collected. 200 digital radiographic (DR) images were labeled manually as training set, and the remaining 300 images were used to validate by mean absolute error (MAE), Pearson or spearman correlation coefficients, and intra/interclass correlation coefficients (ICCs). The relationship between accuracy of vertebral boundary identification and the subjective image quality score was evaluated.

Results

The PT, MT, and TL/L Cobb angles were measured by the automated framework within 300 milliseconds. Remarkable 2.92° MAE, .967 ICC, and high correlation coefficient (r = .972) were obtained for the major curve. The MAEs of PT, MT, and TL/L were 3.04°, 2.72°, and 2.53°, respectively. The ICCs of these 3 curves were .936, .977, and .964, respectively. 88.7% (266/300) of cases had a difference range of ±5°, with 84.3% (253/300) for PT, 89.7% (269/300) for MT, and 93.0% (279/300) for TL/L. The decreased bone/soft tissue contrast (2.94 vs 3.26; P=.039) and bone sharpness (2.97 vs 3.35; P=.029) were identified in the images with MAE exceeding 5°.

Conclusion

The fully automated framework not only identifies the vertebral boundaries, vertebral sequences, the upper/lower end vertebras and apical vertebra, but also calculates the Cobb angle of PT, MT, and TL/L curves sequentially. The framework would shed new light on the assessment of AIS curvature.

Introduction

Adolescent idiopathic scoliosis (AIS) is 1 of the most common spinal deformities and affects millions of children worldwide. ¹ The Cobb ² method, introduced in 1948, has been widely considered as the ‘gold standard’ to estimate the severity of scoliosis in patients. The magnitude of the Cobb angle is directly related to the diagnosis and the decision-making in the watchful waiting, bracing, or surgery, and even the selection of fusion segments for AIS patients. ³ However, the traditional approach is regarded to have an interobserver variance ranging from 4-8° and to be time-inefficient, because of the need to draw a vertical line at the upper/lower end vertebra endplate. ⁴ New innovative approaches, such as the end vertebra tilt angle method and the smartphone-aided measuring, have drawn endplate lines to measure Cobb angles electronically. Altogether, these reported methods have not achieved the spinal curve measurements in automation and have not provided comprehensive assessments.^4-6 As such, an automated calculation and comprehensive assessments in the Cobb angle is worthy of being consideration and would be useful in clinical practice.

With the development of image processing technology and artificial intelligence,^7-9 the calculation of the Cobb angle via multiview correlation networks and deep learning approaches is promising method in obtaining the Cobb angle measurement.^10-12 A fully automated analysis of the spine shape has been proposed using deep learning models with vertebral end plate centers, hip joint centers, and margins of the S1 end plate labeled. ¹¹ Although this method measured the measurement of the major curve of the Cobb angle, the sequence of vertebral body, end vertebrae and apical vertebra were unavailable.

As such, artificial neural networks and oscillograms of end-plate distribution forming an automated framework in the measurement of Cobb angles was proposed in this study, which provides a fully automated and reliable solution for comprehensive spinal curvature assessment (Supplemental video1). This includes the proximal thoracic (PT) curve, the main thoracic (MT) curve, and the thoracolumbar/lumbar (TL/L) curve, as well as the identification of end vertebras and apical vertebra in the clinical workflow. The proposed method was designed to accomplish 3 tasks automatically: 1) the identification of vertebral boundaries and vertebral sequences, 2) the recognition of upper and lower end vertebras, apical vertebra, and 3) the estimation of the Cobb angles of PT, MT, and TL/L curves. Additionally, the convolutional neural networks were compared to the manual measurements in the AIS Cobb angle and confirmed that convolutional neural networks had an efficient performance in practice.

Materials and Methods

Data Harvest

This non-interventional prospective study obtained ethical committee approval (HX-2020-741) in September 2020. Written informed consent has been obtained from all patients. X-ray images from 500 AIS patients were acquired from the picture archive and communication system of our hospital. The diagnosis of AIS is based on radiographic image characteristics, natural history, and clinical presentation. Notably, congenital scoliosis, neuromuscular disorders and syndromic disorders have been excluded, as well as abnormal vertebral body and irregular sequence. ³ Patients with only 1 preoperative anteroposterior (AP) whole-spine radiography, were included. Data desensitization was performed before further use.

The Labelling of the Boundary Vertices of the Vertebra

A senior spine surgeon (LM L) selected 200 AP whole-spine radiographies independently to ensure adequate inclusion of all Lenke-type curves (Figure 1A). The AP whole-spine radiographs were labeled by a fellowship-trained spine surgeon (XM H) using specially developed software. The 4 boundary vertices of each vertebral body from T1 to L5 were labeled. Different colors were used to distinguish the vertebral sequences (Figure 1B). All labels were re-examined by an attending spine surgeon (M L), who adjusted any inappropriate locations before the labels were used for training the convolutional neural networks.

Figure 1.

200 anteroposterior whole-spine radiographies were selected (A) and the boundary vertices of the vertebra from T1 to L5 were labeled manually (B). The labeled images were used for training convolutional neural networks (C). The Cobb angle of PT, MT, and TL/L curves were identified according to the oscillogram (D). 300 untrained whole-spine AP radiographs were used to evaluate the reliability of the automated method.

The Identification of the Boundary Vertices Automatically via Convolutional Neural Networks

With the import of the format of digital imaging and communications in medicine (DICOM) images, the pre-processing was conducted with the developed software, including the identification of the largest connected area by the morphological operations and the crop of the human foreground area. The grayscale of input image was resized to 1024 × 512 (Figure 2A). Additionally, the largest external rectangle of the vertebral mask was calculated, and vertebral bodies were segmented using HRNet. ¹³ The HRNet maintained high-resolution representations by connecting parallel representations of different resolutions and repeated multiscale fusion. The backbone of the network was drawn from ResNet101, which improved training effect of deep neural networks and achieved maximum accuracy and efficiency with 101 layers. ¹⁴ The sizes of the feature maps are presented as height × width × channels. The skip connections combined the shallow and deep layers through the skip connection module (Figure 2B). The keypoint-based detection model was developed to automatically identify 4 boundary vertices of each vertebra from T1 to L5 via convolutional neural networks (Figure 2C). We managed to augment the training set via horizontal flip, rotating ±15°, randomizing brightness and contrast, cutout, adding pixels.

Figure 2.

Identifying the largest connected area using morphological operations and cropping the human foreground area, the grayscale of input image is resized to 1024 × 512 (A). The backbone of the network is from ResNet101. The sizes of the feature maps are presented as height × width × channels. E and D represent encoder and decoder, respectively (B). The 4 boundary vertices of each vertebra from T1 to L5 were identified automatically via convolutional neural networks (C).

The Calculation of the Cobb Angle in Oscillograms

Using the keypoint-based detection model, 68 boundary vertices of the 17 vertebras were automatically identified. To determine the boundary of a single vertebral body, 4 labelled vertices with the same color were grouped. One endplate line was established by connecting the 2 upper/lower boundary vertices. Therefore, 34 endplate lines were determined by this method for 12 thoracic vertebrae and 5 lumbar vertebrae.

Furthermore, the software automatically calculated the horizontal-inclination angle of each endplate line, where 0° presents horizontal, + presents left-oblique and - presents the right-oblique. An oscillogram was adopted with the vertebra sequence as the abscissa and the endplate inclination angle as the ordinate. The upper/lower end vertebras were defined as the most inclined endplate lines and the apical vertebra (or intervertebral disc) was defined as inclination angle most close to 0°. The end vertebras were corresponded to the vertices with the largest absolute values of ordinates in the oscillogram, including the upper and lower end vertebras. If there are parallel vertebras in upper/lower end, the end vertebra should be the proximal/distal 1. The difference value between the maximum and minimum ordinate values in the oscillogram indicated the Cobb angle of the major curve. According to the inflexion vertices of the oscillogram and the vertebral sequences from the keypoint-based detection model, the program measured the Cobb angles of PT, MT, and TL/L curves and identified the major curve simultaneously and accurately (Figure 1D). It is available at the website: https://www.medspine.net/#/login.

Reliability Evaluation

Intraobserver and Interobserver Reliability

A senior spine surgeon (LM L) selected 50 original radiographs from the 200 whole-spine AP images that were previously labeled. The patient’s information was desensitized, and the other 2 observers (M L and GJ F)were blinded to all previous results. To assess intraobserver reliability, the attending surgeon (M L) measured the Cobb angle of the major curve twice at an interval of 2 weeks, and the intraclass correlation coefficient (ICC) for the same observer was .977. To evaluate interobserver reliability, the attending surgeon (M L)and senior surgeon (GJ F) measured the Cobb angle of the major curve independently, and the interobserver ICC was .969. The ICC values indicated an optimal agreement for the manual evaluators.

The Evaluation of the Reliability of Automated Measurements

The automated method we proposed was tested and compared with the manual method to evaluate its accuracy. A senior spine surgeon (LM L) selected 300 untrained whole-spine AP radiographs independently to ensure adequate inclusion of all Lenke-type curves, and the patients’ information was desensitized. The manual method was carried out by the attending spine surgeon (M L) with Surgimap (version 2.3.2), a reliable software for spinal measurement.^15,16 With the import of the whole-spine AP radiographs, the measurements of PT, MT, and TL/L were outputted automatically. The measurements were recorded by a software developer with no clinical experience in measuring Cobb angle. The mean absolute error (MAE), Pearson or spearman correlation coefficient, and ICC were used to evaluate agreement between automated and manual methods.

Assessments of Image Quality

To determine the influence of image quality on the accuracy of key point identification, image quality of the major curve with MAEs exceeding 5° was assessed by pairing images with the same gender and age from the remainings with MAE≤5°. The quality of the images were evaluated on 6 dimensions, including bone/soft tissue contrast, bone sharpness, visibility of the processus spinosus, delignation of the intervertebral spaces, assessment of the spinal curve, and assessment of the Risser grade. ¹⁷ 1 attending and 1 fellowship-trained spine surgeons (M L, XM H) independently assessed the 50 images. Differences were resolved by consensus with the participation of a senior spine surgeon (X Y).

Statistical Analysis

Statistical analysis was performed with R software (version 3.5.3) and MedCalc software (version 19.0.7). Data were compared between 2 groups with the use of t-tests for continuous variables and chi-square tests for categorical variables. Intraobserver and interobserver reliabilities for manual evaluators were estimated by calculating the ICC with corresponding 95% confidence interval (CI). ¹⁸ The consistency of the Cobb angle derived by automatic and manual methods was assessed by MAE, Bland-Altman plot, ICC, and Pearson or spearman correlation analyses.

Results

Demographic Data

Whole-spine images from 500 AIS individuals were collected. 200 of them were labeled manually to train the keypoint-based detection model, and the remaining 300 images were used to evaluate the reliability of the automated measurement method. The average age of the AIS patients was 16.2 years, and 84% of them were female. The Cobb angle of the major curve was 39.4°, with 19.7° for the PT curve, 36.5° for the MT curve, and 28.4° for the TL/L curve (Table 1).

Table 1.

Demographic data of 500 adolescent idiopathic scoliosis patients.

Variable	Mean ± SD	Range
Age (year)	16.2 ± 3.1	10 to 24
Female	421 (84%)	-
Major curve (degree)	39.4 ± 14.7	11.3 to 92.5
Proximal thoracic curve (degree)	19.7 ± 8.2	.1 to 49.8
Main thoracic curve (degree)	36.5 ± 13.9	8.3 to 92.5
Thoracolumbar/lumbar curve (degree)	28.4 ± 10.1	8.1 to72.8

Oscillograms of Different Lenke Types of AIS

The oscillogram was plotted by the tilt angle of 34 endplates (T1 to L5), and the oscillograms of different Lenke-type curves represented unique characteristics. The oscillogram of Lenke 1 curve was characterized as sinusoid. The difference between peak and trough indicates the Cobb angle of the main thoracic curve and the major curve (Figure 3A). Compared to Lenke 1, the oscillogram of Lenke 2 curve had an extra trough at the initial segment, which represented a structural PT curve (Figure 3B). The oscillogram of Lenke 3 curve was similar to Lenke 6 curve, but the distinction lies in the 2 successional differences between the peak and trough, which indicated the MT and TL/L, respectively (Figure 3C and F). The oscillogram of Lenke 4 curve was similar to Lenke 2 curves, but the distinction lies in the larger difference between the trough and the second peak in the Lenke 4 curve, which indicated a structural TL/L (Figure 3D). The oscillogram of Lenke 5 curve was characterized by a peak and a trough at the posterior segment, which indicated a structural TL/L curve (Figure 3E).

Figure 3.

Oscillograms of different Lenke types of adolescent idiopathic scoliosis.

Consistency Between Automatic and Manual Methods

The automatic method achieved to measure the PT, MT, and TL/L Cobb angles within 300 milliseconds after the image was imported. The MAEs, Bland-Altman plots, Pearson or spearman correlation coefficients, and ICCs were used to evaluate the reliability of the automated measurement method. 2.92° MAE and .967 ICC for the major curve was obtained. The MAEs of PT, MT, and TL/L were 3.04°, 2.72°, and 2.53°, respectively (Table 2). The ICCs of these 3 curves were .936, .977, and .964, respectively (Table 3). In the subgroup analysis of the major curve, the ICCs of 10°-20°, 20°-40°, 40°-60°, and >60° were .580, .851, .824, and .729, respectively (Table 3). The Bland-Altman plots showed the distribution of the differences between the 2 methods. For the consistency of the major curve, difference value of 88.7% (266/300) cases were within 5°, with 84.3% (253/300) for PT, 89.7% (269/300) for MT, and 93.0% (279/300) for TL/L (Figure 4). The results of the automatic method were strongly associated to the manual method with an excellent correlation coefficient (r = .972) for the Cobb angle of the major curve, with .934, .982, and .960 for PT, MT, and TL/L curves, respectively (Figure 5).

Table 2.

Mean absolute error of automated and manual methods.

Variable	Mean ± SD	Range
Major curve (degree)	2.92 ± 1.75	0 to 8.9
Proximal thoracic curve (degree)	3.04 ± 1.94	0 to 8.1
Main thoracic curve (degree)	2.72 ± 1.74	0 to 8.9
Thoracolumbar/lumbar curve (degree)	2.53 ± 1.60	0 to 7.6

Table 3.

Intraclass Correlation Coefficient of automated and manual methods.

Variable	ICC (95% CI)	P Value
Major curve total (n = 300)	.967 (.957, .975)	<.001
10-20° (n = 17)	.580 (.160, .824)	.006
20-40° (n = 147)	.851 (.785, .896)	<.001
40-60° (n = 113)	.824 (.706, .890)	<.001
>60° (n = 23)	.729 (.273, .894)	<.001
Proximal thoracic curve	.936 (.918, .950)	<.001
Main thoracic curve	.977 (.963, .984)	<.001
Thoracolumbar/lumbar curve	.964 (.955, .971)	<.001

Figure 4.

The Bland-Altman plots showed the distribution of the differences between automatic and manual methods.

Figure 5.

Pearson or spearman correlation analyses between automatic and manual methods.

Assessment of Image Quality

To verify whether the image quality has influenced on the assessment accuracy, 34 images with MAEs over 5° and the equivalent ones under 5° were compared. Decreased overall scores (17.76 vs 19.35; P=.008), bone/soft tissue contrast (2.94 vs 3.26; P = .039), and bone sharpness (2.97 vs 3.35; P = .029) were found in the images with MAEs over 5°. There were no significant differences in the visibility of the processus spinosus, delignation of the intervertebral spaces, assessment of the spinal curve, or assessment of the Risser grade between the 2 groups (Table 4).

Table 4.

Comparison of image quality between MAE>5° and MAE≤5°.

Variable	MAE>5 (n = 34)	MAE≤5 (n = 34)	P Value
Overall score	17.76 ± 2.57	19.35 ± 2.24	.008
Bone/soft tissue contrast	2.94 ± .65	3.26 ± .62	.039
Bone sharpness	2.97 ± .80	3.35 ± .60	.029
Visibility of processus spinosus	2.44 ± .77	3.03 ± .58	.094
Delignation of the intervertebral spaces	2.94 ± .74	3.12 ± .77	.337
Assessment of the spinal curve	3.44 ± .75	3.62 ± .55	.271
Assessment of the risser grade	2.74 ± .83	2.97 ± .76	.226

Discussion

We proposed a fully automated measurement of Cobb angle in AIS patients using convolutional neural networks. Convolutional neural networks identified the boundary vertices of vertebral bodies by the encoder-decoder framework and provided efficient spinal curvature measurements via oscillograms automatically. The Cobb angles of PT, MT, and TL/L curve were measured simultaneously within 300 milliseconds. A reliability evaluation with 300 AIS patients showed an impressive 2.92° MAE in the Cobb angle of major curves. Our method, therefore, provides spinal surgeons with an assessment framework of spinal curvature in AIS patients.

Satisfactory consistency was obtained for the fully automated measurement framework through convolutional neural networks. We evaluated the accuracy of automated methods with 4 indicators: MAEs, Bland-Altman plots, ICCs, and Pearson or spearman correlation coefficients. The Cobb angle of major curves determined the management strategy for AIS patients. ³ Therefore, the Cobb angle is considered to be the most important radiological parameter in scoliosis. The fuzzy Hough transform was proposed to find line structures in the vertebral edge images, and high agreement between automatic and manual measurements was achieved (ICC>.95). ¹⁰ Sardjono et al. ¹⁹ tested the accuracy of an automatic Cobb angle determination method with an MAE of 3.91°. Similarly, Wu et al. ¹¹ proposed a fully automated spinal curvature assessment using MVC-Net, achieving an MAE of 4.04°. Recently, an automated method was presented based on deep learning approaches by extracting anatomical parameters from biplanar radiographs of the spine. The standard error of the Cobb angle in scoliosis was as high as 9.9°. ¹² In our study, the reliability evaluation with 300 images showed impressive 2.92° MAE, .967 ICC, 89% coincidence rate, and excellent correlation coefficient (r = .972) for major curves. Compared with previous studies, our method obtained higher measurement accuracy.

Accurate identification of vertebral boundary vertices ensures the accuracy of automated measurements. A recent study reported a spline construct from vertebra centroids, which obtained superior reliability compared to the traditional Cobb method. ²⁰ Considering that with the aggravation of the lateral curvature, the vertebral body may appear as a wedge change, which will lead to nonparallelization of the upper and lower endplates. ²¹ Therefore, this study did not adopt the median line of the vertebral body to assess the inclination angle of the vertebral body. Instead, the 4 boundary vertices of the vertebral body are the key to automatic image recognition. First, 68 vertebral boundary vertices representing 12 thoracic and 5 lumbar vertebras, and the order of T1 to L5, were labeled manually. In all, 200 X-ray images from AIS patients were labeled for training the convolutional neural networks. Then, the vertebral boundary point and vertebral sequence of the imported images are automatically recognized using the trained networks. The identification of vertebral boundary vertices is the cornerstone for the inclined angle measurement of the vertebral endplate in the next step.

The oscillogram model is another important part of ensuring the accuracy of automated measurement. After the identification of boundary vertices based on convolutional neural networks, the computer program determines the upper and lower endplates of each vertebral body based on the 4 boundary vertices, and thereby determines 34 endplate lines for the 17 vertebras. The software automatically measured the horizontal inclination angle of the 34 endplate lines and plots the oscillogram. The peaks and troughs of the oscillogram represent the vertebral endplate with the greatest inclination angle. Compared with the previous local measurement model, our oscillogram model truly realized comprehensive and accurate quantification of the upper and lower endplates of the vertebras from T1 to L5, which avoided intraobserver and interobserver instability of selecting end vertebra. If the endplates of end vertebra are parallel, then the peak/trough is horizontal where an additional vertebra should be included in the curve. In addition, this method measured PT, MT, and TL/L at 1 time within 300 milliseconds. The oscillogram procedure seems to be more related to the principle of the Cobb angle in characterizing the severity of the curvature.

To explore the influence of image quality on the accuracy of automatic measurement, we evaluated image quality and found that the scores for bone/soft tissue contrast and bone sharpness were significantly lower for MAEs exceeding 5°. The identification of vertebral boundary vertices is the basis of the oscillogram, and it is integral for accurate automatic measurement. Identification of vertebral boundaries would be hampered if the vertebral contours were not well defined with sharp bones. Compared with MT and TL/L, PT has the worst consistency among the 4 indicators, including MAE, ICC, coincidence rate, and correlation coefficient. One possible reason for this finding is the interference of anatomical structures, including the first and second ribs, the clavicle, the costovertebral joints, aorta and even the mediastinum, resulted in a blurred boundary of the upper thoracic vertebras.

Some limitations of our work have to be considered. This automatic measurement method is only applicable to AIS, and congenital scoliosis is not applicable for the time being. Our method is based on the 4 boundary vertices of the vertebral body, whose accuracy is dependent on identifying vertebral boundary vertices. Unfortunately, congenital scoliosis has complex vertebral abnormalities, including multiple hemivertebrae, poor vertebral formation, or segmental insufficiency, which will reduce the accuracy for identifying vertebral boundary vertices. Second, most of the lateral bends included in our study are moderate partly due to limited data on severe AIS in our database, and partly because of severe lateral bends and severe vertebral body tilt. Currently, we are still conducting targeted algorithm optimization. Third, this preliminary study does not include X-ray images with metal artifacts, and potential degradation in landmark accuracy might occur when processing images with metal artifacts caused by spinal bracing. Fourth, vertebral body rotation and pelvic tilt which will significantly alter the assessment are not taken into consideration at present, because the segmentation of pedicle root and spinous process is difficult. Last, assessment of lateral view is absent in our study. For a comprehensive evaluation of scoliosis, further study should combine anteroposterior and lateral view.

Conclusions

Interobserver variance and the time-consuming nature of manual methods to assess Cobb angle are baffling to clinicians. We, therefore, proposed a fully automated framework to measure the Cobb angle in AIS patients using convolutional neural networks, which identified vertebral boundaries and vertebral sequences, recognized apical vertebra, upper and lower end vertebras, and estimated Cobb angles of PT, MT, and TL/L curves sequentially. A reliability evaluation with 300 AIS patients showed an impressive 2.92° MAE in the Cobb angle of the major curves in an average of 300 milliseconds. Compared to non-automated softwares, such as surgimap and smartphone software, the automated method provides a more efficient, objective and time-saving framework in the assessment of spinal curvature to clinicians and a potential auto diagnostic system to adolescents and families. Additionally, this would reduce the diagnostic errors from the fatigue in clinical practice.

ORCID iDs

Xianming Huang, MMed https://orcid.org/0000-0002-2332-3457

Xi Yang, MD https://orcid.org/0000-0001-7925-3323

Zhongjie Zhou, MD https://orcid.org/0000-0003-4041-1958

Yueming Song, MD https://orcid.org/0000-0002-2377-0740

Zongke Zhou, MD, PhD https://orcid.org/0000-0002-9037-4756

Shishu Huang, PhD https://orcid.org/0000-0002-5373-103X