A Review of the Methods on Cobb Angle Measurements for Spinal Curvature

Abstract

Scoliosis is a common disease of the spine and requires regular monitoring due to its progressive properties. A preferred indicator to assess scoliosis is by the Cobb angle, which is currently measured either manually by the relevant medical staff or semi-automatically, aided by a computer. These methods are not only labor-intensive but also vary in precision by the inter-observer and intra-observer. Therefore, a reliable and convenient method is urgently needed. With the development of computer vision and deep learning, it is possible to automatically calculate the Cobb angles by processing X-ray or CT/MR/US images. In this paper, the research progress of Cobb angle measurement in recent years is reviewed from the perspectives of computer vision and deep learning. By comparing the measurement effects of typical methods, their advantages and disadvantages are analyzed. Finally, the key issues and their development trends are also discussed.

1. Introduction

Spinal deformity refers to the appearance of morphological abnormalities in the coronal, sagittal, or axial position of the spine that deviate from the normal position. In 2014, Li, Y.W. et al. [1] screened 15,000 children who were randomly selected in the Luohe area of Henan Province, China, for spinal deformities and found that nearly two out of every 1000 children in the area suffered from congenital spinal deformities. The incidence of acquired spinal deformities is even more alarming. In the case of scoliosis, for example, epidemiological surveys show that more than 2% of adults in China and the United States suffer from scoliosis, and most of the onset occurs after adolescence [2,3].

Adolescent idiopathic scoliosis (AIS) is mostly mild and does not cause major health problems. However, there is a 10–20 percent chance that minor scoliosis will worsen further [4], while severe scoliosis can affect breathing [5]. Due to the progressive properties of scoliosis and the significantly higher probability of postoperative complications in adult patients than in adolescent patients, medical imaging diagnosis and screening for scoliosis are important [6].

In order to scientifically assess the progression of the disease, a quantitative analysis of scoliosis is a necessary step in scoliosis treatment. Compared to other methods of assessing spinal curvature, the Cobb angle method is preferred for its better reproducibility [7], easier application, and suitability for measuring more severe spinal curvature. In 1966, the Scoliosis Research Society (SRS) adopted the Cobb angle as a standard method for quantifying scoliosis deformities. It remains the most commonly used method for assessing the curvature of the spine to date. The schematic of the Cobb angle is shown in Figure 1.

Figure 1.

Schematic of Cobb angle measurements: the Cobb angle is defined as the angle between the extension line of the upper end plate of the most inclined vertebral body in a curved segment and the extension line of the lower end plate of the most inclined vertebral body below.

2. The Challenges of Cobb Angle Measurement

Clinically, a more than 5° change in Cobb angle can represent the progression of the disease course, which, in turn, affects the formulation of the doctor’s diagnosis and treatment plan. However, Jeffries, B.F. et al. [8] noted that a 5° change in Cobb angle might be affected by measurements rather than by an improvement or exacerbation of the course of the disease. Yang, W.Z. [9]’s research supported this opinion by pointing out the high inter- and intra-operator variability during the Cobb angle measurement. The sources of errors may be equipment errors, image errors, and subjective errors in the operator’s selection of the vertebral end. Therefore, computer-aided methods are considered superior in terms of reliability [10,11,12,13] because they can remove subjective errors. There are some computer-aided methods that measure spinal curvature without the help of traditional medical imaging (X-ray, CT, MRI, etc.), including body surface image screening [14,15] and motor system examination [16]. However, these methods either only allow for early screening or are too expensive to monitor the condition regularly. Therefore, traditional medical imaging is still an indispensable auxiliary tool in scoliosis treatment. However, scoliosis measurement software, which has a high market share at present, such as Surgimap [17], uses semi-automatic measurement methods. The operator needs to manually select the upper and lower ends of the vertebral body. Clearly, there is a huge market gap for commercial software for fully automatic measurement of Cobb angles using traditional medical imaging.

With the increasing demand for intelligent medical devices, automatic Cobb angle measurement is expected to assist in solving more complex problems in realistic medical scenarios and, therefore, faces more challenges. For example, during surgery, fully automated Cobb measurements can be used to assist with bone fixation. However, these methods suffer from parallax problems due to the narrow field of view for imaging equipment in the operating room. Additionally, in the regular Cobb angle review of the postoperative population, metal implants are prone to metal artifacts in 3D images and affect the robustness of the measurement method.

The rest of this paper is organized as follows. Several typical medical situations using automatic Cobb angle measurement and the preprocessing methods designed for them will be introduced in Section 3. In Section 4 and Section 5, we will introduce several automatic measurement methods developed for 2D and 3D images. In the last two sections, we will discuss the shortcomings of the current methods and future directions for the fully automated measurement of Cobb angles.

3. Typical Application Scenarios & Preprocessing Methods

Many factors can affect the imaging quality of medical images, so we selected several typical scenarios and summarized three preprocessing methods. These are considered to be effective in improving the robustness of automatic algorithms.

3.1. PDE Filters Used for Feature Enhancement

In X-ray images of the spine, the edges of the vertebral body are often degenerated and difficult to discern. The partial differential equation filter identifies the edges by calculating the gradient of the image (the area with a large gradient is most likely to be the edge) and smooths the image of the area, except the edge, hoping to strengthen the weak edge of the vertebral body and suppress image noise. Osher and Rudin et al. [18] proposed an iterative PDE to suppress noise. However, the inflection points of noise may be mistaken for edges. In response to this problem, Alvarez and Mazorra [19] proposed a shock-diffusion model with an anisotropic diffusion term. The proposed method provided a sharp cutoff at the edge, while the image was regularized. However, with a small standard deviation, the smoothing of the method will be insufficient. To solve the above problem, Gilboa et al. [20,21] improved their diffusion function to propose a complex diffusion partial differential equation filter based on the standard impact diffusion model.

Duong L et al. [22] compared these PDE models in terms of their noise reduction effects on spine images. The standard PDE model enhanced the noise while enhancing the vertebral body. Images processed by anisotropic diffusion PDE models were somewhat degenerate at the edges of the vertebral body. The complex diffusion PDE model performed best, not only effectively enhancing the vertebral body to eliminate noise but also minimizing the degeneration of the vertebral body edges.

Duong, L.et al. [22] also compared the classification accuracy of the SVM classifier before and after using the PDE filter. It was found that the accuracy of classifying the original image was between 50.2% and 84.0% without denoising, and the classification accuracy increased to 65.6–90.7% after using the PDE filter.

3.2. Preprocessing Methods Used to Deal with Parallax

Automated measurement of the Cobb angle can be used during surgery to confirm the restoration of the functional alignment of the spine. The X-ray machine used specifically for surgery usually has a narrow field of view [23], and the image of the spine is longer than the machine’s field of view, so it is necessary to stitch multiple captured images to form a complete image of the spine during surgery. Yaniv, Z. [24] proposed a method of moving the C-arm parallel to the front for image stitching to obtain panoramic images. However, he did not address the parallax problem during image registration. Methods of stitching X-ray images can be divided into intensity-based registration and feature-based registration. Intensity-based registration relies on large overlapping areas, thus increasing the radiation dose received by patients and medical staff. In response to this, Wang, L. [25] proposed a moving C-arm shooting method that does not require overlapping, thus reducing the number of shots and reducing the amount of radiation. The feature-based registration method is limited by the fact that the feature intensities of different images taken with X-rays may be completely different, making the stitching difficult. Since the spine consists of multiple vertebral bodies that cannot be completely parallel to the calibration plane, parallax effects can cause measurement errors in the image. There are manifold methods [26], multi-view methods [27], and pure rotating camera methods [28] to eliminate parallax, but they all have certain drawbacks.

Medtronic’s O-arm mobile 3D Cone-Beam CT imagers exhibit high intra- and inter-observer agreement. [29]. However, these images were of poor quality in obese patients and at the cervicothoracic junction. The 2D long film function in Medtronic’s latest O-arm has shown better performance in splicing imaging [30].

3.3. Preprocessing Methods Used for Metal Artifacts Reduction (MAR)

The corrective treatment of scoliosis is sometimes assisted by metal implants. Metals tend to cause metal artifacts in 3D images. MRI is not commonly used for spinal curvature measurement (unless spinal cord injury is suspected), while US is less disturbed by metal implants, and metal occlusion can be avoided by multi-angle scanning. However, metallic implants are prone to high-amplitude hardening artifacts in CT [31]. Therefore, algorithms have been proposed to eliminate metal artifacts in CT images. Metal artifact removal is usually carried out by segmenting the area affected by the artifact in the sinogram and replacing it with synthetic values, such as interpolation [32,33] and iterative reconstruction [34,35]. The interpolation method has high parameter requirements and can easily lead to the blurring of structures near the implant. The iterative reconstruction method has high computational complexity. Some unsupervised deep learning algorithms, such as AND-net [36] and CycleGAN [37], were introduced. Although good results have been achieved, unsupervised algorithms do not always produce filling information that conforms to the anatomical structure and are prone to secondary artifacts. Additionally, small metal implants tend to disappear in high-rise networks [38]. In addition, the existing MAR algorithms are all data-driven, so the dataset is extremely critical. However, when limited by the actual situation, it is impossible to find datasets that can implement supervised algorithms. Therefore, some researchers have introduced metal masks to images without metal artifacts to generate metal artifact images, conduct supervised training, and achieve good results [39,40].

4. Evaluation Methods of Cobb Angle in 2D Images

Due to its controllable radiation dose and low cost [41], X-ray is still the main way to assess scoliosis progression, and so it is used as the main medical imaging method for Cobb angle measurement. There are two main categories for the current commonly used methods of computer-aided measurements, which are image-enhancement-based methods and machine-learning-based methods.

4.1. Image Enhancement Method for 2D Image

X-rays of the spine are characterized by large noise, blurry edges, and possible occlusion between structures. Therefore, the image enhancement method is committed to extracting the features we need through means such as filtering. The biggest challenge the researchers encountered was how to segment the vertebral body from the image background.

4.1.1. Semi-Automatic Method

In earlier studies, because the vertebral body boundary is blurred and difficult to segment, researchers often use a semi-automatic method. Chockalingam, N. et al. [42] proposed a multi-segmented measurement using AP images. (Unless otherwise specified, all the 2D automatic measurement methods below use AP images). The program automatically divides the area of the spine (manually selected by the operator) into eight equal segments. The program automatically fits the midline of the spine according to the intersection of each parallel line and the spinal area that the operator manually identifies and then calculates the angle. An example diagram of its method is shown in Figure 2. Chockalingam, N. tested the intra-observer error and inter-observer error of the octa segmentation method and compared it with the manual method. The octave method has an excellent performance in the Technical Error of Measurement (TEM) and Average Reliability Coefficient (R). Table 1 shows the statistical measures for variability of the methods.

Figure 2.

Schematic of octa segmentation method proposed by Chockalingam, N. et al. [42].

Table 1.

Statistical measures for variability of the methods.

Measure	Definition
Technical Error of Measurement (TEM)	$TEM=\frac{{\sum }_1^N[{\sum }_1^{K}M{\left(n\right)}^2-\frac{{\left({\sum }_1^{K}M\left(n\right)\right)}^2}{K}]}{N\left(K-1\right)}$
Standard Deviation (SD)	$SD=\sqrt{\frac{1}{N}{\displaystyle {\displaystyle \sum }_{i=1}^N}\left(m_i-\overline{m}\right)}$
Average Reliability Coefficient (R)	$R=1-\left[{\left(TEM\right)}^2/{\left(SD\right)}^2\right]$
Intra- or Inter-class Correlation Co-efficient (ICC)	$ICC=\frac{S{D}_{\left\{intra or inter\right\}}^2}{S{D}_{intra}^2+S{D}_{inter}^2}$
Mean Absolute Difference (MAD)	$MAD=\frac{1}{N}{\displaystyle {\displaystyle \sum }_{i=1}^N}\left|m_i-{\widehat{m}}_i\right|$
Mean Square Error (MSE)	$MSE=\frac{{\sum }_{i=1}^n{\left(m_i-\overline{m}\right)}^2}{N}$

However, Chockalingam, N.’s method [42] can neither find the vertebral body with the largest tilt angle nor be sensitive in mild scoliosis scenarios. In order to measure the Cobb angle more accurately on more occasions, segmentation of the vertebral body is still unavoidable. Samuvel, B. et al. [43] proposed an automated Cobb angle measurement method for segmenting the vertebral body from the image using a mask. Yang, D. et al. [44] designed a two-stage, semi-automatic, ultrasound curve angle measurement method for adolescent idiopathic scoliosis. The method was composed of two processes: manual identification of spinal transverse processes and automatic angle measurement based on manually placed masks. The results showed that the proposed method had almost the same effect as the conventional method.

4.1.2. Full-Automatic Method

Semi-automatic methods perform similarly to or better than manual methods. However, it is difficult for semi-automatic methods to remove subjectivity, which affects the measurement accuracy. There have been some works using image enhancement methods for the full-automatic measurement of Cobb angles. Some researchers calculate the Cobb angle by fitting the midline of the spine, while others measure it by extracting the upper and lower ends of the vertebral body. A very intuitive idea is to enhance the edges in the image and then abstract the irregular vertebral edges into quadrilateral shapes to separate the upper and lower ends of the vertebrae. Zhang, J. et al. [45] used Canny Operator to enhance the edges, making it easier for Hough transform to extract vertebral bodies from images. However, the Canny operator not only strengthens the edges of the vertebral body but also strengthens the edges of unrelated structures. Therefore, some researchers consider suppressing noise with filters. Anitha, H. et al. [46] proposed a method for image enhancement using three filters which are used for noise suppression, region of interest preservation, and edge extraction. This method automatically separates the vertebral ends and measures the Cobb angle. Comparing Anitha, H.’s method and manual measurement from the three dimensions of mean, standard deviation, and error, from the perspective of within and between observers, it was found that the former is better than the latter.

Using filters may preserve some interference features, such as some of the images of the rib with high grayscale. Some researchers have proposed using the active shape model (ASM) to fit the edges of the vertebral body. Williams, D.J. and Shah, M. [47] proposed an active contour model, the snake model, and successfully segmented the vertebral body from the image. However, the snake model has higher requirements for the initial setup. If the initial position of the snake deviates too much from the target, there is a possibility that the snake will not be pulled towards the boundary. Anitha, H. et al. [48] used the GVF snake model to search for the edge of the spine. They replaced the external force with a gradient vector flow (GVF) force in the snake model, which can be defined by the equilibrium solution of the vector diffusion equation. GVF force has fewer requirements for the initial position. As a result, the model acts better in terms of edge extraction. Figure 3 shows the result of Anitha H et al.’s [48] processing of the spine images with the GVF-snake model.

Figure 3.

The X-ray images processed by the GVF-snake model [48].

Although the GVF-snake model reduces the active shape model’s dependence on the initial setup, the researchers found that, in some areas with blurred boundaries, GVF-snakes are prone to losing features, especially when the vertebral bodies obscure each other. Roberts, M.G. et al. [49] proposed an active apparent model (AAM) for the identification of spinal vertebral bodies. The active apparent model has two advantages over the active shape model: (1) simple constraints are placed on the parameters, and the resulting shape remains within a reasonable range; (2) a global shape model is used to simulate the correlation between different parts implicitly. Therefore, even if some of the details are occluded, the approximate shape of the occluded part can be estimated by AAM. However, the main idea of its search method is based on the assumption of a linear relationship between the difference between the texture of the model and the area enclosed by the shape and the change in the parameters. This assumption is only reliable within a certain range of bias, and its coverage is highly dependent on the training set. Therefore, the robustness of this method is limited when searching.

In scoliosis cases, the vertebral body may have concomitant aberrations, but neither the active shape model nor the active apparent model can handle the topological changes in the underlying shape. Jalba, A.C. and Wilkinson, M. et al. [50] proposed a shape deformation method based on particle systems: the CPM algorithm. The CPM algorithm treats the grayscale image as an electric field, and the amount of negative charge in the electric field is proportional to the image gradient. A large number of positively charged particles that can be moved freely in the system are gradually concentrated at the edges under the influence of the Lorentz force and the Coulomb force. Sardjono, T.A. et al. [51] improved the CPM algorithm by limiting the direction of movement of the particles and using it to look for the left and right edges of the vertebral body. The improved CPM algorithm performs better for the segmentation of the spinal vertebral body than conventional CPM and GVF-snake models. Since the CPM model does not depend on the initial position of the particles, allowing particles to cross a portion of the boundary freely, and the electric field acts over longer distances relative to other energy fields, it can perform better when the underlying shape topology transformation occurs. Another advantage of CPM is that it can adapt to many unknown topologies and complex shapes, such as edges with large curvature and unclosed profiles. As particles are only subject to force and are not constrained by motion and morphology, they show good robustness in the edge capture of unknown shapes and complex shapes.

4.2. Machine Learning Methods for 2D Image

4.2.1. Segment the Vertebral Body to Measure the Cobb Angle

While the image enhancement method has successfully measured Cobb angles on some spine images, it requires precise feature engineering. Thus, there are disadvantages, such as its high computational cost and unstable performance on different images. Machine learning methods can construct a classifier to map a given dataset into several given categories. Some researchers use machine learning methods to segment spine images from X-ray images and then calculate the Cobb angle through spine midline fitting or angle regression network.

Kusuma, B.A. [52] uses a Canny filter to extract the edges of the vertebral body and cooperate with K-means clustering to find the centroid of the spine segment to fit the midline of the spine. To avoid interference from unrelated structures such as ribs, Duong, L. [53] delineated a region of interest (ROI) for each vertebral body in the training set (where ROI is defined as an area that includes the corresponding vertebral body but is slightly larger than each vertebral body). Then, a mean model was calculated to estimate the areas of each vertebral body’s probability in the test set. A Soft-Margin SVM classifier was trained to segment the vertebral body image from each ROI. Finally, the program fitted the midline of the spine and calculated the Cobb angle by calculating the centroid of the vertebral body.

The successful application of the mean model to the selected vertebral ROI demonstrated the usefulness of prior knowledge of anatomy for segmenting medical imaging. However, scoliosis has great individual variability, and the reliability of the mean model is doubtful. Alharbi, R.H. et al. [54] trained convolutional neural networks (CNN) with transfer learning to calibrate ROIs for each spinal vertebral body. CNN performs better in terms of generalization capacity than the mean models. However, this method can only mark the approximate position of each vertebra with a frame, and a classifier is required to know the specific position of each vertebra and the rotation direction.

Ronneberger, O. et al. [55] proposed U-Net, which is a semantic segmentation network model based on FCN [56] structure. It is widely used in the field of medical imaging due to its suitability for small datasets and good performance in medical image segmentation. Some researchers have improved the U-net model and used it to measure the Cobb angle. Tu, Y. et al. [57] proposed a DU-net for spine contours’ segmentation. Wang Z. et al. [58] used Resnet to modify U-Net to increase network computing speed while ensuring segmentation accuracy. The structure of the proposed model is shown in Figure 4. The preprocessed image is divided by the improved U-Net into a binary image, where the white area is the vertebral body, and the black area is the non-vertebral body. Then, the minimum external matrix for each spinal vertebral body is calculated, which yields the upper and lower end plates of the vertebral body. Comparing the results of the scoliosis surgeon’s manual method with the improved U-net method, the ICC of both automatic and manual measurement results is higher than 0.96, and the MAD is about 3°. Error analysis also proves good consistency between automatic and manual measurement methods.

Figure 4.

The structure of the improved U-Net [55]: Compared with the original U-net, Wang Z. [58] has made two modifications. First, in order to reduce the amount of calculation, modify conv (3 × 3) to conv (1 × 1) + conv (3 × 3) + conv (1 × 1). In order to reduce the overall parameters, conv (3 × 3) is replaced with conv (1 × 1) when dealing with the same dimension and dimension reduction calculation.

Lin, Y. et al. [59] proposed a two-stage learning framework, Seg4Reg. Instead of just using deep learning to segment images, Seg4Reg takes the spine segmentation result as the input of the angle regression network and won first place in the spine image segmentation competition held by MICCAI.

4.2.2. Locate the Landmark of the Spine to Measure the Cobb Angle

We found that, as long as the accurate landmarks of the vertebral body are found, the segmentation can be replaced, and the Cobb angle can be directly estimated. Zhang, C. et al. [60], Chen, B. et al. [61], Kim, K.C. et al. [62], and Zhang, J. et al. [63] have proposed methods to find landmarks of the vertebral body. However, there are also two problems in calculating the Cobb angle by the vertebral landmark: (1) precise feature engineering is required due to image noise; (2) a small error in the landmark may lead to a large deviation of the Cobb angle. Therefore, researchers should not only consider the accuracy of landmark estimation but also consider the impact of reducing landmark errors on Cobb angle measurements.

Sun, H. et al. [64] improved the SVR method and proposed a new multi-output regression measurement Cobb angle scheme, S²VR. This method introduces manifold regularization into the framework and obtains a joint estimation of Cobb angles and spine landmarks using their high correlation. They compared the Cobb angle estimated by the S²VR method with the manually measured Cobb angle, with a Correlation Coefficient (ICC) of 0.923 for the middle angle and 0.884 and 0.902 for the remaining two angles, respectively. The framework of S²VR is shown in Figure 5.

Figure 5.

The framework of S²VR [64]: SVR is used to capture the angle and landmark output correlation structure matrix, improved to obtain S2VR. Sun, H. et al. [64] introduced manifold regularization and trained with kernel alignment method.

S²VR improves the accuracy of Cobb angle and landmark output by considering the explicit dependencies between multiple outputs. However, Wu, H. et al. [65] found that the accuracy of Cobb angle measurements was also affected by outliers in the training data (including subjective error and imaging artifacts). Therefore, they came up with a new framework for automated landmark evaluation: Boostnet. It combines convolutional neural networks with statistical methods. Not only is multi-output regression achieved, but the robustness of landmark estimation is improved. Boostnet uses a dependency matrix of adjacent landmarks to solve the problem of explicit dependencies between multiple outputs. Wu, H. et al. validated Boostnet’s ability to estimate spinal landmarks and found that its mean squared error (MSE) on the training set was only 0.0046. The structure of Boostnet is shown in Figure 6.

Figure 6.

The structure of Boostnet [65]. The Boostnet consists of three parts: a convolutional layer for feature extraction, a boostlayer for removing outliers, and a multi-output layer for relieving the pressure of small datasets by capturing inter-landmark dependencies.

Single-view images tend to miss some features due to the mutual occlusion of vertebral body structures. Wu, H. et al. [66] and Xu Q. [67] noted that multi-view images could complement the missing features of single-view images. Therefore, Wu, H. et al. designed a multi-perspective correlation network (MVC-net) using the X module to perform a weighted learning of feature maps of AP and LAT images to obtain joint features. For the phase-wrapping characteristics between the Cobb angle output and the landmark output, they designed a loss function, considering spinal landmark regression loss, spinal landmark-related loss, and Cobb angle output loss to minimize error. Xu, Q. [67] improved the loss function on the basis of the former and used extrapolation to obtain more accurate angle measurements. The structure of the MVC-net is shown in Figure 7.

Figure 7.

The structure of the MVC-net [67]. MVC-net consists of three parts: a convolutional layer for feature extraction (the convolutional layers are connected with X modules for feature joint learning); a spine landmark output layer and a Cobb angle output layer.

Considering the accumulation of errors in the two-stage framework and the fact that the cascaded network cannot guarantee global optimality, Yi Lin et al. [68] proposed Seg4Reg+ by combining the two methods of angle regression and landmark monitoring. Seg4Reg+ trains angle regression and landmark detection methods in parallel and optimizes them globally.

5. Evaluation Methods of Cobb Angle in 3D Images

Due to factors such as missing dimensions, measurement noise, and varied postures [69], 2D measurements face great challenges in terms of their accuracy and comprehensiveness. Yang W. et al. [70] measured Cobb angles on three-dimensional reconstructed spinal structures and on X-rays and found that the former has improved accuracy. As 3D imaging technology has become the mainstream means of detection, the methods of 3D Cobb angle measurement using CT, 3D Ultrasonic, or MRI images have gradually become abundant. CT exposes patients to larger doses of radiation [71], which have a negative effect on their health. MRI requires patients to be in a recumbent position, and due to gravity, images of the spine taken in a recumbent position are thought to be very different from those taken in a standing position [72,73,74,75]. Ultrasound imaging is considered especially suitable for adolescent idiopathic scoliosis (AIS) patients for its real-time, cost-effective, and radiation-free characteristics [76]. Although the measurement of 3D images has the characteristics of good continuity and comprehensive observation, due to its complexity for computation, a 3D ultrasound system is still unreliable for clinical scoliosis assessment [77]. Therefore, how to improve the degree of refinement and automation of measurement is also the focus of research by relevant scholars.

5.1. Image Enhancement Method for 3D Image

5.1.1. Semi-Automatic Method

Similar to the measurement method used for 2D images, identifying key points through image enhancement and feature detection and then using the prior knowledge of spine shape to fit the curved lines of the spine in a three-dimensional space is the most-used method at present.

Cheung, C.J. et al. [78] developed a freehand 3D ultrasound system for scoliosis curvature measurement. The method used the transverse process and spinous profile as the landmarks. Based on the coronal images, the Cobb angle was measured by volume projection imaging. Vo, Q.N. et al. [79] proposed a method for finding the maximum curvature plane in 3D ultrasound images and using it to measure spinal curvature. The method uses voxel-based reconstruction techniques and bilinear interpolation to reconstruct 3D spine images and then measures the axial rotation of the vertebral body on the 3D images to find the plane of maximal curvature.

5.1.2. Full-Automatic Method

The above methods, which rely on landmark detection, are often used for semi-automated measurements and do not effectively reduce the workload. The full-automatic method generally uses template matching for spinal curve detection.

Volume projection imaging (VPI) is a commonly used method in ultrasound imaging to draw the contours of spiny columns on the coronal surface using the averaging mixing method and to assess scoliosis. Zhou, G. et al. [80] reported an automated method for determining spinal curvature using the ultrasound images reconstructed by VPI. The method identifies spinous tips, using histogram equalization to reduce speckle noise and positioning the darkest points line by line in images after noise suppression. The detected points are then polynomial-fitted to a curve, and the curvature angle of the spine is calculated based on the inflection points on the curve. Polynomial fitting, to some extent, mitigates the effect of misidentified feature points on the measurement results. This method selects the point with the least intensity as the marker. Image intensity is affected by the singularity of the intensity distribution, and when the quality of ultrasound images is low, such as for people whose BMI is high, the accuracy of curve point detection is significantly reduced [81].

In addition, Zhou, G. et al. [82] proposed another method, using prior knowledge of vertebral anatomy to look for features in ultrasound volume projection imaging (VPI). They used a two-fold thresholding strategy to extract the skeletal features, which can obtain information on symmetrical and asymmetric metrics from phase consistency. The spinal contour is detected from the segmented spinal regions and can be used to extract the spinal curve for curvature measurements. However, the recognition accuracy and anti-interference ability of this method are far from satisfactory. In order to solve this problem, Zhou, G. et al. [83] proposed a fully automatic method that facilitates the segmentation of the spinous column profile by adaptively adjusting the frequency bandwidth of the log-Gabor filter to calculate the directional phase consistency.

Visualization techniques such as VPI primarily demonstrate spinal anatomy on the coronal surface, which are inadequately capable of evaluating three-dimensional spinal deformities. Li, D.S. et al. [84] developed a new method based on projection imaging to obtain sagittal images of spinal anatomy. Sagittal projection images and coronal surface VPIs are obtained by deriving spine curves from coronal images at different depths and constructing non-planar surfaces. As long as the sagittal projection image has been calculated, the Cobb angles can be measured automatically.

Some researchers look for markers, such as spinous processes (SP), on images of the spine and fit the three-dimensional curves used to represent the spine. Sagittal projection angle (SPA) is a very common quantitative indicator of three-dimensional spinal curvature. Zeng, H. et al. [85] applied a fully automated method to measure spinous process angle, which was verified to have a high consistency with the Cobb angle, on coronal spinal imagery. This method uses a gradient vector flow (GVF) snake model to locate the spinous process (SP) on ultrasound (US) images of the transverse vertebral body.

Zheng, R. et al. [86,87] proposed an automatic method to measure the Sagittal projection angle (SPA). The method consists of four steps, including spinous process detection, data point clustering, curve fitting, and SPA estimation. Compared with ultrasonic manual measurements, the results of the automated method have a higher correction rate and smaller differences. SPA makes full use of the three-dimensional features of the spine image, but spinal rotation, which is common in scoliosis cases, causes the spinous process to shift more towards the concave side of the spine. Therefore, Herzenberg, J.E. et al. [88] pointed out that the angle of the spinous process may underestimate the severity of curvature of the spine compared to the Cobb method.

For three-dimensional image data, how to improve the image processing speed, considering the image quality and calculation time, is an important problem that needs to be solved. Chen, H.B. et al. [89] used a point projection (FDP) algorithm using voxel-based nearest neighbor (VNN), multiplane interpolation (MPI), and pixel nearest neighbor (PNN) protocols. This method effectively shortens the reconstruction time and can meet the requirements for immediate demonstration while ensuring reliability and accuracy when assessing scoliotic angle measurements. The above methods have a good performance in terms of calculation accuracy and noise suppression. However, in terms of calculation time, 1–2 mins are still needed to process a case, and the efficiency still needs to be further improved.

5.2. Machine Learning Method for 3D Image

Scoliosis detection methods based on image characteristics and prior knowledge are susceptible to interference from image noise, resulting in large measurement deviations or even measurement errors. Meanwhile, in medical images, some artificial noise will also be introduced, such as gray features and texture features in US [90] and secondary artifacts in CT. To solve the above problems, researchers have begun to study robust detection methods with noise suppression, mainly using machine learning, deep neural networks, and other frameworks [91,92,93,94].

Considering that a single weak classifier cannot deal with various sources of noise, Wang, C.H. [95] jointly trained multiple weak classifiers through AdaBoost and successfully segmented the spinal cord center points and fitted the spine curve on a CT dataset.

Liu, Z. et al. [96] constructed a fully automatic framework based on Faster R-CNN. The Faster R-CNN is trained to detect vertebral lamina. While the detected lamina pairs are used to fit the midline of the spine curve, a spinal curvature estimator calculates the scoliotic angles based on the curve. This method uses the structural features of the spine, such as ribs and lamina, which are adjacent to each other, as a priori knowledge, but there is still a possibility of false positives in the detection of the spine.

Convolutional neural networks have great advantages in terms of skeletal segmentation and feature extraction, which will facilitate the accurate measurement of scoliosis angles. The most-used method is the U-Net. Huang, Z. et al. [97] introduced a segmentation method called RSN-U-net, as shown in Figure 8. Aiming at the spot and regular occlusion noise that are prone to appear in ultrasound medical images, this method uses total variance (TV) loss to train the neural network and successfully improves the robustness of spot and regular occlusion noise, effectively segmenting the bone features in ultrasound spine images. Agrawal, A. et al. [40] simultaneously used three U-nets to segment the vertebral bodies in the coronal and sagittal planes and the pelvic region of the CT dataset and calculated the sagittal Cobb angle.

Figure 8.

The architecture of RSN-U-net [97]: Similar to U-net, RSN-U-net is also composed of an encoder and decoder. The encoder section consists of four repeated encoder stacks. Each encoder stack contains two conv (3 × 3) convolutional layers and one max-pooling (2). Each decoder stack contains two conv (3 × 3) and one Upsampling (2).

Banerjee, S. et al. [98] presented a Light-convolution Dense Selection U-Net (LDS U-Net) to identify lateral bony features from ultrasound spine bony features automatically. In order to export the selective features, the method suppresses irrelevant information with a gating mechanism. In addition, it also uses multi-scale skip-paths to enhance feature fusion.

In addition to the use of 3D medical images, some scholars have used patient back images to directly obtain scoliosis angles through deep learning. This method can avoid radiation damage to the human body and does not require complex medical equipment operation, but the accuracy of its detection and its use in medicine remains to be seen. Yang, J. et al. [15] proposed a deep learning algorithm for automated scoliosis screening using images of uncoated backs, as shown in Figure 9. The algorithm uses Faster-RCNN to locate the back image and classifies the image with Resnet. When performing the initial screening task for scoliosis, the algorithm was both more accurate and efficient than human experts.

Figure 9.

Block diagram and the architectures of Faster-RCNN and Resnet [15]: Yang, J. et al. (a) The entire DLA workflow; (b) The architecture and workflow of Faster-RCNN; (c) The architecture of Resnet. [15]’s algorithm using convolutional layer to obtain a feature map of the back. The algorithm finds the ROI in the feature map, pools it, and classifies it with a classifier.

Although the experimental results of these methods show that the Cobb angle can be measured effectively, there are also problems, such as calculation time cost and insufficient recognition accuracy, which is also a common problem in two-stage (2-stage) deep learning methods. How to achieve a balance between image quality and the efficiency of computation time is still a major challenge for the current 3D angle measurement algorithms.

6. Discussion

With the rapid development of information technology and medical imaging technology, the technical bottleneck of computer-aided measurement of spinal curvature has been continuously broken through. However, the main problems faced by researchers remain challenging.

Due to its low cost and convenience, using 2D medical images for the Cobb angle measurement is always the most important. The greatest problem that researchers encountered at first was the segmentation of the vertebral part of the image. In view of the characteristics of larger noise, blurred edges, and more occlusion between structures, some researchers have used filters, active shape models, and active apparent models to extract the edges of the spine, but the effect is far from satisfactory. The active shape model successfully captures contour features in X-rays but is highly dependent on the initial setup. Furthermore, it fails to solve the problem of overlapping structures in medical images caused by scoliosis, which can lead to the loss of a large number of feature points. The active apparent model has certain constraints on the shape of the object to be segmented, so it performs better in images where structures obscure each other. However, neither the active shape model nor the active apparent model can resolve the underlying changes in topology. While the CPM model can effectively solve the above problem, it is computationally expensive and susceptible to errors caused by variations in X-ray images.

Some researchers have noted that, although there are individual differences in spinal imagery, there are still statistical regularities. Therefore, they use machine learning to segment the spine image and calculate the Cobb angle. One of the most representative methods is Seg4Reg, a two-stage learning framework. However, the stability of the Cobb angle output is still a problem. The output stability of the measurement is affected by a variety of factors, including phase wrapping and vertebral coordinate error. Focusing on the phase wrapping in the angle output, methods such as S2VR, MVC-net, and MVE-net focus on the correlation between the output of the vertebral landmarks and the output of the Cobb angle, with a specific loss function design to ensure a smooth output. Since the appearance of a noise point may affect the selection of the vertebral end and cause a large deviation in the results, researchers proposed the boost-layer method to exclude outliers with large coordinate errors. The AAM model can use statistical methods to estimate the approximate location of feature points. However, it is unable to cope with the fusion and deformation of the vertebral bodies in scoliosis cases.

Machine learning methods and image enhancement methods have their own advantages due to their different technical focus on problem-solving. As Duncan and Ayache [53] pointed out in their review of current medical imaging trends, combining these two areas will help improve accuracy and robustness.

Researchers have identified two drawbacks to using 2D medical images to monitor scoliosis relative to newer technologies: (1) 2D medical images can only show a single perspective, which is prone to information loss caused by occlusion of features; (2) scoliosis is a 3D structural change. The projection on the light image does not reflect the true spine curvature. Some 3D medical imaging technologies, such as MRI and ultrasound, show their advantages, and researchers believe that ultrasound medical imaging is a promising 3D medical imaging for the regular assessment of scoliosis. The information in 3D medical images is often so abundant that the biggest problem facing researchers is how to eliminate interfering information. A problem that troubles researchers is how to reconstruct the 3D structure of the spine. Some researchers have used volume projection imaging (VPI) to describe the spine profile in the coronal plane, but this cannot adequately represent the 3D structural changes in scoliosis. Therefore, some researchers measure the Cobb angle together with VPI by generating a projection of the sagittal plane. Another solution is to reconstruct a curve that describes the trend in the spine by identifying features of the spine, such as spinous or transverse processes, through polynomial fitting. Soon, the researchers encountered two problems. One was that the SPA underestimated the degree of spinal curvature. The researchers realized that we do not have a well-established quantitative assessment of curvature for 3D spinal structures, which is a major source of error in the computer-aided measurement of 3D angles. The other is that the ultrasound images were noisy, and precise feature engineering is required to find the spinous and transverse processes. Therefore, some researchers use filters or introduce machine learning architectures to obtain more robust detection algorithms with noise suppression.

The automatic measurement method for Cobb angle shows its potential application in various medical scenarios, but the existing algorithms are not complete enough. We believe that a complete automatic Cobb angle measurement algorithm should meet five requirements: accurate segmentation and classification, high positioning accuracy, a small angle output error, better robustness for a wider range of application scenarios, and a shorter running time. To achieve this goal, we believe that improvements should be made in the following four areas.

(1) Dataset: The existing automatic measurement algorithms are all data-driven. However, we lack large public datasets for multiple real-world scenarios, and some sources of interference cannot be ignored. For example, intraoperative images have measurement errors caused by image stitching, and there may be interference from surgical instruments. Postoperative imaging may show metallic implants and anatomical abnormalities (e.g., transitional anatomy and missing vertebrae). However, most of the existing databases do not reflect these interference items, which greatly limits their ability to verify the algorithm.

(2) Algorithm design: Current fully automatic algorithms have poor transferability between different datasets, which is reflected in their poor generalization in the face of abnormal anatomy. In the Verse Vertebral Segmentation Competition held by MICCAI in 2019, the vast majority of algorithms performed poorly on the test set, with many anatomical anomalies (such as transitional anatomy and missing vertebral bodies) [99]. Only the algorithm of Payer, C. [100] is stable. In 2020, the situation improved. The algorithm design should not only be based on existing datasets but should also consider the idiotype of the application scenario as much as possible.

(3) Deep learning framework: How to balance image quality and computation time is a big challenge for fully automated Cobb angle measurement methods. It was mentioned earlier that Cobb angle measurements could be used for intraoperative spinal alignment. However, intraoperative automatic measurement requires an extremely high real-time performance, which cannot be solved by the deep learning framework used by existing methods. Therefore, a deep learning framework that considers both efficiency and accuracy is needed.

(4) Application scenarios: Initially, fully automated measurements of Cobb angles were only used for a quantitative assessment of the disease. Now, they can also be used to determine whether the vertebrae are properly aligned during surgery. Future researchers can explore more application scenarios for a fully automated Cobb angle measurement in combination with medical practice. For example, combined with a three-dimensional reconstruction of the spine, the treatment plan, and surgical effect can be evaluated by establishing a mechanical model or combined with real-time registration technology for intraoperative real-time monitoring.

7. Conclusions

Measurements of scoliosis angles such as Cobb angles are semi-automatic at present, requiring the manual labeling of marker points to calculate the Cobb angle. These methods are inefficient and inconsistent, and fully automated measurement methods are urgently needed. Research based on 2D images such as X-Ray images is the hotspot and has made great progress; automatic Cobb angle measurements can be achieved in some specific scenarios. For a more comprehensive assessment of scoliosis and lower radioactivity, 3D detection technologies are increasingly used in the assessment of scoliosis, especially adolescent idiopathic scoliosis (AIS). With the improvement in the dataset and the optimization of the deep-learning-based algorithm, automatic measurement of the Cobb angles will be implemented for 3D images, making them applicable to the diagnosis and treatment of spinal curvature.

Author Contributions

Conceptualization, G.Y. and C.J.; methodology, S.W.; formal analysis, E.L. and Z.L.; investigation, C.J.; data curation, C.J.; writing—original draft preparation, C.J.; writing—review and editing, G.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded partially by Beijing Natural Science Foundation, grant number L192015 and partially by National Nature Science Foundation, grant number 61973300.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.