Intra- and inter-observer reliability of determining radiographic sagittal parameters of the spine and pelvis using a manual and a computer-assisted methods

Abstract

Sagittal imbalance is a significant factor in determining clinical treatment outcomes in patients with deformity. Measurement of sagittal alignment using the traditional Cobb technique is frequently hampered by difficulty in visualizing landmarks. This report compares traditional manual measurement techniques to a computer-assisted sagittal plane measurement program which uses a radius arc methodology. The intra and inter-observer reliability of the computer program has been shown to be 0.92–0.99. Twenty-nine lateral 90 cm radiographs were measured by a computer program for an array of sagittal plane measurements. Ten experienced orthopedic spine surgeons manually measured the same parameters twice, at least 48 h apart, using a digital caliper and a standardized radiographic manual. Intraclass correlations were used to determine intra- and interobserver reliability between different manual measures and between manual measures and computer assisted-measures. The inter-observer reliability between manual measures was poor, ranging from −0.02 to 0.64 for the different sagittal measures. The intra-observer reliability in manual measures was better ranging from 0.40 to 0.93. Comparing manual to computer-assisted measures, the ICC ranged from 0.07 to 0.75. Surgeons agreed more often with each other than with the machine when measuring the lumbar curve, the thoracic curve, and the spino-sacral angle. The reliability of the computer program is significantly higher for all measures except for lumbar lordosis. A computer-assisted program produces a reliable measurement of the sagittal profile of the spine by eliminating the need for distinctly visible endplates. The use of a radial arc methodology allows for infinite data points to be used along the spine to determine sagittal measurements. The integration of this technique with digital radiography’s ability to adjust image contrast and brightness will enable the superior identification of key anatomical parameters normally not available for measurement on traditional radiographs, improving the consistency of sagittal measurement.

Introduction

The importance of sagittal spino-pelvic alignment has gained renewed attention with recent studies demonstrating that proper sagittal plane balance is critical to long term patient satisfaction [6]. Previous studies described the pelvic radius technique to determine pelvic orientation, focusing on understanding the relationship of global sagittal alignment to the orientation of the pelvis, as the entire balance of the spine is dependent on its geometric configuration [7–10]. More recent studies employ a sagittal measurement computer program using a method for calculating the best-fit arcs passing through the spine which are used to calculate thoracic kyphosis and lumbar lordosis, thus eliminating the need for distinctly visualized endplates resulting in significantly improved reproducibility in the measurement of various spinal sagittal and pelvic plane parameters [1, 5]. A validation study comparing the Cobb measurement technique to the best-fit arcs methodology demonstrated that both techniques exhibit similar reliability and accuracy [2, 3]. Additional studies using the computerized best-fit arcs technique have established normal human sagittal plane alignment patterns and identified pathological variations by combining geometric pelvic measurements with various sagittal spinal alignment parameters [14, 18, 22].

The difficulty of reliably measuring anterior–posterior (AP) scoliosis radiographs by experienced spinal surgeons using the manual Cobb technique for the classification of scoliosis has already been established [3, 4, 17, 20]. Recently, a computerized digital measurement technique for scoliosis demonstrated improved precision and good correlation with manually measured radiographs validating this methodology for the measurement of the spine in the coronal plane [12]. Similarly, the accuracy and reliability in the measurement of the spine in the sagittal plane is often more difficult due to variable measurement criteria, manual measurement errors, and difficulty visualizing measurement landmarks. The purpose of this study was to compare intra-observer and inter-observer reliability of experienced spinal surgeons in manually measuring various sagittal plane parameters to the best-fit arcs computer-aided measurement system to determine which technique is most reproducible.

Materials and methods

Twenty-nine 30 × 90 cm lateral radiographs of the spine and pelvis from a cohort of 160 asymptomatic young adult volunteers from a previous study [1] were randomly selected. These conventional standing lateral X-rays of the spine, pelvis, and femoral heads were obtained at a fixed distance of 228 cm from the X-ray source with the subject in a comfortable standing position, the knees fully extended, and the upper limbs resting on two arm supports. The hardcopy films were then independently reviewed by ten experienced spinal surgeons at five medical centers. Fourteen sagittal plane measurements were determined manually with a digital caliper on unmarked films on two occasions, at least 48 h apart. The measurements were the following:

• Pelvic incidence Angle subtended by a line which is drawn from the center of the femoral head to the midpoint of the sacral endplate and a line perpendicular to the sacral endplate [16].

• Sacral slope The angle subtended by a horizontal reference line and the sacral endplate line.

• Pelvic tilt The angle subtended by a vertical reference line originating from the center of the femoral head and the midpoint of the sacral endplate.

• Lumbar lordosis, thoracic kyphosis, cervical lordosis Measured using the Cobb technique.

• L4/L5 incidence Angle subtended by a line which is drawn from the center of the femoral head to the midpoint of the superior endplate of L4 (L5) and a line perpendicular to the superior endplate of L4 (L5).

• L4–L5 angle/L5–S1 angle Angle formed by a line through the superior endplate of the vertebral body (L4 or L5) and the superior endplate of the next distal vertebral body (L5 or S1).

• L4 or L5 pinch Angle formed by a line through the superior endplate of the vertebral body and the inferior endplate of the same vertebral body.

• Spino-sacral angle Angle formed by a line through the sacral endplate and the vertical line drawn parallel to the radiograph edge from the C7 centroid.

• Grade Quantifies the forward translation of L5 with respect to the sacral endplate. Grade I defines a slip between 0 and 25% of the linear distance of the sacral endplate while Grades 2, 3, and 4 represent translations of L5 up to 50, 75, and 100%, respectively. When L5 is completely anterior and distal to the S1 endplate, it is Grade 5 [19].

These manual measurements were compared to those obtained from a computer-aided radiologic measurement methodology which utilized customized sagittal profile software (SagittalSpine^©, Optimage, Group of Applied Research in Orthopedics, Lyon, France). This program allows the user to measure various data points on the radiograph, manipulating image contrast and size [1]. Two sets of data points are necessary to generate the geometric model of the spine and pelvis. The first set consists of five anterior spinal vertebrae surface anatomic landmarks; one at the L5 inferior endplate (P1), one at the apex of the lumbar lordosis (P2), one at the thoracolumbar junction (P3), one point at the apex of the thoracic kyphosis (P4), and one at the top of the thoracic kyphosis (P5) (Fig. 1). The program determines the best fit arcs that pass through these five points and displays the sagittal contour of the spine along with the calculation of the thoracic kyphosis and lumbar lordosis. The thoracic kyphosis and lumbar lordosis is calculated via two arcs of circles tangent to the apex of the curve and are the sum of the kyphosis and lordosis, respectively, above and below the apex. Additional radiographic data points placed on the pelvis allow calculation of the various other parameters measured in the study, including sacral slope, pelvic incidence, and pelvic tilt. The baseline sagittal measurement data from a previous study using the same computer program and the same films were used in this study for comparison to the manual measurements.

Fig. 1.

Example illustrating the generation of a geometrical model of the spine and pelvis from a standard marked and digitised radiograph. a Spine anatomical landmarks and method of calculating TK and LL: each of the curves is described by two arcs, LL by arcs of circles 1 and 2, and TK by arcs of circles 3 and 4. For each arc, the following are specified: the radius R (in mm) divided by the distance (d, in mm) measured from the pelvis; the angle θ (in degrees) subtended by the arc; and the number of vertebrae (n) included by the arc of the circle. The ratio θ/n represents the mean curvature of each arc. To simplify interpretation, both arcs are taken together to represent lordosis (arcs 1 and 2) and kyphosis (arcs 3 and 4). Thus, the resultant angle (θ_TK and θ_LL), the number of vertebrae (n_TK and n_LL) involved in the curved region and the corresponding curvature (θ_TK/n_TK and θ_LL/n_LL) can all be derived. The vertebrae marking the extremes of the lordosis and kyphosis segments are marked and the linear segment between the two curved segments is drawn. b Pelvic anatomical landmarks and method for calculating PI, SS, PT and d

Inter-rater and intra-rater reliability between manual measures, reliability of computer-aided measures and inter-rater variability between manual and computer-aided measures were analyzed using Kappa statistics for categorical variables (determination of end vertebrae), Pearson’s correlation coefficients and intra-class correlations for continuous variables (angular measures). A two-tailed test of significant differences between reliability coefficients between manual measures and computer-aided measures was done using Fisher’s Z transformation on the correlation coefficients and calculating a z-score for the difference in the Fisher’s Z values.

Results

There was poor to slight agreement [15] between manual measures in determining the end-vertebrae for lumbar lordosis, thoracic kyphosis and cervical lordosis, with kappa values ranging from −0.01 to 0.11. However, there was fair to moderate intra-rater reliability among manual measures in determining end vertebrae with kappa coefficients ranging from 0.47 to 0.72 (Table 1). There was poor agreement between the manual measures and computer-aided measures in determining end-vertebrae with kappa values ranging from 0.00 to 0.20 (Table 2).

Table 1.

Kappa coefficients for inter- and intra-observer reliability in determining end vertebrae among manual measures

	Inter-rater reliability		Intra-rater reliability
	%Agreement	Kappa	%Agreement	Kappa
LEV lumbar	39.20	−0.03	84.10	0.72
UEV lumbar	34.50	0.11	63.60	0.52
LEV thoracic	33.90	0.04	62.90	0.47
UEV thoracic	22.30	0.00	62.50	0.49
LEV cervical	29.00	−0.01	63.30	0.48
UEV cervical	19.30	0.00	78.40	0.72

LEV lower end vertebrae; UEV upper end vertebrae

Table 2.

Kappa coefficients for inter-observer reliability in determining end vertebrae between manual measures and computer-aided measures

Parameters	%Agreement	Kappa
LEV lumbar	N/A	N/A
UEV lumbar	38.10	0.20
LEV thoracic	32.30	0.08
UEV thoracic	31.90	0.03
LEV cervical	15.30	0.00
LEV thoracic	19.70	0.04

Computer-aided measures always use S1 endplate

LEV lower end vertebrae; UEV upper end vertebrae

The intra-rater reliability for the manual measures of sagittal plane parameters varied from a low ICC of 0.40 (0.29–0.49) for determining the L4–L5 angle to a high of 0.93 (0.91–0.95) for determining the L4 incidence. The inter-rater reliability for the manual measures also varied from a low ICC of −0.02 (−0.04 to 0.03) for the L4 incidence to a high of 0.64 (0.61–0.67) for the sacral slope (Table 3). The intra-rater reliability using Pearson’s coefficient showed the same variability for the manual measures of sagittal plane parameters with a low of 0.34 for determining the L4–L5 angle to a high of 0.94 for determining the L4 incidence. The inter-rater reliability for the manual measures also varied from a low Pearson’s coefficient of −0.08 for the L4 incidence to a high of 0.71 for the sacral slope (Table 3).

Table 3.

Intra- and Inter-rater reliability of manual measures

Parameters	Inter-rater reliability		Intra-rater reliability
	Intra-class correlations	Pearson’s coefficient	Intra-class correlations	Pearson’s coefficient
Pelvic incidence	0.41 (0.36–0.45)	0.29	0.69 (0.62–0.74)	0.65
Sacral slope	0.64 (0.61–0.67)	0.61	0.77 (0.72–0.81)	0.71
Pelvic tilt	0.42 (0.37–0.46)	0.44	0.60 (0.53–0.67)	0.55
Lumbar curve	0.57 (0.52–0.59)	0.54	0.90 (0.87–0.92)	0.88
Thoracic curve	0.54 (0.50–0.57)	0.46	0.80 (0.76–0.84)	0.74
Cervical curve	0.12 (0.06–0.17)	0.56	0.89 (0.86–0.91)	0.84
L5 incidence	0.12 (0.07–0.18)	0.04	0.79 (0.74–0.83)	0.88
L5–S1 angle	0.39 (0.35–0.44)	0.51	0.75 (0.70–0.80)	0.69
Pinch or angle body L5	0.49 (0.45–0.53)	0.47	0.69 (0.62–0.74)	0.62
L4 incidence	−0.02 (−0.04–0.03)	−0.08	0.93 (0.91–0.95)	0.94
L4–L5 Angle	0.07 (0.02–0.12)	0.14	0.40 (0.29–0.49)	0.34
Pinch or angle body L4	0.45 (0.40–0.49)	0.48	0.64 (0.57–0.71)	0.52
Spino-sacral angle	0.12 (0.04–0.19)	0.29	0.90 (0.87–0.92)	0.90
Grade	0.26 (0.17–0.34)		0.79 (0.73–0.84)

These values were compared to measures of reliability of computer-aided measurements from a previously published [1] study using the same computer measurement software that measured the same sagittal plane measurements using the same patients, and which showed very good to almost perfect agreement with ICC’s ranging between 0.88 and 0.99 (Table 4). Pearson’s coefficients comparing computer-aided measures to each surgeon’s manual measures showed a low coefficient of 0.10 for the L4 incidence compared to 0.72 for the sacral slope (Table 5).

Table 4.

Intra-class correlations for computer-aided measures used by permission of authors, Berthonnaud et al.

Parameters	Inter-rater ICC	Intra-rater ICC
Pelvic incidence	0.98 (0.98–0.99)	0.99 (0.98–0.99)
Sacral slope	0.98 (0.97–0.99)	0.99 (0.98–0.99)
Pelvic tilt	0.99 (0.99–1.00)	0.99 (0.99–1.00)
Lumbar curve	0.97 (0.96–0.99)	0.88 (0.64–0.95)
Thoracic curve	0.91 (0.86–0.95)	0.94 (0.90–0.97)

Table 5.

Pearson correlation coefficients for computer-aided measures compared to each surgeon’s manual measures

Parameters	Pearson’s coefficient
Pelvic incidence	0.59
Sacral slope	0.72
Pelvic tilt	0.63
Lumbar curve	0.68
Thoracic curve	0.59
Cervical curve	0.64
L5 incidence	0.25
L5–S1 angle	0.66
Pinch or angle body L5	0.58
L4 incidence	0.10
L4–L5 angle	0.30
Pinch or angle body L4	0.57
Spino-sacral angle	0.26

The test of significant differences between reliability coefficients showed that the reliability of computer-aided measures was significantly higher in all parameters except for lumbar lordosis where the difference between the manual measures and the computer-aided measures was not significantly different (Table 6).

Table 6.

Test of significant differences between reliability coefficients

Parameters	Inter-rater comparison P (two-tailed)	Intra-rater comparison P (two-tailed)
Pelvic incidence	<0.0001	<0.0001
Sacral slope	<0.0001	<0.0001
Pelvic tilt	<0.0001	<0.0001
Lumbar curve	<0.0001	0.51062
Thoracic curve	<0.0001	<0.0001

Discussion

The evaluation of the global spino-pelvic alignment using a reliable method is a significant challenge, especially in the sagittal plane where the technique of manually measuring the spine and pelvis has changed little over the past 50 years. The traditional Cobb measurement technique has been hampered by a variety of limitations including poor visualization of pertinent anatomical landmarks due to differing anatomic X-ray absorption gradients and the lack of the consistent use of high-quality lateral 90 cm radiographs of the spine and pelvis to determine the global sagittal balance. Current techniques tend to concentrate on isolated measurement of thoracic kyphosis and lumbar lordosis as separate anatomic structures, ignoring the role of the pelvis in overall sagittal alignment. This highlights the need for a consistent method of sagittal plane radiographic measurement other than the traditional manual method and the critical need for full length 90 cm radiographs for all sagittal plane deformities. Regardless of whether or not the pathology is a fracture, infection, structural kyphosis or a high grade spondylolisthesis, the restoration of reasonable sagittal balance is paramount to the long term surgical success and patient satisfaction.

The importance of maintaining proper spino-pelvic alignment cannot be overstated. The inevitable loss of lumbar lordosis with aging depends on the compensatory ability of the pelvis and hips to maintain acceptable sagittal alignment [1, 2, 5, 7–11, 14, 18, 21, 22, 23]. Unlike coronal plane deformities (i.e., scoliosis) where the width of the pelvis allows for more potential deformity and resultant compensatory balance, the narrow anterior–posterior depth of the pelvis restricts the amount of compensation available to correct sagittal imbalance. Although the geometry of the pelvis is fixed by adulthood [2], the pelvis does allow for significant sagittal compensation by increasing the retroversion of pelvis (pelvic tilt) along with the extension of the hips and flexion of the knees in an attempt to move the sagittal balance line of weight bearing toward the center of the sacrum. This ability of the pelvis and hips to compensate for spinal sagittal imbalance is variable, with certain individuals being better able to maintain their sagittal balance as the typical increase in thoracic kyphosis and decrease in lumbar lordosis develops with advancing age. Once all the available pelvic compensation is reached, the forward angulation of the spine in the sagittal plane results in the spine overhanging the zone of spinal balance contained within the pelvic brim allowing for the translation of the C7 plumb and gravity line anterior to the sacrum. The importance maintaining appropriate sagittal spino-pelvic balance can best be demonstrated from the adult deformity patient’s perspective. Patients with significant sagittal imbalance had poor two-year health-related quality of life measures [6].

Jackson et al. have provided key insight into sagittal spino-pelvic balance and alignment by establishing that lumbar lordosis is dependent on pelvic morphology and that total sagittal spinal alignment must include the a comprehensive evaluation of both [8, 9]. Specifically, the authors described the reliability of the pelvic radius technique and its importance in evaluating the contribution of the pelvis’s fixed morphology relationship to total sagittal profile, particularly lumbar lordosis [7, 10]. The authors manually measured 90 cm lateral radiographs from a wide variety of normal and pathological subjects to determine lumbar lordosis, thoracic kyphosis and pelvic morphology, specifically the pelvic radius-sacral line (PRS1 angle). The pelvic radius technique measures the relationship of the femoral heads to the position of the sacrum. They found that the PRS1 angle remains fixed and strongly correlates with the total lumbosacral lordosis with individuals with a larger than average angle having less total lordosis while those with a smaller angle have more lordosis. They concluded that lordosis is specifically dependent on the fixed pelvic morphology and its contribution to the global sagittal alignment and that the manually measured pelvic radius technique is a reliable and valuable tool to assess the overall sagittal spino-pelvic alignment. Finally, the authors clearly established the need for 90 cm radiographs (including the hips) and the difficulty in manually measuring the radiographs with various parameters due to lack of clarity. Difficulty in visualization was specifically noted in identifying the center of the C7 vertebrae, which was necessary to establish the weight bearing plumb line; the upper thoracic spine due to the shoulders, and the top of the sacrum which was often obscured by the pelvis or previous instrumentation.

With the advent of digital radiography there has been the concurrent development of computer based radiographic spinal measurement technology to more accurately measure the global spino-pelvic alignment. Recently, digital computer measurement has been shown to have excellent intra- and inter-observer reliability and correlated well with manual measurements for most parameters of adolescent idiopathic scoliosis in the coronal plane [12]. Although the authors concluded that digital measurement is more accurate with preoperative and coronal measurements than with postoperative and sagittal measurements, they still felt that digital computer measurement is a valid technique [12]. The authors stressed that as digital radiography use becomes more frequent, the use of the digital measurement techniques will become increasing more common [12]. The parallel development of a digital radiographic computer technique for the measurement of spino-pelvic alignment has also been studied for its intra- and inter-observer variability, accuracy and reproducibility when compared to manually measured radiographs using the Cobb technique [1]. The technique measures digitized radiographs using the arcs of circle geometric methodology to measure the normal sagittal contours following the assignment of relevant data points on the spine. The program also allows for the measurement of key fixed pelvic parameters including sacral slope (SS), pelvic incidence (PI), and pelvic tilt (PT) that have previously been described by Duval-Beaupere [5]. The authors showed high reproducibility of measurements obtained with digitized sagittal computer-aided measurement. The computer measurement values of the spino-pelvic axis established in this study were used for comparison to the manually measured spino-pelvic parameters in the current study [1]. Recent studies have further validated the superior accuracy, reproducibility and versatility of this computer methodology in the measurement of the sagittal spino-pelvic axis in various pathological conditions including spondylolisthesis and degenerative conditions [11, 13, 14, 21, 23, 24].

Studies of this nature by default may tend to have a built in bias toward more reproducible results since the individuals measuring the radiographs, digitizing the radiographs, and using the computer program to measure them inevitably have had extensive experience in the technique. Therefore, even though these carefully controlled computer studies have demonstrated superb reliability in measuring the sagittal plane, there is still a significant need to evaluate how other spine specialist’s manual measurements compare to each other and to this technique.

The results of this study highlights the fact that even experienced spine specialists have significant difficulty in reliably and reproducibly measuring traditional 90 cm sagittal radiographs in the sagittal plane. This difficulty arises primarily due to the inability to identify strategic anatomic parameters on the radiographs and the latitude the Cobb measurement technique allows the measurer in the identification and marking of the endplates when determining thoracic kyphosis and lumbar lordosis. The surgeons’ intra-observer ICC generally demonstrated moderate to good reliability. However, all of the manual measures when compared to the near perfect computer-aided measure ICC results are found to be significantly inferior. The interobserver comparison of the surgeon results showed only fair ICC, pointing out how poorly the measurers were able to reproduce each other’s results. Again, these results when compared to the inter-observer reliability of the computer-aided method are poor. Finally, comparing the surgeon’s measurements to the computer-aided method showed poor correlation, with lumbar lordosis being the only parameter that showed good reliability. These results indicate that a computer-aided technique such as the one used in this study simply reduces the human measurement error by first clarifying the anatomical landmarks available for measurement due to the ability to adjust the contrast with digital radiography, and second, the computer program’s automatic and precise calculation of all measurement parameters.

Conclusion

This study highlights the inconsistency of the manual measurement of the sagittal plane, even among experienced spinal surgeons. The computer-assisted program used in this study produced reliable measurements of the sagittal-pelvic plane by eliminating the need for distinctly visible endplates by the use of best-fit arcs passing through the spine. This allows for the measurement of numerous data points along the spine while at the same time minimizing human measurement error. The integration of such techniques with digital radiography’s ability to adjust image contrast and brightness enabled the superior identification of key anatomical parameters normally not available for measurement on traditional radiographs. The integration of this computer methodology appears to improve the clinician’s ability to accurately and reliably measure the sagittal-pelvic plane when compared to manual measurement.