RELIABILITY OF VARIOUS OBSERVERS IN DETERMINING COMMON RADIOGRAPHIC PARAMETERS OF ADULT HIP STRUCTURAL ANATOMY

Abstract

Background

Radiographic evaluation of the hip is extremely important in the diagnosis and treatment decisionmaking process for pre-arthritic hip disease. Many different radiographic measurements have been described as indicators of underlying structural hip deformity. The purpose of this study was to determine the interobserver and intraobserver reliability of various musculoskeletal physicians in performing selected measurements of adult structural hip anatomy.

Methods

A blinded review of 45 sets of radiographs from patients with developmental dysplasia, femoro-acetabular impingement, and normal anatomy was performed. Data points included the lateral center-edge angle (LCEA), vertical-center-anterior angle (VCA), head-neck offset ratio (UNO), alpha angle, Tonnis angle, Tonnis osteoarthritis grade and a radiographic diagnosis. One orthopaedic fel low, two orthopaedic residents, and two attending musculoskeletal physiatrists analyzed radiographs on two separate occasions. One sports medicine orthopaedic attending physician completed a single analysis of the image sets. Intraobserver and interobserver reliability was established using intra-class correlation coefficients (ICC) for continuous variables. Agreement regarding categorical variables was performed using the kappa coefficient

Results

Excellent intraobserver reliability was found for the following: LCEA (ICC = 0.88), VCA (0.88), Tonnis angle (0.83), HNO on the frog lateral (0.78), alpha angle on the frog lateral (0.76), HNO on the cross-table lateral (0.75), and angle alpha on the cross-table lateral (0.76). Intraobserver reliability for osteoarthritis grade was poor (weighted kappa = 0.57). For all data points, interobserver reliability was considerably worse, with 95% confidence intervals spanning below 0.55.

Conclusions

While the described measurements of adult structural hip anatomy provide excellent reliability for a given reader, these measurements are less reliable across readers. Taken in isolation, these measurements, as performed by observers with varied clinical experience and clinical backgrounds, are limited in determining a consistent radiographic diagnosis.

INTRODUCTION

The majority of chronic, non-inflammatory hip disorders in the young adult population are mechanically-based and are associated with structural instability of the hip and/or femoroacetabular impingement (FAT).^1,2 While structural instability of the hip and FAI are well-described, many patients with symptomatic disease experience major delays in diagnosis, inaccurate diagnoses, and ineffective or inappropriate treatments. ^3,4 Consultation with healthcare providers in various disciplines (primary care, orthopaedics, physiatry, physical therapy, rheumatology and chiropractic) is commonly necessary for these patients to obtain an accurate diagnosis and appropriate treatment.^3,4 Thus, there exists a major need to improve the diagnostic skills of healthcare providers involved in the evaluation and treatment of patients with groin, lateral hip, and posterior pelvic pain.

Proper diagnosis of pre-arthritic hip disease in the skeletally mature patient is dependent on the patient history, physical examination, and imaging studies. The radiographic examination is critical in screening patients for underlying structural abnormalities, establishing an accurate diagnosis, and in developing an appropriate treatment strategy. Many different radiographic measurements have been described as indicators of structural hip disease. In particular, measurements such as the lateral center-edge angle of Wiberg,⁵ the anterior center-edge angle of Lequesne,⁶ and the Tonnis angle⁷ have been consistently used as primary markers for acetabular dysplasia. Similarly, measurements of the head-neck offset (initially described by Eijer)^8,9 and the alpha angle¹⁰ have recently been used to identify potential cases of femoroacetabular impingement A complete review of any hip series in an adult patient also requires an overall assessment of the degree of osteoarthritis, such as that outlined in the classification system described by Tonnis.¹¹

While the reliability of some of these measurements has been evaluated in the pediatric patient population, less has been published in the orthopaedic literature regarding their reliability in the skeletally mature hip.^12-17 Much of the information available on the reliability of specific measurement tools for the adult hip is buried within larger studies or is relevant to hip specialists only.^18-25,18-27 In order to have a standardized set of radiographic parameters to help reliably define structural hip disorders, it is imperative that the radiographic measurements be reproducible. Ideally, the reliability of these measurements should be established for various healthcare providers and not limited to experienced hip specialists, since many of these patients present to a variety of physicians involved in musculoskeletal care. Therefore, the purpose of this study is to evaluate the reliability of healthcare providers with varied clinical experience and medical backgrounds in performing selected measurements of adult structural hip anatomy.

METHODS

The computerized patient database of the senior author (housing lists all patients having undergone surgical intervention) was utilized to select preoperative images for 15 patients with acetabular dysplasia and 15 patients with impingement (cam, pincer, or combined). A control group of 15 patients was selected independently, consisting of a previously reported patient cohort⁹ that was evaluated clinically and radiographically by the senior author and found to have no clinical evidence of hip disease. Specifically, none of the patients had groin pain, a positive impingement test, or hip irritability on examination. All of the patients had alternative findings suggestive of a disorder not involving the hip (i.e. degenerative disc disease of the lumbar spine). The image sets for patients in the dysplasia and impingement groups were randomly selected from an alphabetized list of consecutive operative cases from 2001-2005. Patients were included in these groups if they had surgical treatment for symptomatic disease, intraoperative findings consistent with the preoperative diagnosis, and a positive response to surgery. All patients in the dysplasia group underwent periacetabular osteotomy,²⁸ and all patients in the impingement group underwent either surgical dislocation with osteochondroplasty²⁹ or hip arthroscopy with limited open osteochondroplasty.²⁶

Image sets were excluded from the study if the radiographs demonstrated any evidence of previous surgery, or if the radiographs were taken before 2001, as images prior to this date did not meet the Digital Imaging and Communications in Medicine (DICOM) standard. Prior Institutional Review Board approval existed for the protocol.

All study patients had a series of four radiographs: an anteroposterior (AP) pelvis, cross-table lateral, frog-lateral, and a false profile view of the hip. Though multiple radiology technicians obtained the image sets, all radiographs were taken in the same imaging center using a standardized imaging protocol. This protocol has been previously published and is available for reference. ²⁴

All identifying data was removed from the radiographs, and each patient was assigned a study number. Additionally, all observers in the study were blinded with regard to the patient's clinical history, physical examination, and underlying diagnosis. The observers had various clinical backgrounds and experience levels in order to simulate the evaluation of radiographs by healthcare providers who are not hip specialists. The radiographs were retrospectively reviewed by one adult reconstruction fellow, two third year orthopaedic surgery residents, an attending sports medicine orthopaedic surgeon with specific interest in knee and shoulder, and two attending physiatrists who treat all musculoskeletal disorders. All six observers had some exposure to treating hip disorders, yet none had an established practice focus in this area.

The radiographic parameters measured included: (Figure 1) the lateral center-edge angle (LCEA) of Wiberg,⁵ (Figure 2) vertical-center-anterior (VCA) angle of Lequesne,⁶ (Figure 3) Tonnis angle,⁷ (Figure 4) head-neck offset ratio^8,9,11 (on frog-lateral and cross-table lateral radiographs), and (Figure 5) alpha angle10 (on frog-lateral and cross-table lateral radiographs). Observers were also asked to estimate the degree of joint degeneration based on the Tonnis classification of osteoarthritis. ¹¹ The specific techniques for making these measurements have been previously reviewed³⁰ and are succinctly summarized below.

Figure 1.

The lateral center-edge angle (LCEA) of Wiberg.

Figure 2.

Vertical-center-anterior (VCA) angle of Lequesne.

Figure 3.

Tonnis angle.

Figure 4.

Head-neck offset ratio.

Figure 5.

Alpha angle.

Before the initial radiographic review, the study participants underwent a training session to review the study protocol and to review the originally described techniques for making the measurements in question. These techniques were summarized in written form and were available to the readers as a reference. Case examples were utilized to demonstrate the measurement techniques. Each observer performed the radiograph review independently and was blinded to the other participants' measurements and diagnoses. To assess intraobserver reliability, a second review of the image sets took place at least 6 weeks after the first. One reader (blinded) provided a single read only for purposes of interobserver reliability. For all other readers, image sets remained the same for the second read, however, the sequence of images was randomly altered and each image set was labeled with a new identification number. Observers were asked to remain blinded to their initial reads, were not allowed access to the images between sessions, and were asked not to discuss their data with other study participants between readings.

Source of funding: Washington University Institute of Clinical and Translational Sciences Research Grant These resources were utilized for research personnel support.

STATISTICS

Before initiating the study, a biostatistician was employed to determine the appropriate number of cases needed. A minimum sample size of 15 per group was determined to provide adequate spread across the rating categories and thus provide adequately precise confidence intervals for the reliability estimates. As the reliability coefficients in this study were not assessed for statistical significance, no formal sample size calculation was performed.

Intraclass correlation coefficients (ICCs) and corresponding 95% confidence intervals (CIs) were calculated to quantify interrater and intrarater reliability for continuous variables. For intraobserver reliability, agreement was calculated for each reader separately as well as for raters combined. Because of the potential for bias and practice effects, we did not calculate interobserver reliability using second readings. Kappa^31,32 values were utilized to determine interobserver and intraobserver reliability for categorical variables (diagnosis and Tonnis OA grade). The simple kappa was reported for the unordered variable of diagnosis. The weighted kappa is a refinement of the kappa coefficient that takes into account the magnitude of the disagreement between ratings and was used for Tonnis OA grade as a result of the ordering of the variable. Kappa values and ICCs of 1.0 are indicative of perfect agreement, whereas values less than 1.0 suggest progressively less agreement between readers (in the case of interobserver reliability) or between reads (in the case of intraobserver reliability). The classification scheme of Landis and Koch³¹ was utilized to provide an additional, more generalized tool for understanding the kappa results. In this system, kappa values of 0.81-1.0 are indicative of excellent agreement; 0.61-0.80, substantial agreement; 0.41-0.60, moderate agreement; 0.21-0.40, fair agreement; 0-0.20, slight agreement; and values less than 0, poor agreement.

RESULTS

In general, readers were substantially more consistent with individual measurements from read to read than they were comparing those same measurements across readers. Intrarater reliability was substantial to excellent, with kappa values greater than 0.7, for all of the continuous variables studied, with the lowest combined intrarater reliability of ICC = 0.75 existing for measurements of head-neck offset on the cross-table lateral (Table 1). Combined intrarater agreement for the single categorical variable (Tonnis OA grade) was also limited, having a kappa value of 0.57 (moderate agreement). The majority of the lowest individual intrarater reliability scores for the continuous variables came from measurements of the alpha angle, with ICC ranging from 0.33-0.47 (fair to moderate agreement). All readers achieved excellent intrarater agreement for LCEA (ICCs a 0.8); whereas agreement across readers for VCA ranged from 0.67 to 0.97. As indicated by the non-overlapping CIs, measures of the femoral head-neck morphology (HNO and alpha angle) were less reliably identified between reads than measures of acetabular morphology (CEA VCA and Tonnis angle). For measurements of head-neck offset ratio, combined intrarater reliability minimally improved on the frog-lateral radiograph compared to the cross-table lateral, unlike the alpha angle, which had the same combined reliability for the two different views. Combined intrarater reliability in making a radiographic diagnosis was substantial, as demonstrated by a kappa value of 0.66.

Table 1.

Variable	RATER					All raters combined
	1	2	3	4	5
CEA	0.82 (0.70,0.90)	0.88 (0.79,0.93)	0.92 (0.86,0.95)	0.86 (0.76,0.92)	0.87 (0.78,0.93)	0.88 (0.85,0.91)
VCA	0.67 (0.47,0.80)	0.75 (0.59,0.86)	0.96 (0.94,0.98)	0.97 (0.95,0.98)	0.88 (0.79,0.93)	0.88 (0.85,0.91)
Tönnis Angle	0.90 (0.83,0.94)	0.72 (0.55,0.84)	0.87 (0.77,0.92)	0.72 (0.55,0.84)	0.93 (0.87,0.96)	0.83 (0.79,0.87)
HNO (Frog)	0.77 (0.62,0.87)	0.63 (0.41,0.77)	0.90 (0.83,0.94)	0.84 (0.72,0.91)	0.78 (0.64,0.88)	0.78 (0.72,0.83)
α (Frog)	0.70 (0.51,0.82)	0.68 (0.49,0.81)	0.84 (0.73,0.91)	0.33 (0.05,0.57)	0.58 (0.35,0.74)	0.76 (0.70,0.81)
HNOQCTL)	0.53 (0.29,0.71)	0.45 (0.18,0.65)	0.36 (0.08,0.59)	0.82 (0.69,0.89)	0.50 (0.24,0.69)	0.75 (0.69,0.80)
α CCIL)	0.47 (0.21,0.67)	0.29 (0.01,0.54)	0.74 (0.57,0.85)	0.93 (0.88,0.96)	0.64 (0.43,0.78)	0.76 (0.70,0.81)
Tönnis OA grade	0.31 (0.09,0.53)	0.57 (0.35,0.78)	0.93 (0.83,1.00)	0.53 (0.23,0.82)	0.49 (0.31,0.67)	0.57 (0.47,0.66)

Table 1. Intrarater reliability for 1^st and 2^nd reading per rater for continuous (CEA, VCA, Tönnis angle, Head-neck offset ratio (HNO) and alpha angle) and categorical variables (Tönnis osteoarthritis grade and diagnosis). Data are ICC (95% CI for ICC) for continuous variables and kappa values for categorical variables. Abbreviations: CEA = lateral center-edge angle; VCA = anterior center-edge angle; α = alpha angle; Frog = frog leg lateral; XTL = cross table lateral; OA = osteoarthritis.

Interrater values were substantially lower than those noted for combined intrarater reliability. The LCEA was the most reproducible between readers, having an ICC = 0.64 (Table 2). Measures of acetabular morphology were collectively more reproducible than those for femoral morphology, though the interrater value for head-neck offset ratio on the frog-lateral had the second highest kappa value (0.55). Interrater values were significantly lower on the cross-table lateral radiograph than they were on the frog-lateral radiograph. For measures of femoral morphology, readers had more difficulty reproducing the alpha angle than the HNOR (0.55 vs. 0.21 on the frog-lateral and 0.08 vs. -0.08 on the cross-table lateral). Readers had only slight agreement in making a determination of the degree of osteoarthritis (kappa = 0.17).

TABLE 2.

Variable	κ or ICC (95% CI)
CEA	0.64 (0.52,0.75)
VCA	0.38 (0.26,0.53)
Tönnis Angle	0.42 (0.29, 0.56)
HNO (Frog)	0.55 (0.42,0.68)
α (Frog)	0.21 (0.11,0.36)
HNO (XTL)	0.08 (-0.01, 0.20)
α (XTL)	-0.08 (-0.12,-0.01)
OA grade	κ-0.17

Table 2. Interrater reliability for agreement across multiple raters (using 1^st readings only).

Data are multiple kappas (K) for categorical variables (OA grade, film quality, diagnosis) and ICCs for continuous variables (95% CI for ICC).

Abbreviations: CEA=lateral center-edge angle; VCA=anterior center-edge angle; α= alpha angle; Frog = frog leg lateral; XTL = cross table lateral; OA = osteoarthritis.

DISCUSSION

The purpose of this study was to assess the ability of a collection of readers (with differing specialties and varied levels of clinical experience in treatment of hip disorders) to reliably calculate multiple commonly used radiographic parameters of structural hip anatomy. None of the observers had an established practice focus on the treatment of this patient population. This specific collection of observers was utilized to simulate the variety of healthcare providers that evaluate these patients.^3,33 In general, we found that readers were able to reproduce their individual measurement techniques from read to read, but that readers as a whole were unable to reliably agree on precise measures of acetabular and femoral morphology.

There are certain limitations to the study. First, despite a standardized imaging protocol, inconsistencies in radiographic technique were inevitable as multiple technicians were involved in the creation of the radiographs. This was particularly reflected in the quality of cross-table lateral films which lacked uniformity in both limb positioning (degree of femoral rotation) and beam penetration. This likely contributed to the markedly lower kappa values and inter-rater reliability noted on these views compared to the frog-leg lateral.

Secondly, an argument could be made that the limited number of readers with specific training in joint reconstructive surgery negatively impacted the results. This may be true, yet our goal was to assess a variety of health care providers to mimic clinical practice.

Other investigators have recently reported on hip radiographic reading reliability. Using two experienced readers and 100 hips, Tannast et a.²² evaluated the lateral center-edge angle (LCEA), anterior center edge angle (VCA), and acetabular inclination in their study on validation of the Hip2Norm software. They found in-traobserver reliability for acetabular inclination to range from a kappa value of 0.74-0.89 (compared to a range of 0.72-0.93 in our study), and for interobserver reliability, an interclass correlation coefficient of 0.61 (compared to 0.42 in our study). They noted intraobserver kappa values of 0.97 to 0.98 for the lateral center-edge angle (compared to 0.82 - 0.92), and an interobserver ICC = 0.92 (compared to 0.64). For the anterior center edge angle, they noted intraobserver kappa values of 0.54 to 0.69 (compared to 0.67-0.97), and an interobserver ICC = 0.63 (compared to 0.38). While our interobserver numbers remain comparatively lower, in the case of acetabular inclination this might be partially explained by the use of the lateral edge of the acetabular sourcil as the lateral marker for inclination (as opposed to the most laterally projected point of the acetabulum, as done in the Tannast study). Though the inclination of the weight-bearing dome of the acetabulum is comparatively more representative of structural instability, finding the true lateral aspect of the sourcil can be difficult, and is somewhat dependent on radiographic technique. For purposes of creating a more reproducible marker of dysplasia, use of the technique as performed by Tan-nast et al. might prove more reliable, even in the hands of a larger number of readers. Additionally, this study highlights impaired reliability with experienced readers and with a computerized software system.

Steppacher et al²¹ evaluated the reliability of the Ton-nis classification of osteoarthritis utilizing two readers and 50 image sets. They found intraobserver reliability of κ=0.73-0.76 and interobserver reliability of κ=0.74. Both values are higher than the mean combined intraobserver kappa value of 0.57 and the interobserver kappa value of 0.17 noted in our study. likewise, in our previous study with expert readers, ²⁴ kappa values were also lower than those noted by Steppacher et al with combined intraobserver values of 0.60 and interobserver values of 0.59. As we noted previously, the difference in values might be secondary to the introduction of a “normal” cohort of patients in both of our studies (versus a dysplastic only cohort in their study), as the distinction between grade 0 osteoarthritis (seen in the normal cohort) and grade 1 osteoarthritis (seen in some of the less severe cases of impingement and dysplasia) can sometimes be difficult and dependent on radiographic quality.

In summary, this study highlights the limited ability of various readers to reproduce radiographic measurements for structural hip anatomy. Nevertheless, individual readers are able to reliably recreate such measurements. These data are important and suggest that there is a need to standardize and educate providers regarding the radiographic evaluation of the hip. Doing so may enhance the ability of healthcare providers to more effectively screen and treat patients with pre-arthritic hip disease. Future efforts should be targeted at improving the reliability of hip radiographic analysis by both primary musculoskeletal caregivers and hip specialists. This may be facilitated by refining or clarifying the definitions for specific radiographic measurements, establishing detailed methodology for making these measurements, and incorporating imaging software programs to improve the reliability between observers.