Reliability of radiographic measurement of fracture angulation in children with femoral shaft fractures

Abstract

Introduction

Clinical decisions are based on x-ray measures of radiographs. However, the reliability of assessing the angular measurement of fractured femurs in children is unknown. The purpose of this study was to determine the intra-and interrater reliability of an x-ray assessment of femoral shaft fracture angulation in children.

Methods

On 2 occasions, 3 raters evaluated 30 radiographs of children aged 4 to 10 years, 3 to 24 months after a femoral fracture. The radiographs were evaluated by an orthopedic surgeon, an orthopedic fellow and a research assistant, using a standardized methodology. Reliability was assessed with the intraclass correlation coefficient (ICC), an index of concordance that ranges from 0–1, where > 0.75 indicates excellent concordance.

Results

The ICC for the intrarater ranged from 0.75 to 0.97. The interrater reliability ICC for anterior or posterior angulation was 0.91 (95% confidence interval [CI] 0.84–0.95) and for varus or valgus angulation was 0.90 (95% CI 0.79–0.95). A change in measurement of more than 5 degrees exceeds the variation in measurement and, therefore, can be attributed to a true shift in fracture position.

Conclusions

With specific standardized protocols, radiographic assessment of femoral fractures is reliable, irrespective of the examiner's level of experience.

Femoral fractures are common^1,2 and expensive.^3–5 Numerous options are available for treatment, including traction, casting, external fixation and flexible intramedullary nails. Treatment is influenced by many factors and varies widely within the Pediatric Orthopedic Society of North America.⁶ Irrespective of treatment, clinical decisions are made using a variety of information, including physical assessment and interpretation of radiographs. Initially, radiographic measurements are used to assess satisfactory fracture position.⁷ One of the major components of satisfactory position is the angulation of the fracture fragments. Subsequently, a change between radiographic measurements may indicate a shift in fracture position that might require such treatment as cast wedging, adjustment of a fixator or change to a different modality. However, a change between x-ray measurements may also be due to variability of measurement, in which case, no change in treatment would be indicated. Thus, measures need to be as reliable as possible to attribute change in measures to true clinical change.⁸ The purpose of this study was to evaluate the reliability of angulation measurements from radiographs of children with femoral fractures.

Methods

The radiographs used in this study were those of children aged 4 to 10 years, who were admitted to 4 pediatric hospitals, had consented to participate in a prospective trial and had received either early application of hip spica or external fixation.⁹ The study received research ethics board approval in each site, and we obtained informed consent from the participants' parents or guardians. Radiographs consisting of anterior or posterior and lateral views were obtained at injury and at 3, 9, 15 and 24 months postinjury.

Three examiners, including a staff pediatric orthopedic surgeon, a pediatric orthopedic fellow and a research assistant, reviewed the radiographs. The research assistant had no prior experience evaluating radiographic fracture positions.

The measurement techniques, discussed below, were decided a priori. Although all examiners knew their measurements were part of a reliability study, no joint session or common discussions occurred. Each rater interpreted the radiographs independently and was allowed unrestricted time on each set of radiographs. Raters were blinded to their own and to other raters' evaluations. Radiographs were numbered sequentially with all data regarding patient identity or study site removed or blocked out.

All x-ray measurements were performed with a soft-lead pencil and the same protractor. After the measurements had been recorded, the markings were erased with isopropyl alcohol. Each rater evaluated the angular position of the fractures on the radiographs on 2 separate occasions, with at least a 14-day interval between measurements. We chose 14 days to minimize the recall of prior measurements. During the subsequent evaluations, radiographs were presented in random order to guard against any recall bias from the first interpretation.

For the measurement of fracture positions, radiographs were oriented to the upright position, and 2 lines were drawn so they intersected in the centre of the intramedullary canal, parallel to the shaft of the proximal segment of the femoral shaft, and in the centre of the intramedullary canal, parallel to the shaft of the distal segment of the femoral shaft. This technique is essential to avoid using the centre of the rounded-off apex of the fracture, thereby falsely minimizing the angular deformity.¹⁰ The protractor was aligned with the zero mark against the proximal segment. The angle reported was then measured as varus or valgus angulation, using the anterior or posterior view, and the anterior or posterior angulation was measured using the lateral view.

Reliability was quantified with SPSS 10.1 for Windows for both intrarater and interrater reliability, using the intraclass correlation coefficient (ICC). The ICC, a test of concordance or agreement for continuous data, such as measurement angles, ranges from 0 to 1, with ICC greater than 0.75 indicating excellent reliability.¹¹ The random effects model allows for extrapolation to all raters and is useful for the intrarater and interrater trials.⁸ To assess interrater reliability, the average ICC over 2 occasions was used.

The sample size for the study was estimated for a random effects ICC. Assuming an alpha level of 0.05, beta level of 0.20, ICC of 0.8 and taking 2 measurements per subject, the estimated sample size was 30 patients.¹²

To quantify the variability of measurements obtained by the 3 raters, we performed limits of agreement. In this technique, described by Bland and Altman,¹³ the limits of agreement are the mean of the difference plus or minus 2 standard deviations (SDs). That is, 95% limits of agreement = 2s. Change in fracture position beyond the limits of agreement can be more confidently attributed to true change rather than to simple variability of measurement.

We performed correlations to examine the covariance between pairs of raters for both varus or valgus and anterior or posterior angles.¹⁴

Results

Of the 30 patients included in this study, 17 were male and 13 were female. Sixteen received hip spica and 14 received external fixation (all casts and fixators had been removed at the time of the radiographs). Of the 30 patients, 5 had spiral, 11 had oblique and 5 had spinal fractures. Patients were obtained from all 4 centres and thus are typical of children with femoral fractures in the age range of 4 to 10 years. The definition of malunion, the primary outcome of the study, was defined as more than 10º of varus or valgus, or 15º of anterior or posterior angulation. The results of the intrarater ICC are presented in Table 1. The research assistant achieved the best rating, with an ICC of 0.97 for both anterior or posterior and varus or valgus angulation. The staff surgeon's initial ICC was 0.86 and 0.55 for anterior or posterior and varus or valgus angulation, respectively. After another review was performed, the ICCs were 0.90 and 0.75, respectively, using the first and third x-ray readings.

Table 1

Interrater reliability for anterior or posterior angulation was 0.91 (95% confidence interval [CI] 0.84–0.95) and varus or valgus angulation was 0.90 (95% CI, 0.79–0.95). The overall reliability for both varus or valgus and anterior or posterior angulation was excellent.

The limits of agreement, as shown in Table 2, demonstrate that the 3 raters, regardless of the angle, were almost identical. Agreement between the research assistant and the staff orthopedic surgeon was the strongest.

Table 2

The Pearson correlation coefficients, as displayed in Table 3, were positive and extremely strong for all pairs of rater comparisons with r = 0.9 (p = 0.01).

Table 3

Discussion

Measurement of fracture angulation is fundamental to fracture care. For all fractures, including femoral fractures, radiographs are used to determine an appropriate reduction and to subsequently provide criteria for malunion. Reliability is a basic prerequisite of all clinic measurements and is defined as obtaining the same measurement by the same or by a different clinician.⁸ Variability in measurement can arise from 3 sources: the patient, the procedure and the clinician.⁸

This paper focuses on the intra-rater and interrater reliability of clinicians' reading of radiographs. Physicians who see patients with a femoral fracture should know the precision and reproducibility of the angles that are measured. There is no universally accepted definition of an acceptable reduction for femoral fractures in children. However, in the randomized trial, we used more than 10° of varus or valgus or 15° of anterior or posterior angulation.⁹ To guide clinicians on what constitutes a true change in fracture position, we used the limits of agreement. The results of this study suggest that a change in angulation of more than 5° can be attributed to a true shift in fracture position. This may seem intuitive, but it should be reassuring to clinicians who frequently assess patients. When fractures move beyond this limit, and if the progression is into the unacceptable range, then treatment, such as cast wedging or fixation adjustment, may be required.

The least experienced rater in this study performed as well or better than the more experienced raters. This indicates that skill in rating radiographs is not dependent on clinical experience. Although the initial reliability of the staff was moderate, the absolute variance was relatively small. Remeasurement demonstrates that precision can be improved simply through awareness.

The most important source of variability not assessed in this study was the procedure involved in taking x-rays, specifically patient positioning. For the appraisal of angular deformity, a potential source of variability is the effect of rotation of the limb. Given the radiation exposure, repeated x-rays of patients at the same time is not ethical. However, previous research in cadaveric models has shown that rotation of up to 10º has a minimal effect on angular deformity.¹⁵ Although the results of this study show high levels of reliability for the radiographic assessment of fracture position, this estimate provides only a minimal level of reliability, given the inability to perform repeat x-rays on the same visit. The clinicians involved had neither special training nor a joint session to practice their technique. Standardizing the technique and tools controlled the measurement process. If the results of this study are to be generalized to other settings, clinicians should use a similar method of determining where to place the lines, as described in this study.

In conclusion, the measurement of femoral fracture angulation from radiographs, using a standardized technique, is reliable regardless of the clinician's experience. With the method described in this study, clinicians can be confident that a true change in fracture angulation of 5° probably constitutes a true change in fracture position.

Accepted for publication July 29, 2005.

Competing interests: None declared.

Correspondence to: Dr. James G. Wright, The Hospital for Sick Children, 555 University Ave., Toronto ON M5G 1X8; fax 416 813-6433; james.wright@sickkids.ca