Abstract
Purpose
Distal radius fractures are a common injury. In the emergency room, trainees regularly assess these fractures using visual estimation. Our hypothesis is that assessment of radiographic parameters has sufficient accuracy for rendering treatment consistent with formal measurements.
Methods
This study compared visual measurements made by 25 orthopaedic residents and attending physicians to formal measurements made by a single fellowship trained musculoskeletal radiologist in a series of patients with distal radius fractures. A search was performed utilizing the ICD-9 code for distal radius fracture in all patients presenting to a single institution emergency department. Participants used visual estimation to rate 25 radiographs. Parameters estimated included radial inclination, radial height, volar tilt, and the presence of intra-articular displacement. Analysis using Lin concordance coefficients, Bland Altman plots, and the Kappa statistic evaluated the agreement between visual estimation and formal measurements. The proportion of raters whose estimates would have resulted in a course of treatment that conflicted with the formal reading quantified the potential impact of visual estimation on treatment.
Results
Concordance coefficients were poor for radial inclination (ρc = 0.13), radial height (ρc = 0.24), and volar tilt (ρc = 0.46). The Kappa statistic for intra-articular displacement was 0.4. Analysis performed according to level of training did not result in substantial improvements in these statistics. Treatment based on visual estimates conflicted with formal readings 34 % of the time for radial inclination, 38 % of the time for radial height, 27 % of the time for volar tilt, and 31 % of the time for intra-articular displacement.
Discussion
Visual estimation is not an adequate form of measurement for evaluation of patients with distal radius fractures. Physicians should be mindful of these results when developing treatment plans based solely upon visual estimation.
Keywords: Distal radius fracture, Radiographic interpretation, Radial inclination
Introduction
Physicians routinely evaluate radiographic parameters when developing initial management plans for patients with distal radius fractures. Many parameters have been used to describe fracture alignment including ulnar variance, palmar tilt, radial tilt, radial inclination, radial height, intra-articular step-off, teardrop angle, and anterior-posterior distance [1, 2]. Visual estimates are compared to established parameters (i.e., a radial inclination of 23° (Fig. 1a), a radial height of 12 mm (Fig. 1b), a volar tilt of 12° (Fig. 1c), and—for intra-articular fractures—an incongruent radiocarpal joint with a step-off), which, in conjunction with age, may predict stability of distal radius fractures [3, 4]. Widely accepted guidelines for nonoperative treatment following reduction may include less than a 5° loss of radial inclination (preferably to about 20°), restoration of the radial height to within 2–3 mm of normal, neutral volar tilt, and less than 1 mm of radiocarpal joint step-off [5–8]. As pre-injury or contralateral films are not always available, we use these guidelines at our institution.
Fig. 1.
a Radiographic measurement of radial inclination, b radiographic measurement of radial height, c Radiographic measurement of volar tilt (−13°), d Radiographic measurement of intra-articular displacement
Emergency room personnel frequently describe radiographic findings of distal radius fractures to on-call orthopaedic surgeons. In these situations, personnel visually estimate radiographic parameters despite current user-friendly digital software. This may be due to time constraints, lack of availability, or provider confidence in visual measurements. While computed tomography (CT) offers the capability to improve evaluation of fracture alignment, additional imaging is rarely necessary unless plain films are inconclusive [9–13].
The purpose of this study was to compare visual estimation to formal digital measurements of radiographic parameters in patients with distal radius fractures. To our knowledge, there are no reports published in the literature comparing visual estimation to formal digital measurements of radiographic parameters of distal radius fractures.
Methods
This was a retrospective comparative study of estimated visual measurements made by orthopaedic residents and attending physicians to formal digital measurements made by a fellowship trained musculoskeletal radiologist in a series of patients with distal radius fractures prior to reduction or treatment. To simulate the emergency department (ED) environment, radiographs with three appropriate views were included, regardless of fracture severity or displacement. The Institutional Review Board reviewed and approved the protocol prior to study initiation.
Plain Radiographs
A search was performed utilizing the ICD-9 code for distal radius fracture in all patients presenting to our institution who had three wrist radiographic views (posteroanterior, oblique, and lateral) from 2008–present. A random selection process identified 25 patients. AO/OTA (fracture classification and treatment received) classifications of the 25 radiographs were as follows: eight were 23-C2, seven were 23-A2, five were 23-A3, four were 23-C3, and one was 23-B3. The AO/OTA classification system uses an alphanumeric format to categorize fractures based on their anatomical location and morphology. The number 23 refers to the distal radius (2-radius, 3-distal segment). The letters A (extra-articular), B (partial articular), and C (complete articular) are used to broadly described the type of fracture, whereas the numbers following the letter allow for subgrouping based on specific details of the fracture (A1 simple, A2 wedge, A3 complex; B1 split, B2 depression, B3 split-depression; and C1 simple articular, simple metaphyseal, C2 simple articular, complex metaphyseal, C3 complex articular, complex metaphyseal). Table 1 presents the AO/OTA classification and treatment received by each of the 25 patients included in this study.
Table 1.
AO/OTA fracture classification and treatment choice
| AO/OTA Classification | Treatment Received | |
|---|---|---|
| Radiograph | ||
| 1 | 23-C2 | Nonoperative |
| 2 | 23-A2 | Nonoperative |
| 3 | 23-C2 | Nonoperative |
| 4 | 23-A3 | Operative |
| 5 | 23-A3 | Operative |
| 6 | 23-C2 | Nonoperative |
| 7 | 23-C2 | Nonoperative |
| 8 | 23-C3 | Operative |
| 9 | 23-A3 | Operative |
| 10 | 23-A2 | Nonoperative |
| 11 | 23-A2 | Operative |
| 12 | 23-A2 | Nonoperative |
| 13 | 23-A2 | Operative |
| 14 | 23-C3 | Operative |
| 15 | 23-A2 | Operative |
| 16 | 23-C2 | Operative |
| 17 | 23-C3 | Nonoperative |
| 18 | 23-C3 | Operative |
| 19 | 23-C2 | Operative |
| 20 | 23-A3 | Operative |
| 21 | 23-C2 | Nonoperative |
| 22 | 23-C2 | Operative |
| 23 | 23-A2 | Operative |
| 24 | 23-A3 | Operative |
| 25 | 23-B3 | Operative |
| Total | ||
Raters
Twenty-five physicians served as raters for this study. Twenty-one were orthopaedic surgery residents (five PGY1, five PGY2, four PGY3, three PGY4, and four PGY5), two were attending orthopaedic physicians, and two were attending orthopaedic physicians with formal hand fellowship training. Raters provided visual estimates for the radiographic parameters and a choice of treatment (operative vs. nonoperative), without the use of any measuring devices. All raters had experience in primary trauma call at our institution and were blinded to all patient identification factors, as well as knowledge of prior radiographic analysis and the type of treatment received at the time of fracture presentation.
Radiograph Assessment
To simulate how distal radius fractures present in the ED, we asked the raters to evaluate plain radiographs presented in slideshow form. Raters evaluated radiographic images projected from a monitor and magnified onto a large screen in a room with ambient light. Raters were asked to visually estimate the following parameters:
Radial inclination: is the angle between the articular surface of the radius and the radial styloid, measured on the posteroanterior (PA) view (Fig. 1a).
Radial height: is the difference in length between the ulnar head and the tip of the radial styloid on the PA view (Fig. 1b).
Volar tilt (palmar inclination): at the articular surface of the radius, a tangent line is drawn from dorsal-to-volar, followed by a line perpendicular to the long axis of the radius. The angle formed measures the volar tilt (Fig. 1c). Raters used the lateral view to measure volar tilt. Dorsal angulation of the distal radius articular surface correlates with a negative volar tilt.
Intra-articular displacement: Raters measured the amount of articular displacement at the radiocarpal joint in millimeters. Two millimeters or greater (gap or step-off) constituted a rating of intra-articular displacement (Fig. 1d). Raters used the PA and oblique views to measure intra-articular displacement.
A fellowship trained musculoskeletal radiologist at our institution blinded to the raters’ visual estimations made formal digital measurements of the radiographic parameters for each distal radius fracture. The radiologist evaluated the images using the Phillips-Stentor iSite Picture Archiving and Communication System (PACS). The radiologist made a single measurement under no magnification in a dark room from a computer monitor utilizing the same guidelines for measuring each radiographic parameter as described previously.
Statistical Analysis
Analysis using Lin concordance coefficients and Bland Altman plots for continuous variables and the Kappa statistic for binary variables (intra-articular displacement) evaluated the utility of visual estimation. Lin concordance analysis evaluates both precision and accuracy, and is therefore preferable in comparison to Pearson correlation coefficient, which does not take into account systematic error. There are no known categories for concordance values, however, other investigators have suggested broad categories as follows: High (ρc ≥ 0.8), good (0.7 ≤ ρc < 0.8), fair (0.6 ≤ ρc < 0.7), and poor (ρc < 0.6) [1].
This analysis incorporated Bland Altman plots to graphically represent agreement and measurement error. The mean value of each visual estimation and formal measurement is calculated and plotted on the x-axis. The difference between each visual estimation and formal measurement is calculated and plotted on the y-axis. Each point represents one measurement by a rater. A horizontal line running through the zero point of the y-axis represents perfect agreement. A point falling on this line would indicate no difference between the formal and visual readings. The closer points are to the centerline plot, the more agreement in the measurements. The lines above and below the centerline demarcate the 95 % limits of agreement. To examine visual estimation according to level of training, raters were stratified. Inter-rater reliability of rater’s visual estimates was also calculated. The Intraclass Correlation with absolute agreement (ICC; two way random model, with absolute agreement) was calculated to measure inter-rater reliability for radial inclination, radial height, and volar tilt, whereas the Kappa statistic was calculated to measure inter-rater reliability for intra-articular displacement.
For each parameter, a clinically relevant threshold was established. The formal reading served as the gold standard for determining treatment (operative vs. nonoperative), although in clinical practice this determination is of course based upon a multitude of patient-specific factors. The proportion of raters whose visual estimates fell outside the clinical threshold was calculated. This proportion represents the percentage of visual estimates that conflicted with the treatment indicated by the formal reading, thereby resulting in a different course of treatment. Radial inclination of 17° or less, radial height of 8 mm or less, negative volar tilt, and intra-articular displacement served as the thresholds for this analysis. For example, the parameter for radial inclination is 23° with a post reduction loss of up to 5° considered acceptable. In this regard, if a given radiograph’s formal reading was 19° (indicating nonoperative treatment) and the rater’s estimate was 12° (indicating operative treatment), the visual reading would have crossed the threshold. An alpha level of 0.05 was set for all analyses. Stata 12 (StataCorp. 2011. Stata Statistical Software: Release 12. College Station, TX: StataCorp LP) was used for all analysis.
This was an exploratory study aimed at evaluating the utility of visual estimates and as such a power analysis was not performed. Specifically, an a priori meaningful difference in rater reliability was not identified, which is a requirement of a sample size calculation. The authors decided a sample of 25 radiographs would give an appropriate idea of the reliability of these measures.
Results
Table 2 presents the concordance coefficients. Overall, concordance coefficients were relatively low for radial inclination (RI, ρc = 0.13; p < 0.01), radial height (RH, ρc = 0.24; p < 0.01), and volar tilt (VT, ρc = 0.46; p < 0.01). The Kappa statistic for intra-articular displacement was 0.4 (p < 0.01). Subgroup analysis according to level of training demonstrated small changes in reliability and agreement (Table 2). Figure 2a–c displays the Bland Altman plots for radial inclination, radial height, and volar tilt. The limits of agreement were as follows: radial inclination (−22, 24), radial height (−16, 15), and volar tilt (−44, 34). ICC values for inter-rater reliability were as follows: radial inclination 0.06, radial height 0.12, and volar tilt 0.16. The Kappa statistic for inter-rater reliability for intra-articular displacement was 0.42.
Table 2.
Rater reliability and agreement
| Concordance coefficent (rho_c) | p value | 95 % limits of agreement | |
|---|---|---|---|
| All raters (n = 25) | |||
| Radial inclination | 0.13 | <0.01 | −22, 24 |
| Radial height | 0.24 | <0.01 | −16, 15 |
| Volar tilt | 0.46 | <0.01 | −44, 34 |
| Fellowship trained (n = 2) | |||
| Radial inclination | 0.04 | 0.79 | −22, 24 |
| Radial height | 0.35 | 0.01 | −13, 13 |
| Volar tilt | 0.62 | <0.01 | −38, 35 |
| Not fellowship trained (n = 2) | |||
| Radial inclination | 0.10 | 0.47 | −24, 22 |
| Radial height | 0.19 | 0.11 | −19, 12 |
| Volar tilt | 0.51 | <0.01 | −43, 35 |
| PGY3 through 5 (n = 11) | |||
| Radial inclination | 0.17 | <0.01 | −21, 23 |
| Radial height | 0.27 | <0.01 | −13, 14 |
| Volar tilt | 0.47 | <0.01 | −42, 34 |
| PGY1 and 2 (n = 10) | |||
| Radial inclination | 0.13 | 0.04 | −23, 26 |
| Radial height | 0.32 | <0.01 | −13, 15 |
| Volar tilt | 0.4 | <0.01 | −48, 34 |
Fig. 2.
Bland Altman plots for Radial Inclination (a), Radial Height (b), and Volar Tilt (c)
The average agreement between the rater’s recommendation of treatment and actual treatment received by the patient was 60 %. Treatment based on visual estimates conflicted with formal reading thresholds 34 % of the time for radial inclination, 38 % of the time for radial height, 27 % of the time for volar tilt, and 31 % of the time for intra-articular displacement. Subgroup analysis according to level of training did not reveal substantial changes in clinically relevant threshold analysis (Table 3).
Table 3.
Subgroup analysis
| Radial inclination | Radial height | Volar tilt | IA Displacement | |
|---|---|---|---|---|
| All raters | 34 | 38 | 27 | 31 |
| Fellowship trained | 34 | 34 | 20 | 24 |
| Not fellowship trained | 40 | 42 | 28 | 26 |
| PGY3 through 5 | 33 | 37 | 23 | 32 |
| PGY1 and 2 | 34 | 38 | 32 | 32 |
Percentage of visual estimates that fell outside the clinical threshold thereby resulting in a course of treatment that conflicted with the treatment indicated by the digital reading
Discussion
The purpose of this study was to compare visual estimation to formal digital measurements of radiographic parameters in patients with distal radius fractures. In comparison to formal measurements, visual estimation demonstrated poor reliability in all radiographic parameters. Analysis performed according to the level of training did not result in substantial improvements in the accuracy of visual estimations.
The wide limits of agreement and poor reliability observed in this study suggest that visual estimation alone may not be a sufficient form of measuring radiographic parameters in patients with distal radius fractures. Taking into account anatomic variation, radiographic parameters defined as within normal limits include a radial inclination of 23° (range 13–30°), a radial height of 12 mm (range 8–18 mm), and a volar tilt of 12° (range 1–21°) [3]. The limits of agreement for radial inclination (−22, 24), radial height (−16, 15), and volar tilt (−44, 34) fall outside the ranges of these parameters. This study’s rating of intra-articular displacement demonstrated a Kappa coefficient of 0.4, which is indicative of poor inter-observer agreement.
Experience and training may improve the accuracy of measurement by visual estimation [2]. In our study, accounting for fellowship training and postgraduate year in training did not substantially improve concordance coefficients or kappa values. These statistics improved marginally among attending physicians for volar tilt and intra-articular displacement, with the greatest improvements seen among hand fellowship-trained physicians. Interestingly, radial inclination measures were worse as level of training increased. Physicians in training had higher concordance coefficients for radial inclination than attending physicians. Despite these trends, concordance coefficients and Kappa values remained poor among the subgroups.
In an effort to completely evaluate the accuracy of visual estimation, we collected data regarding treatment recommendations and performed an analysis based on clinical thresholds. In this study, rater recommended treatment agreement with treatment rendered was approximately 60 %. Analysis according to clinically relevant thresholds indicated that visual estimation alone would have resulted in treatment that conflicted with the formal reading approximately 27 to 38 % of the time. These findings indeed suggest that visual estimation alone may not be an accurate predictor of treatment received. The use of pre-reduction films in this study is an important consideration. Ideally, a study evaluating the relationship between visual estimation and treatment would also of course take into account post-reduction films.
Our methodology is an important consideration for accurately interpreting this study. We designed this study to allow the results to be generalizable to the ED, which is frequently where patients who have sustained a distal radius fracture first present. In this regard, the ED radiographs of randomly selected patients served as the study sample in an effort to create a representative sample of patients. While this approach helps to improve the generalizability of study findings, the lack of controlled selection over the radiographs may have resulted in some views that may not be ideally suited to visual estimation.
There are several limitations of this study. Raters made visual measurements on a large screen in ambient light, whereas radiologist measurements were on a computer screen in a dark room. The lack of repeated measures in the protocol prohibited calculation of intra-rater reliability. Because the musculoskeletal radiologist’s digital measurements served as our control, the validity of this study is dependent upon the accuracy of this rater. It should be mentioned that CT scan measurements were not used as the control (since CT scans were not performed on a vast majority of these patients), although this imaging modality can of course be utilized clinically when plain radiographs are insufficient [9–13].
A number of factors, including success or failure of initial treatment, hand-dominance, activity level, general health of the patient, social restrictions, functional status, and age are important considerations for determining treatment of distal radius fractures. In our study, treatment recommendation was based solely on radiographic evaluation rather than a global clinical scenario. Despite the inherent limitations of this study, its results do indeed suggest that visual estimation alone may not be a sufficient form of measuring radiographic parameters in patients with distal radius fractures.
Acknowledgments
The authors would like to acknowledge and thank the following individuals for their help in various roles of data collection and analysis: Greg Polkowski MD; Vinayak Sathe, MD; Michael Abramov, MD; Kyle Nuland, BS; and Bryan Lawlor BS.
Conflict of Interest
Michael P. O’Malley declares that he has no conflict of interest. Craig Rodner declares that he has no conflict of interest. Andrew Ritting declares that he has no conflict of interest. Mark P. Cote declares that he has no conflict of interest. Robin Leger declares that she has no conflict of interest. Harlan Stock is a reviewer for the Journal of Hand Surgery. Jennifer M. Wolf receives compensation as an editor for the Journal of Hand Surgery, as well as grant funding from OREF, AFSH, and NIH.
Statement of Human and Animal Rights
Procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000 and 2008.
Statement of Informed Consent
All identifying factors are eliminated form the text, including but not limited to patients' names, initials, or hospital numbers. There are no patients that are identifiable within the text.
References
- 1.Catalano LW, 3rd, Barron OA, Glickel SZ. Assessment of articular displacement of distal radius fractures. Clin Orthop Relat Res. 2004:79-84. [DOI] [PubMed]
- 2.Cohen MS, Jupiter JB. Fractures of the Distal Radius." Skeletal Trauma. 4. Philadelphia: Saunders Elsevier; 2009. pp. 1405–458. [Google Scholar]
- 3.Geissler WB. Arthroscopic Reduction and Fixation of Distal Radius and Ulnar Styloid Fractures. In: Operative Techniques in Orthopaedic Surgery. Philadelphia: Lippincott William & Wilkins; 2011:2171-2182.
- 4.Grafe MW, Kim PD, Rosenwasser MP. Distal radius fractures: External fixation. In: Wilkins LW, editor. Masters techniques in orthopaedic surgery: Fractures. 2. Philadlephia: Lippincott Williams & Wilkins; 2006. pp. 169–179. [Google Scholar]
- 5.Harness NG, Ring D, Zurakowski D, Harris GJ, Jupiter JB. The influence of three-dimensional computed tomography reconstructions on the characterization and treatment of distal radial fractures. J Bone Joint Surg Am. 2006;88:1315–1323. doi: 10.2106/JBJS.E.00686. [DOI] [PubMed] [Google Scholar]
- 6.Kreder HJ, Hanel DP, McKee M, Jupiter J, McGillivary G, Swiontkowski MF. X-ray film measurements for healed distal radius fractures. J Hand Surg [Am] 1996;21(1):31–9. doi: 10.1016/S0363-5023(96)80151-1. [DOI] [PubMed] [Google Scholar]
- 7.Koval KJ, Zuckerman JD. Distal Radius. In: Handbook of Fractures. Third ed. Philadelphia:: Lippincott Williams & Wilkins; 2006:226-236.
- 8.Lafontaine M, Hardy D, Delince P. Stability assessment of distal radius fractures. Injury. 1989;20:208–210. doi: 10.1016/0020-1383(89)90113-7. [DOI] [PubMed] [Google Scholar]
- 9.Medoff RJ. Essential radiographic evaluation for distal radius fractures. Hand Clin. 2005;21:279–288. doi: 10.1016/j.hcl.2005.02.008. [DOI] [PubMed] [Google Scholar]
- 10.Metz VM, Gilula LA. Imaging techniques for distal radius fractures and related injuries. Orthop Clin North Am. 1993;24(2):217–28. [PubMed] [Google Scholar]
- 11.Pruitt DL, Gilula LA, Manske PR, Vannier MW. Computed tomography scanning with image reconstruction in evaluation of distal radius fractures. J Hand Surg [Am] 1994;19(5):720–7. doi: 10.1016/0363-5023(94)90174-0. [DOI] [PubMed] [Google Scholar]
- 12.Rachkidi R, Ghanem I, Kalouche I, El Hage S, Dagher F, Kharrat K. Is visual estimation of passive range of motion in the pediatric lower limb valid and reliable? BMC Musculoskelet Disord. 2009;10:126. doi: 10.1186/1471-2474-10-126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ruch DS, McQueen MM. Distal radius and ulna fractures. Rockwood & Green's Fractures in Adults. 7. Philadelphia: Lippincott Williams & Wilkins; 2010. pp. 831–80. [Google Scholar]