Abstract
Background
Loss of contact between radial head fracture fragments is strongly associated with other elbow or forearm injuries. If this finding has adequate interobserver reliability, it could help examiners identify and treat associated ligament injuries and fractures (eg, forearm interosseous ligament injury or elbow dislocation).
Questions/purposes
(1) What is the interobserver agreement on radiographic loss of contact between radial head fracture fragments? (2) Are there factors associated with the observer such as location of practice or subspecialization that increase interobserver reliability?
Methods
Fully trained practicing orthopaedic and trauma surgeons from around the world evaluated 27 anteroposterior and lateral radiographs of radial head fractures on a web-based platform for the following characteristics: (1) loss of contact between at least one radial head fracture fragment and the remaining radial head and neck; (2) a gap between fragments of 2 mm or greater; (3) anticipated fracture instability (mobility) on operative exposure; (4) anticipated associated ligament injuries; and (5) recommendation for treatment. Agreement among observers was measured using the multirater kappa measure. Kappas for various observer characteristics were compared using 95% confidence intervals.
Results
The overall interobserver agreement was moderate (range, 0.49–0.55) for each question except associated ligament injury, which was fair (0.33). Shoulder and elbow surgeons had substantial agreement (range, 0.51–0.61) in many areas, but kappas were generally in the moderate range (0.41–0.59) based on number of years in practice, radial head fractures treated per year, and trainee supervision.
Conclusions
Radiographic signs of radial head fracture instability such as loss of contact have moderate reliability. This characteristic seems clinically useful, because loss of contact between at least one radial head fracture fragment and the remaining radial head and neck is strongly associated with associated ligament injury or other fractures.
Level of Evidence
Level III, diagnostic study. See Guidelines for Authors for a complete description of levels of evidence.
Introduction
Some classification systems for fracture of the radial head are based on the degree of displacement and the size of the fracture fragments [2, 5, 10, 14, 15]. The classification by Mason, being the most widely used, is based on fragment characteristics but does not quantify displacement. According to the modified classification by Broberg and Morrey, displacement of more than 2 mm and/or involvement of at least 30% of the articular surface indicates a displaced radial head fracture [2]. A key distinction in fractures of the radial head is between stable fractures where the stepoff is the issue (but usually for block to motion, not arthrosis, which seems rarely to be a problem after these injuries) and unstable fractures where the issue is stability (eg, terrible triad, where the radial head plays a critical role in elbow stability) [8].
Fracture instability at operative exposure (meaning that the fracture is loose and mobile) seems important, because unstable fractures are typically associated with other injuries to the elbow or forearm [6, 11, 17]. Many partial fractures are depressed 2 mm or more but are impacted and stable (it takes force to move them on operative exposure) with an intact periosteum. We are curious whether radiographic factors such as a gap between fracture fragments or loss of contact indicate which fractures are mobile on operative exposure and—more importantly—which are associated with other fractures or ligament injuries [17]. Rineer and colleagues [17] reported that partial articular (Mason 2) radial head fractures with at least one fracture fragment without bony contact (ie, no area where the fracture fragments are immediately adjacent and it is possible that the fracture is stable and difficult to move on operative exposure) are 21 times more likely to be associated with a complex, unstable injury pattern. For this radiographic finding to be useful, its interobserver reliability needs to be demonstrated.
This study addresses the following study questions: (1) Is there agreement between observers on radiographic gap, loss of contact between radial head fracture fragments, anticipated fracture instability/mobility on operative exposure, anticipated associated ligamentous injury, and decision for surgery? (2) Are there factors associated with the observer such as location of practice or subspecialization that increase interobserver reliability?
Patients and Methods
Study Design and Setting
The institutional research board at the principal investigator’s (DR) hospital approved this study. Twenty-seven sets of AP and lateral elbow radiographs from patients treated for a radial head fracture were selected based on image quality to provide a spectrum of displacement and injury patterns. Seven of 23 fractures (30%) were whole head fractures (Figs. 1, 2) and 14 of 23 fractures (61%) were displaced (Figs. 3, 4) according to the criteria by Broberg and Morrey. We invited the members of the Science of Variation Group–fully trained practicing orthopaedic and trauma surgeons from around the world–to evaluate the radiographs on a web-based study platform (SurveyMonkey, Palo Alto, CA, USA).
Fig. 1.
AP radiograph showing a displaced whole head radial head fracture.
Fig. 2.
Lateral radiograph showing a displaced whole head radial head fracture.
Fig. 3.
AP radiograph showing a nondisplaced partial radial head fracture.
Fig. 4.
Lateral radiograph showing a nondisplaced partial radial head fracture.
Participants
Three hundred thirty-three invitations were sent, 179 responses were received (54%; not all of the members treat elbow fractures), and 168 surgeons (92% men, 8% women) completed the online survey (94% of the initial responders). The majority practiced in the Unites States (58%), had more than 5 years of experience (79%), were specialized in either the hand and wrist (41%) or orthopaedic traumatology (45%), had trainees in the operating room (81%), and treated more than 10 elbow fractures a year (53%) (Table 1).
Table 1.
Observer demographics (n = 168)
| Demographic | Number (%) |
|---|---|
| Sex | |
| Male | 155 (8) |
| Female | 13 (13) |
| Area | |
| United States | 97 (58) |
| Europe | 30 (18) |
| Asia | 13 (8) |
| Other | 12 (7) |
| Canada | 10 (6) |
| Australia | 4 (2) |
| United Kingdom | 2 (1) |
| Specialization | |
| General orthopaedic surgeons | 5 (3) |
| Orthopaedic traumatology | 75 (45) |
| Shoulder and elbow | 12 (7) |
| Hand and wrist | 69 (41) |
| Other | 7(4) |
| Treated elbow fractures/year | |
| 0–5 elbow fractures in 1 year | 22 (13) |
| 6–10 elbow fractures in 1 year | 41 (24) |
| 11–20 elbows in 1 year | 47 (28) |
| > 20 elbows in 1 year | 58 (35) |
| Trainees in the operating room | |
| Yes | 136 (81) |
| No | 32 (19) |
Study Description
After logon, the observers were asked general information about their practices. Subsequently, they were asked to answer five multiple-choice questions about fracture characteristics and treatment options for each of the 27 patients: (1) Is there a gap of more than 2 mm between one of the radial head fragments and the intact radius? (2) Is there complete loss of contact between a fracture fragment and the rest of the proximal radius? (3) Is the fracture unstable? (4) Are there likely to be associated ligament injuries or fractures? (5) Would you recommend operative treatment?
Variables, Outcome Measures, Data Sources, and Bias
Independent variables were observer characteristics and fracture characteristics. Dependent variables were binominal (yes/no answers to the questions). Agreement among observers was determined using the multirater kappa measure described by Siegel and Castellan [18]. The multirater kappa measure is a frequently used statistic measure to describe chance-corrected agreement between ratings made by multiple observers (interobserver reliability) or between ratings made by one observer on multiple occasions (intraobserver reliability) [16]. The generated kappa values were interpreted according to the guidelines by Landis and Koch [12]: values of 0.01 to 0.20 indicate slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and more than 0.81, almost perfect agreement. Zero indicates no agreement beyond that expected resulting from chance alone, –1.00 means total disagreement, and +1.00 represents perfect agreement.
The only incentive for observers to participate was group authorship.
Statistical Analysis
As a measure of power, the precision was determined for estimating the true value of kappa in the population based on a 95% confidence interval for a kappa of 0.30 in 27 subjects (ie, 27 radiographs) of radial head fracture rated by 168 observers. According to the Fleiss-Cuzick estimator of kappa in the case of equal numbers of ratings for each subject, the 168 observers and 27 subjects provide a 95% precision of the true kappa value of ± 0.18 around the observed value of kappa [20]. Multirater kappas were calculated with use of SPSS for Windows, 21.0.0.0 2012 (SPSS Inc, Chicago, IL, USA).
Results
Interobserver Agreement
The interobserver agreement of the surgeons who participated in the survey was moderate (reference value, 0.41–0.60) for diagnosing a gap between radial head fracture fragments of more than 2 mm (κ = 0.55) and also for diagnosing a complete loss of bony contact between fragments (κ = 0.43), classifying a fracture of the radial head as unstable (κ = 0.49), and for recommending operative treatment (κ = 0.52). There was fair (reference value, 0.21–0.40) interobserver reliability for diagnosing anticipated ligament injuries associated with the radial head fracture (κ = 0.33) (Table 2).
Table 2.
Interobserver agreement
| Variable | Categorical | κ | (95% CI)* |
|---|---|---|---|
| Gap of > 2 mm | Moderate | 0.55 | (0.53–0.58) |
| Complete loss of cortical contact | Moderate | 0.43 | (0.43–0.44) |
| Unstable fracture | Moderate | 0.49 | (0.48–0.50) |
| Associated ligament injuries/fractures | Fair | 0.33 | (0.32–0.34) |
| Recommend operative treatment | Moderate | 0.52 | (0.51–0.54) |
* CI calculated (x− [1.96 x SEM], x + [1.96 x SEM]); CI = confidence interval.
Factors Associated With Increased Interobserver Agreement
Shoulder and elbow surgeons were the only subset of surgeons that agreed substantially (range, 0.51–0.61) in diagnosing a gap of more than 2 mm, anticipating fracture instability and recommending operative treatment. General orthopaedic surgeons had only fair or slight agreement on all five questions. Surgeons specialized in either orthopaedic traumatology or hand and wrist scored moderate on all five questions except for anticipated ligament injuries (Table 3).
Table 3.
Interobserver agreement for specialization
| Variable | κ 1 (n = 5) | Classification | κ 2 (n = 75) | Classification | κ 3 (n = 12) | Classification | κ 4 (n = 69) | Classification | κ 5 (n = 7) | Classification |
|---|---|---|---|---|---|---|---|---|---|---|
| Gap of > 2 mm | 0.26 | Fair | 0.56 | Moderate | 0.61 | Substantial | 0.56 | Moderate | 0.49 | Moderate |
| Complete loss of cortical contact | 0.2 | Slight | 0.45 | Moderate | 0.51 | Moderate | 0.42 | Moderate | 0.39 | Fair |
| Unstable fracture | 0.3 | Fair | 0.51 | Moderate | 0.61 | Substantial | 0.47 | Moderate | 0.49 | Moderate |
| Associated ligament injuries/fractures | 0.16 | Slight | 0.34 | Fair | 0.35 | Fair | 0.34 | Fair | 0.25 | Fair |
| Recommend operative treatment | 0.29 | Fair | 0.53 | Moderate | 0.61 | Substantial | 0.52 | Moderate | 0.53 | Moderate |
1 = general orthopaedics; 2 = orthopaedic traumatology; 3 = shoulder and elbow; 4 = hand and wrist; 5 = other.
The interobserver variability of complete loss of cortical contact, anticipated fracture instability, and anticipated associated ligament injuries were also affected somewhat by years in practice, supervision of trainees in the operating room, and number of elbow fractures treated in a year (Tables 4, 5, 6).
Table 4.
Interobserver agreement for years in independent practice
| Variable | κ 1 (n = 36) | Classification | κ 2 (n = 45) | Classification | κ 3 (n = 52) | Classification | κ 4 (n = 35) | Classification | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gap of > 2 mm | 0.5 | Moderate | 0.55 | Moderate | 0.58 | Moderate | 0.59 | Moderate | ||||||
| Complete loss of cortical contact | 0.43 | Moderate | 0.45 | Moderate | 0.43 | Moderate | 0.41 | Moderate | ||||||
| Unstable fracture | 0.5 | Moderate | 0.52 | Moderate | 0.53 | Moderate | 0.39 | Fair | ||||||
| Associated ligament injuries/fractures | 0.41 | Moderate | 0.36 | Fair | 0.29 | Fair | 0.27 | Fair | ||||||
| Recommend operative treatment | 0.53 | Moderate | 0.54 | Moderate | 0.53 | Moderate | 0.47 | Moderate | ||||||
1 = 0–5 years in independent practice; 2 = 6–10 years in independent practice; 3 = 11–20 years in independent practice; 4 = 21–30 years in independent practice.
Table 5.
Interobserver agreement for number of elbow fractures treated in 1 year
| Variable | κ 1 (n = 22) | Classification | κ 2 (n = 41) | Classification | κ 3 (n = 47) | Classification | κ 4 (n = 58) | Classification |
|---|---|---|---|---|---|---|---|---|
| Gap of > 2 mm | 0.53 | Moderate | 0.53 | Moderate | 0.59 | Moderate | 0.54 | Moderate |
| Complete loss of cortical contact | 0.4 | Fair | 0.46 | Moderate | 0.44 | Moderate | 0.42 | Moderate |
| Unstable fracture | 0.5 | Moderate | 0.52 | Moderate | 0.51 | Moderate | 0.45 | Moderate |
| Associated ligament injuries/fractures | 0.39 | Fair | 0.33 | Fair | 0.31 | Fair | 0.34 | Fair |
| Recommend operative treatment | 0.52 | Moderate | 0.51 | Moderate | 0.56 | Moderate | 0.5 | Moderate |
1 = 0–5 elbow fractures treated; 2 = 6–10 elbow fractures treated; 3 = 11–20 elbow fractures treated; > 20 elbow fractures treated.
Table 6.
Interobserver agreement for supervision of trainees in the operating room
| Variable | κ 1 (n = 136) | Classification | κ 2 (n = 32) | Classification |
|---|---|---|---|---|
| Gap of > 2 mm | 0.56 | Moderate | 0.53 | Moderate |
| Complete loss of cortical contact | 0.44 | Moderate | 0.42 | Moderate |
| Unstable fracture | 0.49 | Moderate | 0.48 | Moderate |
| Associated ligament injuries/fractures | 0.33 | Fair | 0.33 | Fair |
| Recommend operative treatment | 0.52 | Moderate | 0.5 | Moderate |
1 = yes; 2 = no.
Discussion
Loss of contact between radial head fracture fragments is strongly associated with other elbow or forearm injuries [17]. A reliable radiographic sign of radial head fracture instability could help examiners anticipate and properly treat associated fractures and ligament injuries. We aimed to assess interobserver agreement on radial head fracture characteristics based on radiographs and to try to identify factors associated with increased interobserver reliability.
This study should be interpreted in light of several limitations. The quality of the radiographs was determined by what was obtained in the emergency department and was not standardized. No CT scans were available in the survey. Observers had no information about the patient or the injury. Also, we did not give observers any training or reference values on, for example, what defined “loss of contact” or “gap.” Also, this study was limited to interobserver agreement only, because intraobserver agreement is less relevant to clinical practice and because it is more practical to have the members of the collaborative volunteer one 20- to 30-minute session. Intraobserver agreement tends to be much less of a problem for most types of radiological diagnosis and studies consistently show that more sophisticated imaging improves intraobserver variation more than interobserver variation [1, 13, 19]. We also asked many secondary study questions of the data, including variations based on training and experience, all of which should be interpreted with caution and considered primarily hypothesis-generating. The members of our collaborative and those who chose to participate in this study may not be representative of the average surgeon. Our collaborative includes many types of surgeons and a decision not to participate generally means they are either not familiar with the topic or are too busy at the time.
Because this is a reliability study, we did not have an interoperative evaluation of the fracture as a reference standard and cannot comment on the validity of the answers of the surgeons. The next step would be interoperative verification of loss of bony contact as a radiographic predictor of radial head fracture instability and associated fractures or ligament injuries and determining to what extent these findings changed the prognosis and treatment. Finally, intuitively one might expect higher kappas for, for example, loss of contact, than our study results depict. It is possible that in imbalance in marginal totals (eg, the way in which the fractures with and without loss of contact have been distributed in our study) have resulted in the “ kappa paradox,” a result with lower kappas than one might expect [3, 4, 9].
The interobserver agreement on the radiographic diagnosis of loss of bony contact was moderate overall, suggesting that this is a useful radiographic finding that can be used to guide management and counsel patients. When a surgeon evaluating a radiograph of a radial head fracture diagnoses a gap between fracture fragments and loss of bony contact, he or she should understand that this diagnosis is moderately reliable and strongly associated with a risk of associated ligament injuries or fractures of the forearm or elbow. Our findings are consistent with Doornberg et al. [7] who measured substantial intraobserver and moderate interobserver reliability of classification of radial head fractures according to the modified Mason classification by Broberg and Morrey.
The observations based on training and experience were inconsistent with only the subgroup of shoulder and elbow surgeons having somewhat better agreement, so these should not be overinterpreted. Our interpretation of the findings of these secondary analyses is that there is no single factor (such as experience for instance) that accounts for interobserver variation. Interobserver variation is greater than one would expect even for simple diagnoses such as radial head contact and a large amount of the variation among observers remains unexplained. Interobserver variation therefore merits additional study to determine ways to reduce observer variation other than more sophisticated imaging and simplified classifications (which have only helped to a limited degree).
Loss of contact of radial head fracture fragments on radiographs is associated with intraoperative fragment mobility/instability and usually indicates other injuries to the elbow or forearm, which can be important but subtle (eg, interosseous ligament injury of the forearm). This study documents moderate reliability in the diagnosis of a gap between fracture fragments on radiographs. We recommend that clinicians inspect radiographs for loss of contact and a gap between radial head fracture fragments and scrutinize patients with this finding carefully for possible interosseous ligament injury, self-reduced elbow dislocation, or associated fractures. Future studies can address whether other tests (eg, CT) are even more reliable and accurate.
Footnotes
Each author certifies that he or she, or a member of his or her immediate family, has no funding or commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.
All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request.
Each author certifies that his or her institution approved the human protocol for this investigation, that all investigations were conducted in conformity with ethical principles of research, and that informed consent for participation in the study was obtained.
This work was performed at the Orthopaedic Hand and Upper Extremity Service, Massachusetts General Hospital, Boston, MA, USA.
Science of Variation Group:
Kevin Eng, Vispi Jokhi, David O. Oloruntoba, Sanjev Jain, Parag Melvanki, Frede Frihagen, Iain McGraw, Philipp Lenzlinger, K. J. Ponsen, Andrew Schmidt, Bernhard Ciritsis, Joseph M. Conflitti, Matthias Turina, Rudolf W. Poolman, Ronald Liem, R. S. Gulve, James Wagg, Peter Kloen, Elena Grosso, Matt Mormino, Pradeep Choudhari, Robert D. Zura, Rodrigo Pesantez, George Thomas, Peter Brink, Marc Swiontkowski, Daphne Beingessner, Niels Schep, Nikolaos Kanakaris, R. W. Peters, J. Andrew, I. Trenholm, Ladislav Mica, M. H. J. Verhofstad, Lisa Taitsman, Daniel Hernandez, Ian Harris, Kenneth Egol, Kyle Jeray, Lars C. Borris, Antonio Barquet, James Kellam, John L. Marsh, Jonathan L. Hobby, Parag Sancheti, Lob Guenter, Thomas DeCoster, Brett D. Crist, Gregory J. Della Rocca, Milind Merchant, Michael P. Leslie, Richard Jenkinson, David Weiss, Edward K. Rodriguez, Rena Stewart, John Wixted, Peter Althausen, Peter Schandelmaier, Brad Petrisor, Platz Andreas, Henry Broekhuyse, Clifford Jones, Edward Harvey, Fryda Medina Rodríguez, Hans J. Kreder, Michael Prayson, Raymond Malcolm Smith, Toni M. McLaurin, Arie B. van Vugt, Richard Buckley, Vishwanath M. Iyer, Fred Baumgaertel, Ross Leighton, Reto H. Babst, Qiugen Wang, Edward Yang, Eric Mark Hammerberg, Leon Elmans, Alan Kawaguchi, David B. Carmack, Amal Basak, George Kontakis, Jeremy Hall, Richard S. Page, Taco Gosens, Denise Eygendaal, Martin Richardson, Thomas Wright, George S. M. Dyer, Charalampos Zalavras, Frank Walter, Scott Duncan, Grant Garrigues, Steven Rhemrev, Abhijeet L. Wahegaonkar, Jonathan Rosenfeld, Sander Spruijt, Martin Boyer, Peter Jebson, Ramon de Bedout, John T. Bolger, Thomas J. Fischer, Waldo E. Floyd III, Julie Adams, Francisco Javier Aguilar Sierra, M. Jason Palmer, Sidney Jacoby, Todd Siff, Carrie Swigart, Gregory DeSilva, John Taras, Jeffrey Wint, Jim Calandruccio, Russell Shatford, Douglas T. Hutchinson, Takashi Sasaki, Richard Uhl, Jose Nolla, Sanjeev Kakar, Ekkehard Bonatz, Rick Papandrea, Michael Nancollas, Gustavo Mantovani Ruggiero, Richard Barth, Brent Bamberger, John McAuliffe, Marc J. Richard, Michael Baskies, Scott Mitchell, Daniel Polatsch, Jorge Boretto, Lisa Lattanza, Rozental, Charles Cassidy, David M. Kalainov, Eric Hofmeister, Lawrence Weiss, Leon Benson, Taizoon Baxamusa, Doug Hanel, Fabio Suarez, Alberto Pérez Castillo, David R. Miller, Chris Wilson, Timothy G. Havenhill, Michael W. Grafe, Saul Kaplan, Catherine Spath, Christopher J. Walsh, Kendrick Lee, John Howlett, Loren Potter, Patrick T. McCulloch, Boyd Lumsden, Nick Meyer, Ben Sutker, Charles Metzger, Jose A. Ortiz Jr, Steve Helgemo, Jason Fanuele, Jay Pomerance, W. Arnnold Batson, Shep Hurwit, J. C. Goslings, John G. DeVine, Scott Edwards, Thomas Hughes, Mahmoud I. Abdel-Ghany, I. J. V. Kleinlugtenbelt.
References
- 1.Bernstein J, Adler LM, Blank JE, Dalsey RM, Williams GR, Iannotti JP. Evaluation of the Neer system of classification of proximal humeral fractures with computerized tomographic scans and plain radiographs. J Bone Joint Surg Am. 1996;78:1371–1375. doi: 10.2106/00004623-199609000-00012. [DOI] [PubMed] [Google Scholar]
- 2.Broberg MA, Morrey BF. Results of treatment of fracture-dislocations of the elbow. Clin Orthop Relat Res. 1987;216:109–119. [PubMed] [Google Scholar]
- 3.Bruinsma WE, Guitton TG, Warner JJ, Ring D. Interobserver reliability of classification and characterization of proximal humeral fractures: a comparison of two and three-dimensional CT. J Bone Joint Surg Am. 2013;95:1600–1604. doi: 10.2106/JBJS.L.00586. [DOI] [PubMed] [Google Scholar]
- 4.Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol. 1990;43:551–558. doi: 10.1016/0895-4356(90)90159-M. [DOI] [PubMed] [Google Scholar]
- 5.Cutler CW. Fractures of the head and neck of the radius. Ann Surg. 1926;83:267–278. doi: 10.1097/00000658-192602000-00015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Davidson PA, Moseley JB, Jr, Tullos HS. Radial head fracture. A potentially complex injury. Clin Orthop Relat Res. 1993;297:224–230. [PubMed] [Google Scholar]
- 7.Doornberg J, Elsner A, Kloen P, Marti RK, van Dijk CN, Ring D. Apparently isolated partial articular fractures of the radial head: prevalence and reliability of radiographically diagnosed displacement. J Shoulder Elbow Surg. 2007;16:603–608. doi: 10.1016/j.jse.2006.10.015. [DOI] [PubMed] [Google Scholar]
- 8.Duckworth AD, McQueen MM, Ring D. Fractures of the radial head. Bone Joint J. 2013;95:151–159. doi: 10.1302/0301-620X.95B2.29877. [DOI] [PubMed] [Google Scholar]
- 9.Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43:543–549. doi: 10.1016/0895-4356(90)90158-L. [DOI] [PubMed] [Google Scholar]
- 10.Hotchkiss RN. Displaced fractures of the radial head: internal fixation or excision? J Am Acad Orthop Surg. 1997;5:1–10. doi: 10.5435/00124635-199701000-00001. [DOI] [PubMed] [Google Scholar]
- 11.Itamura J, Roidis N, Mirzayan R, Vaishnav S, Learch T, Shean C. Radial head fractures: MRI evaluation of associated injuries. J Shoulder Elbow Surg. 2005;14:421–424. doi: 10.1016/j.jse.2004.11.003. [DOI] [PubMed] [Google Scholar]
- 12.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
- 13.Lozano-Calderon S, Blazar P, Zurakowski D, Lee SG, Ring D. Diagnosis of scaphoid fracture displacement with radiography and computed tomography. J Bone Joint Surg Am. 2006;88:2695–2703. doi: 10.2106/JBJS.E.01211. [DOI] [PubMed] [Google Scholar]
- 14.Mason ML. Some observations on fractures of the head of the radius with a review of one hundred cases. Br J Surg. 1954;42:123–132. doi: 10.1002/bjs.18004217203. [DOI] [PubMed] [Google Scholar]
- 15.Müller M. The Comprehensive Classification of Fractures in Long Bones. Berlin, Germany: Springer-Verlag; 1990. [Google Scholar]
- 16.Posner KL, Sampson PD, Caplan RA, Ward RJ, Cheney FW. Measuring interrater reliability among multiple raters: an example of methods for nominal data. Stat Med. 1990;9:1103–1115. doi: 10.1002/sim.4780090917. [DOI] [PubMed] [Google Scholar]
- 17.Rineer CA, Guitton TG, Ring D. Radial head fractures: loss of cortical contact is associated with concomitant fracture or dislocation. J Shoulder Elbow Surg. 2010;19:21–25. doi: 10.1016/j.jse.2009.05.015. [DOI] [PubMed] [Google Scholar]
- 18.Siegel S, Castellan N. Nonparametric Statistics for the Behavioral Sciences. New York, NY, USA: McGraw-Hill; 1988. [Google Scholar]
- 19.Stieber J, Quirno M, Cunningham M, Errico TJ, Bendo JA. The reliability of computed tomography and magnetic resonance imaging grading of lumbar facet arthropathy in total disc replacement patients. Spine (Phila Pa 1976). 2009;34:E833–840. [DOI] [PubMed]
- 20.Zou G, Donner A. Confidence interval estimation of the intraclass correlation coefficient for binary outcome data. Biometrics. 2004;60:807–811. doi: 10.1111/j.0006-341X.2004.00232.x. [DOI] [PubMed] [Google Scholar]