Reliability of Diagnosis of Partial Union of Scaphoid Waist Fractures on Computed Tomography

Abstract

Computed tomography (CT) is increasingly used not just to diagnose union but also to estimate the percentage of the fracture gap that is bridged by healing bone. This study tested the primary null hypothesis that there is no agreement between observers on the extent of union of a scaphoid waist fracture on CT. CT scans of 13 nondisplaced scaphoid waist fractures treated nonoperatively were rated by 145 observers. CT scans were done 10 to 12 weeks after injury. Observers were asked to “eyeball” measure percentage of union. We found that there was a moderate agreement on the categorical degree of partial union of a scaphoid waist fracture on CT (k = 0.34). Agreement on the location of bony bridging was slight (k = 0.31). We concluded that there is limited reliability of diagnosis of partial union of a scaphoid waist fracture on CT and that this should be taken into account in both patient care and research. This is a Level III, diagnostic study.

Introduction

Scaphoid waist fractures can result in nonunion. Radiographs are often used to diagnose union, and some surgeons may prolong cast immobilization based on radiographic findings. Dias demonstrated that radiographs are not reliable for diagnosis of union. 1

Computed tomography (CT) is increasingly used not just to diagnose union but also to estimate the percentage of the fracture gap that is bridged by healing bone. Singh et al 2 categorized partial union of a fracture of the scaphoid waist into percentage union ranges as follows: 0 to 24%, 25 to 49%, 50 to 74%, and 75 to 99% union. Prior research suggested that CT can be used to identify certain features related to nonunion or prolonged union time of a scaphoid fracture. 3 Moreover, the concept of percent union on CT scans is often used in studies of scaphoid waist fractures.

This study tested the primary null hypothesis that there is no agreement between observers on the extent of union of a scaphoid waist fracture on CT.

Methods

After approval by our institutional review board, members of the Science of Variation Group (SOVG) with an interest in hand surgery or fractures were invited to participate in this study. Among the 150 surgeons who felt the study was appropriate for their expertise and interests, 145 completed the questionnaire.

CT scans of patients with scaphoid waist fractures made 10 to 12 weeks after injury (per the routine practice of one surgeon—CSM) were obtained by using billing codes via a research fracture database. Relevant patients were than manually identified and assessed in the electronic medical record system, at our institution. Inclusion criterion included patients aged 18 years or older treated nonoperatively for a nondisplaced fracture of the scaphoid waist. All scaphoid CT scans had a slice thickness of 1.25 mm or less. All scans were made in the plane of the scaphoid. We chose waist fractures, as they are the most common type of scaphoid fracture treated by surgeons involved in fracture care. In addition, the SOVG consists of members from various subspecialties. Choosing the waist fracture allowed us to include a larger cohort of respondents who were familiar with interpretation of a scaphoid CT scan.

Participants were shown windowed videos of the 13 CT scans in coronal and sagittal fashion and were asked to diagnose the percentage bony bridging and the location of the bony bridging and to indicate the doubt of their answer. The percentage of bony bridging was categorized according to Singh et al 2 (0%, 1–24%, 25–49%, 50–74%, 75–99%, and 100%). Participants were informed of this measurement technique and were asked to “eyeball” their measurement percentages. The location of the bony bridging was divided into anatomic areas, which included dorsal, volar, radial, and ulnar. All scans were viewed on the observers’ monitors, and there was no restriction of time allowed to view each video. Videos could be viewed more than once. We also recorded each observer’s sex, location of practice, years of practice, supervision of trainees, and specialization.

Observer Characteristics

The vast majority of the 145 participants were men ( n = 139, 96%). Thirty-eight percent specialized in hand and wrist surgery ( n = 56) and 35% in traumatology ( n = 51) ( Table 1 ). Five participants did not complete the survey and were excluded from further analysis.

Table 1. Characteristics of observers ( n = 145) .

a One observer did not fill out years of practice, supervising trainees, and his specialization.
Sex
Male	139
Female	6
Location of practice
United States/Canada	75
Europe	57
Asia	3
Australia	6
Other	10
Years in practice a
0–5	55
6–10	30
11–20	42
21–30	17
Supervising trainees a
Yes	126
No	18
Specialization a
Hand-wrist	56
Traumatology	51
Shoulder/elbow	16
General orthopedics	9
Other	12

Statistical Analysis

The multirater kappa measure described by Siegel 4 was used to measure interobserver agreement. Using the guidelines of Landis and Koch, 5 the generated kappa values were interpreted as follows: 0.01 to 0.20 defines slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; 0.81 to 0.99, almost perfect agreement; and 1.00, perfect agreement. Zero indicates no agreement beyond chance alone, whereas −1.00 indicates total disagreement.

Results

The mean amount of scaphoid fractures treated by the observers on a yearly basis was 2.7. All the observers treated at least one scaphoid fracture per year and three surgeons treated five scaphoid fractures per year. There was fair agreement on the categorical degree of partial union of a scaphoid waist fracture on CT (k = 0.34). Agreement on the location of bony bridging was also fair (k = 0.31) ( Table 2 ). The average doubt of the answers of the observers was 3.8 on a scale from 0 to 10 (0 = no doubt and 10 = really in doubt).

Table 2. Interobserver agreement ( n = 145) .

	Bony bridging		Location bony bridging
	Kappa	Agreement	Kappa	Agreement
Overall	0.34	Fair	0.31	Fair
Area
United States	0.35	Fair	0.31	Fair
Europe	0.33	Fair	0.33	Fair
Other	0.36	Fair	0.32	Fair
Years in independent practice
0–5 y	0.34	Fair	0.30	Fair
> 5 y	0.34	Fair	0.32	Fair

Discussion

Measuring percentage of union on a CT scan could be helpful in studying and treating scaphoid waist fractures. To be useful, this diagnostic test must be sufficiently reliable and accurate. This study measured the reliability of diagnosis of percent union on CT of healing scaphoid waist fractures.

This study has several limitations. First, all cases were retrieved from a single surgeon’s practice that might have caused a selection bias, although this is unlikely because we used a consecutive series of nonoperatively treated nondisplaced scaphoid waist fractures. Second, most SOVG participants work in an academic setting (82% supervise trainees), and their values, training, and practice might differ from the larger community of surgeons. Third, images were presented in a standardized fashion, which did not allow observers to adjust and manipulate images as would be the case in their daily practice. However, a study found that the format of imaging visualization has limited influence on interpretation. 6 We asked the observers to “eyeball” measure the percentage and location of bony bridging, without giving them further instructions. The purpose was to make the study resemble actual clinical situations, across a spectrum of surgeons’ practice settings. Though this might have caused some uncertainty in the answers, it shows the wide variability in interpretation of CT scans in scaphoid fractures.

Surgeon reliability of diagnosis of partial union is fair. This is consistent with the evidence that diagnosis of scaphoid fracture characteristics on radiologic images has limitations. For instance, in one study two observers rated 151 scaphoid waist fractures on radiographs and measured fair to moderate agreement on the Herbert, Russe, and Compson classifications, as well as diagnosis of fracture comminution, displacement, and the fracture location. 7 In the past decade, CT scans have become more popular in the management of scaphoid fractures. In one study, 66 patients had a CT scan of the scaphoid 12 to 18 weeks after an acute scaphoid fracture treated in a below-elbow cast for 8 to 12 weeks; 14 fractures did not show any evidence of union, 30 had union across the entire fracture, and 22 were partially united. A study of CT diagnosis of union by three observers of 48 patients with a scaphoid waist fracture found moderate interobserver agreement (k = 0.58). 8 A study of the reliability and accuracy of diagnosis of union on CT scans among 59 raters found substantial reliability, but a negative predictive value of 0.41 (0–0.84) and a positive predictive value of 0.99 (0.97–1), indicating that CT scans are reliable for diagnosis of union but inadequate for ruling out nonunion of scaphoid waist fractures between 6 and 10 weeks after injury. 9

Agreement on the location of a healing bridge was slight. This is likely due to adding the unreliability of diagnosing union to the unreliability of determining the location of healing.

We found raters to be doubtful of their responses to some extent. It is quite possible that this could be due to low confidence in diagnosing the location of bony bridging and in the ability to quantify the percentage of bony bridging. We can speculate that we might have found a better confidence among the raters if we had simply asked if the raters thought whether there was or there was no bony bridging. Furthermore, our speculation would extend to a better confidence among raters if we had asked whether bony bridging was more or less than 50% (two categories).

We think that quantifying degree of bony bridging may have an impact on treatment. If there is only 0 to 24% bony bridging after 6 to 10 weeks of nonoperative treatment, physicians might consider surgery as a means of ensuring perhaps more rapid fracture union. However, we urge caution in using only radiographic images as a means of guiding treatment in the presence of a partially united fracture. Clinical fracture tenderness or lack thereof may also play a role in deciding ongoing treatment. The influence of a partial CT scan union on clinical fracture tenderness or lack thereof (and vice versa) has not been studied to the best of our knowledge. Therefore, our thought process outlined above may very well be considered speculative. That being said, all the fractures included in this study went on to uneventful union, a fact that may add some credibility to our speculation.

Diagnosis of partial union may not be reliable enough for use in research, as suggested by contemporary data. Future research might further test reliability of this imaging modality. The accuracy of diagnosis of partial union might not be possible to study due to the lack of a reference standard as to what constitutes union as well as the location of union. An alternative approach would be to consider serial CT scans done 12 weeks apart, and analyzed by the same set of independent observers, which would allow quantification of ongoing union, as applicable in clinical care settings, especially when combined with a consistently applied clinical examination by the same set of observers. This approach may very well allow us to develop guidelines as to when a scaphoid fracture can truly be considered to be clinically and radiologically united.

Authors of the Science of Variation Group

John M. Stephenson; Bernard F. Hearon; Marinis Pirpiris; Brian PD Wills; John T. Capo; Gregory DeSilva; SH van Helden; Dr. Minoo Patel; Asif Ilyas; Hans Goost; Thomas J. Fischer; Jonathan L. Hobby; Prosper Benhaim; Seth Dodds; Maarten W. G. A. Bronkhorst; Raymond Malcolm Smith; Nicholas L. Shortt; Frank L. Walter; Thomas Mittlmeier; James S. Huntley; David Ruchelsman; Krzysztof A. Tomaszewski; Charalampos Zalavras; Robin N. Kamal; Reid W. Draeger; John Itamura; Max Talbot; Lawrence Weiss; Ian Harris; Milind Merchant; Edward F. Ibrahim; Hervey L. Kimball; M. Jason Palmer; Christian Deml; Jeremy Hall; Joseph M. Conflitti; Andrew L. Terrono; Jan Biert; Kabir; Kyle Jeray; Greg Merrell; Diederik O. F. Verbeek; Jason C. Fanuele; Edward K. Rodriguez; Theodoros Tosounidis; L. M. S. J. Poelhekke; Antonio Barquet; Kendrick Lee; John M. Erickson; Babis; Eric Hofmeister; Ante Prkic; Andrew J. R. Oppy; Daniel Polatsch; Craig Rodner; Cyrus Klostermann; John McAuliffe; G. A. Kraan; Matej Kastelec; J. Sandoval; Peter Schandelmaier; Randy Hauck; Tomo Havliček; Michel P. J. van den Bekerom; Aakash Chauhan; Paul A. Martineau; Michael W. Grafe; Scott F. Duncan; Constanza L. Moreno-Serrano; Lewis B. Lane; Stephen A. Kennedy; Peter Jebson; Quell; John Munyak; Richard S. Gilbert; Leon Elmans; Lars E. Adolfsson; H. Brent Bamberger D. O.; Thomas DeCoster; Nikolaos K. Kanakaris; John Taras; Betsy M. Nolan; P. V. van Eerten; L. C. Bainbridge; van der Plaat; Ippokratis Pountos; Peter Hahn; Carl Ekholm; Marcos Sanmartin-Fernandez; Ladislav Mica; Prashanth Inna; Aita M. A.; F. Thomas D. Kaplan; Juan Miguel Rodríguez Roiz; Emilie Sandman; Jesus Moreta; Todd Bafus; Vochteloo A. J. H.; Ziogas Kleanthis; Manit Arora; Camilo Jose Romero Barreto; Anze Kristan; Marc Swiontkowski; Matthias Knobe; Franz Josef Seibert; Sergio Rowinski; Robert D. Zura; Leonid Katolik; Rob Parisien; Frank W. Bloemers; George S. M. Dyer; T. Schepers; Niels W. L. Schep; Iain McGraw; Gereon Schiffer; Sebastian Farr; Minos Tyllianakis; Vincenzo Giordano; Valentin Neuhaus; Nelson Elias; Martin Richardson; Apard T.; Mario Di Micoli; E. T. Walbeehm; DFP van Deurzen; Marco Rizzo; Taizoon Baxamusa; Richard Buckley; Adam B. Shafritz; Ezequiel E. Zaidenberg; Michael Nancollas; Juan M Patiño; Timothy Omara; Ramon de Bedout; Eric T. Tolo; Desirae M. McKee; Henry Broekhuyse; Jorge G. Boretto; Alexandre Leme Godoy-Santos; Tim Chesser; Carlos Henrique Fernandes; Anna N. Miller; Peter J. Evans; Georgios Panagopoulos; Mr Andrew John Powell; Efstathios G. Ballas; Charles A. Goldfarb; Scott G Kaar; Amy Wasterlain.

Ethical Approval

This study was approved by Ethics in Clinical Practice Committee, Massachusetts General Hospital, Boston, United States.

Funding Statement

Funding None.