CORR Insights®: What Are the Interobserver and Intraobserver Variability of Gap and Stepoff Measurements in Acetabular Fractures?

Where Are We Now?

Gap and stepoff have commonly been used in orthopaedic fracture assessment since Knirk and Jupiter [7] presented their work on distal radius fractures in young adults in 1986. They described a significantly increased (91%) risk of post-traumatic arthritis in incongruent joints at approximately 7 years in patients in whom a step of more than 2 millimeters was identified on the distal radius surface on plain radiographs. Since then, this 2-millimeter cutoff has been used in daily practice for articular fracture fixation, as well as in other smaller and larger joints. Unfortunately, this is often done without support by further research specific to the joint concerned. House and Jupiter [5] revisited the classic paper [7] in 2009 and self-critiqued the lack of interobserver agreement, among other methodologic flaws.

Stepoff and gap are commonly described as incongruence in the articular surface, with a step in alignment or a gap between fragments. Verbeek et al. [19] specifically researched acetabular fractures and defined acceptable postoperative reduction as a stepoff less than 1 mm and a gap less than 5 mm on CT. Identifying such clear cutoffs assumes that these measures are both reliable and reproducible before they are accurately correlated to clinical outcomes. Questioning the interobserver and intraobserver agreement in the way Meesters et al. [13] do in the current article in Clinical Orthopaedics and Related Research® sheds a different light not only on Verbeek’s study [19], but also on other studies in which hard gap and stepoff measurements were related to outcomes after fracture care [10, 11].

There is no doubt that modern techniques such as three-dimensional printing (3-D) printing have improved our understanding and treatment of acetabular fractures [3, 16]. Despite this, we need to know that our methods of assessing fractures (and reductions of fractures) possess excellent interobserver and intraobserver agreement. An earlier study focusing on the presence rather than the actual size of stepoff and gap in acetabular fractures demonstrated excellent interobserver agreement [2], but another study including measurements reported only moderate intraclass correlation coefficients [18]. Given the well-established relationship between inadequate articular acetabular reduction and the clinical outcome of acetabular fractures, there is a lot at stake [19]. Inadequate or unreliable measurements of articular reduction might not only compromise patient care, but also impede clinical research.

Meesters et al. [13] performed a diagnostic imaging study to determine the interobserver and intraobserver variability in gap and stepoff measurements in acetabular fractures. Five observers (two experienced and three less-experienced surgeons) measured preoperative and postoperative radiographs and CT images as well as the intraoperative fluoroscopy images of 60 patients. They found there is insufficient interobserver and intraobserver agreement about the actual size of gap and stepoff measurements in acetabular fractures. Therefore, the usefulness of these measurements to support clinical decision-making in acetabular fracture surgery is, at best, questionable. This study clearly demonstrates a critical shortcoming in how we assess and treat patients with acetabular fractures.

Where Do We Need To Go?

Although Meesters et al. [13] clearly state that 2‐D measuring techniques are consistent with what is currently used in daily practice worldwide, it does bear the question: rather than using the same techniques that have been used for decades, perhaps we could come up with something better? Because of the shortcomings of 2‐D techniques for measuring complex 3‐D structures, there is a demand for measurement techniques that will consider the 3‐D shape and potential 3‐D displacement of fractures. In acetabular and other fractures involving joint surfaces, it is important to identify the acceptable limits of fracture reduction; in order to achieve this, we will need to be able measure these limits accurately.

In a previous study, Meesters et al. [12] introduced a 3‐D CT method for measuring acetabular fractures, demonstrating that 2‐D CT measurements underestimate the initial and residual displacement in complex acetabular fractures compared with 3‐D CT measurements. The total gap area was identified as a valuable measurement with an excellent intraclass correlation coefficient. However, I do not know of subsequent studies linking these 3‐D CT measurements to either the surgical indication or clinical outcome. In other areas, quantitative 3‐D CT measurements have been advocated to determine the morphology of odontoid fractures [17] and analyze tibia plateau fractures [1]. Again, these studies did not correlate quantitative 3‐D CT measurements with clinical outcomes.

To improve the reliability of gap and stepoff measurements in acetabular fractures, we could learn from the work done in ankle fractures, especially reduction of posterior malleolar fractures [4]. De Muinck Keizer et al. [4] evaluated the reduction of fracture fragments using quantitative 3-D CT, which proved to be a promising and innovative radiographic technique. In addition to the classic stepoff and gap measurement, 3‐D displacement and the total gap surface were identified as valuable measurements. The interrater reliability of these new quantitative 3‐D CT measurements was good to excellent, with intraclass correlation coefficients of 0.64 and 0.92 for 3‐D displacement and the total gap surface, respectively. In a subsequent study by Meijer et al. [14], at 1 year after injury, a larger stepoff identified on quantitative 3‐D CT correlated with higher levels of pain and poorer symptoms, as reflected by the foot and ankle outcome scores of patients with rotational ankle fractures.

How Do We Get There?

Soon, studies could seek to replicate the current study [13] using quantitative 3‐D CT techniques. This should establish the interobserver and intraobserver variability among surgeons with different experience in measuring the gap, stepoff, 3‐D displacement, and gap area, such that the reliability of this technique in modern daily practice can be ascertained.

Subsequently, these measurements could be used in a clinical study aiming to relate quantitative 3‐D CT measurements to clinical outcomes such as patient-reported outcome measures, osteoarthritis, and conversion to hip arthroplasty. This might lead to new, qualitative, 3‐D CT-based, validated standards for the acceptable limits and tolerance, if any, of fracture fragment reduction in acetabular surgery. This study should ideally be conducted by an international data-sharing collaborative group. This collective of researchers would share their data, sometimes from historical trials, to combine their work into much larger datasets, which are typically required to allow for machine-learning tools to be developed. An example of such a successful collaborative is the Machine Learning Consortium [6]

I envision that in the future, deep-learning algorithms enabling computer vision software to identify, for example, fractures on images (that is, image recognition) could be used. These algorithms could be further developed to a level where fracture segmentation on CT might occur automatically. The first steps on the pathway of image recognition in orthopaedic surgery have demonstrated promising results in wrist, ankle, and scaphoid fractures [8, 15]. Currently, image recognition using systems driven by artificial intelligence are, at best, on par with humans’ abilities [8, 15], although it seems only a matter of time before automated image recognition becomes superior to humans. I have no doubt that the gap, stepoff, 3‐D displacement, and gap area will also be quantified automatically in the future as an extension of image recognition. This will remove the human variability of measurements and thus improve reproducibility [9]. Finally, machine-learning tools should be developed and combined with image recognition to create a system that, with the use of further artificial intelligence, will provide acetabular surgeons globally with the tools needed to make individualized clinical decisions.