Orthopedic registry research — limitations and future perspectives

In 2015, there is nothing spectacular about registers. To collect information on the outcome of treatments is as old as medicine itself. Registries merely took the concept several steps further with (often) nationwide, prospective collection of outcome data combined with rigorous follow-up. This process requires some sort of reliable patient identification system, and due to their use of personal identification numbers, the Nordic countries became the first to implement registries. Orthopedic surgeons were pioneers in getting nationwide registries up and running: the Swedish Knee Arthroplasty Register (SKAR, www.knee.se), started in 1975, became a prototype that inspired orthopedic surgeons around the world, and the Swedish Hip Arthroplasty Register (SHAR, www.shpr.se) followed in 1979. The findings generated by these and other registries improved results after arthroplasty surgery in an evolutionary process that eliminated “bad” implants and inferior techniques. Over the years, orthopedic registry-based research has therefore become an integral part of the scientific literature. This being said, recent examples illustrate that the orthopedic community has not always been quick enough at picking up inferior concepts; take, for instance, the history of the articular surface replacement (ASR) device, which was withdrawn much too late.

Limitations to registry-based research

Observational studies must fulfill a number of critical methodological prerequisites. External validity of registry studies depends on a high degree of completeness and coverage. These numbers have to be supplied in each registry-derived publication, and the quality of the registration process must be continuously validated. But, even if such basic requirements are fulfilled, registry-based research can be criticized and questioned due to several methodological limitations:

• By mathematical necessity, stunningly large numbers of patients and procedures can give rise to correspondingly small p-values and narrow confidence intervals related to a given set of hypotheses. But statistically significant findings may not always be clinically relevant. One can be tempted to genuflect at Fisher’s 0.05-altar, but does, for instance, a minimal change in patient-reported outcome (with a p-value of < 0.001) always make a clinically relevant difference? We must always interpret registry-based results within a framework of clinical relevance, irrespective of whether we analyze implant survival, the incidence of a specific complication, or subjective outcome measures.

• An ongoing debate about the interpretation of mortality data derived from the National Joint Registry of England, Wales and Northern Ireland (www.njrcentre.org.uk) highlights the problem of selection bias. Patient selection to uncemented or resurfacing hip arthroplasty is influenced by age and gender, and also by general health and a number of other confounders not accounted for in the analysis. The calculation of life expectancy in this material leads to the prediction that about a third of all patients receiving a certain type of hip resurfacing will become centenarians, illustrating the impact of selection bias that cannot be adequately adjusted for (Kandala et al. 2014). A failure to understand this phenomenon can lead to erroneous interpretations.

• Residual confounding is a related and under-recognized problem in registry-based research. This term describes the amount of variation not explained by variables included in a regression analysis, and this residual confounding can be unsettlingly large: The remaining unexplained variation can amount to more than 80%, or, to put it differently, even the nicest Cox regression model based on routine registry data can leave us clueless when it comes to predicting outcomes. The inclusion of additional confounders such as comorbidities or socioeconomic factors can reduce the amount of residual confounding and improve the predictive accuracy of a specific model (Whitehouse et al. 2014), but we have to realize that the pseudo-R² values of regression models in our field are poor and that the remaining uncertainty is large.

Based on these and many other limitations, it appears to be mandatory that scientists, scientific journals, and international associations devoted to registry-based research should define how registry research should be presented, building on existing guidelines and protocols (von Elm et al. 2007, Ranstam et al. 2011). Ideally, this process could result in a structured reporting protocol that minimizes methodological flaws and creates greater transparency in the description of registry-based observational studies.

Future perspectives

But is, then, the answer to the above-described limitations of observational studies to collect more and more data? Having started off as quite simple datasets that included some baseline demographic information, implant types, and outcomes in terms of revisions, registries are now moving towards more extensive data collection. This data collection on an industrial scale, sometimes referred to as “Big Data”, generates an immense workload. Smart IT solutions such as linking hospital records to registry databases and a more sensible distribution of the tasks associated with data collection and imputation must prevent orthopedic surgeons and their staff from spending more time at the keyboard than with their patients. It has been rightly stated that registries have reduced the revision burden in arthroplasty surgery, but much remains to be done in order to reduce the “administrative burden” associated with Big Data collection.

Observational studies are usually referred to as hypothesis-generating, preparing the ground for randomized controlled studies that represent a higher level of evidence. On the other hand, it is obvious that relatively small differences in outcomes require unrealistically high numbers of patients to be included in randomized controlled trials (Altman 1982). The term “cluster randomization” describes a procedure whereby clusters of individuals rather than single individuals are randomized to different treatment groups, often within a nationwide registry. This approach combines the methodological superiority of randomization with the large numbers of patients required to investigate clinically relevant issues where differences between outcomes are small but relevant. Cardiologists have demonstrated that cluster randomization of different coronary care units to 1 of 2 different treatment strategies combined with the follow-up of many thousands of patients enrolled within a registry setting gives answers to questions that cannot be addressed only within a registry setting or only in a classical RCT (Flather et al. 2011). Perhaps our large orthopedic registries should join forces with the legislative bodies and the implant industry in order to introduce new implants or techniques in such cluster-randomized registry studies.

The name of Göran Bauer, the former Editor-in-Chief of Acta Orthopaedica and initiator of the Swedish Knee Arthroplasty Register, is closely connected with the tradition of publishing registry-based research in Acta Orthopaedica. Our journal holds a record of having published the highest number of registry-based articles in the field of orthopedics. The issue that is presently in your hands is filled to the brim with high-quality orthopedic research, including 9 articles based on registry data. Among these is an article by Jameson and co-authors that won the Acta Award at the Meeting of the International Society of Arthroplasty Registers (ISAR) in Stratford-upon-Avon. In addition, 2 guest editorials in this issue cover different aspects of the introduction and monitoring process of novel implants, showing how registries could help to prevent future disasters.

I will refrain from citing all the articles that appear in the present issue in this editorial, which might be interpreted as being a rather cheap trick to enhance the impact factor. We at Acta Orthopaedica merely want our readers to have interesting and thought-provoking reading, and we are convinced that—despite all the obvious limitations of registry-based research—such studies will continue to play an important role in the advancement of orthopedic science. We must, however, always make sure that the orthopedic registries, established and developed by orthopedic surgeons, are not handed over to the sometimes shortsighted interests of different stakeholders, including manufacturers and politicians.