Where Are We Now?
Austin and colleagues performed a retrospective analysis of a longitudinal, single-institutional primary TKA database and found that there was no association between suture type (barbed suture vs. standard interrupted suture) and 90-day all-cause reoperations. However, the study did find that barbed sutures were associated with a greater risk of arthrotomy failure. The authors conclude, appropriately, that this is a finding that needs further exploration since using barbed sutures for arthrotomy closure is widespread, and that having a larger sample size would be essential for better examination of the relationships between suture type and 90-day all-cause reoperations and arthrotomy failures.
The conclusion that further study is needed is not unique, but the current study brings up two interesting questions related to research in general, and particularly in orthopaedic surgery: (1) How can we reconcile the desire for good evidence with the logistical realities of doing clinical research? (2) How can we best interpret statistical results, including power, sample size, confidence intervals, and effect size when we are dealing with rare events like complications after surgery?
In orthopaedic surgery, researchers often face logistical and ethical issues while trying to perform randomized controlled clinical trials [14], and it’s the lack of these trials that is often cited as the fundamental flaw in using existing data to establish practice guidelines. In fact, there is still debate over how to reconcile sometimes conflicting (and often overlapping) evidence [13, 17]. Observational studies, especially retrospective ones like the current study, often remain the only options we have to make decisions about the effectiveness of interventions.
Where Do We Need To Go?
Additionally, researchers can struggle to gather a large enough sample to achieve enough power to detect differences between suture types. The event of interest in this study—reoperations and other complications after primary TKA—is rare. Currently, less than 2% of patients experience complications of any kind after total knee replacements [1, 2]. Therefore, we must turn to two larger concerns: (1) How best to handle the need for large samples and (2) how to effectively use advanced statistical methods to best model differences in outcomes between groups. Pooling data across studies either requires cooperation of different research sites working on the same project (with the attendant problems of securing contracts, data, and funding) or advanced statistical methods.
While larger studies with higher levels of evidence are needed, determining the most-effective treatments for patients is not going to be easy, and will require other approaches. We have many large administrative databases available, such as the National Inpatient Sample and the Medicare data files, but research using this information has limitations as well, including issues with data quality and inclusivity, as well as the challenges associated with doing longitudinal analysis because of confidentiality concerns [16]. Increasing the availability of data gleaned from the clinical record, along the lines of the National Surgical Quality Improvement Program database, would go a long way in improving the applicability and quality of database research.
How Do We Get There?
I believe that advanced statistical approaches offer great promise to resolve these issues. It seems clear enough that clinical research puts too much emphasis on the p value [9, 10, 18], when what really matters to clinicians is the when what matters to clinicians is the effect size or minimum clinically important difference [12]. What started with Neyman and Pearson and the “simple” p value in the 1920s [4, 15] has evolved to ever more complex methods for modeling research outcomes. For example, meta-analysis as a statistical approach has long been employed to combine results from multiple studies, but new methods, such as meta-regression, systematic applications of fixed-effect versus random-effects modeling, and careful examination of various methods for pooling rare events have been shown to decrease the impact of potential bias in results from meta-analyses [3, 7]. Other studies have shown how adding correction factors to binary logistic regression models can lead to more reliable parameter estimation [11].
The launch of the Comparative Effectiveness of Pulmonary Embolism Prevention after Hip and Knee Replacement (PEPPER) trial shows that it is, in fact, possible to generate large sample sizes and thereby study rare events [5, 6]. This 5-year randomized trial will recruit up to 25,000 patients at 25 sites to compare the effectiveness of different prophylaxis regimens (aspirin, warfarin, and rivaroxaban). Outcomes of interest include not only the rates of pulmonary embolisms and deep vein thrombosis events, but also other clinically important outcomes related to the use of chemoprophylactic agents such as bleeding.
We need to continue increasing the number of Level I and II studies published in orthopaedic surgery journals [8], and a trial of the size of the PEPPER study is a step in the right direction. Just as importantly, we must look to our statistician colleagues for assistance in interpreting study results and incorporating newer methods in all orthopaedic surgery research. Our attention needs to turn away from focusing solely on what is statistically significant to truly understanding the clinical importance (or lack thereof) of our research.
Footnotes
This CORR Insights® is a commentary on the article “Are Barbed Sutures Associated With 90-day Reoperation Rates After Primary TKA?” by Austin and colleagues available at: DOI: 10.1007/s11999-017-5474-7.
The author certifies that neither she, nor any members of her immediate family, have any commercial associations (such as consultancies, stock ownership, equity interest, patent/licensing arrangements, etc.) that might pose a conflict of interest in connection with the submitted article.
All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research ® editors and board members are on file with the publication and can be viewed on request.
The opinions expressed are those of the writers, and do not reflect the opinion or policy of CORR ® or The Association of Bone and Joint Surgeons®.
This CORR Insights® comment refers to the article available at DOI: 10.1007/s11999-017-5474-7.
This comment refers to the article available at: https://doi.org/10.1007/s11999-017-5474-7.
References
- 1.American Academy of Orthopaedic Surgeons. Total knee replacement. Available at: http://orthoinfo.aaos.org/topic.cfm?topic=a00389. Accessed August 28, 2017.
- 2.American Association of Orthopaedic Surgeons. Preventing venous thromboembolic disease in patients undergoing elective hip and knee arthroplasty. Evidence based guidelines and evidence report. Available at: http://www.aaos.org/research/guidelines/VTE/VTE_full_guideline.pdf. Accessed September 6, 2017.
- 3.Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A. Much ado about nothing: A comparison of the performance of meta-analytical methods with rare events. Statist Med. 2007;26:53–77. doi: 10.1002/sim.2528. [DOI] [PubMed] [Google Scholar]
- 4.Cohen J. A power primer. Psychol Bull. 1992;112:155–159. doi: 10.1037/0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
- 5.Comparative Effectiveness of Pulmonary Embolism Prevention After Hip and Knee Replacement (PEPPER). Available at: https://clinicaltrials.gov/ct2/show/NCT02810704. Accessed August 28, 2017. [DOI] [PMC free article] [PubMed]
- 6.Comparative Effectiveness of Pulmonary Embolism Prevention After Hip and Knee Replacement (PEPPER): Balancing Safety and Effectiveness. Available at: https://www.pcori.org/research-results/2015/comparative-effectiveness-pulmonary-embolism-prevention-after-hip-and-knee). Accessed August 28, 2017.
- 7.Cumming G. The new statistics. Psychol Sci. 2013;25:7–29. doi: 10.1177/0956797613504966. [DOI] [PubMed] [Google Scholar]
- 8.Cunningham BP, Harmsen S, Kweon C, Patterson J, Waldrop R, McLaren A, McLemore R. Have levels of evidence improved the quality of orthopaedic research? Clin Orthop Relat Res. 2013;471:3679–3686. doi: 10.1007/s11999-013-3159-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Freiman JA, Chalmers TC, Smith H, Jr, Kuebler RR. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 “negative” trials. N Engl J Med. 1978;299:690–694. doi: 10.1056/NEJM197809282991304. [DOI] [PubMed] [Google Scholar]
- 10.Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. Eur J Epidemiol. 2016;31:337–350. doi: 10.1007/s10654-016-0149-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.King G, Zeng L. Logistic regression in rare events data. Polit Anal. 2001;9:137–163. doi: 10.1093/oxfordjournals.pan.a004868. [DOI] [Google Scholar]
- 12.Leopold SS, Porcher R. Editorial: The minimum clinically important difference—The least we can do. Clin Orthop Relat Res. 2017;475:929–932. doi: 10.1007/s11999-017-5253-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Moher D. The problem of duplicate systematic reviews. BMJ. 2013;347:f5040. doi: 10.1136/bmj.f5040. [DOI] [PubMed] [Google Scholar]
- 14.Mundi R, Chaudhry H, Mundi S, Godin K, Bhandari M. Design and execution of clinical trials in orthopaedic surgery. Bone Joint Res. 2014;3:161–168. doi: 10.1302/2046-3758.35.2000280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Neyman J, Pearson ES. On the use and interpretation of certain test criteria for purposes of statistical inference: Part I. Biometrika. 1928;20A:175–240. [Google Scholar]
- 16.Patel AA, Singh K, Nunley RM, Minhas SV. Administrative databases in orthopaedic research: pearls and pitfalls of big data. J Am Acad Orthop Surg. 2016;24:172–179. doi: 10.5435/JAAOS-D-13-00009. [DOI] [PubMed] [Google Scholar]
- 17.Siontis KC, Hernandez-Boussard T, Ioannidis JP. Overlapping meta-analyses on the same topic: survey of published studies. BMJ. 2013;347:f4501. doi: 10.1136/bmj.f4501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Wasserstein RL, Lazar NA. The ASA’s statement on p-values: Context, process, and purpose. Am Stat. 2016;70:129–133. doi: 10.1080/00031305.2016.1154108. [DOI] [Google Scholar]
