CORR Insights™: Is Limited Incision Better Than Standard Total Hip Arthroplasty? A Meta-Analysis

Abstract

This CORR Insights™ is a commentary on the article ‘‘Is Limited Incision Better Than Standard Total Hip Arthroplasty? A Meta-Analysis ” by Joseph T. Moskal MD and Susan G. Capps PhD available at DOI: 10.1007/s11999-012-2717-5.

Usually, the difficulty of using meta-analysis to inform outcome-based decision-making is confined to the quality of the data. There are different levels of evidence; mixing Levels I and III does not average out to Level II. Retrospective studies are always shadowed by numerous kinds of bias, including the possibility of inconsistent reporting and different standards of reporting across the different institutions whose data are included in the meta-analysis. Some studies may be excluded by language barriers. Moskal and Capps face all the usual questions, and then they make the task even harder by considering a range of outcomes.

They meet this greater challenge through careful attention to detail. They recognized that the articles had to be screened for multiple uses of the same population, so no dataset would bias the results by being counted twice. For three articles, they recognized that the data could be broken into two substudies, and they incorporated each study independently. They considered at least four references per variable, which I rather liked. They used a random-effects model to ensure that the results were as general as possible; this approach recognizes that the true effect size may vary among institutions. Even so, testing for heterogeneity would have been helpful; had there been little heterogeneity, they could have surmised that there was not much difference in effect size across institutions, and they then could have used a less-conservative fixed-effects model.

Why not use the more conservative test every time? The trade-off is that an unnecessarily conservative test will reject findings that may in fact be meaningful because they might not turn up as statistically significant. Small sample sizes and confounding variables are common in this kind of research; we do not want to trade away our ability to detect differences unnecessarily.

Another issue is publication bias, which is the idea that negative studies are less likely to be submitted or accepted for publication. This phenomenon, to the degree it exists, affects meta-analysis by systematically increasing apparent effect sizes; if “positive” studies are more likely to be published, as may be the case for new approaches and new technologies, and meta-analyses survey mainly the published literature, we are left with a more optimistic estimation of the novel treatment’s effect than we should be. There are statistical approaches that allow us to estimate whether publication bias likely affects the literature being surveyed; Moskal and Capps present no funnel plots or any other explicit tests to determine this. It would have been nice if they had, as there are techniques available to compensate for publication bias (eg, Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1: 247–262) by imputing the missing results.

Fortunately, publication bias is not likely to have been major issue in this article because Moskal and Capps correctly recognize that statistical significance and clinical impact are two separate things. Estimated blood loss is 73 mL, or a sixth of a pint. One day in hospital may or may not be meaningful. Differences in pain and Harris hip scores also were modest, although they are large enough to have some clinical impact.

For all the effort and care that can go into collecting, screening, and analyzing the data, we always have some uncertainty in the final answer. Meta-analyses can polish the answer but cannot remove the uncertainty. Moskal and Capps end their paper on a point worth repeating: evidence-based practice “grows through implementation of more well-designed, prospective comparative studies (RCTs or not) and through continued quantitative analysis of the body of knowledge as it matures.”