A journey into clinical evidence: from case reports to mixed treatment comparisons

Why bother with clinical evidence?

Medical decision making can be based on several approaches. Indeed, evidence based medicine represents just one of the several conceivable means to decide how to manage patients. For instance, eminence, experience, vehemence, eloquence, elegance, providence, diffidence, nervousness, and confidence can all guide (or misguide) clinical decisions instead of evidence [1]. Nonetheless, it now appears clear that prior clinical evidence, disseminated through peer-review publication, is the only viable approach to foster improvements in the delivery of health care [2].

Moving through the evidence based medicine hierarchy: bottom-up or top-down?

The hierarchy of evidence based medicine moves from in vitro studies, animal studies, case reports, and case series to observational clinical studies and randomized clinical trials (Figure 1) [3].

Figure 1.

Parallel hierarchy of evidence based medicine.

Accordingly, in the context of secondary research [4], progressively greater emphasis is placed on editorials, reviews, systematic reviews, study level meta-analyses, and patient level meta-analyses [3].

Practice guidelines occupy a unique seat in the feast, as their actual role largely depend on the quality of data gathering which lies behind the guideline itself, and on the specific agenda of the drafting committee [5]. A new entry in the evidence based medicine scenario is the network meta-analysis, most recently renamed mixed treatment comparison. This type of meta-analysis, always stemming from a prior systematic review, includes both direct and indirect comparison studies, borrowing appropriately from the latter to reinforce the former. It is based on a straightforward statistical concept (Figure 2) [6,7], but may also exploit sophisticated statistical methods, including Bayesian hierarchical models [8].

Figure 2.

Theoretical basis for indirect treatment comparisons. Ln=natural logarithm; OR=odds ratio; Var=variance

For instance, when trying to compare three different drugs (A, B and C), we may have to rely on three randomized trials: trial 1 - comparing A vs B, trial 2 - comparing A vs C, and trial 3 - comparing B vs C. In a typical meta-analysis of randomized clinical trials, we would exploit only trial 1. However, we may generate interaction odds ratios (OR) for A vs B using trials 2 and 3, according to the following: Ln (OR_{A vs. B}) = Ln (OR_{A vs. C}) - Ln (OR_{B vs. C}), and Var [Ln (OR_A vs.B)] = Var [Ln (OR_{A vs. C})] + Var [Ln (OR_{B vs. C})], where Ln is the natural logarithm, and Var is the variance. Such interaction OR can then be pooled with the OR from trial 1, with a typical random-effect inverse-variance weighting process [6].

Despite the well established role of randomized clinical trials and systematic reviews, case reports and series should not be considered altogether faulty or unreliable. Whenever uncommon events occur or novel insights are available, substantial information can be gained even by a handful of data, if well collected, thoroughly reported, and carefully discussed [9]. Accordingly, single-operator case series can inform on learning curve and skill acquisition [10]. Nonetheless, the major improvements in clinical medicine have left room, in most cases, only for small and subtle developments, which mandate large and simple, yet carefully conducted, randomized trials, or more complex pooling efforts such as patient level meta-analyses or mixed treatment comparisons [2].

Table 1.

Selected publication types from MEDLINE/PubMed, ordered according to their total number, with accompanying definition.

What does the future hold?

It remains difficult to predict the role and shape of evidence based medicine ten or twenty years from now. The internet revolution has dramatically changed the way information is gathered, analyzed and disseminated. Cloud computing and the universal availability of powerful handheld computer devices will probably empower most if not all clinical practitioners with sophisticated data analysis capability. Yet user friendly tools to analyze complex information and synthesize data from different types of studies and sources represent a formidable challenge, as explicitly stated in their piece on teleoanalysis by Wald and Morris [11].

The intriguing, yet possibly disturbing, feature of teleoanalysis is indeed the fact that it combines data from different types of evidence rather than from a single study design.

In the meanwhile, we remain adamant that every piece of evidence, be it a clinical vignette, a randomized trial, an editorial, or a practice guideline, should be viewed and appraised constructively, yet avoiding the illusion that it can alone guide righteously the practitioner’s hand.