Included Full Reports
|
Nikolakopoulouet al (2014) [9]
|
Develop an understandingof characteristics of andproblems encountered inpast network meta-analyses. |
Reports published prior to theend of 2012 that included anyform of indirect comparison;networks had to include aminimum of 4 treatments and adescription of data analysis. |
186 |
Outcome, number of included studies, synthesismethod (e.g. Bucher method, hierarchical model,meta-regression), approach to evaluation of consistency, control treatment, number of competing interventions, network shape (e.g. star shape, closed loops), category of comparison (pharmacologic versus placebo, etc), outcome measure (e.g. odds ratio, risk ratio, etc) |
While NMA validity is highly reliant on assumptions, the reporting and evaluation of these assumptions are commonly not addressed by authors |
Bafeta et al[18] (2013) |
To examine whether NMAsfollow the key methodologicalrecommendations for reportingand conduct of systematicreviews. |
All NMAs from 2003-July 2012that compared clinical efficacyof three or more interventions basedon randomised controlled trials,excluding meta-analyses with anopen loop network of threeinterventions. |
121 |
Reporting of general characteristics and keymethodological components of the systematic review process (e.g. PRISMA and AMSTAR) using twocomposite outcomes. For some components, ifreporting was adequate, the authors assessed theirconduct quality. Authors assessed whether NMAsmentioned or discussed the assumptions required(based on homogeneity, similarity, consistency, andexchangeability). Information regarding assessment of reporting bias and other details were also collected. |
Essential methodological components of the systematic review process are frequently lacking in reports of NMAs, even when published in journals with high impact factors. |
Tan et al[19] (2013) |
To establish guidance and currentpractice on summarizing resultsfrom indirect comparisons andmixed treatment comparisons; to produce recommendations toimprove current practice; toidentify research priorities forimproving reporting. |
HTA programme reports publishedbetween 1997–2011 by the NIHRin the UK that employed methodsfor indirect comparisons of mixedtreatment comparisons. |
19 (8 indirect comparisons,11 mixed treatmentcomparisons) |
Reporting of input data (presentation of the numberof interventions, study level data and the relationship structure of the interventions and the studies included in the analysis), methods (specification of Bayesian or frequentist statistical models, software used, presentation of prior distributions used, sensitivity analyses, model convergence assessment), results (presentation of relative and absolute effects, probability of treatment being best, ranking of interventions). |
A variety of methods for reporting reviews involving these methods were identified. There is no current standard approach used. Standardization of reporting methods and improvement in the use of graphical approaches to present information are needed. |
Coleman [17](2012) |
Summarize guidance for NMA;summarize traits of publishedNMAs; understand reasons forchoice and implementation ofNMA |
Studies from 2006– July 2011 wereincluded. Studies had to include 3or more treatments compared usingRCTs; published in full text;Bayesian or Frequentist approachaccepted; not included in a methodspublication or economic evaluation;English publication. |
33 Bayesian (81%), 8frequentist, 1 both |
Type of analysis (bayesian vs frequentist); reportingof methods choices (prior choice, inconsistencyevaluation, heterogeneity assessment, modelconvergence assessment); number of treatments;number of trials and patients; funding and country;clinical area; network pattern; involvement of amethodologist; fixed or random effects; use ofcovariate adjustments; account for multi-arm trials;software used; reporting details (type of outcome,summary measure, probabilities and claims ofequivalence or non-inferiority. |
Further guidance on proper conduct, interpretation and reporting of network meta-analyses is required. |
Donegan [16](2010) |
Review quality of publishedindirect treatment comparisons to add to empirical data supporting a need for improvements ofreporting of such work. |
Inclusion: reviews applying statistical methods to indirectly compare effects of 2 interventions based on RCTs. A statistical method was considered to be a quantitative summary of the ITC or a description of overlap of the two related CIs. Exclusion: ITCs shown forillustrative purposes; ITCs based onnon-randomized data; costeffectiveness and narrative reviews;trial reports; qualitative ITCs;MTC/NMA reviews. |
43 ITCs (review excluded NMAs); all based on frequentist methods |
Study inclusion criteria; ITCs made; clinical area ofstudy; # of trials and patients involved in ITC; thetype of data and measure of effect; consideration &assessment of similarity, homogeneity and consistency(with detailed criteria for each); approach to reportingof results and interpretation. Criteria based on theabove components used to identify higher quality andlower quality ITCs. |
The assumptions of ITCs aren’t always explored or reported. Reporting should be improved by more routine assessment of assumptions and clear statement of methods used for assessments. |
Song [15](2009) |
Assess the basic assumptions and other current limitations in the use of indirect comparisons in systematic reviews of healthcareinterventions |
Systematic reviews/meta-analysespublished between 2007–2007which used an indirect comparison(based on title/abstract). |
88 indirect comparisons (49/88 frequentist adjusted ITC, 18/88 NMA or Bayesian MTC, 13/88 informal ITC, 6/88 naive ITC, 2/88 unclear) |
Collected data related to comprehensiveness ofliterature search for trials included in indirectcomparisons, methods used to conduct indirectcomparison, availability of head-to-head trial data,presence/absence of explicit mention of the similarityassumption (and related efforts to explore/improvesimilarity) |
Several key methodologic problems related to authors’ unclear understanding of key assumptions for ITC. Sub-optimal search for evidence, statement and exploration/resolution of trial similarity and inappropriate combination of direct and indirect evidence are all current problems. |
Included Abstracts
|
Brooks-Renney[23] (2011; abstract) |
Review the use of NMAs within published health technology appraisals from NICE. |
Technology appraisals from NICEpublished between 2006–2011containing the term ‘mixed treatment comparison’. Submissions which werewithdrawn, terminated or suspended were not included. |
17 included technology appraisals |
Specific criteria gathered are not described. Providesinformation related to methodologic limitationsidentified in the conduct of network meta-analysessubmitted by manufacturers. |
NMA and ITC are increasingly common in technology assessments, however robust design and methdologies are needed to enesure maximum uptake of their findings. |
Bending [21](2011; abstract) |
Review the methodology andimpact of ITCs and NMAssubmitted by manufacturerson the NICE committee’sappraisal of pharmaceuticals. |
Technology appraisals from NICEpublished between 2006–2011containing either of the terms‘indirect treatment comparison’or ‘mixed treatment comparison’.Submissions which were withdrawn,terminated or suspended were notincluded. |
24 included technologyappraisals |
Number of trials included, availability of head-headevidence, disease area, treatment comparisons made,justification of study selection, sensitivity analysesrelated to trial selection, outcomes assessed. Alsocollected related information on key critiques of ITCsand NMAs. |
There is wide variation in the reporting and validity of ITCs and NMAs. There is a need for guidance for the conduct of NMAs and ITCs. |
Buckley [22] (2009 abstract) |
To investigate factors impactingoutcome of health technologyassessment submissions involvingindirect and mixed treatmentcomparisons. |
Technology appraisals from NICEpublished between 2003–2008 |
19 published technologyappraisals |
Collection of evidence presented in HTA submissionsincluding therapeutic area, clinical comparisons made,degree of direct and indirect evidence, suitability andcriticisms of type of analysis and statisticalmethodologies used, outcome appraisal (full/partialrecommendation versus not recommended) |
There is a clear increase and acceptance in use of indirect comparison methods in technology assessments. To maximize their quality, past criticsms related to use of validated methods, proper accounting for population heterogeneity, clear justification of analysis decisions made need to be considered. |