A Bayesian Mixed Treatment Comparisons Meta-Analysis of Treatments for Alcohol Dependence and Implications for Planning Future Trials

Abstract

Background

Several treatments for alcohol dependence have been tested in randomized clinical trials, giving rise to systematic reviews with a network of evidence, or mixed treatment comparisons, structure. Within the network, there are few direct comparisons of active treatments. Thus far, this network has not been adequately analyzed. For example, “indirect comparisons” between treatments (e.g., the comparison of treatments B:C obtained via estimates from A:B and A:C trials) have not been incorporated into estimates of treatment effects. This has implications for the planning of future randomized controlled trials.

Methods

We apply recent developments in Bayesian mixed treatment comparisons (MTC) meta-analysis to analyze the network of evidence. Using these results, we propose a methodology to inform, design, and power a hypothetical trial in the context of an updated meta-analysis, for treatments that have been infrequently compared, and therefore whose effects sizes are not well informed by meta-analysis.

Results

MTC meta-analysis provides more accurate estimates than pairwise meta-analysis and uncovers decisive differences between active treatments that have been infrequently directly compared. Weighting across all outcomes indicates that combination (naltrexone+acamprosate) treatment has the highest posterior probability of being the “best” treatment. If a new clinical trial were to be performed of combination therapy versus acamprosate alone, there is no feasible sample size that would result in a decisive meta-analysis.

Conclusions

MTC meta-analysis should be used to estimate treatment effects in networks where direct and indirect evidence are consistent, and to inform the design of future studies.

Background

Alcohol dependence is an important health risk factor that can lead to disability and death. Alcohol consumption is potentially avoidable, and effective strategies to help people who are dependent on alcohol to reduce excessive drinking and maintain abstinence following detoxification are needed. Several treatments have been tested in randomized controlled trials (RCTs) with various comparators of interest; thus, evidence synthesis strategies are needed for both decision-making, and planning new research studies based on the existing body of RCT literature.

The clinical practice guidelines that inform medical decision-making should rely on evidence-based evaluations of treatment options, primarily systematic reviews of randomized controlled trials (RCTs). ¹ Several systematic reviews of RCTs for pharmacological treatments for alcohol dependence have been conducted. ^2-6 Very recently, two systematic reviews were published in the Cochrane data base,^7,8 which represents the pinnacle of evidence synthesis. Cochrane reviews naturally give rise to a mixed treatment network of evidence structure (more technically termed an incomplete randomized block design of trials). Within a network, ‘direct evidence’ refers to evidence about 2 treatments that have been directly compared (e.g., A:B), while ‘indirect evidence’ refers to evidence about 2 treatments obtained from comparisons via a common comparator (e.g., the evidence for A:B derived from A:C and B:C trials).⁹ There is a growing body of literature termed ‘mixed treatment comparison’ (MTC) meta analysis^10-16 or ‘network meta-analysis’, where the objective is to combine all available evidence into an internally consistent set of estimates while respecting the randomization. The role of MTC synthesis is particularly clear in medical decision making where a choice must be made on which of several treatments to adopt in national treatment recommendations, which treatments to further study, and how to design those studies.

The goal of this paper is to assimilate the existing network of evidence using Bayesian MTC methods, and based on the findings, outline a methodology that could be followed to design a new trial. We first assess the assumptions of MTC, such as evidence consistency across direct and indirect comparisons.^17-19 We then report and compare the resulting MTC estimates with those obtained from pairwise meta-analysis, and demonstrate improved efficiency of the MTC estimates. We present an overall ranking system weighted across the four outcomes of interest, to arrive at the “best” treatment. Finally, we use results and predictions from the Bayesian MTC model in a novel way to guide the planning of a hypothetical trial for treatments infrequently directly compared that takes into account the entire existing network of evidence.

Methods

Data

Two Cochrane database systematic reviews of naltrexone (NAL) and acamprosate (ACA) have been performed and updated as of 2008⁷ and 2011,⁸ respectively. Figure 1 shows the RCT network structure where the bracketed number represents the number of trials reporting each outcome as extracted from the Cochrane Database for the return to drinking outcome only. The network demonstrates that all direct comparisons have been studied, but naltrexone and acamprosate have only been directly compared three times, and their combination has only been directly compared to placebo twice. For additional information, the reader is referred to Table 1 in the Supplementary Material, which presents summary data from each trial in the multi-arm network. Given that there are only 3 direct comparisons of ACA versus NAL in the literature, results for this comparison have been equivocal, i.e., neither treatment has been shown via pairwise meta-analysis to be superior. In addition, given only 2 direct comparisons of combined NAL+ACA with any treatment, no significant effects have been found. Since few studies consider direct comparisons of active treatments, it is essential that meta-analysis incorporate the plethora of indirect comparisons.

Figure 1.

Mixed Treatment Comparisons Model

We perform a random effects mixed treatment comparisons meta-analysis for the randomized controlled trials reported in the Cochrane database. All outcomes are modeled independently (univariately). We use a Bayesian hierarchical framework to model three binary outcomes, relapse, return to drinking, and discontinuation (R, RD, and D, respectively) and one continuous outcome, percent days abstinent (PDA). We use odds ratio to summarize the binary outcomes R, RD, and D, and mean differences to summarize PDA.

For the binary model, let the total number of events in the i^th study for intervention k be denoted by y_ik, and the corresponding number of patients be denoted by n_ik. The probability of an event (p_ik) is given based on the observed number events in each study arm. We use the logistic regression difference parameterization of Lu and Ades ¹⁷ whereby each study, i, is assigned a reference intervention arm, b, and the corresponding log-odds for that intervention is denoted μ_ib. The parameter δ_ibk denotes the i^th trial log odds ratio for intervention k relative to intervention, b, where δ_ibk is assumed to be normally distributed with pooled mean treatment effect, d_bk, and common (homogeneous) variance, σ². The odds ratio model can be expressed as,

$$\begin{array}{c}\hfill \mathit{logit}\left(p_{\mathit{ik}}\right)=\{\begin{array}{ccc}\hfill {\mu }_{\mathit{ib}}\hfill & \hfill b=A,B,C,D\hfill & \hfill \mathit{if}k=b\hfill \\ \hfill {\mu }_{\mathit{ib}}+{\delta }_{\mathit{ibk}}\hfill & \hfill k=B,C,D\hfill & \hfill \mathit{if}k\mathit{is}\text{after}b\hfill \end{array}\phantom{\}}\hfill \\ \hfill {\delta }_{\mathit{ibk}}~N\left(d_{\mathit{bk},}{\sigma }^2\right)\hfill \end{array}$$

Where p_ik is the probability of an event in trial i under treatment k and. Taking treatment A as the overall MTC baseline, all relative treatment effects can be written in terms of the following basic parameters, the consistency equations, ¹⁷

$$\begin{array}{cc}\hfill & d_{\mathit{BC}}=d_{\mathit{AC}}-d_{\mathit{AB}}\hfill \\ \hfill & d_{\mathit{BD}}=d_{\mathit{AD}}-d_{\mathit{AB}}\hfill \\ \hfill & d_{\mathit{CD}}=d_{\mathit{AD}}-d_{\mathit{AC}}.\hfill \end{array}$$

We estimate the posterior densities of the parameters of interest using MCMC methods implemented in freely available WinBUGS software.^19,20 We adapt the WinBUGS code presented in ^21,22(program 1c for binary outcomes and 5a for normal outcomes). We specify vague prior distributions for all the parameters, allowing the data to drive inference. Specifically, we assign each μ_i and d_i to be N(0,10000), which are essentially quite vague though proper. For the log odds ratio model we assign the common standard deviation, σ, a Uniform (0,5) prior, which is assumed to be the same for all pairwise comparisons. This prior distribution provides the appropriate weight to the possibility of small or zero variance. For the continuous outcome (percent days abstinent), we use the specification of program 5a,^22,23 with a normal likelihood, identity link, and normal random effects (we refer the reader to the original paper for the model set up). We specify the prior distribution for between trial variance as a Uniform(0, 10). For each outcome, we also graphically assess model sensitivity to other prior distributions.

Mixed Treatment Comparison Model Evaluation

To assess model convergence we use trace plots and the Brooks-Gelman-Rubin statistic. For each analysis we run two chains, each employing 40,000 iterations burn-in and monitoring an additional 40,000 iterations, basing posterior inference on every second sample to reduce autocorrelation in the sample. Thinning is chosen based on the observation of autocorrelation plots with the goal of minimizing autocorrelation in the resulting chain.

When carrying out MTC meta-analysis, it is essential to first check the consistency of direct and indirect sources of evidence using a hypothesis testing framework. We assess consistency based on both “node splitting” and the deviance information criteria (DIC). ^18,24 Node splitting separates evidence available for a particular treatment comparison (i.e., node in a directed acyclical graph, DAG) into ‘direct’ and ‘indirect,’ ¹⁸ and the conflict between direct and indirect evidence sources is measured by comparing their posterior distributions. Using the methods of 18 we obtain the posterior mean treatment effect d_{A, B}^dir, from the pair-wise meta-analysis of the trials comparing A and B directly, and obtain d_{A, B}^indir from the MTC meta-analysis of the remaining studies, (i.e., trials including A and C and the trials including B and C). Note that the underlying model of node-splitting is the same as the model presented in the Methods, except that the trial-specific LORs δ_{A, B} are assumed to be drawn from N(d_A,B^dir, σ²) and N(d_A,B^indir, σ²) for the direct and indirect evidence, respectively. We then examine the posterior distribution of the inconsistency parameter, ω_A,B = d_A,B^dir − d_A,B^indir to determine whether the data satisfies the evidence consistency null hypothesis, H0: d_A,B^dir = d_A,B^indir (i.e., that direct and indirect effect sizes arise from the same distribution). This is accomplished by fitting the model and using MCMC results to compute the proportion (prob) of times ω_XY > 0 in the sample. The reported p-value for the hypothesis test of evidence consistency obtained by node splitting is p= 2 × min(prob,1-prob). As a second evidence consistency check, we compare the DIC resulting from the full MTC model to that resulting from the node-split models; a reduced DIC for the node split model signifies a potential inconsistency between the direct and indirect sources of evidence on that node. Although DIC has no meaningful scale, typically differences of 5-10 (smaller is better) signal an improvement in fit, and this is how we interpret model comparisons in this paper. ²⁴

For each outcome we also compare Bayesian MTC meta-analysis posterior mean effects and 95% credible intervals (CIs) with those from Bayesian random effects pairwise meta analyses of head to head (direct) comparisons. For the outcomes relapse, return to drinking, and discontinuation, all results are presented in terms of posterior odds ratios (OR) (95% CI) where a value > 1 indicates a worse outcome. For percent days abstinent, results are reported in terms of posterior mean difference in percent days abstinent (95% CI). In general, a decisive or “significant” effect corresponds to a 95% credible interval excluding 1 for ORs and 0 for mean differences. As in Ades et al (2011),²⁵ we report the standard deviation (σ) and its 95% CI, the posterior mean residual deviance, the effective number of parameters, and the DIC for each MTC model.

Planning a new trial

The effect size used to generate the sample size for new trials is typically derived from observed effect sizes from previous clinical or preclinical trials, or at best, from a reported meta-analysis. This effect size is then used to power the trial in isolation of accumulated evidence. However, the empirical distribution of observed meta-analysis effect sizes should not be used to plan future trials since this distribution of estimates is overdispersed. New research suggests that basing the sample size of a future trial on the results of an updated meta-analysis that will include it, is more sensible than powering a future trial in isolation.²⁶

Posterior distributions of effect sizes and posterior predictive distributions for a new observation are natural consequences of Bayesian estimation and sensible tools on which to base future studies. A predictiveeffect size, δ_bk(new), can be generated and given a fixed baseline group effect size, used to generate a new observation, y_ik(new), from a fixed sample size. However, no closed form expression for this distribution exists and hence sampling from it within a simulation framework such as that offered by Bayesian MCMC methods would appear to be the most straightforward.

We apply the following approach in the MTC meta-analytic setting adapted from (21) for planning a 2 arm RCT with equal patient allocation and binary outcome based on odds ratios, and testing H₀: δ_bk = 0 versus H₁: δ_bk is not equal to 0:

1. While fitting the MTC meta-analysis in WinBUGS, generate an estimate, δ_bk(new).

2. Given the predicted effect size δ_bk(new) and assumed known reference group (b) event rate, calculate the event rate in group k and stochastically generate a new set of data, y_k(new) from a binomial distribution with fixed sample size, n_k(new).

3. Using the standard hypothesis testing framework, run 1000 simulations whereby each data set generated under a fixed sample size, n_k(new) is included in an updated MTC metaanalysis.

4. The percentage of times the meta-analytic finding is decisive (or “significant”), in the new MTC meta-analysis is the proposed power of the study. Thus, power is calculated by adding the study to the MTC meta-analysis for each simulation iteration, and assessing the proportion of times which the MTC meta-analytic estimate for k:b is decisive (i.e., 95% credible interval excluding 1).

5. Repeat 1-4 for user-specified fixed sample sizes with range [0,2000].

Results

Mixed Treatment Comparison Model Evaluation

The exact p-values for the test of the null hypothesis of evidence consistency formulated by node splitting comparisons of direct and indirect evidence, as well as node splitting and full model DICs are reported in Table 1. All hypothesis tests of evidence consistency are non-significant at the .05 alpha-level, indicating consistent directionality of evidence and therefore, the appropriateness of MTC meta-analysis for this treatment network. Further, the DICs reported for the full MTC model are consistently lower than those reported from the node-split model, which provides additional evidence of consistency.¹⁸

Table 1.

Goodness of fit statistics for Node splitting models. Dbar = posterior mean of the residual deviance, DIC=Deviance Information Criteria, pD=effective degrees of freedom.

Treatment Comparison	Node splitting Posterior mean SD (95% CI)	Dbar	DIC	pD	MTC Posterior mean SD (95% CI)	Dbar	DIC	pD	Inconsistency (p-value)
Relapse:
NAL:PLB	0.23(0.01,0.52)	291.4	328.5	37.1					0.14
ACA:PLB	0.25(0.01,0.51)	290.7	328.0	37.3					0.87
NAL+ACA:PLB	0.25(0.01,0.25)	291.8	329.7	37.9	0.25(0.02,0.58)	177.7	199.8	22.1	0.45
ACA:NAL	0.23(0.01,0.52)	291.0	328.3	37.3					0.16
NAL:NAL+ACA	0.26(0.01,0.54)	291.0	329.1	38.1					0.39
ACA:NAL+ACA	0.25(0.01,0.54)	291.7	329.5	37.8					0.64
Return to Drinking:
NAL:PLB	0.47(0.28,0.69)	503.7	578.7	75.0					0.84
ACA:PLB	0.46(0.27,0.68)	504.6	579.1	74.5					0.16
NAL+ACA:PLB	0.49(0.30,0.71)	503.1	578.4	75.3	0.43(0.21,0.66)	398.8	456.0	57.2	0.50
ACA:NAL	0.47(0.29,0.69)	503.9	578.9	75.0					0.77
NAL:NAL+ACA	0.49(0.31,0.70)	503.1	578.5	75.4					0.71
ACA:NAL+ACA	0.48(0.29,0.70)	503.7	578.6	74.9					0.30
% Days Abstinent:‡
NAL:PLB	8.59(6.45,9.93)	341.7	394	52.1					0.49
ACA:PLB	8.56(6.41,9.93)	340.7	393	52.0					0.70
NAL+ACA:PLB	8.49(6.26,9.93)	342.0	394	51.8	7.29(5.05,9.69)	246.2	290.7	44.4	0.08
ACA:NAL	8.60(6.47,9.94)	341.2	393	52.0					0.34
NANL:NAL+ACA	--	--	--	--					--
ACA:NAL+ACA	8.50(6.29,9.93)	343.4	397	53.3					0.86
Discontinuation:
NAL:PLB	0.41(0.27,0.58)	546.8	624.6	77.8					0.63
ACA:PLB	0.41(0.26,0.57)	547.3	624.8	77.5					0.28
NAL+ACA:PLB	0.41(0.27,0.58)	546.9	624.7	77.8	0.39(0.24,0.57)	459.5	523.4	66.9	0.62
ACA:NAL	0.41(0.27,0.58)	547.3	625.5	78.2					0.58
NAL:NAL+ACA	0.42(0.27,0.58)	546.6	624.4	77.8					0.59
ACA:NAL+ACA	0.41(0.27,0.58)	546.9	624.5	77.6					0.29

Table 1 also shows that for the binary outcomes, the posterior mean standard deviation is moderate-sized (range = [0.25,0.43]), and different from zero for all three binary outcomes but within an expected range as per previous reports (e.g.,²⁵). For the continuous outcome PDA, the posterior mean SD is also moderate-sized. Global goodness of fit tests including posterior mean of the residual deviance, DIC, and effective degrees of freedom (pD), are also reported in Table 1. Figure 2 shows that the posterior mean estimate of σ for a range of uniform priors remains unchanged for the log odds ratio model, and also remains unchanged for upper bound greater than or equal to 10 for the continuous model. Thus, we use a prior distribution of U(0,10) for the continuous outcome .

Figure 2.

Posterior mean and 95% credible intervals for pairwise (left) and MTC (right) meta-analysis. An OR < 1 and a positive % days abstinent (PDA) both favor the first treatment in the comparisons.

Mixed Treatment Meta-Analysis Results

Figure 3 presents results of the pairwise meta-analysis of head to head trials (left) and MTC meta-analysis (right) for all possible treatment comparisons. MTC and pairwise results are directionally similar, which is expected given confirmed evidence consistency. However, for infrequently compared active treatments, 95% CIs are much narrower for the MTC model. The MTC model clearly benefits from greater precision since common comparator information available from the network of evidence is utilized to improve estimates.

Figure 3.

Posterior mean standard deviation (y-axis) for a sensitivity analysis to prior specification for the standard deviation parameter, σ (x-axis).

Of primary interest is how ACA compares to NAL since only 3 direct comparisons of these treatments have been performed, resulting in not statistically significant pairwise meta-analytic findings. From MTC meta-analysis, ACA increases the odds of relapse versus NAL (OR=1.64 [1.10,2.56]). This effect previously was not found to be significant, although was in the same direction.^7,8 Similarly, ACA increases the percent days abstinent over NAL (mean difference=7.14[4.13, 10.11]), which was also not previously detected as significant in pairwise meta-analysis. These findings are consistent with the clinical literature, which posits naltrexone as a treatment for relapse and acamprosate for the maintenance of abstinence.²⁷ Thus, MTC analysis confirms that these treatments appear to have beneficial effects on different aspects of drinking behavior.

Figure 3 also indicates that combined NAL+ACA versus placebo reduces the odds of returning to drinking (OR=0.42[0.16,0.84]). While this is a greater estimated effect size than either NAL or ACA versus placebo, MTC results indicate that combination treatment is not better than either NAL or ACA alone; however, combination treatment has rarely been directly compared with active treatments.

The posterior probabilities of being among the k^th best treatments are presented as rankograms in Figure 4.²⁵ The 4 panels show the probability of being the first, second, third, and fourth best treatment for each outcome. While these estimates should not be interpreted conclusively (because they are close in value due to the considerable uncertainty in inference about relative treatment effects, as demonstrated in Figure 3), they are potentially informative for deciding which treatments to further study. It is notable that placebo is never the best treatment. Naltrexone appears to be superior for reducing the odds of relapse (mean probability of being the best = 0.59) while ACA appears to be superior for percent days abstinent (mean probability of being best = 0.82). With respect to discontinuation, all treatment groups appear to be equivalent. We further estimate a weighted rank across the four outcomes using an equal weighting scheme in the absence of prior information as to which outcome is more important (0.25 for each outcome). The posterior probability of being the best treatment is 0, 0.35, 0.24, and 0.41 for PLB, ACA, NAL, and NAL+ACA, respectively. Although MCT meta-analysis is able to provide decisive estimates of treatment effects when pairwise meta-analysis cannot, the posterior rankings for the 3 active treatments are still inconclusive.

Figure 4.

Treatment rankings: probability that each treatment is the kth best (k=1,2,3,4) on reducing relapse, return to drinking, discontinuation, and increasing percent days abstinent.

Planning a New Trial

There is some indication by the weighted (omnibus) posterior rankings, that the treatment combination of NAL+ACA has the highest posterior probability of being the best treatment. This finding is driven by the outcomes RD and relapse. However, decisive effects of the combination treatment versus ACA alone are not detected by MTC meta-analysis (MTC OR = 0.81[0.31,1.58] for RD and 0.69[0.32,1.14] for relapse); however, few head to head trials of combined therapy with any comparator exists. A decisive effect is also not detected in pairwise meta-analysis, a result based on only 2 head to head trials (Figure 1). Specifically, one of these head to head trials did not ²⁷ and one did ²⁸ show a significant effect of combined therapy over ACA monotherapy. Importantly, the trial that did not show significance was 10 times larger than the one that did, thus it provides the majority of the weight in meta-analysis.

Given the equivocal results in light of very little direct evidence for this comparison, subject matter researchers might still question whether there is any benefit of the treatment combination over ACA. In fact, studies are currently underway to detect such effects. ^29,30 As a result, we plan a hypothetical new 2-arm trial for RD and relapse powered in the context of an updated MTC meta-analysis to further investigate these interventions. Since most RCTs favor the 2 arm approach, we consider this case for ease of presentation, although the approach easily could be extended to any number of treatment arms. The following serves as an example of how such a study might be planned.

For the outcomes relapse and RD, we obtain predictive estimates for the effect size (log OR), δ_{NAL+ACA:ACA(new)}, where the event rate of ACA (the reference group) is assumed known from the 24 placebo-controlled trials of acamprosate (Figure 1). The distribution of 1000 sampled predictive effect sizes for the outcomes is shown in Figure 5. Following the methodology outlined in the Methods, Figure 6 shows the power curve for the hypothetical trial designed in two ways: 1.) in isolation, as a single new clinical trial that is not part of the network of evidence, and 2.) in the context of the network of evidence presented in Figure 1, which after the trial’s completion, would be subject to a new MTC meta-analysis. Designing this trial in isolation requires a feasible sample size to obtain 80% power to detect a significant effect on either outcome; however, obtaining 80% power to detect a decisive effect in a new MTC meta-analysis is infeasible, as the power curve asymptotes at approximately 0.60 and 0.18 for relapse and RD, respectively, for any reasonable sample size. The reason for this is that irrespective of the size of the new study, its influence in random effects meta-analysis is limited by the degree of heterogeneity in the analysis. In fact, the sample size directly depends on the amount of heterogeneity in the existing network of trials, as the weight of the i^th study in classical random effects meta-analysis is $\frac{1}{({\sigma }^2+{\tau }_i^2)}$ where ${\tau }_i^2$ is the estimated variance of the effect size in the i^th (i.e., new) study. ²⁶ In our MTC meta-analysis, the posterior estimate σ² is moderately large (Table 1), which means the contribution of the new study is proportionally down-weighted. The result is that designing a trial in isolation using the observed random effects meta-analytic effect size will underestimate the size of the trial that needs to be performed to add to the existing network of evidence. This process was repeated by breaking up the new study into several smaller studies. While power improves slightly, 80% power is still not achieved.

Figure 5.

Sampled effect sizes for relapse (left) and return to drinking (right).

Figure 6.

Power and sample size curves for a study of naltrexone+acamprosate versus acamprosate designed in isolation (dashed line) and a study designed to detect significance in a new MTC meta-analysis (solid line) for the outcomes relapse (top) and return to drinking (bottom).

Conclusion

Mixed treatment evidence structures are more common than pairwise structures, ¹⁷ and research supports refining evidence synthesis by incorporating both direct and indirect treatment comparisons to better inform health care decisions. ^{9,16,17,31,32} The role of MTC synthesis is particularly clear in medical decision making where a choice must be made on which of several treatments to adopt in national treatment recommendations, as well as which treatments to further study and how to design those studies. We have shown that consistency of evidence is established for treatments for alcohol dependence, which indicates that a relatively homogenous internal structure exists. We have shown that utilizing MTC evidence allows for the detection of significant effects for treatments that are infrequently directly compared (e.g., ACA vs NAL and NAL+ACA vs Placebo), tighter credible intervals when indirect effects are exploited, and posterior rank ordering of treatments.

The results of our MTC analysis can assist planning and decision-making by determining which treatment comparisons lack direct and/or indirect evidence, and need to be further compared. Specifically, this is achieved by determining effect sizes from a new trial using posterior or posterior predictive estimates from MTC meta-analysis, and then designing the trial in the context of a new MTC meta-analysis. We have presented a methodology for using Bayesian MTC meta-analysis results to determine which treatments should be further studied, for generating predicted clinical trial data, and for powering the new study in the context of refitting the meta-analysis. The simulations show that the power for designing a trial in isolation versus in the context of meta-analysis can be dramatically different, and depends both on the number of direct comparisons and between trial heterogeneity.

Although we did not pursue it in this paper, the value of information (VoI) may also be used for to design and prioritize future trials. ³³ This methodology involves evaluating the likelihood that the decision will change following the new study, as well as the net benefit in performing the new study. These quantities of interest depend on source of variation in the current meta-analysis and the predictive heterogeneity. Greater certainty and smaller study bias result in larger value of information. VoI can be evaluated for a range of different types of studies, for example, muti-arm and multi-center, in order to prioritize the RCT to perform next.

Our approach easily could be extended to designing a study for any number of treatment arms, which is a scenario of importance due to the difficulty of such designs. In general, when planning a >2 arm clinical trial, identifying a priori the correlation structure between several treatment effects can be difficult. In the context of a heterogeneous MTC model for multi-arm networks of evidence, the correlation among treatment groups within a study would be propagated through the Bayesian MTC meta-analytic model to the posterior predictive estimates of effect size for a new study; thus, this correlation structure would automatically be taken into account when designing the new study.

Limitations of the current study along with further research directions must be addressed. Firstly, our analysis of several outcomes likely inflates the type I error rate. Rather than enforcing a strict correction for multiple comparisons, this could be addressed in future research via a multivariate MTC meta-analysis, which would help control the type I error rate by exploiting correlation in outcomes, and by borrowing information in light of missing outcomes. Secondly, while naltrexone decreases relapse and acamprosate improves percent days abstinent, which is consistent with their biochemical nature, when taking all 4 outcomes into consideration simultaneously, there is no consensus as to which treatment is best.