Performing Arm-Based Network Meta-Analysis in R with the pcnetmeta Package

Abstract

Network meta-analysis is a powerful approach for synthesizing direct and indirect evidence about multiple treatment comparisons from a collection of independent studies. At present, the most widely used method in network meta-analysis is contrast-based, in which a baseline treatment needs to be specified in each study, and the analysis focuses on modeling relative treatment effects (typically log odds ratios). However, population-averaged treatment-specific parameters, such as absolute risks, cannot be estimated by this method without an external data source or a separate model for a reference treatment. Recently, an arm-based network meta-analysis method has been proposed, and the R package pcnetmeta provides user-friendly functions for its implementation. This package estimates both absolute and relative effects, and can handle binary, continuous, and count outcomes.

1. Introduction

In diverse scientific fields, such as social and medical research, summaries of cumulative knowledge are increasingly based on the results of meta-analyses (Hunter and Schmidt 1996; Lindholm et al. 2005; Cooper et al. 2009). Meta-analysis is a statistical method for combining and contrasting a collection of estimated effect sizes, such as odds ratios, from multiple independent studies (DerSimonian and Laird 1986). Various approaches for treatment comparisons in meta-analysis have been introduced (Hedges and Olkin 1985; Sutton et al. 2000; Higgins and Green 2008).

Traditional meta-analysis focuses on direct pairwise comparisons between two treatments in the collected studies. In some cases, however, not enough studies directly compare two treatments of interest. For example, suppose A, B, and C are three treatments for a disease, and researchers aim to compare A vs. B, while published studies only compare A vs. C or B vs. C. Although direct evidence is not available, the comparisons A vs. C and B vs. C provide indirect evidence. Based on this idea, network meta-analysis, also known as mixed treatment comparisons, was developed to simultaneously compare multiple treatments by synthesizing direct and indirect evidence (Lumley 2002; Lu and Ades 2004, 2006; Salanti et al. 2008; Lu and Ades 2009; Zhang et al. 2014). This technique has been widely applied in medical research (e.g., Psaty et al. 2003; Elliott and Meyer 2007; Cipriani et al. 2009). Currently, two types of approaches are used for network meta-analysis. The first approach is contrast-based. It focuses on modeling treatment contrasts (relative effect sizes, typically log odds ratios) within each study (Lu and Ades 2004). Another approach is arm-based (e.g., Zhang et al. 2014, 2015a; Hong et al. 2016a,b), which focuses on describing population-averaged absolute effect sizes for each treatment arm.

We use an illustrative example to show the difference between the contrast- and arm-based approaches. Suppose that the outcome in a network meta-analysis is binary, and y_ik and n_ik are the numbers of events and participants, respectively, in treatment group k in the ith study. For such data, both the contrast- and arm-based models use the binomial likelihood y_ik ~ Binomial(n_ik, p_ik); they differ in the way they model the underlying absolute risks p_ik in each study’s treatment group. Specifically, the contrast-based method needs to specify a baseline treatment b(i) in the ith study. For convenience, we simply denote b(i) as b. Then, the Bayesian hierarchical model for this approach is (Lu and Ades 2004, 2009):

$$\begin{array}{ll}\hfill g(p_{ik})& ={\mu }_i+X_{ik}{\delta }_{\mathit{ibk}};\hfill \\ \hfill {\delta }_{\mathit{ibk}}& ~N(d_{bk},{\sigma }_{bk}^2),\hfill \end{array}$$

where g(·) is a link function and X_ik is a dummy variable taking the value 0 if k = b or 1 if k ≠ b. Also, μ_i is the baseline effect for treatment b in the ith study, and δ_ibk is the relative effect of treatment k compared with the baseline b on the g-transformed scale. Note that this model treats the μ_i’s as nuisances and uses non-informative priors for them. This model is described as contrast-based because it focuses on the overall relative effects d_hk between treatment pairs (h, k), which are estimated using the evidence consistency equation d_hk = d_bk − d_bh. This model does not permit a back-transformation from the relative effects to absolute effects, unless the absolute effect of a given “reference” treatment group can be accurately estimated from external data, or can be estimated using a separate model to analyze the existing data in the “reference” treatment group (Welton et al. 2012; Dias et al. 2013b).

Population-averaged absolute effects are preferred in some situations such as cost-effectiveness analysis and patient decisions (Dias et al. 2013b). For example, consider two scenarios comparing treatments A and B according to one-year survival rates: (i) p_A = 0.8 vs. p_B = 0.5; (ii) p_A = 0.004 vs. p_B = 0.001. Both scenarios yield an odds ratio 4.0, but patients would prefer treatment A in scenario (i) more strongly than in scenario (ii). Therefore, an absolute effect or absolute difference is preferred in this case. Compared with the contrast-based method, the arm-based model provides a straightforward way to estimate absolute effects and various types of relative effects. The model is specified as (Zhang et al. 2014, 2015a):

$$\begin{array}{ll}\hfill g(p_{ik})& ={\mu }_k+{\nu }_{ik};\hfill \\ \hfill {({\nu }_{i1},{\nu }_{i2},\dots ,{\nu }_{iK})}^{\top }& ~\mathit{MVN}(\mathbf{0},{\mathbf{\sum }}_K),\hfill \end{array}$$

where Σ_K is the variance-covariance matrix of the vector of random effects specific to study i. The μ_k’s are treatment-specific parameters reflecting absolute effects. Based on the absolute effects, various types of relative effects can be obtained. In some cases (e.g., the missingness of treatment arms is not at random), the effect size estimates produced by the arm-based method may be less biased than those given by the contrast-based method (Zhang et al. 2015a). Moreover, the arm-based method can use information contained in single-arm studies, in which only one treatment group is available or of interest. Single-arm studies cannot be included in the contrast-based model but they may provide valuable information for treatment comparisons and enhance the robustness of a network meta-analysis (Lin et al. 2016a). Although the arm-based approach has many advantages, the convergence of Markov chain Monte Carlo (MCMC) algorithms for parameter estimation may be slower compared with the contrast-based approach. The estimates using the arm-based models may not converge well if some treatments are only included in a few (say, less than three) studies. Also, some researchers have concerns about the arm-based model, for example, that absolute effects tend to be highly variable compared to relative effects, and that pooling arm-level data may not fully respect the randomization process in randomized controlled trials (Dias and Ades 2016). However, these concerns are mainly raised because the arm- and contrast-based models use different assumptions about treatment effects. Specifically, the contrast-based model assumes that relative effects are exchangeable across studies, while the arm-based model assumes absolute effects are exchangeable (Hong et al. 2016b). Although the assumption of exchangeable relative effects is popular in meta-analysis, the assumption of exchangeable absolute effects is also accepted in the literature (see, e.g., van Houwelingen et al. 1993; Shuster et al. 2007; Senn 2010; Chu et al. 2012). More details of the arm-based model are discussed in Dias and Ades (2016) and Hong et al. (2016b).

Plenty of software packages are dedicated to conducting traditional meta-analysis (e.g., Rosenberg et al. 2000; Borenstein et al. 2005; Viechtbauer 2014; Schwarzer 2015), but very limited software is available specifically for network meta-analysis. The R package netmeta (Rücker et al. 2015) provides models in a frequentist framework described in Rücker (2012). The R package gemtc (van Valkenhoef and Kuiper 2015) and the Stata (StataCorp 2015) command network perform contrast-based analyses. Neither package provides estimates for population-averaged treatment-specific parameters. This article introduces the R package pcnetmeta (Lin et al. 2016b), which performs network meta-analysis using the arm-based model and provides estimates for various effect sizes. This package is available from Comprehensive R Archive Network (R Core Team 2015) at http://CRAN.R-project.org/package=pcnetmeta. It uses MCMC techniques on the R platform through JAGS (Plummer 2015b, 2016). JAGS is a program for analyzing Bayesian hierarchical models using MCMC simulation, which is available for diverse computer platforms including Windows and Mac OS X.

The package pcnetmeta provides user-friendly functions to perform network meta-analysis for various types of data. Convergence of the MCMC routine can be assessed by the function outputs. The package also provides functions to draw network plots which illustrate the comparisons between multiple treatments. In addition, plots for 95% credible intervals of treatment-specific and relative effect sizes are provided to visually display treatment effects and their comparisons.

This article is organized as follows. Section 2 presents an overview of arm-based Bayesian hierarchical models as implemented in the pcnetmeta package. Section 3 illustrates the use of the package with several examples, and discusses the output structures. Finally, Section 4 closes with suggested future improvements.

2. Arm-based models for network meta-analysis

2.1. Arm-based model for binary outcomes

Suppose a network meta-analysis reviews I studies on K treatments, where each study only investigated a subset of the K treatments. Let T_i (1 ≤ i ≤ I) be the set of treatments compared in the ith study. Also, in the ith study, the total number of events/participants in treatment group k (k ∈ T_i) is denoted by y_ik/n_ik. Zhang et al. (2014) specified the following arm-based model using the probit link function:

$$\begin{array}{ll}\hfill y_{ik}& ~\text{Binomial}(n_{ik},p_{ik})k\in T_i\hfill \\ \hfill {\mathrm{\Phi }}^{-1}(p_{ik})& ={\mu }_k+{\nu }_{ik}\hfill \\ \hfill {({\nu }_{i1},{\nu }_{i2},\cdots ,{\nu }_{iK})}^{\top }& ~\mathit{MVN}(\mathbf{0},{\mathbf{\sum }}_K),\hfill \end{array}$$ (1)

where Φ(·) denotes the standard normal cumulative distribution function. In this model, μ_k is a fixed effect for Treatment k, and the random effects (ν_i1, ν_i2, …, ν_iK)^⊤ are correlated within each study with variance-covariance matrix Σ_K. Based on this model, the absolute risk of Treatment k can be estimated as $p_k=\mathrm{E}[p_{ik}\mid {\mu }_k,{\sigma }_k]=\mathrm{\Phi }\left({\mu }_k/\sqrt{1+{\sigma }_k^2}\right)$ (Zeger et al. 1988), where ${\sigma }_k^2$ is the kth diagonal element in Σ_K. Since this estimate is a marginal expectation of p_ik given μ_k and σ_k, we can interpret p_k as the population-averaged absolute risk of Treatment k. With the estimates p_k for absolute risk, we can further estimate the risk difference (RD), odds ratio (OR), and risk ratio (RR), which are defined as RD_kl = p_k − p_l, ${\text{OR}}_{kl}=\frac{p_k/(1-p_k)}{p_l/(1-p_l)}$, and RR_kl = p_k/p_l, respectively. Other link functions may be also considered in the arm-based model, but they do not yield simple expressions for population-averaged absolute risks, though some approximations exist. For example, using the logit link we can approximate $p_k=\mathrm{E}[p_{ik}\mid {\mu }_k,{\sigma }_k]\approx {\left[1+exp\left(-{\mu }_k/\sqrt{1+C^2{\sigma }_k^2}\right)\right]}^{-1}$, where $C=\frac{16\sqrt{3}}{15\pi }$ (Zeger et al. 1988). This article and the package pcnetmeta use the probit link function for simplicity.

2.2. Arm-based model for continuous outcomes

Researchers can also perform network meta-analysis on studies with continuous outcomes (e.g., Kasiske et al. 1998; Philbrook et al. 2007; Zhang et al. 2015b). We continue to use i and k to index the studies and treatments and n_ik as the total number of participants receiving Treatment k in the ith study. The summary data include sample mean ȳ_ik and within-study sample standard deviation s_ik. We can specify the arm-based model for continuous outcomes as

$$\begin{array}{ll}\hfill {\overline{y}}_{ik}& ~N({\theta }_{ik},s_{ik}^2/n_{ik})k\in T_i\hfill \\ \hfill {\theta }_{ik}& ={\mu }_k+{\nu }_{ik}\hfill \\ \hfill {({\nu }_{i1},{\nu }_{i2},\cdots ,{\nu }_{iK})}^{\top }& ~\mathit{MVN}(\mathbf{0},{\mathbf{\sum }}_K).\hfill \end{array}$$ (2)

In this model, μ_k is of interest because it can be interpreted as the overall effect of Treatment k by noticing that E[θ_ik|μ_k, σ_k] = μ_k, and we may further estimate the effect difference between Treatments k and l, which is defined as d_kl = μ_k − μ_l.

2.3. Arm-based model for count datasets

Dias et al. (2013a) discussed methods for count datasets in network meta-analysis. Their models are contrast-based, but we can consider the corresponding arm-based versions.

In some network meta-analyses, the available data are in the form of counts over a certain time period. The total number of person-years at risk is supplied rather than the total number of participants. Let y_ik be the number of events in treatment group k in the ith study, and E_ik is the corresponding exposure time in person-years. Suppose λ_ik is the treatment-specific rate for treatment group k in the ith study, and we are interested in the population-averaged treatment-specific rate. We consider the following arm-based model with a Poisson likelihood and the log link function:

$$\begin{array}{ll}\hfill y_{ik}& ~\text{Poisson}(E_{ik}{\lambda }_{ik})k\in T_i\hfill \\ \hfill log({\lambda }_{ik})& ={\mu }_k+{\nu }_{ik}\hfill \\ \hfill {({\nu }_{i1},{\nu }_{i2},\cdots ,{\nu }_{iK})}^{\top }& ~\mathit{MVN}(\mathbf{0},{\mathbf{\sum }}_K).\hfill \end{array}$$ (3)

A key assumption for this model is that the rates λ_ik are constant over the follow-up period. Based on this model, the population-averaged treatment specific rate can be estimated as ${\lambda }_k=\mathrm{E}[{\lambda }_{ik}\mid {\mu }_k,{\sigma }_k]=exp({\mu }_k+{\sigma }_k^2/2)$.

A similar model is available for studies that report the proportion of patients developing an event within a given follow-up period, where the follow-up time may differ for each study (e.g., Psaty et al. 2003; Elliott and Meyer 2007). In this case, the event probability depends on the length of follow-up. Specifically, we have y_ik and n_ik as the number of events and participants, respectively, in treatment group k in the ith study, and the study-specific follow-up times are denoted f_i. Following Dias et al. (2013a), we assume a latent Poisson process with rate λ_ik for each treatment group in each study; therefore the time T_ik until an event occurs is distributed as exponential with rate λ_ik and survivor function Pr(T_ik > f_i) = exp(−λ_ikf_i). Thus, the event probability is p_ik = 1 − exp(−λ_ikf_i). Using the complementary log-log link cloglog(t) = log(−log(1 − t)) for p_ik, we have cloglog(p_ik) = log(f_i) + log(λ_ik). Again, we can model log(λ_ik) using model (3) above, and an arm-based model is constructed as follows:

$$\begin{array}{ll}\hfill y_{ik}& ~\text{Binomial}(n_{ik},p_{ik})k\in T_i\hfill \\ \hfill \text{cloglog}(p_{ik})& =log(f_i)+log({\lambda }_{ik})\hfill \\ \hfill log({\lambda }_{ik})& ={\mu }_k+{\nu }_{ik}\hfill \\ \hfill {({\nu }_{i1},{\nu }_{i2},\cdots ,{\nu }_{iK})}^{\top }& ~\mathit{MVN}(\mathbf{0},{\mathbf{\sum }}_K).\hfill \end{array}$$ (4)

The parameter of interest is still the population-averaged treatment-specific rate ${\lambda }_k=exp({\mu }_k+{\sigma }_k^2/2)$.

2.4. Parameter estimation for arm-based models

Let ν_i = (ν_i1, ν_i2, …, ν_iK)^⊤. The full likelihood function for the arm-based model is

$$L\propto \prod _{i=1}^I{\int }_{{ℝ}^K}\left(\prod _{k\in T_i}{L}_c(y_{ik};{\mu }_k,{\nu }_{ik})\right)\times \frac{exp(-\frac{1}{2}{\nu }_i^{\top }{\mathrm{\sum }}_K^{-1}{\nu }_i)}{{(2\pi )}^{K/2}{\mid {\mathrm{\sum }}_K\mid }^{1/2}}d{\nu }_i$$

where L_c(y_ik; μ_k, ν_ik) is the conditional likelihood given the random effects ν_ik. For example, for binary outcomes, the conditional likelihood is

$$L_c(y_{ik};{\mu }_k,{\nu }_{ik})={[g^{-1}({\mu }_k+{\nu }_{ik})]}^{y_{ik}}{[1-g^{-1}({\mu }_k+{\nu }_{ik})]}^{n_{ik}-y_{ik}},$$

where g⁻¹(·) is the inverse of the link function. To obtain the maximum likelihood estimates, the maximization problem is subject to the condition that Σ_K is positive definite. When the outcome is binary or count, the full likelihood function cannot be expressed in a closed form; for continuous outcomes, it may have a closed form. However, maximizing the likelihood may be unstable and converge slowly if the number of treatments K is large and the number of collected studies I is small.

Alternatively, we can, and pcnetmeta does, apply MCMC to obtain Bayesian estimates for parameters of interest. In arm-based models, vague N(0, 1000) priors are used for the treatment-specific fixed effects μ_k. As suggested in Gelman and Hill (2007), we may assign an inverse-Wishart prior to the unstructured variance-covariance matrix Σ_K with the scale matrix being the K × K identity matrix I_K and degrees of freedom K + 1, i.e., ${\mathbf{\sum }}_K^{-1}~\text{Wishart}({\mathbf{I}}_K,K+1)$. This has the effect of setting a uniform prior on the individual correlation parameters. Alternatively, the separation strategy by Cholesky decomposition can be used to specify a vague prior to Σ_K (Barnard et al. 2000; Lu and Ades 2009; Wei and Higgins 2013). We denote this model as HET-COR because the variances of the random effects are heterogeneous.

To reduce model complexity, we may assume an exchangeable correlation structure for Σ_K (Zhang et al. 2015a), that is, Σ_K = DR_exD, where D = diag(σ₁, σ₂, …, σ_K) and

$${\mathbf{R}}_{\text{ex}}=\left[\begin{array}{cccc}1& \rho & \cdots & \rho \\ \rho & 1& \cdots & \rho \\ ⋮& ⋮& \ddots & ⋮\\ \rho & \rho & \cdots & 1\end{array}\right].$$

To guarantee that R_ex is positive definite, ρ must be greater than $-\frac{1}{K-1}$. A vague uniform prior on $\left(-\frac{1}{K-1},1\right)$ can be used for ρ. We denote models with this exchangeable correlation matrix as HET-EQCOR for heterogeneous variances σ_k. If we further assume homogeneity of variances, that is, σ_k = σ for all k = 1, 2, …, K, the model is denoted as HOM-EQCOR.

To avoid overfitting, we can use the deviance information criterion (DIC) proposed by Spiegelhalter et al. (2002) for model selection. A smaller penalized deviance implies a better model.

Finally, to implement the hierarchical models in JAGS, the package pcnetmeta automatically generates initial values for the parameters and specifies different random number generators (RNGs) for different chains. Zero is used to initialize the random effects ν_ik; the initial values for the variance parameters Σ_K are the means of their prior distributions. For example, since ${\mathbf{\sum }}_K^{-1}~\text{Wishart}({\mathbf{I}}_K,K+1)$ in the HET-COR model, the initial value for ${\mathbf{\sum }}_K^{-1}$ is the mean of this Wishart distribution, (K + 1)I_K. For the fixed effects μ_k, the initial values are the naïve estimates of the corresponding absolute effects computed by simply pooling the data in each treatment group. For example, consider the continuous outcome in model (2), the initial values for μ_k’s are Σ_{i:k∈Ti}ȳ_ikn_ik/Σ_{i:k∈Ti}n_ik.

3. Using the R package pcnetmeta

The R package pcnetmeta provides user-friendly functions to perform arm-based network meta-analysis using the models described above. Users can download its source file at http://CRAN.R-project.org/package=pcnetmeta, or directly install it within R by typing install.packages(“pcnetmeta”). Note that the pcnetmeta depends on the R packages rjags (Plummer 2016) and coda (Plummer 2015a). The pcnetmeta package does not include a copy of the JAGS library, so users must install JAGS separately. JAGS is freely available at its homepage http://mcmc-jags.sourceforge.net/. Also, the package pcnetmeta requires JAGS version ≥ 4.0.0; the earlier versions of JAGS may not guarantee exact reproducibility of the results. In this section, we introduce the basic usage of this package.

3.1. Data structure for network meta-analysis

To begin, we briefly introduce the necessary dataset structures. In the package pcnetmeta, four datasets, smoke, parkinson, dietaryfat, and diabetes, are provided as illustrative examples.

The dataset smoke contains 24 studies on smoking cessation with binary outcomes, reported in Hasselblad (1998) and Lu and Ades (2006). This network meta-analysis compares four treatments, labeled as: 1) no contact (NC); 2) self-help (SH); 3) individual counseling (IC); and 4) group counseling (GC). We display the dataset’s first few rows below. The column s.id contains IDs for the 24 studies, and t.id labels the treatments included in each study. For example, Study 1 compares Treatments 1, 3, and 4. The columns r and n are the number of events (successful cessation) and participants, respectively.

R> library(“pcnetmeta”)
R> data(“smoke”)
R> head(smoke)
s.id  t.id    r      n
1     1      1    9  140
2     1      3  23  140
3     1      4  10  138
4     2      2  11    78
5     2      3  12    85
6     2      4  29  170

The dataset parkinson is a collection of studies with continuous outcomes, reported in Dias et al. (2013a). It contains 7 studies on 5 treatments. The outcome is the mean off-time reduction in patients given dopamine agonists as adjunct therapy in Parkinson’s disease. One treatment is placebo, coded by 1, and the other four treatments are active drugs, coded 2 to 5. The dataset is displayed below. The columns s.id, t.id and n have the same meanings as in the dataset smoke, while mean and sd report the sample means and standard deviations of the continuous outcome.

R> data(“parkinson”)
R> parkinson
s.id  t.id    mean     sd    n
1       1      1   -1.22  3.70   54
2       1      3   -1.53  4.28   95
3       2      1   -0.70  3.70 172
4       2      2   -2.40  3.40 173
5       3      1   -0.30  4.40   76
6       3      2   -2.60  4.30   71
7       3      4   -1.20  4.30   81
8       4      3   -0.24  3.00 128
9       4      4   -0.59  3.00   72
10     5      3   -0.73  3.00   80
11     5      4   -0.18  3.00   46
12     6      4   -2.20  2.31 137
13     6      5   -2.50  2.18 131
14     7      4   -1.80  2.48 154
15     7      5   -2.10  2.99 143

The datasets dietaryfat and diabetes serve as examples for models (3) and (4). This article uses diabetes to illustrate estimation of treatment-specific rates and rate ratios. This dataset was analyzed by Elliott and Meyer (2007) to assess the effects of antihypertensive agents on incident diabetes, and includes the follow-up times (in years) for each study. Twenty-two clinical studies are included, covering six different treatments: 1) diuretic; 2) placebo; 3) beta blocker (BB); 4) calcium-channel blocker (CCB); 5) angiotensin-converting-enzyme inhibitor (ACEI); and 6) angiotensin-receptor blocker (ARB). Users can apply data() in R to load the datasets dietaryfat and diabetes; we do not display the detailed dataset.

Note that NA is not allowed in the dataset for the package pcnetmeta, because the published articles collected in a network meta-analysis typically report all summary results (such as both mean and variance for continuous outcomes). Also, each row in the dataset represents one treatment group in a study, so a single-arm study is straightforwardly input as a single row in the dataset for analysis using the arm-based models in the pcnetmeta package.

3.2. Plotting the network

The function nma.networkplot() in package pcnetmeta provides a visual overview of treatment comparisons in network datasets. Calling this function produces a network graph in an R plot window. Each vertex in the network plot represents a treatment and each edge between two nodes stands for a direct comparison between the corresponding two treatments. The usage of the function is as follows:

R> args(nma.networkplot)
function (s.id, t.id, data, title = “”, trtname, alphabetic = TRUE,
       weight.edge = TRUE, adjust.thick = 5, weight.node = TRUE,
       adjust.node.size = 10, node.col = “orange”, edge.col = “black”,
       text.cex = 1, adjust.figsizex = 1.1, adjust.figsizey = 1.1)
NULL

Users need to input s.id and t.id for study and treatment IDs respectively. The argument title gives the graph title, and trtname specifies the treatment names. If trtname is not specified, the treatment IDs given in t.id are used. The argument alphabetic is a logical value indicating whether to sort the treatment nodes alphabetically in the clockwise direction according to the treatment names; if alphabetic = FALSE, the nodes are sorted according to treatment IDs ( t.id). The logical argument weight.edge = TRUE causes the edge thickness to be drawn proportional to the number of direct treatment comparisons; weight.node = TRUE causes the node size to be proportional to the number of direct comparisons which contain that treatment node.

The following code produces network plots for the datasets smoke, parkinson, and diabetes respectively.

R> data(“smoke”)
R> nma.networkplot(s.id, t.id, data = smoke,
+      trtname = c(“NC”, “SH”, “IC”, “GC”))
R> data(“parkinson”)
R> nma.networkplot(s.id, t.id, data = parkinson)
R> data(“diabetes”)
R> nma.networkplot(s.id, t.id, data = diabetes,
     trtname = c(“Diuretic”, “Placebo”, “BB”, “CCB”, “ACEI”, “ARB”))

Figure 1 shows the resulting graphs. In the left panel for the smoking cessation data, every pair of treatment nodes is connected by an edge, so all pairs of treatments are directly compared. For the dataset parkinson in Figure 1b, we did not specify treatment names, so the function used the treatment IDs from 1 to 5 as the names. Its network plot shows no edge between some pairs of treatments, e.g., Treatments 2 and 3. This means that no study directly compares Treatments 2 and 3. As a result, if the aim is to compare the effects of Treatments 2 and 3, only indirect evidence is available, e.g., from the comparisons Treatments 2 vs. 1 and 3 vs. 1. Figure 1c is the network plot for the dataset diabetes; all pairs of treatments are directly compared except ACEI and ARB.

Figure 1.

Network plots generated by the function nma.networkplot().

3.3. Performing arm-based network meta-analysis

The major functions in package pcnetmeta are nma.ab.bin(), nma.ab.cont(), nma.ab.py(), and nma.ab.followup(), which perform arm-based network meta-analysis for different types of data using the models introduced in Section 2. In particular, nma.ab.bin() analyzes binary outcomes, while nma.ab.cont() is used for continuous outcomes. These two functions are based on models (1) and (2), respectively. Functions nma.ab.py() and nma.ab.followup() can be used when exposure times or follow-up times are available, and are based on the models (3) and (4), respectively. The commands in each of the following subsections may take around 5{20 minutes on an Intel 2.60 GHz processor. The actual runtime depends on the complexity of the treatment network and the user’s processor.

Function nma.ab.bin() for binary outcomes

The arguments of the function nma.ab.bin() are as follows

R> args(nma.ab.bin)
function (s.id, t.id, event.n, total.n, data, trtname, param = c(“AR”,
       “LOR”, “LRR”, “RD”, “rank.prob”), model = “het_cor”, prior.type,
       a = 0.001, b = 0.001, c = 10, higher.better = FALSE, digits = 4,
       n.adapt = 5000, n.iter = 1e+05, n.burnin = floor(n.iter/2),
       n.chains = 3, n.thin = max(1, floor((n.iter - n.burnin)/1e+05)),
       conv.diag = FALSE, trace = NULL, dic = FALSE, postdens = FALSE,
       mcmc.samples = FALSE)
NULL

As in nma.networkplot(), the arguments s.id and t.id are numeric or character vectors indicating study and treatment IDs. Users also specify each study’s number of events and participants using event.n and total.n respectively. The argument model can be specified as “het_cor”, “het_eqcor”, or “hom_eqcor”, which corresponds to the models described in Section 2.4. When model = “het_cor” (the default), users can specify prior.type = “invwishart” (the default) to assign an inverse-Wishart prior to the variance-covariance matrix of random effects. Alternatively, by assigning prior.type = “chol”, the separation strategy by Cholesky decomposition is used for the variance-covariance matrix, and uniform priors U(0, c) are assigned to the standard deviations and vague priors are assigned to the correlation components (Barnard et al. 2000; Lu and Ades 2009; Wei and Higgins 2013). When “het_eqcor” and “hom_eqcor” are used, the correlation matrix of the random effects has an exchangeable correlation structure. For the “hom_eqcor” and “het_eqcor” models, two types of priors for the random-effect variances can be used. A popular prior for variances is inverse-Gamma(ε, ε) where ε can be set to a low value (e.g., 0.001) (Spiegelhalter et al. 2003). However, the posterior may be sensitive to the choice of ε, and a uniform prior for standard deviation is preferred in some cases (Gelman 2006). Users choose the prior by specifying prior.type = “unif” (the default) or “invgamma”, and the prior parameters a, b, and c. If inverse-Gamma priors are used, inverse-Gamma(a, b) with density $\frac{b^a}{\mathrm{\Gamma }(a)}{x}^{-a-1}{e}^{-b/x}$ is assigned to the variances of the random effects; if uniform priors are used, U(0, c) is assigned to the standard deviations. The default prior parameters are a = b = 0.001 and c = 10.

The argument param is a character string vector which indicates the effect sizes to be estimated. As in Section 2.1, param can include absolute risk ( “AR”), odds ratio ( “OR”), log odds ratio ( “LOR”), risk ratio ( “RR”), log risk ratio ( “LRR”), risk difference ( “RD”). In addition, researchers may be interested in treatment ranks (Salanti et al. 2011). Users can estimate the rank probabilities of different treatments (i.e., probabilities of the treatment having ranks 1, 2, …, K) by adding “rank.prob” to the argument param. When “rank.prob” is added, users need to specify the logical argument higher.better; TRUE indicates that a higher event probability implies a better treatment. For example, the event in the dataset smoke is smoking cessation, and a higher smoking cessation probability implies a better treatment. Many outcomes in medical studies are the events of developing a disease (e.g., Thijs et al. 2008), in which case a better treatment should lead to a lower event probability.

The arguments n.adapt, n.iter, n.burnin, n.chains, and n.thin control the MCMC algorithm run by rjags (Plummer 2016). The argument n.adapt is the number of iterations for adaptation (the default is 5,000); this is used to maximize the sampling efficiency. The argument n.iter determines the number of iterations in each MCMC chain, and n.burnin is the number of burn-in iterations at the beginning of each chain which are discarded. The argument n.chains is the number of MCMC chains; the default is 3. Additionally, n.thin is the thinning rate for MCMC chains, which is used to save memory and computation time if n.iter is large. For example, if n.iter is 10⁶ and n.thin is 10, then only one sample would be kept in every 10 samples in each chains, and the remaining number of iterations is 10⁵.

The argument conv.diag specifies whether to compute potential scale reduction factors (PSRFs) proposed by Gelman and Rubin (1992) for convergence diagnostics. The argument trace is a character string vector which can be chosen from the elements specified in param except for “rank.prob”. Trace plots of the specified effect sizes are saved in users’ current working directory as .png files. A trace plot is a plot of the sampled parameter estimates at each iteration against iteration number. Both PSRFs and trace plots can be used to examine whether the MCMC chains are drawn from stationary distributions. Finally, if dic = TRUE, the function will provide the deviance information criterion (DIC) statistic proposed by Spiegelhalter et al. (2002); conventionally the model with smallest DIC is considered the best among the candidate models. The posterior density plot of treatment-specific effect sizes can be obtained as a .pdf file by setting postdens = TRUE.

The function nma.ab.bin() returns a list with effect size estimates, which lists the posterior mean, standard deviation, median, and a 95% credible interval (CI) with 2.5% and 97.5% quantiles as the lower and upper bounds.

Here is an example to demonstrate the function’s usage. We called the function nma.ab.bin() on the dataset smoke as follows:

R> data(“smoke”)
R> set.seed(12345)
R> smoke.out <- nma.ab.bin(s.id, t.id, r, n, data = smoke, trtname = c(“NC”,
+     “SH”, “IC”, “GC”), param = c(“AR”, “OR”, “RR”, “LOR”, “LRR”, “RD”,
+     “rank.prob”), model = “het_cor”, higher.better = TRUE, digits = 3,
+     n.adapt = 10000, n.iter = 200000, n.thin = 1, conv.diag = TRUE, 
+     dic = TRUE, trace = “LOR”, postdens = TRUE)

The following messages were outputted:

Start running MCMC…
Compiling model graph
       Resolving undeclared variables
       Allocating nodes
Graph information:
      Observed stochastic nodes: 50
      Unobserved stochastic nodes: 29
      Total graph size: 646
Initializing model
   |++++++++++++++++++++++++++++++++++++++++++++++++++| 100%
   |**************************************************| 100%
   |**************************************************| 100%
Start calculating MCMC convergence diagnostic statistics…
Start calculating deviance information criterion statistics…
   |**************************************************| 100%
Start saving trace plots…
Start saving posterior density plot for absolute risk…

When a JAGS model is compiled, it may require an initial sampling phase during which the samplers adapt their behavior to maximize their efficiency (e.g., a Metropolis-Hastings random walk algorithm may change its step size) (Plummer 2016). The warning of “adaptation incomplete” may occasionally occur if the number of iterations for the adaptation process (i.e., the argument n.adapt) is not sufficient, so the MCMC algorithm may not achieve the maximum efficiency. This warning generally has little impact on the posterior estimates of the treatment effects. To avoid this warning, users may increase n.adapt.

The results are saved in the object smoke.out, a list containing AbsoluteRisk, OddsRatio, LogOddsRatio, RelativeRisk, LogRelativeRisk, RiskDifference, TrtRankProb, and DIC. We can use these effect size names to display the corresponding estimates. For example, the estimates of absolute risks (posterior mean and standard deviation, and posterior median and 95% credible interval) can be displayed as

R> smoke.out$AbsoluteRisk
$Mean_SD
       Mean (SD)
NC 0.082 (0.014)
SH  0.166 (0.053)
IC   0.183 (0.028)
GC 0.229 (0.061)
$Median_CI
       Median (95% CI)
NC 0.081 (0.058, 0.114)
SH  0.158 (0.085, 0.293)
IC   0.181 (0.134, 0.243)
GC 0.222 (0.127, 0.369)

The argument digits in the function nma.ab.bin() can be used to change the number of digits to the right of the decimal point. Here, we used digits = 3. Each list element in the object smoke.out consists of two sublists: Mean_SD contains posterior sample means with sample standard deviations; Median_CI contains posterior medians with 95% CIs. For example, users can output the medians and 95% CIs for the log odds ratio as follows:

R> smoke.out$LogOddsRatio$Median_CI
       NC                               SH                                IC
NC --                                  -0.752 (-1.610, 0.032)  -0.917 (-1.420, -0.436)
SH 0.752 (-0.032, 1.610)    --                                 -0.167 (-0.944, 0.686)
IC  0.917 (0.436, 1.420)     0.167 (-0.686, 0.944)  --
GC 1.170 (0.399, 1.970)    0.424 (-0.637, 1.420)  0.252 (-0.505, 1.050)
       GC
NC -1.170 (-1.970, -0.399)
SH  -0.424 (-1.420, 0.637)
IC   -0.252 (-1.050, 0.505)
GC --

Since the log odds ratio is a relative effect size comparing a pair of treatments, the output estimates are displayed in a K × K matrix, where K is the number of treatments. In this example, K = 4. The element in the ith row and jth column is the estimated log odds ratio of treatment i compared to treatment j. To statistically test the difference between treatments, one can examine whether the effect size under the null hypothesis (i.e., the two treatments do not differ) is within the corresponding 95% CI; under the null hypothesis, ORs and RRs are 1, while LORs, LRRs, and RDs are 0. From the output, the 95% CI of the log odds ratio for SH vs. NC is (−0.032, 1.610) which contains 0; therefore, there is not sufficient evidence to reject the null hypothesis. However, the 95% CI of the log odds ratio comparing GC to NC is (0.399, 1.970), which does not contain 0; therefore, these two treatments differ significantly.

Also, in this example, we included “rank.prob” in the argument param to estimate treatment rank probabilities, and dic was specified as TRUE to calculate the DIC statistic. Recall that for the smoking cessation dataset, a higher event probability implies a better treatment, so we specified the argument higher.better as TRUE. Users can access the treatment rank probabilities and DIC statistics using

R> smoke.out$TrtRankProb
       rank1   rank2   rank3   rank4
NC 0.0000 0.0002 0.0319 0.9680
SH 0.1520 0.2390 0.5780  0.0300
IC  0.1860 0.5420 0.2720  0.0002
GC 0.6620 0.2190 0.1180  0.0020
R> smoke.out$DIC
D.bar 278.6972
pD       44.1862
DIC  322.8834

From the output, treatment GC has the highest probability of being the best treatment (66.2%). As for the DIC statistic, D.bar is the posterior expectation of the deviance, which reflects the model fit; it is usually lower when more parameters are used in the model. However, complex models may lead to overfitting. To balance between the number of parameters and fitting effects, pD is used to penalize D.bar; it reflects the number of effective parameters used in the model. DIC is the penalized deviance, calculated as the sum of D.bar and pD; a model with smaller DIC is preferred.

Trace plots of LOR are generated because trace = “LOR” was specified. Figure 2 shows the trace plots of the LOR comparing IC and GC. Since we used the default n.chains = 3, three trace plots are drawn. Each trace plot shows evidence that the posterior samples of LOR are drawn from the stationary distribution.

Figure 2.

Trace plots generated by R function nma.ab.bin() for the log odds ratio comparing IC and GC in the smoking cessation data.

A posterior density plot (Figure 3a) for treatment-specific absolute risks is also generated by specifying postdens = TRUE. This density plot is smoothed by the R function density(). This plot shows visualized treatment effects, and we may also evaluate treatment differences. For example, NC clearly has lower event probability than IC and GC, and its posterior density only overlaps with the densities of IC and GC in tiny regions.

Figure 3.

Posterior density plots generated by the functions in package pcnetmeta.

Function nma.ab.cont() for continuous outcomes

For continuous outcomes, the arguments of nma.ab.cont() are mostly similar to those of nma.ab.bin(). The major difference is that users need to specify the summaries of continuous outcomes (sample means and standard deviations) for the arguments mean and sd in nma.ab.cont(). Also, the effect sizes to be estimated include continuous treatment-specific effects and their differences. Users can specify the argument param as "mu" to estimate treatment-specific effects and "diff" to estimate effect differences. Also, "rank.prob" can be included in param to estimate treatment rank probabilities. The network dataset parkinson is used as an example:

R> data("parkinson")
R> set.seed(12345)
R> parkinson.out <- nma.ab.cont(s.id, t.id, mean, sd, n, data = parkinson,
+     model = "hom_eqcor", prior.type = "unif", digits = 3, n.adapt = 10000,
+     n.iter = 100000, n.thin = 1, conv.diag = TRUE, trace = "mu",
+     postdens = TRUE)

In this example, we used the model "hom_eqcor", which assumes an exchangeable correlation structure for the correlations between treatment effects. We display the medians and the corresponding 95% CIs for treatment-specific effects and effect differences as follows.

R> parkinson.out$TrtEffect$Median_CI
        Median (95% CI)
Trt1 -0.717 (-1.590, 0.172)
Trt2 -2.520 (-3.540, -1.520)
Trt3 -0.984 (-1.840, -0.147)
Trt4 -1.340 (-1.970, -0.614)
Trt5 -1.930 (-2.930, -0.951)
R> parkinson.out$EffectDiff$Median_CI
       Trt1                                Trt2                            Trt3
Trt1 --                                   1.810 (0.762, 2.860)  0.274 (-0.835, 1.360)
Trt2 -1.810 (-2.860, -0.762) --                                -1.530 (-2.770, -0.341)
Trt3 -0.274 (-1.360, 0.835)  1.530 (0.341, 2.770)   --
Trt4 -0.618 (-1.570, 0.400)  1.190 (0.131, 2.330)   -0.338 (-1.240, 0.549)
Trt5 -1.210 (-2.480, -0.016) 0.602 (-0.780, 1.910) -0.910 (-2.290, 0.218)
        Trt4                               Trt5
Trt1 0.618 (-0.400, 1.570)    1.210 (0.016, 2.480)
Trt2 -1.190 (-2.330, -0.131) -0.602 (-1.910, 0.780)
Trt3 0.338 (-0.549, 1.240)    0.910 (-0.218, 2.290)
Trt4 --                                   0.552 (-0.215, 1.730)
Trt5 -0.552 (-1.730, 0.215)  --

The effect difference in the ith row and jth column is calculated as the effect of treatment i minus that of treatment j. To statistically test the difference between two treatments, users may check whether 0 is within the corresponding 95% CI for the effect difference. For instance, from the output, the 95% CI for the effect difference between Treatments 1 and 2 is (−2.860, 0.762), which does not contain 0. Therefore, Treatments 1 and 2 differ significantly. The posterior density plot (Figure 3b) for treatment-specific effects is obtained by specifying postdens = TRUE. From the density plot, the overlap region of densities for Treatments 1 and 2 is fairly small, and this supports the above conclusion.

Functions nma.ab.py() and nma.ab.followup() for count datasets

For models (3) and (4), treatment effects can be related to the follow-up times of participants. Some studies report the total exposure times in person-years (e.g., Hooperet al. 2000) for each treatment group, and some report the mean follow-up time for each study (e.g., Psaty et al. 2003; Elliott and Meyer 2007). The functions nma.ab.py() and nma.ab.followup() can be used for these two types of datasets, corresponding to models (3) and (4), respectively. In these two functions, the argument param can include “rate” (treatment-specific rate), “lograte” (log rate), “ratio” (rate ratio), “logratio” (log rate ratio), and “rank.prob” (treatment rank probabilities). Since the two functions are similar, this article focuses on using the dataset diabetes to illustrate the usage and output of nma.ab.followup(). The function is called as follows:

R> data(“diabetes”)
R> set.seed(12345)
R> diabetes.out <- nma.ab.followup(s.id, t.id, r, n, folup, data = diabetes,
+     trtname = c(“Diuretic”, “Placebo”, “BB”, “CCB”, “ACEI”, “ARB”),
+     model = “het_cor”, digits = 3, n.adapt = 10000, n.iter = 200000, 
+     n.thin = 2, conv.diag = TRUE, trace = “lograte”, postdens = TRUE)

Log rate ratio (the treatment in row compared to that in column) for each pair of treatments can be displayed as follows:

R> diabetes.out$LogRateRatio$Median_CI
              Diuretic  Placebo                                                              BB
Diuretic --            0.090 (-0.662, 0.801)                                         0.102 (-0.565, 0.855)
Placebo                -0.090 (-0.801, 0.662)  --                                   0.014 (-0.621, 0.740)
BB                       -0.102 (-0.855, 0.565)   -0.014 (-0.740, 0.621)  --
CCB                    -0.163 (-0.911, 0.487)   -0.075 (-0.792, 0.545)  -0.065 (-0.573, 0.465)
ACEI                   -0.266 (-0.979, 0.397)   -0.176 (-0.796, 0.358)  -0.164 (-0.721, 0.421)
ARB                    -0.292 (-1.110, 0.609)   -0.205 (-1.000, 0.663)  -0.192 (-0.896, 0.658)
                            CCB                              ACEI                            ARB
Diuretic                0.163 (-0.487, 0.911)   0.266 (-0.397, 0.979)   0.292 (-0.609, 1.110)
Placebo                0.075 (-0.545, 0.792)   0.176 (-0.358, 0.796)   0.205 (-0.663, 1.000)
BB                        0.065 (-0.465, 0.573)  0.164 (-0.421, 0.721)   0.192 (-0.658, 0.896)
CCB                     --                                 0.100 (-0.470, 0.661)   0.126 (-0.667, 0.782)
ACEI                    -0.100 (-0.661, 0.470) --                                 0.028 (-0.815, 0.717)
ARB                     -0.126 (-0.782, 0.667) -0.028 (-0.717, 0.815) --

From the output, all of the 95% CIs contain 0, so the difference between any pair of treatments is not significant. The posterior density plot for log rates is shown in Figure 3c, which also indicates that treatment-specific density curves do not differ much.

3.4. Plotting 95% credible intervals

When presenting network meta-analysis results, it is helpful to report the 95% CIs for effect sizes of interest. The package pcnetmeta provides functions absolute.plot() and contrast.plot() to draw the 95% CIs for treatment-specific and relative effect sizes, respectively. Users can simply call these functions on objects obtained from nma.ab.bin(), nma.ab.cont(), nma.ab.py(), and nma.ab.followup(). Here, we use the three objects obtained from previous sections as examples to generate treatment-specific 95% CI plots:

R> absolute.plot(smoke.out, width = 5, height = 1.5)
R> absolute.plot(parkinson.out, width = 5, height = 1.5)
R> absolute.plot(diabetes.out, width = 8, height = 2.5)

The generated plots are shown in the left panels of Figure 4. Contrast plots showing comparisons to a reference treatment can be generated by the following code:

Figure 4.

The left three panels show the plots for treatment-specific (absolute) effects generated by the function absolute.plot(); the right three panels show the plots for relative effects generated by the function contrast.plot().

R> contrast.plot(smoke.out, reference = “NC”, width = 5, height = 1.5)
R> contrast.plot(parkinson.out, reference = “Trt1”, width = 5, height = 1.5)
R> contrast.plot(diabetes.out, reference = “Placebo”,
+     width = 8, height = 2.5)

The argument reference specifies the reference treatment to be compared against. The right panels of Figure 4 display the contrast plots.

Figure 4a shows the treatment-specific 95% CI plot for the smoking cessation data. Clearly, the 95% CIs of IC and GC do not overlap with the 95% CI of NC. This indicates significant differences between IC vs. NC and GC vs. NC. This can be confirmed by Figure 4b: The 95% CIs of ORs comparing IC vs. NC and GC vs. NC do not intersect with the vertical line at OR = 1. Figure 4c is the treatment-specific 95% CI plot for the Parkinson’s disease data. Note that the 95% CIs for Treatments 1 and 2 overlap only in a tiny region. Correspondingly, the 95% CI for the effect difference between the two treatments in Figure 4d does not intersect with the vertical line at 0. Finally, Figures 4e and 4f show the 95% CI plots for treatment-specific and relative effects for the diabetes dataset. All of the treatment-specific 95% CIs have large overlap with other CIs. The 95% CIs of the rate ratios compared with placebo intersect with the vertical line at 1; therefore, the active treatments do not differ significantly from placebo.

3.5. Plotting treatment rank probabilities

Function rank.prob() is used to graph the probabilities of each treatment having each of the different possible ranks among the treatments. Users can call this function for the objects smoke.out, parkinson.out, and diabetes.out as follows:

R> rank.prob(smoke.out, cex.axis = 2, cex.lab = 2)
R> rank.prob(parkinson.out, cex.axis = 2, cex.lab = 2)
R> rank.prob(diabetes.out, cex.axis = 1, cex.lab = 2)

Figure 5 shows the plots of treatment rank probabilities. In the plots, each vertical bar represents probabilities that a specific treatment has different possible ranks. A darker area indicates the probability of being a higher rank, thus the black areas show the probabilities of being the best treatment. Therefore, from Figures 5a and 5b, treatments GC and Trt2 have much higher probabilities of being the best treatment, compared with other treatments in their respective studies. Figure 5c shows the rank probabilities plot for the dataset diabetes. Treatment ARB has highest probability of being the best treatment, although the probability for Treatment ACEI is close to the highest probability, so ARB and ACEI do not differ much.

Figure 5.

Plots of treatment rank probabilities generated by the function rank.prob().

4. Discussion

This article presents an overview of the R package pcnetmeta. Arm-based models are introduced to demonstrate the underlying methods of the functions. Practical usage of various functions is illustrated with examples of real network meta-analyses. Also, the package provides several plots for interpretation of network meta-analysis outputs.

MCMC convergence diagnostics have been extensively discussed in the literature (Cowles and Carlin 1996; Kass et al. 1998). The PSRFs and trace plots provided by the package pcnetmeta are used to examine whether the MCMC chains are drawn from stationary distributions; however, additional techniques are required to determine the effective sample size for adequate convergence (Robert and Casella 2004, p. 500). By specifying the argument mcmc.samples = TRUE in nma.ab.bin(), nma.ab.cont(), nma.ab.followup(), and nma.ab.py(), the MCMC posterior samples are saved in the output objects. Functions in other packages developed for MCMC convergence and sample-size adequacy, such as the R package mcmcse (Flegal and Hughes 2012), can be called for these posterior samples.

The current version of pcnetmeta does not detect inconsistency in arm-based network meta-analysis. Future work would add functions for network consistency assessment (Zhao et al. 2016). Moreover, both the contrast-based and arm-based methods can be extended to handle individual patient data (IPD) (Jansen 2012; Saramago et al. 2014; Hong et al. 2015; Veroniki et al. 2015); a future update of the package may include functions for IPD.