A Bayesian Multivariate Meta-Analysis of Prevalence Data

Abstract

When conducting a meta-analysis involving prevalence data for an outcome with several subtypes, each of them is typically analyzed separately using a univariate meta-analysis model. Recently, multivariate meta-analysis models have been shown to correspond to a decrease in bias and variance for multiple correlated outcomes compared to univariate meta-analysis, when some studies only report a subset of the outcomes. In this article, we propose a novel Bayesian multivariate random effects model to account for the natural constraint that the prevalence of any given subtype cannot be larger than that of the overall prevalence. Extensive simulation studies show that this new model can reduce bias and variance when estimating subtype prevalences in the presence of missing data, compared to standard univariate and multivariate random effects models. The data from a rapid review on occupation and lower urinary tract symptoms by the Prevention of Lower Urinary Tract Symptoms (PLUS) Research Consortium are analyzed as a case study to estimate the prevalence of urinary incontinence and several incontinence subtypes among women in suspected high risk work environments.

1 |. INTRODUCTION

Meta-analysis plays an important role in synthesizing evidence from multiple sources, supporting the recent rapid growth of “evidence-based medicine” ¹. Multivariate and network meta-analysis (NMA) methods have been developed for meta-analyses of data consisting of multiple outcomes, multiple treatments, or multiple diagnostic tests ^2–13. NMA uses both indirect and direct comparisons of multiple treatments within a network, while multivariate meta-analysis allows for the joint analysis of multiple outcomes by incorporating information about their correlations ^5;9. These models are therefore able to use information normally unavailable when each treatment or outcome is analyzed separately, a statistical concept known as “borrowing strength” that is particularly useful in the presence of missing data ^4;14;15. For example, Williams and Bürkner ¹⁶ jointly modeled the effects of intranasal oxytocin on multiple symptoms of schizophrenia in a Bayesian multivariate meta-analysis, resulting in increased precision compared to previous analyses that modeled symptoms separately.

Multivariate meta-analysis models have also been increasingly used in the evaluation of diagnostic tests, as they allow for the joint modeling of accuracy indices such as sensitivity and specificity ^17–21. The use of NMA in meta-analyses of clinical trials assessing multiple treatments has also increased sharply over the past decade ^9;22;23. However, many practitioners conducting systematic reviews involving multiple correlated outcomes of interest still analyze these individually using separate univariate models, thus ignoring any correlations between outcomes. Riley ¹⁵ suggested that reasons for this hesitancy may include “tradition, the increased complexity of the multivariate approach, the need for speciali[zed] statistical software and a lack of understanding of the consequences of ignoring correlation in meta-analysis”, which are all still relevant issues.

This widespread use of separate univariate models is especially prevalent in the context of observational data. For instance, we found only three examples of multivariate meta-analysis of observational data in the literature: 1) Fawcett et al. ²⁴ presented a multivariate method that used data on prevalences of individual disorders in order to estimate their overall prevalence. To our knowledge, this is the only study to perform a multivariate meta-analysis of prevalence and incidence data. 2) The Fibrinogen Studies Collaboration ²⁵ used a bivariate random effects meta-analysis in order to jointly model partially and fully adjusted estimates of the association between fibrinogen level and incidence of coronary heart disease, and 3) Lin and Chu ²⁶ proposed a Bayesian multivariate meta-analysis simultaneously analyzing multiple factors. While meta-analyses are most frequently conducted in order to estimate effect sizes such as odds ratios (OR’s), risk differences, or mean differences, they can also be used to estimate the pooled disease frequency such as incidence rates and prevalence proportions ²⁷. This may include multiple related outcomes consisting of an overall prevalence and several subtypes of the measured outcome. For example, Rona et al. ²⁸ conducted univariate meta-analyses estimating the prevalence of any food allergy as well as the prevalences of specific types such as allergies to peanuts and shellfish. Similarly, Williams et al. ²⁹ separately estimated the prevalences of typical autism and all autism spectrum disorders (ASD). For this type of data, it is reasonable to expect that the prevalence of any given subtype will be both correlated with and constrained by the overall prevalence. Therefore, these outcomes are not independent. Additionally, the subtypes measured in observational data may be particularly susceptible to outcome reporting bias (ORB). If a study is smaller or performed in a population in which the overall prevalence is expected to be lower, investigators may be more likely to only report the overall outcome ¹⁰.

Multivariate meta-analysis models have been shown to be effective in reducing ORB when there is correlation between outcomes, in addition to providing more precise estimates in the presence of missing data ^{9;10;14;15;30}. Modeling multiple correlated prevalences using univariate models ignores studies in which each particular prevalence is not reported. This can result in biased estimates if any of the studies are subject to ORB. Instead, multivariate models allow us to “borrow” information on missing observations across outcomes using the within-study correlations ⁹. However, one reason that practitioners may be reluctant to use multivariate meta-analysis methods, is that these within-study correlations have to be estimated ⁶. In this paper, we address that issue through the use of a Bayesian multivariate meta-analysis framework. This framework also gives us greater flexibility in parameterizing the multivariate random effects model. Prevalences of individual subtypes are subject to the natural constraint that they cannot be larger than the overall prevalence. While Trikalinos et al. ¹⁰ jointly modeled multiple categorical outcomes that are mutually exclusive or subsets of each other using a multinomial distribution, our method differs in that the multiple subtypes modeled need not imply a set of mutually exclusive categories. We introduce a case study as a motivating example in Section 2. We then present fully Bayesian univariate and multivariate models for estimating the prevalence of each outcome in Section 3, including a novel parameterization of the multivariate random effects model that accounts for the natural constraints in the data, thereby incorporating additional information into the model. We then compare these three different approaches in simulation studies (Section 4) and when applied to the case study (Section 5). Section 5.2 presents a sensitivity analysis of the missing at random (MAR) assumption for the case study. Section 6 gives a brief discussion of these results.

2 |. A MOTIVATING STUDY

Recently, members of the PLUS research consortium ³¹ conducted a rapid review of studies reporting lower urinary tract symptoms in women in suspected high risk working environments ³². Of the studies collected, 26 report the overall number of women in the study as well as the number experiencing any form of urinary incontinence (UI). Additionally, many studies provide data on two subtypes of UI: stress urinary incontinence (SUI) and urgency urinary incontinence (UUI). We present the data in Table W.1, which was not included in the original paper by Markland et al. ³² . Let y_i0, y_i1, and y_i2 denote the number of women experiencing any UI, SUI and UUI in study i , respectively, and let n_i denote the total number of women in each study. While all studies provide counts for the total number of women experiencing any urinary incontinence, many studies do not report one or both subtype counts. We note that y_i0 ≥ y_i1; y_i2 and the counts for each subtype do not necessarily sum to the overall UI count when they are all reported, as they are not mutually exclusive. Therefore a model based on a multinomial distribution such as that used by Trikalinos et al. ¹⁰ would not be appropriate. Our goal is to estimate the population-averaged marginal prevalence of urinary incontinence (π₀), as well as that of each subtype (π₁, π₂).

Currently, standard practice would be to estimate each prevalence individually, using univariate random-effects models. However, it is reasonable to expect that the prevalences of the different subtypes of urinary incontinence outcomes will be correlated with one another, in addition to being correlated with the overall prevalence. This can allow us to use data from non-missing outcomes to address ORB and increase the precision of our pooled subtype estimates. Therefore, we first fit a Bayesian multivariate random effects model in order to incorporate information about the correlations between the UI outcomes (π₀, π₁, π₂). We then compare these results to those found using a novel parameterization of the model that incorporates the natural constraint that π₀ ≥ π₁, π₂. Finally, we compare the results found using both of these models to those found using separate univariate random effects models.

3 |. METHODOLOGY

3.1 |. A univariate random effects model

As mentioned previously, one way to model the overall and subtype prevalences of a condition, is to model each using separate univariate random effects models ³³. In using random effects models, we assume that the true prevalences vary across studies. Let π_ij be the probability of having the jth outcome in study i ∈ {1, …, N }, where j ∈ {0, 1, …, J }, and let n_i be the number of participants in study i . Here, π_i0 refers to the overall prevalence in study i , while π_i1, …, π_iJ refer to the J subtype prevalences. Let S_i be the set of outcomes that are reported in study i , and D_i = {(y_ij , n_i), j ∈ S_i} denote the available data from study i . Let ϕ(z) and ϕ(·) denote the probability density function and cumulative density function (CDF) of the standard normal distribution, respectively. If we use a probit link function to separately model the number of cases y_ij for each of the J + 1 total outcomes, we have

$$y_{ij}~\text{Binomial}\left(n_j,{\pi }_{ij}\right),{\varphi }^{-1}\left({\pi }_{ij}\right)={\mu }_j+v_{ij},v_{ij}~N\left(0,{\sigma }_j^2\right),j\in S_i,i=1,\dots ,N,$$ (1)

where µ_j is the fixed effect for each outcome, while v_ij is the random effect for outcome j within study i . Therefore, σ_j describes the between-study variability in outcome j . If we assume that conditional on the π_ij ’s, the y_ij ’s are independent, this gives us the following observed data likelihood function combining the J + 1 independent random effects models:

$$L_1\propto \prod _{i=1}^N\prod _{j\in S_i}{\int }_{-\infty }^{\infty }{\left[\varphi \left({\mu }_j+{\sigma }_{j}z\right)\right]}^{y_i}{\left[1-\varphi \left({\mu }_j+{\sigma }_{j}z\right)\right]}^{n_i-y_i}\varphi \left(z\right)dz.$$ (2)

The population averaged prevalence for each outcome can be estimated as ${\pi }_j=E\left[{\pi }_j|{\mu }_j,{\sigma }_j\right]=\varphi {\left(\frac{{\mu }_j}{\sqrt{1+{\sigma }_j^2}}\right)}^{13},j\in \left\{0,1,\dots ,J\right\}$.

3.2 |. A multivariate random effects model

We can extend this univariate approach to jointly estimate the prevalences for multiple outcomes, using a multivariate Bayesian random effects model. This incorporates the correlations between outcomes to improve estimation when missing data are present, otherwise known as borrowing strength. Zhang et al. ¹³ present a hierarchical Bayesian random effects model with a probit-link in the context of “arm-based” network meta-analysis. In “arm-based” NMA, the focus is on calculating the event probabilities for each treatment arm ^13;34;35. Therefore, we adapt this framework in order to estimate the event probabilities for our overall outcome and each of the subtypes, treating them as separate “arms.”

We first let each y_ij be independently binomially distributed, conditional on π_ij . We then let the probit-transformed (π_i0, …, π_iJ)^T follow a multivariate normal distribution:

$$\begin{gathered} y_{ij}~\text{Binomial}\left(n_i,{\pi }_{ij}\right) \\ {\left[{\varphi }^{-1}\left({\pi }_{i0}\right),{\varphi }^{-1}\left({\pi }_{i1}\right),\dots ,{\varphi }^{-1}\left({\pi }_{iJ}\right)\right]}^T={\left({\mu }_0+v_{i0},{\mu }_1+v_{i1},\dots ,{\mu }_j+v_{iJ}\right)}^T, \\ {\mathbf{v}}_i^T={\left(v_{i0},v_{i1}\dots ,v_{iJ}\right)}^T~MVN\left(0,{\Sigma }_{J+1}\right),j\in S_i, \end{gathered}$$ (3)

where Σ_J+1 = diag (σ)R_J+1diag (σ). R_J+1 is the within-study correlation matrix and the ${\sigma }_j^2$ terms capture the between study variation in each outcome. Let L_ij denote the conditional likelihood given v_ij , defined as

$$L_{ij}\left(y_{ij};{\mu }_j,v_{ij}\right)={\left[\varphi \left({\mu }_j+v_{ij}\right)\right]}^{y_{ij}}{\left[1-\varphi \left({\mu }_j+v_{ij}\right)\right]}^{n_i-y_{ij}}.$$ (4)

Then the observed data likelihood function can be written as follows:

$$L_2\propto \prod _{i=1}^N{\int }_{{ℝ}^j}\left(\prod _{j\in S_i}{L}_{ij}\left(y_{ij};{\mu }_k,v_{ij}\right)\right)\frac{exp\left(-\frac{1}{2}{\mathbf{v}}_i^T{\sum }_{J+1}^{-1}{\mathbf{v}}_i\right)}{{\left(2\pi \right)}^{\left(J+1\right)/2}{\left|{\sum }_{J+1}\right|}^{1/2}}d{\mathbf{v}}_i,$$ (5)

where ${ℝ}^J$ is the J-dimensional real space. The population averaged prevalence for each outcome can be estimated using the same method used in the univariate model as ${\pi }_j=\varphi \left(\frac{{\mu }_j}{\sqrt{1+{\sigma }_j^2}}\right),j\in \left\{0,1,\mathrm{..},J\right\}$.

3.3 |. A new multivariate random effects model accounting for the natural constraint

While the above multivariate model allows us to incorporate information about the correlation between outcomes into our estimation of the population-averaged prevalences, it fails to account for the natural constraint in the data when we model an overall outcome along with the prevalence of different subtypes. We present a different parameterization of the previous model in order to account for this natural constraint: y_i0 ≥ y_i1, y_i2, …, y_iJ, without requiring the subtypes be mutually exclusive.

First, let y_i0 be binomially distributed with parameter π_i0 as for the other two approaches. Then, let p_ij denote the proportion of cases in study i that fall into the subtype j ∈ {1, …, J}. Therefore, y_ij is binomially distributed with denominator y_i0 and probability p_ij . Let µ₀ denote the fixed effect for the overall outcome, and v_i0 are the within-study random effects for this overall outcome. Let ${\mu }_j^{*}$ be the fixed effect corresponding to the proportion of outcomes that fall into subtype j and the $v_{ij}^{*}\text{'}\text{s}$ be the corresponding random effects. We again use a probit link function to model π_i0 and p_ij :

$$\begin{gathered} \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} y_{i0}~\text{Binomial}\left(n_i,{\pi }_{i0}\right),{\varphi }^{-1}\left({\pi }_{i0}\right)={\mu }_0+v_{i0}, \\ {y}_{ij}~\text{Binomial}\left(y_{i0},p_{ij}\right),{\varphi }^{-1}\left(p_{ij}\right)={\mu }_j^{*}+v_{ij}^{*},j\in \left\{1,\dots ,J\right\},j\in S_i, \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} {{\mathbf{v}}^{*}}_i={\left(v_{i0},v_{i1}^{*},\dots ,v_{iJ}^{*}\right)}^T~MVN\left(0,{\Sigma }_{J+1}^{*}\right), \end{gathered}$$ (6)

where ${\mathbf{\Sigma }}_{J+1}^{*}=diag\left(\sigma *\right){{\mathbf{R}}^{*}}_{J+1}diag\left(\sigma *\right)$. ${{\mathbf{R}}^{*}}_{J+1}$ is the within-study correlation matrix and the ${\sigma }_j^{*2}$ terms capture the between study variation in the overall outcome and proportion of events that fall into each subtype. The key difference between Σ_J+1 and Σ*_J+1 is that the final J variance components ${\sigma }_1^{*2},\dots ,{\sigma }_J^{*2}$ in Σ*_J+1 describe the within-study random effects in the proportions of the overall outcome that fall into the individual subtypes ${\left(v_{i1}^{*},\dots ,v_{iJ}^{*}\right)}^T$, while ${\sigma }_1^2,\dots ,{\sigma }_j^2$ in Σ_J+1 describe the within-study random effects in the prevalences of the individual subtypes (v_i1,…, v_iJ )^T . Furthermore, the correlation terms between subtypes can be interpreted as reflecting the correlation between the subtype event rates, conditional on the overall count in each study.

This results in the following observed data likelihood function:

$$L_3\propto \prod _{i=1}^N{\int }_{{ℝ}^J}\left(\prod _{j\in S_i}{L}_{ij}\left(y_{ij};{\mu }_j^{*},v_{ij}^{*}\right)\right)\frac{exp\left(-\frac{1}{2}{\mathbf{v}}_i^{*T}{\sum }_{J+1}^{-1}{\mathbf{v}}_i^{*}\right)}{{\left(2\pi \right)}^{\left(J+1\right)/2}{\left|{\sum }_{J+1}^{*}\right|}^{1/2}}d{\mathbf{v}}_i^{*},$$ (7)

where the conditional likelihood L_i0 is defined as

$$L_{i0}\left(y_{i0};{\mu }_0,v_{i0}\right)={\left[\varphi \left({\mu }_0+v_{i0}\right)\right]}^{y_{i0}}{\left[1-\varphi \left({\mu }_0+v_{i0}\right)\right]}^{n_i-y_{i0}},$$ (8)

and the conditional likelihood L_ij for j ∈ {1,…, J} is defined as

$$L_{ij}\left(y_{ij};{\mu }_j^{*},v_{ij}^{*}\right)={\left[\varphi \left({\mu }_j^{*}+v_{ij}^{*}\right)\right]}^{y_{ij}}{\left[1-\varphi \left({\mu }_j^{*}+v_{ij}^{*}\right)\right]}^{y_{i0}-y_{ij}}.$$ (9)

We can then estimate π₀ by:

$${\pi }_0=E\left[{\pi }_{i0}|{\mu }_{0,}{\sigma }_0\right]={\int }_{-\infty }^{\infty }\varphi \left({\mu }_0+{\sigma }_{0}z\right)\varphi \left(z\right)dz=\varphi \left({\mu }_0/\sqrt{1+{\sigma }_0^2}\right),$$ (10)

where ϕ(·) denotes the standard normal cumulative distribution function, Z ∼ N(0, 1), and ϕ(·) is the density of the standard normal distribution. Let (X , Y ) be a standard bivariate normal with covariance $Co{v}_{X,Y}=\frac{1}{\sqrt{1+{\sigma }_0^2}\sqrt{1+{\sigma }_0^{*2}}}{\sum }_{1,j+1}^{*}$.

As shown in the Supplementary Materials, we can estimate ${\pi }_j=E\left[{\pi }_{i0}{p}_{ij}|{\mu }_0,{\mu }_j^{*},{\sigma }_0,{\sigma }_j^{*}\right],j\in \left\{1,\mathrm{..},J\right\}$ using:

$$\begin{gathered} {\pi }_j={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\varphi \left({\mu }_0+{\sigma }_0{z}_0\right)\varphi \left({\mu }_j^{*}+{\sigma }_j^{*}{z}_j\right)\varphi \left(z_0,z_j\right)\hspace{0.17em}d{z}_{0}d{z}_j \\ \hspace{1em}=P\left(X<\frac{{\mu }_0}{\sqrt{1+{\sigma }_0^2}},Y<\frac{{\mu }_j^{*}}{\sqrt{1+{\sigma }_j^{*2}}}\right). \end{gathered}$$ (11)

This new parameterization of the multivariate random effects model truncates the density of each study-level subtype prevalence at the current estimate for the overall prevalence. We hypothesized that this would decrease bias and increase precision when estimating the population-averaged subtype prevalences.

3.4 |. Prior specifications

We use Markov chain Monte Carlo (MCMC) methods to obtain Bayesian posterior estimates for each π_j and Σ*_J+1, with N(0, 1000) priors for each µ_j . For the univariate models, we put a Unif(0, 10) prior on each σ_j . The inversegamma(ε, ε), where ε is small, has previously been a popular choice of prior for ${\sigma }_j^2$, as it is conditionally conjugate ³⁶. However, the results can be sensitive to the choice of ε, particularly for small σ ³⁶. We use the same choice of priors with a spherical decomposition for both Σ*_J+1 and Σ_J+1. The commonly used conjugate inverse-Wishart prior for the precision matrix of multivariate normal random vectors can result in inflated estimates of the variances and shrinkage of the correlations towards zero, particularly when the true variances are small ^12;37. Thus, we use a separation strategy ³⁸ in order to specify the priors on Σ_J+1 and ${\Sigma }_{J+1}^{*}$, which involves modeling the variance and correlation components separately. Specifically, in this case we use the spherical decomposition described by Lu and Ades ³⁹ and Wei and Higgins ¹² and implemented in the pcnetmeta R package ⁴⁰, with Unif (0, π) priors on the coordinate parameters³⁹.

3.5 |. Missing at random (MAR) assumption

While these methods do not require that all studies measure each outcome, they do require the assumption that the data are missing at random (MAR). That is, the probability of an outcome being measured for any given study may depend only the observed outcomes for that study. Since we are currently assuming that the overall outcome is observed for all studies, when we simulate data that are MAR, we let the probability of being missing depend only on the overall prevalence.

To illustrate, let m_ij = 1 if the number of events y_ij for the jth outcome of the ith study is not reported. We can then model the probability that m_ij = 1 by:

$$\text{logit}\left(P\left(m_{ij}=1\right)\right)={\alpha }_{0j}+{\alpha }_{1j}f\left({\pi }_{i0}\right)+{\alpha }_{2j}g\left({\pi }_{ij}\right),$$ (12)

where f(·) and g(·) can be any functions. The specific functions we used in our simulation studies are described in Section 4.1. The likelihood function modeling the missing data is $L_m\propto {{\prod }_{i=1}^N{\prod }_{j=1}^{J}P{\left(m_{ij}=1\right)}^{m_{ij}}\left[1-P\left(m_{ij}=1\right)\right]}^{1-m_{ij}}$. Multiplying the likelihood for the observed data as in equation (5) or (7) with L_m, we obtain the total likelihood function which incorporates the missing mechanism. If the data are missing completely at random (MCAR), then α_1j = α_2j = 0, and the probability of the j^th outcome being missing is some constant determined by α_0j . If the data are missing at random, then α_2j = 0 and the probability of being missing only depends on the underlying event rates for the data that are fully observed. Finally, if α_2j ≠ 0, then the data are missing not at random (MNAR), since the probability of an outcome being missing depends directly on the underlying event rate. We can evaluate the impact of the missing data assumptions by simulating patterns of missingness that correspond to different values of α_1j and α_2j.

4 |. SIMULATION STUDIES

4.1 |. Methods of simulation

In order to compare the performance of the univariate models, the standard multivariate model, and the new multivariate model, we performed two main sets of simulations: one with data for the two subtypes under MCAR and one under MAR. For each setting, we simulated 2000 data sets containing 30 studies. Each data set contains the overall count for each study (y_i0), the counts for two subtypes (y_i1, y_i2), and the overall number of participants in each study (n_i). We let y_i0 be distributed binomial with denominator n_i and success probability π_i0, an.d y_i1 and y_i2 each be distributed binomial with denominator y_i0 and success probabilities p_i1. and p_i2., respectively. The $\left({\varphi }^{-1}\left({\pi }_{i0}\right),{\varphi }^{-1}\left(p_{i1}\right),{\varphi }^{-1}\left(p_{i2}\right)\right)$ are distributed as a multivariate normal with different variances. For simplicity, we let all of the pairwise correlations be identical and equal to ρ. The overall count y_i0 is observed in all studies, while the mean probabilities of missing each subtype across all studies, ${\overline{m}}_{.1}$ and ${\overline{m}}_{.2}$ are 0.5. Each condition is repeated twice, with n_i, the sample size for each study, equal to 100 or 500. All models are fit using JAGS version 4.3.0⁴¹, run using R version 3.3.3⁴² and packages rjags ⁴³ and coda ⁴⁴, and consist of 2 independent chains with 20,000 samples each. We also use a burn-in period of 5,000 samples and a thinning interval of 2. The 2,000 simulations for each condition were run in parallel using the Minnesota Supercomputing Institute (MSI) resources.

We first simulated data for 30 studies (N = 30), where observations for the two subtypes (y_i1; y_i2) were all MCAR with probability 0.5. We set the pairwise correlations between ϕ⁻¹(π_i0), ϕ⁻¹(p_i1), ϕ⁻¹(p_i2) to all be equal to ρ, which had possible values of (0, 0.4, 0.8). The (σ₀, σ₁, σ₂) were set to be either (0.5, 0.5, 0.5) or (0.5, 1, 1). The µ₀, µ₁ and µ₂ are set such that the population-averaged prevalence (π₀, π₁, π₂) were equal to (0.3, 0.15, 0.05), respectively. This corresponded to 6 different conditions, which we repeated with n_i = (100, 500) for each of the 30 studies, for a total of 12 conditions. To investigate the performance of proposed methods under small number of studies, we also included a set of simulations when the number of studies N = 10 and the sample size per study n_i = 100, giving an additional 6 scenarios.

Second, we repeat each of the MCAR conditions described above, but with y_i1 and y_i2 under MAR and the marginal probability of being missing across all studies P(m_ij = 1) = 0.5 for j = 1, 2. We let the probability of missingness for the jth subtype of the ith study, P (m_ij = 1) depend on π_i0, the observed overall study-specific prevalence. For simplicity, we assume P(m_i1 = 1) = P (m_i2 = 1), since these only depend on the overall study-specific prevalence. From equation (12), we let α_0j = logit(0.5), α_1j = 3, j = 1, 2, and $f\left({\pi }_{i0}\right)=\text{logit}\left({\pi }_{i0}\right)-\text{logit}\left({\overline{\pi }}_{i0}\right)$ such that P (m_ij = 1) is inversely proportional to the difference in the logit of overall study-specific prevalence and its mean, i.e.,

$$P\left(m_{ij}=1\right)={\text{logit}}^{-1}\left(\text{logit}\left(0.5\right)-3(\text{logit}\left({\pi }_{i0}\right)-\text{logit}\left({\overline{\pi }}_{.0}\right)\right).$$ (13)

Therefore, studies with smaller prevalence will be more likely to be missing, leading to ORB as described in Section 1.

Finally, we also repeated each of the 6 conditions where n_i = 100 for the case were there were no missing data. This included separate cases where each data set included 30 studies or 10 studies. Overall, this corresponded to 18 MCAR and MAR conditions and 12 no missing data conditions, for a total of 48 conditions.

4.2 |. Simulation results

Table 1 summarizes the results where N = 100 and approximately 50% of the subtype data are MCAR across studies. All three models gave similar results for the fully observed overall outcome, with the univariate model having slightly less bias and shorter credible intervals than the two multivariate models. However, using the new parameterization reduced both bias and 95% credible interval width (CIW) for the two subtypes under all conditions, outperforming both the univariate and original multivariate models. This reduction in bias and CIW for the two subtypes became larger as both ρ and the subtype variance increased. The original multivariate model still reduced bias and CIW over the univariate approach, except when ρ = 0, where it corresponded to a larger bias, CIW, or both. We observe qualitatively similar results for each condition when N = 500, as well as the conditions where each data set contained only 10 studies, which are given in Tables W.2 and W.4 in the Supplementary Materials, respectively.

TABLE 1.

(N = 30, n_i = 100, MCAR) Bias, 95% credible interval width (CIW) and coverage probability (Cov.) for univariate model, original multivariate model, and new parameterization, across 2000 simulations containing 30 studies, where n_i = 100, the true prevalences are (0.3, 0.15, 0.05), and data are missing completely at random (MCAR).

		Overall			Subtype 1			Subtype 2
		Bias	CIW	Cov.	Bias	CIW	Cov.	Bias	CIW	Cov.
σ = (0.5, 0.5, 0.5), ρ = 0	Univariate	0.003	0.121	0.95	0.009	0.127	0.96	0.008	0.078	0.96
	Original	0.003	0.122	0.96	0.007	0.102	0.95	0.008	0.071	0.95
	New Param.	0.004	0.124	0.96	0.002	0.097	0.95	0.005	0.061	0.95
σ = (0.5, 0.5, 0.5), ρ = 0.4	Univariate	0.003	0.121	0.95	0.010	0.143	0.97	0.011	0.094	0.97
	Original	0.003	0.122	0.96	0.007	0.109	0.96	0.009	0.076	0.95
	New Param.	0.004	0.124	0.96	0.003	0.103	0.95	0.006	0.066	0.94
σ = (0.5, 0.5, 0.5), ρ = 0.8	Univariate	0.002	0.121	0.96	0.012	0.158	0.97	0.013	0.105	0.96
	Original	0.002	0.121	0.95	0.006	0.114	0.96	0.007	0.074	0.96
	New Param.	0.003	0.122	0.96	0.004	0.107	0.96	0.005	0.066	0.95
σ = (0.5, 1, 1), ρ = 0	Univariate	0.002	0.121	0.95	0.013	0.162	0.96	0.018	0.132	0.97
	Original	0.003	0.123	0.96	0.012	0.146	0.95	0.021	0.133	0.95
	New Param.	0.004	0.124	0.96	0.001	0.121	0.96	0.010	0.089	0.95
σ = (0.5, 1, 1), ρ = 0.4	Univariate	0.002	0.121	0.96	0.015	0.183	0.97	0.023	0.153	0.97
	Original	0.003	0.123	0.96	0.012	0.152	0.96	0.022	0.137	0.95
	New Param.	0.004	0.124	0.96	0.003	0.129	0.96	0.011	0.097	0.95
σ = (0.5, 1, 1), ρ = 0.8	Univariate	0.002	0.121	0.96	0.018	0.203	0.97	0.025	0.166	0.96
	Original	0.002	0.121	0.96	0.010	0.148	0.96	0.018	0.123	0.96
	New Param.	0.003	0.122	0.96	0.004	0.132	0.95	0.011	0.096	0.95

The reduction in bias for the two subtypes for the two multivariate parameterizations increased when the data were MAR, as shown in Table 2 (N = 100) and Table W.3 (N = 500). This includes the conditions in the MCAR scenarios (where ρ = 0) where the original multivariate model sometimes had larger bias than the univariate models for the subtype outcomes. In the MAR case where ρ = 0, σ = (0.5, 1, 1), and N = 100, using the original multivariate parameterization reduced bias by 52.9% and 5.0% for subtypes 1 and 2, respectively, while using the new parameterization reduced it by 97.1% and 72.5%, respectively, when compared to the univariate models. As expected, the coverage probabilities for the univariate models were quite low when the data were MAR (as low as 23.4% for the N = 500, σ = (0.5, 0.5, 0.5), ρ = 0.8 scenario). We again observed similar results when N = 10 (Table W.5).

TABLE 2.

(N = 30, n_i = 100, MAR) Bias, 95% credible interval width (CIW) and coverage probability (Cov.) for for univariate model, original multivariate model, and new parameterization, across 2000 simulations containing 30 studies, where n_i = 100, the true prevalences are (0.3, 0.15, 0.05), and data are missing at random (MAR).

		Overall			Subtype 1			Subtype 2
		Bias	CIW	Cov.	Bias	CIW	Cov.	Bias	CIW	Cov.
σ = (0.5, 0.5, 0.5), ρ = 0	Univariate	0.003	0.121	0.95	0.064	0.127	0.38	0.027	0.081	0.65
	Original	0.003	0.123	0.96	0.017	0.121	0.94	0.014	0.089	0.93
	New Param.	0.004	0.124	0.96	0.002	0.092	0.96	0.005	0.057	0.96
σ = (0.5, 0.5, 0.5), ρ = 0.4	Univariate	0.003	0.121	0.95	0.078	0.150	0.35	0.037	0.102	0.56
	Original	0.003	0.122	0.96	0.014	0.116	0.94	0.013	0.080	0.94
	New Param.	0.004	0.124	0.96	0.005	0.099	0.95	0.007	0.060	0.96
σ = (0.5, 0.5, 0.5), ρ = 0.8	Univariate	0.003	0.121	0.96	0.092	0.161	0.27	0.045	0.108	0.42
	Original	0.003	0.121	0.95	0.013	0.113	0.94	0.010	0.070	0.95
	New Param.	0.003	0.122	0.96	0.008	0.104	0.95	0.007	0.058	0.94
σ = (0.5, 1, 1), ρ = 0	Univariate	0.003	0.121	0.95	0.068	0.189	0.65	0.040	0.159	0.83
	Original	0.004	0.124	0.96	0.032	0.195	0.95	0.038	0.185	0.93
	New Param.	0.004	0.124	0.96	0.002	0.114	0.96	0.011	0.084	0.95
σ = (0.5, 1, 1), ρ = 0.4	Univariate	0.003	0.121	0.96	0.090	0.202	0.50	0.052	0.171	0.67
	Original	0.004	0.123	0.96	0.025	0.172	0.95	0.029	0.152	0.93
	New Param.	0.005	0.124	0.97	0.007	0.123	0.96	0.013	0.087	0.95
σ = (0.5, 1, 1), ρ = 0.8	Univariate	0.003	0.121	0.96	0.112	0.210	0.34	0.063	0.173	0.55
	Original	0.003	0.121	0.96	0.016	0.141	0.93	0.016	0.104	0.95
	New Param.	0.004	0.123	0.96	0.011	0.128	0.94	0.011	0.084	0.94

The results for the cases where there were no missing data are presented in Tables W.6 and W.7 in the Supplementary Materials for N = 30 and N = 10, respectively. Here, the original multivariate model provided little to no benefit over using separate univariate models. Under many conditions, this model had larger bias and wider credible intervals on average than the separate univariate models. We hypothesize that without any missing data, the original multivariate model’s ability to borrow information across outcomes may not make up for the additional model complexity. These results are also consistent with previous literature ^14;15. However, the new parameterization still had reduced bias and credible interval widths even without missing data. We hypothesize that this is due to the reduction in density where the subtypes would have greater prevalence than the overall outcome. Furthermore, in the case where the subtype outcomes are distributed as binomial with the denominator equal to the overall count, the probit (or logit) transformed prevalences likely do not follow a normal distribution. This could further explain the reduction in bias found with the new parameterization.

In summary, using the two multivariate parameterizations did not improve performance when focusing solely on estimating the overall prevalence. However, these models improved estimation of the prevalences of the two subtypes over using separate univariate models, particularly under the MAR conditions, with the new parameterization outperforming the original multivariate model under all conditions.

5 |. THE CASE STUDY

5.1 |. Results under missing at random assumption

Table 3a presents estimates of the overall urinary incontinence (UI) prevalence and that of each subtype (SUI, UUI) found using separate univariate models, the multivariate model, and the new parameterization accounting for the subtype constraint. Each model was fit using 3 independent chains with 250,000 samples each and a burn-in period of 5,000 samples. We used a substantially larger number of iterations for the case study in order to generate smooth plots of the estimated density of the posterior predictive distribution. We also report the posterior mean and standard deviation of the between study variances and the correlations between outcomes in Table 3b. We expect the estimates of the covariance matrices to differ between the two parameterizations, since the random effects for the subtypes in the new parameterization refer to the variability in the proportion of overall cases that fall into that subtype, rather than the variability in study specific prevalence.

TABLE 3.

Case Study Results

(a) Posterior mean (SD) of marginal prevalences for overall outcome (UI) and two subtypes (SUI, UUI) with corresponding 95% credible interval width (CIW)
Model	UI	CIW	SUI	CIW	UUI	CIW
Univariate	0.274 (0.024)	0.096	0.127 (0.022)	0.088	0.066 (0.021)	0.082
Multivariate	0.275 (0.025)	0.098	0.128 (0.022)	0.088	0.064 (0.019)	0.072
New Parameterization	0.275 (0.025)	0.099	0.123 (0.02)	0.078	0.061 (0.016)	0.064

(b) Posterior mean (SD) of components in estimated covariance matrices for original multivariate model and new parameterization ($\widehat{\mathbf{\Sigma }}$, $\widehat{\mathbf{\Sigma }}*$
	${\widehat{\sigma }}_1$	${\widehat{\sigma }}_2$	${\widehat{\sigma }}_3$
Univariate	0.383 (0.060)	0.435 (0.089)	0.607 (0.131)
Multivariate	0.394 (0.062)	0.457 (0.093)	0.6 (0.121)
	${\widehat{\sigma }}_1^{*}$	${\widehat{\sigma }}_2^{*}$	${\widehat{\sigma }}_3^{*}$
New Param.	0.399 (0.064)	0.755 (0.160)	0.683 (0.157)
	${\widehat{\rho }}_{12}$	${\widehat{\rho }}_{13}$	${\widehat{\rho }}_{23}$
Multivariate	0.462 (0.204)	0.681 (0.158)	0.395 (0.220)
	${\widehat{\rho }}_{12}^{*}$	${\widehat{\rho }}_{13}^{*}$	${\widehat{\rho }}_{23}^{*}$
New Param.	−0.115 (0.249)	0.364 (0.233)	0.103 (0.270)

All three models gave similar results for the overall prevalence of UI. The estimates of the SUI prevalence differed slightly across the three models, with the new parameterization giving the smallest estimate. The proportion falling into the SUI subtype also had low correlation with the overall outcome under the new parameterization (−0.115) and larger variance (0.755). The 95% credible interval was wider for the original parameterization, than the univariate model, similar to the σ = (0.5, 1, 1) and ρ = 0 scenarios from Section 4.2. However, the new parameterization still reduced 95% credible interval width by 11.4% compared to the univariate model.

The estimated marginal prevalence of UUI was highest under the univariate model, and lowest under the new parameterization. The proportion falling into the UUI category had a higher estimated correlation with the overall outcome under the new parameterization (0.364) and slightly lower estimated variance (0.600) when compared to the SUI outcome. Jointly modeling the outcomes resulted in a 12.2% and 22.0% reduction in 95% credible interval width for the original and new multivariate parameterizations, respectively. Based off of the original model, the estimated unconditional correlation between the SUI and UUI subtypes was relatively low (0.395). As estimated by the new parameterization, the estimated correlation conditional on the overall count was even smaller (0.103).

Figure 1 presents forest plots for each outcome, showing the shrunken estimates for the study-level prevalences under the three different models. The estimates and credible intervals are similar across all three methods for the overall prevalence of UI, as well for the subtypes in studies where the given outcome is observed. While the univariate model cannot estimate study level prevalence for unobserved subtypes, the two multivariate parameterizations can do so using information from the correlations and observed outcomes. The new parameterization gave narrower credible intervals than the original multivariate model when estimating subtype prevalence corresponding to the unobserved outcomes. Similarly, Figure W.1 presents posterior density plots for each outcome, by model. This illustrates the reduction in density at larger values when using the new parameterization.

FIGURE 1. Forest Plot of Study Level Estimates.

Posterior mean and 95% credible interval of marginal and study level prevalences for each of the three outcomes across 26 studies

We further investigated this reduction in joint density by the new parameterization using the posterior predictive bivariate density plots, as shown in Figure 2. We estimated these posterior predictive distributions by drawing a random sample representing the prevalences of a future study from the joint posterior distributions of parameters at each of the 750,000 total iterations. Of the samples generated according to the results from the original multivariate model, 64 samples and 1 sample gave estimates where the overall UI prevalence was smaller than the corresponding estimates for the SUI and UUI prevalences, respectively. The new parameterization on the other hand specifically prevents this from occurring, which explains the reduction in density between (a) and (c) in Figure 2. Finally, we tested the sensitivity to the choice of the separation strategy prior on the covariance matrix by rerunning the two multivariate models using an inverse-Wishart prior. The results are included in Table W.8 in the Supplementary Materials, and are similar to those found using the previous choice of prior.

FIGURE 2. Bivariate Density Plots for Predicted Prevalences in New Study.

(a) Overall and SUI posterior predicted prevalences for new study based on original multivariate model results, (b) Overall and UUI with original multivariate model, (c) Overall and SUI with new parameterization, (d) Overall and UUI with new parameterization

5.2 |. Sensitivity analysis results under MNAR

When fitting each model, we assume that the probability of each study missing a subtype outcome does not directly depend on the underlying prevalence. As we cannot directly test whether the data are MNAR, we conduct a sensitivity analysis assuming several different patterns of missingness to evaluate the impact of different degrees of MAR violations. We assume that m_ij ∼ Ber(q_ij), where.q_ij is the probability of subtype j for study i being missing. Specifically, we specify a logistic model for q_ij as logit (q_ij) = α_0j + α_2j, where α_2j is not identifiable. Instead, we specify values for α_2j ∈ [−2, 2], with increments of 0.1 and observe how the estimates of the marginal prevalences π₀, π₁, π₂ vary when (12) is incorporated into the likelihoods for the original multivariate model (5) and new parameterization (7). For simplicity, we let α₂₁ = α₂₂ = α₂ for each α_2j ∈ [−2, 2].

Figure 3 presents the posterior means and 95% credible intervals of π₀, π₁, and π₂ for the two models under each value of α₂ ∈ [−2, 2]. As expected, the posterior mean for the overall UI prevalence (π₀) remained approximately constant across values of α₂, since it is fully observed. However, the posterior means for the two subtype prevalences (π₁, π₂) generally increased with α₂. Larger values of α₂ corresponded to studies with larger π_ij missing with higher probability, thus the estimated prevalence became larger in order to incorporate this fact. The posterior mean of π₁ ranged from (0.123, 0.133) and from (0.120, 0.126) for the original and new parameterizations, respectively, while the posterior mean of π₂ ranges from (0.061, 0.067) and from (0.059, 0.064). Therefore, violating the MAR assumption to this extent led to a slightly larger difference in estimates for the original model than for the new parameterization. We also note that the upper limit of the 95% credible interval from the new parameterization is consistently lower than that from the original model for the SUI and UUI panels of Figure 3.

FIGURE 3. Sensitivity Analysis.

Posterior mean and 95% credible intervals for UI, SUI, and UUI marginal prevalence across values of α₂

In general, the mechanism of MNAR is unobservable. The sensitivity analysis above was intended to examine the risk of bias under a few MNAR mechanisms. Missingness may depend on other unobserved characteristics of the study population and even if missingness is only related to subtype prevalences, the dependency may differ from what we considered.

6 |. DISCUSSION

Jointly modeling the overall and each subtype prevalence using multivariate random effects models generally improved both bias and precision for the marginal subtype estimates by “borrowing strength” across outcomes. Incorporating the natural subtype constraint improved estimation further, likely by eliminating density in regions where the subtype prevalence would be larger than the overall prevalence for an individual study. This occurred even under the few conditions where the original multivariate parameterization failed to improve estimation, including when there were no missing data. Using the original multivariate model was the least beneficial when correlations between outcomes were low, variances were large, and the data were MCAR. We hypothesize that the multivariate models cannot borrow much information when ρ = 0. In this case, the multivariate model required estimating far more parameters than the univariate models, thus increasing variability. However, the new parameterization improved estimation under these conditions by taking into account the natural constraint in the data. Both models improved bias when the data were MAR and correlations were large, with the new parameterization still outperforming the original multivariate model.

This behavior was reflected in the results of the case study. The SUI subtype had low correlation with the overall outcome under the new parameterization and slightly larger variance. This likely led to the original multivariate model giving a wider 95% credible interval than the univariate model, while the new parameterization having a slightly narrower credible interval. The UUI subtype had a slightly higher correlation with the overall outcome and slightly lower variance. In this case, the original multivariate model had a slightly shorter 95% credible interval than the univariate model, consistent with the results of the simulations. We were also able to directly observe the reduction in density in the posterior predictive distributions for a single study associated with using the new parameterization, as shown in Figure 2. Finally, using the original and new multivariate parameterizations to jointly model the case study outcomes allowed us to estimate the correlation between subtypes both unconditionally and conditioned on the overall outcome.

Because the simulation conditions were generated under the new parameterization in order to ensure the subtype counts did not exceed the overall counts, the correlations used to generate the data correspond to the overall outcome and proportions falling into each subtype. Large positive or negative correlations between the two subtypes would imply that the subtypes had a high co-morbidity or tended toward mutual exclusivity, respectively. Therefore, we hypothesize that using the new parameterization would be the most advantageous when a disease or set of outcomes are suspected to have either of these characteristics. However, if the subtypes were known to be mutually exclusive, a multinomial model would be more appropriate. We also caution that the methods may not be appropriate in the case of very rare diseases, resulting in very sparse counts, as the resulting estimates may not be stable.

Because we based the simulated marginal prevalences on those observed in the case study, all simulations were conducted using (π₀, π₁, π₂) = (0.3, 0.15, 0.05). Therefore, the elimination of density by the new parameterization excluded all values above approximately 0.3, omitting about 70% of the original space. Furthermore, we hypothesize that this reduction in density may have the most impact in cases where at least one subtype has a high prevalence relative to the overall outcome. In this situation, the original multivariate model would likely have a greater number of samples where the subtype prevalence estimate was incorrectly higher than the overall prevalence. While we assume in this paper that the overall outcome is fully observed, these methods could be used when some studies do not report the overall count, such as in the food allergy ²⁸ and autism spectrum disorder (ASD) ²⁹ meta-analyses mentioned in Section 1. While both multivariate methods can be implemented with multiple subtypes, the estimates may become unstable in higher dimensions depending on the amount of missing data, variances, and correlations between outcomes. More work would be needed to evaluate these models’ performance in higher dimensional scenarios.

As mentioned in Section 12, both multivariate methods require that any missing data be MAR or MCAR. Our analysis of the case study data included a sensitivity analysis of a specific MNAR mechanism, but the true missingness mechanism remains unknowable. However, we hypothesize that it may be less common for the type of prevalence data we have described to be MNAR. If the data were MNAR, the probability of a given subtype being missing would directly depend on the underlying prevalence of each study. One such scenario would be given by investigators choosing not to report the prevalence of a subtype with a low count. We view this as unlikely, since if a study aiming to estimate the prevalence of a subtype encountered a lower frequency than expected, the outcome would still be of interest. Furthermore, while we have described in Section 1 the possible situation of studies measuring a lower overall prevalence being less likely to measure the lower frequency subtypes, this would be a case of the data being MAR, as the probability of being missing would depend on the overall count.

As discussed in Section 3.4, we use the separation strategy initially proposed by Barnard et al. ³⁸ for the priors on the covariance matrices in the two multivariate models. The separation strategy has been shown to improve estimation of the variance and covariance terms over the use of an inverse-Wishart conjugate prior by adding more flexibility. However, this method also greatly increased computational time, as the multivariate models took up to a few hours to fit, depending on the number of iterations used. Using an HMC sampler such as STAN may decrease computational time over JAGS (a Gibbs based sampler) ³⁷.

To the best of our knowledge, there is only one other existing multivariate meta-analysis of multivariate prevalence data ²⁴. This serves as a case study illustrating the potential improvement in estimation by jointly modeling multivariate prevalence data, particularly when incorporating additional natural constraints into the model parameterization. The methods used in this analysis allowed us to better compare prevalence rates of specific lower urinary tract symptom types across different occupation types in working women. These comparisons have informed future research on a wider range of lower urinary tract symptoms, in addition to urinary incontinence.