Bayesian multiple imputation for missing multivariate longitudinal data from a Parkinson’s disease clinical trial

Abstract

In Parkinson’s disease (PD) clinical trials, Parkinson’s disease is studied using multiple outcomes of various types (e.g. binary, ordinal, continuous) collected repeatedly over time. The overall treatment effects across all outcomes can be evaluated based on a global test statistic. However, missing data occur in outcomes for many reasons, e.g. dropout, death, etc., and need to be imputed in order to conduct an intent-to-treat analysis. We propose a Bayesian method based on item response theory to perform multiple imputation while accounting for multiple sources of correlation. Sensitivity analysis is performed under various scenarios. Our simulation results indicate that the proposed method outperforms standard methods such as last observation carried forward and separate random effects model for each outcome. Our method is motivated by and applied to a Parkinson’s disease clinical trial. The proposed method can be broadly applied to longitudinal studies with multiple outcomes subject to missingness.

1 Introduction

Parkinson’s disease (PD), a degenerative disorder of the central nervous system, is one of the most common movement disorders affecting about 1% of people older than 60 years.¹ The symptoms of PD include primary motor symptoms (e.g. resting tremor, slow movement, rigidity, and postural instability, etc.), secondary motor symptoms (e.g. speech problems, cramping, swallowing difficulty, etc.), and non-motor symptoms (e.g. sleep problems, pain, depression, etc.). Because there is no cure for PD, many drugs have been studied over the last 20 years to decrease the progression and disability of the disease.

A major challenge when studying PD is that impairment is multidimensional and progressive. Thus, for efficient development, targeting, and evaluation of treatments, it is necessary to collect longitudinally multiple outcomes (e.g. clinical rating scales or patient completed questionnaires) of various types (e.g. binary, ordinal, continuous). Common approaches for handling multiple outcomes include choosing a single outcome as a primary outcome, the use of a linear combination of several outcomes, multiple tests with adjustment to the overall significance level, omnidirectional tests of any type of treatment difference (e.g. Hotelling’s T² test, MANOVA, Wald test, and ² test), and hierarchical models. Detailed review and discussion of these methods can be found in Pocock² or Tilley et al.³ These methods address a directionless scientific question, i.e. whether there is any difference between two treatments in these outcomes. However, the major scientific question of many PD clinical trials is directional, i.e. whether target treatment is preferred over placebo in slowing PD progression based on all primary outcomes measured. When the treatment slows progression across all outcomes, the aforementioned approaches tend to statistically obscure findings and lose power.^2,3 To address this issue, O’Brien⁴ has introduced a global statistical test (GST) to combine information across outcomes and to examine whether a treatment has a global benefit without the need of multiple tests. The GST is more powerful than a univariate test or other multiple test procedures when there is a common dose effect across outcomes (i.e. treatment effect is of similar magnitude across all outcomes). The GST was further extended and discussed by many authors,^5-15 and has found broad application in medical studies on toxicity,¹⁶ stroke,^17,18 dermatology,¹⁹ asthma,²⁰ lower back pain,²¹ restless legs syndrome,²² neuropsychological impairment,²³ multiple sclerosis,^24,25 and breast cancer.²⁶

The rank-sum-type GST proposed by O’Brien⁴ is conducted as follows. First, for each outcome, all individuals are ranked based on their measured outcomes. Then each individual’s ranks are summed to obtain a rank sum. Finally, a two-sample t-test is applied to compare the rank sums between the treatment and placebo groups. This rank-sum-type GST can combine the treatment effect in all outcomes without making any parametric assumption on the outcome distributions and their correlation. However, Huang et al.¹² has proved that O’Brien’s rank-sum-type GST cannot control type I error asymptotically when two groups have different joint cumulative distribution functions. To address this issue, Huang et al.¹² proposed an adjusted rank-sum-type GST (referred to as adjusted GST) for the case where variances in two groups are different. The adjusted GST is computed similarly to O’Brien’s rank-sum-type GST, but it is divided by an adjustment factor. In addition, Huang et al.¹³ introduced the concept of global treatment effect (GTE), which measures a treatment’s overall benefit across multiple outcomes. In this article, we have selected the adjusted GST and GTE to assess the efficacy of PD treatment in a clinical trial.

Another challenge in longitudinal PD clinical trials is that missing data are ubiquitous due to missed visits, withdrawals, lost to follow-up, death, etc. However, when missing values are present in the response variables, the adjusted GST and GTE cannot be computed unless all individuals with missing values are excluded. The primary efficacy evaluation in confirmatory clinical trials is often required by agencies to follow the ‘intent-to-treat’ (ITT) principle, i.e. the analysis includes all randomized individuals regardless of the treatment they actually received, drop-out, or withdrawal of consent. By including all patients who are randomized, the ITT analysis preserves the benefits of randomization and is commonly accepted as the most unbiased approach. However, if the individuals drop out of the study, the data after dropout are missing. Therefore, the ITT analysis requires a method of dealing with the missing data. Complete-case (CC) analysis excludingall individuals with any missing values is seriously biased when the missing is not completely at random (MCAR) and less efficient²⁷ and it violates the ITT principle. Other standard methods such as last observation carried forward (LOCF), i.e. imputing with the value at the last available observation or with the worst observation for the group have been used in multiple PD studies.^28,29 Although the LOCF and worst observation imputation methods follows the ITT principle, neither are ideal in this context. They are single imputation methods and underestimate the true variability leading to biased tests.³⁰ The LOCF method underestimates disease progression because the last observed visit (when patient is expected to be less affected) is used in lieu of the final time point. Additionally, when there are multiple outcomes, both the LOCF and worst observation methods fail to account for the correlation among outcomes. A good review of the missing data methods for longitudinal studies can be found in Ibrahim et al.³¹ and Ibrahim and Molenberghs.³²

A better approach is to impute the missing outcomes using some imputation model. Imputation uncertainty is handled by multiple imputation (MI), where M > 1 sets of imputed values are created for the missing values in the dataset, as draws from the predictive distribution of the missing values under an assumed imputation model.³³ Among the multivariate longitudinal outcomes, there are three-sources of correlation, i.e. inter-source (different measures at the same visit), intra-source (same measure at different visits), and cross correlation (different measures at different visits).³⁴ One may conduct multiple imputation for each outcome using separate random effects models. But this method (referred to as separate models) ignores the inter-source and cross correlation. To the best of our knowledge, there is no available method that readily imputes the missing values in multivariate longitudinal response variables of mixed type while accounting for all sources of correlation. Our objectives in this article are to develop a Bayesian method based on item response theory (IRT) to perform multiple imputation (MI) for the missing multivariate longitudinal outcomes while accounting for all sources of correlation and to assess a treatment’s global effect across multiple outcomes. Because imputation and statistical inference are carried out separately with the MI method, the MI method has the flexibility that the imputation model differs from the model used for assessing treatment effect.³⁵

The remainder of the article is organized as follows. In section 2.1, we introduce a motivating ongoing PD clinical trial where a moderate amount of missing values has been occurring and in section 2.2 we illustrate the primary analysis methods for this trial. In section 3, we describe the imputation model, multiple imputation using MCMC, and the specification of prior distributions. In section 4, sensitivity analysis is performed under various scenarios to evaluate several methods. Section 5 applies the proposed methodology to a PD clinical trial dataset. A discussion is provided in section 6.

2 A motivating ongoing clinical trial and the primary analysis methods

2.1 A motivating ongoing clinical trial

This article is motivated by the ongoing NIH Exploratory Trials in Parkinson’s Disease (NET-PD) Long-term Study-1 (referred to as LS-1 study) funded by the National Institute of Neurological Disorders and Stroke (NINDS). The enrollment ended in 2010 and follow-up will be completed in 2015. The LS-1 study is a randomized, multi-center, double-blind, placebo-controlled Phase III study of creatine (the study drug, 10 g daily) in individuals with early treated PD. A total of 1741 individuals from 45 sites in the US and Canada were equally randomized to either 10 g creatine/day or matching placebo with annual follow-up for a minimum of 5 years. The LS-1 study evaluates the long-term effects of creatine as measured by change from baseline in a variety of clinical domains including modified Rankin score (Rankin), Schwab and England activities of daily living (SEADL), ambulatory capacity (AC), PDQ-39 summary score (PDQ-39), and symbol digit modalities (SDM). Rankin (an ordinal variable with integer value from 0 to 5, with larger value reflecting worse clinical outcomes) is a measurement of overall clinical assessment.³⁶ SEADL (an ordinal variable with integer value from 0 to 100 incrementing by 5, with larger value reflecting better clinical outcomes) is a measurement of activities of daily living.³⁷ AC is an ordinal variable with integer value from 0 to 20, with larger value reflecting worse clinical outcome.³⁸ PDQ-39 (an approximate continuous variable with integer value from 0 to 156, with larger value reflecting worse clinical outcomes) is a measurement of quality of life.³⁹ SDM (an approximate continuous variable with integer value from 0 to 110, with larger value reflecting better clinical outcomes) is a measurement of cognitive function.⁴⁰ The measurement of these five outcomes are taken at baseline, annual visits from year 1 to 5. The LS-1 study represents the largest number of patients with early treated PD ever enrolled in a clinical trial. Details of the LS-1 study can be found on the study website http://parkinsontrial.ninds.nih.gov/index.htm. It is expected that there will be approximately 20% missing data across the various outcomes. The primary analysis is to compare the creatine versus placebo groups using the adjusted GST and GTE based on these five outcomes’ changes from baseline to the last visit of year 5. Because the computation of the adjusted GST and GTE requires all outcomes to be in the same direction, we recode outcomes SEADL and SDM so that higher values in all outcomes are worse clinical conditions. We will next illustrate how to compute the adjusted GST and GTE in section 2.2.

2.2 The primary analysis methods

Suppose K outcomes are measured for N individuals at a total of J visits. Let y_ijk (binary, ordinal, and continuous) be the observed outcome k (k = 1, …, K) from individual i (i = 1, …, N) at visit j(j = 1, …, J, where j = 1 is baseline). We assume that the first n₁ individuals are in the placebo group and the next n₂ individuals are in the treatment group, and then N = n₁ + n₂. Let z_ik = y_iJk − y_i1k be the change from baseline to the last visit (e.g. 5 years) for outcome k (referred to as outcome change). Throughout the article, we code all outcomes so that larger observation values are worse clinical conditions. Because the clinical outcomes of PD patients generally deteriorate over time, we expect z_ik to be positive. For each outcome change, we rank all N individuals. Define the midrank of an observation as either the regular rank when there is no tie on the observation or the average rank among the tied observations.⁴¹ Let R_ik = midrank(z_ik). In general, higher z_ik values correspond to larger R_ik. The rank sum of individual i is $R_i={\Sigma }_{k=1}^K{R}_{ik}$. O’Brien’s rank-sum-type test statistic T is defined as the regular univariate two-sample t-test with pooled standard deviation for the two rank sum samples from the placebo and treatment groups.⁴ Specifically

$$T=\frac{{\stackrel{‒}{R}}_2-{\stackrel{‒}{R}}_1}{\sqrt{{\widehat{\sigma }}_2^2∕n_2+{\widehat{\sigma }}_1^2∕n_1}}$$ (1)

where ${\stackrel{‒}{R}}_1={\Sigma }_{i=1}^{n_1}{R}_i∕n_1$, ${\stackrel{‒}{R}}_2={\Sigma }_{i=n_1+1}^N{R}_i∕n_2$, ${\widehat{\sigma }}_1^2=\frac{1}{n_1-1}{\Sigma }_{i=1}^{n_1}{(R_i-{\stackrel{‒}{R}}_1)}^2$, and ${\widehat{\sigma }}_2^2=\frac{1}{n_2-1}{\Sigma }_{i=n_1+1}^N{(R_i-{\stackrel{‒}{R}}_2)}^2$. O’Brien’s test statistic T rejects the null hypothesis of no treatment effect at significant level α when |T|>t_df,α/2, where t_df,α/2 is the (1−α/2)th percentile of the t_df distribution with df degree of freedom, with df=[ζ²/(n₁−1)+(1−ζ)²/(n₂−1)]⁻¹ and $\zeta =({\widehat{\sigma }}_1^2∕n_1)({\widehat{\sigma }}_1^2∕n_1+{\widehat{\sigma }}_2^2∕n_2)$.¹²

However, Huang et al.¹² has proved that O’Brien’s unadjusted rank-sum-type GST cannot control type I error asymptotically when two groups have different joint cumulative distribution functions. Instead, Huang et al.¹² proposed an adjusted rank-sum GST test statistic T_a (referred to as adjusted GST statistics)

$$T_a=\frac{{\stackrel{‒}{R}}_2-{\stackrel{‒}{R}}_1}{\sqrt{\widehat{h}({\widehat{\sigma }}_2^2∕n_2+{\widehat{\sigma }}_1^2∕n_1)}}$$ (2)

where $\widehat{h}$ (see Huang et al.¹² for the details of computation) is the estimate of the asymptotic variance of O’Brien’s test statistic when the variances in the two groups are different. Huang et al.¹³ has shown that as the N increases, T_a converges to a normal distribution with asymptotic mean ${\mu }_{T_a}=K\stackrel{‒}{\psi }\sqrt{N}∕\left(2\sqrt{J^{\prime }\Sigma J}\right)$ and variance 1, where J is a K-dimensional vector with all its elements equal to one, $\stackrel{‒}{\psi }$ is GTE defined next. Please see Huang et al.¹³ for the formula to compute Σ and $\stackrel{‒}{\psi }$. In this article, we use the adjusted GST statistics to test the efficacy of the treatment. We refer to ${\stackrel{‒}{R}}_2-{\stackrel{‒}{R}}_1$ in T_a as rank sum difference and $\widehat{h}({\widehat{\sigma }}_2^2∕n_2+{\widehat{\sigma }}_1^2∕n_1)$ as estimated variance of the rank sum difference. Note that if the treatment is efficacious in slowing the PD progression, we expect ${\stackrel{‒}{R}}_2<{\stackrel{‒}{R}}_1$ and T_a>0.

To measure a treatment’s overall benefit across multiple outcomes, Huang et al.¹³ defined GTE as the difference of two probabilities: the probability that a control group individual will do better if the individual is in the treatment group, and the probability that a treatment individual will do better if the individual is in the control group. Let Z_1k and Z_2k be the change from baseline to the last visit (e.g. 5 years) for outcome k in the placebo and treatment groups, respectively. We expect Z_1k and Z_2k to be positive and Z_1k > Z_2k if a treatment is efficacious in slowing the PD progression. Let the treatment effect on outcome k be ψ_k = p(Z_1k > Z_2k)−p(Z_1k < Z_2k). The GTE across all K outcomes is $\stackrel{‒}{\psi }={\sum }_{k=1}^K{\psi }_k∕K$. The GTE takes values between −1 and 1 and it measures the degree of dissimilarity between two groups. When $\stackrel{‒}{\psi }=0$, there is no global preference between groups. When $\stackrel{‒}{\psi }$, the treatment group is preferred the most. When $\stackrel{‒}{\psi }=-1$, the placebo group is preferred the most. Higher positive $\stackrel{‒}{\psi }$ corresponds to a higher degree of treatment group preference. We will next illustrate how to impute the missing data so that the ITT principle can be followed when computing the adjusted GST and GTE.

3 Models used in MI inference

3.1 Imputation model

In this section, we propose the imputation model based on the item response theory for the multivariate longitudinal data. Let y_ij = (y_ij1, …, y_ijk, …, y_ijK)’ be the vector of observation for individual i at visit j, with elements possibly being binary, ordinal, and continuous. Let y_i = (y_i1, …, y_iK)’ be the outcome vector across visits. We model the binary outcomes, the cumulative probabilities of ordinal outcomes, and the continuous outcomes using a two-parameter model,⁴² graded response model,⁴³ and common factor model,⁴⁴ respectively.

$$\mathrm{logit}\left\{p\right(y_{ijk}=1\mid {\theta }_{ij}\left)\right\}=a_k+b_k{\theta }_{ij}$$ (3)

$$\mathrm{logit}\left\{p\right(y_{ijk}\le l\mid {\theta }_{ij}\left)\right\}=a_{kl}-b_k{\theta }_{ij},\text{with}l=1,2,\dots ,n_k-1$$ (4)

$$y_{ijk}=a_k+b_k{\theta }_{ij}+{∊}_{ijk}$$ (5)

where random error ${∊}_{ijk}\sim N(0,{\sigma }_{∊}^2)$,a_k is the outcome-specific ‘difficulty’ parameter and b_k is the outcome-specific ‘discriminating’ parameter that is always positive and represents the discrimination of outcome k, i.e. the degree to which the outcome discriminates between individuals with different latent disease severity θ_ij. In model (4), the ordinal outcome k has n_k categories and n_k − 1 thresholds a_k1, …, a_kl, …, a_knk−1 that must satisfy the order constraint a_k1< …, a_kl< … <a_knk−1. The probability that individual i selects category l on outcome k at visit j is p(Y_ijk = l|θ_ij) = p(Y_ijk ≤ l|θ_ij)− p(Y_ijk ≤l − 1|θ_ij). The latent variable θ_ij is continuous and it indicates individual i’s unobserved disease severity at visit j, with a higher value denoting more severe status. We refer to θ_ij as disease severity. The mean of θ_ij is modeled as a function of covariates and visit time

$${\mu }_{ij}=E\left[{\theta }_{ij}\right]=\left[{\mathit{X}}_i\beta \right]t_j$$ (6)

where X_i is individual i’s covariate vector including some covariates of interest (e.g. treatment assignment) and potential confounding variables (e.g. age, gender), t_j is the visit time variable with t₁ = 0 for baseline. For example, if X_i only includes the treatment assignment, μ_ij = [β₀ + β₁I_i(trt)]t_j, where I(·) is an indicator function, I_i(trt) = 1 if individual i is in the treatment group. The significant negative coefficient β₁ indicates that the treatment significantly improves the disease severity. Model (6) is a latent trait regression model assuming that the disease severity is expected to change linearly over the course of the study, with slope depending on the covariate vector X_i.⁴⁵

The latent disease severity vector θ_i = (θ_i1, …, θ_iJ)’ is assumed to be independently and identically distributed with normal probability density function h(θ_i, Σ), i.e. θ_i|Σ ~ N_J(μ_i, Σ), where μ_i = (μ_i1, …, μ_iJ)’ is the mean vector of θ_i, Σ is the J × J covariance matrix of θ_i with σ_lm being the (l, m) element. Various assumptions can be made on the covariance matrix, e.g. uniform (equal variance and equal correlation), autoregressive (correlation decreases as time separation increases), heteroscedastic (unequal variance and equal correlation), and more generally, unstructured (unequal variance and unequal correlation).⁴⁶ We use an unstructured covariance matrix in this article. This IRT model accounts for all three sources of correlations illustrated in section 1. Specifically, the inter-source correlation (different measures at the same visit) is modeled by the disease severity θ_ij. Both the intra-source correlation (same measure at different visits) and cross correlation (different measures at different visits) are modeled by the correlation between θ_ij and θ_ij’ with j ≠ j’ through the off-diagonal elements of covariance matrix Σ.

It is well-known that the item-response models are over-parameterized because they have more parameters than can be estimated from the data.^47,48 Hence additional constraints are required to make models identifiable. In the longitudinal setting discussed here, the mean and variance of θ_ij at one visit may be specified to establish the location and scale of the disease severity distribution.⁴⁹ Specifically, we set σ₁₁ = 1 (the variance of θ_ij at baseline) and additionally we set t₁ = 0 at baseline to make μ_i1 = 0 through model (6) to ensure parameter identifiability.

Under the local independence assumption (i.e. conditioning on the disease severity θ_ij, all components in y_ij are independent), the full likelihood of individual i across all visits is

$$p({\mathit{y}}_i,{\theta }_i)=\left[\prod _{j=1}^J\prod _{k=1}^{K}p(y_{ijk}\mid {\theta }_{ij})\right]\cdot h\left({\theta }_i\right)$$ (7)

For notation convenience, we let the difficulty parameter vector be a = (a₁, …, a_K)’, the discrimination vector be b = (b₁, …, b_K)’, and the parameter vector Φ = (a, b, β, Σ)’.

3.2 Bayesian multiple imputation

In this section, we illustrate how to use Markov chain Monte Carlo (MCMC) algorithm to iteratively draw samples for the parameter vector Φ and the missing outcome data from conditional distributions and how to perform multiple imputation to make appropriate statistical inference. Let Y_mis and Y_obs be the missing and observed responses, respectively. Multiple imputations under models (3), (4), and (5) are M independent draws ${\mathit{y}}_{\mathrm{mis}}^{\left(1\right)},\dots ,{\mathit{y}}_{\mathrm{mis}}^{\left(M\right)}$ from the posterior predictive distribution for the missing data, $p({\mathit{Y}}_{\mathrm{mis}}\mid {\mathit{Y}}_{\mathrm{obs}})=\int p({\mathit{Y}}_{\mathrm{mis}}\mid {\mathit{Y}}_{\mathrm{obs}},\Phi )p(\Phi \mid {\mathit{Y}}_{\mathrm{obs}})\mathrm{d}\Phi$. The observed-data posterior density is $p(\Phi \mid {\mathit{Y}}_{\mathrm{obs}})\propto \pi \left(\Phi \right)p({\mathit{Y}}_{\mathrm{obs}}\mid \Phi )$, where π(Φ) is the prior distribution, the observed likelihood function $p({\mathit{Y}}_{\mathrm{obs}}\mid \Phi )=\int p({\mathit{Y}}_{\mathrm{obs}},{\mathit{Y}}_{\mathrm{mis}}\mid \Phi )\mathrm{d}{\mathit{Y}}_{\mathrm{mis}}$. As in most missing data problems with unknown missing patterns, the posterior predictive distribution p(Y_mis|Y_obs) cannot be simulated directly. We use a Gibbs sampling algorithm to draw the missing values from p(Y_mis|Y_obs). Specifically, the current versions of the parameter vector Φ^(t), the missing data ${\mathit{y}}_{\mathrm{mis}}^{\left(t\right)}$, and the disease severity θ^(t) are updated in three steps:

$$\left(1\right){\mathit{y}}_{i\left(\mathrm{mis}\right)}^{(t+1)}\sim p({\mathit{y}}_{i\left(\mathrm{mis}\right)}\mid {\mathit{y}}_{\mathrm{obs}},{\theta }^{\left(t\right)}{\Phi }^{\left(t\right)})\text{for}i=1,\dots ,N.$$

$$\left(2\right){\theta }_i^{(t+1)}\sim p({\theta }_i\mid {\mathit{y}}_{\mathrm{obs}},{\mathit{y}}_{\mathrm{mis}}^{(t+1)},{\Phi }^{\left(t\right)})\text{for}i=1,\dots ,N.$$

$$\left(3\right){\Phi }^{(t+1)}\sim p(\Phi \mid {\mathit{y}}_{\mathrm{obs}},{\mathit{y}}_{\mathrm{mis}}^{(t+1)},{\theta }^{(t+1)}).$$

Starting from a reasonable initial values Φ⁽⁰⁾ and ${\mathit{y}}_{\mathrm{mis}}^{\left(0\right)}$, these steps define a cycle of Gibbs sampler. Repeating the above Gibbs sampling algorithm with large enough number of iterations, it creates stochastic sequences $\{{\Phi }^{\left(t\right)}:t=1,\dots ,T\}$ and $\{{\mathit{y}}_{\mathrm{mis}}^{\left(t\right)}:t=1,\dots ,T\}$ whose limiting distributions are p(Φ|Y_obs) and p(Y_mis|Y_obs), respectively. We implement this algorithm in WinBUGS.⁵⁰ We use the trace plots available in WinBUGS and view the absence of apparent trend in the plots as evidence of convergence. In addition, we run multiple chains with overdispersed initial values and compute the Gelman-Rubin scale reduction statistics $\widehat{R}$ to ensure $\widehat{R}$ of all parameters are smaller than 1.1.⁵¹ The length of burn-in is assessed by trace plots and autocorrelation for each parameter. If diagnostics tools suggest that convergence is achieved after T₀ iterations, we would retain simulated missing values every (T − T₀)/M iterations starting from t = T₀ + 1 and treat them as M imputed datasets.

After M imputed datasets have been created, they can be analyzed using the primary analysis methods illustrated in section 2.2, resulting in M sets of rank sum differences and associated estimated variances computed based on the change of all outcomes from baseline to the last visit. The M sets of rank sum differences and associated estimated variances are combined to create one multiple-imputation inference by Rubin’s MI rules³³ as follows. Let D_m and V_m denote the estimated rank sum difference and the associated estimated variance from the mth imputed dataset, respectively. The overall rank sum difference D is estimated by $\stackrel{‒}{D}=M^{-1}{\Sigma }_m{D}_m$ and its variance is estimated by $\widehat{V}=M^{-1}{\Sigma }_m{V}_m+(1+M^{-1})B$ where the between-imputation variance $B={(M-1)}^{-1}{\Sigma }_m{(D_m-\stackrel{‒}{D})}^2$. The test statistic $T_a=\stackrel{‒}{D}∕\sqrt{\widehat{V}}$ has approximately a central t-distribution under the null hypothesis of no treatment effect with degree of freedom $\nu =(M-1){(1+{\Sigma }_m{V}_m∕[(M+1)B\left]\right)}^2$.

Multiple imputation usually provides good MI efficiency and minimizes sampling variability with a moderate M. Specifically, we use M = 100 in this article. After M sets of GTE are computed based on M imputed datasets, the overall GTE is computed as the average of M sets of GTE, but its variance cannot be computed because the variance formula is unavailable.

3.3 Specification of prior distribution

In this section, we illustrate selection of the prior distribution π(Φ). We use vague prior distributions on all elements in the parameter vector Φ. Specifically, the prior distributions of a_k, k = 1, …, K, all elements in β are N(0, 100). We use the prior distribution b_k ~ Uniform[0, 20], k = 1, … K, to ensure positivity. To obtain the prior distributions for the threshold parameters of ordinal outcome k, we first define unconstrained auxiliary parameters $a_{k1}^{\ast },\dots ,a_{kl}^{\ast },\dots ,a_{k{n}_k-1}^{\ast }$ such that $a_{kl}^{\ast }\sim N(0,100)$ for l = 1, …, n_k − 1, and set a_kl equal to the l-th order statistic of the auxiliary parameters. This approach to modeling threshold parameters is recommended by Plummer.⁵² For the ease of sampling for Σ, we use an approach based on the Cholesky decomposition.⁵³ Let Σ=ΩΩ’, where Ω is a matrix with zero entries above the main diagonal, and let ω_lm be the (l, m)th entry for 1 ≤ m ≤ l ≤ J. Consider a latent vector z_i = (z_i1, …, z_iJ)’ with N(0, 1) independent components. Then the linear reparameterization of ${\theta }_i=\Omega z_i+{\mu }_i$ (with element being ${\theta }_{ij}={\Sigma }_{l=1}^j{\omega }_{jl}{z}_{il}+{\mu }_{ij}$, e.g. ${\theta }_{i2}={\omega }_{21}{z}_{i1}+{\omega }_{22}{z}_{i2}+{\mu }_{i2}$) has mean μ_i and variance Σ. The entries of the matrix Σ are computed as ${\sigma }_{lm}={\Sigma }_{k=1}^{l\wedge m}{\Omega }_{lk}{\Omega }_{mk}$, 1 ≤ l, m ≤ J, where l ∧ m = min(l, m). We impose Uniform(0, 20) prior distribution on ω_kk to ensure non-negativity and N(0, 100) prior distribution on ω_lm where l ≠ m to allow for possible negative correlation.

To summarize the multiple imputation methods illustrated in section 3, we fit item-response models (3), (4), and (5) with mean model (6) to obtain M full datasets with missing outcome values at all visits imputed, compute the change from baseline to the last visit for each outcome, rank all individuals on each outcome, add up each individual’s ranks to obtain a rank sum, compute the estimated rank sum difference and the estimated variance using equation (2) and compute the GTE, the results are combined using Rubin’s MI rules.

4 Simulation

In this section, we perform a sensitivity analysis under various scenarios. We conduct extensive simulation studies to evaluate the performance of the LOCF, separate models (use linear mixed models and proportional odds models as imputation models for continuous and ordinal outcomes, respectively), and MI method. We simulate the datasets with data structure similar to the LS-1 study, e.g. five outcomes (with the first three ordinal outcomes representing Rankin, SEADL, and AC, respectively and with the last two continuous outcomes representing PDQ-39 and SDM, respectively) and six visits (baseline, annual visits from year 1 to 5). The outcome data are generated from multivariate normal distributions using the following algorithm.

1. From the baseline data in the ongoing the LS-1 study, we obtain the baseline mean vector of the five outcomes μ₀ = (1.2, 91.1, 1.7, 13.2, 44.5)’ and the baseline inter-source covariance matrix V of the five outcomes

   $$V=\left(\begin{array}{ccccc}\hfill 0.22\hfill & \hfill -1.46\hfill & \hfill 0.31\hfill & \hfill 2.01\hfill & \hfill -1.25\hfill \\ \hfill -1.46\hfill & \hfill 39.51\hfill & \hfill -4.53\hfill & \hfill -32.23\hfill & \hfill 18.37\hfill \\ \hfill 0.31\hfill & \hfill -4.53\hfill & \hfill 2.36\hfill & \hfill 6.20\hfill & \hfill -4.37\hfill \\ \hfill 2.01\hfill & \hfill -32.23\hfill & \hfill 6.20\hfill & \hfill 111.90\hfill & \hfill -18.74\hfill \\ \hfill -1.25\hfill & \hfill 18.37\hfill & \hfill -4.37\hfill & \hfill -18.74\hfill & \hfill 135.87\hfill \end{array}\right).$$
2. We select the intra-source correlation coefficient matrix

   $$W=\left(\begin{array}{cccccc}\hfill 1.00\hfill & \hfill 0.75\hfill & \hfill 0.69\hfill & \hfill 0.67\hfill & \hfill 0.67\hfill & \hfill 0.65\hfill \\ \hfill 0.75\hfill & \hfill 1.00\hfill & \hfill 0.75\hfill & \hfill 0.69\hfill & \hfill 0.74\hfill & \hfill 0.70\hfill \\ \hfill 0.69\hfill & \hfill 0.75\hfill & \hfill 1.00\hfill & \hfill 0.81\hfill & \hfill 0.77\hfill & \hfill 0.75\hfill \\ \hfill 0.67\hfill & \hfill 0.69\hfill & \hfill 0.81\hfill & \hfill 1.00\hfill & \hfill 0.84\hfill & \hfill 0.70\hfill \\ \hfill 0.67\hfill & \hfill 0.74\hfill & \hfill 0.77\hfill & \hfill 0.84\hfill & \hfill 1.00\hfill & \hfill 0.84\hfill \\ \hfill 0.65\hfill & \hfill 0.70\hfill & \hfill 0.75\hfill & \hfill 0.70\hfill & \hfill 0.84\hfill & \hfill 1.00\hfill \end{array}\right).$$

   The covariance matrix Σ of five outcomes across six visits is Σ = V ⨷ W, where ⨷ is a Kronecker product and the dimension of Σ is 30 × 30.

3. From the protocol of the LS-1 study, the expected one year outcome change for the placebo and treatment groups are d_P = (0.2, −2, 0.25, 3, −1.1)’ and d_T=(0.16, −1.6, 0.184, 2.4, −0.8)’, respectively. The mean vector across six visits for the placebo and treatment groups are ${\mu }_P={({\mu }_0^{\prime },{\mu }_0^{\prime }+d_p^{\prime },{\mu }_0^{\prime }+2d_p^{\prime },{\mu }_0^{\prime }+3d_p^{\prime },{\mu }_0^{\prime }+4d_p^{\prime },{\mu }_0^{\prime }+5d_p^{\prime })}^{\prime }$ and ${\mu }_T={({\mu }_0^{\prime },{\mu }_0^{\prime }+d_T^{\prime },{\mu }_0^{\prime }+2d_T^{\prime },{\mu }_0^{\prime }+3d_T^{\prime },{\mu }_0^{\prime }+4d_T^{\prime },{\mu }_0^{\prime }+5d_T^{\prime })}^{\prime }$, respectively.

4. Simulate 400 individuals in the placebo group from a multivariate normal distribution with mean μ_P and covariance matrix Σ.

5. Simulate 400 individuals in the treatment group from a multivariate normal distribution with mean μ_T and covariance matrix Σ.

6. If the simulated outcomes are out of the reasonable bounds illustrated in section 2.1, they are truncated to the closest bound. Round the outcomes Rankin, AC, PDQ-39, and SDM to the closest integers. Divide the outcome SEADL by 5 and round it to the closest integer because SEADL increments by 5.

7. Rescale outcomes SEADL and SDM so that larger values in all five outcomes represent worse clinical outcomes. To avoid that some categories have zero observations, we combine some categories in outcomes SEADL and AC so that they have seven and six categories, respectively.

We totally simulate 100 datasets. Let r_ijk (1 if missing, 0 otherwise) be the missing indicator of individual i’s outcome k at visit j. To simulate the missing data, we specify a model that describes the missing data process as follows.

$$\mathrm{logit}\left\{p\right({\mathit{r}}_{ijk}=1\left)\right\}={\varphi }_{k0}+{\varphi }_{k1}{y}_{ijk}+{\varphi }_{k2}{y}_{i,j-1,k}$$ (8)

where φ_k1 = 0 implies that missingness only depends on the observed data in the last visit and is ignorable dropout mechanism (missing at random, MAR), while φ_k1 ≠ 0 implies an outcome-dependent non-ignorable dropout mechanism (missing not at random, MNAR).³³ We do not simulate any missing data at baseline and hence j = 2, …, 6 in model (8). In the simulation studies, we investigate two missing data mechanisms (MAR and MNAR), and two missing data patterns, i.e. non-monotone (an individual can be missing at one visit and then measured again at later visit) and monotone (sequences of measurements on some individuals terminate prematurely). To generate monotone missing pattern, we exclude y_ij’k for j’ ≥ j if r_ijk = 1. We select the appropriate φ_k,MAR = (φ_k0, φ_k2)’ and φ_{k, MNAR} = (φ_k0, φ_k1, φ_k2)’ for k = 1, …, 5 so that the overall missing percentages are 20%, 30%, and 40% in all combinations of missing mechanisms and missing patterns.

After the datasets with missing data are simulated, we first compute the adjusted GST statistics T_a and GTE based on the LOCF method, which does not require multiple imputation. Next, we compute T_a and GTE based on the separate models and MI methods. Specifically, we first fit separate models to generate 100 multiply imputed full datasets for each simulated dataset with missing values, then compute the adjusted GST statistics T_a and GTE based on Rubin’s MI rules illustrated in section 3.2. Next, we fit IRT models (4) and (5) and include the binary treatment assignment variable in the mean model (6), i.e. μ_ij = [β₀ + β₁ I_i(trt)]t_j, to each simulated dataset. We run two chains with 10,000 iterations per chain. The first 5000 iterations are discarded as burn-in, and the inference is based on the remaining 5000 iterations. The MCMC convergence and mixing properties are assessed by visual inspection of the history plots of all parameters and the Gelman–Rubin statistics. In all simulation studies, the MCMC chains mix well after a burn-in of 5000 iterations. For each simulated dataset with missing data, we obtain M = 100 full datasets imputed by retaining simulated missing values every 100 iterations starting from the 5001th iteration on each chain. We then compute the T_a and GTE based on Rubin’s MI rules. To evaluate and compare the performance of the LOCF, separate models, and MI methods, we compute the root mean square error (RMSE) of T_a and GTE defined as $\sqrt{\frac{1}{100}{\Sigma }_n^{100}{({\widehat{T}}_{n,a}-\mu T_{n,a})}^2}$ and $\sqrt{\frac{1}{100}{\Sigma }_n^{100}{({\widehat{\psi }}_n-{\psi }_n)}^2}$ respectively, where the subscript n denotes the nth with missing data, ${\widehat{T}}_{n,a}$ and ${\widehat{\psi }}_n$ are the estimated adjusted GST and GTE using various methods, the asymptotic mean μT_n, (see Huang et al.,¹³ p.3090) and ψ_n are computed based on the nth simulated dataset without generating any missing data (i.e. the true dataset).

Table 1 displays the RMSE of T_a and GTE using the LOCF, separate models, and MI methods under various scenarios, i.e. the combinations of three missing percentages (20%, 30%, and 40%), two missing data mechanisms (MAR and MNAR), and two missing data patterns (non-monotone and monotone). Within each combination of missing data mechanism and missing data pattern, RMSE increases as the missing percentage increases. All methods perform better under MAR than under MNAR. While both the LOCF and separate models methods performs better under non-monotone than under monotone pattern, there is no obvious trend observed for the MI method in this comparison. In all simulation settings, the separate models method has the largest RMSE and the MI method always outperforms the LOCF method (except in the first scenario) with a larger advantage as the missing percentage increases.

Table 1.

Root mean square error (RMSE) of the adjusted GST statistics Ta and GTE for change from baseline to 5 years using the LOCF, separate models, and MI methods under various scenarios.

			Ta			GTE
Missing mechanism	Missing pattern	Missing %	LOCF	Separate	MI	LOCF	Separate	MI
MAR	Non-monotone	20	1.257	2.963	1.248	0.035	0.079	0.039
		30	2.078	3.714	1.631	0.057	0.098	0.049
		40	2.938	4.366	1.874	0.078	0.113	0.056
	Monotone	20	1.529	3.047	1.192	0.043	0.082	0.037
		30	2.370	3.884	1.632	0.064	0.101	0.049
		40	3.314	4.516	1.908	0.087	0.116	0.056
MNAR	Non-monotone	20	1.803	3.383	1.642	0.049	0.089	0.049
		30	2.639	4.164	2.233	0.070	0.107	0.063
		40	3.469	4.709	2.805	0.091	0.120	0.076
	Monotone	20	1.864	3.405	1.534	0.051	0.089	0.046
		30	2.717	4.184	1.976	0.073	0.108	0.057
		40	3.717	4.817	2.666	0.098	0.123	0.073

LOCF: last observation carried forward; MI: multiple imputation; GST: global statistical test; GTE: global treatment effect; MAR: missing at random; MNAR: missing not at random.

In the first simulation scenario, the RMSE of GTE based on the MI method is slightly larger than that based on the LOCF method. We have run additional simulation for this scenario. Specifically, we have doubled the expected one year outcome change in both groups to d_P = (0.4, −4, 0.5, 6, −2.2)’ and d_T = (0.32, −3.2, 0.368, 4.8, −1.6)’. The RMSE of T_a from methods LOCF, separate models, and MI are 4.044, 4.331 and 1.812 and the RMSE of GTE are 0.094, 0.101, and 0.055, respectively. The results suggest that the advantage of the proposed MI method over the LOCF and separate models methods increases as the outcome rates of change get larger. This is because the LOCF method assumes no disease progression after dropout and its performance deteriorates as the disease progression rate increases.

In conclusion, the LOCF method dampens the treatment–response relationships and produces misleading results. The separate models method fails to account for the inter-source and cross correlations. It is not surprising that the LOCF method outperforms the separate models method in the simulation studies because slow disease progression is simulated. In contrast, the proposed MI method based on the IRT model can account for the within-subject correlation across visits and successfully model the treatment effect of the multiple longitudinal outcomes. As a result, the proposed MI method performs better than the LOCF and separate models methods, with larger advantage as the missing percentage and the outcome rates of change increase.

5 Data analysis

In this section, we apply the proposed methodology to analyze a clinical trial dataset. Because the LS-1 study illustrated in section 2.1 is ongoing and the final dataset is not available until 2015, we use the data from a PD clinical trial with the data structure similar to the LS-1 study. DATATOP (Deprenyl And Tocopherol Antioxidative Therapy Of Parkinsonism) was a double-blind, placebo-controlled multicenter clinical trial to determine if deprenyl and/or tocopherol administered to patients with early PD will slow the progression of PD. Totally 800 individuals were randomly assigned in a 2 × 2 factorial design to receive double-placebo, active tocopherol alone, active deprenyl alone, and both active tocopherol and deprenyl. We combine the individuals who did not receive deprenyl (double-placebo and active tocopherol alone groups, 401 individuals) and refer to as placebo group. We combine the individuals who received deprenyl (active deprenyl alone and both active tocopherol and deprenyl groups, 399 individuals) and refer to as treatment group. The details of DATATOP trial can be found in Shoulson et al.⁵⁴

The outcomes collected include UPDRS (Unified Parkinson’s Disease Rating Scale) total score, Schwab and England activities of daily living (SEADL), Mini-Mental State Exam (MMSE), and Hamilton Depression Scale, measured at five visits, i.e. baseline, 1 month, 3 months, 9 months, and 15 months. UPDRS total score is the sum of 44 questions each measured on a 5-point scale (0–4) and it is approximated by a continuous variable with integer value from 0 (not affected) to 176 (most severely affected). SEADL is illustrated in section 2.1. Mini-Mental State Exam, a measurement of cognitive impairment, is an ordinal variable with integer value from 0 (severe) to 30 (normal). Hamilton Depression Scale, a depression test measuring the severity of clinical depression symptoms, is an ordinal variable with integer value from 0 (normal) to 52 (severe). For ordinal variables, we combine some categories with zero or small number of individuals and have 7, 7, and 10 categories in SEADL, MMSE, and Hamilton depression scale, respectively.

Before the end of the study, some individuals (222 and 154 individuals in the placebo and treatment groups, respectively) reached a level of functional disability sufficient to warrant the initiation of dopaminergic therapy, which is a symptomatic therapy to provide temporary relief of PD symptoms for a short period. In this case, only the observed outcomes before the initiation of dopaminergic therapy can be used in the assessment of treatment efficacy because dopaminergic therapy can significantly change the values of the outcomes collected. Therefore, these individuals would have missing data after the initiation of dopaminergic therapy and this missing mechanism is most likely MAR because the missingness depends only on the observed variables before therapy start. In addition, missing data occurred due to withdrawals, lost to follow-up, missed visits, etc. To visualize and explore the missing patterns in the dataset,⁵⁵ we plot the missingness map of four outcomes in the placebo and treatment groups in Figures 1 and 2, respectively. The observed values are plotted with dark gray and the missing values are in white. The sporadic white bars indicate non-monotone missing pattern. The numbers in the figures are the number of individuals and the corresponding percentage (in parenthesis). For example, in the placebo group, the numbers and percentages of individuals who have measurements of UPDRS up to 15 months, 9 months, 3 months, 1 month, and baseline only are 119 (29.7%), 130 (32.4%), 113 (28.4%), 30 (7.5%), and 6 (1.5%), respectively. Figures 1 and 2 suggest that most of the missing data are monotone. Although the overall missing percentages are roughly 24% and 16% in the placebo and treatment groups, respectively, the missing percentages at the last visit are around 70% and 50% in the placebo and treatment groups, respectively. In order to compute the adjusted GST and GTE based on the change from baseline to the last visit and follow the ITT principle, we need to impute the missing values.

Figure 1.

The missing patterns for four outcomes in the placebo group. Dark gray area indicates observed data while white area indicates that missing data. The sporadic white bars indicate non-monotone missing pattern. The numbers in the plots are the number of individuals and the percentage (in parenthesis) in the placebo group.

Figure 2.

The missing patterns for four outcomes in the treatment group. Dark gray area indicates observed data while white area indicates that missing data. The sporadic white bars indicate non-monotone missing pattern. The numbers in the plots are the number of individuals and the percentage (in parenthesis) in the treatment group.

In the LOCF analysis, we use the last observed value of the missing outcome to impute the observation at the last visit (15 months) and obtain the sample size of 401 and 398 in the placebo and treatment groups, respectively, after deleting one individual in the treatment group who did not have any data for UPDRS. In separate models analysis, we use linear mixed models and proportional odds models as imputation models for continuous and ordinal outcomes, respectively. In MI analysis, we use item-response models (4) and (5) and include the binary treatment assignment variable in the mean model (6), i.e. μ_ij = [β₀ + β₁ I_i(trt)]t_j. We have run two parallel chains with overdispersed initial values and 10,000 iterations per chain. After discarding a burn-in of 5000 iterations, we obtain 100 multiply-imputed datasets by taking every 100 iterations from the remaining 5000 iterations of each chain. To assess the efficacy of deprenyl, we compute the change from baseline to the last visit for all four outcomes, from which we compute the rank sum difference, the associated variance, and the adjusted GST statistics T_a and GTE using the LOCF, Separate, and MI methods.

Table 2 displays the estimates of the rank sum difference, the associated standard errors, the adjusted GST statistics T_a, p-values, and GTE based on the three methods. The treatment deprenyl shows significant efficacy in all three methods. After consolidating the 100 sets of estimates using Rubin’s rules, the MI method has the most negative rank sum difference, the most negative adjusted GST statistics T_a (with degree-of-freedom being 205), and the smallest p-value. The results of extremely small p-values are consistent with the published DATATOP study results.⁵⁴ The estimated GTE (i.e. 0.161) from the MI method is the largest among all three methods, indicating that the MI method estimates the largest global treatment effect from deprenyl. The interpretation of GTE is that an individual in the control groups would have a [(100% + 16.1% − P)/2 = (58.1% − P/2)] probability of having a better overall outcome if he/she had been assigned to the treatment group, where P is the probability that the placebo and treatment groups have tied observations. If no ties are observed, then the probability would be equal to 58.1%. Moreover, after fitting the IRT model, we obtain the following estimates of latent variable regression parameters, i.e. ${\widehat{\beta }}_0$ (0.179, sd: 0.013, CI: [0.152, 0.207]), and ${\widehat{\beta }}_1$ (−0.110, sd: 0.016, CI: [−0.140, −0.079]). The CI of ${\widehat{\beta }}_1$ not covering zero indicates the significant benefits of deprenyl in slowing PD progression. This is consistent with the significant adjusted GST statistics T_a in Table 2.

Table 2.

Results by various imputation methods for change of last visit.

Method	Rank sum difference (SE)	Ta	p-value	GTE
LOCF	−202.6 (37.8)	−5.4	8.32 e–8	0.127
Separate	−111.5 (40.1)	−2.8	5.71 e–3	0.070
MI	−257.1 (42.9)	−6.0	9.20 e–9	0.161

LOCF: last observation carried forward; MI: multiple imputation.

6 Discussion

In clinical trials for many complex diseases, a single outcome is often insufficient to determine the efficacy of treatments. In PD clinical trials, researchers have longitudinally measured multiple mixed type outcomes including quality of life, motor fluctuations, depression, and cognition, etc. During the course of follow-up period, missing values have become a common phenomena in clinical trial practice. In this article, we present a Bayesian method based on the item response theory (IRT) to impute the missing data in the multivariate longitudinal data structure, and the inference based on the adjusted GST and GTE is combined using Rubin’s MI rules. In the IRT models, the outcomes are representations of a latent variable which is a function of individual-specific covariates and visit time. All sources of correlation have been accounted for in the IRT modeling framework. We have performed sensitivity analysis under various scenarios. The simulation results suggest that the proposed MI method generally performs better than the standard methods such as last observation carried forward and separate random effects model for each outcome, with larger advantage as the missing percentage and the outcome rates of change increase. Although we did not compare it to other ad hoc methods such as imputation with the mean or imputation with the worst observation, it is likely that such single imputation methods are inferior because they fails to incorporate the missing-data uncertainty and do not account for all sources of correlation.³⁰ The proposed method offers a novel and flexible way to impute the missing multivariate longitudinal data and allows the imputation model to differ from the final analysis model. In the analysis of a PD clinical trial (DATATOP), the treatment of deprenyl has efficacious global effects across all four outcomes indicated by the significant adjusted global statistical test and the significant treatment effect coefficient in the IRT model.

Our modeling framework based on the item response theory has some limitations that we view as future research directions. Note that the discrimination parameter b_k controls both within-individual correlation in different outcomes and outcome-specific treatment effect β expressed in equation (6). If there is low within-individual correlation but a large treatment effect, this model may underestimate the treatment effect and overestimate the correlation.⁵⁶