Review and Evaluation of Imputation Methods for Multivariate Longitudinal data with Mixed-type Incomplete Variables

Summary

Estimating relationships between multiple incomplete patient measurements requires methods to cope with missing values. Multiple imputation is one approach to address missing data by filling in plausible values for those that are missing. Multiple imputation procedures can be classified into two broad types: joint modeling (JM) and fully conditional specification (FCS). JM fits a multivariate distribution for the entire set of variables, but it may be complex to define and implement. FCS imputes missing data variable-by-variable from a set of conditional distributions. In many studies, FCS is easier to define and implement than JM, but it may be based on incompatible conditional models. Imputation methods based on multilevel modeling show improved operating characteristics when imputing longitudinal data, but they can be computationally intensive, especially when imputing multiple variables simultaneously. We review current MI methods for incomplete longitudinal data and their implementation on widely accessible software. Using simulated data from the National Health and Aging Trends Study, we compare their performance for monotone and intermittent missing data patterns. Our simulations demonstrate that in a longitudinal study with a limited number of repeated observations and time-varying variables, FCS-Standard is a computationally efficient imputation method that is accurate and precise for univariate single-level and multilevel regression models. When the analyses comprise multivariate multilevel models, FCS-LMM-latent is a statistically valid procedure with overall more accurate estimates, but it requires more intensive computations. Imputation methods based on generalized linear multilevel models can lead to biased subject-level variance estimates when the statistical analyses involve hierarchical models.

1 |. INTRODUCTION

Missing data are often inevitable in longitudinal studies. A primary reason is that non-response can occur at any time in the study. Individuals’ responses may be missing because they have moved out of the area, missed an appointment, were too ill to attend, or died. In studies involving annual surveys, missing data also occur when participants refuse to answer or do not know the answer.

Commonly used statistically valid methods can be classified into three broad types: (1) likelihood and Bayesian methods; (2) weighting methods; and (3) imputation methods.¹ The likelihood and Bayesian methods define a model for the observed and unobserved variables. Using computational techniques such as the EM algorithm² or Data Augmentation,³ it provides estimates for the estimands of interest. These methods may result in biased estimates when the model is mis-specified or when the missing data mechanism is non-ignorable. Weighting is an alternative approach to handling missing data. This approach weights the observed data to account for missing observations using the estimated probabilities of non-response.^4–6 Weighting methods are best suited for monotone missing data patterns and are commonly used when the missing variable is scalar. In contrast to the former two methods, imputation methods explicitly “fill in” the missing values with plausible values. Single imputations have been shown to result in sampling variance estimates that are too small.⁷ Multiple imputation procedures circumvent this issue by replacing each missing value with a set of B plausible values drawn from the predictive distribution of the missing data. Based on these values, B sets of complete datasets are generated. Each dataset is analyzed separately, and final estimates are obtained using common combination rules.⁸

Although the idea behind multiple imputation seems simple, developing procedures to produce plausible values is more complex. Multiple papers have summarized and compared possible procedures to impute scalar variables,⁹ multiple non-clustered variables,^10–13 and multiple continuous longitudinal variables.^14–18 However, methods to impute mixed-type variables in longitudinal studies are more limited and dispersed. Kalaycioglu et al¹⁹ compared three chained equations imputation approach,²⁰ the multivariate Normal imputation,¹¹ and a Bayesian imputation approach to impute time-varying binary, categorical, skewed, and normally distributed variables. Huque et al²¹ presented a comparison study with twelve currently available imputation methods for longitudinal data with incomplete continuous and binary variables. Other studies compared imputation methods for continuous and binary multilevel data,^16,22–25 and multilevel categorical data²⁶ in non-longitudinal settings. This manuscript reviews available imputation methods for multiple variables of various types in longitudinal studies. Using simulations based on the National Health and Aging Trend Study (NHATS),²⁷ we compare the operating characteristics of different methods for handling missing values in both outcomes and explanatory variables. Code in R²⁸ for implementing all of the methods for the simulations and the real-data example is provided.²⁹

The paper proceeds as follows. Section 2 introduces the multiple imputation approach. Sections 3 and 4 review the fully conditional specification (FCS) imputation procedures and the joint modeling (JM) imputation procedures, respectively. Section 5 describes the simulation analyses and presents the results of the simulations. An application for estimating the associations between hospital admissions, physical, cognitive abilities, and skilled nursing facilities admissions by different imputation methods is demonstrated in Section 6. Section 7 provides a discussion and conclusions.

2 |. MULTIPLE IMPUTATION FOR LONGITUDINAL STUDIES

2.1 |. Notations and assumptions

Let Y = {y_ijk} represent a multivariate longitudinal dataset such that y_ijk is the value of variable Y_j, j ∈ {1,…, J}, for subject i ∈ {1,…, n} at time t_k, k ∈ {1,…,K}. The data Y can be saved in either a wide or long matrix format. In wide format, Y is stored as a n × JK matrix with each row representing an individual and each column corresponding to variable Y_j measured at time t_k, denoted by Y_jk. In long format, Y is a nK × (J + 1) matrix, with the last column describing the time point that the observation is recorded and the other columns are the J variables. For simplicity, we define Y_l, l ∈ {1,…,L}, to be a column in either the wide or long format. In wide format, L = JK and Y_l is a vector of size n. In long format, L = J and Y_l is a vector of size nK. Tables 1 and 2 present an example of data arranged in wide format and long format, respectively, with K = 4 time points and J = 3 variables. Question marks represent missing values.

TABLE 1.

Example of data arranged in the wide format

ID	Time t1			Time t2			Time t3			Time t4
	Y11	Y21	Y31	Y12	Y22	Y32	Y13	Y23	Y33	Y14	Y24	Y34
1	y 111	y 121	y 131	y 112	y 122	y 132	?	?	?	?	?	?
2	y 211	y 221	y 231	y 212	y 222	y 232	y 213	y 223	y 233	?	?	?
3	y 311	y 321	y 331	y 312	y 322	y 332	?	y 323	?	y 314	y 324	y 334

TABLE 2.

Example of data arranged in the long format

ID	Y1	Y2	Y3	Time
1	y 111	y121	y131	t 1
1	y 112	y 122	y 132	t 2
1	?	?	?	t 3
1	?	?	?	t 4
2	y 211	y 221	y 231	t 1
2	y 212	y 222	y 232	t 2
2	y 213	y 223	y 233	t 3
2	?	?	?	t 4
3	y 311	y 321	y 331	t 1
3	y 312	y 322	y 332	t 2
3	?	y 323	?	t 3
3	y 314	y 324	y 334	t 4

Let M = {m_ijk} be a matrix of indicators, such that m_ijk = 0 when y_ijk is observed and m_ijk = 1, otherwise. In addition, let $Y_l^{\mathit{\text{obs}}}$ and $Y_l^{\mathit{\text{mis}}}$ be the observed (m_ijk = 0) and the missing (m_ijk = 1) parts of Y_l, respectively. Monotone and intermittent missing data are two missing data patterns that are commonly observed in longitudinal data.⁷ Monotone missing data occurs when subjects drop out from the study and do not return for follow-up appointments. If subject i drops out from the study at time point t_k, then m_ijk* = 1, ∀j ∈ {1, …, J} and k* ≥ k. Intermittent missing data usually occur when subjects skip an interview or refuse to answer certain questions. In such case, m_ijk = 1 for any j ∈ {1, …, J} and k ∈ {1,…,K}. Imputation of $Y_l^{\mathit{\text{mis }}}$ involves modeling the relationships between Y_l, complete covariates X = {x_ip}, p ∈ {1, .., P}, and all the other L − 1 variables, Y_−l.

To obtain valid inferences when data comprise missing values, one should consider the missing data mechanism,⁷ P(M|Y,x,ϕ), where ϕ are the parameters governing this distribution. In many software packages, the default method for missing values is the list-wise deletion procedure, which assumes that the data are missing completely at random (MCAR). MCAR implies that the missing data mechanism is unrelated to missing and observed values, P(M|Y,x,ϕ) = P(M|ϕ). In aging research, older adults may be too ill to complete the study, and assuming MCAR can lead to biased estimates. One way to relax the MCAR assumption is to assume that missing data depends only on observed values, also know as missing at random (MAR), P(M|Y,x,ϕ) = P(M|Y^obs,x,ϕ), where $Y^{\mathit{\text{obs}}}=\left\{Y_l^{\mathit{\text{obs}}}\right\}$. Under MAR, a variety of analytic strategies to address missing data can be considered. The third type of missing data mechanism is not missing at random (NMAR). Under NMAR, missing data depends on both the observed and unobserved values. Because missing data depends on unobserved values, methods to handle missing data under NMAR rely on assumptions that are not verifiable from the observed data. Thus, many authors emphasized the need for sensitivity analysis to assess inferences under different plausible assumptions.^1,30 This paper describes multiple imputation methods that assume that the missing data mechanism is MAR, and it examines their performance when these methods are applied to multiple incomplete longitudinal variables of different types.

2.2 |. Multiple imputation for multivariate data

Assuming MAR, a multiple imputation (MI) procedure generates B plausible values for each missing value resulting in B complete datasets. Each complete dataset is analyzed separately, and point and interval estimates are obtained using common combination rules. The procedure for creating imputations for all partially observed variables in Y consists of three steps:

1. Specify the model P(Y₁, …, Y_L|θ,x) and prior distributions of parameters P(θ) and calculate the posterior distribution of θ based on Y^obs;

2. Draw a value θ* from its posterior distribution P(θ|Y^obs,x);

3. Draw imputations $Y_1^{*},\dots ,Y_L^{*}$ from the conditional posterior predictive distribution of Y^mis given Y^obs, θ*, and x, P(Y^mis|Y^obs,θ*,x).

After the imputation for all incomplete variables is completed, researchers can conduct any statistical analysis that they would have performed on a complete dataset. Using common combination rules, an estimate for a scalar parameter of interest, β, is derived as the average of ${\widehat{\beta }}^{(u)}$, u ∈ {1,…,B}, which is the estimate of β within complete dataset u. And its sampling variance, $\text{var}(\widehat{\beta })$, is estimated by summing the average sampling variances within imputation and the variance between imputations⁸. Formally,

$$\widehat{\beta }=\frac{1}{B}\sum _{u=1}^B{\widehat{\beta }}^{(u)},\text{var}(\widehat{\beta })=\frac{1}{B}\sum _{u=1}^B{U}_B+\left(1+\frac{1}{B}\right)W_B,\text{ where }{U}_B=\sum _{u=1}^B\widehat{\text{var}}\left({\beta }^{(u)}\right),W_B=\frac{1}{B-1}\sum _{u=1}^B{\left({\widehat{\beta }}^{(u)}-\widehat{\beta }\right)}^2.$$ (2.1)

Two main strategies for specifying a distribution of all incomplete variables have been proposed: joint modeling (JM) and fully conditional specification (FCS).¹ The JM approach defines a multivariate distribution for all incomplete variables, P(Y₁,…,Y_L|θ,x). The FCS approach specifies a set of univariate conditional distributions for each incomplete variable given the other variables, {P(Y_l|θ_l,Y_−l,x)}. Compared to the JM approach, the FCS approach provides more flexibility for imputing different types of variables, but it may suffer from theoretical limitations, because the joint distribution based on the different conditional models may not exist.¹³

Under either the FCS or JM strategy, the imputation models used to create $Y_l^{*}$ depend on the data format. With long format, multilevel models are commonly used as imputation models, where the variance at the subject level captures the correlation between repeated measurements.^31–33 In wide format data, single-level models are specified to impute an incomplete variable at a specific time point. The correlation between the repeated measurements of an incomplete variable is accounted for by adjusting for its values measured at all the other time points as constant effects.³⁴ This approach assumes an unstructured correlation structure between the repeated measurements of the incomplete variables. While a wide format can be used when the same observation is recorded for the same unit over time, it may be impractical when the same observation is recorded for different units, but units are grouped within clusters. The imputation methods using wide format data should generally be used for balanced longitudinal data in which information on all individuals is recorded over similar intervals. When individuals’ reporting is recorded over different time intervals, special care should be given to the time between reports.

3 |. IMPUTATION BY FULLY CONDITIONAL SPECIFICATIONS

The FCS approach is also referred to multivariate imputation by chained equations (MICE). MICE iterates through sampling from the conditional posterior distributions of model parameters, $P\left({\mathit{\theta }}_l\mid Y_l^{\mathit{\text{obs }}},Y_{-l},X\right)$, and sampling from the conditional posterior predictive distributions $P\left(Y_l^{\mathit{\text{mis}}}\mid Y_l^{\mathit{\text{obs}}},Y_{-l},X,{\mathit{\theta }}_l^{*}\right)$. Formally, the t-th iteration of MICE involves sampling

$$\begin{array}{lll}{\mathit{\theta }}_1^{*(t)}\hfill & ~\hfill & P\left({\mathit{\theta }}_1\mid Y_1^{obs},Y_{-1}^{(t-1)},X\right),\hfill \\ {\mathbf{Y}}_1^{*(t)}\hfill & ~\hfill & P\left(Y_1\mid Y_1^{obs},Y_{-1}^{(t-1)},X,{\mathit{\theta }}_1^{*(t)}\right),\hfill \\ \hfill & ⋮\hfill & \hfill \\ {\mathit{\theta }}_L^{*(t)}\hfill & ~\hfill & P\left({\mathit{\theta }}_L\mid Y_L^{obs},Y_{-L}^{(t)},X\right),\hfill \\ Y_L^{*(t)}\hfill & ~\hfill & P\left(Y_L\mid Y_L^{obs},Y_{-L}^{(t)},X,{\mathit{\theta }}_L^{*(t)}\right).\hfill \end{array}$$ (3.1)

Because the conditional models in MICE may not represent a joint distribution, there are no theoretically supported methods to assess convergence. One possible method to assess convergence is by examining whether the imputation of each variable has converged. For example, by examining the convergence of a summary statistic that utilizes the imputed values in each incomplete variable over multiple chains.³⁵ The MICE algorithm is implemented in the mice package³⁶ in R, where researchers specify a set of univariate imputation models for each incomplete variable. Other available software for implementing the MICE algorithm include IVEware,³⁷ PROC MCMC in SAS with FCS statement. In all of these implementations, FCS model specifications commonly depend on the data format.

3.1 |. FCS using wide format data

For wide format longitudinal data, imputation models treat repeated measurements of an incomplete variable Y_j at time t_k as K distinct variables, {Y_jk;k ∈ {1, …,K}}. If Y_j contains missing values at and after time t_k, then variables {Y_jt;∀t ≥ k} are incomplete variables. To impute a missing variable Y_jk, the imputer specifies a linear regression model or a generalized linear model where Y_jk is the dependent variable, and x, the variables {Y_jt;∀t ≠ k} and all the other variables at all time points $\left\{Y_{\tilde{j}t};\forall \tilde{j}\ne j,t\in \left\{1,\dots ,K\right\}\right\}$ are the independent variables. Another method for imputing incomplete continuous and count variables is the predictive mean matching (PMM) procedure.⁴ PMM is a semi-parametric method that imputes data using observed values, making it less sensitive to model mis-specification than purely parametric methods.³⁸ For count variables with large number of zeros, the zero-inflated Poisson and the zero-inflated Negative-Binomial models can be used.³⁹ These models are implemented in the countimp package.⁴⁰ The application of MICE to wide data format has been referred to as FCS-Standard²¹ or as imputation by chained equations with fixed effects regression models (ICE-FS).¹⁹

As the number of waves in longitudinal studies increases, FCS-Standard can result in numerical instabilities because of the lack of identification that arises from specifying many explanatory variables in the conditional models. Nevalainen et al⁴¹ proposed to impute variables recorded at time t_k only with variables that are recorded within (t_k −δ, t_k +δ) time window. This procedure assumes that a partially recorded variable observed at time t_k is independent from variables recorded at time t_k ±(δ +ϵ) (ϵ > 0) conditional on variables recorded at time t_k ± δ. This reduces the number of covariates used within each conditional model. Additional details of this FCS method are provided in Welch et al.⁴²

3.2 |. FCS– Multilevel linear model

When data are saved in long format, multilevel linear models have been proposed as conditional imputation models for continuous variables. The first level of the model describes the repeated observations of subjects across time, and it is nested within a second-level, which is subject-level information. Formally, to impute the missing observation of a continuous variable Y_l at time point k for the i-th subject, the following multilevel linear model is used,

$$y_{ilk}=X_{ik}^T{\mathit{\beta }}_l+Z_{ik}^T{b}_i+e_i,b_i~N_q\left(0,V_b\right),e_i~N\left(0,{\sigma }_e^2\right),$$ (3.2)

where x_ik and z_ik are p×1 vector of covariates and q×1 vector of subject-level covariates, respectively, and V_b is an unstructured covariance matrix of the subject-level effects. Both x and z may comprise complete and other incomplete variables. Parameters β_l are regression coefficients corresponding to covariates x, and b_i correspond to subject-level variations in covariates Z. Model (3.2) assumes a common conditional variance ${\sigma }_e^2$ across all subjects. Applying MICE with Model (3.2) involves sampling ${\theta }_l^{*}=\left({\beta }_l^{*},V_b^{*},{\sigma }_e^{2*}\right)$, subject-level effects $b_i^{*}$, and imputation values $y_{ilk}^{*}$. Commonly, conjugate prior distributions are assumed for ${\theta }_l^{*}$. With continuous incomplete variables, a multivariate Gaussian prior distribution for β_l, an Inverse-Wishart distribution for V_b, and an Inverse-Gamma distribution for ${\sigma }_e^2$ are assumed. Samples of b_i can be drawn from its conditional posterior distribution N(μ_bi,Ψ_bi), with ${\mu }_{bi}=V_b{Z}_i^T{\left(Z_i{V}_b{Z}_i^T+{\sigma }_e^2{I}_K\right)}^{-1}\left(y_{il}-X_i^T{\beta }_l\right)$ and ${\mathit{\text{Ψ}}}_{bi}=V_b-V_b{Z}_i^T{\left(Z_i{V}_b{Z}_i^T+{\sigma }_e^2{I}_K\right)}^{-1}{Z}_i{V}_b$, where y_il = (y_il1, ⋯,y_ilK)^T is the responses of subject i measured at all K time points and I_K denotes a K ×K identity matrix. Imputed values of $y_{ilk}^{*}$ are sampled from Model (3.2) given ${\theta }_l^{*}$ and $b_i^{*}$. The sampling procedure relies on a MCMC algorithm, which can be computationally burdensome because many samples are required for the chain to converge to its equilibrium distribution.⁴³ A possible approximation procedure samples ${\theta }_l^{*}$ from their large sample Normal approximation and $b_i^{*}$ from $N\left({\mu }_{b_i},{\mathit{\text{Ψ}}}_{b_i}\right)$.^16,25

To impute discrete variables, one approach is to sample from multilevel linear models and round the imputed continuous values to the nearest valid discrete values. However, this rounding step can result in biased estimates.⁴⁴ To address this, Yucel et al⁴⁵ have proposed a calibration method that is similar to posterior predictive checks in Bayesian analysis⁴⁶ to improve imputed rounded values.

3.3 |. FCS– Multilevel linear model with latent variables

Another approach for imputing binary and categorical variables is to sample from a multilevel linear model with latent variables.^47,48 For a binary variable Y_l, this model assumes that there is a latent Normal variable ${\tilde{Y}}_l$, such that y_ilk = 1 if ${\tilde{y}}_{ilk}>0$, and y_ilk = 0 otherwise. The latent variable is assumed to follow a multilevel linear model. This representation is equivalent to assuming that y_ilk follows a multilevel probit model. Formally,

$${\Phi }^{-1}\left(P\left(y_{ilk}=1\mid X_{ik},b_i,{\mathit{\theta }}_l,\tau \right)\right)={\tilde{y}}_{ilk}=X_{ik}^T{\mathit{\beta }}_l+Z_{ik}^T{b}_i+e_i,b_i~N_q\left(0,V_b\right),e_i~N(0,1),$$ (3.3)

where Φ⁻¹ is the inverse cumulative distribution function (CDF) of the standard Normal distribution. The sampling procedure for drawing imputations from Model (3.3) is the same as the procedure for Model (3.2), which can be implemented by the function mice.impute.2l.jomo in the micemd R package.

For ordinal categorical variables Y_l with H > 2 levels, the latent variable imputation model is based on a cumulative probit model. The model assumes that Y_l is determined by a latent Normal variable ${\tilde{Y}}_l$ partitioned by H − 1 threshold parameters τ = {τ_h}, h ∈ {1, …,H}, such that y_il = h, if ${\tau }_{h-1}<{\tilde{y}}_{il}<{\tau }_h\left({\tau }_0=-\infty ,{\tau }_H=\infty \right)$. In addition, the model assumes that the latent variable ${\tilde{Y}}_l$ follows a multilevel linear model as Model (3.3). Formally, a cumulative probit model of Y_l is defined as

$${\Phi }^{-1}\left(P\left(y_{ilk}\le h\mid X_{ik},b_i,{\mathit{\theta }}_l\right)\right)={\Phi }^{-1}\left(P\left({\tilde{y}}_{ilk}<{\tau }_h\mid X_{ik},b_i,{\theta }_l\right)\right)={\tau }_h-\left(X_{ik}^T{\mathit{\beta }}_l+Z_{ik}^T{b}_i\right).$$ (3.4)

The sampling procedure at iteration t starts with updating ${\tau }^{(t)}~N\left(\tau \mid Y_l,{\tilde{Y}}^{(t-1)},{\theta }_l^{(t-1)},b_i^{(t-1)}\right)$ followed by sampling ${\tilde{Y}}_l~TN\left({\tilde{Y}}_l\mid Y,{\tau }^{(t)},{\theta }_l^{(t-1)},b_i^{(t-1)}\right)$ (a truncated Normal distribution) for all subjects, where τ^(t) are the truncation parameters at iteration t, and ${\theta }_l^{(t-1)}={\left({\beta }_l^{(t-1)},V_b^{(t-1)}\right)}^T$, and $b_i^{(t-1)}$ are the sample of θ_l = (β_l, V_b)^T and b_i at iteration t − 1, respectively. Samples of the parameters ${\theta }_l^{(t)}$ are drawn from its conditional posterior distribution $P\left({\theta }_l\mid Y_l,{\tilde{Y}}_l^{(t)},{\tau }^{(t)},b_i^{(t-1)}\right)$ and $b_i^{(t)}$ are drawn from $P\left(b_i\mid Y_l,{\tilde{Y}}_l^{(t)},{\tau }^{(t)},{\theta }_l^{(t)}\right)$. The conjugate prior distributions that are used for Model (3.3) are commonly specified for θ_l and b_i. Additional technical details are provided in Enders et al.⁴⁸

The multilevel multinomial probit model⁴⁹ can be used to impute a nominal categorical variable Y_l with H_l categories. This model expands Y_l into H_l binary variables $Y_l^h$, h ∈ {1, …,H_l}, that indicate whether y_il = h for subject i. An underlying latent Normal variable ${\tilde{Y}}_l^h$ corresponding to $Y_l^h$ is defined by the probability of $y_{il}^h=1$. If ${\tilde{y}}_{il}^h$ is greater than ${\tilde{y}}_{il}^{h*}$ for all h* ≠ h, then $y_{il}^h=1$ and y_il = h. For identifiability purposes, a multivariate linear multilevel model is assumed for the first H_l − 1 latent variables, such that b_i ~ MV N(0, V_b) and the within-subject variance Σ_e is the identity matrix. The sampling procedure is similar to Model (3.3), except that ${\tilde{Y}}_l^h$ are generated by an accept-reject algorithm.⁴⁷ These models for categorical variables using latent Normal variables are implemented in the software Blimp.⁵⁰ We refer to the multilevel linear models with latent variables as FCS-LMM-latent.

3.4 |. FCS– Multilevel generalized linear model

The multilevel generalized linear models are a flexible approach to model skewed or non-normally distributed variables. These models are also commonly referred to as generalized linear mixed models (GLMM). Assuming that an incomplete variable Y_l conditional on item-level covariates X and Z has an exponential family probability density function or probability mass function. A GLMM is defined as

$$g\left(E\left(y_{ilk}\mid X_{ik},Z_{ik},b_i,{\mathit{\theta }}_l\right)\right)=X_{ik}^T\beta +Z_{ik}^T{b}_i,b_i~N_q\left(0,V_b\right),$$ (3.5)

where g(·) is a function linking the expected value of response y_ilk to linear predictors. Sampling from the posterior distribution of P(θ_l|X,Z,Y_l) can be implemented using MCMC. The latent multilevel variable model in Section 3.3 can be viewed as Model (3.5) with a probit link function. The probit model can perform well for binary and categorical variable, but may not suitable for skewed or count variables. A different commonly used link function is the logit link for binary or the log-link for count variables. Compared to the probit link, sampling from P(b_i|y_i,x_i, θ_l) with the logit or log link functions can be more complex. A possible approximation can be obtained by sampling b_i from its marginal posterior distribution N(0,Ψ_b), where Ψ_b is estimated by the REML²³ or the Fisher scoring method.¹⁶ Using GLMM models for imputation of binary and count missing variables are implemented in the micemd package in R. Availability of software that implements multilevel generalized linear models using the log-Normal or Gamma likelihoods for incomplete skewed continuous data are limited. Throughout, we refer to this method as FCS-GLMM.

4 |. IMPUTATION BY JOINT MODELING

4.1 |. JM– General location model

The JM approach specifies a multivariate distribution for all incomplete variables. A multivariate Normal distribution is often used when the data are arranged in wide format and consist of only continuous variables.^14,17 For a mixture of continuous and discrete variables, the general location model is proposed as a possible imputation model.¹¹ This model describes the joint distribution of Y = (W,C) in terms of a marginal distribution for all discrete variables, $W=\left(W_1,\dots ,W_{S_1}\right)$, and a conditional distribution of all continuous variables $C=\left(C_1,\dots ,C_{S_2}\right)$ given the discrete variables, where S₁ + S₂ = L(= JK). The general location model is defined as

$$\begin{gathered} P(W,C)=P(W)P(C\mid W) \\ =\mathit{\text{Multinomial}}\left(N,{\pi }_d\right)\cdot \mathit{\text{Normal}}\left({\mu }_d,\mathit{\text{Σ}}\right). \end{gathered}$$ (4.1)

The marginal distribution P(W) of S₁ discrete variables is modeled by a multinomial distribution on the cell counts of a S₁-dimensional contingency table with $D=\prod _{l=1}^{S_1}{H}_l$ cells and cell probabilities π_d, d ∈ {1, ..,D}, where H_l is the number of distinct levels of variable W_l. Within each cell of the contingency table, continuous variables C follow a multivariate Normal distribution with mean μ_d and covariance matrix Σ_d. In finite samples, as the number of categorical variables increases, some cells may be empty. This may lead to unstable estimation.⁵¹ In these situations, the restricted general location model can be applied. The restricted model assumes that a contingency table cell counts follow a log-linear model, which is fitted by a subset of W_l, l ∈ {1, …, S₁}, and possibly their interactions. Continuous variables are modeled by a multivariate linear regression model with the categorical variables as the independent variables. Another possible limitation of both the general location model and the restricted general location model is their reliance on the multivariate Normal distributions for continuous Y_l. This may result in inaccurate imputation when Y_l is skewed or multi-modal. The general location model and the restricted location model are implemented in the mix package⁵² in R. Throughout, we refer to the general location model as JM-GL.

4.2 |. JM– Multivariate multilevel linear model

When Y comprise only continuous variables and is in a long format, a possible joint model for imputing Y is the multivariate linear multilevel model. Let y_ik = (y_i1k, ⋯, y_iLk)^T be a column vector of L continuous responses of the i-th subject measured at time point k. The multivariate multilevel linear model (MLMM) is

$$\begin{gathered} y_{ik}=\left(I_L\otimes X_{ik}^T\right)\mathit{\beta }+\left(I_L\otimes Z_{ik}^T\right)b_i+e_i, \\ {b}_i~N\left(0,V_b\right),e_i~N\left(0,{\mathit{\text{Σ}}}_e\right), \end{gathered}$$ (4.2)

where the column vector β has pL elements and column vector b_i has qL elements. The symbol ⊗ is the Kronecker product. Schafer and Yucel¹⁷ proposed a MCMC procedure to sample b_i, θ, and Y^mis jointly. It assumed that Σ ~ Inv-Wishart(υ₁,Λ₁), V_b ~ Inv-Wishart(υ₂,Λ₂), and an improper uniform density over ℛ^PL for β. This model is implemented in R package pan.⁵³

4.3 |. JM-Multivariate multilevel linear model with latent variables

For imputations of both continuous and categorical variables, a multivariate multilevel linear model (MLMM) can be used to model latent Normal variables that correspond to each of the categorical variables together with other continuous variables. Formally, a MLMM assumes that a set of incomplete continuous variables, $y_{ik}^{(c)}={\left(y_{i1k}^{(c)},\dots ,y_{iCk}^{(c)}\right)}^T$, and a set of latent variables ${\tilde{y}}_{ik}^{(w)}={\left({\tilde{y}}_{i1k}^{(w)},\dots ,{\tilde{y}}_{iWk}^{(w)}\right)}^T$ of categorical variables $y_{ik}^{(w)}={\left(y_{i1k}^{(w)},\dots ,y_{iWk}^{(w)}\right)}^T$ are distributed as

$$\begin{gathered} y_{ik}^{(c)}=\left(I_C\otimes X_{ik}^T\right){\mathit{\beta }}_c+\left(I_C\otimes Z_{ik}^T\right)b_{ci}+e_{ci}, \\ {\tilde{y}}_{ik}^{(w)}={\Phi }^{-1}\left(P\left(y_{ik}^{(w)}=1\right)\right)=\left(I\otimes X_{ik}^T\right){\mathit{\beta }}_w+\left(I_W\otimes Z_{ik}^T\right)b_{wi}+e_{wi}, \\ {b}_i={\left(b_{ci},b_{wi}\right)}^T~N\left(0,V_b\right),e_i={\left(e_{ci},e_{wi}\right)}^T~N\left(0,{\mathit{\text{Σ}}}_e\right),{\text{ where Σ}}_e=\left(\begin{array}{cc}{\sigma }_e^2{I}_C& \mathit{\text{cov}}\left(e_c,e_w\right)\\ \mathit{\text{cov}}\left(e_c,e_w\right)& I_W\end{array}\right). \end{gathered}$$ (4.3)

Under this model, a vector stacking the latent variables ${\tilde{y}}^{(w)}$ and the continuous variables y^(c) follows a multivariate Normal distribution, where V_b is an unstructured covariance matrix of the subject-level effects, and the covariance matrix Σ_e captures the associations between the two sets of variables. The imputation algorithm is similar to sampling from the Model (4.2). However, Σ_e can not be sampled from the Inverse-Wishart distribution. Instead, the elements of Σ_e should be updated individually using a Metropolis-Hastings procedure. Detailed descriptions of the imputation algorithm are provided in Carpenter and Kenward⁵⁴ chapters 4–5. We refer to this model as JM-MLMM-latent and it is implemented in the jomo package in R⁵⁵ and the REALCOM program in MATLAB.⁵⁶

4.4 |. JM– Multivariate generalized multilevel linear model

Extending the FCS-GLMM to its JM version is an another approach for handling mixed-type incomplete variables. Let y_i = {y_i1, ⋯, y_iL} be a K × L response matrix of subject i, which consists of different types of variables. We assume a multivariate generalized linear mixed model (MGLMM) for P(Y₁, …,Y_l|θ) defined as

$$p\left(y_{i1},\dots ,y_{iL}\mid \theta \right)=\int \prod _{l=1}^L{p}_l\left(y_{il}\mid b_{il},{\theta }_l\right)p_b\left(b_i\mid V_b\right)d{b}_i,$$ (4.4)

where p_l(·) are density functions, b_i = (b_i1, …, b_iL)^T is a vector of subject-level effects which follows a multivariate Normal distribution with mean zero and covariance matrix V_b. The Model (4.4) links a set of univariate generalized linear multilevel models by introducing correlations between the variance components of the subject-level effects. The model assumes that the Y_ls are independent conditional on b_i and x. We refer to this model as JM-MGLMM. For example, a shared-random intercepts model with two outcomes (L = 2) in Equation (4.4) is

$$\begin{gathered} E\left(y_{i1k}\mid b_{i0},X_{ik},{\beta }_{01}\right)=g_1^{-1}\left({\beta }_{01}+b_{i10}+{\beta }_1{X}_{ik}\right), \\ E\left(y_{i2k}\mid b_{i0},X_{ik},{\beta }_{02}\right)=g_2^{-1}\left({\beta }_{02}+b_{i20}+{\beta }_2{X}_{ik}\right), \\ {b}_{i0}=\left(b_{i10},b_{i20}\right)~N_2\left(0,V_b\right),V_b=\left(\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right), \end{gathered}$$ (4.5)

where g₁ and g₂ are link functions for outcomes Y₁ and Y₂, respectively, and the latent correlation of outcomes at the subject-level is identified by ρ. A possible extension of Model (4.5) involves adding random slopes in E(y_i1k|b_i0) and E(y_i2k|b_i0). Including all of the covariates as subject-level random slopes in b_i = (b_i10, …, b_i1p, …, b_iL0, …, b_iLp) would increase the dimension of V_b to (L × (1 + p))². To complete the Bayesian model, diffused prior distributions can be used. Specifically, β_p ~ N(0, 100), V_b ~ Inverse-Wishart(I_q, q) where q is the cardinality of b_i. Sampling from posterior distributions of the parameters can be implemented using the JAGS software⁵⁷, which requires the users to specify both the prior distributions and the likelihood functions. We have provided a code example on the GitHub website.

5 |. SIMULATIONS

The National Health and Aging Trend Study (NHATS) collects information on a nationally representative sample of Medicare beneficiaries ages 65 and older. Beginning in 2011, annual interviews are conducted and detailed information on a broad range of variables related to sociodemographic factors, physical, cognitive capacity, and health outcomes are collected. We use the NHATS data collected from 2011 to 2014 (rounds 1–4) and select a set of variables including four incomplete longitudinal variables of varying types: an indicator of whether a person had an overnight hospital stay, a person’s body mass index (BMI), comorbidity index, and the count of devices paid to assist with daily activities during the past year (paid assistive devices). The comorbidity index is defined as a count of 10 chronic conditions, including heart disease (e.g., angina or congestive heart failure), hypertension, arthritis, osteoporosis, diabetes, lung disease, Alzheimer’s disease, and related dementia, cancer, and whether they experienced a heart attack or a stroke in the past. The number of paid assistive devices (ranges from 0–9) includes the following aids: vision aids, hearing aid, cane, walker, wheelchair, scooters, grabbers, special dress items, and adapted utensils. Almost 98% of the missing values resulted from loss to follow-up. Patients’ information collected at the first interview is used as covariates for imputation and analysis, including age, gender, self-rated health, an indicator for whether participants take prescribed medicines, and an indicator for whether participants or their spouse/partner have any medical bills that are being paid off over time. Our goal is to estimate the associations between the comorbidity index, BMI, the paid assistive devices, and the hospital stay status after adjustment for baseline variables.

We design a simulation study to evaluate the operating characteristics of different imputation methods described in Sections 3 and 4. Our simulations are based on 5309 participants that were observed for 4 all rounds, and we simulate different missing data patterns.

5.1 |. Missing data mechanism and missing data patterns

In the simulation, we assume that the missing data mechanism is MAR for the monotone missing data pattern. In longitudinal studies with many covariates, it is reasonable to believe that P(m_ijk|x_i, Y_i1,…, Y_iL) would depend on at least one missing Y_iL value for some i, where Y is recorded in a wide format. Thus, for intermittent missing data, we assume an NMAR missing data mechanism. We simulate both missing data patterns on the wide-formatted NHATS data composed of only the completed cases. Missing data indicators are sampled from the Bernoulli distribution with the event probabilities predicted by models estimated from the original NHATS data.

To simulate an intermittent missing data pattern, we generate the missing indicators m_ijk = 1 for each of the incomplete variables independently. For subject i at round k ∈ {2, 3, 4}, the probability of Y_ijk being missing is

$$P\left(m_{ijk}=1\mid Y_i,X_i\right)={\text{logit}}^{-1}\left({\alpha }_{0jk}^{*}+\sum _{j=1}^J\sum _{k=1}^{k-1}{\widehat{\alpha }}_{jk}^{*}{y}_{ijk}+\sum _{p=1}^P{\widehat{\psi }}_{jp}{x}_{ip}\right),$$ (5.1)

where the covariates include all of the time-varying variables prior to round k and time-invariant variables. We set ${\alpha }_{0jk}^{*}$ to ensure that the average missing proportions of Y_jk at round 2, 3, and 4 are 20%, 35%, and 40%, respectively. The coefficients ${\widehat{\alpha }}_{jk}^{*}$ and ${\widehat{\psi }}_{jp}$ are the maximum likelihood estimates of Model (5.1) using the original NHATS data.

To simulate a monotone missing data pattern, we generate the drop-out indicators for each subject. Let r_ik represent a drop-out indicator for subject i at round k ∈ {2, 3, 4}. If subject i drops out at round k*, then r_ik* = 1 and values for subject i are not observed at round k* and in subsequent rounds for all variables (m_ijk = 1, ∀k ≥ k* and ∀j ∈ {1, …, J}). The probability that subject i is lost to follow-up at round k is

$$P\left(r_{ik}=1\mid Y_i^{\mathit{\text{obs}}},X_i\right)={\text{logit}}^{-1}\left({\tilde{\alpha }}_{0k}+\sum _{j=1}^J\sum _{k=1}^{k-1}{\widehat{\alpha }}_{jk}{y}_{ijk}+\sum _{p=1}^P{\widehat{\psi }}_p{x}_{ip}\right),$$ (5.2)

where the covariates include all of the time-varying variables that are fully observed priorto round k and time-invariant variables. The intercept ${\tilde{\alpha }}_{0k}$ is set to ensure a pre-specified proportion of participants who drop out. Based on the original NHATS data, we set the proportion of participants who start to drop out from the study at rounds 2, 3, and 4 to 20%, 15% and 10%, respectively. Cumulatively, the proportion of individuals with missing information at round 4 is approximately 45%. The coefficients ${\widehat{\alpha }}_{jk}^{*}$ and ${\widehat{\psi }}_{jp}$ are the maximum likelihood estimates of Model (5.2) using the drop-out indicators observed in NHATS.

5.2 |. Study design

We consider three configurations with different numbers and types of incomplete variables. The variables include a binary hospital stay status (Y₁), a discrete bounded comorbidity index (Y₂), a continuous BMI (Y₃), and a discrete count number of paid assistive devices (Y₄). Configuration 1 assumes that Y₁ and Y₂ are incomplete and the other two variables are fully observed. Configuration 2 assumes that in addition to Y₁ and Y₂, Y₃ is incomplete. Lastly, Configuration 3 assumes that all four variables are incomplete. Because monotone missing data pattern is observed for most of the individuals in NHATS, we evaluate the performance of the different imputation methods using 500 simulated incomplete datasets with monotone missing data patterns for each of the three configurations. We also compare the different methods on 300 incomplete simulated datasets with intermittent missing data patterns for Configuration 3.

For every simulated dataset, we conduct a multiple imputation procedure with B = 5 imputations using 10 imputation methods with different choices of imputation models described in Section 3 and 4. For the FCS-Standard method, we consider three imputation models to impute the count variables: linear regressions, predictive mean matching, and Poisson regressions. When applying FCS-LMM-latent, we use latent variable models for binary variables and linear models for continuous and count variables. For the FCS-GLMM method, we specify a logit link for binary variables, an identity link for continuous variables, and either an identity or a log link for count variables. When applying JM-MLMM-latent, we consider both homoscedastic and heteroscedastic within-subject variance. For JM-MGLMM, we assume a logit link for binary variables, an identity link for continuous variables, and a log link for count variables. A complete list of the different methods is provided in Table 3.

TABLE 3.

Summary of imputation methods for mixed-type longitudinal data

Approach	Data format	Method	Imputation models
FCS			Imputation model for binary variables	Imputation model for count variables
	Wide	FCS-Standard (LM)	Logistic regression	Linear regression
		FCS-Standard (PMM)	Logistic regression	Predictive mean matching
		FCS-Standard (Poisson)	Logistic regression	Poisson regression
	Long	FCS-LMM-latent	Multilevel linear regression on latent variables	Multilevel linear regression
		FCS-GLMM (Gaussian)	Multilevel logistic regression	Multilevel linear regression
		FCS-GLMM (Poisson)	Multilevel logistic regression	Multilevel Poisson regression
JM	Wide	JM-GL	General location model
	Long	JM-MLMM-latent (common)	Multivariate multilevel linear model with latent variables and homoscedastic within-subject variance
		JM-MLMM-latent (random)	Multivariate multilevel linear model with latent variables and heteroscedastic within-subject variance
		JM-MGLMM	Multivariate multilevel generalized linear model using a logit and log link for binary, count variables

Imputing the missing data is usually performed as part of the data preparation process, and the ultimate goal is to generate unbiased estimates of the associations and conditional associations between variables. Our simulations mimic situations in which no specific statistical analysis is specified prior to the imputation, and the imputed datasets are used for multiple analyses. We conduct three types of analyses: univariate generalized hierarchical model, latent growth model, and bi-variate generalized hierarchical model. An overview of the simulation is presented in Table 4.

TABLE 4.

Summary of simulation design

Configuration	Incomplete variables	Missing Pattern	Statistical Analysis
1	Hospital stay (Binary)	Monotone	Univariate GLMM
	Comorbidity index (Count)
2	Hospital stay (Binary)
	Comorbidity index (Count)	Monotone	Univariate GLMM
	BMI (Continuous)
3	Hospital stay (Binary)		Univariate GLMM Growth Curve Model Bivariate GLMM
	Comorbidity index (Count)	Monotone
	BMI (Continuous)	Intermittent
	Number of devices paid for caring (Count)

5.2.1 |. Univariate generalized linear hierarchical model

Within each imputed dataset, a multilevel logistic regression model is used to model the conditional associations between the comorbidity index and the hospital stay status,

$$\text{logit}\left(P\left(y_{i1k}=1\mid \mathit{\gamma },b_{0i},X_i,t_{ik},y_{i2k},y_{i3k},y_{i4k}\right)\right)={\gamma }_0+b_{0i}+{\gamma }_1{y}_{i2k}+{\gamma }_2{y}_{i3k}+{\gamma }_3{y}_{i4k}+{\gamma }_4{t}_{ik}+X_i^T{\gamma }_p,$$ (5.3)

where X_i comprises of five baseline covariates for subject i, t_ik is the round of time that individual i is being interviewed, $b_{0i}~N\left(0,{\sigma }_b^2\right)$ denotes the subject-level effects, and γ_l, l ∈ {0, …, 8}, denotes a set of unknown coefficients. Let ${\widehat{\gamma }}_1^{(u)},{\widehat{\gamma }}_2^{(u)},{\widehat{\gamma }}_3^{(u)}$ be the estimates of Model (5.3) within imputed data set u = {1,…, 5}. and ${\widehat{\sigma }}_{{\gamma }_1}^2,{\widehat{\sigma }}_{{\gamma }_2}^2,{\widehat{\sigma }}_{{\gamma }_3}^2$ be their corresponding sampling variances. The final estimates are obtained using the common combination rules described in Section 2.

5.2.2 |. Latent growth curve model

Researchers may be interested in the trajectory of individuals over time. For the imputed datasets in Configuration 3, we fit a latent linear growth curve model (LGCM) on the trajectory of BMI over time. The model includes all the time-invariant variables and the time-varying comorbidity index Y₂ as the predictors. The LGCM requires the data to be structured in wide format, and it can be broken down into two latent constructs, the intercept factor and the slope factor.⁵⁸ Let t_i ∈ {0, 1, 2, 3} denote the round that subject i is observed. The adjusted LGCM is expressed by a multilevel model that consists of an intercept model, π_0i and a slope model, π_1i,

$$\begin{gathered} y_{i3t}={\pi }_{0i}+{\pi }_{1i}{t}_i+{ϵ}_{it}, \\ {\pi }_{0i}={\eta }_{00}+{\eta }_{01}{y}_{i2t}+X_i^T{\mathit{\eta }}_0+e_{0i}, \\ {\pi }_{1i}={\eta }_{10}+{\eta }_{11}{y}_{i2t}+X_i^T{\mathit{\eta }}_1+e_{1i}, \end{gathered}$$ (5.4)

where the residuals of the intercept and slope models are (e_0i,e_1i)^T ~ N(0₂,Σ), and Σ is an unknown unstructured covariance matrix.

5.2.3 |. Bivariate generalized linear hierarchical model

In some studies, researchers are interested in examining the associations between multiple factors and multiple outcomes simultaneously.^59–61 This can be achieved by jointly modeling multiple outcomes. Compared to a univariate model, joint models are computationally more complex. We examine a joint model comprising of hospital stay (Y₁) and paid assistive devices (Y₄) on datasets with imputed comorbidity index (Y₂) and BMI (Y₃) for Configuration 3. The bivariate multilevel generalized linear model used in the analysis is

$$\begin{gathered} E\left(y_{i1k}\mid b_{i0},y_{i2k},y_{i3k},X_i,{\lambda }_{10},{\lambda }_{11},{\lambda }_{12}\right)=\text{logi}{\text{t}}^{-1}\left({\lambda }_{10}+b_{i10}+{\lambda }_{11}{y}_{i2k}+{\lambda }_{12}{y}_{i3k}+{\lambda }_1{X}_{ik}\right), \\ E\left(y_{i4k}\mid b_{i0},y_{i2k},y_{i3k},X_i,{\lambda }_{20},{\lambda }_{21},{\lambda }_{22}\right)=\text{exp}\left({\lambda }_{20}+b_{i20}+{\lambda }_{21}{y}_{i2k}+{\lambda }_{22}{y}_{i3k}+{\lambda }_2{X}_{ik}\right), \\ \text{where }{b}_{i0}={\left(b_{i10},b_{i20}\right)}^T~N_2\left(0,{\mathit{\text{Σ}}}_b\right),{\mathit{\text{Σ}}}_b=\left(\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right)\mathit{\text{, }} \end{gathered}$$ (5.5)

where a logit link function is applied to hospital stay and the log link function to the paid assistive devices. The subject-level effect b_i0 = (b_i10, b_i20)^T is assumed to follow a bivariate Normal distribution with zero means and an unstructured covariance matrix Σ_b. The correlation between the separate random intercepts ρ represents the interdependence between the two outcomes at the subject-level.

5.2.4 |. Congeniality of Analysis Models

Congeniality⁶² between the different imputation methods and the three analyses varies. Not all of the imputation procedures had the same or more general model specification compared to the three analysis models. For the univariate GLMM, methods that include multilevel modeling encompass the analysis model, whereas methods based on the wide format data are mis-specified. For the bivariate GLMM analysis, JM-MLMM and JM-MGLMM methods are the only two methods that encompass the analysis model. For the latent growth curve, all imputation methods are mis-specified because time was not adjusted for in any of the imputation models.

5.2.5 |. Performance assessment metrics

Estimates of the three analyses are obtained using the common combination rules described in Section 2. For each configuration and each replication we estimate the relative bias $\left({\widehat{\theta }}^{\mathit{\text{meth }}}-{\widehat{\theta }}^{\mathit{\text{comp }}}\right)/{\widehat{\theta }}^{\mathit{\text{comp }}}$, where ${\widehat{\theta }}^{\mathit{\text{meth }}}$ is the estimate obtained after implementation of multiple imputation procedure meth described in Table 3, and ${\widehat{\theta }}^{\mathit{\text{comp }}}$ is the estimate from the complete dataset. We record the root of mean squared error (RMSE), the 95% interval estimate width, whether the interval estimate covers the estimate with complete data. Additionally, we estimate the fraction of missing information (FMI) that measures the uncertainty in the imputed values for missing elements.^8,63 For each parameter estimate, the FMI estimate is

$${\widehat{\lambda }}_m=\frac{r_m+2/(v+3)}{r_m+1},$$ (5.6)

where $r_m=\left(1+m^{-1}\right)\frac{W_B}{U_B}$, $v=(m-1){\left(1+\frac{1}{r}\right)}^2$. W_B and U_B are calculated using Equation (2.1), and represent the variance between the m complete-data estimates and the average of the m complete-data variances, respectively. We summarize these metrics by averaging across all replications.

5.3 |. Results of the univariate GLMM across all configurations

5.3.1. | Relative bias of coefficients and the subject-level variance

Figure 1 presents the relative bias of regression coefficients estimates associated with the incomplete variables and the subject-level variance estimate, ${\sigma }_b^2$, for study design Configurations 1–3 with monotone missing data patterns. The first row depicts the change in relative bias of ${\widehat{\gamma }}_1$, the conditional log-odds ratio of having a hospital stay with one unit increase in the comorbidity index, as the number of incomplete variables increases from two to four. The boxplots of relative biases are similar for each method as the number of incomplete predictors increases. Across Configurations 1–3, JM-GL results in the smallest mean relative bias of −0.02. However, it has the largest variability in relative biases of ${\widehat{\lambda }}_1$ compared to other methods for all configurations. FCS-Standard with either linear regressions (LM) or with predictive mean matching (PMM), and FCS-LMM-latent result in comparable mean relative biases to JM-GL. Across Configurations 1–3, the averages relative bias for LM is between −0.04 to −0.03, and −0.03 to −0.02 for PMM and FCS-LMM-latent. Imputation methods that assume Poisson regressions for the count variables, FCS-Standard, FCS-GLMM, and JM-MGLMM have mean relative biases that are greater than −0.29. The JM-MLMM methods generally have the second largest relative bias. Across Configurations 1–3, JM-MGLMM leads to the largest mean relative bias of approximately −0.40.

FIGURE 1.

Relative bias of regression coefficients estimates associated with the incomplete covariates and the random effect estimate. Each column represents a simulation setting. From left to right, the column corresponds to Configuration 1 to Configuration 3.

Similar trends are observed for the relative bias of ${\widehat{\gamma }}_2$ in Configurations 2 and 3 (second row of Figure 1). The mean relative biases produced by FCS-Standard with either LM or PMM, FCS-LMM-latent, and JM-GL are close to zero. All of the methods show higher variability in relative biases for ${\widehat{\gamma }}_2$ compared to those observed for ${\widehat{\gamma }}_1$ and ${\widehat{\gamma }}_3$. The interquartiles (IQR) of the relative biases of ${\widehat{\gamma }}_2$ in Configurations 2–3 are above 0.20 for all of the imputation methods, whereas the IQRs of the relative bias for ${\widehat{\gamma }}_1$ and ${\widehat{\gamma }}_3$ are around 0.05 and 0.08 on average. This is because BMI is not significantly associated with hospital stay status after adjustment for the other covariates. In Configuration 3, the mean relative biases of ${\widehat{\gamma }}_3$ after using FCS-standard with Poisson regressions, FCS-GLMM, and JM-MGLMM with multilevel Poisson regressions are 0.007, −0.004 and −0.124, respectively. In contrast, the mean relative biases of ${\widehat{\gamma }}_1$ produced by these methods ranges from −0.3 to −0.4.

The relative bias of the subject-level variance, ${\sigma }_b^2$, is depicted in the last row of Figure 1. All of the methods present similar trends across all configurations. The FCS-LMM-latent has the smallest mean relative bias compared to all other methods. The mean relative bias produced by JM-GL is close to zero; however, the variability of the relative bias across replications is the largest. JM-MLMM performs similarly to JM-GL. The two FCS-GLMM methods lead to the largest mean relative biases for estimating ${\sigma }_b^2$ compared to all other methods. Their mean relative biases across all configurations are greater than 0.17.

The relative biases of regression coefficient estimates associated with fully observed predictors are presented in Figure 2. The relative biases of statistically significant coefficients (p-value<0.001) show similar trends for all methods. Generally, the average values and the variability of the relative biases across replications are smaller compared to the relative biases of the statistically insignificant coefficients (the top four rows compared to the bottom two rows). In Configuration 3, the mean relative bias of FCS-Standard is close to zero for all fully observed covariates, except for FCS-Standard with Poisson regression. FCS-LMM-latent has a similar performance to FCS-Standard methods other than the one with Poisson regressions. JM-MGLMM and JM-MLMM have large relative biases for the coefficients of gender, an individual’s status of taking prescribed medicines, and having medical bills being paid off over time.

FIGURE 2.

Relative bias of regression coefficients estimates associated with the fully observed predictors. Each column represents a simulation setting. From left to right, the column corresponds to Configuration 1 to Configuration 3.

5.3.2 |. The RMSE, Interval Width and Coverage

Because all of the methods have produced similar trends across all configurations, we only present the RMSEs, Interval widths, and 95% coverage rates for all imputations methods in Configuration 3 (Table 5). Results for Configurations 1 and 2 are provided in the Appendix. Generally, FCS-LMM-latent, FCS-Standard with either LM or PMM, and JM-GL outperform the other methods in terms of RMSE and average interval width across all regression coefficients. FCS-LMM-latent has the smallest RMSE when estimating γ₁, γ₂, γ₇, γ₈, γ₉, and smaller RMSEs when estimating other coefficients compared to FCS-Standard with Poisson regression, FCS-GLMM, JM-MLMM, and JM-MGLMM. Across all methods, the average interval widths for all coefficients are similar, and the differences in RMSE are mainly driven by biases.

TABLE 5.

The root mean squared error (RMSE), average interval width and empirical coverage of the 95% CI of coefficients estimates and subject-level variance estimate for the univariate analysis in Configuration 3 with the monotone missing data pattern. The incomplete predictors are marked in bold.

Estimates	Metrics	FCS-Standard (LM)	FCS-Standard (PMM)	FCS-Standard (Poisson)	FCS-LMM-latent	FCS-GLMM (Gauss.)	FCS-GLMM (Poisson)	JM-General Location	JM-MLMM (common)	JM-MLMM (random)	JM-MGLMM
Intercept ${\widehat{\gamma }}_0^{\mathit{\text{comp}}}=-{3.89}^{**}$	RMSE	0.150	0.154	0.149	0.156	0.255	0.237	0.176	0.131	0.165	0.125
	Width	1.157	1.136	1.127	1.110	1.039	1.033	1.094	1.131	1.144	1.055
	Coverage(%)	100	99	100	100	99	100	100	100	100	100
Comorbidity Index ${\widehat{\gamma }}_1^{\mathit{\text{comp}}}={0.38}^{**}$	RMSE	0.021	0.020	0.136	0.019	0.025	0.112	0.020	0.031	0.071	0.152
	Width	0.117	0.118	0.117	0.117	0.110	0.105	0.113	0.117	0.113	0.100
	Coverage(%)	99	100	0	100	100	0	99	100	22	0
BMI ${\widehat{\gamma }}_2^{\mathit{\text{comp}}}=-{0.02}^{*}$	RMSE	0.003	0.004	0.004	0.003	0.004	0.003	0.004	0.004	0.006	0.005
	Width	0.029	0.029	0.028	0.028	0.026	0.026	0.027	0.028	0.030	0.027
	Coverage(%)	100	100	100	100	100	100	100	100	99	100
Paid assistive devices ${\widehat{\gamma }}_3^{\mathit{\text{comp }}}={0.36}^{**}$	RMSE	0.026	0.022	0.021	0.026	0.019	0.018	0.025	0.019	0.052	0.048
	Width	0.149	0.152	0.150	0.150	0.142	0.141	0.144	0.149	0.151	0.143
	Coverage(%)	100	100	100	99	100	100	99	100	87	92
Age ${\widehat{\gamma }}_4^{\mathit{\text{comp}}}={0.11}^{**}$	RMSE	0.014	0.014	0.015	0.016	0.018	0.015	0.019	0.017	0.018	0.012
	Width	0.112	0.112	0.111	0.111	0.104	0.101	0.107	0.112	0.111	0.106
	Coverage(%)	100	100	100	100	100	100	100	100	100	100
Gender ${\widehat{\gamma }}_5^{\mathit{\text{comp}}}={0.28}^{**}$	RMSE	0.044	0.041	0.073	0.056	0.048	0.072	0.053	0.040	0.054	0.090
	Width	0.310	0.311	0.306	0.307	0.282	0.282	0.299	0.307	0.304	0.287
	Coverage(%)	100	100	99	100	100	96	100	100	100	96
Self-rated health ${\widehat{\gamma }}_6^{\mathit{\text{comp}}}={0.20}^{**}$	RMSE	0.024	0.024	0.052	0.024	0.021	0.049	0.028	0.022	0.040	0.072
	Width	0.162	0.162	0.164	0.157	0.146	0.146	0.155	0.157	0.158	0.148
	Coverage(%)	100	99	95	100	100	93	100	100	98	60
Have med. bill paid off overtime ${\widehat{\gamma }}_7^{\mathit{\text{comp}}}={0.34}^{*}$	RMSE	0.064	0.066	0.060	0.048	0.058	0.066	0.131	0.093	0.102	0.067
	Width	0.483	0.483	0.480	0.473	0.444	0.435	0.462	0.473	0.475	0.451
	Coverage(%)	100	100	100	100	100	100	93	99	99	100
Took prescribed med. ${\widehat{\gamma }}_8^{\mathit{\text{comp }}}=0.30$	RMSE	0.095	0.096	0.129	0.071	0.096	0.156	0.110	0.155	0.206	0.183
	Width	0.702	0.714	0.692	0.666	0.647	0.644	0.667	0.684	0.674	0.650
	Coverage(%)	100	100	100	100	100	100	99	100	99	98
Time ${\widehat{\gamma }}_9^{\mathit{\text{comp}}}={0.10}^{**}$	RMSE	0.020	0.022	0.048	0.016	0.027	0.023	0.021	0.016	0.017	0.018
	Width	0.109	0.108	0.108	0.104	0.102	0.105	0.099	0.105	0.106	0.100
	Coverage(%)	99	100	66	100	96	98	99	100	100	100
Random effect ${\widehat{\sigma }}_{b_{\mathit{\text{comp}}}}^2={1.13}^{**}$	RMSE	0.048	0.044	0.070	0.037	0.202	0.220	0.050	0.035	0.040	0.107
	Width	0.235	0.239	0.233	0.238	0.217	0.212	0.221	0.241	0.240	0.220
	Coverage(%)	98	99	95	100	0	0	94	100	99	56

The interval coverages of all coefficients estimated using FCS-Standard methods, FCS-LMM-latent, and JM-GL are above nominal. The two JM-MLMM methods produce comparable RMSE, average interval widths, and coverages to FCS-LMM-latent only for γ₂ and γ₄. However, JM-MLMM with heteroscedastic Σ_ei has significantly lower than nominal coverage for γ₁ and γ₃. Compared to other methods, FCS-GLMM and JM-MGLMM with multilevel Poisson regressions have shorter interval widths for all coefficients and comparable RMSE for most coefficients. However, they result in large RMSEs and below nominal coverages when estimating γ₁ and ${\sigma }_b^2$.

5.3.3 |. Results of GLMM for intermittent missing data pattern

In Configuration 3 with intermittent missing data, the FCS-standard methods and JM-GL perform similarly to the one observed for the monotone missing pattern, in terms of the mean relative biases (MRB), interval coverages, and RMSE for most of the coefficients (Appendix table A1). Comparing the monotone to the intermittent missing data pattern, we observe an increase in the RMSE of the coefficients of the incomplete variables for FCS-LMM-latent and FCS-GLMM, and their 95% coverage rates for the co-morbidity index coefficient decrease to 85% (FCS-LMM-latent) and 44% (FCS-GLMM). The performances of the JM-MLMM and JM-MGLMM also deteriorate with intermittent missing data. Specifically, their average 95% coverage rates for the coefficients of the incomplete variables, γ₁ − γ₃, drop from above nominal to below 60%. The two FCS-GLMM methods have higher relative biases, wider interval widths, and interval estimates that do not cover the true parameter for the subject-level variance estimate.

5.4 |. Results of the latent growth curve model for Configuration 3

The results of the mean relative biases, RMSE, and average interval coverages of the estimated coefficients associated with the slope factor at each time point are presented in Table 6. Overall, the performances of the JM methods are inferior to those of the FCS methods. Among all methods, JM-MGLMM results in the largest mean relative biases and below nominal interval coverages for all coefficients. Among the FCS methods, the FCS-Standard methods outperform the FCS-LMM-latent and FCS-GLMM. For the comorbidity index coefficient after round 2, FCS-Standard methods result in nominal average interval coverages, while FCS methods with hierarchical modeling result in below nominal coverages. For intermittent missing data patterns, the average interval coverages of FCS-LMM-latent and FCS-GLMM are below 75%. Comparing the results between monotone and intermittent missing data patterns, we observe lower relative biases and better interval coverages for the monotone missing data pattern for all of the FCS methods and JM-GL across all coefficient estimates. JM-MLMM with heteroscedastic variances and JM-MGLMM result in larger relative biases for the intercepts at rounds 3 and 4 in simulations with intermittent missing data patterns, and their interval estimates do not cover the true intercepts.

TABLE 6.

The mean relative bias (MRB), the root mean squared error (RMSE), average empirical coverage of the 95% CI, average FMI estimates of slope factor’s coefficients estimates for the latent growth curve analysis in Configuration 3 with intermittent (Inter.) and monotone (Mono.) missing data patterns.

Slope Estimates	Metrics	FCS-Standard (LM)		FCS-Standard (PMM)		FCS-Standard (Poisson)		FCS-LMM-latent		FCS-GLMM (Gaussian)		FCS-GLMM (Poisson)		JM-General Location		JM-MLMM (common)		JM-MLMM (random)		JM-MGLMM
		Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.	Inter.	Mono.
Round 1 Intercept	MRB	−0.03	0.00	−0.02	0.00	−0.03	−0.01	0.11	0.06	0.11	0.06	0.11	0.06	−0.03	−0.01	0.11	0.05	0.16	0.32	0.89	0.05
	RMSE	0.21	0.16	0.21	0.17	0.21	0.16	0.22	0.15	0.22	0.15	0.22	0.15	0.21	0.16	0.22	0.15	0.31	0.56	1.63	0.15
	Coverage(%)	74	85	77	87	72	86	60	85	60	84	58	88	71	83	61	88	64	26	13	93
	FMI	0.40	0.32	0.47	0.46	0.38	0.31	0.31	0.24	0.29	0.26	0.29	0.25	0.26	0.23	0.30	0.25	0.44	0.68	0.08	0.34
Round 1 Comorbidity	MRB	−0.43	0.08	−0.48	0.40	−2.15	1.96	−4.01	−1.57	−4.14	−1.54	−4.07	−2.87	−0.48	−0.15	−1.19	−0.05	−1.57	4.84	−12.53	−2.47
	RMSE	0.03	0.02	0.03	0.02	0.04	0.04	0.06	0.03	0.06	0.03	0.06	0.05	0.03	0.02	0.03	0.02	0.11	0.20	0.19	0.04
	Coverage(%)	100	100	100	100	94	94	70	98	65	98	44	82	97	100	100	100	93	97	0	96
	FMI	0.42	0.31	0.42	0.31	0.45	0.53	0.37	0.26	0.35	0.28	0.38	0.50	0.25	0.23	0.38	0.28	0.84	0.92	0.14	0.41
Round 2 Intercept	MRB	0.01	0.01	0.02	0.00	0.02	0.01	0.00	0.01	0.00	0.01	0.00	0.02	0.01	0.01	0.00	0.02	0.46	0.37	4.75	0.23
	RMSE	0.18	0.16	0.19	0.16	0.18	0.16	0.12	0.12	0.12	0.12	0.12	0.12	0.18	0.16	0.12	0.13	0.94	0.76	9.50	0.48
	Coverage(%)	64	71	84	81	66	71	83	79	85	79	85	80	60	66	83	78	09	11	0	5
	FMI	0.42	0.36	0.74	0.59	0.43	0.34	0.31	0.28	0.34	0.28	0.32	0.29	0.30	0.27	0.34	0.29	0.93	0.89	0.24	0.45
Round 2 Comorbidity	MRB	−0.23	−0.08	−0.24	0.01	−0.86	0.41	−1.34	−0.61	−1.37	−0.59	−1.39	−1.06	−0.23	−0.12	−0.45	−0.30	−0.36	1.37	−2.31	−0.93
	RMSE	0.03	0.02	0.04	0.02	0.05	0.03	0.08	0.04	0.08	0.04	0.08	0.06	0.04	0.02	0.04	0.03	0.13	0.23	0.14	0.06
	Coverage(%)	99	100	98	100	77	95	39	94	38	96	9	62	91	100	99	99	89	96	9	63
	FMI	0.52	0.33	0.53	0.33	0.60	0.64	0.44	0.32	0.44	0.34	0.52	0.64	0.32	0.25	0.47	0.34	0.90	0.95	0.24	0.51
Round 3 Intercept	MRB	−0.04	−0.01	−0.02	−0.03	−0.04	−0.01	0.17	0.10	0.17	0.10	0.17	0.10	−0.04	−0.01	0.17	0.10	1.09	0.74	9.47	0.52
	RMSE	0.20	0.14	0.20	0.14	0.19	0.14	0.30	0.20	0.31	0.20	0.30	0.21	0.20	0.14	0.30	0.20	1.85	1.26	16.04	0.89
	Coverage(%)	62	76	76	83	62	75	13	43	14	44	19	43	52	70	18	45	0	0	0	0
	FMI	0.55	0.39	0.77	0.62	0.53	0.39	0.39	0.35	0.40	0.34	0.41	0.34	0.36	0.29	0.41	0.36	0.97	0.94	0.22	0.50
Round 3 Comorbidity	MRB	−0.21	−0.11	−0.22	−0.05	−0.66	0.13	−1.01	−0.50	−1.03	−0.48	−1.07	−0.79	−0.21	−0.13	−0.44	−0.36	−0.31	0.76	−0.99	−0.73
	RMSE	0.04	0.02	0.04	0.02	0.07	0.03	0.10	0.05	0.10	0.05	0.10	0.08	0.05	0.03	0.05	0.04	0.16	0.24	0.10	0.07
	Coverage(%)	95	99	97	99	54	97	16	81	17	82	1	39	82	99	90	92	84	93	46	25
	FMI	0.59	0.37	0.60	0.37	0.71	0.73	0.49	0.35	0.49	0.37	0.60	0.70	0.36	0.29	0.52	0.37	0.92	0.96	0.31	0.54
Round 4 Intercept	MRB	−0.03	−0.02	−0.04	−0.03	−0.02	−0.01	0.10	0.02	0.10	0.02	0.10	0.02	−0.03	−0.02	0.09	0.02	1.04	0.60	9.99	0.45
	RMSE	0.33	0.21	0.33	0.21	0.33	0.21	0.24	0.14	0.24	0.14	0.24	0.14	0.34	0.22	0.23	0.14	2.03	1.19	19.36	0.88
	Coverage(%)	62	78	70	87	63	79	72	92	72	91	69	92	54	70	73	91	1	5	0	0
	FMI	0.61	0.40	0.76	0.70	0.61	0.43	0.42	0.32	0.42	0.32	0.41	0.33	0.40	0.30	0.43	0.32	0.95	0.88	0.16	0.45
Round 4 Comorbidity	MRB	−0.32	−0.16	−0.33	−0.10	−0.81	0.02	−1.06	−0.50	−1.09	−0.47	−1.14	−0.82	−0.30	−0.17	−0.48	−0.42	−0.29	0.80	−1.18	−0.76
	RMSE	0.05	0.03	0.06	0.03	0.08	0.04	0.10	0.05	0.11	0.05	0.11	0.08	0.06	0.03	0.05	0.05	0.18	0.26	0.12	0.07
	Coverage(%)	93	99	94	99	58	97	23	87	22	88	3	46	76	97	92	93	84	93	50	35
	FMI	0.61	0.39	0.63	0.39	0.74	0.74	0.49	0.34	0.50	0.36	0.59	0.65	0.36	0.30	0.53	0.35	0.91	0.95	0.35	0.52

5.5 |. Results of the multivariate GLMM for Configuration 3

The relative biases of the coefficients estimates of the hospital stay outcome are summarized in Figure 3. For all imputation methods, estimates of model parameters generally have larger relative biases when using a joint analysis model compared to the univariate model analysis. FCS-LMM-latent has the smallest mean relative bias and narrower IQR of relative biases for all coefficients except for the coefficient of the indicator variable of having any prescribed medicine. JM-GL has comparable performance to FCS-LMM-latent in terms of having small mean relative biases, but it has larger variability in relative biases for all coefficients. JM-MLMM and JM-MGLMM lead to small relative biases only in coefficients that are statistically significant (age, BMI, gender, self-rated health). FCS-Standard with either LM or Poisson regressions have higher mean relative biases for coefficients of incomplete variables compared to their performance in the univariate analysis. FCS-Standard with PMM leads to mean relative biases close to zero for coefficients of BMI, gender, self-rated health, and having any prescribed medicine, but it has the widest IQRs of relative biases compared to other imputation methods.

FIGURE 3.

Relative bias of regression coefficients estimates for hospital stay.

Table 7 summarizes the RMSEs, average interval widths and coverages for the coefficients that represent the associations between incomplete explanatory variables and the two incomplete outcomes. When estimating the conditional association between comorbidity index and the binary outcome hospital stay, λ₁₁, FCS-GLMM with identity links for count variables, FCS-LMM-latent, FCS-GLMM with identity links for count variables, JM-GL, and JM-MLMM have interval coverages above 95%, with small RMSE of approximately 0.04. However, these methods produce larger RMSEs (> 0.11) and lower than nominal interval coverages when estimating the conditional associations between the comorbidity index and the paid assistive devices, λ₂₁. FCS-Standard, FCS-GLMM and JM-MGLMM, which assume Poisson regressions for count variables, lead to small RMSE (0.01, 0.02, 0.01, respectively) and interval coverages around nominal (96%, 92%, 98%, respectively). When estimating the conditional association between BMI and the two outcomes, FCS-LMM-latent produces the smallest RMSE for λ₂₁ and λ₂₂, and it has interval coverages above nominal. Generally, all methods lead to higher RMSE when estimating λ₂₁ and λ₂₂ compared to the estimation of λ₁₁ and λ₁₂. This is because of the insignificant association between BMI and the number of paid assistive devices.

TABLE 7.

The root mean squared error (RMSE), average interval width and empirical coverage of the 95% CI of coefficients estimates of the incomplete predictors and the correlation estimate for the joint analysis in Configuration 3.

Estimates	Metrics	FCS-Standard (LM)	FCS-Standard (PMM)	FCS-Standard (Poisson)	FCS-LMM-latent	FCS-GLMM (Gauss.)	FCS-GLMM (Poisson)	JM-General Location	JM-MLMM (common)	JM-MLMM (random)	JM-MGLMM
Outcome 1: Hospital stay (Yes)
Comorbidity ${\widehat{\lambda }}_{11}^{\mathit{\text{comp}}}={0.44}^{**}$	RMSE	0.064	0.160	0.143	0.043	0.038	0.113	0.070	0.046	0.059	0.193
	Width	0.216	0.890	0.520	0.482	0.768	0.614	0.596	1.140	0.633	0.339
	Coverage	81	88	85	100	100	80	100	100	100	35
BMI ${\widehat{\lambda }}_{12}^{\mathit{\text{comp }}}=-{0.02}^{**}$	RMSE	0.006	0.008	0.006	0.004	0.005	0.005	0.006	0.005	0.007	0.007
	Width	0.032	0.062	0.049	0.042	0.055	0.052	0.043	0.074	0.055	0.039
	Coverage	99	100	100	100	100	100	100	100	100	99
Outcome 2: Paid assistive devices
Comorbidity ${\widehat{\lambda }}_{21}^{\mathit{\text{comp }}}={0.086}^{**}$	RMSE	0.016	0.028	0.013	0.020	0.019	0.016	0.020	0.021	0.109	0.011
	Width	0.051	0.054	0.057	0.052	0.048	0.05	0.049	0.051	0.256	0.049
	Coverage	87	44	96	81	76	92	75	79	78	98
BMI ${\widehat{\lambda }}_{22}^{\mathit{\text{comp }}}=0.003$	RMSE	0.003	0.002	0.003	0.002	0.002	0.002	0.003	0.002	0.005	0.003
	Width	0.012	0.014	0.012	0.012	0.012	0.012	0.013	0.011	0.021	0.013
	Coverage	100	100	99	100	100	100	97	97	99	99
Correlation between the two outcomes
$\widehat{\rho }=0.30$	RMSE	0.038	0.037	0.037	0.035	0.045	0.051	0.044	0.035	0.039	0.045
	Width	0.231	0.226	0.230	0.230	0.252	0.241	0.220	0.243	0.233	0.221
	Coverage	100	100	100	100	100	98	100	100	100	97

For the correlation estimate between the two outcomes, ρ, the FCS-LMM-latent and JM-MLMM have the smallest RMSE compared to the other methods. However, all methods have relatively similar interval estimates which lead to interval coverages that are above nominal.

5.6 |. Fraction of missing information and computational time

The FMI estimates of the univariate GLMM analysis are summarized in Table 8. Generally, the average FMI estimates from simulations with monotone missing data patterns are smaller than those with intermittent missing data patterns for all imputation methods. When estimating the coefficients associated with incomplete variables and the subject-level variance, larger differences between the FMI estimated from the intermittent and monotone missing data patterns are observed for FCS-based methods and JM-MLMM with heteroscedastic variances compared to the other imputation methods. Across all simulation scenarios, the range of the average FMI estimates is between 0.14 and 0.51. Most imputation methods have average FMI estimates between 0.20 to 0.30. Among all of the methods, JM-GL results in the smallest FMI estimates for all of the constant effects coefficients.

TABLE 8.

The fraction of missing information estimated for the univariate GLMM analysis in Configuration 3.

Estimates	Missing pattern	FCS-Standard (LM)	FCS-Standard (PMM)	FCS-Standard (Poisson)	FCS-LMM-latent	FCS-GLMM (Gauss.)	FCS-GLMM (Poisson)	JM-General Location	JM-MLMM (common)	JM-MLMM (random)	JM-MGLMM
Intercept	Monotone	0.28	0.29	0.31	0.23	0.23	0.22	0.19	0.23	0.29	0.23
	Intermittent	0.32	0.30	0.31	0.26	0.27	0.26	0.17	0.25	0.35	0.23
Comorbidity Index	Monotone	0.26	0.27	0.51	0.23	0.23	0.37	0.19	0.23	0.36	0.38
	Intermittent	0.37	0.38	0.48	0.33	0.36	0.47	0.20	0.34	0.45	0.36
BMI	Monotone	0.28	0.27	0.30	0.24	0.24	0.23	0.19	0.23	0.33	0.23
	Intermittent	0.33	0.33	0.33	0.27	0.30	0.32	0.18	0.28	0.46	0.25
Paid assistive devices	Monotone	0.24	0.30	0.23	0.21	0.19	0.18	0.16	0.21	0.37	0.24
	Intermittent	0.49	0.51	0.47	0.41	0.40	0.39	0.26	0.37	0.48	0.31
Age	Monotone	0.27	0.25	0.28	0.21	0.24	0.24	0.17	0.22	0.23	0.22
	Intermittent	0.25	0.25	0.25	0.20	0.26	0.26	0.14	0.21	0.21	0.20
Gender	Monotone	0.27	0.26	0.28	0.21	0.24	0.24	0.18	0.22	0.22	0.21
	Intermittent	0.27	0.25	0.26	0.20	0.27	0.28	0.15	0.22	0.22	0.19
Self-rated health	Monotone	0.30	0.30	0.35	0.22	0.24	0.26	0.19	0.23	0.26	0.23
	Intermittent	0.27	0.27	0.30	0.23	0.27	0.29	0.16	0.24	0.28	0.21
Have med. bill paid off overtime	Monotone	0.23	0.23	0.23	0.17	0.22	0.22	0.16	0.20	0.20	0.20
	Intermittent	0.23	0.23	0.22	0.18	0.26	0.26	0.11	0.19	0.20	0.18
Took prescribed med.	Monotone	0.34	0.34	0.35	0.23	0.25	0.25	0.23	0.23	0.23	0.23
	Intermittent	0.36	0.35	0.35	0.26	0.28	0.29	0.20	0.29	0.31	0.23
Time	Monotone	0.32	0.32	0.36	0.26	0.30	0.30	0.21	0.29	0.28	0.25
	Intermittent	0.36	0.36	0.37	0.29	0.32	0.32	0.18	0.33	0.33	0.27
Subject-levelvariance	Monotone	0.36	0.35	0.37	0.33	0.22	0.23	0.24	0.35	0.35	0.28
	Intermittent	0.44	0.45	0.47	0.40	0.31	0.33	0.23	0.41	0.43	0.33

The average FMI estimates of the coefficients in the latent curve model when assuming a monotone missing data pattern are smaller than those assuming the intermittent missing data pattern (Table 6). Overall, the FMI estimates for the latent growth curve model coefficients are larger than the univariate GLMM coefficients. The FCS-based methods and JM-GL result in relatively smaller FMI that are around 0.25 compared to the JM-MLMM with heteroscedastic variances.

We recorded the computational time of all of the 10 methods on a standard Windows PC intel Xeon Core, 2.40GHz processor. For a single imputation, the FCS-standard with the linear regressions and PMM procedures, and JM-GL required the least computational time (less than 2s). FCS-Standard with the Poisson regressions had an average computational time of 22s. The three JM-based methods with multivariate multilevel modeling required approximately 280s of computation time. The computational time of FCS-GLMM methods was around 800s. The most computationally intensive method was the FCS-LMM-latent, which required several hours to complete a single imputation.

6 |. CASE STUDY ON SKILLED NURSING HOME ADMISSIONS

We applied the different imputation methods to study the relationship between sociodemographic factors, the number of hospital admissions, measures of physical, cognitive abilities, and skilled nursing facilities admissions. The data are based on NHATS Rounds 1–5, linked to data from the Center for Medicare and Medicaid (CMS). We selected 4836 participants who were enrolled in the Medicare fee for service at baseline. Deceased participants and those who left the Medicare fee for service during the study were assumed to be censored in contrast to missing. The proportions of missing participants because of loss to follow up at each round were approximately 16%, 14%, 12%, and 6%.

The sociodemographic factors comprised baseline age, sex, race/ethnicity (non-Hispanic white and other), educational level (college and higher, no school or < 9 grade, 9–12 grade and high school, and vocational training), and time-varying cohabitation status (living alone versus not living alone) and Medicare-Medicaid dual eligibility. The cognitive ability is a 5-points Likert scale ranging from excellent (1 point) to poor (5 points). The status of time-varying Alzheimer’s Disease and Related Dementias (ADRD) was defined based on the Chronic Condition Data Warehouse (CCW) algorithm. The time-varying comorbidity index comprises a count of 20 common chronic conditions among older adults identified through the CCW algorithm.⁶⁴ The function and frailty factors include two time-varying count variables: activities of daily living (ADL) and frailty score. ADL is derived as a count of limitations in seven daily living activities. Frailty score is defined by the sum of five criteria: exhaustion, low activity, weakness, slowness, and shrinking. Among all explanatory variables, only cohabitation status, ADL, and frailty score are subject to missing values.

We implemented FCS-Standard, FCS-LMM-latent, FCS-GLMM, JM-GL, JM-MLMM, and JM-MGLMM to impute the missing data. We applied a multilevel logistic regression model to examine the relationship between SNF admission status and sociodemographic factors adjusted for other factors that assess the physical and cognitive abilities. The results are displayed in Figure 4. Patients who are non-Hispanic whites are associated with 75% higher conditional odds of having an SNF admission compared to other race/ethnicity groups. For one unit increase in the number of hospital admissions, the conditional odds of having an SNF admission will quadruple. Being diagnosed with ADRD, with a higher ADL disabilities value and a higher comorbidity index, especially frailty, were associated with increased conditional odds of SNF admission. Not living alone and having Medicare-Medicaid dual eligibility were associated with decreased conditional odds of SNF. Education showed a statistically insignificant relationship with any SNF admission after adjustment for the other covariates.

FIGURE 4.

Odds ratios estimates of sociodemographic factors, the number of hospital admissions, measures of physical and cognitive abilities on skilled nursing facilities admissions for different imputation methods.

All imputation methods resulted in similar estimates of the conditional odds ratio and 95% interval estimates for the fully observed variables. FCS-based methods and JM-based methods result in different estimates for variables with missing values. FCS-Standard produces higher conditional odds ratios for cohabitation status and the number of hospital admissions, and it has lower conditional odds ratios for ADL disabilities and frailty scores compared to other methods. Compared to FCS-based methods, JM-based methods have higher conditional odds ratios for estimates of ADL disabilities and frailty scores.

7 |. DISCUSSION

Addressing missing data in longitudinal studies is challenging and involves advanced statistical techniques. We review existing MI methods for incomplete longitudinal mixed data, and their implementations on widely accessible software that requires limited additional coding. We compare these methods using simulation analyses and describe an applied example based on the NHATS data.

Among all of the methods examined, the FCS-LMM-latent method had the best performance for the univariate and multivariate multilevel modeling in terms of having small biases, relatively low FMI, and interval coverages that were at or above nominal. FCS-Standard and JM-GL resulted in the best performance for the growth curve model analysis, and comparable performance to FCS-LMM-latent for univariate hierarchical regression modeling, but sub-optimal performance when estimating some of the coefficients of multivariate multilevel modeling. Across all analyses, all imputation methods displayed better performance for data with monotone missing data patterns compared to intermittent missing data. This is partly because our intermittent missing data pattern is NMAR, while the monotone missing data pattern examined is MAR.

FCS-standard and JM-GL posit that the same variable measured at a different time point is a different variable, and use conditional associations between variables measured at different time points to impute the ones that are incomplete. This process essentially assumes an unstructured association structure between the repeated measurements. FCS-LMM-latent models adjust for the associations between repeated measurements recorded at different time points through subject-specific intercepts. The FCS-Standard methods are computationally more efficient and more flexible for imputing non-Normally distributed variables compared to FCS methods with multilevel modeling. The disadvantage of FCS-Standard methods is that they can result in non-identifiable models as the number of waves and variables within waves increases. In such cases, the FCS-twofold method⁴¹ is suggested for building imputation models. In our simulation study, we did not implement the FCS-twofold method because the simulation data comprise a small number of waves and variables within each wave. Possible disadvantages of the FCS-LMM-latent method are its intensive computational time and its possible sub-optimal performance when estimating growth curve models.

The JM-based methods with multilevel modeling, JM-MLMM-latent, and JM-MGLMM use fully observed variables as predictors and model the associations between incomplete variables through the correlations of the subject-level intercepts. This may result in biased estimates of the association between an outcome and a predictor in the analysis model when both variables are incomplete. In addition, the simulations display that using JM-MLMM-latent with a common covariance Σ_e across subjects had slightly better performance compared to JM-MLMM-latent with heterogeneous-covariances ${\mathit{\text{Σ}}}_{e_i}$. The poor performance of JM-MLMM-latent with heterogeneous-covariances may arise from the lack of convergence of the sampling algorithm. The Gibbs sampler used by JM-MLMM-latent (random) is slow to converge with a large number of subjects and poorly estimated subject-level variance.¹⁴

Imputation of count variables is sensitive to distributional assumptions. The simulations show that methods using Poisson regression may produce biased estimates of the comorbidity coefficient compared to a multilevel linear model or predictive mean matching. This may arise because many subjects have no comorbidities and the total number of comorbidities is bounded. Thus, a Poisson or a Negative-Binomial regression does not approximate this variable well. In situations where Poisson or Negative-Binomial hierarchical regression models approximate the data well (e.g. the number of paid assistive devices), our simulations show that these methods perform similarly to multilevel linear regression or predictive mean matching. Another option for imputing non-Normal continuous variables is to transform them so they would appear approximately Normal during the imputation step and transform the variables back in the statistical analyses step. However, these transformations may not preserve the original correlation structure between variables.^19,65

In univariate analysis, FCS-GLMM methods had point estimates of regression coefficients with small biases, but it resulted in poor operating characteristics for point and interval estimates of subject-level variance when using a multilevel model for analysis. This may arise from the computational algorithm that is being used to impute binary variables. The MCMC algorithm that has been used to sample from a multilevel logistic regression in the micemd package samples $b_i^{*}$ from its marginal posterior distribution N(0,Ψ_b). This can result in underestimation of the variance of b_i in the imputation process which carries over to the analysis model.

In our simulations, FCS-Standard performs well in cases with missing predictors but results in worse operating characteristics if these incomplete variables are used as dependent variables. This finding relates to the differences between the imputation of explanatory variables and outcomes. When both explanatory variables and outcomes are missing, Little⁶⁶ suggested that imputing outcome variables provides limited information for the subsequent regression of analysis. Von Hippel⁶⁷ proposed the multiple-imputation-deletion procedure that includes all incomplete variables in the imputation step and excludes missing response variables values from the substantive analysis. However, this procedure was proposed when the response variables and the subsequent analyses are defined in advance. In this work, we considered situations where no specific statistical analysis is specified before the imputation step, and the imputed datasets can be used for multiple analyses.

Our results agree with the conclusions of Huque et al²¹ that FCS-Standard provides reliable estimates for univariate hierarchical model analysis. However, we also demonstrated that FCS-Standard methods produce higher bias for coefficient estimates of level-1 covariates when analyzing multiple outcomes simultaneously in joint multilevel models. When imputing count variables, our results are consistent with the findings of Demirtas et al⁶⁵ and Kalaycioglu et al,¹⁹ who demonstrated that assuming a linear regression model for a non-Normally distributed variable may result in smaller biases compared to a mis-specified non-Gaussian model. Our simulations show that imputations using either the predictive mean matching or a multilevel linear model with rounding result in good operating characteristics of subsequent analyses.

In addition to the methods discussed in the manuscript, there are methods that can be used for imputing incomplete longitudinal data but require more technical coding and advanced statistical knowledge. One option is the fully Bayesian approach.¹⁹ It requires users to specify prior on the parameters of both imputation and analysis models and implement additional coding with software, such as STAN or PROC MCMC in SAS. The MCMC algorithm updates the parameters of the imputation model, draws an imputation for the incomplete variable, and updates the parameters of the analysis model sequentially at each iteration. More details on this approach can be found in Kalaycioglu et al.¹⁹ Another option is to implement nonparametric modeling techniques, such as sequential regression trees, random forests, and Dirichlet process mixture models.^68–71 These methods are beyond the scope of this manuscript, because their implementation with multiple variables that experience missing values is not trivial and they require the use of advanced statistical theory and coding. A future direction of our comparative work is to consider non-parametric imputation methods and extend the study simulations to data with more incomplete variables measured at a larger number of waves.

In conclusion, in longitudinal data with a small number of waves and a limited number of variables, when the analysis models comprise univariate regression models, FCS-standard is a computationally efficient method that results in precise and accurate estimates for both single and multilevel models. However, if the analysis models comprise multivariate multilevel models FCS-LMM-latent is a valid statistical method that produces more accurate estimates at the cost of more intensive computations.

APPENDIX

A SIMULATION RESULTS OF UNIVARIATE MODEL ANALYSIS FOR CONFIGURATIONS 3 WITH INTERMITTENT MISSING DATA PATTERN

B SIMULATION RESULTS OF UNIVARIATE MODEL ANALYSIS FOR CONFIGURATIONS 1 AND 2

TABLE A1.

The mean relative bias (MRB), the root mean squared error (RMSE), average interval width and empirical coverage of the 95% CI of coefficients estimates and subject-level variance estimate for the univariate analysis in Configuration 3 with intermittent missing data pattern. The incomplete predictors are marked in bold.

Estimates	Metrics	FCS-Standard (LM)	FCS-Standard (PMM)	FCS-Standard (Poisson)	FCS-LMM-latent	FCS-GLMM (Gauss.)	FCS-GLMM (Poisson)	JM-General Location	JM-MLMM (common)	JM-MLMM (random)	JM-MGLMM
Intercept ${\widehat{\gamma }}_0^{\mathit{\text{comp}}}=-{3.89}^{**}$	MRB	−0.018	−0.031	0.080	−0.045	0.039	0.086	0.153	0.101	0.170	0.224
	RMSE	0.170	0.176	0.166	0.201	0.236	0.219	0.252	0.480	0.535	0.255
	Width	1.170	1.154	1.149	1.123	1.027	1.009	1.075	1.116	1.170	1.026
	Coverage(%)	100	100	100	99	100	99	97	76	67	99
Comorbidity Index ${\widehat{\gamma }}_1^{\mathit{\text{comp }}}={0.38}^{**}$	MRB	0.046	0.011	0.027	−0.099	0.002	−0.027	0.050	−0.001	−0.197	−0.332
	RMSE	0.026	0.029	0.152	0.046	0.060	0.141	0.027	0.041	0.116	0.239
	Width	0.127	0.127	0.115	0.122	0.112	0.106	0.113	0.124	0.120	0.089
	Coverage(%)	98	96	0	85	44	0	95	90	3	0
BMI ${\widehat{\gamma }}_2^{\mathit{\text{comp }}}=-{0.02}^{*}$	MRB	−0.036	−0.050	−0.401	−0.112	−0.153	−0.371	−0.017	−0.099	−0.301	−0.634
	RMSE	0.004	0.004	0.005	0.004	0.004	0.006	0.005	0.010	0.010	0.011
	Width	0.030	0.030	0.029	0.029	0.026	0.026	0.027	0.029	0.033	0.024
	Coverage(%)	100	100	99	100	100	99	99	94	98	62
Paid assistive devices ${\widehat{\gamma }}_3^{\mathit{\text{comp }}}={0.36}^{**}$	MRB	0.031	0.089	0.185	−0.089	−0.247	−0.154	−0.140	0.085	−0.126	0.120
	RMSE	0.041	0.036	0.035	0.043	0.025	0.025	0.048	0.026	0.077	0.122
	Width	0.183	0.185	0.181	0.172	0.159	0.158	0.154	0.167	0.167	0.146
	Coverage(%)	96	99	100	94	100	100	88	99	63	1
Age ${\widehat{\gamma }}_4^{\mathit{\text{comp }}}={0.11}^{**}$	MRB	−0.013	−0.018	−0.001	−0.043	−0.053	−0.047	−0.007	−0.120	−0.132	0.057
	RMSE	0.015	0.015	0.014	0.025	0.017	0.014	0.017	0.029	0.025	0.016
	Width	0.111	0.111	0.110	0.109	0.098	0.098	0.106	0.110	0.108	0.102
	Coverage(%)	100	100	100	100	100	100	100	99	99	100
Gender ${\widehat{\gamma }}_5^{\mathit{\text{comp }}}={0.28}^{**}$	MRB	−0.051	−0.052	−0.241	−0.254	−0.212	−0.298	−0.088	−0.268	−0.350	−0.458
	RMSE	0.040	0.041	0.077	0.076	0.067	0.088	0.052	0.081	0.102	0.132
	Width	0.311	0.307	0.304	0.299	0.272	0.272	0.293	0.305	0.301	0.280
	Coverage(%)	100	100	98	100	99	94	100	99	95	62
Self-rated health ${\widehat{\gamma }}_6^{\mathit{\text{comp}}}={0.20}^{**}$	MRB	−0.002	0.011	0.354	0.000	0.000	0.217	0.014	0.068	0.287	0.668
	RMSE	0.022	0.022	0.073	0.018	0.019	0.047	0.026	0.023	0.061	0.135
	Width	0.160	0.160	0.159	0.157	0.141	0.140	0.151	0.159	0.161	0.141
	Coverage(%)	100	100	66	100	100	94	99	100	87	0
Have med. bill paid off overtime ${\widehat{\gamma }}_7^{\mathit{\text{comp}}}={0.34}^{*}$	MRB	−0.035	−0.046	0.010	−0.204	−0.108	−0.072	−0.033	−0.233	−0.195	0.092
	RMSE	0.059	0.059	0.062	0.048	0.051	0.055	0.093	0.059	0.076	0.091
	Width	0.481	0.481	0.473	0.469	0.424	0.419	0.453	0.478	0.473	0.444
	Coverage(%)	100	100	100	100	100	100	97	100	100	100
Took prescribed med. ${\widehat{\gamma }}_8^{\mathit{\text{comp }}}=0.30$	MRB	0.031	0.039	0.429	0.081	0.261	0.495	−0.129	−0.247	0.081	1.055
	RMSE	0.104	0.101	0.165	0.065	0.111	0.168	0.215	0.105	0.081	0.331
	Width	0.722	0.722	0.710	0.677	0.632	0.630	0.658	0.682	0.683	0.637
	Coverage(%)	100	99	98	100	100	99	97	100	100	47
Time ${\widehat{\gamma }}_9^{\mathit{\text{comp}}}={0.10}^{**}$	MRB	0.009	0.016	−0.191	−0.091	−0.143	−0.260	−0.083	0.510	0.473	−0.594
	RMSE	0.017	0.018	0.025	0.016	0.029	0.021	0.024	0.017	0.020	0.019
	Width	0.110	0.111	0.110	0.105	0.105	0.104	0.098	0.109	0.108	0.102
	Coverage(%)	100	100	99	100	95	99	97	100	99	100
Random effect ${\widehat{\sigma }}_{b_{\mathit{\text{comp}}}}^2={1.13}^{**}$	MRB	−0.027	−0.027	−0.041	−0.018	−0.297	−0.305	−0.010	0.007	−0.017	−0.099
	RMSE	0.053	0.055	0.063	0.042	0.338	0.346	0.067	0.038	0.042	0.120
	Width	0.256	0.257	0.257	0.247	0.232	0.232	0.219	0.250	0.253	0.225
	Coverage(%)	98	97	96	100	0	0	90	100	100	53

TABLE B2.

The root mean squared error (RMSE), average interval width and empirical coverage of the 95% CI of coefficients estimates and subject-level variance estimate for the univariate analysis in Configuration 1. The incomplete predictors are marked in bold.

Estimates	Metrics	FCS-Standard (LM)	FCS-Standard (PMM)	FCS-Standard (Poisson)	FCS-LMM-latent	FCS-GLMM (Gauss.)	FCS-GLMM (Poisson)	JM-General Location	JM-MLMM (common)	JM-MLMM (random)	JM-MGLMM
Intercept ${\widehat{\gamma }}_0^{\mathit{\text{comp }}}=-{3.89}^{**}$	RMSE	0.133	0.134	0.144	0.115	0.164	0.145	0.159	0.384	0.349	0.153
	Width	1.156	1.143	1.125	1.119	1.047	1.041	1.078	1.116	1.116	1.121
	Coverage	100	100	100	100	100	100	100	91	96	100
Comorbidity Index ${\widehat{\gamma }}_1^{\mathit{\text{comp }}}={0.38}^{**}$	RMSE	0.020	0.018	0.132	0.018	0.027	0.110	0.018	0.024	0.075	0.150
	Width	0.117	0.118	0.117	0.115	0.108	0.106	0.114	0.115	0.185	0.101
	Coverage	100	99	0	100	98	0	99	100	75	0
BMI ${\widehat{\gamma }}_2^{\mathit{\text{comp}}}=-{0.02}^{*}$	RMSE	0.004	0.004	0.006	0.004	0.004	0.005	0.005	0.006	0.005	0.005
	Width	0.029	0.028	0.028	0.028	0.026	0.026	0.027	0.028	0.028	0.028
	Coverage	100	100	99	100	100	100	99	100	100	100
Paid assistive devices ${\widehat{\gamma }}_3^{\mathit{\text{comp }}}={0.36}^{**}$	RMSE	0.026	0.024	0.024	0.055	0.023	0.022	0.035	0.022	0.021	0.022
	Width	0.151	0.153	0.149	0.147	0.142	0.141	0.141	0.150	0.148	0.148
	Coverage	99	100	100	84	100	100	96	100	100	100
Age ${\widehat{\gamma }}_4^{\mathit{\text{comp}}}={0.11}^{**}$	RMSE	0.015	0.016	0.015	0.014	0.016	0.014	0.018	0.024	0.022	0.013
	Width	0.113	0.113	0.111	0.110	0.103	0.099	0.106	0.113	0.111	0.109
	Coverage	100	100	100	100	100	100	100	100	100	100
Gender ${\widehat{\gamma }}_5^{\mathit{\text{comp}}}={0.28}^{**}$	RMSE	0.039	0.040	0.065	0.052	0.050	0.072	0.053	0.084	0.099	0.077
	Width	0.31	0.309	0.305	0.305	0.287	0.279	0.295	0.309	0.307	0.302
	Coverage	100	100	100	100	100	99	100	99	98	99
Self-rated health ${\widehat{\gamma }}_6^{\mathit{\text{comp}}}={0.20}^{**}$	RMSE	0.021	0.021	0.053	0.023	0.016	0.046	0.028	0.027	0.051	0.081
	Width	0.161	0.163	0.159	0.159	0.147	0.144	0.155	0.160	0.178	0.157
	Coverage	100	100	96	100	100	97	99	100	94	49
Have med. bill paid off overtime ${\widehat{\gamma }}_7^{\mathit{\text{comp }}}={0.34}^{*}$	RMSE	0.065	0.064	0.065	0.054	0.058	0.060	0.099	0.058	0.060	0.078
	Width	0.484	0.479	0.478	0.475	0.443	0.436	0.461	0.488	0.481	0.471
	Coverage	100	100	100	100	100	100	98	100	100	100
Took prescribed med. ${\widehat{\gamma }}_8^{\mathit{\text{comp}}}=0.30$	RMSE	0.078	0.084	0.135	0.066	0.089	0.144	0.099	0.197	0.148	0.197
	Width	0.701	0.724	0.689	0.674	0.648	0.638	0.669	0.686	0.697	0.674
	Coverage	100	100	99	100	100	100	100	97	100	99
Time ${\widehat{\gamma }}_9^{\mathit{\text{comp }}}={0.10}^{**}$	RMSE	0.02	0.021	0.046	0.015	0.022	0.020	0.018	0.018	0.017	0.019
	Width	0.104	0.107	0.112	0.103	0.101	0.103	0.099	0.104	0.108	0.104
	Coverage	100	99	75	100	99	100	98	99	99	98
Random effect ${\widehat{\sigma }}_{b_{\mathit{\text{comp}}}}^2={1.13}^{**}$	RMSE	0.042	0.040	0.060	0.029	0.204	0.222	0.044	0.041	0.037	0.041
	Width	0.239	0.237	0.240	0.237	0.213	0.218	0.218	0.237	0.236	0.237
	Coverage	100	100	99	100	0	0	99	99	100	100

TABLE B3.

The root mean squared error (RMSE), average interval width and empirical coverage of the 95% CI of coefficients estimates and subject-level variance estimate for the univariate analysisin Configuration 2. The incomplete predictors are marked in bold.

Estimates	Metrics	FCS-Standard (LM)	FCS-Standard (PMM)	FCS-Standard (Poisson)	FCS-LMM-latent)	FCS-GLMM (Gauss.)	FCS-GLMM (Poisson)	JM-General Location	JM-MLMM (common)	JM-MLMM (random)	JM-MGLMM
Intercept ${\widehat{\gamma }}_0^{\mathit{\text{comp }}}=-{3.89}^{**}$	RMSE	0.138	0.140	0.139	0.154	0.239	0.228	0.175	0.483	0.520	0.222
	Width	1.167	1.131	1.129	1.125	1.047	1.027	1.096	1.111	1.134	1.117
	Coverage	100	100	100	100	98	100	99	71	67	100
Comorbidity Index ${\widehat{\gamma }}_1^{\mathit{\text{comp}}}={0.38}^{**}$	RMSE	0.018	0.019	0.133	0.015	0.025	0.108	0.017	0.023	0.044	0.15
	Width	0.119	0.118	0.116	0.117	0.108	0.103	0.113	0.115	0.130	0.102
	Coverage	100	100	0	100	99	0	100	100	89	0
BMI ${\widehat{\gamma }}_2^{\mathit{\text{comp }}}=-{0.02}^{*}$	RMSE	0.004	0.004	0.005	0.003	0.004	0.003	0.004	0.007	0.008	0.008
	Width	0.029	0.028	0.028	0.028	0.026	0.026	0.027	0.029	0.030	0.027
	Coverage	100	100	100	100	99	100	100	99	97	98
Paid assistive devices ${\widehat{\gamma }}_3^{\mathit{\text{comp}}}={0.36}^{**}$	RMSE	0.024	0.022	0.023	0.054	0.022	0.020	0.041	0.020	0.019	0.022
	Width	0.150	0.150	0.149	0.147	0.145	0.143	0.143	0.149	0.152	0.15
	Coverage	100	100	100	91	100	100	93	100	100	100
Age ${\widehat{\gamma }}_4^{\mathit{\text{comp}}}={0.11}^{**}$	RMSE	0.016	0.015	0.015	0.015	0.019	0.016	0.017	0.027	0.026	0.013
	Width	0.113	0.112	0.112	0.112	0.103	0.101	0.107	0.111	0.112	0.109
	Coverage	100	100	100	100	100	100	100	99	100	100
Gender ${\widehat{\gamma }}_5^{\mathit{\text{comp}}}={0.28}^{**}$	RMSE	0.041	0.039	0.071	0.053	0.049	0.074	0.047	0.090	0.096	0.078
	Width	0.313	0.315	0.306	0.304	0.285	0.279	0.298	0.314	0.304	0.303
	Coverage	100	100	99	100	100	98	99	98	96	98
Self-rated health ${\widehat{\gamma }}_6^{\mathit{\text{comp }}}={0.20}^{**}$	RMSE	0.021	0.023	0.051	0.023	0.017	0.045	0.025	0.025	0.036	0.074
	Width	0.162	0.163	0.162	0.159	0.15	0.146	0.154	0.158	0.161	0.156
	Coverage	100	100	93	100	100	97	100	100	99	55
Have med. bill paid off overtime ${\widehat{\gamma }}_7^{\mathit{\text{comp }}}={0.34}^{*}$	RMSE	0.051	0.058	0.049	0.050	0.053	0.055	0.096	0.055	0.058	0.077
	Width	0.485	0.493	0.476	0.473	0.435	0.440	0.459	0.480	0.485	0.466
	Coverage	100	100	100	100	100	100	96	100	100	100
Took prescribed med. ${\widehat{\gamma }}_8^{\mathit{\text{comp }}}=0.30$	RMSE	0.084	0.087	0.138	0.067	0.096	0.153	0.130	0.229	0.205	0.189
	Width	0.731	0.708	0.703	0.667	0.654	0.639	0.657	0.698	0.687	0.676
	Coverage	100	100	100	100	100	100	99	95	95	99
Time ${\widehat{\gamma }}_9^{\mathit{\text{comp }}}={0.10}^{**}$	RMSE	0.019	0.018	0.044	0.014	0.022	0.022	0.017	0.017	0.017	0.017
	Width	0.106	0.105	0.110	0.104	0.104	0.103	0.099	0.104	0.104	0.105
	Coverage	99	100	80	100	100	100	100	100	100	100
Random effect ${\widehat{\sigma }}_{b_{\mathit{\text{comp}}}}^2={1.13}^{**}$	RMSE	0.041	0.040	0.065	0.036	0.200	0.218	0.047	0.043	0.041	0.046
	Width	0.237	0.242	0.235	0.240	0.214	0.215	0.219	0.236	0.240	0.236
	Coverage	100	100	96	100	0	0	97	100	99	100

DATA AVAILABILITY STATEMENT

The data that support the findings of the simulation study are openly available from the National Health and Aging Trends Study (NHATS) at https://nhats.org/researcher/data-access/public-use-files. The data that support the findings of the real data analysis are available at https://nhats.org/researcher/data-access/sensitive-data-files?id=restricted_data_files. Restrictions apply to the availability of these data, which were used under license for this study.