A Tractable Method to Account for High-dimensional Nonignorable Missing Data in Intensive Longitudinal Data

Abstract

Despite the need for sensitivity analysis to nonignorable missingness in intensive longitudinal data (ILD), such analysis is greatly hindered by novel ILD features, such as large data volume and complex nonmonotonic missing-data patterns. Likelihood of alternative models permitting nonignorable missingness often involves very high-dimensional integrals, causing curse of dimensionality and rendering solutions computationally prohibitive to obtain. We aim to overcome this challenge by developing a computationally feasible method, nonlinear indexes of local sensitivity to nonignorability (NISNI). We use linear mixed effects models (LMMs) for the incomplete outcome and covariates. We use Markov multinomial models to describe complex missing-data patterns and mechanisms in ILD, thereby permitting missingness probabilities to depend directly on missing data. Using a second-order Taylor series to approximate likelihood under nonignorability, we develop formulas and closed-form expressions for NISNI. Our approach permits the outcome and covariates to be missing simultaneously, as is often the case in ILD, and can capture U-shaped impact of nonignorability in the neighborhood of the MAR model without fitting alternative models or evaluating integrals. We evaluate performance of this method using simulated data and real ILD collected by the Ecological Momentary Assessment method.

1. Introduction

With development of technology, collecting real-time data using electronic devices is becoming increasingly common. Researchers now can conduct extensive longitudinal studies involving numerous participants and collect information about their daily experiences, behaviors, and environments.¹ Data from such studies involve long, intensively collected measurements that can capture changes in multiple variables concurrently and over time and are usually referred to as intensive longitudinal data (ILD).² One major method of obtaining ILD is Ecological Momentary Assessment (EMA), where prompts are sent to electronic devices, and participants are asked to report their current behaviors and experiences randomly or when events happen. EMA can minimize recall bias, achieve greater ecological validity, and allow researchers to capture relationships among different variables over time.^3–5

Missingness is ubiquitous and often unavoidable in ILD. In EMA, given that participants are prompted frequently over time, it is expected that noncompliance or dropout will occur during data collection, leading to missing data. Currently, a majority of statistical analyses assume that missing data do not influence missing probability after conditioning on observed data, known as missing at random (MAR).⁶ Under MAR and the additional assumption that parameters governing the outcome model and the missing-data mechanism (MDM) are distinct, MDM becomes ignorable^6,7 so that a valid likelihood-based/Bayesian inference does not need to model MDM. Thus, the assumption of MAR or (closely related) ignorability greatly simplifies analysis. However, this critical assumption cannot be verified robustly because observed data provide little information about MDM.^8,9 In self-reported EMA data, it is possible that reasons for non-response relate to the variable of interest, after conditioning on the observed information. Under such nonignorable missingness, assuming MAR generally yields biased and invalid estimation results.^10–13 It is crucial to conduct sensitivity analysis to assess validity and reliability of MAR analysis to alternative missing-data assumptions.^8,9,14

Such sensitivity analysis should consider several novel features and notable challenges in ILD. Intensiveness in data collection often means numerous missing data per subject as compared to traditional short-panel data, even if the rate of non-responded prompts is modest per subject. Furthermore, unlike short-panel data, there are typically numerous intermittent missingness and nonmonotonic missing-data patterns. In fact, in our EMA data, the number of unique missing-data patterns was equal to the number of subjects, leading to extremely sparse data for each missing-data pattern. Finally, there is often simultaneous missingness in regression outcome and covariates arising from non-responses. Because of such factors, sensitivity analysis encounters curse of dimensionality in such high-dimensional missing-data problems, which calls for general and flexible sensitivity analysis methods that are also computationally feasible to use with ILD.

Herein, we introduce a tractable sensitivity index method of evaluating sensitivity of a multilevel ILD analysis to alternative missing-data assumptions. We develop a joint selection model that augments (1) a multilevel model for notional complete outcome of interest with (2) a product conditional model for missing covariates and (3) a Markov Multinomial Missing-data Model (M2-MDM) that permits intermittent missingness and dropouts to depend directly on unobserved values in outcome and covariates by nonignorability parameters. Brute-force sensitivity analysis then varies the nonignorability parameters in a plausible range and estimates the resulting range of joint selection models. Curse of dimensionality in the ILD missing-data problem manifests here as high-dimensional integrals in the likelihood of the aforementioned joint selection models. The likelihood function, which is to be optimized, must be evaluated for combinations of potential unobserved data at non-responded occasions for each subject. Thus, computational workload required to evaluate such integrals increases exponentially with increasing number of intermittent missingness and missing covariates and can become computationally prohibitive with increasing data intensiveness. Such rapid increase in complexity with increasing number of missing values per subject is an example of curse of dimensionality. To solve such problems, we developed closed-form expressions for the sensitivity indexes, based on a second-order Taylor series approximation to the likelihood of the joint selection model. Computation of the sensitivity indexes avoids fitting complicated joint selection models or evaluating high-dimensional integrals. As a result, the method is tractable to use in ILD.

Our approach aims to answer the question: how are results from multilevel ILD modeling affected by possible violation of MAR assumption? Our approach has the advantage of directly addressing this question by providing a parsimonious sensitivity index for each parameter estimate in the multilevel model. One alternative approach to answering this question is to jointly estimate all the parameters in the aforementioned selection model. Although this is possible in principle, numerous studies have shown that the likelihood function for a nonignorable selection model is not well behaved. The likelihood function can be flat, non-convex, or multi-modal,^8,15–17 even if the nonignorability parameter is weakly identified and conclusions sensitively depend on model assumptions untestable in the presence of missing data.¹⁸ Reliable estimation of the joint selection model may require additional data such as availability of instrumental variables or refreshment samples.^19,20 In ILD, such estimation is further complicated by the aforementioned curse of dimensionality, which can render the joint estimation approach computationally unfeasible. The method developed herein can be viewed as a tool for quickly screening for sensitivity before investing a great deal of time and resources to overcome conceptual and computational challenges associated with such arduous joint estimation tasks and is consistent with the recommendation that the selection model is best used as a provisional device to perform sensitivity analysis.^8,9,15,21

Another alternative is to consider sensitivity analysis based on a semiparametric selection model.^22,23 With a sufficiently unstructured semiparametric or nonparametric marginal mean model for the response variable, the nonignorability parameter in this approach can be made unidentifiable. Another benefit of this approach is that the Generalized Estimating Equation (GEE) avoids numerical integrals encountered in likelihood-based sensitivity analysis by inverse weighting for missingness. There have been debates about marginal models versus conditional (multilevel) models and prior work demonstrating benefits of conditional models for longitudinal data.^24,25 Our modeling approach is motivated by the fact that multilevel models are well suited for and have been a predominant method for ILD.² Such multilevel models are preferred because they provide explicit modeling and estimation of heterogeneity among individuals in ILD analysis. Furthermore, multilevel models yield valid and efficient results under MAR without modeling why data are missing, whereas valid GEE analyses require modeling the MDM even under MAR, and efficient GEE MAR estimators can be difficult to find in ILD, which typically have irregular data collection occasions and complicated nonmonotonic missing-data patterns and reasons. It is fair to say that multilevel models remain a major method of analyzing longitudinal data and more so of ILD. It is therefore of great interest to directly gauge sensitivity of multilevel model parameter estimates to MAR assumption.

Our work builds on past work on local sensitivity analysis and extends it for use with ILD. We extend the index of local sensitivity to nonignorability (ISNI) method first proposed by Troxel et al.²⁶ and extend the more-recent nonlinear ISNI (NISNI) method developed by Gao et al.¹² to ILD. ISNI utilizes the idea of local sensitivity^15,17,21 to assess change in inference in the neighborhood of MAR model and has been extended to a range of statistical models and data types.^27–33 Although an R package isni is available for common distributions and models,³⁴ such methods are restricted to settings of missing outcome and monotonic impacts of nonignorable missingness, which limit their use in ILD. Gao et al.¹² relaxed these assumptions and developed the NISNI method for a cross-sectional subset (i.e., the first prompt of each subject) of EMA data. Herein, we extend this approach to full ILD.

In Section 2, we develop NISNI for ILD under the selection model framework. In Section 3, we discuss the method of calibrating NISNI. We apply the method to simulated data in Section 4 and to an EMA dataset in Section 5. We conclude with discussion in Section 6.

2. NISNI method for intensive longitudinal data

In EMA, data are collected by sending prompts to participants’ electronic devices, so failing to respond to any prompt will lead to missingness in data for all the questions in that particular diary entry. In such circumstances, outcome variable and covariates for that occasion will be missing simultaneously, which is common in ILD. In the following sections, we develop a joint model that permits a nonignorable missing-data process in EMA data or other types of ILD alike.

2.1. Selection model for incomplete outcome and covariates

2.1.1. Multilevel model for notional complete outcome

Denote the notional complete data vector for a continuous outcome for the ith subject as Y_i, where i = 1, 2, …, N, and Y_ij is the jth element in vector Y_i, j = 1, 2, …, n_i. We consider a linear mixed effects model (LMM), an instance of a multilevel model:

$$Y_i=W_i{\theta }_w+X_i{\theta }_x+Z_{yi}{B}_i+{\epsilon }_i^y,$$ (1)

where W_i is an n_i × P_w design matrix for P_w covariates that are fully observed for all the subjects, and P_w × 1 vector θ_w contains all the fixed effect parameters for W_i; X_i is an n_i × P_x matrix for P_x covariates that are not fully observed, and θ_x contains all the fixed effect parameters for W_i; X_i is an n_i × P_x matrix for P_x covariates that are not fully observed, and θ_x contains all the fixed effect parameters for X_i; and Z_yi is a subset of (W_i, X_i) and is the design matrix for Q_y × 1 random effects B_i, where $B_i\stackrel{i.i.d}{~}\text{MVN}\left(0,V_y\right)$, and V_y is the var-cov matrix for B_i. Assume the residual ${\epsilon }_i^y\mid B_i~\text{MVN}\left(0,{\sigma }_{ey}^2{I}_{n_i}\right)$. The var-cov matrix for Y_i after integration over the random effects is ${\Sigma }_{Y_i}\left({\theta }_2\right)={\sigma }_{ey}^2{I}_{n_i}+Z_{yi}{V}_y{Z}_{yi}^T$, where vector θ₂ contains all the unique parameters in V_y and σ_ey. Let θ₁ = (θ_w, θ_x), and the entire design matrix for the mean of Y_i is (W_i, X_i). Denote θ = (θ₁, θ₂). Then marginal distribution for outcome Y_i is given by

$$Y_i\mid X_i,W_i;\theta ~\text{MVN}\left(\left(W_i,X_i\right){\theta }_1,{\Sigma }_{Y_i}\left({\theta }_2\right)\right)$$

2.1.2. Product conditional multilevel model for notional complete covariates

Let $X_i=\left(X_{1i},\cdots ,X_{P_{x}i}\right)$ denote P_x time-varying covariates collected simultaneously with outcome Y at each occasion. We model joint conditional distribution f_ψ(X_i|W_i) using a product conditional model:

$$f_{\psi }\left(X_i\mid W_i\right)={\prod }_{p=1}^{P_x}{f}_{{\psi }_p}\left(X_{pi}\mid W_i,{\tilde{X}}_{pi}\right),$$

where ${\tilde{X}}_{pi}$ is null when p = 1, and ${\tilde{X}}_{pi}=\left(X_{1i},X_{2i},\dots ,X_{(p-1)i}\right)$ when p > 1. Unlike the product conditional model for cross-sectional data,^12,35 the notional complete data for the pth covariate are $X_{pi}=\left(X_{pi1},\cdots ,X_{pi{n}_i}\right)$, which is a vector of repeated measures from subject i. We thus model $f_{{\psi }_p}\left(X_{pi}\mid W_i,{\tilde{X}}_{pi}\right)$ using a multilevel model as follows:

$$X_{pi}=W_i{\psi }_{pw}+{\tilde{X}}_{pi}{\psi }_{p\tilde{x}}+Z_{pi}{b}_{pi}+{\epsilon }_{pi}^x.$$

For a specific p, ${\psi }_{p\tilde{x}}=\left({\psi }_{1p},{\psi }_{2p}\dots ,{\psi }_{(p-1)p}\right)$. Similar to the outcome model, Z_pi is a subset of $\left(W_i,{\tilde{X}}_{pi}\right)$ and is the design matrix for Q_p × 1 random effects b_pi, where $b_{pi}\stackrel{i.i.d}{~}\text{MVN}\left(0,V_p\right)$, and V_p is the var-cov matrix for b_pi. Assume the residual ${\epsilon }_{pi}^x\mid b_{pi}~\text{MVN}\left(0,{\sigma }_{ep}^2{I}_{n_i}\right)$. Let ${\psi }_{p1}=\left({\psi }_{pw},{\psi }_{p\tilde{x}}\right)$, and ψ_p2 is a vector that contains all the unique parameters in V_p and σ_ep. After integration over b_pi,

$$X_{pi}\mid W_i,{\tilde{X}}_{pi};{\psi }_p~\text{MVN}\left(\left(W_i,{\tilde{X}}_{pi}\right){\psi }_{p1},{\Sigma }_{pi}\left({\psi }_{p2}\right)\right),\text{ where }{\Sigma }_{pi}\left({\psi }_{p2}\right)={\sigma }_{ep}^2{I}_{n_i}+Z_{pi}{V}_p{Z}_{pi}^T.$$

2.1.3. Markov multinomial missing-data model (M2-MDM)

In EMA, participants may skip some assessments (causing intermittent missingness), or participants may quit during the study (causing dropouts). Furthermore, the prompt response behavior and outcome variable both could be affected by common idiosyncratic (i.e., time-varying) factors which, if unobserved or excluded from the analysis, could lead to dependence of missingness probability on unobserved outcome values. Owing to such considerations, we use a Markov multinomial model to capture arbitrary and nonmonotonic missing-data patterns and nonignorable missingness, which extends the simple binary missing-data model of Gao et al.¹² to ILD. For each subject i, there is a corresponding n_i × 1 missing status vector G_i with its jth component as:

$$G_{ij}=\{\begin{array}{ll}\hfill 0\hfill & \text{ if subject i is observed at occasion j}\hfill \\ \hfill \text{I}\hfill & \text{ if subject i is intermittently missing at occasion j}\hfill \\ \hfill \text{D}\hfill & \text{ if subject i drops out at occasion j}\hfill \end{array}$$

and we can write the joint distribution for $\left(G_{i1},\cdots G_{i{n}_i}\right)$ as a product of univariate transitional probabilities: $\left(G_{i1},\dots ,G_{i{n}_i}\mid Y_i,X_i,\gamma \right)=\left(G_{i1}\mid Y_i,X_i,\gamma \right){\prod }_{j=2}^{n_i}\left(G_{ij}\mid G_{i1},\dots ,G_{i,j-1},Y_i,X_i,\gamma \right)$. We then can model univariate conditional distributions separately. ILD typically contains numerous unique missingness patterns. Therefore, conditioning on the full missingness-status history (G_i1, ⋯ , G_i,j−1) can cause data sparsity within each missing-data pattern, which may lead to unstable or even non-converging estimation. One approach to solving this problem is to adopt a finite-order Markov model. Although our method can easily generalize to a higher-order Markov model, we consider the following first-order Markov model for expositional simplicity:

$$\begin{array}{l}\left(G_{ij}\mid G_{i1},\dots ,G_{i,j-1}=g_{i,j-1},Y_i,X_i,\gamma \right)=\left(G_{ij}\mid G_{i,j-1}=g_{i,j-1},Y_i,X_i,\gamma \right)\hfill \\ ~\text{Multinomial}\left(1,\left[P_{ij}^{0g_{i,j-1}},P_{ij}^{\text{I}{g}_{i,j-1}},P_{ij}^{{\text{Dg}}_{i,j-1}}\right]\right).\hfill \end{array}$$

Given missingness status at the previous occasion G_i,j−1, G_ij becomes independent of all other past missing status variables. We model missingness status transition probability $P_{ij}^{u{g}_{i,j-1}}$ with the following multinomial logit model:

$$\text{ln}\left(\frac{P_{ij}^{uv}}{P_{ij}^{0v}}\right)=S_{ij}^T{\gamma }_0^{uv}+{\gamma }_{1y}{Y}_{ij}+{\gamma }_{1x}^T{X}_{ij},u=\text{I},\text{\hspace{1em}D},v=g_{i,j-1},$$ (2)

where u is missingness status at the current occasion with u = O as the reference level, and υ = g_i,j−1 ∈ (O, I, D) is missingness status at the previous occasion. S_ij contains all the observed missingness predictors, including variables in fully observed W_ij as well as the last observed components in outcome Y_i,j−1 and in covariates X_i,j−1. Under this first-order M2-MDM, missing probability is a constant for certain occasions; specifically, $P_{ij}^{0\text{D}}=P_{ij}^{\text{ID}}=0$, $P_{ij}^{\text{DD}}=1$ (definition of dropout); $P_{ij}^{\text{DI}}=0$ (definition of intermittent missing).

Nonignorable parameter vector γ₁ = (γ_1y, γ_1x) captures dependence of missingness probability on potentially unobserved values of contemporaneous variables at the current occasion; MDM reduces to MAR when γ₁ = 0. The model is a parsimonious nonignorable selection model that approximates a more-complex model wherein given the outcome and covariates at the current occasion, missingness depends additionally on past and future unobserved outcomes and time-varying covariates: we can integrate out these past and future unobserved outcome and covariates so that the resulting selection model depends only on values at the current visit³⁶. Although our method is general and can be extended to the more-general M2-MDM, we prefer the more-parsimonious specification, which reduces the number of parameters for nonignorable non-responses and permits easily interpretable sensitivity analysis, which is desirable³⁷. In addition, we can permit γ₁ to depend on u and υ, which increases the number of nonignorability parameters in sensitivity analysis. It seems reasonable in practice that the direction of nonignorability is roughly the same for different values of u and υ. Thus, we can fix these sensitivity parameters at the maximum number of γ₁ for a parsimonious and conservative sensitivity analysis.

2.1.4. Selection model likelihood

We partition Y_i and X_i into $\left(Y_i^{\text{obs}},Y_i^{\text{mis}}\right)$ and $\left(X_i^{\text{obs }},X_i^{\text{mis }}\right)$ for observed and missing components, respectively. Our method can be extended to partially observed Z_yi and Z_pi with slightly more complex closed-form expressions for certain terms in our sensitivity index formulas. For expositional simplicity, we focus on Z_yi and Z_pi containing only fully observed variables. The log-likelihood for model parameters based on observed data with N independent subjects is:

$$L\left(\theta ,\psi ,{\gamma }_0,{\gamma }_1;Y^{\text{obs}},X^{\text{obs}},W,S,G\right)=\sum _{i=1}^N\left[\text{ln}{f}_{\theta }\left(Y_i^{\text{obs}}\mid X_i^{\text{obs}},W_i\right)+\sum _{p=1}^{P_x}\text{ln}{f}_{{\psi }_p}\left(X_{pi}^{\text{obs}}\mid W_i,{\tilde{X}}_{pi}^{\text{obs}}\right)+\sum _{j:g_{ij}=0}\text{ln}{f}_{\gamma }\left(G_{ij}\mid S_{ij},X_{ij}^{\text{obs}},Y_{ij}^{\text{obs}},G_{i,j-1}\right)\right]+\sum _{i:n_i^{\text{obs }}<n_i}\text{ln}\left[{\int }_{{\Omega }_{Y_i}}{\int }_{{\Omega }_{X_{P_{x^i}}}}\dots {\int }_{{\Omega }_{X_{2i}}}{\int }_{{\Omega }_{X_{1i}}}\left(\prod _{j:g_{ij}\ne 0}{f}_{\gamma }\left(G_{ij}\mid S_{ij},Y_{ij}^{\text{mis}},X_{ij}^{\text{mis}},G_{i,j-1}\right)\right)f_{\theta }\left(Y_i^{\text{mis}}\mid Y_i^{\text{obs}},X_i^{\text{obs}},X_i^{\text{mis}},W_i\right)\prod _{p=1}^{P_x}{f}_{{\psi }_p}\left(X_{pi}^{\text{mis}}\mid X_{pi}^{\text{obs}},W_i,{\tilde{x}}_{pi}\right)d{X}_{1i}^{\text{mis}}d{X}_{2i}^{\text{mis}}\dots d{X}_{P_{x}i}^{\text{mis}}d{Y}_i^{\text{mis}}\right],$$

where n_i is the number of planned data collection occasions for subject i, $n_i^{\text{obs}}$ is the number of occasions that have data observed, and is the sample space for a random variable.

2.2. Review of ISNI method

As described in the Introduction, it is well recognized in the literature that non-identifiability arises when attempting to use the aforementioned nonignorable selection model likelihood to jointly estimate all the model parameters because data provides little information about the nonignorability parameter, γ₁. A more judicious approach is to consider this nonignorable selection model as tentative and perform sensitivity analysis over a range of values for γ₁. For a given γ₁, other model parameters Θ = (θ, ψ, γ₀) are estimable by maximizing likelihood over Θ. To investigate impacts of different assumptions on the degree of nonignorable missingness, as captured by γ₁ on the other parameters Θ, we can obtain parameter profile maximum likelihood estimations (MLEs), given γ₁, denoted as $\widehat{\Theta }\left({\gamma }_1\right)$. If $\widehat{\Theta }\left({\gamma }_1\right)$ changes little as γ₁ varies around 0, sensitivity analysis concludes that MAR inference is robust and trustworthy. Such profile MLEs $\widehat{\Theta }\left({\gamma }_1\right)$ can be obtained by maximizing likelihood for a set of fixed values of γ₁ by iterative methods such as Newton-Raphson or expectation-maximization methods. However, as we shall show, this is computationally infeasible for ILD. The ISNI method overcomes this challenge by eliminating the need to perform such brute-force model estimation and requiring only estimates readily available under MAR.

The foregoing likelihood expression indicates that unless γ₁ = 0, a multi-dimension integral is included for each subject, and its dimension increases with increasing number of non-responses and variables with missing data. Take the EMA baseline wave data as an example. Figure 1 shows the missing-data pattern for each subject. The maximum number of prompts is 128. Green indicates answered prompts, red indicates unanswered prompts, and white indicates there was no planned prompt. Each subject has a unique missing-data pattern, and only 1 out of 461 subjects has fully observed data. On average, each subject has 42 planned random prompts, and the percentage of non-responded prompts is 27% but is as high as 91%. To perform sensitivity analysis for a model including two covariates missing simultaneously with the outcome, the average number of dimensions required for integration per subject will be approximately 34 (42 × 0.27 × 3). Certain subjects have more than 100 prompts, among which 70% are missing. Integration dimension will be higher than 210 for each of these subjects. It will be computationally infeasible to perform sensitivity analysis by direct likelihood optimization. Using Taylor-series expansion, we propose an extended ISNI method that avoids evaluating integrals and fitting the nonignorable selection models, thereby providing a feasible method of examining the parameter sensitivity to nonignorability in ILD.

Figure 1:

Missing-data patterns.

The ISNI method simplifies sensitivity analysis by approximating $\widehat{\Theta }\left({\gamma }_1\right)$ using a first-order Taylor series expansion around γ₁ = 0. For the kth element of Θ:

$${\widehat{\Theta }}^k\left({\gamma }_1\right)\approx {\widehat{\Theta }}^k(0)+{\frac{\partial {\widehat{\Theta }}^k(0)}{\partial {\gamma }_1}|}_{{\gamma }_1=0}\times {\gamma }_1.$$

Troxel et al. defines ${\frac{\partial \widehat{\Theta }\left({\gamma }_1\right)}{\partial {\gamma }_1}|}_{{\gamma }_1=0}$ as ISNI with the following expression:

$$\text{ISNI}=-{\nabla }^2{L}_{\Theta ,\Theta }^{-1}{\nabla }^2{L}_{\Theta ,{\gamma }_1},$$

where ${\nabla }^2{L}_{a,b}={\frac{{\partial }^{2}L}{\partial a\partial b}|}_{(\widehat{\Theta }(0),0)}$, and L is the log-likelihood of a joint selection model. First-order Taylor series expansion and linear ISNI have been shown to adequately capture local sensitivity when only outcome is subject to missingness.

Gao et al.¹² have shown that for cross-sectional data, linear ISNI (ISNIL) may not be sufficient for estimating $\widehat{\Theta }\left({\gamma }_1\right)$ accurately when outcome and covariates are missing concurrently. They extended the method by expanding the Taylor-series to second order, thereby developing the quadratic index of sensitivity to nonignorability (ISNIQ). ISNIL and ISNIQ together are named as nonlinear indexes of sensitivity to nonignorability (NISNI). In the following section, we extend NISNI to overcome curse of dimensionality in selection model likelihood for ILD.

2.3. NISNI development for Y-dependent nonignorability

Similar to cross-sectional data, for ILD the profile MLE can have a U-shape in the neighborhood of MAR when the outcome and covariates are missing simultaneously. To accurately capture the shape, we apply the following Taylor-series expansion:

$$\widehat{\Theta }\left({\gamma }_1\right)\approx \widehat{\Theta }(0)+{{\gamma }_1^T\frac{\partial \widehat{\Theta }\left({\gamma }_1\right)}{\partial {\gamma }_1}|}_{{\gamma }_1=0}+{\frac{1}{2}{\gamma }_1^T\frac{{\partial }^2\widehat{\Theta }\left({\gamma }_1\right)}{\partial {\gamma }_1\partial {\gamma }_1^T}{\gamma }_1|}_{{\gamma }_1=0}.$$

Define $\text{ISNIL}={\frac{\partial \widehat{\Theta }\left({\gamma }_1\right)}{\partial {\gamma }_1}|}_{{\gamma }_1=0}$ and $\text{ISNIQ}={\frac{{\partial }^2\widehat{\Theta }\left({\gamma }_1\right)}{\partial {\gamma }_1\partial {\gamma }_1^T}|}_{{\gamma }_1=0}$. As in Supplemental Materials Appendix A:

$$\text{ISNIL}(\widehat{\theta })=-{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^2{L}_{\theta ,{\gamma }_1}$$ (3)

$$\text{ISNIQ}(\widehat{\theta })=-2{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,\theta ,{\gamma }_1}\text{ISNIL}\left(\widehat{\theta }\left({\gamma }_1\right)\right)-2{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,\psi ,{\gamma }_1}\text{ISNIL}\left(\widehat{\psi }\left({\gamma }_1\right)\right)-2{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,{\gamma }_0,{\gamma }_1}\text{ISNIL }\left({\widehat{\gamma }}_0\left({\gamma }_1\right)\right)-{\nabla }^2{L}_{\theta ,\theta }^{-1}\left(\sum _{j=1}^{n_{\theta }}\text{ISNIL}\left({\widehat{\theta }}^j\right){\nabla }^3{L}_{\theta ,\theta ,{\theta }^j}\right)\text{ISNIL}\left(\widehat{\theta }\left({\gamma }_1\right)\right)-{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,{\gamma }_1,{\gamma }_1^T},$$ (4)

where ${\nabla }^2{L}_{ab}={\frac{{\partial }^{2}L}{\partial a\partial b}|}_{{\gamma }_1=0}$ and ${\nabla }^3{L}_{abc}={\frac{{\partial }^{3}L}{\partial a\partial b\partial c}|}_{{\gamma }_1=0}$. To help understand the formulas, consider Y-dependent nonignorability. For any subject i, let ${\Sigma }_{Y_i}\left({\theta }_2\right)=\left(\begin{array}{ll}{\Sigma }_{Y_i,oo}\hfill & {\Sigma }_{Y_i,om}\hfill \\ {\Sigma }_{Y_i,mo}\hfill & {\Sigma }_{Y_i,mm}\hfill \end{array}\right)$, ${\Sigma }_{pi}\left({\psi }_{p2}\right)=\left(\begin{array}{cc}{\Sigma }_{pi,oo}& {\Sigma }_{pi,om}\\ {\Sigma }_{pi,mo}& {\Sigma }_{pi,mm}\end{array}\right)$ and denote $\left(W_i^{\text{obs}},X_i^{\text{obs}}\right)$ as $D_{Y_i,o}$, $\left(W_i^{\text{mis}},X_i^{\text{mis}}\right)$ as $D_{Y_i,m}$, $\left(W_i^{\text{obs }},{\tilde{X}}_{pi}^{\text{obs }}\right)$ as D_pi,o, and $\left(W_i^{\text{mis}},{\tilde{X}}_{pi}^{\text{mis}}\right)$ as D_pi,m. Denote conditional expectations $E\left(Y_i^{\text{mis}}\mid Y_i^{\text{obs}},X_i,W_i\right)$ as E(Y_i)^m|o and $E\left(X_{pi}^{\text{mis}}\mid X_{pi}^{\text{obs}},{\tilde{X}}_{pi},W_i\right)$ as E(X_pi)^m|o. To obtain NISNI, we must derive expressions for the second and third derivatives of the log likelihood w.r.t. Θ evaluated at γ₁ = 0. From the models described in section 2.1, the main required conditional distributions are:

$$Y_i^{\text{obs }}|X_i^{\text{obs }},W_i^{\text{obs }}~\text{MVN}\left(D_{Y_i,o}{\theta }_1,{\Sigma }_{Y_i,oo}\right),X_{pi}^{\text{obs }}|{\tilde{X}}_{pi}^{\text{obs }},W_i^{\text{obs }}~\text{MVN}\left(D_{pi,o}{\psi }_{p1},{\Sigma }_{pi,oo}\right),$$

$$Y_i^{\text{mis}}|Y_i^{\text{obs}},X_i,W_i~\text{MVN}\left(E{\left(Y_i\right)}^{\text{m}}|\text{o},{\Sigma }_{Y_i}^{\text{m}|\text{o}}\right),X_{pi}^{\text{mis}}|X_{pi}^{\text{obs}},{\tilde{X}}_{pi},W_i~\text{MVN}\left(E{\left(X_{pi}\right)}^{\text{m}|o},{\Sigma }_{pi}^{\text{m}|o}\right),$$

where $E{\left(Y_i\right)}^{\text{m}\mid \text{o}}=D_{Y_{i,m}}{\theta }_1+{\Sigma }_{Y_i,mo}{\Sigma }_{Y_i,oo}^{-1}\left(Y_i^{obs}-D_{Y_i,o}{\theta }_1\right)$ and $E{\left(X_{pi}\right)}^{\text{m}\mid \text{o}}=D_{pi,m}{\psi }_{p1}+{\Sigma }_{pi,mo}{\Sigma }_{pi,oo}^{-1}\left(X_{pi}^{obs}-D_{pi,o}{\psi }_{p1}\right)$. Therefore, each derivative evaluated at γ₁ = 0 has an explicit closed-form expression without evaluating high-dimensional integrals. Derivation result details are shown in Supplemental Materials Appendix B.

2.3.1. Simple example to illustrate NISNI

Consider a case where each subject i is prompted twice. The outcome model contains the intercept and one covariate X that is concurrently missing with Y, and the model for X only contains the intercept. Both models only include random intercepts:

$$Y_i={\theta }_w+{\theta }_x{X}_i+Z_{yi}{B}_i+{\epsilon }_i^y,X_i={\psi }_0+Z_{xi}{b}_{xi}+{\epsilon }_i^x.$$

Because there are only two occasions, only dropout can happen, and M2-MDM reduces to a binary logit model with $P_{ij}^{00}=\frac{1}{1+\text{exp}\left({\gamma }_0+{\gamma }_1{Y}_{ij}\right)}$. Let n_oo denote the number of subjects with both occasions observed, and n_om denote the number of subjects who drop out at the second occasion. $X_i^{\text{obs}}$ represents observed X in subject i, and p_m denotes the missing proportion. ${\widehat{\theta }}_w$, ${\widehat{\theta }}_x$, ${\widehat{\sigma }}_{ey}$ and ${\widehat{\sigma }}_{ex}$ represent MAR estimators. When σ_vy = σ_υx = 0, expressions for terms used in NISNI calculations are described in Supplemental Materials Appendix C. NISNI terms for ${\widehat{\theta }}_1=\left({\widehat{\theta }}_w,{\widehat{\theta }}_x\right)$ are:

$$\text{ISNIL}\left({\widehat{\theta }}_w,{\widehat{\theta }}_x\right)=\left(p_m{\widehat{\sigma }}_{ey}^2,0\right)$$

$$\text{ISNIQ}\left({\widehat{\theta }}_w,{\widehat{\theta }}_x\right)=2p_m\left(1-p_m\right){\widehat{\theta }}_x{\widehat{\sigma }}_{ey}^2{\widehat{\sigma }}_{ex}^2\left({\widehat{\psi }}_0\frac{2n_{oo}+n_{om}}{{\sum }_i{X}_{io}^2-\frac{{\left({\sum }_i{X}_{io}\right)}^2}{2n_{oo}+n_{om}}},-\frac{2n_{oo}+n_{om}}{{\sum }_i{X}_{io}^2-\frac{{\left({\sum }_i{X}_{io}\right)}^2}{2n_{oo}+n_{om}}}\right).$$

$\text{ISNIL}\left({\widehat{\theta }}_x\right)=0$ while $\text{ISNIQ}\left({\widehat{\theta }}_x\right)$ is not zero, indicating the need to use NISNI to capture parameter sensitivity to nonignorability. Furthermore, $\text{ISNIL}\left({\widehat{\theta }}_w\right)$ increases with increasing p_m, and the absolute values of both $\text{ISNIQ}\left({\widehat{\theta }}_w\right)$ and $\text{ISNIQ}\left({\widehat{\theta }}_x\right)$ will increase with increasing p_m when p_m < 0.5, holding all other quantities constant. The NISNI expression will be more complicated when considering correlations and when ${\Sigma }_{Y_i}$ and Σ_Xi have more complex structures.

2.4. NISNI development for Y- and X-dependent nonignorability

When considering Y- and X- dependent nonignorability, the M2-MDM takes the general expression, as given by Eq.(2). ${\gamma }_1=\left({\gamma }_{1y},{\gamma }_{1x_1},\dots ,{\gamma }_{1x_{P_x}}\right)$ and ISNIL now become:

$$\text{ISNIL}(\widehat{\theta })={\left[{\text{ISNIL}}_y(\widehat{\theta }),{\text{ISNIL}}_x^T(\widehat{\theta })\right]}^T={\left[-{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^2{L}_{\theta ,{\gamma }_{1y}},0\right]}^T.$$

$\text{ISNIQ}(\widehat{\theta })$ becomes a matrix wherein each element represents a second derivative of $\widehat{\theta }$ w.r.t. γ₁. Denote $\frac{{\partial }^2\widehat{\theta }\left({\gamma }_1\right)}{{\partial }^2{\gamma }_{1y}}$, $\frac{{\partial }^2\widehat{\theta }\left({\gamma }_1\right)}{\partial {\gamma }_{1y}\partial {\gamma }_{1x_p}}$ and $\frac{{\partial }^2\widehat{\theta }\left({\gamma }_1\right)}{\partial {\gamma }_{1x_p}\partial {\gamma }_{1x_q}}$ as ${\text{ISNIQ}}_{yy}(\widehat{\theta })$, ${\text{ISNIQ}}_{yp}(\widehat{\theta })$, and ${\text{ISNIQ}}_{pq}(\widehat{\theta })$, respectively, where ${\gamma }_{1x_p}$ and ${\gamma }_{1x_q}$ are the pth and qth elements in vector ${\gamma }_{1x}=\left({\gamma }_{1x_1},\dots ,{\gamma }_{1x_{P_x}}\right)$. To compute ISNIQ, we must additionally derive closed-form expressions for terms related to γ_1x. From derivation details shown in Supplemental Materials Appendix D:

${\text{ISNIQ}}_{yy}(\widehat{\theta })$ has the same expression as Eq.(4)

$${\text{ISNIQ}}_{yp}(\widehat{\theta })=-{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,{\psi }_p,{\gamma }_{1y}}{\text{ISNIL}}_p\left({\widehat{\psi }}_p\right)-{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,{\gamma }_0,{\gamma }_{1y}}{\text{ISNIL}}_p\left({\widehat{\gamma }}_0\right)-{\nabla }^2{L}_{\theta ,\theta }^{-1}{\nabla }^3{L}_{\theta ,{\gamma }_{1y},{\gamma }_{1x_p}}$$

$${\text{ISNIQ}}_{pq}(\widehat{\theta })=0.$$

3. NISNI calibration

3.1. Y-dependent nonignorability

To facilitate interpretation, a scale-free NISNI calibration is needed. Let ${\tilde{\gamma }}_1$ be the minimum absolute value for γ₁ so that $\left|{\widehat{\theta }}^k\left({\gamma }_1\right)-{\widehat{\theta }}^k(0)\right|=\text{SE}$, i.e., the minimum degree of nonignorability needed to change an MLE by one standard error. That is, ${\tilde{\gamma }}_1=\text{min}\left(\left|{\text{arg}}_{{\gamma }_1}\left(\left|\text{ISNIL}\times {\gamma }_1+\frac{\text{ISNIQ}}{2}\times {\gamma }_1^2\right|=\text{SE}\right)\right|\right)$. For a continuous outcome, the meaning of ${\tilde{\gamma }}_1$ depends on the unit of Y. For scale-free interpretation, we define $\text{MinNI}=\tilde{\gamma }{\sigma }_Y$, where σ_Y is the standard deviation (SD) of the outcome. MinNI defined as such means that the minimum magnitude of nonignorability needed to change an estimate by one standard error is such that a $\frac{1}{\text{MinNI}}-\text{SD}$ unit change in outcome is associated with an e¹ = 2.7−fold change in the odds of missingness. In the special case of using ISNIL alone to measure sensitivity, MinNI reduces to a simpler form as ${\text{MinNI}}^L=\left|\frac{\text{SE}\times {\sigma }_Y}{\text{ISNIL}}\right|$. A small MinNI value suggests large sensitivity to nonignorability because modest nonignorability can change an estimate significantly whereas a large MinNI value means that MAR result remains robust to all but extreme nonignorability. We use 1 as a cuto value for important sensitivity.²⁶

3.2. Y- and X-dependent nonignorability

Building upon the approach of calibrating the scalar γ₁ for Y-dependent nonignorability, we now consider index calibration for a vector of nonignorability parameter γ₁ = (γ_1y, γ_1x) with Y- and X-dependent nonignorability. Following Gao et al.,¹² let ${\gamma }_1^{*}$ represent a standardized cumulative magnitude of nonignorability over all the variables with missing values, and ${\gamma }_1^{*}=\sqrt{{\gamma }_{1y}^2{\sigma }_Y^2+{\gamma }_{1x_1}^2{\sigma }_{X_1}^2+\dots +{\gamma }_{1x_{P_x}}^2{\sigma }_{X_{P_x}}^2}$. Thus, to find the range of estimates for parameter θ^k given a magnitude of overall nonignorability, ${\gamma }_1^{*}=R$, we simply find the minimum and maximum values of ${\widehat{\theta }}^k\left({\gamma }_{1y},{\gamma }_{1x}\right)={\widehat{\theta }}^k(0)+{\text{ISNIL}}_y{\gamma }_{1y}+\frac{1}{2}{\text{ISNIQ}}_{yy}{\gamma }_{1y}^2+{\sum }_{p=1}^{P_x}{\text{ISNIQ}}_{py}{\gamma }_{1y}{\gamma }_{1x_p}$ subject to the constraint ${\gamma }_{1y}^2{\sigma }_Y^2+{\gamma }_{1x_1}^2{\sigma }_{X_1}^2+\dots +{\gamma }_{1x_{P_x}}^2{\sigma }_{X_{P_x}}^2=R^2$.

Now MinNI^Q is defined as the minimum ${\gamma }_1^{*}$ to have $\left|{\widehat{\theta }}^k\left({\gamma }_1\right)-{\widehat{\theta }}^k(0)\right|=\text{SE}$. For a given θ^k, this corresponds to finding the minimum ${\gamma }_1^{*}$ subject to the constraint $\left|{\text{ISNIL}}_y{\gamma }_{1y}+\frac{1}{2}{\text{ISNIQ}}_{yy}{\gamma }_{1y}^2+{\sum }_{p=1}^{P_x}{\text{ISNIQ}}_{yp}{\gamma }_{1y}{\gamma }_{1x_p}\right|=\text{SE}$. We use the Lagrange multipliers method to perform the mathematical optimization.

4. Simulation study to illustrate NISNI

In the simulation study, we set n_i = 3, i = 1, 2, …, 100, and for each subject:

$$Y_i~\text{MVN}\left(W_i{\theta }_w+X_i{\theta }_x,{\Sigma }_{Y_i}\right),X_i~\text{MVN}\left(W_i{\psi }_w,{\Sigma }_{X_i}\right).$$

W is the intercept and θ_w = 1, θ_x = 4, and ψ_w = 0. ${\Sigma }_{Y_i}$ and ${\Sigma }_{X_i}$ are both compound symmetric with ${\sigma }_{ey}^2=3.6$, ${\sigma }_{vy}^2=0.4$ and ${\sigma }_{ex}^2=0.9$, ${\sigma }_{vx}^2=0.1$. We only consider dropout in M2-MDM for computational feasibility with $\text{logit}\left(P_{ij}^{\text{DO}}\right)={\gamma }_{00}+{\gamma }_{01}{Y}_{i,j-1}+{\gamma }_1{Y}_{ij}$. Figure 2 shows the analysis performed using one simulated dataset with a missing proportion of 12.3%. The solid line is the exact sensitivity curve; that is, the profile MLE ${\widehat{\theta }}_x\left({\gamma }_1\right)$ obtained by maximizing the log-likelihood of the joint selection model for a range of specified γ₁ values around 0. The dashed line is the sensitivity curve approximated using the equation

$${\tilde{\theta }}_x\left({\gamma }_1\right)={\widehat{\theta }}_x(0)+\text{ISNIL}\times {\gamma }_1+\frac{1}{2}\text{ISNIQ}\times {\gamma }_1^2.$$

${\widehat{\theta }}_x(0)$ is MAR estimate 3.940, ISNIL = 0.0088, and ISNIQ = 1.8903, all of which were computed without estimating any nonignorable selection models. The dashed line is very close to the solid one, indicating that NISNI captures the exact U-shaped sensitivity curve well without requiring computationally intensive estimation of nonignorable models.

Figure 2:

Application of NISNI to simulation data showing 12.33% missingness. Parameter setting in M2-MDM is γ₀₀ = −5, γ₀₁ = −0.1, and true γ₁ = −1. Two red vertical dashed lines represent ${\gamma }_1=\pm \frac{1}{{\sigma }_Y}$ correspondingly.

We then varied the missing proportion by changing γ₀₀ and γ₁ while fixing γ₀₁ = −0.5. Table 1 summarizes the results obtained from 12 datasets. $\frac{\Delta {\widehat{\theta }}_x}{\Delta {\gamma }_1}$ and $\frac{{\Delta }^2{\widehat{\theta }}_x}{\Delta {\gamma }_1^2}$ are the first and second derivatives, respectively, at the MAR model calculated directly from the exact sensitivity curve. ISNIL and ISNIQ calculated using Eq.(3) and (4), respectively, are close to the true values, showing that NISNI indeed accurately calculates first and second derivatives without fitting any nonignorable models. Computation time is reduced from hours to mere seconds. The increase in computational efficiency is higher for datasets with larger missing proportions. Over all the datasets, ISNIL values are all very small while MinNI^L values are large, indicating negligible linear sensitivity of ${\widehat{\theta }}_x(0)$. In contrast, ISNIQ values are relatively large with MinNI^Q approaching the cuto value of 1. This is expected because when the X and Y are missing simultaneously, ${\widehat{\theta }}_x\left({\gamma }_1\right)$ can be U-shaped. Hence, ISNIL may not adequately capture sensitivity around the MAR model and it is necessary to examine NISNI. Furthermore, ISNIL and ISNIQ values are not related to γ₁ values. This is expected because sensitivity analysis should not inform nonignorability magnitude.

Table 1:

NISNI results for simulation data, γ₀₁ = −0.5.

γ00	γ1	Prop_M	${\widehat{\theta }}_x(0)$ (S.E.)	$\frac{\Delta {\widehat{\theta }}_x}{\Delta {\gamma }_1}$	$\frac{{\Delta }^2{\widehat{\theta }}_x}{\Delta {\gamma }_1^2}$	ISNIL	MinNIL	ISNIQ	MinNIQ
−1	1	42.0%	3.96 (0.14)	−0.025	0.986	−0.021	28.47	0.978	2.18
	0.5	33.0%	3.95 (0.13)	0.005	1.153	0.006	105.55	1.168	2.10
	−1	23.7%	3.74 (0.14)	−0.038	2.252	−0.041	13.74	2.190	1.35
	−0.5	19.0%	3.94 (0.14)	−0.042	1.635	−0.045	12.95	1.637	1.64
−2	1	39.0%	3.96 (0.13)	0.003	1.291	−0.001	707.45	1.289	1.95
	0.5	26.0%	3.99 (0.13)	−0.009	1.424	−0.006	105.69	1.436	1.89
	−1	10.7%	3.80 (0.14)	−0.009	2.300	−0.009	63.06	2.357	1.36
	−0.5	16.0%	3.95 (0.13)	−0.021	1.245	−0.022	25.20	1.255	1.89
−5	1	20.0%	3.89 (0.13)	−0.001	2.035	−0.00004	13128.22	2.027	1.48
	0.5	9.0%	4.01 (0.12)	−0.026	0.767	−0.026	20.37	0.806	2.30
	−1	7.0%	3.87 (0.12)	−0.004	1.194	−0.006	80.03	1.225	1.84
	−0.5	3.7%	3.99 (0.11)	0.006	0.633	0.006	87.42	0.648	2.65

5. Application to EMA data

5.1. Baseline wave analysis

We applied our method to an EMA dataset obtained from a longitudinal study on adolescent smoking history(PO1CA098262, PI R. Mermelstein).^4,5,12 Data for 461 adolescents (55.1% female; 56.8% white; 52.7% grade 10; mean age 15.7 years) were included in baseline wave analysis. Subjects were all self-reported ever-smokers in grades 9 and 10 at the baseline. During a 7-day period, they provided responses to random prompts and three types of smoking-related event interviews (smoking; decide not to smoke; want to smoke but could not). The random prompts were sent to subjects’ hand-held computers several times every day, and each prompt recorded the corresponding date-time information. Random interviews asked questions about mood, activity, location, companionship, presence of other smokers, substance use, and other behaviors. Adolescents were trained to provide a smoking report when they smoked and completed prompt questions just after smoking. Questions in smoking reports included the same questions asked in the random prompts plus additional smoking-related items. With the enormous volume of data collected, we are interested in exploring potential factors related to changes in adolescents’ positive moods.

It is reasonable to suspect that some non-responses are related to participant mood. For example, adolescents tend not to respond when their moods are low, so conventional analysis methods assuming MAR could yield biased results. Therefore, we applied NISNI to evaluate potential impact of such nonrandom missingness on MAR estimates.

5.1.1. Model

Outcome variable posaff (marginal mean 6.80 with SD 1.94) refers to positive affect calculated by taking the average of the following positive mood assessment items: Happy, Relaxed, Cheerful, Confident, and Accepted, each rated from 1 to 10. We consider the following multi-level ideal outcome model for posaff in the form of Eq.(1) with

$${\text{posaff}}_{ij}={\left(\text{Int},\text{ Hour, Smoker, AloneBS, Male, Grade10, Weekday}\right)}_{ij}^T{\theta }_w+{\text{sociso}}_{ij}{\theta }_x+B_{0i}+B_{1i}{\text{Hour}}_{ij}+{\epsilon }_{ij}^y,$$

where Hour is time in hours after midnight. Following Hedeker et al.,⁴ we included several potential predictors, random intercept and slope for Hour in the LMM. AloneBS indicates the proportion of random prompts wherein a subject was alone. An adolescent was defined as Smoker=1 if he or she had at least one smoke interview record. Male is a demographic binary variable where 1 indicates male, and Grade10 is a demographic binary variable where 1 indicates 10th grade. Weekday is a nominal variable indicating day of the week. Sociso measures social isolation and is missing simultaneously with the outcome variable. It is calculated by taking the average of social isolation assessment items: Lonely, Left out, and Ignored, each rated from 1 to 10. We may include sociso in the model for several reasons. It could be a confounding or mediating variable for the relationship between posaff and Smoker such that the analyst would like to include it in the outcome model. Sociso might also be related to missingness, and including it in the analysis can render MAR assumption more plausible. We posit the following ideal covariate model for sociso: ${\text{sociso}}_{ij}={\left(\text{Int},\text{ Hour},\text{ Smoker, AloneBS, Male, Grade10, Weekday}\right)}_{ij}^T{\psi }_w+b_{0i}+b_{1i}{\text{Hour}}_{ij}+{\epsilon }_{ij}^x$. We assume the following two types of M2-MDMs for sensitivity analysis (u = I, D, υ = g_i,j−1).

(1) M2-MDM for Y-dependent nonignorability:

$$\text{ln}\left(\frac{P_{ij}^{uv}}{P_{ij}^{0v}}\right)={\left({\left(\text{Int},\text{\hspace{0.17em}Hour},\text{\hspace{0.17em}Smoker}\right)}_{ij},{\text{ posaff}}_{i,j-1}\right)}^T{\gamma }_0^{uv}+{\text{ posaff}}_{ij}{\gamma }_1,$$

or (2) M2-MDM for Y-and X-dependent nonignorability:

$$\text{ln}\left(\frac{P_{ij}^{uv}}{P_{ij}^{0v}}\right)={\left({\left(\text{Int},\text{\hspace{0.17em}Hour},\text{\hspace{0.17em}Smoker}\right)}_{ij},{\text{posaff}}_{i,j-1}\right)}^T{\gamma }_0^{uv}+{\text{ posaff}}_{ij}{\gamma }_{1y}+{\text{sociso}}_{ij}{\gamma }_{1x}.$$

5.1.2. Results

We first conduct MAR analysis and then calculate NISNI for each parameter, using formulas derived in Section 2. A correction interval is calculated for each parameter, representing the range of the corresponding profile MLE when nonignorability is moderate (i.e. when ${\gamma }_1=\pm \frac{1}{{\sigma }_Y}$ for M2-MDM (1) or $\Vert {\gamma }_1^{*}\Vert =1$ for M2-MDM (2)). Results are shown in Tables 2 and 3 for M2-MDM (1) and M2-MDM (2), respectively.

Table 2:

Y-dependent nonignorability NISNI results for EMA baseline data, N = 461, $\sum n_i^{\text{obs}}=14105$, $\sum n_i^{\text{mis}}=5645$.

Parameter	MAR	SE	ISNIL	ISNIQ	Linear Correction		Nonlinear Correction
	Est				$\text{Est }\left({\gamma }_1=\pm \frac{1}{{\sigma }_Y}\right)$	MinNIL	$\text{Est }\left({\gamma }_{1y}=\pm \frac{1}{{\sigma }_Y}\right)$	MinNIQ
Intercept	7.712***	0.130	0.816	0.466	[7.291, 8.134]	0.31	[7.353, 8.196]	0.32
Hour	0.019***	0.003	−0.012	0.004	[0.013, 0.025]	0.54	[0.013, 0.026]	0.57
smoker	−0.032	0.095	0.099	0.043	[−0.083, 0.019]	1.86	[−0.078, 0.025]	2.64
sociso	−0.418***	0.007	−0.0004	−0.189	[−0.418, −0.418]	38.54	[−0.443, −0.418]	0.54
AloneBS	−0.900**	0.322	−1.284	0.252	[−1.563,−0.236]	0.49	[−1.530, −0.202]	0.50
Male	0.199*	0.098	0.230	−0.144	[0.081, 0.318]	0.82	[0.061, 0.299]	0.73
Grade10	0.031	0.096	0.091	−0.002	[−0.016, 0.078]	2.03	[−0.017, 0.078]	2.00
Tuesday	−0.051	0.043	0.045	0.032	[−0.075,−0.028]	1.85	[−0.070, −0.023]	1.46
Wednesday	−0.103*	0.043	0.037	0.013	[−0.122, −0.084]	2.28	[−0.121, −0.082]	3.24
Thursday	−0.060	0.044	−0.006	0.009	[−0.063,−0.057]	14.99	[−0.062, −0.056]	4.85
Friday	0.053	0.043	0.084	−0.041	[0.010, 0.097]	1.00	[0.004, 0.091]	0.90
Saturday	0.203***	0.044	0.193	−0.036	[0.104, 0.303]	0.44	[0.099, 0.298]	0.43
Sunday	0.120**	0.044	0.031	−0.050	[0.104, 0.136]	2.74	[0.097, 0.129]	1.63

^***p < .001,

^**p < .01,

^*p < .05.

Table 3:

Y-and X-dependent nonignorability NISNI results for EMA baseline data, N = 461, $\sum n_i^{\text{obs}}=14105$, $\sum n_i^{\text{mis}}=5645$.

Parameter	MAR	SE	ISNIL	ISNIQ	Linear Correction		Nonlinear Correction
	Est			(ISNIQyy,ISNIQxy,ISNIQxx)	$\text{Est}\left({\gamma }_{1y}=\pm \frac{1}{{\sigma }_Y}\right)$	MinNIL	$\text{Est }\left({\gamma }_1^{*}=1\right)$	MinNIQ
Intercept	7.712***	0.130	(0.816,0)	(0.466,−0.550,0)	[7.291, 8.134]	0.31	[7.345, 8.201]	0.29
Hour	0.019***	0.003	(−0.012,0)	(0.004,−0.005,0)	[0.013, 0.025]	0.54	[0.013, 0.026]	0.52
smoker	−0.032	0.095	(0.099,0)	(0.043,−0.057,0)	[−0.083, 0.019]	1.86	[−0.078, 0.025]	1.51
sociso	−0.418***	0.007	(−0.0004,0)	(−0.189,0.219,0)	[−0.418, −0.418]	38.54	[−0.449, −0.412]	0.41
AloneBS	−0.900**	0.322	(−1.284,0)	(0.252,−0.038,0)	[−1.563,−0.236]	0.49	[−1.530, −0.202]	0.47
Male	0.199*	0.098	(0.230,0)	(−0.144,0.121,0)	[0.081, 0.318]	0.82	[0.061, 0.300]	0.73
Grade10	0.031	0.096	(0.091,0)	(−0.002,0.037,0)	[−0.016, 0.078]	2.03	[−0.017, 0.078]	1.88
Tuesday	−0.051	0.043	(0.045,0)	(0.032,0.002,0)	[−0.075,−0.028]	1.85	[−0.070, −0.023]	1.46
Wednesday	−0.103*	0.043	(0.037,0)	(0.013,0.006,0)	[−0.122, −0.084]	2.28	[−0.121, −0.082]	1.92
Thursday	−0.060	0.044	(−0.006,0)	(0.009,−0.002,0)	[−0.063,−0.057]	14.99	[−0.062, −0.056]	4.33
Friday	0.053	0.043	(0.084,0)	(−0.041,−0.001,0)	[0.010, 0.097]	1.00	[0.004, 0.091]	0.90
Saturday	0.203***	0.044	(0.193,0)	(−0.036,−0.019,0)	[0.104, 0.303]	0.44	[0.099, 0.298]	0.43
Sunday	0.120**	0.044	(0.031,0)	(−0.050,−0.032,0)	[0.104, 0.136]	2.74	[0.097, 0.130]	1.55

^***p < .001,

^**p < .01,

^*p < .05.

MAR estimates show that effects of Intercept, Hour, sociso, AloneBS, Male, Wednesday, Saturday, and Sunday are significant, indicating that the adolescents feel more positive at night on weekend, when they are less isolated, have fewer chances to be alone, or are male. Intercept, Hour, AloneBS, Male, and Saturday have MinNI statistics less than 1 under both linear and nonlinear sensitivity analyses, suggesting that potential impact of nonignorability is considerable and that such estimates can be sensitive to nonignorable missingness. Especially for variables AloneBS and Male, the p-value may be greater than 0.05 when nonignorability is moderate, thereby changing the significance level. MinNI^L and MinNI^Q values are similar and lead to the same qualitative conclusions based on a cut-off value of 1 for important sensitivity, indicating that for fully observed covariates, ISNIL will be sufficient to measure sensitivity to nonignorability. Because Intercept measures the conditional mean of positive mood at random prompts, it is understandable that nonignorable missingness has a noticeable impact on such estimations. The sign of ISNIL for Intercept informs the direction for adjustment of the MAR estimate. For example, if the true γ₁ > 0, missingness probability will increase with increasing posaff, and smaller posaff values are more likely to be observed. Therefore, MAR estimate yields underestimated results, and a positive ISNI means that the estimate should be adjusted upward. MinNI statistics for sociso are 38.54 and 0.54 for linear and nonlinear sensitivity analyses, respectively, as listed in Table 2, demonstrating the importance of using NISNI to capture U-shaped impact of nonignorable missingness on regression coefficient parameters for a covariate concurrently missing with outcome. The same finding is listed in Table 3.

When considering Y- and X-dependent nonignorable missingness (i.e., as in M2-MDM (2)), MinNI^Q decreases, and some correction intervals widen (i.e., as in Table 3 vs. Table 2), which is expected because Y-dependent nonignorability is a special case of Y- and X- dependent nonignorability. Thus, for the same total size of nonignorability, the correction interval obtained when only considering γ_1y should be nested within that obtained when considering both γ_1y and γ_1x. However, no significant difference is detected between sensitivity measures of the two M2-MDMs, indicating that origin of nonignorability missingness may not significantly impact the maximum sensitivity to nonignorability for parameters in the outcome model.

5.2. Multi-wave analysis

5.2.1. Model

We also applied our method to analyze multiple waves of EMA data. Besides the baseline wave, each subject was followed-up at 6, 15, and 24 months and possibly at 5 and 6 years, thereby generating at most 5 more waves of data. For follow-up years 5 and 6, only the subset of participants who had provided smoking data on their EMA prompts were again recruited to participate in the years 5 and 6 EMA portion of the study, given the interest then to focus on smoking contexts. Among those 461 subjects who had baseline wave measurements, 123 subjects had at least one observation in waves 5 or 6 data and are included in the multi-wave analysis. Because sociso was not collected in waves 5 and 6 data, tirbor, indicating the tired/bored scale, is included in the model instead. Similarly, tirbor is calculated by taking the average of the tired and bored assessment items: Tired, Bored, and Trouble Concentrating, each rated from 1 to 10. Since the 123 subjects all eventually became smokers, smoker indicates a smoker in the baseline wave. AloneBS is calculated separately for each wave. The ideal outcome model and ideal covariate model are:

$${\text{posaff}}_{ij}={\left({\text{Int}}_{\text{wave}},\text{\hspace{0.17em}Hour},\text{\hspace{0.17em}Smoker},\text{\hspace{0.17em}AloneBS},\text{\hspace{0.17em}Male},\text{\hspace{0.17em}Grade10},\text{\hspace{0.17em}Weekday}\right)}_{ij}^T{\theta }_w+{\text{tirbor}}_{{\text{wave}}_{ij}}{\theta }_x+B_{0i}+B_{1i}{\text{Hour}}_{ij}+{\epsilon }_{ij}^y,\text{ and}$$

$${\text{tirbor}}_{{\text{wave}}_{ij}}={\left(\text{Int},\text{\hspace{0.17em}Hour},\text{\hspace{0.17em}Smoker},\text{\hspace{0.17em}AloneBS},\text{\hspace{0.17em}Male},\text{\hspace{0.17em}Grade}10,\text{\hspace{0.17em}Weekday}\right)}_{ij}^T{\psi }_w+b_{0i}+b_{1i}{\text{Hour}}_{ij}+{\epsilon }_{ij}^{X_{\text{wave}}}.$$

5.2.2. Results

Results are summarized in Table 4. For each wave, tirbor has a very significant negative impact on posaff, showing that the adolescents feel less positive when they are tired or bored or have trouble concentrating. Similar to conclusions in the baseline wave analysis, ISNIL tirbor values are very close to zero, while ISNIQ values are non-zero, and MinNI^Q values are smaller than 1, which again demonstrates the necessity of using NISNI to capture U-shaped impact of nonignorable missingness for tirbor.

Table 4:

NISNI results for multi-wave EMA dataset, N = 123.

Parameter	MAR	SE	ISNIL	ISNIQ	Linear Correction		Nonlinear Correction
	Est				$\text{Est}\left({\gamma }_1=\pm \frac{1}{{\sigma }_Y}\right)$	MinNIL	$\text{Est}\left({\gamma }_{1y}=\pm \frac{1}{{\sigma }_Y}\right)$	MinNIQ
Wave 1	8.050***	0.220	0.723	0.765	[7.666, 8.434]	0.57	[7.774, 8.542]	0.72
Wave 2	7.525***	0.220	0.753	0.613	[7.125, 7.925]	0.55	[7.211, 8.011]	0.64
Wave 3	7.723***	0.221	0.898	0.677	[7.246, 8.200]	0.46	[7.342, 8.295]	0.52
Wave 4	7.806***	0.220	0.774	0.561	[7.395, 8.217]	0.54	[7.474, 8.296]	0.61
Wave 5	8.140***	0.219	0.455	0.365	[7.899, 8.382]	0.91	[7.950, 8.433]	1.23
Wave 6	7.940***	0.220	0.504	0.400	[7.673, 8.208]	0.82	[7.729, 8.265]	1.06
Hours	0.008**	0.002	−0.005	−0.002	[0.005, 0.011]	0.91	[0.005, 0.010]	0.83
Tirbor 1	−0.284***	0.011	0.007	−0.134	[−0.286, −0.281]	2.94	[−0.306, −0.284]	0.65
Tirbor 2	−0.193***	0.011	0.007	−0.105	[−0.197, −0.190]	3.05	[−0.211, −0.193]	0.75
Tirbor 3	−0.199***	0.011	−0.005	−0.126	[−0.202, −0.196]	3.92	[−0.220, −0.199]	0.73
Tirbor 4	−0.211***	0.011	−0.010	−0.116	[−0.216, −0.206]	2.18	[−0.232, −0.211]	0.68
Tirbor 5	−0.244***	0.010	−0.003	−0.071	[−0.245, −0.242]	6.85	[−0.255,7 −0.244]	0.92
Tirbor 6	−0.209***	0.011	−0.001	−0.078	[−0.210, −0.209]	16.99	[−0.221, −0.209]	0.96
smoker	0.064	0.204	0.005	0.050	[0.061, 0.067]	78.60	[0.064, 0.074]	5.19
AloneBS	−0.492***	0.074	−0.712	0.073	[−0.870, −0.114]	0.19	[−0.860, −0.103]	0.20
Male	−0.026	0.177	0.144	−0.162	[−0.103, 0.051]	2.31	[−0.126, 0.028]	1.57
Grade10	0.060	0.173	0.007	−0.066	[0.056, 0.064]	47.48	[0.047, 0.060]	4.12
Tuesday	−0.103**	0.034	−0.015	0.024	[−0.111, −0.096]	4.28	[−0.108, −0.092]	2.20
Wednesday	−0.082*	0.034	−0.004	0.021	[−0.084, −0.080]	15.26	[−0.082, −0.077]	3.03
Thursday	−0.071*	0.034	−0.014	0.013	[−0.079, −0.064]	4.70	[−0.077, −0.062]	2.74
Friday	−0.047	0.034	0.016	−0.003	[−0.055, −0.038]	3.99	[−0.056, −0.039]	3.39
Saturday	−0.057	0.034	0.117	−0.007	[−0.119, 0.005]	0.56	[−0.119, 0.004]	0.55
Sunday	−0.092**	0.034	0.050	0.011	[−0.119, −0.066]	1.28	[−0.118, −0.064]	1.39

^***p < .001,

^**p < .01,

^*p < .05.

To compare magnitudes of sensitivity across waves, we examine Figure 3, which shows trends of missing percentage, |ISNIQ|, and MinNI^Q across all 6 waves. The patterns for missing percentage and |ISNIQ| are similar, although they are not parallel, which is consistent with the amount of missing data being an important determinant of sensitivity. To help understand why, consider the simple closed-form expressions of NISNI for θ_w and θ_x in Section 2.3.1. For the intercept, |ISNIL| is positively associated with the missing percentage, and |ISNIQ| for both θ_w and θ_x are related to p_m(1 − p_m). Therefore, when other quantities in the formula remain the same, |ISNIQ| will increase with increasing p_m from 0 to 0.5, reaching the maximum when p_m = 0.5. However, as shown in the NISNI formula in Section 2.3.1, other quantities such as MAR estimates can impact both |NISNI| and MinNI. For example, if only referring to p_m, we would expect |ISNIQ| for wave 1 to be greater than that for wave 2. However, the results are reversed, which may be because the MAR estimates for the two waves are quite different (−0.286 versus −0.197) compared to the small difference in missingness percentage (0.252 versus 0.269), respectively. Thus, NISNI values can be considered as a generalization of missing percentages that account for complex features of models and data when measuring sensitivity.

Figure 3:

Relationships among missing percentages, |ISNIQ|, and MinNI^Q for ${\widehat{\beta }}_{\mathit{\text{Tirbor}}}$ across all 6 waves.

Although the effect of Male is no longer significant, the finding that the adolescents feel more positive at night or when less alone remains valid. However, Monday now becomes the day that adolescents feel more positive on average, and their moods tend to be lower especially during the middle of the week or on Sunday. AloneBS and Saturday have MinNI^L and MinNI^Q values less than 1, suggesting that potential impact of nonignorability may be large, and these estimates can be sensitive to nonignorable missingness. Especially for AloneBS, the p-value may be greater than 0.05 when nonignorability is moderate, thereby changing the significance level. For Thursday and Sunday, although MinNI values are larger than 1, the upper bound of the sensitivity interval will have a p-value slightly greater than 0.05.

6. Discussion

Missing data are often unavoidable in ILD. Currently, regular ILD analysis often makes the strong MAR assumption about missing-data processes, which cannot be verified using only observed data. If the assumption is invalid, selection bias can occur, and the analysis results may be incorrect. Therefore, sensitivity analysis is necessary, and it helps researchers gauge credibility of standard analysis. The NISNI method developed herein relaxes the MAR assumption and provides a computationally feasible method of screening local sensitivity to nonignorability for parameters of interest in ILD analyses.

We have extended the NISNI method to ILD showing missingness in both outcome and covariates. Analyses of simulated data and real EMA data show that when outcome and covariates are missing simultaneously, impact of nonignorability on regression coefficients for missing covariates is not monotonic around the MAR model. Application of NISNI to EMA data in both baseline and multi-wave analyses of the adolescent smoker cohort reveals that in estimating the function of mood regulation, nonignorable missingness leads to attenuation bias (i.e., under-estimation of true effects) in the MAR effect estimates of the time-varying predictors missing simultaneously with mood outcome, regardless of whether better or worse mood is more likely to cause non-responses. This new finding of unidirectional impact of nonignorability is useful for EMA researchers to understand the nature of such nonmonotonic impacts on EMA data analysis and to measure/control for such impacts. In such cases, ISNIL is no longer sufficient to quantify parameter sensitivity to nonignorability. By developing ISNIQ, our NISNI method can readily capture U-shaped impact of nonignorability on parameters of interest.

This method is especially useful for addressing curse of dimensionality encountered in high-dimensional nonignorable missing data in ILD, which usually involves numerous measurement occasions for each subject. To perform sensitivity analysis on nonignorable missingness, the brute-force approach to fitting complicated nonignorable models becomes computationally prohibitive. The NISNI method avoids fitting nonignorable models, making such sensitivity analysis feasible to perform in modern data-rich environments.

Recently, shared-parameter mixed effects models have been developed to relax the MAR assumption in EMA data analysis.^13,38 Such models assume conditional independence of missingness and unobserved data given random effects. This differs from our models, which permit missingness probability to depend directly on time-varying unobserved outcome and covariates. The difference between the two modeling approaches is analogous to the difference between handling time-constant and time-varying confounders. As a result, shared-parameter models do not have the curse of dimensionality issues, as encountered in our models, so long as dimensionality of random effects does not increase with increasing number of missing occasions. Even so, Cursio et al.¹³ documented slow convergence issues (i.e., hours of computation time) when fitting their model to EMA data. Thus, it is not surprising that use of brute-force to perform sensitivity analysis by directly estimating a range of nonignorable selection models, as in our modeling approach, is computationally prohibitive, and we must solve the curse of dimensionality problem for practical sensitivity analysis.

Although we illustrate our approach using EMA data, the method also is applicable to other types of ILD. More studies on pain, diet, and physical activity involve data collection methods similar to those used to collect EMA data, and there are similar missing-data issues in such studies for which our NISNI method is applicable. Our method also can be applied to traditional short-panel clinical trials and observational data, where both outcome and time-varying covariates can be missing when a subject misses a visit.

Given the modeling flexibility and computational simplicity of ISNI analysis, researchers can specify different M2-MDMs and perform ISNI analysis to examine robustness of MAR estimates under a given set of posited scenarios. Results to date have demonstrated that conclusions about local sensitivity to nonignorability are considerably less sensitive to MDM specification, than conclusions drawn from joint estimation of all the parameters in nonignorable models.^15,28,32 This is intuitive because ISNI analysis fixes the sensitivity parameter γ₁, for which data provide little information and which is the major source of sensitivity to model assumptions.

When ISNI analysis flags MAR estimates as being unreliable, researchers may choose to conduct speculative nonignorable analysis. To increase confidence in such an analysis, it is helpful to garner additional information on MDM for more-robust modeling and estimation of nonignorable selection models, e.g., by collecting refreshment samples or ascertaining a sample of missing values. In such situations, ISNI analysis can be useful for informing researchers about the need for and allocation of resources for additional data collection efforts. For instance, statisticians may consider performing ISNI analysis during data collection to monitor sensitivity to nonignorable missingness and assess need for additional data collection.

One limitation of the current study is that we only developed the NISNI method under the condition that each of the variables subject to missingness follows a multivariate normal (MVN) distribution and is modeled using LMM. By introducing random effects, LMM can model complex var-cov matrices. Furthermore, marginal distribution after integration over random effects is also MVN, allowing us to obtain closed-form expressions for all the NISNI terms. Therefore, the NISNI method avoids numerical evaluation of derivatives and expectations and further reduces computational workload. However, in real datasets, there are other data distributions that are more appropriately modeled using generalized linear mixed effects models. Therefore, it would be useful to extend the NISNI method to other data types. Although closed-form expressions for NISNI terms may be unavailable, computational workload can still be considerably less than that required for brute-force sensitivity analysis.