Nonstandard conditionally specified models for nonignorable missing data

Significance

We consider data-analysis settings where data are missing not at random. In these cases, the two basic modeling approaches are 1) pattern-mixture models, with separate distributions for missing data and observed data, and 2) selection models, with a distribution for the data preobservation and a missing-data mechanism that selects which data are observed. These two modeling approaches lead to distinct factorizations of the joint distribution of the observed-data and missing-data indicators. In this paper, we explore a third approach, apparently originally proposed by J. W. Tukey as a remark in a discussion between Rubin and Hartigan, and reported by Holland in a two-page note, which has been so far neglected.

Abstract

Data analyses typically rely upon assumptions about the missingness mechanisms that lead to observed versus missing data, assumptions that are typically unassessable. We explore an approach where the joint distribution of observed data and missing data are specified in a nonstandard way. In this formulation, which traces back to a representation of the joint distribution of the data and missingness mechanism, apparently first proposed by J. W. Tukey, the modeling assumptions about the distributions are either assessable or are designed to allow relatively easy incorporation of substantive knowledge about the problem at hand, thereby offering a possibly realistic portrayal of the data, both observed and missing. We develop Tukey’s representation for exponential-family models, propose a computationally tractable approach to inference in this class of models, and offer some general theoretical comments. We then illustrate the utility of this approach with an example in systems biology.

Missing data are ubiquitous in the social and biomedical sciences, and the credibility of any data analysis is dependent on the assumed mechanism that leads to the missing data, as well as on the mode of inference (1). Here, we work within a framework in which the estimand involves both observed and missing data (2). An important concept is that of ignorable missing data under which there is no need to specify a model for the missingness indicators to achieve valid Bayesian or likelihood-based inference (3–5).

There are two basic approaches to specify the joint distribution of the complete data (observed and missing) and missingness indicators. The first approach, called selection modeling, is to posit a standard model for the complete data and then specify a model that selects observed data from the complete data, referred to as the missingness mechanism (6). The second approach, called pattern-mixture modeling, is to specify separate distributions for each pattern of observed and missing data, thus eschewing explicit assumptions about the missingness mechanism (7, 8). The fundamental challenge with these two basic approaches is that assumptions about the missingness mechanism, whether explicit or implicit, are rarely testable from the observed data. As a result, literature on inference in the presence of missing data includes strategies for assessing sensitivity to model specifications (1, 7, 9, 10), often using a model assuming ignorability as a baseline.

Contributions

Most statistical analyses involving nonignorable missingness mechanisms use one of two approaches: pattern-mixture models or selection models. Here, we develop an alternative approach, evidently originally proposed by J. W. Tukey in a discussion of ref. 11, and described by ref. 12, which has thus far remained recondite. The key insight is to represent the joint distribution of the complete data and missingness indicators in a way such that assumptions are either assessable or, typically, allow the incorporation of substantive knowledge about the problem at hand, thereby offering a path to elicit a realistic portrayal of the data. This work is related to previous work on, so called, exponential-tilt pattern-mixture models for nonignorable missing data (13–16). We make the connection between exponential tilting and Tukey’s representation, but we focus explicitly on describing the utility of Tukey’s representation for Bayesian inference with nonignorable missing data (16). Also, we introduce a class of flexible and widely applicable models based on logistic missingness mechanisms and exponential-family models for the observed data.

Basic Models for Missing Data.

Discussion of models for missing data can be found in a variety of places, including (11, 17). We introduce ideas in the simple case where the data are exchangeable scalar random variables.

Let $Y={\left(Y_1,Y_2,\dots ,Y_N\right)}^{\prime }$ be the complete data and $R={\left(R_1,R_2,\dots ,R_N\right)}^{\prime }$ represent the response indicators for $Y$; $Y_i$ is “missing” when $R_i$ = 0 and “observed” when $R_i=1$. Assuming independence between observations $\left(Y_i,R_i\right)$, the joint distribution for $\left(Y_i,R_i\right)$ can be written in independent and identically distributed (i.i.d.) form:

$$P\left(Y,R\mid \theta \right)=\prod _{i}f\left(Y_i,R_i\mid \theta \right),$$

where $\theta$ is a parameter vector with a prior distribution. For notational simplicity, we focus on the case without covariates although the approach can easily be extended to include covariates (Theory).

The Selection Factorization.

The selection approach (6) factors the joint distribution of $\left(Y_i,R_i\right)$ as

$$f\left(Y_i|{\theta }_Y\right)f\left(R_i|Y_i,{\theta }_{R|Y}\right),$$ [1]

using the distribution of the complete data, $P\left(Y_i|{\theta }_Y\right)$, and the missingness mechanism, i.e., $P\left(R_i|Y_i,{\theta }_{R|Y}\right)$, which controls which complete-data values are observed and which are missing, where the parameters ${\theta }_Y$ and ${\theta }_{R|Y}$ are for the complete-data and missingness-mechanism parameters, respectively. Typical selection modeling has ${\theta }_Y$ distinct from ${\theta }_{R|Y}$, where “distinct” (3) means in disjoint parameter spaces and a priori independent (if distributions are specified). Common models for $f\left(Y_i|{\theta }_Y\right)$ include the normal, with mean $\mu$ and variance ${\sigma }^2$, i.e., $f\left(Y_i|{\theta }_Y\right)\sim N\left(\mu ,{\sigma }^2\right)$ with ${\theta }_Y=\left(\mu ,{\sigma }^2\right)$, or the Bernoulli with unknown probability of success, i.e., $f\left(Y_i|{\theta }_Y\right)\sim \text{Bern}\left(p\right)$ with ${\theta }_Y=p$. Common missingness mechanisms used in practice include the logistic and probit models (18), for the former $f\left(R_i|Y_i,{\theta }_{R|Y}\right)\sim {\text{logit}}^{-1}\left(\alpha +\beta Y_i\right)$ with ${\theta }_{R|Y}=\left(\alpha ,\beta \right)$.

The Pattern-Mixture Factorization.

The pattern-mixture approach (7, 8) is the alternative basic factorization. Here, the complete-data distribution is specified as a mixture of observed data and missing-data components,

$$f\left(Y_i,R_i|\theta \right)=f\left(Y_i|R_i,{\theta }_{Y|R}\right)f\left(R_i|{\theta }_R\right),$$ [2]

where ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\left({\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}},{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$ and thus $\theta =\left({\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}},{\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}},{\theta }_R\right)$ are the observed-data parameters, missing-data parameters, and the population fraction of observed data, respectively. Here $f\left(R_i|{\theta }_R\right)$, a Bernoulli distribution, is easily estimated by the fraction of indicators equal to one. The model for $f\left(Y_i|R_i=1,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$ is typically chosen to fit the observed data well. There is no information in the observed data or indicators, $R_i$, about ${\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}}$. This factorization omits an explicit specification for the missingness mechanism, about which there may be substantive knowledge. Tukey’s representation offers another choice.

Tukey’s Representation.

John W. Tukey, recorded in ref. 12, suggested an alternative representation of the joint distribution for $\left(Y_i,R_i\right)$, which he referred to as the “simplified selection model,” with parameters ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}$, ${\theta }_{R|Y}$, and ${\theta }_R$:

$$\begin{array}{ll}\hfill f\left(Y_i,R_i\mid \theta \right)& =f\left(Y_i\mid R_i=1,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\hfill \\ \hfill & \hspace{1em}\times \hspace{0.17em}f\left(R_i=1\mid {\theta }_R\right)\cdot \frac{f\left(R_i\mid Y_i,{\theta }_{R|Y}\right)}{f\left(R_i=1\mid Y_i,{\theta }_{R|Y}\right)},\hfill \end{array}$$ [3]

where ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}$ are the parameters of the observed data density. The missingness mechanism describes the probability that $Y_i$ is observed or missing given its value. Here, we focus on models using Tukey’s representation when the observed-data distribution is an exponential-family distribution and when the missingness mechanism$f\left(R_i=1\mid Y_i,{\theta }_{R|Y}\right)$ is the inverse-logit of some function of $Y_i$. Tukey’s representation can be obtained from Bayes’ rule or through an application of Brook’s lemma (19, 20), commonly referenced in the theory of spatial autoregressive models (21). Brook’s Lemma is only applicable when the so-called “positivity condition” (due to Hammersley and Clifford) is satisfied (20), which for Tukey’s representation means that

$$\begin{array}{ll}\hfill & \text{If}\hspace{0.17em}P\left(R_i=r|{\theta }_R\right)>0\hspace{0.17em}\text{and}\hspace{0.17em}P\left(Y_i=y|{\theta }_Y\right)>0\hfill \\ \hfill & \text{Then}\hspace{0.17em}P\left(R_i=r,Y_i=y|\theta \right)>0\hfill \end{array}$$

for all pairs of values $\left(r,y\right)$, where ${\theta }_Y$ are the parameters for the complete-data distribution of $Y_i$.* This condition enforces regularity in the way the supports of the marginal distributions relate to the support of the corresponding joint distribution and avoids pathological cases (e.g., the case in ref. 22). This condition is not trivially satisfied in missing-data problems. For instance, Tukey’s representation cannot be applied to models where $P\left(R_i=1|Y_i<c,{\theta }_{R|Y}\right)=0$, for some finite $c$, as when the complete data are normal but the observed data are truncated normal. Consequently, here we focus on problems where $P\left(R_i=1|Y_i,{\theta }_{R|Y}\right)>0$, that is, where the support of the missing data is a nontrivial subset of the support of the observed data. Moreover, as we discuss later, the distributions specified in Eq. 3 must imply an integrable joint density (20). With Tukey’s representation, the “integrability condition” constrains the rate at which the tails of the distribution for the observed data decrease relative to the rate at which the odds of a missing value increase. This condition is further discussed in A Note on the Integrability Condition.

Unlike related work with exponential-tilting models discussed in Connections to Exponential Tilting, Tukey’s approach focuses on an explicit formulation of the missingness mechanism. We describe the utility of Bayesian inference with Tukey’s representation, and we show that it is tractable when the missingness mechanism is logistic and the observed-data distribution is in the exponential family.

Advantages and Challenges of Tukey’s Representation.

A notion central to the arguments in this paper is that although joint distributions can be represented mathematically in several ways, a particular representation may involve components that are more easily elicited from investigators or more easily estimated from data. As Holland notes (12), a main advantage of Tukey’s representation is that it involves the observed-data density, $f\left(Y_i\mid R_i=1,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$, and the marginal probability of a missing observation, both of which can be estimated directly, and the missingness mechanism, $f\left(R_i\mid Y_i,{\theta }_{R|Y}\right)$, which is often natural to elicit in the context of a specific application.

Modeling and Inference Using Tukey’s Representation.

Let $f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y\mid R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$ denote the observed-data density parameterized by ${\theta }_{Y\mid R}^{\mathrm{o}\mathrm{b}\mathrm{s}}$. Using Eq. 3, we can write the joint density for $\left(y_i,r_i\right)$ as

$$f\left(y_i,r_i\mid \theta \right)\propto \left\{\begin{array}{cc}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\hspace{1em}\hfill & \text{if}\hspace{0.17em}{r}_i=1\hfill \\ \frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i|{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\hspace{1em}\hfill & \text{if}\hspace{0.17em}{r}_i=0,\hfill \end{array}\right.$$

or

$$f{\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}^{r_i-1}f{\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}^{1-r_i}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right),$$ [4]

with normalization constant

$$\begin{array}{ll}\hfill & Q\left({\theta }_{Y\mid R=1},{\theta }_{R\mid Y}\right)\hfill \\ \hfill & \hspace{1em}={\left(1+\int \frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i|{\theta }_{Y|R}^{obs}\right)\hspace{0.17em}d{y}_i\right)}^{-1}.\hfill \end{array}$$ [5]

Theorem 1 documents that the normalization constant $Q$ is the population fraction of data that are observed. In the class of “exponential-tilt pattern-mixture” models (13), it is assumed that the missing-data density is $f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y_i\right)=e^{m\left(y_i\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\right)$ for some function $m$. Theorem 2 shows that with Tukey’s representation

$$f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y_i\right)\propto \frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\right).$$

As such, Tukey’s representation can be expressed in the exponential-tilt framework where $m\left(y_i\right)\hspace{0.17em}=\hspace{0.17em}\log \left(\frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}\right)\hspace{0.17em}+\text{const}$. Tukey’s representation focuses on parameterizing the meaningful missingness mechanism $f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)$, rather than on a hard to interpret “exponential-tilt function.”

With Bayesian inference with missing data, at each iteration of a Markov chain Monte Carlo (MCMC) procedure, missing data often are imputed, and so in such a setting, it is advantageous for the missing-data density to have a tractable form. Below, we introduce a class of models for which computation of the normalization constant $Q\left({\theta }_{Y\mid R=1},{\theta }_{R\mid Y}\right)$ is tractable and implies simple distributional forms for both the missing-data and complete-data distributions.

Exponential-Family Models.

Suppose the observed-data distribution belongs to an exponential family and that the logit of the missingness mechanism is linear in the sufficient statistics of that family. Formally, let $f^{\mathrm{o}\mathrm{b}\mathrm{s}}$ be an exponential-family distribution with natural parameter ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\eta$, that is,

$$f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)=h\left(y_i\right)g\left(\eta \right)e^{T{\left(y_i\right)}^{\prime }\eta },$$ [6]

where $g\left(\eta \right)$ is the normalization function and $T\left(y_i\right)$ is the natural sufficient statistic. A logistic missingness mechanism in $T\left(y_i\right)$, with ${\theta }_{R|Y}=\left(\alpha ,\beta \right)$,

$$f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)={\text{logit}}^{-1}\left(\alpha +T{\left(y_i\right)}^{\prime }\beta \right)$$ [7]

$$=\frac{1}{1+e^{-\alpha -T{\left(y_i\right)}^{\prime }\beta }},$$ [8]

implies that

$$\frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}=e^{-\alpha -T{\left(y_i\right)}^{\prime }\beta }.$$

Then, as shown in Theorem 3, the normalization function $Q$ in Eq. 5 can be written as a simple function of the normalization constant $g\left(\cdot \right)$ in the exponential-family formulation of $f^{\mathrm{o}\mathrm{b}\mathrm{s}}$,

$$Q\left({\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}},{\theta }_{R|Y}\right)=\frac{g\left(\eta +\beta \right)}{g\left(\eta +\beta \right)+e^{\alpha }g\left(\eta \right)}.$$ [9]

For the class of exponential family (EF)-logistic models defined by Eqs. 6 and 7, the missing-data distribution, as specified in Eq. 10, is from the same exponential family as the observed data with natural parameter ${\theta }_{Y|R}^{mis}$ (Theorem 3).† Here, we have ${\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}}=\eta +\beta$, and

$$f^{mis}\left(y|\eta ,\beta \right)=h\left(y\right)g\left(\eta +\beta \right)e^{{T\left(y\right)}^{\prime }\left(\eta +\beta \right)}.$$ [10]

This statement is formalized in Theorem 2. Missing-data imputation in this model class is straightforward. The EF-logistic model corresponds to an exponential-tilt pattern-mixture model where $m\left(y_i\right)$ is parameterized as a linear function of the sufficient statistics of the observed-data distribution. Since a large source of uncertainty about any inferential target is due to the missingness mechanism, specifying a scientifically justifiable prior distribution for ${\theta }_{R|Y}$ is important.

In Modeling Assumptions and in SI Appendix, we describe extensions of this basic logistic-EF model.

Estimation and Inference.

The primary estimands of interest are typically functions of the parameters specifying the complete-data distribution, ${\theta }_Y$. Because the observed- and missing-data densities for EF-logistic models are both exponential families (Eqs. 6 and 10), the complete-data distribution is a mixture of exponential families. In this paper, we focus on Bayesian inference for ${\theta }_Y$ via the likelihood, which is obtained as a product of terms (4) for each observation. In the EF-logistic model, analytic expressions for the normalization constant $Q$ and the likelihood are available, and thus standard MCMC methods are applicable (23).

Note that an alternative strategy for estimation is to estimate $f^{\text{obs}}\left(y_i\right)$ nonparametrically using the empirical distribution of the observed data. When the complete-data estimands are simple univariate summaries of the complete-data distribution, this strategy is straightforward because it is easy to integrate a function with respect to the empirical distribution. However, this approach ignores the uncertainty about the true observed data density and may also suffer because, as a consequence of the positivity condition of Brook’s lemma, Tukey’s representation is appropriate only when the missing-data density is absolutely continuous with respect to the observed-data distribution. Using the empirical distribution as if it were the true observed-data distribution implies that the complete data, missing data, and observed data must all have the same discrete support as the finite observed data.

We take a simple approach to computation via MCMC, which is computationally less demanding than alternative methods that characterize the geometry of the solution space. Consider a simple normal-logistic model for illustration, with $T{\left(y_i\right)}^{\prime }=\left(y_i,y_i^2\right)$ and ${\theta }_{R|Y}=\left(\alpha ,{\beta }_1,{\beta }_2\right)$ with $\alpha$ corresponding to the intercept and $\beta ={\left({\beta }_1,{\beta }_2\right)}^{\prime }$ the rate at which the odds of selection change in $y_i$ and $y_i^2$. Here, we assume the observed data follows a standard normal distribution, for convenience. Later, we consider more general cases, including a normal distribution with arbitrary mean $\mu$ and SD $\sigma$ (Modeling Assumptions). Concretely,

$$\begin{array}{ll}\hfill f\left(r_i=1\mid y_i,\alpha ,\beta \right)& ={\text{logit}}^{-1}\left(\alpha +{\beta }_1{y}_i+{\beta }_2{y}_i^2\right)\hfill \\ \hfill f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\right)& =\text{Normal}\left(\mathrm{0,1}\right),\hfill \end{array}$$ [11]

where the standard normal distribution has fixed and known natural parameters ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\left({\eta }_1,{\eta }_2\right)=\left(0,-1/2\right)$. Rather than specifying a prior distribution on $\left(\alpha ,{\beta }_1,{\beta }_2\right)$, we specify a prior distribution on $Q$ and $\beta$, but not on $\alpha$. We then solve Eq. 9 for $\alpha$ to obtain

$$\alpha \left(Q,\eta ,\beta \right)=\log \left(\frac{g\left({\eta }_1+{\beta }_1,{\eta }_2+{\beta }_2\right)}{g\left({\eta }_1,{\eta }_2\right)}\frac{\left(1-Q\right)}{Q}\right),$$ [12]

for general $\left({\eta }_1,{\eta }_2\right)$. For the normal distribution, $g\left({\eta }_1,{\eta }_2\right)=\sqrt{-{\eta }_2}{e}^{\frac{{\eta }_1^2}{4{\eta }_2}}$.‡ For simplicity, assume ${\beta }_2=0$, that is, the log odds of missingness is linear in $y_i$ only, and recall that the natural parameters for the standard normal are ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\left({\eta }_1,{\eta }_2\right)=\left(0,-1/2\right)$. Then, Eq. 12 simplifies to

$$\alpha =\frac{-{\beta }_1^2}{2}+\log \left(\frac{1-Q}{Q}\right).$$

We use this strategy for inference to demonstrate the utility of Tukey’s representation.

Illustration on Transcriptomic and Proteomic Data.

We demonstrate the utility of Tukey’s representation by revisiting a recent analysis of biological data aimed at quantifying the coordination between transcription and translation (24).

Scientific Question and Data.

In experiments involving measurements of transcriptomic and proteomic data, messenger RNA (mRNA) transcripts and proteins that occur at low levels are less likely to be observed (25, 26). This makes it challenging to infer normalization constants for absolute protein levels (27), cluster genes into functionally related sets (28), infer the degree of coordination between transcription and translation (24), and determine the ratio of dynamic range inflation from transcript to protein levels (29). Here, we demonstrate how data analysis with Tukey’s representation can be used to investigate some of these issues by assessing the sensitivity of estimands to different assumptions about the missingness mechanism.

In this analysis, we explore imputation of missing values in a dataset of mRNA and protein abundances in yeast Saccharoymyces cerevisiae in exponential-growth phase. We model transcript measurements (mRNA) from ref. 30 and protein measurements from ref. 31 on 5,854 genes. About 14% of the transcript measurements have missing values ($n_{\text{obs}}=5034$), while about 36% missing of the protein measurements have missing values ($n_{\text{obs}}=3747$). These data were gathered in experiments that were designed in part to understand the degree of coordination between transcription and translation, as well as to identify the relative dynamic ranges of transcript and protein abundances. We treat the complete-data mean and variance as estimands of interest in the analysis. We also consider the ratio of SDs from the two datasets as the quantity that describes the relative inflation of dynamic range between mRNA and protein. Note that in this application, we focus on estimands that are functions of marginal quantities only and ignore the dependence between mRNA and protein levels. For more complex estimands, one approach to modeling the multivariate structure would be to incorporate a copula, in addition to the marginal models, which describes the dependence between observations.

Modeling Assumptions.

It is standard to assume that both complete-data mRNA and protein levels are log-normally distributed (32, 33), although this assumption may not be justified (34, 35) and is also not testable. Here, we use Tukey’s factorization; we model the observed data as a mixture of normal distributions and specify a prior distribution for the parameters of the logistic missingness mechanism, instead of modeling the complete data directly. Together, these assumptions imply a more flexible distribution over complete-data densities. We found that a mixture with $K=3$ components gave a reasonable approximation to the observed-data density.

In experiments measuring transcript and protein abundances, molecules that occur at lower abundances are typically much harder to measure. Thus, we expect a nonignorable missing-data mechanism in which the probability of observation decreases monotonically with decreasing abundance. Evidence suggests that a logistic missingness mechanism with a strictly positive slope, $\beta$, is plausible (36). However, as noted in ref. 37, missing values can occur for multiple reasons, at different stages of the data-collection process. Thus, we generalize the EF-logistic model to allow the selection mechanism to have a logistic form that asymptotes at some value less than one. The observed-data distribution and missingness mechanism together define the joint distribution for a single observation $i$:

$$f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i|r_i=1,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\sim \sum _{k=1}^K{w}_{k}N\left(y_i;{\mu }_k,{\sigma }_k^2\right),$$ [13]

$$f\left(r_i=1|y_i,{\theta }_{R|Y}\right)=\frac{\kappa e^{\beta y_i+\alpha }}{1+e^{\beta y_i+\alpha }},$$ [14]

with ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\left(\mu ,\sigma ,w\right)$ and ${\theta }_{R|Y}=\left(\alpha ,\beta ,\kappa \right)$. Here, $0<\left(1-\kappa \right)<1$ corresponds to the fraction of data that is missing completely at random, and $\alpha$ and $\beta$ describe the odds of a missing value, with $\beta$ parameterizing the rate at which the odds of a missing value change in $y_i$. Under this model, the implied missing-data distribution is

$$\begin{array}{ll}\hfill & f^{\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}}\left(y_i|r_i=0,\beta ,\kappa ,w_k,{\mu }_k,{\sigma }_k\right)\hfill \\ \hfill & \hspace{1em}=\left(1-{\kappa }^{*}\right)f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i|r_i=1,{\mu }_k,{\sigma }_k\right)\hfill \\ \hfill & +{\kappa }^{*}\left(\sum _{k=1}^K{w}_k^{*}N\left(y_i;{\mu }_k+\beta {\sigma }_k^2,{\sigma }_k^2\right)\right).\hfill \end{array}$$ [15]

The full derivation of the mixture weights $w_k^{*}$ and ${\kappa }^{*}$ (functions of $w_k$ and $\kappa$) is given in SI Appendix.

To complete the specifications for the analysis, we propose prior distributions for the parameters $Q$, $\beta$ and $\kappa$. As in the normal-logistic example in Estimation and Inference, rather than specifying a prior distribution for $\alpha$, we specify a prior distribution on $Q$, which is well identified. Under this specification, $\alpha$ is a deterministic function of $Q,\beta$ and $\kappa$. Computing the value of $\alpha$ is unnecessary as it does not appear in the missing-data density. In this application, our prior specification is

$$\begin{array}{ll}\hfill \beta & \sim \mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(\mathrm{1,3}\right)\hfill \\ \hfill Q& \sim \mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\left(\mathrm{0,1}\right)\hfill \\ \hfill \kappa \mid Q& \sim 1-\left(1-Q\right)\mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(\mathrm{1,2}\right),\hspace{1em}\kappa \ge Q.\hfill \end{array}$$ [16]

We chose a uniform prior on $Q$, the population fraction of observed data. The results are not sensitive to this choice because the population fraction of observed data is well identified. On the contrary, $\kappa$ and $\beta$ are not estimable from data, and thus the results are very sensitive to the prior specifications. Ideally, these specifications should incorporate as much expertize and knowledge about the measurement technology or observation mechanism as possible. In the application we consider, Karpievitch et al. (37) and other authors reported that Missing Completely at Random (MCAR) censoring is expected to affect a relatively small proportion of the proteins (e.g., $<20\%$). The parameter $\kappa$ captures the fraction of missing data that is abundance-specific, not MCAR. Under our prior specification, the fraction of the missing data that is MCAR follows a β(1, 2) distribution. This prior has high variance (reflecting our uncertainty) but implies on average that one-third of the missing data is MCAR. Note that $\kappa$ must be greater than $Q$ because the selection probabilities cannot be less than the population fraction of observed data.

The prior on $\beta$ specifies beliefs about the rate at which log odds of missingness change. This depends on the measurement technology and experimental design and thus is expected to be quite variable across datasets. From Eq. 15, we see that the mean of each missing-data component corresponds to a location shift of $\beta {\sigma }_k^2$ of the observed-data mean. This intuition can help us specify plausible priors on $\beta$. If we set $\beta =0$, then all missingness is MCAR. Alternatively, one may want to calibrate the prior for $\beta$ using ancillary data about the sensitivity of the measurement technology.

Draws from the full prior distribution are shown in Fig. 1, Top Left in gray, around the prior mean in black.

Fig. 1.

Model fit to proteomic abundance data (log molecules per cell [log mol./cell]) from ref. 31 data using two approaches: Tukey’s representation (Left) and the selection factorization (Right). The gray lines in Top Left represent draws of the selection mechanism from the prior distribution provided in Eq. 16. The black, red, and blue lines in Bottom Left and Bottom Right correspond to the estimated densities of complete data, missing data, and observed data, respectively.

Data Analysis.

Fig. 1, Bottom Left shows the fit to the protein measurements (31), when $\beta$ is set to its median posterior value. For comparison, Fig. 1, Bottom Right shows the fit of the selection-factorization model in refs. 24 and 29, which assumes the complete data are distributed according to a lognormal and the missingness mechanism is logistic, with the mean linear in $y_i$. The black, red, and blue lines, in both Fig. 1, Bottom Left and Bottom Right, correspond to the estimated densities of complete data, missing data, and observed data, respectively. Note that under the selection model, the estimated observed-data density (blue) is a poor fit to empirical distribution of the observed data, especially near the mode. The corresponding results for the transcript measurements (30) are provided in SI Appendix.

In Fig. 2, we compare the estimated complete-data means and SDs for the protein measurements using Tukey’s representation model implied by Eqs. 13 and 16 to the estimates from the selection-factorization model in ref. 24. In the selection-model parameterization, not all parameters can be estimated from data (38). We found empirically that the likelihood of the selection factorization has two modes: one where $\kappa =1$ (corresponding to the usual selection model with logistic asymptote of 1) and one where $\kappa =\frac{N_{\text{obs}}}{N}$ (corresponding to a fully MCAR model). For both sets of measurements, the estimates obtained with the MCAR model and with the selection-factorization model bracket the estimates obtained with Tukey’s representation. Under the selection-factorization model, the complete-data SD is large and the mean is small relative to the estimates from Tukey’s representation. The results suggest that the parametric assumptions associated with the selection-factorization models overly constrain the fit to the observed data.

Fig. 2.

Posterior distributions of the complete-data mean (Left) and complete-data SD (Right) for protein data (31). The MCAR estimates (red) and an estimate assuming normality of the complete data (blue) are shown as vertical lines for comparison. Under the prior distribution in Eq. 16, estimates using the MCAR and the selection-factorization models are at opposite ends of these posterior distributions.

Table 1 reports exact numerical estimates of the two estimands of interest.

Table 1.

Estimates for the quantities of interest obtained with three models, from protein and mRNA data

Estimand	Tukey’s Rep.	Selection	MCAR	Dataset
Mean	7.42 (7.18, 7.73)	6.84	7.82	Prot.
SD	1.66 (1.52, 1.94)	2.01	1.55	Prot.
Mean	0.51 (0.44, 0.59)	0.35	0.60	mRNA
SD	1.13 (1.07, 1.23)	1.23	1.08	mRNA
Ratios	1.48 (1.28, 1.73)	1.62	1.43	Both

The dynamic range ratios are computed using both datasets. We report maximum-likelihood point estimates for both the MCAR and selection models. We report posterior medians and 95% posterior intervals (in parentheses) for Tukey’s representation. Units are in log molecules per cell. Tukey’s Rep., Tukey’s representation.

Recent published analyses of data using the selection factorization found that translational regulation widens the dynamic range of protein expression (24, 29). One way to quantify the relative dynamic ranges of mRNA and protein is by computing the ratio of the SDs between log-mRNA and log-protein levels. A value of this ratio less than 1 suggests that the dynamic range of protein levels is smaller than that of mRNA and is taken as evidence of a suppressive role of translational regulation. A value greater than 1 is taken as evidence of amplification.

We used posterior estimates of the complete-data SDs, obtained from the three competing models fit to both protein and mRNA measurements (30, 31) to estimate the distribution of the dynamic range ratios, displayed in Fig. 3. The results obtained with Tukey’s representation are consistent with those reported by ref. 29, suggesting that translational regulation reflects amplification of protein levels.

Fig. 3.

Posterior distribution of dynamic range ratios obtained using Tukey’s representation (histogram), the maximum-likelihood estimation dynamic range on the normal selection-factorization model (blue) (29) and the MCAR model (red).

This brief case study demonstrates the relative ease of applied data analysis with Tukey’s representation models and the increased flexibility of models specified using this full conditional specification. By directly modeling the observed data, we avoid the need for Monte Carlo integration of the missing data and do not require parametric specifications for the complete-data density as is typical for selection models. By modeling the selection function directly, we are also able to express uncertainty about the missing-data density beyond the simple location and scale changes typical in pattern-mixture model sensitivity analyses.

Discussion

Tukey’s representation provides a powerful alternative for specifying missing-data models. It allows analysts to eschew some difficult questions about identifiability in models for nonignorable missing data (38) by factoring the joint distribution of the complete data, $Y$, and missing-data indicators, $R$, in such a way that the missingness mechanism is the only component that must rely on assumptions unassessable using observed data.

Theory.

Thus far, we largely worked with exponential-family models. Here, we make formal statements about exponential-family models, as well as give results that hold more generally.

Theorem 1.

The normalization constant $Q\left({\theta }_{Y\mid R}^{\mathrm{o}\mathrm{b}\mathrm{s}},{\theta }_{R\mid Y}\right)$, given in Eq. 5, is equal to the population fraction of observed data:

$$\begin{array}{ll}\hfill & {\left(1+\int \frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\hspace{0.17em}d{y}_i\right)}^{-1}\hfill \\ \hfill & \hspace{1em}=\hspace{0.17em}\mathbb{E}\left[r_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}},{\theta }_{R|Y}\right].\hfill \end{array}$$

Proof (SI Appendix):

A consequence of Theorem 1 is that the missing-data density can be expressed as a function of the observed-data density.

Theorem 2.

If the positivity condition is satisfied, i.e., $f^{\mathrm{m}\mathrm{i}\mathrm{s}}$ is absolutely continuous with respect to $f^{\mathrm{o}\mathrm{b}\mathrm{s}}$, then $f^{\mathrm{m}\mathrm{i}\mathrm{s}}$ can be expressed as a function of the observed-data density and the selection function

$$\begin{array}{ll}\hfill & f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y_i\mid {\theta }_{R|Y},{\theta }_{Y|R}\right)\hfill \\ \hfill & \hspace{1em}=\hspace{0.17em}\frac{Q\left({\theta }_{Y|R},{\theta }_{R|Y}\right)}{1-Q\left({\theta }_{Y|R},{\theta }_{R|Y}\right)}\frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}\hspace{0.17em}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}\right).\hfill \end{array}$$ [17]

Proof (SI Appendix):

This result is a consequence of setting $r_i=0$ in the complete-data likelihood (Eq. 4). Eq. 17 can help assess the plausibility of various missingness mechanisms—not at random, completely at random, and at random (5)—by viewing them as functions of the odds of a missing value versus an observed value, $\frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}$. For instance, when the odds have low variance, it may be more reasonable to assume the missing-data mechanism is completely at random, or at random.

Eq. 17 also leads to a general understanding of the main result regarding exponential families, which can be summarized in the following statement.

Theorem 3.

Assume the observed-data distribution, $f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$, belongs to an exponential family, with natural parameter ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\eta$ and natural sufficient statistic $T\left(y\right)$, and that the selection function, $f\left(r=1\mid y,{\theta }_{R|Y}\right)={\text{logit}}^{-1}\left(\alpha +T{\left(y\right)}^{\prime }\beta \right)$ with ${\theta }_{R|Y}=\left(\alpha ,\beta \right)$. Then the implied missing-data distribution, $f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y\mid {\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}}\right)$, is in the same exponential family as the observed-data distribution, with natural parameter ${\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}}=\eta +\beta$. When $T\left(y\right)$ is P-dimensional, $\eta$ and $\beta$ are also P-dimensional vectors. The normalization constant of the complete-data distribution has the form

$$\begin{array}{ll}\hfill & {\left(1+\int \frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\hspace{0.17em}d{y}_i\right)}^{-1}\hfill \\ \hfill & \hspace{1em}=\hspace{0.17em}\frac{g\left(\eta +\beta \right)}{g\left(\eta +\beta \right)+e^{\alpha }g\left(\eta \right)},\hfill \end{array}$$ [18]

where $g\left(\eta \right)$ is the expression for the normalization constant in the exponential family.

Proof (SI Appendix):

For Corollary 1, assume that the observed-data distribution is a $K$-component mixture of distributions in a common exponential family, with natural parameters ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\left({\eta }^{\left(1\right)},{\eta }^{\left(2\right)},\dots ,{\eta }^{\left(K\right)}\right)$, natural sufficient statistic $T\left(y\right)$, and mixture weights $w_k$,

$$f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)=\sum _{k=1}^K{w}_k{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y\mid {\eta }^{\left(k\right)}\right),$$

and that the selection function is logistic in T(y), $f\left(r=1\mid y,{\theta }_{R|Y}\right)={\text{logit}}^{-1}\left(\alpha +T{\left(y\right)}^{\prime }\beta \right)$. Then the implied missing-data distribution, $f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y|{\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}}\right)$, is a $K$-component mixture of distributions in the same exponential family as the observed-data components, with natural parameters, ${\theta }_{Y|R}^{\mathrm{m}\mathrm{i}\mathrm{s}}=\left({\eta }^{\left(1\right)}+\beta ,{\eta }^{\left(2\right)}+\beta ,\dots ,{\eta }^{\left(K\right)}+\beta \right)$, and weights

$$w_k^{*}=w_k\frac{g_k\left({\eta }^{\left(k\right)}\right)}{g_k\left({\eta }^{\left(k\right)}+\beta \right)}(\sum _{k=1}^K{w}_k\frac{g_k\left({\eta }^{\left(k\right)}\right)}{g_k\left({\eta }^{\left(k\right)}+\beta \right)},$$

where $g\left(\eta \right)$ is the expression for the normalization constant in the common exponential family.

Tukey’s representation can be extended to model incomplete data accounting for observed covariates, for instance, by simply conditioning on $x$,

$$\begin{array}{ll}\hfill & f\left(y_i,r_i\mid x_i,\theta \right)\hfill \\ \hfill & \hspace{1em}\propto \hspace{0.17em}\prod _{i=1}^N\left[f\left(y_i\mid r_i=1,x_i,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\cdot \frac{f\left(r_i\mid y_i,x_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,x_i,{\theta }_{R|Y}\right)}\right].\hfill \end{array}$$ [19]

The factor $f\left(y_i\mid r_i=1,x_i,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$ is estimable from the observed values. A potential challenge in applications that include covariates is the need to specify the selection probabilities, $f\left(r_i\mid y_i,x,{\theta }_{R|Y}\right)$, for all values of $x$.

We can also apply Tukey’s factorization with multivariate data in situations where we have a monotone missing-data mechanism (39). A missing-data pattern is called “monotone” if ${\overrightarrow{y}}_i$ is a $K$-dimensional multivariate random variable that can be ordered such that if $y_i^j$ is missing, then all variables $y_i^k$, $k>j$ are also missing. In this case, the complete-data distribution can be written using Tukey’s representation as

$$\begin{array}{ll}\hfill & f\left(\overrightarrow{y},\overrightarrow{r}\mid \theta \right)\hfill \\ \hfill & \hspace{1em}\propto \hspace{0.17em}\prod _{i=1}^N\prod _{k=1}^K\left[f\left(y_i^k\mid r_i^k=1,r_i^{k-1}=\dots =r_i^1=1,y_i^{k-1},\dots ,\right.\right.\hfill \\ \hfill & \left.y_i^1,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)\hfill \\ \hfill & \left.\times \frac{f\left(r_i^k\mid y_i^k,r_i^1=\dots =r_i^{k-1}=1,y_i^1,\dots ,y_i^{k-1},{\theta }_{R|Y}\right)}{f\left(r_i^k=1\mid y_i^k,r_i^1=\dots =r_i^{k-1}=1,y_i^1,\dots ,y_i^{k-1},{\theta }_{R|Y}\right)}\right],\hfill \end{array}$$ [20]

where, as in the univariate setting, $f\left(y_i^k\mid r_i^k=1,r_i^{k-1}=\dots =r_i^1=1,y_i^{k-1},\dots ,y_i^1,{\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$ is observed but $f\left(r_i^k=1\mid y_i^k,r_i^1=1,\dots ,r_i^{k-1}=1,y_i^1,\dots ,y_i^{k-1},{\theta }_{R|Y}\right)$ is not.

Connections to Exponential Tilting.

Tukey’s representation has an interesting connection to exponential-tilting methods (13–15). Both approaches model the missing-data distribution by modifying the observed-data distribution by a multiplicative factor. However, the strategies to obtain such factors, and their interpretations, differ.

In exponential tilting, we write $f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y\right)$ directly, using a tilting function $q\left(y\right)$, as

$$f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y\right)=\frac{e^{-q\left(y\right)}}{\int e^{-q\left(y\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y\right)dy}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y\right),$$

whereas Tukey’s representation induces $f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y\right)$, indirectly, as

$$f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y\right)=\frac{Q}{1-Q}\frac{1-\pi \left(y\right)}{\pi \left(y\right)}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y\right).$$

The multiplication factor in the exponential-tilting formulation is a direct consequence of the choice of $q\left(y\right)$. The multiplication factor in Tukey’s representation is a function of the fraction of observed data $Q$ and of the odds of missingness.

The relation between exponential tilting and Tukey’s representation is similar in spirit to the relation between a frequentist penalized likelihood and a posterior distribution, in which the penalty is implied by the choice of a prior distribution. Tukey’s representation offers a principled way to derive specific forms of $q\left(y\right)$ from assumptions made on $f^{\mathrm{m}\mathrm{i}\mathrm{s}}$.

Importantly, however, exponential tilting requires specifying the function $q\left(y\right)$ somewhat arbitrarily, and because of that, the data analysis may often require a serious sensitivity analysis, for the results to be defensible. In contrast, Tukey’s representation requires the specification of a fully generative model, the pieces of which are arguably easier to defend in scientific applications.

To explore this equivalence in more detail, we write $f^{\mathrm{m}\mathrm{i}\mathrm{s}}$ in exponential-tilting form by exponentiating the log of the multiplicative factor from Theorem 2,

$$\begin{array}{ll}\hfill & f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y_i\mid {\theta }_{R|Y},{\theta }_{Y|R}\right)\hfill \\ \hfill & \hspace{1em}=\hspace{0.17em}\exp \left\{\log \left(\frac{Q\left({\theta }_{Y|R},{\theta }_{R|Y}\right)}{1-Q\left({\theta }_{Y|R},{\theta }_{R|Y}\right)}\frac{f\left(r_i=0\mid y_i,{\theta }_{R|Y}\right)}{f\left(r_i=1\mid y_i,{\theta }_{R|Y}\right)}\right)\right\}\hspace{0.17em}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\hfill \\ \hfill & \times \left(y_i\mid {\theta }_{Y|R}\right).\hfill \end{array}$$

The right hand side of this equation can be seen as $\exp \left\{-q\left(y_i,{\theta }_{tilt}\right)\right\}\hspace{0.17em}{f}^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}\right)$, where $q\left(y,{\theta }_{tilt}\right)$ is a function-valued sensitivity parameter specified in exponential-tilting models. In Tukey’s representation, when using the logistic selection function, the exponential factor has interpretable components: the log odds of missingness and the log odds of selection. More generally, the parametric form for $q$ is often a complicated function of the selection parameters, even when the equivalent selection function in Tukey’s representation is easily interpretable.

Tukey’s representation opens the door to more transparent analyses in problems that involve missing data. For example, in the model in Illustration on Transcriptomic and Proteomic Data, we can derive the implied tilting function (SI Appendix) given the missingness mechanism $f\left(r_i\mid y_i,{\theta }_{R|Y}\right)$ and the normalization constant $Q\left({\theta }_{Y|R},{\theta }_{R|Y}\right)$. Here, we have the following exponential-tilt function:

$$\begin{array}{ll}\hfill -q\left(y,{\theta }_{tilt}\right)& =\log \left(e^{-\alpha }\left(\sum _k^K\frac{w_k}{\kappa }\frac{g\left(\eta \right)}{g\left({\eta }^{*}\right)}\right)+\frac{1-\kappa }{\kappa }\right)\hfill \\ \hfill & \hspace{1em}+\log \left(\frac{1}{\kappa }{e}^{-\beta y-\alpha }+\frac{1-\kappa }{\kappa }\right),\hfill \end{array}$$ [21]

where ${\eta }^{*}=\eta +\beta$. The complex nature of this function highlights the importance of prior specification for the selection function, as opposed to the tilting function. As another example, in this paper, we focus on cases where we specify a prior distribution for $\beta$, the rate at which the odds of selection change in $T\left(y_i\right)=y_i$. We show that for fixed $\beta$, $\alpha$ is identified in the two-parameter logistic model. However, in some applications, we may have more prior knowledge about $\alpha$, the log odds of missingness given $y_i=0$. In this case, if we specify a prior distribution on $\alpha$, then $\beta$ is identified. This further illustrates how, in many contexts, it is easier to elicit priors and justify parameterizations for the fully generative specification of the selection function.

Even in light of specific equivalences for specific choices of the various factors, working with the $q\left(y\right)$ function implied by Tukey’s generative approach may be more easily defensible, say, in medical, biological, economics, or legal contexts.

A Note on the Integrability Condition.

Not all integrable specifications for $f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}\right)$ and $f\left(r_i\mid y_i,{\theta }_{R|Y}\right)$ imply a proper distribution for $f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y_i\mid {\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}},{\theta }_{R|Y}\right)$. The integrability condition requires the sum ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}+{\theta }_{R|Y}$ to lie in the natural parameter space of the exponential family. In practice, analysts may want to consider missing-data mechanisms that involve a richer set of parameters, ${\stackrel{\sim }{\theta }}_{R|Y}$, such as including an intercept, as demonstrated in Illustration on Transcriptomic and Proteomic Data. In such cases, ${\theta }_{R|Y}$ is taken to denote the subset of parameters in ${\stackrel{\sim }{\theta }}_{R|Y}$ that multiply the sufficient statistics of $f^{\mathrm{o}\mathrm{b}\mathrm{s}}$. The derivations in Modeling and Inference Using Tukey’s Representation can easily be extended to this situation, accordingly.

For example, assume that the natural parameter ${\theta }_{Y|R}^{\mathrm{o}\mathrm{b}\mathrm{s}}=\eta$, and that the missing-data mechanism is logistic with extended parameter vector ${\stackrel{\sim }{\theta }}_{R|Y}=\left(\alpha ,\beta \right)=\left(\alpha ,{\theta }_{R|Y}\right)$ and $f\left(r_i=1\mid y_i,\alpha ,\beta \right)={\left(1+e^{-\left(\alpha +T{\left(y_i\right)}^{\prime }\beta \right)}\right)}^{-1}$. Then, Eqs. 9 and 10 become

$$Q\left(\eta ,\beta \right)=\frac{g\left(\eta +\beta \right)}{g\left(\eta +\beta \right)+g\left(\eta \right)e^{\alpha }},$$ [22]

$$f^{\mathrm{m}\mathrm{i}\mathrm{s}}\left(y\mid \eta ,\beta \right)=h\left(y\right)g\left(\eta +\beta \right)e^{{T\left(y\right)}^{\prime }\left(\eta +\beta \right)}.$$ [23]

The class of EF-logistic models defined in Eqs. 6 and 7 can be further generalized in two useful ways, while maintaining its desirable properties. For instance, generalizing $f^{\mathrm{o}\mathrm{b}\mathrm{s}}$ to be a mixture of exponential families is straightforward (Corollary 1) and does not increase computation substantially. Relaxing assumptions about the missingness mechanism can be more difficult. Still, it is possible to model $f\left(r_i\mid y_i,{\theta }_{R|Y}\right)$ with a mixture of logistic functions, including a missingness mechanism where a fraction of the data is missing completely at random (as is shown in the applied example).

Inferential Strategies.

Recall the simple normal-logistic model,

$$\begin{array}{ll}\hfill f\left(r_i=1\mid y_i,\alpha ,\beta \right)& ={\text{logit}}^{-1}\left(\alpha +\beta y_i\right)\hfill \\ \hfill f^{\mathrm{o}\mathrm{b}\mathrm{s}}\left(y_i\right)& =\text{Normal}\left(\mathrm{0,1}\right).\hfill \end{array}$$

The inferential strategy proposed was to posit prior distributions on $\beta$ and $Q$. Each iteration of the MCMC sampler yields a sample for $\beta$ and $Q$ and implied sample for $\alpha$. A conceptually simpler approach to inference would be to place a prior distribution on all of the parameters of the missingness mechanism (i.e., $\alpha$ and $beta$) and solve for the implied $Q$ at each iteration of the sampler.

In situations where the number of missing values is itself missing, as with truncated data, specifying a prior distribution for all of the parameters of the missingness mechanism would lead to an implied prior distribution for the unknown number of missing values, or equivalently, the population fraction of observed data Q; this approach was also used in the original expectation-maximization algorithm paper (40).

In situations where the number of missing values is known, however, as with censored data, and therefore $Q$ can be estimated from observed data, the support of the likelihood is a constrained parameter space, and a number of choices for the prior distribution on $\beta$ would lead to a posterior distribution that is challenging to explore using Monte Carlo methods. Specifically, because the population fraction of observed data are identifiable, Theorem 1 describes a moment constraint that restricts the region, where the parameters of the missingness mechanism have positive support, to a lower dimensional ridge. Fig. 4 illustrates this phenomenon for the simple normal-logistic model and increasing sample size.

Fig. 4.

The region of positive support for the likelihood, restricted to the parameters of the missingness mechanism, is increasingly constrained as the population fraction of observed data, $Q$, is estimated with increasingly high precision. This intuition is illustrated by the width of the ridge, which is a function of the amount of information about $Q$. We simulated data from a standard normal distribution and a logistic missingness mechanism. The parameters $\left(\alpha ,\beta \right)$ were set to get 90% missing data. The sample size determines the amount of information: $N=100$ (Left) and N = 1,000 (Right).

Sequential Monte Carlo and other specialized Monte Carlo methods that exploit the geometry of the support of posterior distribution may provide solutions in this situation (41–44).

Concluding Remarks

In this paper, we used EF-logistic models to illustrate how Tukey’s representation can be used to encode nonmonotonicity in the missingness mechanism and to model data with complex distributional forms. The EF-logistic models are widely applicable as they can be applied to data that are well approximated by mixtures of exponential families. Although not explored here, similar logic can be applied to facilitate inference for models with nonlogistic selection mechanisms that can be well approximated by mixtures of logistic functions. These EF-logistic models could also be used to facilitate tipping-point analyses (45) or to incorporate subjective model uncertainty via prior distributions on the missingness mechanism (2).

Tukey’s representation is most useful when it is feasible to posit reasonable prior distributions on the selection mechanism. Translating expert knowledge into a functional form can be challenging, in general, and a logistic missingness mechanism is not always a good choice. In practice, Tukey’s representation should be used in concert with strategies for expert prior elicitation (46–48). Nevertheless, prior elicitation for Tukey’s representation is simpler than for other factorizations, because it involves only the set of parameters ${\theta }_{R|Y}$. In contrast, the selection factorization requires additional assumptions about the complete-data density.

In many settings, like the example presented in our applied analysis, we may be able to collect data that partially inform the specification for the selection mechanism. As such, when possible, we can design experiments to learn about the functional form of $f\left(r_i|y_i,{\theta }_{R|Y}\right)$ as well as to further refine prior distributions for ${\theta }_{R|Y}$. Along these lines, Tukey’s representation may be useful in the context of multiphase inference, which is intimately related to problems in missing data (49). In these problems, when preprocessing data, we often have strong knowledge (or control) of the missingness mechanism yet a weaker understanding of the underlying scientific model.

Finally, in this paper, we focus on the class of problems where the data are univariate and i.i.d. Extending this methodology to a broader class of multivariate missing-data problems is challenging. We show that Tukey’s representation is easily extensible to montone missing data, where the observed-data models can easily be replaced by conditional models. For more general missing-data patterns, Tukey’s representation is nontrivial. However, we believe that Tukey’s representation can be a particularly useful tool for specifying joint multivariate distributions using only the full conditionals. Empirical work in this area has shown that imputation using a Gibbs sampler can be effective, even though the specified conditional densities can be incompatible (e.g., do not imply a proper joint distribution) (50, 51). In these so-called partially incompatible Gibbs samplers, each Gibbs would involve missing-data imputation of a single variable given the rest, through Tukey’s representation. Such extensions are the subject of future research by us.

In conclusion, we argue that Tukey’s representation, which represents a hybrid of the selection and pattern-mixture models is an underresearched yet promising alternative for modeling nonignorable missing data.

Data Availability.

All raw input and processed output data are available in Dryad (DOI: 10.5061/dryad.rg367 and DOI: 10.5061/dryad.d644f).