Linear regression with left‐censored covariates and outcome using a pseudolikelihood approach

Abstract

Environmental toxicology studies often involve sample values that fall below a laboratory procedure’s limit of quantification. Such left-censored data give rise to several problems for regression analyses. First, both covariates and outcome may be left censored. Second, the transformed toxicant levels may not be normal but mixtures of normals because of differences in personal characteristics, e.g. exposure history and demographic factors. Third, the outcome and covariates may be linear functions of left-censored variates, such as averages and differences. Fourth, some toxicant levels may be functions of other toxicant levels resulting in a recursive system. In this paper marginal and pseudo-likelihood based methods are proposed for estimation of the means and covariance matrix of variates found in these four settings. Next, linear regression methods are developed allowing outcomes and covariates to be linear combinations of left-censored measures. This is extended to a recursive system of modeling equations. Bootstrap standard errors and confidence intervals are used. Simulation studies demonstrate the proposed methods are accurate for a wide range of study designs and left-censoring probabilities. The proposed methods are illustrated through the analysis of an on-going community-based study of polychlorinated biphenyls, which motivated the proposed methodology.

1 |. INTRODUCTION

A common problem in environmental exposure assessment is the occurrence of contaminant levels that fall below a laboratory’s ability for accurate measurement. Values below this limit of quantification (LOQ) are known as left-censored observations. This problem occurs when a laboratory assay’s LOQ is high relative to typical field exposure, or when the sampling period is short (Hewett, 2006, p. 415). Thousands of chemicals are in use today, with more introduced every year, and laboratory assays of their levels are frequently not quantifiable at the lower levels. This is an important public health and policy issue since values below the LOQ may be toxic or bioaccumulate to toxic levels. Quantification limits are common with air, water, soil/sediment, food, blood, urine and tissue samples. A few examples include herbicides/pesticides/nitrates in the water supply (Holden et al., 1992; Jones et al. 2016); toxic metals (e.g. lead, chromium) in highway runoff (Shumway et al., 2002) and public water supply lines, as in Flint, MI; okadaic acid, arsenic and cadmium in food (EFSA, 2010; The Economist, 2017); endotoxin levels of environments with organic dust (Thorne et al., 2010); and polychlorinated biphenyls (PCBs) in air and blood samples (Korwel et al., 2005; Marek et al., 2013). The motivation for the proposed methods of this paper derives from an on-going community-based study assessing blood PCB levels and the need to easily handle left-censoring of the predictors as well as the outcome variable in linear regression models.

The distribution of environmental contaminants is typically right skewed and assumed to follow a lognormal distribution for which industrial hygienists and environmental toxicologists historically have estimated summary statistics for a single variate. Although maximum likelihood estimation (MLE) was introduced early by Hald (1949) to handle left censoring due to measurements below the LOQ, the methods were computationally laborious and required special tables. This led to single imputation of left-censored values by the LOQ, LOQ/2 (Nehls & Akland, 1973) and $\text{LOQ}/\sqrt{2}$ (Hornung & Reed, 1990). Many researchers (e.g. Hewett & Ganser, 2007) have studied these imputation methods via simulation, showing that MLE is far more accurate. For regression problems historical development focused first on the problem in which only the outcome variate is censored. Maximum likelihood regression methods for censored outcomes with completely observed covariates have a long successful history in both the statistical (Kalbfleisch & Prentice, 2002) and econometric literature (Tobin, 1958), and yield consistent, asymptotically normal estimates. The problem of longitudinal or spatial outcomes subject to left censoring is more complex but has been tackled by full likelihood (Gadda et al., 2000), Monte-Carlo EM (Hughes, 1999) and Bayesian methods (Fridley & Dixon, 2006).

Recently, there has been increased interest in regression problems having usually one, sometimes more, censored covariates, but assuming a completely observed outcome. Several approaches have been proposed. A simple regression approach is the complete-case analysis that omits subjects with any censored variates. When only the covariates are subject to censoring, the complete-case analysis will produce unbiased parameter estimates so long as the regression error term remains uncorrelated with the observed covariates. The key concern is that efficiency decreases as the sample size decreases which can be substantial with heavy censoring. A second approach is single imputation, which includes the previously mentioned functions of LOQ and the expected value below the LOQ (Richardson & Ciampi, 2003). These tend to exhibit either bias in the parameter estimate, its standard error or both (Nie et al., 2010). A third and more sophisticated approach is multiple imputation. For the outcome Y modeled as a linear function of a right-censored Z and uncensored U, Atem et al. (2017) sampled from the predictive distribution of Z given the values of U and Y, substituting in parameter estimates derived from a complete-case regression of Y on (Z, U) and a Cox proportional hazards regression model fit to Z given U. Their approach is appropriate for right-censored Z, but less so for our left-censored setting, in part because this family does not contain the lognormal and, as they noted, due to their extrapolations beyond the range of uncensored values, especially problematic for heavily censored variates. The fourth approach is the EM algorithm. May et al. (2011) introduced approximate ML estimation based on a weighted Monte-Carlo EM algorithm in which the conditional Gaussian distribution of censored variate(s) given the values of uncensored values, including the outcome, were repeatedly sampled using the Gibbs sampler and the adaptive rejection metropolis algorithm. Unlike other authors, May et al. (2011) allowed more than one covariate to be censored.

On another note, there has also been recent interest in estimation of the covariance matrix for a large number of left-censored variates. In 2015, our paper (Jones et al.) and two others (Hoffman & Johnson, 2015; Pesonen et al., 2015) tackled this problem. Although the three methods differed somewhat in their approaches, each came to the conclusion that a pairwise likelihood approach to estimating the correlations was the key to finding a computationally efficient method for finding a consistent set of estimators. For the most part, all assumed multivariate normality of variates. However, this assumption is not realistic enough for many real world applications since the variates may only be normal within homogeneous strata of the population defined by both discrete and continuous variables (e.g. gender and age), thereby resulting in a complex mixture of normals.

An important but so far ignored area of research concerns environmental problems in which both the outcome variate and covariates are subject to left censoring, or more generally, are linear functions of censored variates, such as a difference or an average. This is the goal of this paper. In Section 2, we introduce methods for estimating the covariance matrix of left-censored variates that can be modeled in terms of a set of linear equations based on uncensored variates and normal error terms. This is then generalized to covariance matrices of linear combinations of left-censored variates. These estimators are used as building blocks in Section 3 to create multiple linear regression methods in which both the outcome and covariates can be linear functions of left-censored variates. This is further generalized to incorporate a recursive system of regression equations to accomodate censored covariates that are generated from other censored covariates. Section 4 studies in detail three designs one might find in environmental epidemiology, the last of which extends the framework of the recursive system of regressions. This section also reports on simulation studies of the performance of these methods. Section 5 provides an in-depth application of the proposed methods to the PCB study data that motivated our methodology. Section 6 concludes with some overall discussion.

2 |. ESTIMATION OF THE COVARIANCE MATRIX

Section 2.1 begins by postulating that the censored variates can be written as a set of linear functions of the uncensored covariates plus normal possibly correlated error terms. We propose an extension to the maximum pairwise pseudo-likelihood technique of Jones et al. (2015) for estimating the covariance matrix. Section 2.2 considers the special case of no uncensored covariates and provides a warning. Section 2.3 describes a straightforward approach to estimation of the covariance matrix for variates that are linear combinations of the censored and uncensored variates.

2.1 |. Estimation for uncensored and left-censored variates

Let ${\mathbf{D}}_i^{°}={\left(U_{i1},\dots ,U_{iq},Z_{i1}^{°},\dots ,Z_{ip}^{°}\right)}^{\prime }$ be the i-th unit’s variates of which the Z°’s will be subject to left censoring, whereas the U’s are uncensored and so measured exactly. The special case q = 0 is allowed and will be discussed later. Let ${\mathbf{L}}_i={\left(L_{i1},\dots ,L_{ip}\right)}^{\prime }$ be the laboratory lower limits of quantification for measuring the $Z_i^{°}$ variates. (The lower limit may depend on the i-th unit’s sample volume or other characteristics and hence is subscripted by i.) There are three basic assumptions. (A1) The vector of Z° variates are conditionally multivariate normal given the U variates. It will be convenient for later use to write this out explicitly as

$$\begin{array}{c}{Z}_{i1}^{°}={\varphi }_{10}+\\ ⋮\\ Z_{ip}^{°}={\varphi }_{p0}+\end{array}\begin{array}{l}{\varphi }_{11}{U}_{i1}+.\; .\; .+\hfill \\ ⋮\hfill \\ {\varphi }_{p1}{U}_{i1}+.\; .\; .+\hfill \end{array}\begin{array}{c}{\varphi }_{1q}{U}_{iq}+{\epsilon }_{i1}=\\ ⋮\\ {\varphi }_{pq}{U}_{iq}+{\epsilon }_{ip}=\end{array}\begin{array}{c}{\varphi }_{10}+{\varphi }_1^{\prime }{\mathbf{U}}_i+{\epsilon }_{i1}\\ ⋮\\ {\varphi }_{p0}+{\varphi }_p^{\prime }{\mathbf{U}}_i+{\epsilon }_{ip}\end{array}$$ (1)

where ${\left({\epsilon }_{i1},\dots ,{\epsilon }_{ip}\right)}^{\prime }~N_p\left(\mathbf{0},{\Sigma }_{\epsilon }\right)$ for n independent units. Both p and q are fixed. As for regression models in general, one should avoid models with a large number of variates relative to the sample size. No distributional assumptions are made about the U variates. (A2) As usual in regression, we assume the U variates are uncorrelated with the ε error terms. (A3) $Z_{ij}^{°}$ is assumed conditionally independent of L_ij given U_i. If necessary, we assume that the Z^° variates have already been appropriately transformed so that the error structure is normal. For laboratory data this is often the log transform. Next, let $Z_{ij}=\max \left(Z_{ij}^{°},L_{ij}\right)\text{and}{\eta }_{ij}=1\left[Z_{ij}^{°}\ge L_{ij}\right]$ be the observed variates, so that Z_ij is uncensored if n_ij = 1, but is left censored if n_ij = 0. As an example using the PCB study, Z_ij is the log-transformed, possibly left-censored (below LOQ) measurement of PCB congener j in a blood sample from subject i.

The goal of this section is to estimate the marginal mean and marginal covariance matrix of ${\mathbf{D}}^{°}=\left(\mathbf{U},{\mathbf{Z}}^{°}\right)$ from the observed data $\mathbf{D}=\left\{\left({\mathbf{U}}_i,{\mathbf{Z}}_i,{\mathbf{\eta }}_i\right):i=1,\dots ,n\right\}$ where ${\mathbf{U}}_i={\left(U_{i1},\dots ,U_{iq}\right)}^{\prime },{\mathbf{Z}}_i={\left(Z_{i1},\dots ,Z_{ip}\right)}^{\prime }$ and ${\mathbf{\eta }}_i={\left({\eta }_{i1},\dots ,{\eta }_{ip}\right)}^{\prime }$. We propose a straight-forward two-stage approach using marginal likelihood and pairwise pseudo-likelihood, that is particularly appealing when p is reasonably large.

In Stage 1, the marginal means and variances of the Z°’s are estimated. We begin by considering the j^th marginal model from (1), namely, $Z_{ij}^{°}={\varphi }_{j0}+{\varphi }_j^{\prime }{\mathbf{U}}_i+{\epsilon }_{ij}(i=1,\dots ,n)$ with ${\epsilon }_{ij}~N\left(0,{\sigma }_{\epsilon j}^2\right)$ . The parameters $\left({\varphi }_{j0},{\varphi }_j,{\sigma }_{\epsilon j}^2\right)$ are estimated by maximizing the j-th likelihood given U_i

$${\mathcal{L}}_j\left({\varphi }_{j0},{\varphi }_j,{\sigma }_{\epsilon j}^2\right)=\prod _{i=1}^n{\left\{f\left(z_{ij}-{\varphi }_{j0}-{\varphi }_j^{\prime }{\mathbf{U}}_i;0,{\sigma }_{\epsilon j}^2\right)\right\}}^{{\eta }_{ij}}{\left\{F\left(z_{ij}-{\varphi }_{j0}-{\varphi }_j^{\prime }{\mathbf{U}}_i;0,{\sigma }_{\epsilon j}^2\right)\right\}}^{1-{\eta }_{ij}}$$

where f and F are the normal density and distribution function; $\left(0,{\sigma }_{\epsilon j}^2\right)$ refer to their mean and variance. The resultant maximum likelihood estimators $\left({\widehat{\varphi }}_{j0},{\widehat{\mathit{\varphi }}}_j,{\widehat{\sigma }}_{{\epsilon }_j}^2\right)$ are consistent. Maximization is repeated for each Z_j separately $(j=1,\dots ,p)$ and easily performed by existing software, e.g. the survreg function in R (R Development Core Team, 2017). The marginal mean $E{Z}_j^{°}=E_{\mathbf{U}}\left\{E\left(Z_j^{°}|\mathbf{U}\right)\right\}$ can therefore be estimated by

$$\widehat{E}{Z}_j^{°}={\widehat{\varphi }}_{j0}+{\widehat{\varphi }}_{j1}{\overline{U}}_{\cdot 1}+\cdots +{\widehat{\varphi }}_{jq}{\overline{U}}_{\cdot q}(j=1,\dots ,p)$$

where ${\overline{U}}_{\cdot \mathcal{l}}=n^{-\mathrm{l}}\sum _i{U}_{i\mathcal{l}}.$. We shall refer to this estimator as the maximum marginal likelihood estimator (MMLE) since it estimates a marginal mean and uses no information from the other Z variates. The marginal variance is

$$\text{Var}\left(Z_j^{°}\right)={\text{Var}}_{\mathbf{U}}\left\{E\left(Z_j^{°}|\mathbf{U}\right)\right\}+E_{\mathbf{U}}\left\{\text{Var}\left(Z_j^{°}|\mathbf{U}\right)\right\}=Va{r}_{\mathbf{U}}\left\{{\varphi }_j^{\prime }\mathbf{U}\right\}+\text{Var}\left({\epsilon }_j\right)={\varphi }_j^{\prime }{\Sigma }_{\mathbf{U}U}{\varphi }_j+{\sigma }_{{\epsilon }_j}^2$$

which can be estimated by

$$\widehat{\text{Var}}\left(Z_j^{°}\right)={\widehat{\varphi }}_j^{\prime }{\widehat{\Sigma }}_{\text{UU}}{\widehat{\varphi }}_j+{\widehat{\sigma }}_{{\epsilon }_j}^2,$$ (2)

where ${\widehat{\mathbf{\Sigma }}}_{\mathbf{\text{UU}}}$ is the usual sample covariance matrix of U. As typical in regression settings, marginal means and variances of the outcome depend on representative samples of subjects for covariate patterns, and one should be careful in extrapolating these results beyond the range of the data, such as age ranges not covered by the sample. Two generally useful facts are that the population linear regression coefficients and intercept can be written as ${\varphi }_j={\mathbf{\Sigma }}_{\mathbf{\text{UU}}}^{-1}{\mathbf{\Sigma }}_{\mathbf{\text{U}}{Z}_j^0}\text{and}{\varphi }_{j0}=E{Z}_j^{°}-{\varphi }_j^{\prime }E\mathbf{U},$ respectively. Hence, we can estimate the covariance $\text{Cov}\left(\mathbf{U},Z_j^{°}\right)$ by

$${\widehat{\Sigma }}_{\mathbf{U}{Z}_j^{°}}={\widehat{\Sigma }}_{\mathbf{U}U}{\widehat{\varphi }}_j.$$ (3)

In Stage 2, we propose a pairwise pseudo-likelihood approach for estimating the correlation and covariance matrices of Z°. From model (1) and the fact that any pair of error terms is bivariate normal, i.e. $\left({\epsilon }_j,{\epsilon }_m\right)~N_2\left(0,0,{\sigma }_{\epsilon j}^2,{\sigma }_{\epsilon m}^2,{\rho }_{\epsilon jm}\right)$, it follows that

$$\begin{gathered} \text{Cov}\left({\epsilon }_j,{\epsilon }_m\right)=\text{Cov}\{Z_j^{°}-E{Z}_j^{°}-{\Sigma }_{Z_j^{°}\text{U}}{\Sigma }_{\text{UU}}^{-1}(\mathbf{U}-E\mathbf{U}), \\ {Z}_m^{°}-E{Z}_m^{°}-{\Sigma }_{Z_m^{°}\mathbf{U}}{\Sigma }_{\mathbf{U}U}^{-1}(\mathbf{U}-E\mathbf{U})\} \\ =\text{Cov}\left(Z_j^{°},Z_m^{°}\right)-{\Sigma }_{Z_j^{°}\text{U}}{\Sigma }_{\text{UU}}^{-1}{\Sigma }_{\text{U}{Z}_m^{°}} \end{gathered}$$

and hence

$$\text{Cov}\left(Z_j^{°},Z_m^{°}\right)={\Sigma }_{Z_j^{°}\text{U}}{\Sigma }_{\text{UU}}^{-1}{\Sigma }_{\text{U}{Z}_m^{°}}+\text{Cov}\left({\epsilon }_j,{\epsilon }_m\right).$$ (4)

We only need to estimate $\text{Cov}\left({\epsilon }_j,{\epsilon }_m\right),j\ne m$, since the other covariance terms in (4) are estimated in Stage 1.

In Stage 2, the pairwise error correlations ${\rho }_{\epsilon jm}$ are estimated by generalizing the pairwise pseudo-likelihood approach proposed by Jones et al. (2015) in order to handle the setting described by (1). For each pair $\left({\epsilon }_j,{\epsilon }_m\right)$, we form the pair-specific pseudo-likelihood

$$PP{L}_{jm}\left({\rho }_{\epsilon jm}\right)={\mathcal{L}}_{jm}\left(0,0,{\widehat{\sigma }}_{\epsilon j}^2,{\widehat{\sigma }}_{\epsilon m}^2,{\rho }_{\epsilon jm}\right)$$ (5)

where $L_{jm}$ is the left-censored bivariate normal likelihood and $\left({\widehat{\sigma }}_{\epsilon j}^2,{\widehat{\sigma }}_{\epsilon m}^2\right)$ are the MMLEs of the variances found in Stage 1. This likelihood is based solely on $\{\left(r_{ij},{\eta }_{ij},r_{im},{\eta }_{im}\right):i=1,\dots ,n\}$ where

$$r_{ij}=Z_{ij}-\widehat{E}{Z}_{ij}^{°}-{\widehat{\Sigma }}_{Z_j^{°}\text{U}}{\widehat{\Sigma }}_{\text{UU}}^{-1}\left({\text{U}}_{ij}-{\overline{\text{U}}}_{\cdot j}\right).$$

Thus, r_ij is the estimated (uncensored) residual ${\widehat{\epsilon }}_{ij}$ if ${\eta }_{ij}=1$ and is left censored if ${\eta }_{ij}=0$. The pairwise normal likelihood ${\mathcal{L}}_{jm}\left(0,0,{\widehat{\sigma }}_{\epsilon j}^2,{\widehat{\sigma }}_{\epsilon m}^2,{\rho }_{\epsilon jm}\right)$ for left-censored data is

$$\prod _{i=1}^n{\left\{f_{jm}\left(r_{ij},r_{im}\right)\right\}}^{{\eta }_{ij}{\eta }_{im}}{\left\{(\partial /\partial a)F_{jm}{\left(a,r_{im}\right)|}_{a=r_{ij}}\right\}}^{{\eta }_{ij}\left(1-{\eta }_{im}\right)}\times {\{(\partial /\partial b)F_{jm}(r_{ij},b){|}_{b=r_{im}}\}}^{\left(1-{\eta }_{ij}\right){\eta }_{im}}{\{F_{jm}(r_{ij},r_{im})\}}^{\left(1-{\eta }_{ij}\right)\left(1-{\eta }_{im}\right)},$$

where $f_{\mathrm{jm}}\text{and}{F}_{jm}$ are the bivariate normal density and distribution function with means (0, 0), variances evaluated at $\left({\widehat{\sigma }}_{\epsilon j}^2,{\widehat{\sigma }}_{\epsilon m}^2\right)$ and free correlation parameter ${\rho }_{\epsilon jm}$. Maximization of $PP{L}_{jm}\left({\rho }_{\epsilon jm}\right)$ produces the maximum pairwise pseudo-likelihood estimator $\text{(MPPLE)}{\widehat{\rho }}_{\epsilon jm}$. This maximization is repeated for all $p(p-1)/2$ pairings to compute the $\text{MPPLE}\widehat{\mathbf{P}}$ of the error correlation matrix P. Computing formulae for the likelihood components can be found in Jones et al. (2015), along with other details on maximization.

The MPPLE of the covariance matrix ${\mathbf{\Sigma }}_{\epsilon }$ of error terms in (1) is defined as ${\widehat{\mathbf{\Sigma }}}_{\epsilon }={\widehat{\mathbf{V}}}^{1/2}\widehat{\mathbf{P}}{\widehat{\mathbf{v}}}^{1/2},$ where $\widehat{\mathbf{V}}$ is the Stage 1-derived diagonal matrix whose diagonal contains the $\text{MMLEs}\left({\widehat{\sigma }}_{\epsilon 1}^2,\dots ,{\widehat{\sigma }}_{\epsilon p}^2\right).$ Since the $\text{MMLEs}\left({\widehat{\sigma }}_{\epsilon j}^2,{\widehat{\sigma }}_{\epsilon m}^2\right)\text{are}\sqrt{n}$-consistent estimators, the maximizer ${\widehat{\rho }}_{\epsilon jm}$ of the pseudo-likelihood $PP{L}_{jm}\left({\rho }_{\epsilon jm}\right)$ is also consistent by the pseudo-likelihood theory of Gong and Samaniego (1981). By the continuous mapping theorem the $\text{MPPLE}{\widehat{\Sigma }}_{\epsilon }$ is therefore consistent. The $\text{MPPLE}{\widehat{\Sigma }}_{\text{Z}°\text{Z}°}$ of the covariance matrix of Z° uses consistent plug-in estimators for the population parameters in equation (4) calculated in Stage 1 and Stage 2 and hence is also consistent. At this point we have consistent estimators of ${\mu }_{\text{D}°}\text{and}{\Sigma }_{\text{D}°}.$

2.2 |. Case of no uncensored covariates

In the case of no U covariates (q = 0), the assumption in (1) implies, for all (j, m) pairings,

$$\left(Z_{1j}^{°},Z_{1m}^{°}\right),\dots ,\left(Z_{nj}^{°},Z_{nm}^{°}\right)~iid{N}_2\left({\mu }_j,{\mu }_m,{\sigma }_{\epsilon j}^2,{\sigma }_{\epsilon m}^2,{\rho }_{\epsilon jm}\right)$$ (6)

where ${\mu }_{\mathcal{l}}={\varphi }_{\mathcal{l}0}$ in (1). This is precisely the same setting explored by Jones et al. (2015). Stage 1, as described above, is used to estimate $\left({\mu }_j,{\mu }_m\right)\text{and}\left({\sigma }_{\epsilon j}^2,{\sigma }_{\epsilon m}^2\right),$ the marginal mean and variance of $\left(Z_j^{°},Z_m^{°}\right),$ and Stage 2, as described above, is used to estimate ${\rho }_{\epsilon jm},$ the correlation between $Z_j^{°}\text{and}{Z}_m^{°}.$ The MMLEs $\text{MMLEs}{\widehat{\mu }}_j,{\widehat{\mu }}_m,{\widehat{\sigma }}_{\epsilon j}^2,{\widehat{\sigma }}_{\epsilon j}^2\text{and MPPLE}{\widehat{\rho }}_{\epsilon jm}$ are consistent under the assumption of pairwise bivariate normality (6). This assumption may be reasonable in many settings, but should be seriously scrutinized in each problem. As a cautionary tale, consider two types of workers at a plant, those involved in the manufacturing process with high exposure levels to a toxicant and other workers with far less exposure. Blood toxicant levels, i.e. $\left(Z^{°}|\text{group}=g\right),$ may follow a normal distribution for each group but with different means and variances. Ignoring the existence of the binary covariate and combining the two groups into one results in a mixture of normals rather than in a normal, violating the assumption in (6). The correct approach involves model (1) with U covariates describing the exposure patterns.

2.3 |. Estimation for linear functions of left-censored variates

Under the assumptions of multivariate normal errors and no unmeasured confounding in model (1), we have successfully estimated the marginal means ${\widehat{\mathbf{\mu }}}_{\mathbf{\text{D}}°}$ and covariance matrix ${\widehat{\mathbf{\Sigma }}}_{\mathbf{\text{D}}°}$ of all variates as described in the previous section. We can achieve substantially greater generality by considering linear combinations of the uncensored and censored variates, for which we will estimate means as well as covariance and correlation matrices. This will extend the methodology beyond typical censored-data methods by allowing us, for example, to compute summary statistics for possibly weighted averages and differences of left-censored variates.

Again, ${\text{D}}^{°}={\left({\text{U}}^{\prime },{\mathbf{Z}}^0{}^{\prime }\right)}^{\prime }$ consists of the q-vector U, not subject to left censoring, and the p-vector Z°, which is the uncensored version of Z. Let G be a $(k+1)\times (q+p)$ matrix of constants. The mean and covariance matrix of the linear combination GD° are estimated by

$$\widehat{E}\left(\mathbf{G}{\mathbf{D}}^{°}\right)=\mathbf{G}{\widehat{\mu }}_{{\text{D}}^{°}},\widehat{\text{Cov}}\left(\mathbf{G}{\mathbf{D}}^{°}\right)=\mathbf{G}{\widehat{\Sigma }}_{{\text{D}}^{°}}{\mathbf{G}}^{\prime }$$ (7)

where ${\widehat{\mu }}_{\text{D}°}\text{and}{\widehat{\Sigma }}_{\text{D}°}$ are calculated in the two-stage process described earlier.

3 |. METHODS FOR LINEAR REGRESSION

Using the covariance matrix and mean vector from the previous section, Section 3.1 gives straightforward formulae for estimation of the regression coefficients and population multiple R². The nonparametric bootstrap is proposed for standard errors. Section 3.2 covers the typical setting in which all left-censored Z° variates are generated from the uncensored variates, as in (1), and then one of them is chosen to be the outcome Y° to be regressed on the rest. The regression framework (1) for censored variates may not seem reasonable when some censored covariates are generated from other censored variates. Section 3.3 addresses this problem via a recursive system of regression models which are then shown to reduce to the original equation (1) framework, and hence allow estimation and hypothesis testing.

3.1 |. Multiple Linear Regression

Our regression setup will allow both the outcome and covariates to be linear combinations of uncensored and left-censored covariates as defined in Section 2.3. Specifically, our goal is to estimate the regression coefficients of the multiple linear regression model

$$Y_i^{°}={\beta }_0+{\beta }_1{X}_{i1}^{°}+\cdots +{\beta }_k{X}_{ik}^{°}+e_i$$ (8)

where both $Y^{°}\text{and}\left(X_1^{°},\dots ,X_k^{°}\right)$ can be linear combinations of uncensored variates and variates that are subject to left censoring; $e_1,\cdots ,e_n$ have mean zero and constant variance ${\sigma }_e^2,$ and are independent of each other and of the covariates. Let G be a $(k+1)\times (q+p)$ matrix of constants, where G_Y is the first row and G_X is the remaining k rows of G. Construct G so that

$$Y^{°}={\text{G}}_Y{\mathbf{D}}^{°},{\mathbf{X}}^{°}={\mathbf{G}}_{\mathbf{X}}{\mathbf{D}}^{°}.$$

By the previous section, the means and covariances of the $Y°\text{and}\mathbf{X}°$ can be estimated by

$$\begin{array}{c}{\widehat{\mu }}_{Y^{°}}={\mathbf{G}}_Y{\widehat{\mu }}_{\text{D}°},{\widehat{\Sigma }}_{Y^{°}{Y}^{°}}={\mathbf{G}}_Y{\widehat{\Sigma }}_{\text{D}°}{\mathbf{G}}_Y^{\prime },\\ {\widehat{\mu }}_{{\text{X}}^{°}}={\text{G}}_{\text{X}}{\widehat{\mu }}_{{\text{D}}^{°}},{\widehat{\Sigma }}_{{\text{X}}^{°}{\text{X}}^{°}}={\text{G}}_{\text{X}}{\widehat{\Sigma }}_{{\text{D}}^{°}}{\text{G}}_{\text{X}}^{\prime },\\ {\widehat{\Sigma }}_{X^{°}{Y}^{°}}={\mathbf{G}}_X{\widehat{\Sigma }}_{\text{D}°}{\mathbf{G}}_Y^{\prime }.\end{array}$$

Minimization of the mean squared error $E{\left(Y^{°}-{\beta }_0-{\beta }^{\prime }{\mathbf{X}}^{°}\right)}^2$ for an arbitrary linear function leads to ${\Sigma }_{{\text{X}}^{°}{\text{X}}^{°}}\beta ={\Sigma }_{{\text{X}}^{°}{Y}^{°}}\text{and}{\beta }_0=E{Y}^{°}-{\beta }^{\prime }E{\mathbf{X}}^{°}.$ Our proposal is to use the substitution or plug-in principle, so that the MPPLEs of the regression coefficients are

$$\widehat{\beta }={\widehat{\mathbf{\Sigma }}}_{{\mathbf{\text{X}}}^{°}{\mathbf{\text{X}}}^{°}}^{-1}{\widehat{\mathbf{\Sigma }}}_{{\mathbf{\text{X}}}^{°}{Y}^{°}},{\widehat{\beta }}_0={\widehat{\mu }}_{Y°}-{\widehat{\beta }}^{\prime }{\widehat{\mathit{\mu }}}_{\mathbf{\text{X}}°}.$$ (9)

The nonparametric bootstrap is proposed for standard errors and Wald-type confidence intervals. In addition, one can estimate the multiple correlation coefficient

$$R^2={\Sigma }_{{\text{X}}^{°}{Y}^{°}}^{\prime }{\Sigma }_{{\text{X}}^{°}{\text{X}}^{°}}^{-1}{\Sigma }_{{\text{X}}^{°}{Y}^{°}}/{\Sigma }_{Y^{°}{Y}^{°}}$$ (10)

by replacing its population components by their MPPLEs. Note that this R² is a population quantity, and care should be given to interpreting ${\widehat{R}}^2$ as the percentage of the variability of a potentially censored Y explained by the regression function of the observed X’s, also subject to censoring, when using MPPLEs of the regression coefficients for a given data set.

3.2 |. Typical Linear Regression Example

A typical environmental health problem often involves uncensored variates (U₁,...,U_q) and contaminants $\left(Z_1^{°},\dots ,Z_p^{°}\right)$ subject to left censoring for which the linear system of equations (1) holds along with the associated assumptions. The $\epsilon ={\left({\epsilon }_1,\dots ,{\epsilon }_p\right)}^{\prime }$ error terms in (1) are allowed to share unmeasured factors that induce correlations among the Z° variates. In this section we shall assume that no Z° directly generates any other Z° (e.g. no metabolites) so that model (1) is directly applicable. We shall relax this assumption in the next section. Suppose that the scientific problem of interest is how one of the Z° ‘s, say $Y^{°}=Z_p^{°},$ is related to the others by a single regression model

$$Y_i^{°}={\beta }_0+{\beta }_1^{\prime }{\mathbf{U}}_i+{\beta }_2^{\prime }{\mathbf{Z}}_i^{*°}+e_i$$ (11)

where ${\mathbf{Z}}_i^{*°}={\left(Z_1^{°},\dots ,Z_{p-1}^{°}\right)}^{\prime }.$ In this section we explore the parametric connection between (1) and (11). Although somewhat terse, matrix algebra is useful for simplified expressions and derivations. Model (1) can be abbreviated to ${\mathbf{\text{Z}}}_i^{°}={\varphi }_0+\varphi {\mathbf{\text{U}}}_i+{\epsilon }_i,$ where ${\varphi }_0={\left({\varphi }_{10},\dots ,{\varphi }_{p0}\right)}^{\prime }\text{and}{\varphi }^{\prime }=\left({\varphi }_1,\dots ,{\varphi }_p\right),$ the latter being a q × p matrix. The major blocks of the $(q+p)\times (q+p)$ covariance matrix ${\Sigma }_{{\text{D}}^{°}}\text{of}{\mathbf{D}}^{°}={\left({\mathbf{U}}^{\prime },{\mathbf{Z}}^{{°}^{\prime }}\right)}^{\prime }$ can easily be shown to be ${\Sigma }_{\text{UU}},{\Sigma }_{{\text{UZ}}^{°}}={\Sigma }_{\text{UU}}{\varphi }^{\prime }\text{and}{\Sigma }_{{\text{Z}}^{°}{\text{Z}}^{°}}=\varphi {\Sigma }_{\text{UU}}{\varphi }^{\prime }+{\Sigma }_{\epsilon }.$ Denote the vector of predictors in (11) by ${\mathbf{X}}^{°}={\left({\mathbf{U}}^{\prime },{\mathbf{Z}}^{*\mathrm{0\prime }}\right)}^{\prime }.$ Let G be the identity matrix of dimension q + p, G_Y is the last row of G, and G_X is the first q + p − 1 rows of G, so that $Y^{°}={\mathbf{G}}_Y{\mathbf{D}}^{°}\text{and}{\mathbf{X}}^{°}={\mathbf{G}}_{\text{X}}{\mathbf{D}}^{°}.$ Then ${\Sigma }_{{\text{X}}^{°}{\text{X}}^{°}}={\text{G}}_{\text{X}}{\Sigma }_{{\text{D}}^{°}}{\text{G}}_{\text{X}}^{\prime }\text{and}{\Sigma }_{{\text{X}}^{°}{Y}^{°}}={\text{G}}_{\text{X}}{\Sigma }_{{\text{D}}^{°}}{\text{G}}_Y^{\prime },$ so that $\beta ={\Sigma }_{{\text{X}}^{°}{\text{X}}^{°}}^{-1}{\Sigma }_{{\text{X}}^{°}{Y}^{°}}$ and ${\beta }_0=E{Y}^{°}-{\beta }^{\prime }E{\mathbf{X}}^{°}.$ Equivalently, one could write $\text{Cov}\left({\mathbf{X}}^{°},e\right)=\text{Cov}\left({\mathbf{X}}^{°},Y^{°}-{\beta }^{\prime }{\mathbf{X}}^{°}\right)={\Sigma }_{X^{°}{Y}^{°}}-{\Sigma }_{X^{°}{X}^{°}\beta }=0.$ Hence, the $\widehat{\beta }$ estimators are functions of $\widehat{\varphi },{\widehat{\Sigma }}_{\text{UU}}\text{and}{\widehat{\Sigma }}_{\epsilon },$ i.e. estimators of the parameters underlying model system (1).

A different approach, that can offer valuable insight, is to assume the first p — 1 models of (1) for $\left(Z_1^{°},\dots ,Z_{p-1}^{°}\right)$ plus model (11) for Y° and then to derive how they determine the $\varphi$ parameters for $Z_p^{°}=Y^{°},$ i.e. the parameters of the last equation in (1). Since $Z_{ij}^{°}={\varphi }_{j0}+{\varphi }_j^{\prime }{\mathbf{U}}_i+{\epsilon }_{ij}$ by model (1), substitution into (11) gives

$$Y_i^{°}={\beta }_0+{\beta }_1^{\prime }{\mathbf{U}}_i+\sum _{j=1}^{p-1}{\beta }_{2j}\left({\varphi }_{j0}+{\varphi }_j^{\prime }{\mathbf{U}}_i+{\epsilon }_{ij}\right)+e_i=\left({\beta }_0+\sum _{j=1}^{p-1}{\beta }_{2j}{\varphi }_{j0}\right)+{\left({\beta }_1+\sum _{j=1}^{p-1}{\beta }_{2j}{\varphi }_j\right)}^{\prime }{\mathbf{U}}_i+\left(\sum _{j=1}^{p-1}{\beta }_{2j}{\epsilon }_{ij}+e_i\right)={\varphi }_{Y0}+{\varphi }_Y^{\prime }{\mathbf{U}}_i+e_{Yi},$$

with the obvious equivalence of $\left({\varphi }_{Y0},{\varphi }_Y\right)$ to the parameters in the last line of (1) and with $e_{Yi}={\epsilon }_{ip}.$ Furthermore, $e_i={\epsilon }_{ip}-{\sum }_1^{p-1}{\beta }_{2j}{\epsilon }_{ij}.$ Letting $\mathbf{c}=\left(-{\beta }_{21},\dots ,-{\beta }_{2,p-1},1\right),e~N\left(0,\mathbf{c}{\Sigma }_{\epsilon }\mathrm{c\prime }\right)$

3.3 |. Recursive system of regression models

In some settings the model (1) assumption that the Z° variates are generated solely from a set of uncensored U variates may not seem realistic. For example, in drawing a causal graph for a specific real-world problem, some variates with lower limits of quantification may be generated from other variates having a lower limit of quantification. As a simple example, consider the recursive system of models

$$Z_{i1}^{°}={\text{γ}}_{10}+{\text{γ}}_1^{\prime }{\mathbf{U}}_i+{\epsilon }_{i1}^{*}$$ (12)

$$Z_{i2}^{°}={\text{γ}}_{20}+{\text{γ}}_2^{\prime }{\mathbf{U}}_i+{\psi }_{21}{Z}_{i1}^{°}+{\epsilon }_{i2}^{*}$$ (13)

$$Z_{i3}^{°}={\text{γ}}_{30}+{\text{γ}}_3^{\prime }{\mathbf{U}}_i+{\psi }_{31}{Z}_{i1}^{°}+{\psi }_{32}{Z}_{i2}^{°}+{\epsilon }_{i3}^{*}$$ (14)

where $\left({\epsilon }_{i1}^{*},{\epsilon }_{i2}^{*},{\epsilon }_{i3}^{*}\right)$ is multivariate normal with mean 0 and covariance ${\Sigma }_{\epsilon *}.$ As an example, the $Z_{ij}^{°}\text{'s}$ might be the true contaminant levels in years j = 1, 2, 3 of a study. We make the usual assumption of no unmeasured confounders, whereby in each model the U covariates and the Z° ‘s, used as covariates, are uncorrelated with the regression error. This assumption is required for unbiased estimation of the regression parameters when all Z° covariates are measured exactly and is necessary here for the censored-data setting as well. Since $\text{cov}\left(Z_1^{°},{\epsilon }_2^{*}\right)={\text{γ}}_1^{\prime }\text{cov}\left(\mathbf{U},{\epsilon }_2^{*}\right)+\text{cov}\left({\epsilon }_1^{*},{\epsilon }_2^{*}\right),$ the requirement that ${\epsilon }_2^{*}$ be uncorrelated with Z° and U in (13) means ${\epsilon }_1^{*}\text{and}{\epsilon }_2^{*}$ must be uncorrelated. Similarly, one can show that if ${\epsilon }_3^{*}$ is uncorrelated with $\left(\mathbf{U},Z_1^{°},Z_2^{°}\right)$ in model (14), then ${\epsilon }_3^{*}$ is uncorrelated with ${\epsilon }_1^{*}$ and ${\epsilon }_2^{*}.$ Hence, ${\Sigma }_{\mathrm{\epsilon *}}$ is a diagonal matrix, as generally assumed for recursive systems (Johnston, 1972) — see exception below.

The recursive system of equations described in (12)-(14) can be rewritten in the form of (1), that only involves U covariates, by making sequential substitutions of the $Z_{ij}^{°}$ outcome from one model into the $Z_{ij}^{°}$ covariates of succeeding models. This results in the following model (1) parameters: $\left({\varphi }_{10},{\varphi }_1\right)=\left({\text{γ}}_{10},{\text{γ}}_1\right)$ for model (12), $\left({\varphi }_{20},{\varphi }_2\right)=\left({\text{γ}}_{20}+{\psi }_{21}{\text{γ}}_{10},{\text{γ}}_2+{\psi }_{21}{\text{γ}}_1\right)$ for model (13), and $\left({\varphi }_{30},{\varphi }_3\right)=({\text{γ}}_{30}+{\psi }_{31}{\text{γ}}_{10}+{\psi }_{32}{\varphi }_{20},{\text{γ}}_3+{\psi }_{31}{\text{γ}}_1+{\psi }_{32}{\varphi }_2)$ for model (14), with model (1) error terms becoming

$$\left(\begin{array}{l}{\epsilon }_{i1}\hfill \\ {\epsilon }_{i2}\hfill \\ {\epsilon }_{i3}\hfill \end{array}\right)=\left(\begin{array}{ccc}1& 0& 0\\ {\psi }_{21}& 1& 0\\ {\psi }_{31}+{\psi }_{32}{\psi }_{21}& {\psi }_{32}& 1\end{array}\right)\left(\begin{array}{c}{\epsilon }_{i1}^{*}\\ {\epsilon }_{i2}^{*}\\ {\epsilon }_{i3}^{*}\end{array}\right).$$ (15)

Writing (15) succinctly as $\epsilon =\mathbf{B}\mathrm{\epsilon *},$ the model (1) error terms are multivariate normal with mean 0 and covariance $\mathbf{B}{\Sigma }_{\mathrm{\epsilon *}}\mathbf{B}\prime .$ And very importantly, $\epsilon$ is uncorrelated with U. The exception to ${\Sigma }_{\mathrm{\epsilon *}}$ being a diagonal matrix occurs as follows. If ${\psi }_{21}=0,\text{then}{\epsilon }_{i2}^{*}={\epsilon }_{i2}$ and $\text{Cov}\left({\epsilon }_{i1},{\epsilon }_{i2}\right)$ need not be zero. Section 4.3 gives an example of a more general semi-recursive system in which (12) and (13) each contain multiple models rather than just a single model.

The important result here is that the recursive system of models (12)—(14) falls into the framework of model (1) for which the assumption of multivariate normal errors is satisfied. Hence, the means and covariance matrices of $\left(\mathbf{U},{\mathbf{Z}}^{°}\right)$ can therefore be estimated by the two-stage process described in Section 2. Estimation of regression parameters is as follows. For model (13) let $Y^{°}=Z_{i2}^{°}$ represent the outcome and ${\mathbf{X}}_i^{°}={\left({\mathbf{U}}_i^{\prime },Z_{i1}^{°}\right)}^{\prime }$ be the covariates with regression parameters ${\beta }_0={\text{γ}}_{20}\text{and}\beta ={\left({\text{γ}}_2^{\prime },{\psi }_{21}\right)}^{\prime }.$ For model (14) let $Y_i^{°}=Z_{i3}^{°}\text{and}{\mathbf{X}}_i^{°}={\left({\mathbf{U}}_i^{\prime },Z_{i1}^{°},Z_{i2}^{°}\right)}^{\prime }$ with intercept ${\beta }_0={\text{γ}}_{30}$ and regression coefficients $\beta ={\left({\text{γ}}_3^{\prime },{\psi }_{31},{\psi }_{32}\right)}^{\prime }$ In either case let ${\text{Σ}}_{{\text{X}}^{\text{°}}{\text{X}}^{\text{°}}}$ be the covariance matrix of X° and ${\text{Σ}}_{{\text{X}}^{\text{°}}{\text{Y}}^{\text{°}}}$ be the vector of covariances between X° and Y°. These quantities are estimable via (2) - (4) since the recursive system of equations falls into the framework of model (1). Hence, under the usual assumption of no unmeasured confounders, the consistent estimators are again given by $\widehat{\beta }={\widehat{\Sigma }}_{{\text{X}}^{°}{\text{X}}^{°}}^{-1}{\widehat{\Sigma }}_{{\text{X}}^{°}{Y}^{°}}^{-1},{\widehat{\beta }}_0=\widehat{E}{Y}^{°}-{\widehat{\beta }}^{\prime }\widehat{E}{\text{X}}^{°}.$ This estimation process is performed successively down the list of recursive equation models.

In some settings the data analyst may be provided a single regression model with multiple left-censored covariates. The obvious solution is to add model equations, one for each left-censored covariate. For example, if presented with just model (14), models (12) and (13) can be added, with or without ${\psi }_{21}$ depending on subject matter knowledge. The tools of this section are then available.

4 |. SPECIFIC DESIGNS AND SIMULATION STUDIES

In this section we describe the details of the proposed methods for specific designs, and then report on simulation experiments, comparing the new estimators to both the true values and the estimators obtained from analysis of the uncensored version of the data. In Section 4.1 we examine first how well the MMLEs of means and variances and the MPPLEs of correlations from Section 2.1 work and then consider a standard regression setting which parallels Section 3.2. For comparison, we consider the LOQ/2 and $\text{LOQ}/\sqrt{2}$ imputation methods, given the lack of other methods that can handle the general settings under study. As an aside we assume the data have been pre-transformed to normal, hence appropriate replacement of a left-censored value by, say, LOQ/2 is done on the lognormal scale. In Section 4.2 we consider a regression model in which the outcome is the difference in two left-censored variates and the covariates consist of uncensored and left-censored variates. Section 4.3 studies a model in which the outcome is the average of two left- censored variates and the covariates consist of uncensored variates and the average of three left-censored variates. Given the poor performance of the imputation methods in Section 4.1, they are not reported in the later sections.

4.1 |. Standard regression design

In the first simulation experiment suppose a study measures 7 variates on n = 100 subjects. There are 2 uncensored variates: $U_1~\text{Bin}(1,0.5)$ and U₂ is the normal mixture $U_1+\delta ,$ where $\delta ~N(0,4).$ There are also 5 left-censored variates $\left(Z_1^{°},\dots ,Z_5^{°}\right)$ that satisfy model equations (1) with

$$\varphi =\left(\begin{array}{ccc}0.0& 0.5& 1.0\\ 0.5& 0.3& 1.0\\ -0.5& -0.5& -1.0\\ 0.3& 0.0& -1.0\\ 0.2& 0.4& -0.6\end{array}\right)\text{and}{\Sigma }_{\epsilon }=\left(\begin{array}{ccccc}1.0& 0.2& 0.2& 0.3& 0.5\\ 0.2& 1.0& 0.4& 0.3& 0.4\\ 0.2& 0.4& 1.0& 0.5& 0.3\\ 0.3& 0.3& 0.5& 1.0& 0.2\\ 0.5& 0.4& 0.3& 0.2& 1.0\end{array}\right).$$

Furthermore, the percentages of left-censoring are 20%, 30%, 40%, 50% and 35%. The first task is to examine how well the proposed MMLEs/MPPLEs estimate the true means, variances and correlations. For comparison we include estimators based on the uncensored version of the data and the imputed versions based on LOQ/2 and $\text{LOQ}/\sqrt{2}$ approaches. Table 1 summarizes the results over 1000 simulated data sets. For the sake of space only the correlations between $Z_5^{°}$ and the other variates are shown. In all cases the MM- LEs/MPPLEs closely match the true values and uncensored data estimates of the means, variances and correlations. The imputation methods overestimate the means and under-estimate the variances. Their estimates of the correlations are biased towards zero as one might expect in a measurement error case. The second task is to examine estimates of the regression coefficients. Here, $Y^{°}=Z_5^{°}$ is regressed on the other variates using the same data sets. The true values are found as described in Section 3.2. The second panel of Table 1 summarizes the averages and standard deviations of the regression coefficients, and their average standard errors. Fifty bootstrap resamples are used to estimate the standard errors and Wald-type confidence intervals for the proposed MPPLEs. The MPPLE estimates of the regression coefficients are excellent, especially when one considers the heavy censoring of the covariates and outcome. Estimation of the population R² is also very good. The bootstrap standard errors tend to overestimate the simulation standard error, but the confidence coverages range from 95.8% to 97.5%. The imputation estimators are uniformly dismal with a substantial underestimate of R².

Table 1.

Simulation results for standard regression design. Estimation of summary measures, regression coefficients and standard errors, R². Percent left-censoring for (U, Z°) : 0, 0, 20, 30, 40, 50, 35; n = 100 over 1000 simulated data sets.

Means	U1	U2	$Z_1^0$	$Z_2^0$	$Z_3^0$	$Z_4^0$	$Z_5^0$
True	0.50	0.50	0.75	1.15	−1.25	−0.20	0.10
Uncensored	0.500	0.507	0.759	1.156	−1.258	−0.210	0.097
MPPLE	0.500	0.507	0.758	1.156	−1.258	−0.217	0.096
LOQ/2	0.500	0.507	0.882	1.389	−0.860	0.357	0.224
$\text{LOQ}/\sqrt{2}$	0.500	0.507	0.952	1.493	−0.721	0.530	0.345
Variances
True	0.250	4.250	5.562	5.422	5.562	5.250	2.450
Uncensored	0.250	4.239	5.544	5.409	5.547	5.226	2.448
MPPLE	0.250	4.239	5.534	5.409	5.536	5.254	2.452
LOQ/2	0.250	4.239	4.549	3.859	3.266	2.517	1.812
$\text{LOQ}/\sqrt{2}$	0.250	4.239	4.180	3.437	2.829	2.112	1.495
Correlations with $Z_5^0$
True	−0.064	−0.759	−0.535	−0.566	0.752	0.739
Uncensored	−0.061	−0.758	−0.535	−0.565	0.751	0.737
MPPLE	−0.061	−0.760	−0.539	−0.570	0.752	0.739
LOQ/2	−0.057	−0.717	−0.485	−0.495	0.710	0.681
$\text{LOQ}/\sqrt{2}$	−0.056	−0.710	−0.471	−0.477	0.711	0.682

Coefficients	β0	β1	β2	β3	β4	β5	β6	R2
True	0.158	0.167	−1.267	0.442	0.281	0.144	−0.089	0.738
Uncensored Data
Ave	0.156	0.168	−1.273	0.440	0.284	0.1406	−0.088	0.750
Std Dev	0.146	0.180	0.198	0.086	0.091	0.100	0.102
Ave(SE)	0.147	0.186	0.199	0.088	0.092	0.101	0.100
MPPLE
Ave	0.152	0.169	−1.288	0.443	0.288	0.142	−0.098	0.760
Std Dev	0.198	0.223	0.299	0.114	0.135	0.142	0.151
Ave(BSE)	0.221	0.251	0.343	0.132	0.154	0.166	0.179
CI Coverage	95.8	95.8	96.7	96.3	95.8	96.1	97.5
LOQ/2
Ave	0.193	0.230	−0.744	0.300	0.150	0.204	−0.018	0.668
StD Dev	0.177	0.172	0.157	0.089	0.095	0.116	0.128
Ave(SE)	0.169	0.179	0.145	0.089	0.0924	0.109	0.113
$\text{LOQ}/\sqrt{2}$
Ave	0.255	0.215	−0.646	0.270	0.136	0.202	−0.011	0.662
Std Dev	0.181	0.158	0.145	0.084	0.090	0.119	0.135
Ave(SE)	0.169	0.164	0.132	0.085	0.089	0.107	0.114

4.2 |. Regression for a post-minus-pre design

In this problem we consider a difference between two variates subject to left censoring as the outcome variable which is regressed on uncensored and potentially censored variates. A typical application is the post-pre outcome regression model

$$Y_i^{°}=Z_{i2}^{°}-Z_{i1}^{°}={\beta }_0+{\beta }_1{X}_{i1}+{\beta }_2{X}_{i2}+{\beta }_3{X}_{i3}+e_i$$ (16)

where $X_{i1}=U_{i1},X_{i2}=U_{i2},\text{and}{X}_{i3}=Z_{i1}^{°}.$ Furthermore, we suppose that a causal diagram implies the following set of regression equations for the variates subject to left censoring

$$Z_{i1}^{°}={\text{γ}}_{10}+{\text{γ}}_{11}{U}_{i1}+{\text{γ}}_{12}{U}_{i2}+{\epsilon }_{i1}^{*}$$ (17)

$$Z_{i2}^{°}={\text{γ}}_{20}+{\text{γ}}_{21}{U}_{i1}+{\text{γ}}_{22}{U}_{i2}+\psi Z_{i1}^{°}+{\epsilon }_{i2}^{*}$$ (18)

where $\left({\epsilon }_{i1}^{*},{\epsilon }_{i2}^{*}\right)$ are independent normals and uncorrelated with $\left(U_{i1},U_{i2}\right).$ As an example, $Z_{i1}^{°}\text{and}{Z}_{i2}^{°}$ might be the true blood levels of some toxicant in years 1 and 2, respectively, while $U_{i1}\text{and}{U}_{i2}$ might be the subject’s gender and age. Substitution of (17) into (18) yields

$$Z_{i2}^{°}=\left({\text{γ}}_{20}+\psi {\text{γ}}_{10}\right)+\left({\text{γ}}_{21}+\psi {\text{γ}}_{11}\right)U_{i1}+\left({\text{γ}}_{22}+\psi {\text{γ}}_{12}\right)U_{i2}+\left(\psi {\epsilon }_{i1}^{*}+{\epsilon }_{i2}^{*}\right).$$

We can now rewrite (17) and (18) into a form equivalent to (1)

$$Z_{i1}^{°}={\varphi }_{10}+{\varphi }_{11}{U}_{i1}+{\varphi }_{12}{U}_{i2}+{\epsilon }_{i1}$$ (19)

$$Z_{i2}^{°}={\varphi }_{20}+{\varphi }_{21}{U}_{i1}+{\varphi }_{22}{U}_{i2}+{\epsilon }_{i2}$$ (20)

where $\left({\varphi }_{10},{\varphi }_{11},{\varphi }_{12}\right)=\left({\text{γ}}_{10},{\text{γ}}_{11},{\text{γ}}_{12}\right)\text{and}\left({\varphi }_{20},{\varphi }_{21},{\varphi }_{22}\right)=({\text{γ}}_{20}+\psi {\text{γ}}_{10},{\text{γ}}_{21}+\psi {\text{γ}}_{11},{\text{γ}}_{22}+\psi {\text{γ}}_{12})$ and

$$\left(\begin{array}{c}{\epsilon }_{i1}\\ {\epsilon }_{i2}\end{array}\right)=\left(\begin{array}{cc}1& 0\\ \psi & 1\end{array}\right)\left(\begin{array}{c}{\epsilon }_{i1}^{*}\\ {\epsilon }_{i2}^{*}\end{array}\right),$$ (21)

which can be abbreviated as ${\epsilon }_i=\mathbf{Q}{\epsilon }_i^{*}.$ To get unbiased estimators, we require ${\epsilon }_{i1}^{*}$ to be uncorrelated with $\left(U_{i1},U_{i2}\right)\text{and}{\epsilon }_{i2}^{*}$ uncorrelated with $\left(U_{i1},U_{i2},Z_{i1}^{°},{\epsilon }_{i1}^{*}\right).$ Since

$Cov\left(Z_1^{°},{\epsilon }_2^{*}\right)=Cov\left({\text{γ}}_{11}{U}_1+{\text{γ}}_{12}{U}_2+{\epsilon }_1^{*},{\epsilon }_2^{*}\right)=\left({\text{γ}}_{11},{\text{γ}}_{12}\right)Cov\left({\left(U_1,U_2\right)}^{\prime },{\epsilon }_2^{*}\right)+Cov\left({\epsilon }_1^{*},{\epsilon }_2^{*}\right),Cov\left(Z_1^{°},{\epsilon }_2^{*}\right)=0$ if and only if $\left(U_1,U_2,{\epsilon }_1^{*}\right)$ are uncorrelated with ${\epsilon }_2^{*}.$ Hence, by equation (21), $\epsilon ~N_2\left(0,\mathbf{Q}{\Sigma }_{\mathrm{\epsilon *}}{\mathbf{Q}}^{\prime }\right).$ Assumptions (A1)-(A3) are therefore satisfied, which implies that the means and covariance matrix ${\Sigma }_{{\text{D}}^0}\text{of}\left(U_1,U_2,Z_1^{°},Z_2^{°}\right)$ are estimable by Stages 1 and 2 given in Section 2.1. Next, define

$$\mathbf{G}=\left(\begin{array}{cccc}0& 0& -1& 1\\ 1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\end{array}\right).$$

The covariance matrix of $\left(Z_2^{°}-Z_1^{°},U_1,U_2,Z_1^{°}\right)\text{is}\mathbf{G}{\widehat{\Sigma }}_{{\text{D}}^0}{\mathbf{G}}^{\prime }.$. Estimators of the β’s in (16) follow from Section 3.1. One can also estimate the population R² and the multiple partial correlation coefficients. The nonparametric bootstrap is used for estimation of standard errors and Wald-type confidence intervals.

Simulation Study and Results: We generated 1000 data sets, each of size n = 100, with U₁ ~ Bin(1, 0.5), U₂ ~ Unif(0,1) and $\left({\epsilon }_1^{*},{\epsilon }_2^{*}\right)~N_2\left(\mathbf{0},{\Sigma }_{\mathrm{\epsilon *}}\right)$ where the coefficients in (17)–(18) and error covariance matrix are

$$\left(\begin{array}{ccc}{\text{γ}}_{\text{10}}& {\text{γ}}_{\text{11}}& {\text{γ}}_{\text{12}}\\ {\text{γ}}_{\text{20}}& {\text{γ}}_{\text{21}}& {\text{γ}}_{\text{22}}\end{array}\right)=\left(\begin{array}{ccc}1& 1& 1\\ 0.25& -0.50& -1.00\end{array}\right){\Sigma }_{\mathrm{\epsilon *}}=\left(\begin{array}{cc}1& 0\\ 0& 0.7\end{array}\right)$$

and where $\psi =\mathrm{0.75.}$ The percent left censoring for $\left(Z_1^{°},Z_2^{°}\right)$ is (22.0%, 53.6%). As seen in Table 2, the proposed MPPLEs of the β coefficients in regression model (16) have little bias, their bootstrap standard errors are very close to the sample standard deviations of the 1000 estimated β’s and the Wald-type bootstrap confidence coverages are close to the nominal 95% level. Moreover, the average of the 1000 estimates of the population multiple correlation (0.358) for the regression model is very close to the true value (0.323) and that of regression fits based on knowing the true values of the Z° variates (0.343).

Table 2.

Simulation results for Post-Pre design. Estimation of regression coefficients, bootstrap standard errors and 95% confidence interval coverages. n = 100. Percent left-censoring for $Z_1^{°},Z_2^{°}$ : 22.0%, 53.6%

True β	Method	Ave($\widehat{\beta }$)	SD($\widehat{\beta }$)	Ave(SE($\widehat{\beta }$))	CI Coverage
β0 = 0.25	Uncensored	0.259	0.203	0.207	94.9%
	MPPLE	0.253	0.298	0.298	94.7%
β1 = −0.50	Uncensored	−0.497	0.190	0.189	93.9%
	MPPLE	−0.500	0.237	0.233	94.8%
β2=−1.00	Uncensored	−1.007	0.304	0.306	95.6%
	MPPLE	−1.019	0.401	0.386	93.9%
β3 = −0.25	Uncensored	−0.253	0.084	0.085	94.8%
	MPPLE	−0.249	0.127	0.123	93.0%

Note: Multiple R²: 0.323 for true model, 0.343 for uncensored data analysis, 0.358 forMPPLE analysis. Results summarized over 1000 simulated data sets.

4.3 |. Regression design based on averages

In this example, suppose that in the previous year, 3 measurements of a toxicant are taken on unit i with true values $\left(Z_{i1}^{°},Z_{i2}^{°},Z_{i3}^{°}\right),$ and in the current year, 2 more measurements are taken with true values $\left(Z_{i4}^{°},Z_{i5}^{°}\right).$ To make this setting more realistic, we include a covariate U_i1 that would affect last year’s values as well as this year’s Z° values. In addition, suppose that a random sample of the units received an intervention or remediation after last year’s measurements, i.e. U_i2 is 1 for the intervention and is zero otherwise. Suppose that the model of interest is

$$Y_i^{°}={\beta }_0+{\beta }_1{U}_{i1}+{\beta }_2{U}_{i2}+{\beta }_3{X}_i^{°}+e_i$$ (22)

where $Y_i^{°}=\left(Z_{i4}^{°}+Z_{i5}^{°}\right)/2\text{and}{X}_i^{°}=\left(Z_{i1}^{°}+Z_{i2}^{°}+Z_{i3}^{°}\right)/3,$ the true average values of this year’s and last year’s toxicant levels, respectively. All Z° variates making up these averages are subject to left censoring.

To design a strategy for fitting model (22), suppose that a causal graph describing how each of last year’s variates were generated results in

$$Z_{i1}^{°}={\varphi }_{10}+{\varphi }_{11}{U}_{i1}+{\epsilon }_{i1},$$ (23)

$$Z_{i2}^{°}={\varphi }_{20}+{\varphi }_{21}{U}_{i1}+{\epsilon }_{i2},$$ (24)

$$Z_{i3}^{°}={\varphi }_{30}+{\varphi }_{31}{U}_{i1}+{\epsilon }_{i3}.$$ (25)

Keeping in mind that model (22) is our goal, we will model this year’s Z° values conditional on last year’s values as

$$Z_{i4}^{°}={\eta }_{40}+{\eta }_{41}{U}_{i1}+{\eta }_{42}{U}_{i2}+{\eta }_{43}\left(\frac{Z_{i1}^{°}+Z_{i2}^{°}+Z_{i3}^{°}}{3}\right)+{\epsilon }_{i4}^{*},$$ (26)

$$Z_{i5}^{°}={\eta }_{50}+{\eta }_{51}{U}_{i1}+{\eta }_{52}{U}_{i2}+{\eta }_{53}\left(\frac{Z_{i1}^{°}+Z_{i2}^{°}+Z_{i3}^{°}}{3}\right)+{\epsilon }_{i5}^{*},$$ (27)

where $\left({\epsilon }_{i4}^{*},{\epsilon }_{i5}^{*}\right)$ are bivariate normal. Note that U₁ is still allowed to have an effect on this year’s measurements beyond what is contained in last year’s average X°. As generally required for validity of regression models, U₁ is assumed independent of the error terms $\left({\epsilon }_1,{\epsilon }_2,{\epsilon }_3\right)$ in models (23)-(25) and $\left(U_1,U_2,X^{°}\right)$ are independent of $\left({\epsilon }_4^{*},{\epsilon }_5^{*}\right)$ in models (26)-(27). Models (23)-(25) are allowed to contain the same unmeasured covariates, so that the errors $\left({\epsilon }_1,{\epsilon }_2,{\epsilon }_2\right)$ are allowed to be correlated. These unmeasured covariates are contained in $\left(Z_1^{°},Z_2^{°},Z_3^{°}\right)$ so that with X° in models (26) and (27), we assume that $\left({\epsilon }_1,{\epsilon }_2,{\epsilon }_3\right)$ are uncorrelated with $\left({\epsilon }_4^{*},{\epsilon }_5^{*}\right).$ This is the analogue of the zero correlation assumption made in recursive models (Section 3.3). Finally, $\left({\epsilon }_4^{*},{\epsilon }_5^{*}\right)$ are allowed to be correlated. Note that (23)-(27) are an extension of the recursive system of equations of Section 3.3 by allowing a set of covariates to be added to the previous equation(s) rather than only adding one covariate at a time.

Before we can estimate the parameters of (22), we need to reformulate models (26) and (27) into model framework (1). The $\left(Z_1^{°},Z_2^{°},Z_3^{°}\right)$ terms in models (26) and (27) are replaced by their respective models (23)-(25). For j = 4, 5 this results in

$$Z_{ij}^{°}=\left({\eta }_{j0}+{\eta }_{j3}(1/3)\sum _1^3{\varphi }_{\mathcal{l}0}\right)+\left({\eta }_{j1}+{\eta }_{j3}(1/3)\sum _1^3{\varphi }_{\mathcal{l}1}\right)U_{i1}+{\eta }_{j2}{U}_{i2}+\left({\eta }_{j3}(1/3)\sum _1^3{\epsilon }_{i\mathcal{l}}+{\epsilon }_{ij}^{*}\right)={\varphi }_{j0}+{\varphi }_{j1}{U}_{i1}+{\varphi }_{j2}{U}_{i2}+{\epsilon }_{ij}$$ (28)

where $\left({\varphi }_{j0},{\varphi }_{j1},{\varphi }_{j2},{\epsilon }_{ij}\right)$ are implicitly defined in (28). Models (23)-(25) and (28) with j = 4, 5 have now been reconfigured in the form of (1). Next, note that the corresponding errors are

$$\left(\begin{array}{c}{\epsilon }_{i1}\\ {\epsilon }_{i2}\\ {\epsilon }_{i3}\\ {\epsilon }_{i4}\\ {\epsilon }_{i5}\end{array}\right)=\left(\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 1& 0& 0& 0\\ 0& 0& 1& 0& 0\\ {\eta }_{43}/3& {\eta }_{43}/3& {\eta }_{43}/3& 1& 0\\ {\eta }_{53}/3& {\eta }_{53}/3& {\eta }_{53}/3& 0& 1\end{array}\right)\left(\begin{array}{c}{\epsilon }_{i1}\\ {\epsilon }_{i2}\\ {\epsilon }_{i3}\\ {\epsilon }_{i4}^{*}\\ {\epsilon }_{i5}^{*}\end{array}\right):=\mathbf{A}\mathrm{\epsilon *},$$

so that the error vector of equation (1) is multivariate normal with mean 0 and covariance matrix $\mathbf{A}{\mathbf{\Sigma }}_{{\mathbf{\epsilon }}^{*}}{\mathbf{A}}^{\prime }.$ The error terms $\left({\epsilon }_1,{\epsilon }_2,{\epsilon }_3,{\epsilon }_4,{\epsilon }_5\right)$ can be correlated. The only stipulation is that $\left({\epsilon }_1,{\epsilon }_2,{\epsilon }_3\right)$ be independent of $\left({\epsilon }_4^{*},{\epsilon }_5^{*}\right).$ This generalzes the diagonal error covariance matrix assumption made in recursive systems which add only one covariate at a time.

At this point models (23)-(25) plus (28) written out for j = 4 and j = 5 represent models for $Z_1^{°},\dots ,Z_5^{°}$ in the framework of (1), in which the error terms $\left({\epsilon }_1,\dots ,{\epsilon }_5\right)$ follow a multivariate normal. We are now justified in using Stage 1 and Stage 2 estimation methods to estimate the mean ${\widehat{\mu }}_{D°}$ and covariance matrix ${\Sigma }_{{\text{D}}^{°}}$ of ${\text{D}}^{°}=\left(U_1,U_2,Z_1^{°},\dots ,Z_5^{°}\right).$ Next, define the G matrix

$$\mathbf{G}=\left(\begin{array}{ccccccc}0& 0& 0& 0& 0& 1/2& 1/2\\ 1& 0& 0& 0& 0& 0& 0\\ 0& 1& 0& 0& 0& 0& 0\\ 0& 0& 1/3& 1/3& 1/3& 0& 0\end{array}\right),$$

noting that the first row of $G\left(G_Y\right)$ is used to define the outcome $Y^{°}=G_Y{\mathbf{D}}^{°},$ the average of $Z_4^{°}\text{and}{Z}_5^{°},$ and the last 3 rows of $G\left(G_{\text{X}}\right)$ to define the covariates $\left(U_1,U_2,{\mathbf{X}}^{°}\right)=G_{\text{X}}{\mathbf{D}}^{°}.$ Equations (9) and (10) give the estimated regression coefficients $\left(\beta ,{\beta }_0\right)$ of (22) and the estimated populations R², respectively. The nonparametric bootstrap can be used for standard errors and constructing confidence intervals.

Simulation Study and Results.

We generated 1000 data sets, each with n = 100, in which U₁ ~ U(0,1) and U₂ ~ Bin(1, 0.5). The true $\left(Z_1^{°},Z_2^{°},Z_3^{°}\right)$ were generated from (23)-(25) in which their $\varphi$ coefficients were all equal to 1, the error terms were trivariate normal with variances of 1 and covariances each equal to 0.3. In models (26)-(27), $\left({\eta }_{j0},{\eta }_{j1},{\eta }_{j2},{\eta }_{j3}\right)=(0.2,-0.5,-1.0,0.7),\text{for}j=4,5,$ and hence so too were the β’s of the regression model (1). The error terms, $\left({\epsilon }_4,{\epsilon }_5\right),$ were generated from a bivariate normal with variances (0.7,0.7) and covariance 0.4. The lower limits of quantification were chosen so that each observed Z underwent 40% left censoring. The simulation results (Table 3) illustrate that the MPPLEs of the β’s in (22) are very close to the true values and to the analytic results for the uncensored version of the simulated data. The MPPLE standard errors were based on 50 nonparametric bootstrap resamples of a data set. As seen in Table 3 they are a close match to the empirical standard deviations taken over the 1000 simulated data sets; in addition, the confidence interval coverages are quite good. The population R² is precisely estimated.

Table 3.

Simulation results for Design Based on Averages. Estimation for regression coefficients, bootstrap standard errors and 95% confidence interval coverages. n = 100. Percent left-censoring for $Z_1^{°},\dots ,Z_5^{°}$: 40% each

True β	Method	Ave($\widehat{\beta }$)	SD($\widehat{\beta }$)	Ave(SE($\widehat{\beta }$))	CI Coverage
β0 = 0.2	Uncensored	0.200	0.194	0.196	95.7%
	MPPLE	0.197	0.233	0.236	95.3%
β1 = −0.5	Uncensored	−0.494	0.295	0.280	94.2%
	MPPLE	−0.492	0.330	0.316	93.4%
β2 = −1.0	Uncensored	−0.996	0.152	0.149	94.3%
	MPPLE	−0.999	0.173	0.172	95.0%
β3 = 0.7	Uncensored	0.698	0.107	0.103	94.2%
	MPPLE	0.699	0.131	0.129	95.0%

Note: Multiple R²: 0.499 for true model, 0.496 for uncensored data analysis, 0.499 for MPPLE analysis. Results summarized over 1000 simulated data sets.

5 |. REGRESSION ANALYSIS OF PCB DATA

Polychlorinated biphenyls (PCBs) are toxic man-made industrial chemicals that persist in the environment and bioaccumulate up the food chain. Due to their chemical stability, electrical insulating properties and low flammability, they were widely manufactured for use in hydraulic fluids, insulating fluids in transformers, fluorescent light ballasts, and plasticizers in paints, plastics and caulk. Although the intentional manufacture was banned in the United States and European Union, they are still released into the environment through disposal, waste burning and atmospheric recycling of PCB-containing products. They are even transported from industrial nations by air and ocean currents to highly sensitive Arctic ecosystems (Andersen et al., 2001). There is an extensive literature on serious health effects in humans and animals including carcinogenicity (Lauby-Secretan et al., 2013); reproductive abnormalities and fetal toxicity; adverse effects on the immune, nervous and endocrine systems (Brouwer, Reijnders and Koeman, 1989; Henry and DeVito, 2003 and references therein).

In the process of creating PCBs, chlorine atoms are substituted for hydrogen atoms on the biphenyl molecule, a bonded pair of benzene rings. The number and location of the chlorine atoms (up to a maximum of 10) result in 209 different PCB compounds, called congeners, and also determine the physical properties and toxicity. In general, the higher the number of chlorine atoms, the greater the bioaccumulation and the lower rate of metabolism. In the U.S., Monsanto manufactured complex mixtures of PCBs and marketed them under the name Aroclor followed by 4 digits. The last two digits are the percent chlorine by weight.

As part of an on-going U. S. National Institutes of Health-funded study, the AESOP study (Ampelman et al., 2015), blood samples are taken from subjects in East Chicago (urban site) and Columbus Junction, Iowa (rural site). Concentrations of congener-specific levels are determined by gas chromatography with tandem mass spectrometry. Of the 209 congeners, 159 chromatographic peaks can be measured: some are individual congeners, e.g PCB 118, and others co-eluting, e.g. PCBs 99 and 83. Of the 159 congeners measured, 10 have been chosen for this analytical example. All have at least 35% of their values uncensored. As PCBs are lipophilic, these measurements are standardized as the ratio of nanograms (ng) of congener per milliliter (ml) of blood lipids.

Two issues arise with the measurements. The first issue is that the amount of congener, the ratio numerator, may fall below the laboratory LOQ, whereas the amount of blood lipids, the ratio denominator, is measured exactly. The left-censoring point of the ratio differs therefore from one subject to another based on different lipid levels. The second issue is the normality assumption. The ratio data are highly skewed, and hence the log transform has been conventionally used in this field. For this analytical data set only one subject stands out in the normal probability plots as an outlier and has been removed. She has extremely high concentrations of most congeners relative to all subjects, even though her blood lipid level is typical.

Year 2 data from the AESOP study were previously analyzed by Jones, Perry & Thorne (2015). We shall use a subset of the year 2 data consisting solely of the 92 adult women on study. Table 4 provides the names of the 10 PCB congener groupings, referred to as peaks, and the percentage of samples that are above the LOQ (uncensored). Two uncensored variables will be incorporated into all regression models: site, to measure urban vs. rural difference, and age, since older subjects should have greater cumulative PCB exposure. Table 4 lists the MMLEs of the mean and standard deviation of each peak, found separately by use of regression model (1) with peak as the outcome, and site and age as covariates. There is considerable variability among congeners as to the amounts in blood samples.

Table 4.

Analysis of PCB congeners^*. In each line the outcome variable P118 is regressed on 3 covariates: Site, Age and Peak.

Peak	PCB Congener Grouping†	MMLE of Mean (SD)	Percent Uncensored	Linear Regression** Slope (SEB) Wald	R2	Partial Cor Y,Peak††
A	P70+74+61+76	1.832 (0.675)	40.86	0.625 (0.145) 4.322	0.324	0.519
B	P99+83	1.369 (0.613)	90.32	0.724 (0.098) 7.368	0.485	0.665
C	P105	0.339 (0.830)	37.63	0.714 (0.099) 7.192	0.808	0.890
D	P118 (Y)	1.808 (0.677)	83.87
E	P138+129+163	2.150 (0.670)	95.70	0.568 (0.142) 4.006	0.317	0.511
F	P146	0.166 (0.826)	75.27	0.321 (0.114) 2.811	0.191	0.353
G	P153+168	2.275 (0.682)	98.92	0.506 (0.141) 3.600	0.267	0.454
H	P156+157	0.486 (0.786)	58.06	0.532 (0.135) 3.928	0.337	0.531
I	P187	0.790 (0.823)	84.95	0.238 (0.126) 1.890	0.137	0.256
J	P203	−0.246 (0.956)	46.24	0.051 (0.102) 0.501	0.080	0.063

^*Log of congener values used to achieve normality in model (1).

^†For co-eluting congeners, the major congener by Aroclor production is listed first.

Minor congeners are superscripted.

^**Coefficient (SE), Wald Statistic shown for Peak but not Intercept, Site and Age.

^††Partial correlation between P118 (Y) and Peak are conditional on Site and Age.

To illustrate a left-censored outcome regressed on uncensored and left-censored covariates, PCB 118 (Peak D) is chosen as the outcome variable of interest. This particularly toxic dioxin-like congener displays clear carcinogenic activity in rats, specifically, liver cancers, as well as non-neoplastic lesions in the liver, lung, adrenal cortex, pancreas, thyroid, nose and kidney (National Toxicology Program Techinical Report, 2010). Table 4 lists the MPPL estimates of the regression coefficients, bootstrap standard errors and Wald ratios for 9 separate regressions predicting PCB 118 using site, age and a single peak as the 3 covariates. The coefficient estimates for site and age are not given. In addition, Table 4 lists the MPPLEs of the population R² and partial correlation between PCB 118 and each peak given site and age. One of the most striking results is that Peak B has a slightly larger Wald statistic (7.368) than Peak C (7.192), but its partial correlation of 0.665 is much smaller than Peak C’s partial correlation of 0.890. This is also evident when comparing the estimates of the population ${\widehat{R}}^2$ values: 0.485 for Peak B and 0.808 for Peak C. This Wald ratio-partial correlation paradox is due in part to the fact that Peak B has 90.3% of its values uncensored as compared to just 37.6% uncensored values for Peak C. We return to this issue later.

As a further illustration of the proposed multiple linear regression technique using both uncensored and left-censored covariates, we consider multivariable variable selection next. There are several model building strategies, the most popular ones based on significance tests, information criteria and penalized likelihoods. Forward stepwise selection will be exhibited here, since the other methods would require new theoretical development, not done here. Step 1 in Table 5 lists the estimated coefficients, standard errors and Wald statistics for the base model containing site and age. Since Peak B has the largest Wald value (Table 4), it is the first variable added to the base model and is summarized in Step 2 of Table 5. In the next step Peak C has the largest Wald ratio of any peak, and the resulting model is summarized in Step 3. Note that the addition of Peak C has caused the Peak B Wald statistic to not only lose significance but to even change sign. Hence, in the backward glance of the forward stepwise procedure, Peak B is dropped resulting in the model given in Step 4 of Table 5. In the next step Peak I with a Wald statistic of 2.618 is added (Step 5 summary). Note that this addition has almost no effect on the coefficient or standard error of Peak C. The partial correlation between Peak D (PCB 118) and Peak I (PCB 187) given site, age and Peak C (PCB 105) is 0.401. Finally, no other covariate significantly improved on the last model; the highest positive Wald statistic is 0.850 for Peak F and the largest negative Wald statistic is −0.861 for Peak B, which had been dropped in Step 4. Had either of these variates been added, the corresponding estimated population ${\widehat{R}}^2\text{s}$ would be 0.850 and 0.853, which are only slight increases over 0.839. Hence, the final model is given in the last column of Table 5. The corresponding 95% bootstrap confidence intervals are (−0.169, 0.212) for Site, (−0.016, 0.015) for Age, (0.508, 0.896) for P105, (0.043, 0.299) for P187 and (0.825, 2.058) for the intercept.

Table 5.

Forward stepwise regression for the outcome P118 with congeners^* from Table 4 considered as potential covariates. Site and age are forced into each model.

	Summary** of Forward Stepwise Regression Modeling Steps
Term† (Peak)	1	2	3	4	5
Intercept	0.572 (0.534)	0.326 (0.358)	1.121 (0.423)	1.068 (0.347)	1.441 (0.314)
	1.071	0.910	2.652	3.077	4.586
Site	−0.019 (0.147)	−0.095 (0.108)	0.041 (0.118)	0.030 (0.101)	0.022 (0.097)
	−0.132	−0.883	0.345	0.300	0.222
Age	0.031 (0.012)	0.014 (0.008)	0.013 (0.008)	0.012 (0.008)	−0.0005 (0.008)
	2.507	1.800	1.582	1.513	−0.059
P99+83(B)		0.724 (0.098)	−0.071 (0.211)
		7.368	−0.338
P105(C)			0.755 (0.169)	0.714 (0.099)	0.702 (0.099)
			4.471	7.192	7.084
P187(I)					0.171 (0.065)
					2.618
MPPLE-R2	0.076	0.485	0.809	0.808	0.839

^*Log of congener values used to achieve normality in model (1).

^**Tabled values are regression coefficient estimate (standard error) and Wald statistic.

^†For co-eluting congeners, the major congener by Aroclor production is listed first.

Minor congeners are superscripted.

6 |. DISCUSSION

Left-censored linear regression is a very important area of research that is vital to toxicology and environmental epidemiology. In 2010 the journal Epidemiology devoted a special issue to left-censored data (Schisterman and Little, 2010), with a call for further development. Early developmental work focused on the setting in which only the outcome variate is subject to left censoring. For general settings it was common in environmental and toxicological research to replace left-censored covariates and outcome by the LOQ/2 or $\text{LOQ}/\sqrt{2},$ then to log transform the data and use common regression tools, like least squares. This practice is still in common use today. More recent proposals have been to use multiple imputation and the EM algorithm, oftentimes developed for the simpler case of a single left-censored covariate.

The goal of this paper is to address the far more general framework in which both the outcome and multiple covariates are subject to left censoring or are linear combinations of left-censored variates. The proposed approach involves a pseudo-likelihood framework that uses plug-in estimators of lower level parameters in order to estimate higher level parameters. One would expect this approach to lose some efficiency relative to classical maximum likelihood but to have some practical advantages too. Simultaneous maximiza- ton of a joint likelihood in one stage is not as practical due to the many different patterns of left-censoring (maximum of 2^p) and the complicated integrations. This issue with the MLE is described in greater detail in Jones et al. (2015, pp. 87–8). That paper also showed that joint maximization of the means, variances and correlation for the case of p = 2 is hampered by suboptimal convergence rates, and may have no real improvement in bias or mean squared error. The MPPLE approach will be pursued in subsequent papers for other settings, e.g. the analysis of multivariate data including repeated measures, profiles, and principal components. As far as I am aware, all methods proposed in this paper are new for the field of left-censored data analysis. Since the focus in this paper is on measurement involving a lower LOQ, the methods are developed for left-censored data. However, the likelihoods ${\mathcal{L}}_j$ in Stage 1and ${\mathcal{L}}_{jm}$ in Stage 2 are easily modified to handle right-censored or interval-censored data which are then maximized. Usable software is under development and will become available. The framework and proposed methods are briefly summarized next.

After transformation, usually logarithmic, measures of environmental contaminants are very often normally distributed for a homogeneous population given a common exposure. However, this may not be true for non-homogeneous populations or those with different exposure histories. For this reason, toxicants, like PCBs which persist and bioaccumulate so that blood levels may depend on personal characteristics, e.g. age and residence, exhibit a distribution that is a mixture of normals. Modeling framework (1) addresses this issue, by allowing one to adjust for uncensored covariates. A marginal and pairwise pseudo-likelihood approach is proposed to estimate the means and covariance matrix of all uncensored and left-censored covariates in this framework. This estimation is extended to linear combinations, such as differences and averages of left-censored variates. Methods for linear regression involving an outcome and covariates, which can all be linear combinations of left-censored and uncensored variates, are proposed and are shown to perform quite well via simulation studies. This work is further extended into recursive and semi-recursive systems of equations that extend the conceptual framework of (1). The development of these methods for three common research designs is given and are studied via simulation studies. The results are quite good. The proposed linear regression methodology is illustrated on data from an on-going PCB study which motivated the proposed methods.