Semiparametric isotonic regression modelling and estimation for group testing data

Abstract

In the group testing procedure, several individual samples are grouped and the pooled samples, instead of each individual sample, are tested for outcome status (e.g., infectious disease status). Although this cost-effectiveness strategy in data collection is both labor and time efficient, it poses statistical challenges to derive statistically and computationally efficient estimators under semiparametric models. We consider semiparametric isotonic regression models for the simultaneous estimation of the conditional probability curve and covariate effects, in which a parametric form for combining the covariate information is assumed and the monotonic link function is left unspecified. We develop an expectation-maximization algorithm to overcome the computational challenge and embed the pool-adjacent violators algorithm in the M-step to facilitate the computation. We establish the large sample behavior of the proposed estimators and examine their finite sample performance in simulation studies. We apply the proposed method to data from the National Health and Nutrition Examination Survey for illustration.

1. INTRODUCTION

Group testing has been commonly used to save time and cost in screening for infectious diseases (Dorfman, 1943; Lewis et al., 2012; Van et al., 2012), drug discovery (Remlinger et al., 2006), quality control (Johnson and Pearson, 1999) and genetic studies (Gastwirth, 2000). When screening for infection diseases, group testing begins by collecting specimens (e.g., blood, urine,and plasma) from individuals; these samples are then physically combined to form a pooled specimen. The pooled specimen is then tested for the infection of interest. Accordingly, the test response provides evidence of whether or not the pool contains at least one individual who has a positive status for the disease of interest. If a pool tests negative, all individuals within the pool are declared to have negative status for the infection of interest. It is known that group testing is a cost-effective strategy when the disease prevalence is low. Furthermore, the utility of array-based group testing algorithms has been shown to reduce rates of misclassification (Kim and Hudgens, 2009). Another area where group testing methods have been found useful includes biomarker evaluation, in which pooled samples result in the average of the biomarker concentrations over the pooled individuals (Bondell et al., 2007). The cost-effectiveness of group testing does not, however, come without a price. The use of group testing creates challenges for statistical analysis due to the absence of individual responses.

There is extensive literature on estimating procedures for group testing, most of which has focused on nonparametric procedures (Delaigle and Meister, 2011; Delaigle et al., 2012; Wang et al., 2013; Mitchell et al., 2014; Delaigle and Zhou, 2015). In many studies that use the group testing procedure, one or several covariates are also available, and it is of interest to estimate the conditional probability of the response, given such covariate information. Having postulated a parametric regression model, the shape of the link function is assumed to be known up to finitely many unknown regression parameters (Xie, 2001; Zhang et al., 2013). However, in many situations, the researchers have no prior information regarding the mathematical specification of the true link function. When the assumed link function cannot capture the underlying shape function of the conditional density, the fitted results have undesirable properties such as inaccurate estimators and misleading inference. Delaigle et al. (2014) relaxed the parametric regression models and developed semiparametric models by using a nonparametric weight function, for which it requires a determination of the bandwidth.

Isotonic regression is a least squares problem under order restrictions. Pioneering work was conducted by Ayer et al. (1955), and a comprehensive review can be found in Barlow et al. (1972). A unique solution to standard isotonic regression exists and can be obtained using the pool-adjacent violators algorithm (PAVA) (Barlow et al., 1972; Best and Chakravarti, 1990). The computational aspects of PAVA and its fast implementation in R are discussed in Mair et al. (2009). Standard isotonic regression with PAVA cannot be directly applied to group testing data, since individual responses are not available. Our strategy is to use a semiparametric isotonic regression model and a computationally appealing algorithm by integrating PAVA and the expectation-maximization (EM) algorithm.

The remaining sections of this paper are as follows. In Section 2, we introduce the semiparametric isotonic regression model and then describe the proposed computational algorithm for the estimation. In Section 3, we establish the asymptotic properties of the estimators. We conduct simulation studies to assess the finite sample performance of the proposed algorithm, and compare it with existing estimating methods in Section 4. We apply the proposed method to data from the National Health and Nutrition Examination Survey (NHANES) in Section 5 and provide a brief discussion in Section 6.

2. MODELS AND ESTIMATION

Suppose there are N subjects to be tested and each subject is assigned to exactly one of n pools. The binary outcome Y_ij represents the true binary status for subject i in group j, which cannot be observed directly. Here, Y_ij = 1 indicates that the response of this subject is positive, and Y_ij = 0 otherwise. Let $Y_j^{*}$ denote the observed binary status for the jth pool with a size of n_j, $Y_j^{*}={\max }_{\{i=1,\cdots ,n_j\}}{Y}_{ij}$; i.e., $Y_j^{*}=1$ if the pool contains at least one positive subject, $Y_j^{*}=0$, otherwise. For each subject in the pool, a vector of subject characteristics X_ij = (x_ij,1, …, x_ij,d)^T is observed, i = 1, …, n_j; j = 1, …, n. To acknowledge heterogeneity, we assume the mean of Y_ij is related to the subjects characteristics through a monotonic function,

$$P(Y_{ij}=1|{\mathbf{X}}_{ij})=\pi \{v({\mathbf{X}}_{ij};\mathit{\beta })\},$$ (1)

where π(·) denotes the unknown monotonic nondecreasing function, v(X_ij; β) is a pre-specified function of the subjects’ characteristics and β = (β₁, …, β_d)^T is a finite-dimensional parameter. A commonly used linear model summarizes the subject information as v(X_ij; β) = β^TX_ij. Due to the unspecified link function, we need to impose an additional constraint on the regression coefficients to ensure model identifiability. It has been well studied that such a model is identifiable if either one of the components of β is fixed or if ‖β‖ = 1 and the first component of β > 0 (Ichimura, 1993; Hristache et al., 2001; Groeneboom and Hendrickx, 2019). We chose the first option due to its computational ease.

2.1. Estimation algorithm

The mass-density of the observed data from pool j, ($Y_j^{*}$, x_ij, i = 1, …, n_j), is

$$P(Y_j^{*}|{\mathbf{x}}_{ij},i=1,\cdots ,n_j,\mathit{\beta },\pi )\prod _{i=1}^{n_j}g({\mathbf{x}}_{ij})=\prod _{i=1}^{n_j}{\left[1-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}\right]}^{1-Y_j^{*}}{\left(1-\prod _{i=1}^{n_j}\left[1-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}\right]\right)}^{Y_j^{*}}g({\mathbf{x}}_{ij}),$$

where g(.) is the density function of the covariate. Denote, $\mathbf{O}=\{Y_j^{*},{\mathbf{x}}_{ij}:i=1,\dots ,n_j;j=1,\dots ,n\}$ for the observed data. Leaving out g(·), we have the log-likelihood of the observed data

$$l(\mathit{\beta },\pi |O)=\sum _{j=1}^n\left\{Y_j^{*}\log \left(1-\prod _{i=1}^{n_j}\left[1-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}\right]\right)+(1-Y_j^{*})\log \prod _{i=1}^{n_j}\left[1-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}\right]\right\}.$$ (2)

Let B be the range of β, and Π = {π(·) on R: 0 ≤ π(·) ≤ 1, π(·) is monotone nondecreasing}. Then, (β, π) can be estimated by maximizing the above likelihood, subject to the monotonic constraint for π denoted as $(\widehat{\mathit{\beta }},\widehat{\pi })={\mathrm{arg\; max}}_{(\mathit{\beta },\pi )\in (B,\Pi )}l(\mathit{\beta },\pi |\mathbf{O})$. However, directly maximizing the likelihood in Equation (2) is computationally challenging, due to the non-parametric part involved. Alternatively, to fully use the existing computing algorithm for isotonic regression, we treat the unobserved data Y_ij as the missing data and use the EM algorithm to maximize the log-likelihood of the complete data,

$$l_c(\mathit{\beta },\pi )=\sum _{j=1}^n\sum _{i=1}^{n_j}\left[Y_{ij}\log \pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}+(1-Y_{ij})\log (1-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\})\right]$$

subject to the monotonic constraint for π. Following the principle of the EM algorithm, the following iterative steps are used.

E-step: Given an initial estimator (β⁽⁰⁾, π⁽⁰⁾) and O, calculate the expected complete log-likelihood function,

$$E(l_c|\mathbf{O};{\mathit{\beta }}^{(0)},{\pi }^{(0)})=\sum _{j=1}^n\sum _{i=1}^{n_j}E(Y_{ij}|\mathbf{O};{\mathit{\beta }}^{(0)},{\pi }^{(0)})\log \pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}+\left\{1-E(Y_{ij}|\mathbf{O};{\mathit{\beta }}^{(0)},{\pi }^{(0)})\right\}\log \left[1-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}\right],$$ (3)

where $E(Y_{ij}|Y_j^{*}=0,{\mathbf{x}}_{ij};{\mathit{\beta }}^{(0)},{\pi }^{(0)})=0$ and

$$\text{E}(Y_{ij}|Y_j^{*}=1,{\mathbf{x}}_{ij};{\mathit{\beta }}^{(0)},{\pi }^{(0)})=\frac{{\pi }^{(0)}\{\upsilon ({\mathbf{x}}_{ij};{\mathit{\beta }}^{(0)})\}}{1-\prod _{i=1}^{n_j}\left[1-{\pi }^{(0)}\{\upsilon ({\mathbf{x}}_{ij};{\mathit{\beta }}^{(0)})\}\right]}.$$

M-step: Maximize the conditional expectation of the log-likelihood of the complete data from the E-step, and update the estimates for the unknown parameters. Note that directly maximizing the likelihood in Equation (3) with the monotonic constraint for π is not straightforward. Considering that the likelihood in Equation (3) belongs to the exponential family, we can apply PAVA to simplify the computational task. Theorems 1.5.1 and 1.5.2 in Robertson (1988) unify the theory of order-restricted inference, including the maximization of the likelihood function within the exponential family. In our setting, we can easily verify that the regularity conditions in the Theorems are satisfied. It follows that maximizing the likelihood in Equation (3) under the monotonic constraint is equivalent to minimizing the following sum of squares, denoted as Q(β, π) under the same constraint:

$$\begin{gathered} \left({\mathit{\beta }}^{(1)},{\pi }^{(1)}\right)=\underset{\pi \{{\upsilon }_{(1)}(\mathit{\beta })\}\le \cdots \le \pi \{{\upsilon }_n(\mathit{\beta })\}}{\mathrm{arg\; max}}E(l_c|\mathbf{O};{\mathit{\beta }}^{(0)},{\pi }^{(0)}) \\ =\underset{\pi \{{\upsilon }_{(1)}(\mathit{\beta })\}\le \cdots \le \pi \{{\upsilon }_n(\mathit{\beta })\}}{\mathrm{arg\; min}}\sum _{j=1}^n\sum _{i=1}^{n_j}{\left[E(Y_{ij}|\mathit{O};{\mathit{\beta }}^{(0)},{\pi }^{(0)})-\pi \{\upsilon ({\mathbf{x}}_{ij};\mathit{\beta })\}\right]}^2 \\ =\underset{\pi \{{\upsilon }_{(1)}(\mathit{\beta })\}\le \cdots \le \pi \{{\upsilon }_n(\mathit{\beta })\}}{\mathrm{arg\; min}}Q(\mathit{\beta },\pi ), \end{gathered}$$

where v₍₁₎(β), …, v_(n)(β) are given in the following step. The minimization can be carried out through the following three steps:

Step 1. For a given β, sort v(x_ij, β), i = 1, …, n_j, j = 1, … n from the smallest to largest, denoted as v₍₁₎(β) ≤ v₍₂₎(β) ≤ ⋯ ≤ v_(n)(β).

Step 2. Use PAVA to minimize the objection function Q(β, π) with respect to π(·), subject to the condition that if v₍₁₎(β) ≤ v₍₂₎(β) ≤ ⋯ ≤ v_(n)(β) then π{v₍₁₎(β)} ≤ π{v₍₂₎(β)} ≤ ⋯ ≤ π{v_(n)(β)}.

Step 3. Minimize $Q(\mathit{\beta },\widehat{\pi }(\mathit{\beta }))$ with respect to β, and denote the resulting minimizer as (β⁽¹⁾, π⁽¹⁾). In fact, β⁽¹⁾ is a profile maximum likelihood estimator (MLE) at this iteration. See, for example, Severini and Wong (1992); Murphy and Van der Vaart (2000) for the properties of such an estimator.

After obtaining the r-th iteration estimator (β^(r), π^(r)) we re-calculate the E-step E(l_c|O; β^(r), π^(r)) and then obtain the next iteration estimator (β^(r+1), π^(r+1)) by the M-step. With a given convergence criterion, we can iterate the E-step and M-step until the convergence criterion is satisfied.

3. ASYMPTOTIC PROPERTIES

Without loss of generality, we assume the group size n_j to be 2 and n = N/2 for notation simplicity. We establish the asymptotic properties of the estimator $({\widehat{\mathit{\beta }}}_n,{\widehat{\pi }}_n)$ of the true parameters (β₀, π₀). Denote ℓ(β, π|o) be the log-likelihood based on o = {y*, x}, x = (x₁, x₂), ${\dot{\ell }}_{\beta }(\mathit{\beta },\pi |\mathit{o})=\partial \ell (\mathit{\beta },\pi |\mathit{o})/\partial \mathit{\beta }$, ${\dot{\ell }}_{\pi }(\mathit{\beta },\pi |\mathit{o})\left[h\right]$ be the Hadamard derivative of ℓ(β, π|o) with respect to π in the direction h, $\dot{\pi }(t)=d\pi (t)/dt$, and $\dot{v}(\mathit{x},\mathit{\beta })=\partial v(\mathit{x},\mathit{\beta })/\partial \mathit{\beta }$. As a convention, O and X etc. denote random vectors and o and x etc. denote for observations. Denote

$$\begin{gathered} H_1(\mathit{x}|\mathit{\beta },\pi )=E_{({\beta }_0,{\pi }_0)}\left[{\dot{\ell }}_{\beta }(\mathit{\beta },\pi |\mathit{O})|\mathit{X}=\mathit{x}\right] \\ =-\prod _{i=1}^2\frac{1-{\pi }_0(v({\mathit{x}}_i,{\mathit{\beta }}_0))}{1-\pi (v({\mathit{x}}_i,\mathit{\beta }))}\sum _{j=1}^2\dot{\pi }(v({\mathit{x}}_j,\mathit{\beta }))\dot{v}({\mathit{x}}_j,\mathit{\beta }) \\ +\frac{1-\prod _{i=1}^2\left[1-{\pi }_0(v({\mathit{x}}_i,{\mathit{\beta }}_0)\right]}{1-\prod _{i=1}^2\left[1-\pi (v({\mathit{x}}_i,\mathit{\beta })\right]}\sum _{j=1}^2\left[1-\pi (v({\mathit{x}}_j,\mathit{\beta }))\right]\dot{\pi }(v({\mathit{x}}_{3-j},\mathit{\beta }))\dot{v}({\mathit{x}}_{3-j},\mathit{\beta }), \end{gathered}$$

(in the above x_3−j = x₂ for j = 1, and x_3−j = x₁ for j = 2) and

$$\begin{gathered} {\text{H}}_2(\mathit{x}|\mathit{\beta },\pi )=E_{({\beta }_0,{\pi }_0)}\left[{\dot{\ell }}_{\pi }(\mathit{\beta },\pi |\mathit{O})\left[1\right]|\mathit{X}=\mathit{x}\right] \\ =-\prod _{i=1}^2\left[1-\pi (v({\mathit{x}}_i,\mathit{\beta })\right]\sum _{j=1}^2\frac{1-{\pi }_0(v({\mathit{x}}_j,{\mathit{\beta }}_0))}{1-\pi (v({\mathit{x}}_j,\mathit{\beta }))} \\ +\frac{1-\prod _{i=1}^2\left[1-{\pi }_0(v({\mathit{x}}_i,{\mathit{\beta }}_0)\right]}{1-\prod _{i=1}^2\left[1-\pi (v({\mathit{x}}_i,\mathit{\beta })\right]}\sum _{j=1}^2\left[1-\pi (v({\mathit{x}}_j,\mathit{\beta }))\right]. \end{gathered}$$

We assume the following regularity conditions and summarize the asymptotic properties in Theorems 1 and 2, the proofs of which we sketch in the Appendix.

(C1). B is bounded.

(C2). $\dot{\pi }(·)$ and $\dot{v}(\mathit{x},\mathit{\beta })$ exist and $E_{({\mathit{\beta }}_0,{\pi }_0)}{\sup }_{(\mathit{\beta },\pi )\in (B,\Pi )}\left[{\Vert H_1(\mathit{X}|\mathit{\beta },\pi )\Vert }^2+H_2^2(\mathit{X}|\mathit{\beta },\pi )\right]<\infty$. Some discussions on this condition are provided in Supplementary Materials.

(C3). $\dot{v}(\mathit{x},·)$ and $\dot{\pi }(·)$ are Lipschitz uniformly for π in a neighborhood of π₀.

(C4). $v(\mathit{x},·)$ and $\pi (·)$ are second order differentiable and L₁(P) bounded, uniformly for π in a neighborhood of π₀.

Theorem 1. Under regularity conditions (C1)-(C2), the estimators are consistent: ${\widehat{\mathit{\beta }}}_n\stackrel{a.s.}{\to }{\mathit{\beta }}_0$, and ${\text{sup}}_{v\in R}|{\widehat{\pi }}_n(v)-{\pi }_0(v)|\stackrel{a.s.}{\to }0$.

Theorem 2. Under regularity conditions (C1)-(C4), $d(\widehat{\mathit{\beta }},\widehat{\pi };{\mathit{\beta }}_0,{\pi }_0)=O_p(n^{-1/3})$, where $d(\widehat{\mathit{\beta }},\widehat{\pi };{\mathit{\beta }}_0,{\pi }_0)=\Vert \widehat{\mathit{\beta }}-{\mathit{\beta }}_0\Vert +\Vert \widehat{\pi }-{\pi }_0\Vert$, with $\Vert \widehat{\mathit{\beta }}-{\mathit{\beta }}_0\Vert$ the Euclidean distance and $\Vert \widehat{\pi }-{\pi }_0\Vert ={\text{sup}}_{v\in R}|{\widehat{\pi }}_n(v)-{\pi }_0(v)|$.

Remark 1. When β is known, it can be shown that under suitable regularity conditions, $n^{1/3}(\widehat{\pi }(v)-{\pi }_0(v))\stackrel{D}{\to }A(v){\mathrm{arg\; min}}_h\{\mathbb{B}(h)+h^2\}$, for some A(v), with $\mathbb{\text{B}}(·)$ being the two-sided Browning motion starting from zero. This implies $|\widehat{\pi }(v)-{\pi }_0(v)|=O_p(n^{-1/3})$ for each fixed v, and ${\text{sup}}_{v\in R}|{\widehat{\pi }}_n(v)-{\pi }_0(v)|\ge O_p(n^{-1/3})$. When β is unknown to be estimated, we should still have ${\text{sup}}_{v\in R}|{\widehat{\pi }}_n(v)-{\pi }_0(v)|\ge O_p(n^{-1/3})$. Thus the convergence rate n^1/3 given in Theorem 2 is optimal.

Remark 2. As $\widehat{\mathit{\beta }}$ is bundled together with $\widehat{\pi }(·)$, which is a non-smooth step function, the $\sqrt{n}-\text{convergence}$ of $\widehat{\mathit{\beta }}$ and its weak limit is still an open problem (see Section 3.2.2 of Huang and Wellner (1997)). This, in turn, affected the derivation of the asymptotic distribution of $\widehat{\pi }(v)$ which is also currently open. The conjecture is that as long as $|\widehat{\mathit{\beta }}-{\mathit{\beta }}_0|=o_p(n^{1/3})$, it still holds that $n^{1/3}(|{\widehat{\pi }}_n(v)-{\pi }_0(v)|)\stackrel{D}{\to }A(v){\mathrm{arg\; min}}_h\{\mathbb{B}(h)+h^2\}$. But the fact that $|\widehat{\mathit{\beta }}-{\mathit{\beta }}_0|=o_p(n^{1/3})$ needs to be justified which is not easy.

4. SIMULATION RESULTS

We conducted simulation studies to evaluate the finite sample performance of the proposed method and compared it to that of Delaigle et al. (2014) and the standard logistic regression. We generated group testing data with a group size of 2 from n pairs, which were set to be 100, 250, 500, and 5000, representing scenarios with small (N = 200), moderate (N = 500 or 1000), or large (N = 10000) sample sizes. The covariate vector included three components (X₁, X₂, X₃), where X₁ followed a standard normal distribution, X₂ followed a Bernoulli distribution with a probability of 0.5, and X₃ followed a uniform distribution on [−3, 3]. Two models were considered to generate the binary outcome:

$$P(Y=1|\mathbf{x})=\frac{1}{1+\exp \{-(x_1+0.5x_2+0.9x_3-4.5)\}}$$ (i)

$$P(Y=1|\mathbf{x})=\frac{0.1}{1+\exp \{-2(x_1+0.5x_2+0.9x_3+1)\}}+\frac{0.1}{1+\exp \{-3(x_1+0.5x_2+0.9x_3-3)\}}.$$ (ii)

Table 1 summarizes the average estimates and empirical standard deviations of β by the proposed method (left panel), the method of Delaigle et al. (2014) (middle panel) and the logistic regression method (right panel). Under both scenarios, our proposed method clearly outperformed the method of Delaigle et al. (2014), with both smaller empirical biases and standard deviations, especially under scenarios with small (200 or 500) or moderate sample sizes (1000). For example, under scenario (ii) with a sample size of 200, the standard deviation of ${\widehat{\beta }}_2$ by Delaigle et al. (2014) was 2.25 times larger than that of the proposed method. When the sample size increased to 1000 or 10000, the efficiency gain of the proposed method over that of Delaigle et al. (2014) remained, with the ratio of the standard deviation of by ${\widehat{\beta }}_2$ Delaigle et al. (2014) over that of the proposed method being 2.28 and 1.74, respectively. Under Scenario (i), as expected, the logistic regression method outperformed the other two methods, since the logistic regression method used the information on the underlying link function (logistic curve), while the other two semiparametric methods did not. However, under Scenario (ii), the logistic regression method produced substantial biases, due to the misspecified model assumptions. For example, when N=1000, the bias of ${\widehat{\beta }}_3$ was 6.86 times larger than the corresponding empirical standard deviation, resulting in a misleading inference result.

Table 1:

Simulation results of the proposed method, Delaigle (2014) method and logistic regression under Scenario (i) and (ii). MEAN, empirical average; SD: empirical standard deviation.

				Proposed Method		Delaigle (2014)		Logistic
Scenario	N	PARA	true	MEAN	SD	MEAN	SD	MEAN	SD
(i)	200	β 2	0.5	0.487	0.990	0.585	1.673	0.449	0.864
		β 3	0.9	1.048	0.649	1.203	1.213	1.003	0.498
	500	β 2	0.5	0.537	0.604	0.582	0.961	0.561	0.562
		β 3	0.9	0.983	0.487	1.116	0.729	0.928	0.254
	1000	β 2	0.5	0.529	0.496	0.535	0.679	0.511	0.378
		β 3	0.9	0.985	0.366	1.009	0.450	0.923	0.171
	10000	β 2	0.5	0.491	0.165	0.503	0.165	0.496	0.122
		β 3	0.9	0.907	0.123	0.926	0.135	0.898	0.051
(ii)	200	β 2	0.5	0.541	1.001	0.629	2.253	0.158	0.899
		β 3	0.9	0.909	0.858	0.977	1.602	0.296	0.260
	500	β 2	0.5	0.514	0.822	0.449	1.884	0.191	0.525
		β 3	0.9	1.013	0.787	0.943	1.134	0.291	0.159
	1000	β 2	0.5	0.446	0.785	0.567	1.786	0.156	0.375
		β 3	0.9	1.016	0.645	1.201	0.955	0.279	0.105
	10000	β 2	0.5	0.527	0.252	0.585	0.437	0.162	0.114
		β 3	0.9	0.944	0.190	0.985	0.234	0.276	0.032

Figures 1 and 3 display the average estimated curves with 95% empirical percentile confidence intervals under two scenarios. For the proposed method, the empirical biases were relatively large at the right tail of the curves, but were in a reasonable range and decreased with increasing sample sizes. Under the setting of a sample size of 1000 or 10000, the estimated curves were close to the true curve, with indistinguishable biases under both scenarios. Compared with the two semiparametric methods, the logistic regression model was very sensitive to the underlying true link functions. In Figure 1, when true link function is the logistic function, the estimated curves from the logistic regression were very close to the true curves. However, when the true curves deviated from the logistic function (Figure 3), the logistic regression could not provide a reliable risk estimation due to the substantial bias in the estimated curves. Also, such biases did not decrease with increasing sample sizes. Regarding the variation comparison of the estimate curves, we only considered the two semiparametric methods for fair comparison. The estimated curves obtained by our proposed method had smaller variations than those by the method of Delaigle et al. (2014) for small or moderate sample sizes. Particularly, the estimated curves by Delaigle et al. (2014) has poor performance for the left tail (low risk region) of the curves due to data sparsity. Even when the sample size increased to 1000, the empirical confidence intervals for the low risk region were still substantially wider than those by the proposed method. Under the large sample size (N=10000), both methods had similar performance. Figures 2 and 4 illustrate the median, as well as the first and third quartiles of the 200 integrated squared errors obtained by the two methods, where the integrated squared error is defined as $\underset{a}{\overset{b}{\int }}{(\widehat{f}(t)-f(t))}^{2}dt$ and [a, b] is the x-range of the figure. Under the large sample size, (N=10000), the two methods had comparable performance; whereas our method outperformed the method of Delaigle et al. (2014) under the small sample size (N=200 or 500) and moderate sample size (N=1000).

Figure 1(Scenario 1):

True curves (black solid), estimated curves from logistic regression (red dash), estimated curves from the proposed method (blue dash), and the Delaigle (2014) method (purple dash) with 95% shaded confidence intervals. Subplot (1), (3), (5), and (7): proposed method and logistic regression with sample sizes of 200, 500, 1000, and 10000 respectively; Subplot (2), (4), (6), and (8): Delaigle method and logistic regression with sample sizes of 200, 500, 1000, and 10000, respectively.

Figure 3 (Scenario 2):

True curves (black solid), estimated curves from logistic regression (red dash), estimated curves from the proposed method (blue dash), and the Delaigle (2014) method (purple dash) with 95% shaded confidence intervals. Subplot (1), (3), (5), and (7): proposed method and logistic regression with sample sizes of 200, 500, 1000, and 10000, respectively; Subplot (2), (4), (6), and (8): Delaigle method and logistic regression with sample sizes of 200, 500, 1000, and 10000, respectively.

Figure 2(Scenario 1):

True curves (black solid) and estimated curves corresponding to the 50th (dotted), 25th (dashed), and 75th (dot, dash) percentile values of $\underset{\left[a,b\right]}{\int }|\widehat{v}(t)-v(t)|dt$ Subplot (1), (3), (5), and (7): Proposed method with sample sizes of 200, 500, 1000, and 10000, respectively; Subplot (2), (4), (6), and (8): Delaigle (2014) method with sample sizes of 200, 500, 1000, and 10000, respectively.

Figure 4 (Scenario 2):

True curves (black solid) and estimated curves corresponding to the 50th (dotted), 25th (dashed), and 75th (dot, dash) percentile values of $\underset{\left[a,b\right]}{\int }|\widehat{v}(t)-v(t)|dt$ Subplot (1), (3), (5), and (7): Proposed method with sample sizes of 200, 500, 1000, and 10000, respectively; Subplot (2), (4), (6), and (8): Delaigle (2014) method with sample sizes of 200, 500, 1000, and 10000, respectively.

We conducted simulation studies to evaluate the finite sample performance of the proposed method using data with a group size of 5. Table S2 and Figure S2 in the online supplementary materials summarize the simulation results. Overall, we observed similar simulation results, regardless of the group size.

In summary, the simulation studies suggest that the proposed method performs well for the simultaneous estimation of the conditional probability curve and covariate effects. Our method needs prior knowledge that the underlying probability curve is monotonic, which is a commonly used assumption in studies for deriving risk scores. When the underlying probability curve is monotonic and the sample size is small or moderate, our proposed method yields a substantial gain in statistical efficiency for estimating regression coefficients especially compared with the method of Delaigle et al. (2014), which was designed for a more general case without the monotonic constraint. In application, the two methods can provide valuable complementary information for robust estimation and improved efficiency.

5. APPLICATION

We used the same data as those in Delaigle and Meister (2011) and Delaigle et al. (2014) to illustrate the proposed method. NHANES is a large nutrition and health study to assess the health and nutritional status of adults and children in the United States. The survey has been conducted as a series of questionnaires (Zipf et al., 2013). Details of the design and content of each survey are available at .e used the data NHANES 1999-2000, which can be downloaded from the website: ,ith a cutoff date of 11/15/2017. Although the data from NHANES are not grouped, they are ideal for illustrating the effect of data grouping on the estimation precision of parameters of interest. The data and computational codes with instructions are available in the Supplementary Materials.

Following the paper of Delaigle et al. (2014), we considered two covariates X = {X₁, X₂}, where X₁ is the subject’s age and X₂ is the subject’s total cholesterol level measured in a unit of 100 mg/dL. We aimed to estimate the probability of having the antibody to the hepatitis B virus core antigen (Y = 1) given the information from the two covariates. After removing the subjects with missing values, the sample size of the analytic data was 7064. Among those subjects, 339 subjects (4.8%) had the antibody. The median age was 27 years, with a range of (6-85 years), and the median total cholesterol level was 1.83 (100 mg/dL), with a range of (0.72-5.75). We applied isotonic regression with PAVA to the full cohort as the reference standard for the estimated parameters and estimated the standard error by using the bootstrap method with 200 re-samplings. We then grouped the data randomly using a group size of 2 and repeated the grouping process 200 times for estimating the standard errors. For each of the 200 randomly grouped data, we conducted the proposed method and the method of Delaigle et al. (2014) to estimate the portability P(Y = 1|x) assuming a linear combination of the two covariates. Given the age effect of 1, the estimated coefficients of the cholesterol level given by the three methods were very close, and the associated inferences were consistent: 4.30 with a standard error (DE) of 1.86 by using the full data, 4.49 with a SE of 2.14 by the proposed method using the grouped data, and 4.50 with a SE of 2.26 by the method of Delaigle et al. (2014). As expected, the method using the full cohort had the smallest standard error and the method of Delaigle et al. (2014) had the largest standard error. Figure 5 presents the estimated probability curves obtained by using the full data set and grouping the data with our proposed estimation method. As shown in the figure, the estimated probability curve obtained by using the grouped data was close to the one obtained by using the full cohort, although it was not as smooth. The bootstrap confidence intervals obtained by using the grouped data were almost identical to those obtained by using the full cohort except for the right tail, which suggests that group testing did not lose much statistical efficiency for the probability curve.

Figure 5:

Estimated probability curve of having antibody to hepatitis B virus core antigen from NHANES data.

Here we have included bootstrap standard errors of the estimated coefficients and 95% bootstrap percentile confidence intervals of the estimated curves for the full cohort and grouping data to explore the information loss due to the grouping. Note that we only have an n^1/3 convergence rate for the estimated probability curve, and the asymptotic behavior of the bootstrap standard errors and confidence intervals is not clear in our setting. The simulation studies in the online supplementary materials suggest that the finite sample performance of the bootstrap standard errors and confidence intervals have a reasonable performance in our setting.

6. DISCUSSION

Group testing is a cost-effective sampling strategy to reduce the time and labor in large screening studies. In this paper, we have proposed a computationally appealing algorithm that integrates PAVA with the EM algorithm for fitting semiparametric isotonic regression models for group testing data. Although the estimating algorithm involves iterative steps, it is computationally simple and efficient. The conditional expectations in the E-step can be simply obtained with closed forms. In the M-step, the non-specified monotonic function can be estimated easily by using available R function “Isoreg” in the R basic package or R packages “ISO” and “Isotone.” Then the regression coefficients can be estimate by the “optim” function in R. Our simulation studies suggest that the R function Isoreg in the R basic package is sufficient in our setting, and our method is computationally faster than Delaigle et al.’s method regardless of the sample size. For example, in a 200-run simulation under scenario (ii) using a 3.20 GHz desktop CPU, the CPU run times of the proposed method and Delaigle et al.’s method were 0.17 and 0.77 hours for a sample size of 500, and 35.19 and 115.15 hours for a sample size of 10000, respectively.

One advantage of the PAVA algorithm is that it is statistically and computationally efficient. Alternatively, the monotone spline smoothing method can be used to estimate the unspecified link function with the monotonic constraint (He and Shi, 1998; Leitenstorfer and Tutz, 2006), in which the number of knots or the integrated smoothness parameter need extra efforts to be estimated. Given the binary outcome, our simulation study using the R package SCAM indicated that the non-convergence of the monotone spline smoothing with the integrated smoothness parameter is not negligible in some settings. A thorough theoretic and empirical comparison of the two methods is beyond the scope of this paper and worth of further research. Also, although the nonparametric maximum likelihood estimator $\widehat{\pi }(·)$ does not need additional specifications, such as the kernel function and bandwidth, it is a non-smooth step function and has a cubic root n consistency. By using the techniques in Mukerjee (1988) and Groeneboom and Jongbloed (2014), we can generalize the PAVA estimator to a kernel smoothed PAVA estimator for an improved convergence rate, for which we would require additional computation. Further developing a computational procedure to facilitate the corresponding bandwidth section is a worthy objective for future research.

One potential limitation of the EM algorithm is that the algorithm may converge to a local maxima of the likelihood function, depending on the starting values. In our study, to avoid this issue, we chose 10 sets of initial values from a uniform distribution, and selected the maximizers over the 10 resulting estimators. Alternatively, the estimated regression coefficients from logistic regression could be used as resalable starting points for the proposed methods. We compared the simulation results, obtained by using the estimators from the logistic regression as the initial values to those obtained by using 10 different sets of initial values. The results were almost identical.

APPENDIX

Proof of Theorem 1.

Without loss of generality, we assume the group size n_j to be 2 and n = N/2 for notation simplicity. Let f(o|β, π) = P(y*|x₁, x₂, β, π)g(x₁)g(x₂) as given in Section 2.1. Let $({\widehat{\beta }}_n,{\widehat{\pi }}_n)$ be the MLE and (β₀, π₀) be the true parameter, and

$$\text{q}(\mathit{o}|\beta ,r,\Lambda )\text{=log}\frac{f(\mathit{o}|\mathit{\beta },\pi )}{f(\mathit{o}|{\mathit{\beta }}_0,{\pi }_0)}\text{=log}\frac{\text{P}(y^{*}|{\mathit{x}}_1,{\mathit{x}}_2,\mathit{\beta },\pi )}{\text{P}(y^{*}|{\mathit{x}}_1,{\mathit{x}}_2;{\mathit{\beta }}_0,{\pi }_0)}$$

be the log-likelihood ratio. Let P be the probability measure of f(·|β₀, π₀), Pq(O|β, π) = ∫q(o|β, π)dP(o) the true mean of q; P_n be the empirical measure based on the data $\{{\mathit{o}}_i=(y_i^{*},{\mathit{x}}_{1,i},{\mathit{x}}_{2,i}):i=1,\dots ,n\},$ and $P_{n}q(\mathit{O}|\mathit{\beta },\pi )=n^{-1}{\sum }_{i=1}^{n}q({\mathit{o}}_i|\mathit{\beta },\pi )$, the empirical mean of q.

Note that Pq(O|β, π) is the negative Kullback-Leibler divergence of f(o|β, π) from f(o|β₀, π₀) as a function of (β, π) it is always non-positive, attaining its maximum value of 0 at (β₀, π₀) and the model is identifiable, so we have (β₀, π₀) = arg max_{(β, π)∈(B, Π)} Pq(O|β, π). Also, recall that $(\widehat{\mathit{\beta }},\widehat{\pi })$ is the MLE, so $(\widehat{\mathit{\beta }},\widehat{\pi })={\mathrm{arg\; max}}_{(\mathit{\beta },\pi )\in (B,\Pi )}{P}_{n}q(\mathit{O}|\mathit{\beta },\pi )$.

Let d(β, π; β₀, π₀) = ‖β − β₀‖ + sup_t∈R |π(t) − π₀(t)|. By our model specification, Pq(O|β, π) is continuous with respect to (β, π); also identifiability of the model implies (β₀, π₀) is the unique maximizer of Pq(O|β, π). Thus for all η > 0, sup_{(β, π)∈(B,Π):d(β, π;β0, π0)>η} Pq(O|β, π) < Pq(O|β₀, π₀) and by definition of MLE, $P_{n}q(\mathit{O}|\widehat{\mathit{\beta }},\widehat{\pi })\ge P_{n}q(\mathit{O}|{\mathit{\beta }}_0,{\pi }_0)$, so if we show that $\mathcal{Q}=\{q(\mathit{o}|\mathit{\beta },\pi ):(\mathit{\beta },\pi )\in (\mathit{B},\Pi )\}$ is a P-Glivenko-Cantelli class, then by Theorem 5.8 in van der Vaart (2002, p.386), $d(\widehat{\mathit{\beta }},\widehat{\pi };{\mathit{\beta }}_0,{\pi }_0)\stackrel{a.s.}{\to }0$.

Now we show that $\mathcal{Q}$ is P-Glivenko-Cantelli. For any function g, let ‖g‖_L1(P) = ∫|g(o)|dP(o) and $N_{[]}(ϵ,\mathcal{Q},L_1(P))$ be the minimum number of e-brackets needed to cover $\mathcal{Q}$ under the norm ‖·‖_L1(P). We first show that ${\text{N}}_{[]}(ϵ,Q,L_1(P))$ is finite ∀ϵ > 0. By our model specification,

$$\begin{gathered} \ell (\mathit{\beta },\pi |\mathit{o}):=\log P(y^{*}|{\mathit{x}}_1,{\mathit{x}}_2,\mathit{\beta },\pi ) \\ =y^{*}\log \left(1-\prod _{i=1}^2\left[1-\pi (v({\mathit{x}}_i,\mathit{\beta }))\right]\right)+(1-y^{*})\sum _{i=1}^2\log \left[1-\pi (v({\mathit{x}}_i,\mathit{\beta }))\right] \end{gathered}$$

is differentiable in (β, π) where differentiability for π is in the Hadamard sense. Denote ${\dot{\ell }}_{\beta }(\mathit{\beta },\pi |\mathit{o})=\partial \ell (\mathit{\beta },\pi |\mathit{o})/\partial \mathit{\beta }$ and ${\dot{\ell }}_{\pi }(\mathit{\beta },\pi |\mathit{o})\left[h\right]$ be the Hadamard derivative of ℓ(β, π|o) with respect to π, in the direction h. Then ∀ (β_k, π_k) ∈ (B, Π) (k = 1, 2),

$$\begin{gathered} |q(\mathit{o}|{\mathit{\beta }}_2,{\pi }_2)-q(\mathit{o}|{\mathit{\beta }}_1,{\pi }_1)|=|\ell ({\mathit{\beta }}_2,{\pi }_2|\mathit{o})-\ell ({\mathit{\beta }}_1,{\pi }_1|\mathit{o})| \\ \le |{\dot{\ell }}_{\beta }^T(\mathit{\beta },\pi |\mathit{o})({\mathit{\beta }}_2-{\mathit{\beta }}_1)|+|{\dot{\ell }}_{\pi }(\mathit{\beta },\pi |\mathit{o})\left[{\pi }_2-{\pi }_1\right]|, \end{gathered}$$

where (β, π) is an intermediate point between (β₁, π₁) and (β₂, π₂) Note that

$$\begin{gathered} {\dot{\ell }}_{\beta }(\mathit{\beta },\pi |\mathit{o})=-(1-y^{*})\sum _{i=1}^2\frac{\dot{\pi }(\upsilon ({\mathit{x}}_i,\mathit{\beta }))\dot{\upsilon }({\mathit{x}}_i,\mathit{\beta })}{1-\pi (\upsilon ({\mathit{x}}_i,\mathit{\beta }))} \\ +y^{*}\frac{{\sum }_{i=1}^2\left[1-\pi (\upsilon ({\mathit{x}}_i,\mathit{\beta }))\right]\dot{\pi }(\upsilon ({\mathit{x}}_{3-i},\mathit{\beta }))\dot{\upsilon }({\mathit{x}}_{3-i},\mathit{\beta })}{1-{\prod }_{i=1}^2\left[1-\pi (\upsilon ({\mathit{x}}_i,\mathit{\beta }))\right]} \end{gathered}$$

$$\begin{gathered} {\dot{\ell }}_{\pi }(\mathit{\beta },\pi |\mathit{o})\left[{\pi }_2-{\pi }_1\right]=-(1-y^{*})\sum _{i=1}^2\frac{{\pi }_2(\upsilon ({\mathit{x}}_i,\mathit{\beta }))-{\pi }_1(\upsilon ({\mathit{x}}_i,\mathit{\beta }))}{1-\pi (\upsilon ({\mathit{x}}_i,\mathit{\beta }))} \\ +y^{*}\frac{{\sum }_{i=1}^2\left[1-\pi (\upsilon ({\mathit{x}}_i,\mathit{\beta }))\right]({\pi }_2(\upsilon ({\mathit{x}}_{3-i},\mathit{\beta }))-{\pi }_1(\upsilon ({\mathit{x}}_{3-i},\mathit{\beta })))}{1-{\prod }_{i=1}^2\left[1-\pi (\upsilon ({\mathit{x}}_i,\mathit{\beta }))\right]}. \end{gathered}$$

Then by the boundedness of π(·), π₁(·), π₂(·), (C2) and Hölder’s inequality,

$$\begin{gathered} {\Vert {\dot{\ell }}_{\beta }(\mathit{\beta },\pi |·)({\mathit{\beta }}_2-{\mathit{\beta }}_1)\Vert }_{L_1(P)}= \\ {E}_{({\beta }_0,{\pi }_0)}|(-P(Y^{*}=0|\mathit{X})\sum _{i=1}^2\frac{\dot{\pi }(v({\mathit{X}}_i,\mathit{\beta })){\dot{v}}^T({\mathit{X}}_i,\mathit{\beta })}{1-\pi (v({\mathit{X}}_i,\mathit{\beta }))} \\ +P(Y^{*}=1|\mathit{X})\frac{{\sum }_{i=1}^2\left[1-\pi (v({\mathit{X}}_i,\beta ))\right]\dot{\pi }(v({\mathit{X}}_{3-i},\mathit{\beta })){\dot{v}}^T({\mathit{X}}_{3-i},\mathit{\beta })}{1-{\prod }_{i=1}^2\left[1-\pi (v({\mathit{X}}_i,\mathit{\beta }))\right]})({\mathit{\beta }}_2-{\mathit{\beta }}_1)| \\ =E_{({\beta }_0,{\pi }_0)}|(-\prod _{i=1}^2\frac{1-{\pi }_0(v({\mathit{X}}_i,{\mathit{\beta }}_0))}{1-\pi (v({\mathit{X}}_i,\mathit{\beta }))})\sum _{i=1}^2\dot{\pi }(v({\mathit{X}}_i,\mathit{\beta })){\dot{v}}^T({\mathit{X}}_i,\mathit{\beta }) \\ +\frac{1-{\prod }_{i=1}^2\left[1-{\pi }_0(v({\mathit{X}}_i,{\mathit{\beta }}_0))\right]}{1-{\prod }_{i=1}^2\left[1-\pi (v({\mathit{X}}_i,\mathit{\beta }))\right]} \\ \times \sum _{i=1}^2\left[1-\pi (v({\mathit{X}}_i,\mathit{\beta }))\right]\dot{\pi }(v({\mathit{X}}_{3-i},\mathit{\beta })){\dot{v}}^T(X_{3-i},\mathit{\beta })({\mathit{\beta }}_2-{\beta }_1)| \\ =E_{({\beta }_0,{\pi }_0)}\left[H_1^T(\mathit{X}|\mathit{\beta },\pi )({\mathit{\beta }}_2-{\mathit{\beta }}_1)\right] \\ \le {(E_{({\beta }_0,{\pi }_0)}{\Vert H_1(\mathit{X}|\mathit{\beta },\pi )\Vert }^2)}^{1/2}{(E_{({\beta }_0,{\pi }_0)}{\Vert {\mathit{\beta }}_2-{\mathit{\beta }}_1\Vert }^2)}^{1/2}\le C_1{\Vert {\mathit{\beta }}_2-{\mathit{\beta }}_1\Vert }_{L_2(P)}. \end{gathered}$$

Similarly, for some 0 < C₂ < ∞, ${\Vert {\dot{\ell }}_{\pi }(\mathit{\beta },\pi |·)\left[{\pi }_2-{\pi }_1\right]\Vert }_{L_1(P)}$

$$\le {(E_{({\beta }_0,{\pi }_0)}{H}_2^2(\mathit{X}|\mathit{\beta },\pi ))}^{1/2}{(E_{({\beta }_0,{\pi }_0)}{\Vert {\pi }_2-{\pi }_1\Vert }^2)}^{1/2}\le C_2{\Vert {\pi }_2-{\pi }_1\Vert }_{L_2(P)}.$$

Thus, ‖q(·|β₂, π₂) − q(·|β₁, π₁)‖_L1(P) ≤ C₁‖β₂ − β₁‖_L2(P) + C₂‖π₂ − π₁‖_L2(P), and so, ∀ ϵ > 0, $N_{[]}(ϵ,\mathcal{Q},{\Vert ·\Vert }_{L_1(P)})\le N_{[]}(\frac{ϵ}{2C_1},\mathit{B},{\Vert ·\Vert }_{L_2(P)})N_{[]}(\frac{ϵ}{2C_2},\Pi ,{\Vert ·\Vert }_{L_2(P)})$.

By (C1), N_[](ϵ/(2C₁), B, ‖·‖_L2(P)) = N_[](ϵ/(2C₁), B, ‖·‖) = O(1/ϵ^d), with d = dim(β) Since Π is a collection of bounded monotone functions, by Theorem 2.7.5 in van der Vaart and Wellner (1996, p.159), there is constant 0 < C < ∞, for all P and r > 0,

$$N_{[]}(\frac{ϵ}{2C_2},\Pi ,{\Vert ·\Vert }_{L_r(P)})\le \exp \{\frac{C}{\in }\},\forall \in >0.$$

Thus, for some generic constant 0 < C < ∞, ∀ ϵ > 0,

$$N_{[]}(ϵ,\mathcal{Q},{\Vert \cdot \Vert }_{L_1(P)})\le N_{[]}(\frac{ϵ}{2C_1},\mathit{B},\Vert \cdot \Vert )N_{[]}(\frac{ϵ}{2C_2},\Pi ,L_2(P))\le \frac{C}{{ϵ}^d}\exp \{\frac{2C}{ϵ}\}<\infty ,$$

and so by Theorem 2.4.1 in van der Vaart and Wellner (1996, p.122), $\mathcal{Q}$ is a Glivenko-Cantelli class with respect to P.

Proof of Theorem 2.

Let q(o|β, π) and P_n be defined in the proof of Theorem 1. Define ${\mathbb{M}}_n(\mathit{\beta },\pi )=P_{n}q(\mathit{O}|\mathit{\beta },\pi )$ and $\mathbb{M}(\mathit{\beta },\pi )=Pq(\mathit{O}|\mathit{\beta },\pi )$. Then ${\mathbb{M}}_n(\widehat{\mathit{\beta }},\widehat{\pi })\ge {\mathbb{M}}_n(\mathit{\beta },\pi )\ge {\mathbb{M}}_n(\mathit{\beta },\pi )-O_p(r_n^{-2})$, for any (β, π) ∈ (B, Π) and any positive sequence r_n → ∞.

Since conditions of Theorem 1 are satisfied, $d(\widehat{\beta },\widehat{\pi };{\beta }_0,{\pi }_0)\stackrel{P}{\to }0$. Note that $({\beta }_0,{\pi }_0)={\mathrm{arg\; sup}}_{(\beta ,\pi )\in (B,\Pi )}\mathbb{M}(\beta ,\pi )$. Denote ${\dot{\ell }}_{\pi }({\mathit{\beta }}_0,{\pi }_0|\mathit{o})$, ${\dot{\ell }}_{\pi }({\mathit{\beta }}_0,{\pi }_0|\mathit{o})\left[\pi -{\pi }_0\right]$, ${\ddot{\ell }}_{\beta \beta }({\mathit{\beta }}_0,{\pi }_0|\mathit{o})$, ${\ddot{\ell }}_{\beta ,\pi }({\mathit{\beta }}_0,{\pi }_0|\mathit{o})\left[\pi -{\pi }_0\right]$, ${\ddot{\ell }}_{\pi ,\pi }({\mathit{\beta }}_0,{\pi }_0|\mathit{o})\left[\pi -{\pi }_0,\pi -{\pi }_0\right]$, etc. as the first and second order partial derivatives of ℓ(β, π|o) with respect to (β, π). The derivatives with respect to π are in the Hadamard sense. By our specification of the model, these quantities exist. Note that $E_{({\beta }_0,{\pi }_0)}{\dot{\ell }}_{\beta }({\mathit{\beta }}_0,{\pi }_0|\mathit{O})=\mathbf{0}$ and $E_{({\beta }_0,{\pi }_0)}{\dot{\ell }}_{\pi }({\mathit{\beta }}_0,{\pi }_0|\mathit{O})\left[\pi -{\pi }_0\right]=0$. Denote $\ddot{Q}(\mathit{\beta },\pi |\mathit{o})$ as the matrix of all the second order partial derivatives, then

$$\mathbb{M}(\mathit{\beta },\pi )-\mathbb{M}({\mathit{\beta }}_0,{\pi }_0)=\frac{1}{2}{E}_{({\beta }_0,{\pi }_0)}\left({(\mathit{\beta }-{\mathit{\beta }}_0,1)}^T\ddot{Q}(\overline{\mathit{\beta }},\overline{\pi }|\mathit{O})(\mathit{\beta }-{\mathit{\beta }}_0,1)\right)<0$$

and is of order O(d²(β, π; β₀, π₀)), where $(\overline{\mathit{\beta }},\overline{\pi })$ is an intermediate value between (β, π) and (β₀, π₀). Under (C4), the above can be upper bounded by, −Cd²(β, π; β₀, π₀) for some 0 < C < ∞, in the small neighborhood of (β₀, π₀). So for any and any 0 < η_n → 0 and any τ with η_n < τ ≤ η < ∞, for some 0 < C < ∞,

$$\underset{\tau \text{/2<d}(\mathit{\beta },\pi ;{\mathit{\beta }}_0,{\pi }_0)\le \tau ,(\mathit{\beta },\pi )\in (B,\Pi )}{\sup }\mathbb{M}(\mathit{\beta },\pi )-\mathbb{M}({\mathit{\beta }}_0,{\pi }_0)\le -C{\tau }^2.$$

Next we show, with E* for outer expectation,

$$E^{*}\underset{\tau \text{/2<d}(\mathit{\beta },\pi ;{\mathit{\beta }}_0,{\pi }_0,)\le \tau ,(\mathit{\beta },\pi )\in (B,\Pi )}{\sup }\sqrt{n}\left|({\mathbb{M}}_n-\mathbb{M})(\mathit{\beta },\pi )-({\mathbb{M}}_n-\mathbb{M})({\mathit{\beta }}_0,{\pi }_0)\right|\le {\varphi }_n(\tau ),$$ (A.0)

for some decreasing function ϕ_n(·) to be given.

In the proof of Theorem 1, we showed N_[](ϵ, (B, Π), L₂(P)) = O(1/ϵ^d exp{C/ϵ}) = O(exp{C/ϵ}). Similarly, N_[](ϵ, (B, Π), L₄(P)) = O(exp{C/ϵ}). Let C be some generic finite positive constant, we have

$$\begin{gathered} J_{[]}(\tau ,(\mathit{B},\Pi ),L_4(P)):=\underset{0}{\overset{\tau }{\int }}\sqrt{1+\log N_{[]}(ϵ,(\mathit{B},\Pi ),L_4(P))}dϵ \\ =\underset{-\infty }{\overset{\log \tau }{\int }}{e}^t\sqrt{1+C{e}^{-t}}dt\le C\underset{-\infty }{\overset{\log \tau }{\int }}{e}^t{e}^{-t/2}dt=C{\tau }^{1/2}. \end{gathered}$$

In the above we used the fact that for small τ > 0, log τ < 0, so e^−t > 1 and 1 + Ce^−t ≤ (1 + C)e^−t on (−∞, log τ)

Let $\mathcal{Q}(\tau )=\{q(·|\mathit{\beta },\pi ):(\mathit{\beta },\pi )\in (\mathit{B},\Pi ),\tau /2<d(\mathit{\beta },\pi ;{\mathit{\beta }}_0,{\pi }_0)\le \tau \}$, ${\mathbb{G}}_{n}q=\sqrt{n}(P_n-P)q$, ${\Vert {\mathbb{G}}_n\Vert }_{\mathcal{Q}(\tau )}={\sup }_{q\in \mathcal{Q}(\tau )}|{\mathbb{G}}_{n}q|$. Since $\mathcal{Q}(\tau )\subset \mathcal{Q}$ (given in the proof of Theorem 1), so $N_{[]}(ϵ,\mathcal{Q}(\tau ),L_2(P))\le N_{[]}(ϵ,\mathcal{Q}(\tau ),L_2(P))$. In the proof of Theorem 1, we showed $N_{[]}(ϵ,\mathcal{Q}(\tau ),L_1(P))\le N_{[]}(ϵ/C,(\mathit{B},\Pi ),L_2(P))$. by the same way, $N_{[]}(ϵ,\mathcal{Q}(\tau ),L_2(P))\le N_{[]}(ϵ/C,(\mathit{B},\Pi ),L_4(P))$, for some 0 < C < ∞, and so $J_{[]}(\tau ,\mathcal{Q}(\tau ),L_2(P))\le J_{[]}(\tau ,(\mathit{B},\Pi ),L_4(P))\le C{\tau }^{1/2}$.

Also, as in the proof of Theorem 1, with our specification of the likelihood, Pq² < Cτ² and ‖q‖_∞ < C for all $q\in \mathcal{Q}(\tau )$, for some 0 < C < ∞. Thus, by Lemma 3.4.2 in van der Vaart and Wellner (1996, p.324),

$$\begin{gathered} {\text{E}}^{*}{\Vert {\mathbb{G}}_n\Vert }_{\mathcal{Q}(\tau )}\le C{J}_{[]}(\tau ,\mathcal{Q}(\tau ),L_2(P))\left(1+\frac{C{J}_{[]}(\tau ,\mathcal{Q}(\tau ),L_2(P))}{{\tau }^2\sqrt{n}}C\right) \\ \le \text{C}{\tau }^{1/2}(1+{\tau }^{-3/2}{n}^{-1/2}), \end{gathered}$$

which implies (A.0) with ϕ_n(τ) = Cτ^1/2(1 + τ^−3/2n^−1/2) Take r_n = n^1/3 then

$$r_n^2{\varphi }_n\left(\frac{1}{r_n}\right)=Cn{}^{1/3}\le \sqrt{n}.$$

Now, all conditions of Theorem 3.4.1 in van der Vaart and Wellner (1996, p.322) are satisfied, so by this Theorem, $r_{n}d(\widehat{\mathit{\beta }},\widehat{\pi };{\mathit{\beta }}_0,{\pi }_0)=n^{1/3}d(\widehat{\mathit{\beta }},\widehat{\pi };{\mathit{\beta }}_0,{\pi }_0)=O_p(1)$.

There should be just one appendix, for proofs and longer mathematical arguments. These proofs are in the following form: