Comment on “Dynamic treatment regimes: technical challenges and applications”

Abstract

Inference for parameters associated with optimal dynamic treatment regimes is challenging as these estimators are nonregular when there are non-responders to treatments. In this discussion, we comment on three aspects of alleviating this nonregularity. We first discuss an alternative approach for smoothing the quality functions. We then discuss some further details on our existing work to identify non-responders through penalization. Third, we propose a clinically meaningful value assessment whose estimator does not suffer from nonregularity.

1. Introduction

The authors are to be congratulated for their excellent and thoughtful paper on statistical inference for dynamic treatment regimens. They have addressed several important and long-standing issues in this area. As discussed by the authors, nonsmoothness of the problem in some of the parameters of interest leads to estimators that are not smooth in the data. This in turn makes inference for these parameters challenging. In the following, we comment on a few additional strategies to alleviate the resulting nonregularity due to nonsmoothness. First, we discuss replacing the nonsmooth objective functions via a SoftMax Q-learning approach, which directly addresses the trade-off between bias and variance of the maximum operation in the local asymptotic framework. Proofs are given in the appendix.

Nonregularity of the estimators for the parameters associated with the optimal treatment regimes is mainly due to the existence of non-responders to treatments. Therefore, it would be useful and important if we could identify these non-responders. In the second part, we review our existing work on non-responder identification via penalization. We also discuss how this penalization can alleviate, although not solve, some regularity issues.

For the third and final aspect we wish to discuss, we note that in some public health settings, the parameters in the dynamic treatment regime are not as important as the value function which reflects the overall population impact of the estimated regime and is perhaps the most important quantity to focus on for public health policy. We propose a truncated value function which only focuses on those subjects who are expected to have large treatment effects. We claim that this alternative value function is clinically meaningful and does not suffer from nonregularity.

2. SoftMax Q-Learning

In this section we study the effect of replacing the max operator with a smoother version of it in the two-stage Q-learning algorithm discussed by Laber et al. We show that this smoothing can reduce the bias and can be controlled under local alternatives. The proposed SoftMax approach also sheds light on the bias/variance tradeoff which can be obtained by using over/under smoothing. In what follows, we briefly describe the SoftMax Q-learning algorithm, and then present some theoretical and simulation results.

2.1. Proposed Algorithm

Consider the Q-learning algorithm discussed by Laber et al. in Section 2. In step 2 of the algorithm, the stage outcome is predicted by

$$\stackrel{\sim }{Y}=\underset{a_2}{max}{Q}_2(H_2,a_2;{\widehat{\beta }}_2).$$

We propose replacing Ỹ with a SoftMax version of it. Define the SoftMax function by (see Fig 1)

Fig 1.

The function log{exp(x)+exp(0)} in blue and the function max(x, 0) in red. Note that the functions roughly agree for x ∉ [−3, 3].

$$\mathit{SoftMax}(x,y,\alpha )=\frac{1}{\alpha }log\{e^{\alpha x}+e^{\alpha y}\},\alpha >0.$$

Let

$$\begin{array}{l}\stackrel{⌣}{Y}=\mathit{SoftMax}\left(Q_2(H_2,a_{2,1};{\widehat{\beta }}_2),Q_2(H_2,a_{2,2};{\widehat{\beta }}_2)\right)\\ =\frac{1}{\alpha }log\left\{e^{\alpha H_{2,0}^{\prime }{\widehat{\beta }}_{2,0}}+e^{\alpha \left(H_{2,0}^{\prime }{\widehat{\beta }}_{2,0}+H_{2,1}^{\prime }{\widehat{\beta }}_{2,1}\right)}\right\}\\ =H_{2,0}^{\prime }{\widehat{\beta }}_{2,0}+\frac{1}{\alpha }log\left\{1+e^{\alpha H_{2,1}^{\prime }{\stackrel{⌣}{\beta }}_{2,1}}\right\}.\end{array}$$

The estimator β̂₁ of β₁ is given by ${\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{P}}_n{B}_1\stackrel{⌣}{Y}$. We note that the algorithm discussed by Laber et al. is obtained as the limit, as α goes to infinity, of the SoftMax Q-learning algorithm discussed here.

2.2. Theory

In the following we briefly discuss the asymptotic properties of β̂₁. We first discuss the limiting distribution of $\sqrt{n}({\widehat{\beta }}_1-{\beta }_1^{\ast })$. We then discuss this limiting distribution under local alternatives. Finally, we discuss the asymptotic bias. The proofs appear in the Appendix.

Theorem 1

Assume (A1)–(A2) from Laber et al., and let α_n → ∞ such that $\sqrt{n}/{\alpha }_n\to a_{\infty }$ for a_∞ ∈ [0, ∞). then

1. If a_∞ = 0,

   $$\sqrt{n}({\widehat{\beta }}_1-{\beta }_1^{\ast })⇝{\mathbb{S}}_{\infty }+{\mathrm{\sum }}_{1,\infty }^{-1}P({\mathbb{T}}_{\infty }).$$
2. If 0 < a_∞ < ∞, then

   $$\sqrt{n}({\widehat{\beta }}_1-{\beta }_1^{\ast })\to {\mathbb{S}}_{\infty }+{\mathrm{\sum }}_{1,\infty }^{-1}P({\mathbb{T}}_{\infty })+a_{\infty }log(2){\mathrm{\sum }}_{1,\infty }^{-1}P{B}_1\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\},$$
   where

   $${\mathbb{T}}_{\infty }=B_1\left(H_{2,1}^{\prime }{\mathbb{V}}_{\infty }\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+\frac{1}{2}{H}_{2,1}^{\prime }{\mathbb{V}}_{\infty }\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\}\right).$$

For local alternatives the limiting distribution is given below.

Theorem 2

Assume (A1)–(A3) from Laber et al., and let α_n → ∞ such that $\sqrt{n}/{\alpha }_n\to a_{\infty }$ for a_∞ ∈ (0; ∞). Then

$$\sqrt{n}({\widehat{\beta }}_1-{\beta }_1^{\ast })⇝{\mathbb{S}}_{\infty }+{\mathrm{\sum }}_{1,\infty }^{-1}P({\mathbb{T}}_{\infty })+{\mathrm{\sum }}_{1,\infty }^{-1}P({\mathbb{W}}_{\infty }).$$

where

$$\begin{array}{r}{\mathbb{T}}_{\infty }=B_1\left(H_{2,1}^{\prime }{\mathbb{V}}_{\infty }\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+H_{2,1}^{\prime }{\mathbb{V}}_{\infty }{[a_{\infty }^{-1}{H}_{2,1}^{\prime }s]}_{+}\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\}\right)\\ {\mathbb{W}}_{\infty }=B_1\left(a_{\infty }log\left\{1+e^{a_{\infty }^{-1}{H}_{2,1}^{\prime }s}\right\}-{[H_{2,1}^{\prime }s]}_{+}\right)\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\}.\end{array}$$

The bound of the bias, scaled by root-n, under both standard and local alternatives asymptotics, is given below.

Corollary 1

Assume (A1)–(A2) from Laber et al., and let α_n → ∞ such that $\sqrt{n}/{\alpha }_n\to a_{\infty }$ for a_∞ ∈ (0, ∞). Fix c ∈ ℝ^p₂₁. Then

$$\text{Bias}({\widehat{\beta }}_1,c)\le a_{\infty }\Vert {\mathrm{\sum }}_{1,\infty }^{-1}\Vert P\Vert B\Vert \mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}+o_p(1).$$

When (A3) from Laber et al. also holds, then

$$\underset{s\in {ℝ}^{p_{21}}}{sup}\text{Bias}({\widehat{\beta }}_1,c,s)\le a_{\infty }\Vert {\mathrm{\sum }}_{1,\infty }^{-1}\Vert P\Vert B\Vert \mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}+o_p(1).$$

The above results show that by choosing the scale of α, the bias can be controlled. Theorem 2 shows that this control of the bias directly influences the variance, at least under local alternatives.

For inference, we need to discuss two different settings. When holding α fixed, as n goes to infinity, standard inference for the parameters is valid, as the problem becomes regular. However, this comes with the price that the bias does not vanish even asymptotically (see also the discussion in Section 4 ). As proved in Theorem 2, when taking α to infinity, as n goes to infinity, the problem is nonregular. Thus, adaptive confidence intervals, such as the one suggested by Laber et al., are needed in order to perform valid inference.

2.3. Simulations for SoftMax

We compare the small-sample behaviour of SoftMax to that of soft-threshholding using the example setting discussed in Laber et al., Section 3. Let ${\theta }^{\ast }=max({\mu }_0^{\ast },{\mu }_1^{\ast })$. The max estimator is defined by

$$\widehat{\theta }\equiv max({\widehat{\mu }}_0,{\widehat{\mu }}_1)=\frac{{\widehat{\mu }}_0+{\widehat{\mu }}_1}{2}+\frac{\mid {\widehat{\mu }}_1-{\widehat{\mu }}_1\mid }{2}.$$

A soft-thresholdling estimator is defined by

$${\widehat{\theta }}^{\sigma }=\frac{{\widehat{\mu }}_0+{\widehat{\mu }}_1}{2}+\frac{\mid {\widehat{\mu }}_0-{\widehat{\mu }}_1\mid }{2}\left(1-\frac{4\sigma }{n({\widehat{\mu }}_0-{\widehat{\mu }}_1)}\right).$$

Finally, the SoftMax estimator is defined by

$$\stackrel{⌣}{\theta }=\mathit{SoftMax}({\widehat{\mu }}_0,{\widehat{\mu }}_1,\alpha ).$$

Let Y |A ~ N(μ_a, 1), a = 0, 1, and assume that the treatment assignment is perfectly balanced. We use 1000 Monte Carlo replicates to estimate the bias for each parameter setting. Figure 2 below shows the bias as a function of the treatment effect ${\mu }_1^{\ast }-{\mu }_0^{\ast }$ and with tuning parameters σ ∈ [0, 5] and α ∈ [1, 6] for the soft-thresholding and SoftMax, respectively. It appears that the SoftMax does not suffer from large bias on points away from ${\mu }_1^{\ast }-{\mu }_0^{\ast }=0$. Also, as expected from Theorem 1, the bias decreases as α increases.

Fig 2.

Left: Bias for soft-thresholding. Right: Bias for SoftMax. In both panels the bias is measured in units of $1/\sqrt{n}$ for n = 10, as a function of effect size and of the tuning parameters, σ and α, for the soft-thresholding and SoftMax, respectively.

3. Penalized and Adaptive Q-Learning

In Penalized Q-learning (Song et al., 2011 ) and adaptive Q-learning (Goldberg et al., 2013 ), penalties were imposed on the term $H_{2,1}^{\prime }{\beta }_{2,1}$ for each individual. This use of penalized estimation allows us to simultaneously estimate the second stage parameters and select individuals whose value functions are not affected by treatments, i.e., those individuals whose true values of $H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }$ are zero. Although the penalized method does not solve the non-regularity issue in estimating β’s, our numerical studies have demonstrated that penalized Q-learning is not only able to reduce bias, but also provides better coverage of confidence intervals in a number of scenarios, as compared to the hard thresholding method of Moodie and Richardson (2010) and some soft thresholding methods including resampling approaches. Furthermore, the inference approach for penalized methods described in Zhang and (Zhang 2014) appears to be able to handle diverging model perturbations. Finally, a nice feature of our penalized learning is that it enables us to identify non-responders, who may also have small treatment benefits even under a local alternative. Since it is clinically and practically most useful to target groups whose treatment benefit is large, identifying those subjects with small treatment benefits is useful for better allocation of resources and for reducing costs.

4. Truncated Value Function

The non-regularity issue arises primarily in settings where there are some subjects who do not respond to treatments at the second stage and where inference focuses on effect size. In the context of public health policy, we think that (1) the overall benefit (value) may be of greater interest compared to individual effect sizes and (2) those subjects who are not sensitive to treatments (approximate non-responders) should not have a large impact on the overall decision making process. Thus, we propose an appropriate alternative criterion, namely the ε-truncated value, for evaluating the optimal policy as follows:

$$V_{\epsilon }(d_1,d_2)=E_d[(Y_1(d_1)+Y_2(d_2))I(\delta (X_1)>\epsilon ,\delta (X_2)>\epsilon )],$$

where δ(X₁) and δ(X₂) denote the expected treatment effects at the first and second stages respectively. Here, ε is a small constant indicating a clinically meaningful effect size.

Under a SMART trial with randomization probabilities π_k at stage k (k = 1, 2), this truncated value is equal to

$$E[(Y_1+Y_2)I(A_1=d_1(X_1),A_2=d_2(X_2),\delta (X_1)>\epsilon ,\delta (X_2)>\epsilon )/({\pi }_1{\pi }_2)].$$

Compared to the usual value function, we can see that V_ε(d₁, d₂) differs by at most O(ε). Using the Q-learning model, the above value function for the estimated rule is

$$V_{\epsilon }({\widehat{d}}_1,{\widehat{d}}_2)=E\left[(Y_1+Y_2)I(A_1{\widehat{\beta }}_1{X}_1>0,A_2{\widehat{\beta }}_2{X}_2>0,\mid {\widehat{\beta }}_1{X}_1\mid >\epsilon ,\mid {\widehat{\beta }}_2{X}_2\mid >\epsilon )/({\pi }_1{\pi }_2)\right].$$

One advantage of considering this value function is that non-regularity will no longer be an issue since we have excluded the non-responders from the above statistic. One can easily show $\sqrt{n}({\widehat{V}}_{\epsilon }({\widehat{d}}_1,{\widehat{d}}_2)-V_{\epsilon }(d_1,d_2))$ converges to the same normal distribution under local alternatives whether P(β₂X₂ = 0) > 0 or not.

5. Concluding Remarks

We again thank the authors for their very interesting work which likely stimulate additional future research on this crucial topic. It is clear that there are many fundamental and unresolved computational, methodological and theoretical challenges remaining which will benefit from many diverse problem solving approaches. We look forward to seeing this intriguing research area continue to develop.

Appendix A: Proofs

Sketch of proof of Theorem 1

Using the same arguments that lead to Eq. 2 in Laber et al., we have

$$\sqrt{n}({\widehat{\beta }}_1-{\beta }_1^{\ast })={\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{P}}_n{B}_1\left(\stackrel{⌣}{Y}-B_1^{\prime }{\beta }_1^{\ast }\right)={\mathbb{S}}_n+{\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{P}}_n{B}_1{\mathbb{U}}_n,$$

where is smooth and asymptotically normal and

$${\mathbb{U}}_n=\sqrt{n}\left(\frac{1}{{\alpha }_n}log\left\{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\widehat{\beta }}_{2,1}}\right\}-{\left[H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\right]}_{+}\right).$$

Note that

$$\begin{array}{l}{\mathbb{U}}_n=\sqrt{n}\left(\frac{1}{{\alpha }_n}log\left\{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\widehat{\beta }}_{2,1}}\right\}-\frac{1}{{\alpha }_n}log\left\{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}\right\}\right)+\sqrt{n}\left(\frac{1}{{\alpha }_n}log\left\{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}\right\}-{[H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }]}_{+}\right)\\ =\frac{e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}{H}_{2,1}^{\prime }}{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}}\sqrt{n}\left({\widehat{\beta }}_{2,1}-{\beta }_{2,1}^{\ast }\right)+o_P\left(\sqrt{n}\left({\widehat{\beta }}_{2,1}-{\beta }_{2,1}^{\ast }\right)\right)+\sqrt{n}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }),\end{array}$$

where the last equality follows by taking derivatives, and where

$$f(\alpha ,x)=\frac{1}{\alpha }log\{1+e^{\alpha x}\}-max(x,0).$$ (1)

For the remainder term, note that Lemma B.6 shows the consistency of β̂_2,1, and that the expectation of the Hessian of $\frac{1}{{\alpha }_n}log\left\{1+e^{{\alpha }_n{H}^{\prime }\beta }\right\}$ is bounded by Assumption (A1). Hence, by applying Lemma B.5 to the matrix Σ̂₁ that appears in the remainder term, we conclude that

$$\sqrt{n}({\widehat{\beta }}_1-{\beta }_1^{\ast })={\mathbb{S}}_n+{\mathbb{T}}_n+{\mathbb{W}}_n+o_P(1),$$ (2)

where

$$\begin{array}{l}{\mathbb{T}}_n={\mathrm{\sum }}_1^{-1}{\mathbb{P}}_n{B}_1\left[\frac{e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}}{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}}{H}_{2,1}^{\prime }\sqrt{n}\left({\widehat{\beta }}_{2,1}-{\beta }_{2,1}^{\ast }\right)\right],\\ {\mathbb{W}}_n={\widehat{\mathrm{\sum }}}_1^{-1}\sqrt{n}{\mathbb{P}}_n{B}_{1}f\left({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\right).\end{array}$$

Recall that by assumption, α_n and thus fact that

$$\frac{e^{{\alpha }_{n}x}}{1+e^{{\alpha }_{n}x}}\to \{\begin{array}{cc}1& x>0\\ \frac{1}{2}& x=0\\ 0& x<0\end{array},$$

we obtain that for a given h_2,1,

$$\frac{e^{{\alpha }_n{h}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}}{1+e^{{\alpha }_n{h}_{2,1}^{\prime }{\beta }_{2,1}^{\ast }}}\to \mathbf{1}\{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+\frac{1}{2}\mathbf{1}\{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\}.$$

Define the function w: D_p1 × l ^∞( ) × ℝ^p₂₁ × [0, 1] ↦ ℝ^p₁ by w(Σ, μ, ν, a) = Σ⁻¹μ(g(ν, B₁, H_2,1, a)), where

$$g(\nu ,b_1,h_{2,1},a)=\{\begin{array}{cc}{b}_1\left[\frac{e^{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }/a}}{1+e^{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }/a}}{h}_{2,1}^{\prime }\nu \right],& a>0\\ b_1[(\mathbf{1}\{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+\frac{1}{2}\mathbf{1}\{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\})h_{2,1}^{\prime }\nu ],& a=0\end{array}$$

and where = {g(ν, b₁, h_2,1, a), ||ν|| ≤ K}. Using the same arguments as those used in Lemma B.11, one can show that w is continuous at (Σ_1,∞, P, ℝ^p₂₁, 0). Thus, using the continuous mapping theorem, it can be shown that

$$\begin{array}{l}{\mathbb{S}}_n+{\mathbb{T}}_n⇝{\mathrm{\sum }}_{1,\infty }^1\left[{\mathbb{G}}_{\infty }\left(B_1(H_{2,0}^{\prime }{\beta }_{2,0}^{\ast }+{[H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }]}_{+}-B_1^{\prime }{\beta }_1^{\ast })\right)\right]\\ +{\mathrm{\sum }}_{1,\infty }^{-1}\left[P{B}_1\left(H_{2,0}^{\prime }{\mathbb{Z}}_{\infty ,0}+H_{2,1}^{\prime }{\mathbb{Z}}_{\infty ,1}(\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+\frac{1}{2}\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\})\right)\right],\end{array}$$

where

$${({\mathbb{Z}}_{\infty ,0}^{\prime },{\mathbb{Z}}_{\infty ,1}^{\prime })}^{\prime }={\mathrm{\sum }}_{2,\infty }^{-1}[B_2(Y-B_2^{\prime }{\beta }_2^{\ast })].$$

We now discuss , the third term in (3). By Lemma 1 (i) below, when $\sqrt{n}/{\alpha }_n\to 0$,

$$\Vert {\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{W}}_n\Vert \le \sqrt{n}{\mathbb{P}}_n\Vert {\widehat{\mathrm{\sum }}}_1^{-1}\Vert \Vert B_1\Vert f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast })\le \frac{\sqrt{n}log(2)}{{\alpha }_n}{\mathbb{P}}_n\Vert {\widehat{\mathrm{\sum }}}_1^{-1}\Vert \Vert B_1\Vert \to 0,$$

which proves (i).

For (ii), let δ_n → 0 such that α_nδ_n → ∞. Write

$$\begin{array}{l}{\widehat{\mathrm{\sum }}}_1{\mathbb{W}}_n=\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid >{\delta }_n\})\\ +\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid \le {\delta }_n\})\\ =\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid >{\delta }_n\})\\ +\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast })(\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid \le {\delta }_n\}-\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}))\\ +\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\})\\ \equiv A_n+B_n+C_n.\end{array}$$

Note that by Lemma 1 (vi),

$$\Vert A_n\Vert \le {\mathbb{P}}_n\Vert B_1\Vert \frac{\sqrt{n}}{{\alpha }_n}{e}^{-{\alpha }_n{\delta }_n}\stackrel{\mathrm{P}}{\to }0.$$

Let $p(\delta )=P(\mathbf{1}\{\mid h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid \le \delta \}-\mathbf{1}\{\mid h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\})$, and note that p(0) = 0 and thus p(δ) → 0 as δ → 0. Hence,

$$\Vert B_n\Vert \le {\mathbb{P}}_n\Vert B_n\Vert \frac{\sqrt{n}}{{\alpha }_n}log(2)\Vert (\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid \le {\delta }_n\}-\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\})\Vert \stackrel{\mathrm{P}}{\to }0.$$

Summarizing, we obtain that

$${\mathbb{W}}_n\stackrel{\mathrm{P}}{\to }{a}_{\infty }log(2){\mathrm{\sum }}_{1,\infty }^{-1}P{B}_1\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\},$$

which proves (ii).

Sketch of proof of Theorem 2

Using the same arguments that lead to Eq. 2 in Laber et al., we have

$$\sqrt{n}({\widehat{\beta }}_1-{\beta }_{1,n}^{\ast })={\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{P}}_n{B}_1\left(\stackrel{⌣}{Y}-B_1^{\prime }{\beta }_{1,n}^{\ast }\right)={\mathbb{S}}_n+{\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{P}}_n{B}_1{\mathbb{U}}_n,$$

where is smooth and asymptotically normal, and

$${\mathbb{U}}_n=\frac{e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}{H}_{2,1}^{\prime }}{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}}\sqrt{n}\left({\widehat{\beta }}_{2,1}-{\beta }_{2,1,n}^{\ast }\right)+o_P\left(\sqrt{n}\left({\widehat{\beta }}_{2,1}-{\beta }_{2,1,n}^{\ast }\right)\right)+\sqrt{n}\left(\frac{1}{{\alpha }_n}log\left\{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}\right\}-{[H_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }]}_{+}\right).$$

Similarly to the proof of Theorem 1, we have

$$\sqrt{n}({\widehat{\beta }}_1-{\beta }_{1,n}^{\ast })={\mathbb{S}}_n+{\mathbb{T}}_n+{\mathbb{W}}_n+o_P(1),$$ (3)

where

$$\begin{array}{l}{\mathbb{T}}_n={\widehat{\mathrm{\sum }}}_1^{-1}{\mathbb{P}}_n{B}_1\left[\frac{e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}}{1+e^{{\alpha }_n{H}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}}{H}_{2,1}^{\prime }\sqrt{n}\left({\widehat{\beta }}_{2,1}-{\beta }_{2,1}^{\ast }\right)\right],\\ {\mathbb{W}}_n={\widehat{\mathrm{\sum }}}_1^{-1}\sqrt{n}{\mathbb{P}}_n{B}_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }).\end{array}$$

We obtain that for a given h_2,1, and ${\beta }_{2,1,n}^{\ast }={\beta }_{2,1}^{\ast }+\frac{s}{\sqrt{n}}+o(1/\sqrt{n})$, and ${\alpha }_n/\sqrt{n}\to a_{\infty }^{-1}$,

$$\frac{e^{{\alpha }_n{h}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}}{1+e^{{\alpha }_n{h}_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast }}}\to \mathbf{1}\{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+{[a_{\infty }^{-1}{h}_{2,1}^{\prime }s]}_{+}\mathbf{1}\{h_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\}.$$

Using the same arguments given in the proof of Theorem 4.1 (see also proof of Theorem 1 above) it can be shown that + converges in distribution to

$$\begin{array}{l}{\mathrm{\sum }}_{1,\infty }^{-1}\{{\mathbb{G}}_{\infty }\left(B_1(H_{2,0}^{\prime }{\beta }_{2,0}^{\ast }+{[H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }]}_{+}-B_1^{\prime }{\beta }_1^{\ast })\right)\\ +P{B}_1\left(H_{2,0}^{\prime }{\mathbb{Z}}_{\infty ,0}+H_{2,1}^{\prime }{\mathbb{Z}}_{\infty ,1}(\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }>0\}+{[a_{\infty }^{-1}{H}_{2,1}^{\prime }s]}_{+}\mathbf{1}\{H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }=0\})\right)\},\end{array}$$

where

$${({\mathbb{Z}}_{\infty ,0}^{\prime },{\mathbb{Z}}_{\infty ,1}^{\prime })}^{\prime }={\mathrm{\sum }}_{2,\infty }^{-1}[B_2(Y-B_2^{\prime }{\beta }_2^{\ast })].$$

Note that

$$\begin{array}{l}{\widehat{\mathrm{\sum }}}_1{\mathbb{W}}_n=\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid >{\delta }_n,\mid {\alpha }_n^{-1}{H}_{2,1}^{\prime }s\mid >{\delta }_n/2\})\\ +\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid >{\delta }_n,\mid {\alpha }_n^{-1}{H}_{2,1}^{\prime }s\mid \le {\delta }_n/2\})\\ +\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast })(\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid \le {\delta }_n\}-\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}))\\ +\sqrt{n}{\mathbb{P}}_n(B_{1}f({\alpha }_n,H_{2,1}^{\prime }{\beta }_{2,1,n}^{\ast })\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\})\\ \equiv A_n+B_n+C_n+D_n.\end{array}$$

The first three terms can be bounded as follows:

$$\begin{array}{l}\Vert A_n+B_n+C_n\Vert \le {\mathbb{P}}_n\Vert B\Vert \frac{\sqrt{n}}{{\alpha }_n}log(2)\Vert \mathbf{1}\{\mid H_{2,1}^{\prime }s\mid >{\alpha }_n{\delta }_n/2\}\Vert +{\mathbb{P}}_n\Vert B\Vert \frac{\sqrt{n}}{{\alpha }_n}{e}^{-{\alpha }_n{\delta }_n/2}\\ +{\mathbb{P}}_n\Vert B\Vert \frac{\sqrt{n}}{{\alpha }_n}log(2)\Vert \mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid \le {\delta }_n\}-\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}\Vert \stackrel{\mathrm{P}}{\to }0.\end{array}$$

Thus the limiting distribution of depends on that of D_n. For D_n we have

$$\begin{array}{l}{D}_n={\mathbb{P}}_n{B}_1\left(\frac{\sqrt{n}}{{\alpha }_n}log\left\{1+e^{\frac{{\alpha }_n}{\sqrt{n}}{H}_{2,1}^{\prime }s+o(1/\sqrt{n})}\right\}-{[H_{2,1}^{\prime }s+o(1)]}_{+}\right)\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}\\ \stackrel{\mathrm{P}}{\to }P{B}_1\left(a_{\infty }log\left\{1+e^{a_{\infty }^{-1}{H}_{2,1}^{\prime }s}\right\}-{[H_{2,1}^{\prime }s]}_{+}\right)\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\},\end{array}$$

which concludes the proof.

Proof of Corollary 1

We prove only the second assertion, the first can be proved similarly. Noting that both and have mean zero, it is enough to bound the bias of

$${\mathrm{\sum }}_{1,\infty }^{-1}P({\mathbb{W}}_{\infty })={\mathrm{\sum }}_{1,\infty }^{-1}P{B}_1\left(a_{\infty }log\left\{1+e^{a_{\infty }^{-1}{H}_{2,1}^{\prime }s}\right\}-{[H_{2,1}^{\prime }s]}_{+}\right)\mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}.$$

Using Lemma 1 (i) with $\alpha =a_{\infty }^{-1}$ and $x=H_{2,1}^{\prime }s$, we obtain that the bias is bounded by

$$a_{\infty }log(2)\Vert {\mathrm{\sum }}_{1,\infty }^{-1}\Vert P\Vert B\Vert \mathbf{1}\{\mid H_{2,1}^{\prime }{\beta }_{2,1}^{\ast }\mid =0\}.$$

The following lemma is needed for the proofs of Theorems 1–2:

Lemma 1

Let $f(\alpha ,x)=\frac{1}{\alpha }log\{1+e^{\alpha x}\}-max(x,0)$. Then

1. 0 < f(α, x) ≤ log(2)/α.

2. argmax_x f (α, x) = 0 and f (α, 0) = log(2)/α.

3. f(α, x) → 0 as α → ∞.

4. Fix δ > 0. Then ${max}_{x\in (-\infty ,-\delta ]\cup [\delta ,\infty )}f(\alpha ,x)=\frac{e^{-\alpha \delta }}{\alpha }$.

The proof is technical and therefore omitted.