On Computation of Semi-Parametric Maximum Likelihood Estimators with Shape Constraints

Summary:

Large sample theory of semi-parametric models based on maximum likelihood estimation (MLE) with shape constraint on the non-parametric component is well studied. Relatively less attention has been paid to the computational aspect of semi-parametric MLE. The computation of semi-parametric MLE based on existing approaches such as the EM algorithm can be computationally prohibitive when the missing rate is high. In this paper, we propose a computational framework for semi-parametric MLE based on an inexact block coordinate ascent algorithm. We show theoretically that the proposed algorithm converges. This computational framework can be applied to a wide range of data with different structures, such as panel count data, interval-censored data and degradation data, among others. Simulation studies demonstrate favorable performance compared with existing algorithms in terms of accuracy and speed. Two data sets are used to illustrate the proposed computational method. We further implement the proposed computational method in R package BCA1SG, available at CRAN.

1. Introduction

Semi-parametric models usually impose shape constraints on the non-parametric component. This arises naturally in various applications. For instance, the cumulative function of a point process and the cumulative hazard function should be non-decreasing (Sun and Fang, 2003; Kosorok et al., 2004); in economics, the concavity constraint on utility functions follows from the law of diminishing marginal utility (Wu and Sickles, 2017); in engineering, as degradation is an irreversible process of cumulative damage to the product, the mean degradation function in a semi-parametric degradation model is restricted to be non-decreasing (Wang, 2010; Zhou and Xu, 2019).

In the maximum likelihood estimation (MLE) of semi-parametric models, the non-parametric component $\text{Λ}(\cdot )$ is usually estimated by a step function or piecewise linear function (Pan, 1999; Wellner and Zhang, 2007; Wang, 2010), where the knots depend on the observed data. For example, in panel count and interval-censored data, the knots are the set of distinct observation times. The number of knots can be large compared with the sample sizes as in Sun and Wei (2000), Gomez et al. (2000), Giorgio et al. (2012) and Chiou et al. (2018), among others. Another strategy is to approximate $\text{Λ}(\cdot )$ using splines (Lu et al., 2009; Zhang et al., 2010; Zhou et al., 2017) and convert the problem to estimation of the spline coefficients. Similarly, the number of spline coefficients increases with the sample size. Therefore, it is imperative to develop stable and computationally efficient algorithms for semi-parametric MLE (Fleming and Lin, 2000; Tsodikov, 2003; Kosorok, 2009).

For interval-censored and panel count data, Huang and Wellner (1997) and Wellner and Zhang (2007) proposed a doubly iterative algorithm which updates the regression coefficient β and non-parametric function $\text{Λ}(\cdot )$ through the Newton-Raphson and the iterative convex minorant (ICM) algorithm, respectively. As mentioned in Section 2 of Wellner and Zhang (2007), this algorithm is extremely time-consuming even with moderate sample size. This alternating strategy was also adopted in Sun and Shen (2009) and Dumbgen et al. (2011) for computing MLE in semi-parametric regression models. Another approach is to update both the parametric and non-parametric components simultaneously in each iteration. Towards this goal, Pan (1999) reformulated the ICM algorithm as a generalized gradient projection method and extended it to semi-parametric problems for interval-censored data. The algorithm was found to be time-consuming in Cheng et al. (2011), and they further extended it to a projected Newton-Raphson algorithm that makes use of the full Hessian matrix in each iteration. The algorithm takes only a few iterations to converge, yet it takes a long time to compute the full Hessian and its inverse matrix in each iteration, especially when the searching space in the non-parametric function is large.

As some semi-parametric problems can be cast into the missing data framework, the EM algorithm is constantly used for inference. By introducing latent Poisson variables, Wang et al. (2016) developed the EM algorithm for the spline-based semi-parametric proportional hazard model with interval-censored data. Zeng et al. (2016) used the EM algorithm for semi-parametric transformation models with interval-censored data. The algorithm converges slowly when the censoring intervals are wide resulting in high missing rate. For degradation data, Wang (2010), Ye et al. (2014) and Zhai and Ye (2018) developed the EM algorithm for different degradation process models by augmenting the possibly unobserved degradation of a product unit in all distinct observation times. The convergence speed is slow when the observation times are different for the product units, or when the true parameters are close to the boundary of the parameter space.

There are several consequences when an inefficient computational method is used. First, the optimization algorithm can terminate immaturely when common termination rules are used, leading to highly biased parameter estimation. Second, when the bootstrap is used for inference, a large number of replications is needed and the computational time can be infeasible. This is especially relevant in the big data era, where massive data sets, such as those collected from wearable devices in clinical trials and epidemiological studies, require scalable computational algorithms.

In this paper, we propose a new computational framework for MLE in semi-parametric models. Denote $\ell (\mathit{\beta },\mathbf{\Lambda })$ as the log-likelihood function of a general semi-parametric model, where β is the parametric component, and Λ is the vector consisting of all the parameters in estimating $\text{Λ}(\cdot )$. If $\text{Λ}(\cdot )$ is estimated by monotone splines, for example, Λ is the vector of all spline coefficients. It is common that $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ for fixed Λ is nonconcave. In these cases, simultaneously updating all parameters is unstable unless the initial value is close enough to the MLE. A natural remedy is to use the block coordinate ascent (BCA) that iteratively maximizes the log-likelihood function with respect to each of the two parts while keeping the other fixed. From observations in the simulation studies, it is a waste of time to exactly maximize $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ at the first few BCA iterations because β is still far from optimal and the exact maximization takes a non-negligible time. Motivated by this observation, we propose to update Λ by maximizing $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ inexactly at each BCA iteration.

In the gradient projection (Rosen, 1960), the search direction is obtained by projecting the gradient vector g onto the region defined by the active constraints. Borrowing the idea of Newton’s method, Jamshidian (2004) proposed the generalized Rosen (GR) algorithm that uses the generalized gradient $\tilde{\mathit{g}}=W^{-1}\mathit{g}$, where W is a positive definite matrix. Similar to Newton’s method versus the steepest descent, GR converges much faster than the original version if W is properly chosen. GR was used by Zhang and Jamshidian (2004) in computing the non-parametric MLE of the survival function with censored data, Lu et al. (2009) and Zhang et al. (2010) to compute spline-based semi-parametric MLE of panel count data and interval-censored data, and Lu (2010) to compute semi-parametric MLE in partial linear models. Based on the GR algorithm, we propose a new algorithm where the non-parametric vector Λ is updated by implementing only one GR iteration, and the parametric component β is updated through maximizing $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ exactly. The newly proposed algorithm is called the block coordinate ascent with one-step GR (BCA1SG). The one-step GR update significantly speeds up the algorithm in terms of run time. When $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ can be stably maximized and $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is concave, we show that the BCA1SG is convergent. The two conditions above are satisfied in many semi-parametric models, and thus the proposed computational framework has a wide spectrum of applications.

The rest of the paper is organized as follows. Section 2 provides some definitions in constrained optimization and briefly describes the general GR algorithm. In Section 3, we introduce the block coordinate ascent with one-step GR and show that it converges under mild conditions. We apply the proposed algorithm to panel count data, interval-censored data, and degradation data, respectively, in Section 4. Section 5 and Section 6 are devoted respectively to simulation studies and data analysis. We conclude the paper in Section 7. All proofs of main theorems and technical lemmas are collected in the Supporting Information.

2. Preliminaries

2.1. Some definitions in optimization

Consider the following constrained maximization problem that is common in likelihood inference of non-parametric statistical models:

$$\begin{array}{l}\text{maximize}\ell (\mathbf{\Lambda }),\\ \text{subject to}A\mathbf{\Lambda }⩾\mathit{b},\end{array}$$ (1)

where $A={\left({\mathit{a}}_1,{\mathit{a}}_2,\dots ,{\mathit{a}}_p\right)}^T\in {ℝ}^{p\times m}$ is of full row rank, and $\mathit{b}={\left(b_1,b_2,\dots ,b_p\right)}^T\in {ℝ}^p$. In (1), $\ell (\cdot )$ is the log-likelihood function, $\mathbf{\Lambda }\in {ℝ}^m$ is the vector consisting of all the parameters in estimating $\text{Λ}(\cdot )$, and $A\mathbf{\Lambda }⩾\mathit{b}$ represents shape constraints, such as nonnegativity or convexity, on Λ.

As a preliminary, some definitions in constrained optimization are first introduced as follows.

Definition 1:

A point $\mathbf{\Lambda }\in {ℝ}^m$ is feasible if it satisfies the constraint $A\mathbf{\Lambda }⩾\mathit{b}$.

Definition 2:

An inequality constraint ${\mathit{a}}_i^T\mathbf{\Lambda }⩾b_i$ is said to be active at a feasible point Λ if ${\mathit{a}}_i^T\mathbf{\Lambda }=b_i$, and inactive if ${\mathit{a}}_i^T\mathbf{\Lambda }>b_i$.

Definition 3:

A point ${\mathbf{\Lambda }}^{\ast }$ is called a stationary point of problem (1) if $\mathbf{\nabla }\ell {\left({\mathbf{\Lambda }}^{\ast }\right)}^T(\mathbf{\Lambda }-{\mathbf{\Lambda }}^{\ast })⩽0$ for any feasible Λ.

If $\tilde{\mathbf{\Lambda }}$ is a local maximizer of problem (1), then there exists $\mathit{\lambda }\in {ℝ}^p$, which is called the Lagrange multiplier vector, such that

• primal feasibility: $A\tilde{\mathbf{\Lambda }}⩾\mathit{b}$;

• stationarity: $\mathbf{\nabla }\ell (\tilde{\mathbf{\Lambda }})+A^T\mathit{\lambda }=\mathbf{0}$;

• complementary slackness: ${\lambda }_i\left({\mathit{a}}_i^T\tilde{\mathbf{\Lambda }}-b_i\right)=0$, $i=1,2,\dots ,p$;

• dual feasibility: ${\lambda }_i⩾0$, $i=1,2,\dots ,p$.

The above four conditions are called the Karush-Kuhn-Tucker (KKT) conditions (Luenberger and Ye, 2015, Section 11.8). They are necessary conditions for a point Λ to be the maximizer of problem (1).

2.2. The GR algorithm

GR is an active set algorithm developed to solve the constrained optimization problem displayed in (1) (Zhang and Jamshidian, 2004). Let ${\mathbf{\Lambda }}_0$ be the parameter value at the beginning of a GR iteration. Define $𝓐=\left\{i:{\mathit{a}}_i^T{\mathbf{\Lambda }}_0=b_i,i⩽p\right\}$ as the index set of all active constraints. GR first computes the gradient $\mathit{g}=\nabla \ell \left({\mathbf{\Lambda }}_0\right)$ and then the generalized gradient $\tilde{\mathit{g}}=W^{-1}\mathit{g}$ (Jamshidian and Jennrich, 1993), where W is a positive definite matrix that can be either fixed or adaptively updated at each iteration, and $\tilde{\mathit{g}}$ is the gradient of $\ell (\mathbf{\Lambda })$ evaluated at ${\mathbf{\Lambda }}_0$ in the metric induced by the norm ${\Vert \mathbf{\Lambda }\Vert }_W=\sqrt{{\mathbf{\Lambda }}^{T}W\mathbf{\Lambda }}$. When ${\mathbf{\nabla }}^2\ell \left({\mathbf{\Lambda }}_0\right)$ is negative definite, W is often specified as $-{\mathbf{\nabla }}^2\ell \left({\mathbf{\Lambda }}_0\right)$ to accelerate the convergence. To maintain the active constraints, GR projects $\tilde{\mathit{g}}$ onto $\left\{\mathit{d}:{\mathit{a}}_i^T\mathit{d}=0,i\in 𝓐\right\}$ in the metric of W and uses the projection as search direction. Define $A_{𝓐}$ as the matrix which only keeps the ith row of A for $i\in 𝓐$. Then the projection is given by $\tilde{\mathit{d}}=\left(I-W^{-1}{A}_{𝓐}^T{\left(A_{𝓐}{W}^{-1}{A}_{𝓐}^T\right)}^{-1}{A}_{𝓐}\right)\tilde{\mathit{g}}$ (Jamshidian, 2004). After that, compute the largest possible step length $\alpha =\underset{x>0}{\sup }\left\{x;A\left({\mathbf{\Lambda }}_0+x\tilde{\mathit{d}}\right)⩾\mathit{b}\right\}$. Then the step length is computed as $\rho =\arg \underset{0<x⩽\alpha }{\max }\ell \left({\mathbf{\Lambda }}_0+x\tilde{\mathit{d}}\right)$, and the new iterate is given by ${\mathbf{\Lambda }}_0+\rho \tilde{\mathit{d}}$. GR repeats the above procedure until $\tilde{\mathit{d}}=\mathbf{0}$, and the solution is denoted as $\tilde{\mathbf{\Lambda }}$.

Afterwards the four conditions in the KKT conditions are checked one by one. First, $\tilde{\mathbf{\Lambda }}$ obviously satisfies the primal feasibility. Second, because $\tilde{\mathit{d}}=\mathbf{0}$, we have

$$\mathbf{\nabla }\ell (\tilde{\mathbf{\Lambda }})-A_{𝓐}^T{\left(A_{𝓐}{W}^{-1}{A}_{𝓐}^T\right)}^{-1}{A}_{𝓐}{W}^{-1}\nabla \ell (\tilde{\mathbf{\Lambda }})=\mathbf{0}.$$ (2)

Define $I_{𝓐}$ as the matrix which only keeps the ith column of the p × p identity matrix for $i\in 𝓐$, and denote $\tilde{\mathit{\lambda }}=-I_{𝓐}{\left(A_{𝓐}{W}^{-1}{A}_{𝓐}^T\right)}^{-1}{A}_{𝓐}{W}^{-1}\mathbf{\nabla }\ell (\tilde{\mathbf{\Lambda }})$. Then equation (2) is equivalent to

$$\mathbf{\nabla }\ell (\tilde{\mathbf{\Lambda }})+A^T\tilde{\mathit{\lambda }}=\mathbf{0},$$

which means the stationarity condition holds, and $\tilde{\mathit{\lambda }}$ is the respective Lagrange multiplier vector. Third, it is easy to see that ${\tilde{\lambda }}_i=0$ if $i\notin 𝓐$, where ${\tilde{\lambda }}_i$ is the ith component of $\tilde{\mathit{\lambda }}$. On the other hand, ${\mathit{a}}_i^T\tilde{\mathbf{\Lambda }}-b_i=0$ holds for $i\in 𝓐$. Therefore,

$${\tilde{\lambda }}_i\left({\mathit{a}}_i^T\tilde{\mathbf{\Lambda }}-b_i\right)=0,i=1,2,\dots ,p,$$

which means the complementary slackness holds. Lastly, if min $(\tilde{\mathit{\lambda }})⩾0$, the dual feasibility holds and all the four conditions are satisfied. Otherwise, the index corresponding to the smallest Lagrange multiplier is removed from the index set $𝓐$ and the next iteration starts. The algorithm terminates when $\tilde{\mathit{d}}=\mathbf{0}$ and min $(\tilde{\mathit{\lambda }})⩾0$.

For the comprehensive description of the GR algorithm, see Zhang and Jamshidian (2004). Convergence of the active set algorithm was discussed in Section 12.3 in Luenberger and Ye (2015).

3. Block Coordinate Ascent with One-Step GR

Computing the MLE of a semi-parametric model usually requires solving

$$\begin{array}{r}\text{maximize}\ell (\mathit{\beta },\mathbf{\Lambda }),\\ \text{subject to}\mathit{\beta }\in \text{Θ},\\ A\mathbf{\Lambda }⩾\mathit{b},\end{array}$$ (3)

where $\text{Θ}$ is the parameter space for β, and the linear constraint on Λ usually results from the shape constraint on $\text{Λ}(\cdot )$. The linear constraint in (3) is generic because common shape constraints on Λ(·), such as nonnegativity, monotonicity (Zhang and Jamshidian, 2004), and concavity (Dumbgen et al., 2011), can be readily transformed into linear inequalities on Λ. When $\ell (\mathit{\beta },\mathbf{\Lambda })$ is jointly concave in β and Λ, the GR algorithm can be used to solve this problem by setting $-{\mathbf{\nabla }}^2\ell (\mathit{\beta },\mathbf{\Lambda })$. However, the joint concavity does not hold in many cases, such as the semi-parametric inverse Gaussian process model (Ye and Chen, 2014). Under such scenarios, the selection of the matrix W becomes tricky, which limits the application of the GR algorithm.

A close investigation into the log-likelihood reveals that $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is usually concave. As a result, $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ can be readily maximized by the GR algorithm. On the other hand, maximizing $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is almost the same as solving the MLE in the corresponding parametric models, and the maximization is usually well-studied. This motivates the use of BCA algorithm which alternates between the following two procedures until convergence.

• Keeping Λ fixed, update the parametric component by maximizing $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$;

• Keeping β fixed, update Λ by maximizing $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ through the GR algorithm.

According to our simulation experience, the value of β is usually far from optimal at first few BCA iterations. Given a poor estimate of β, the value of Λ which maximizes $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is very likely to be poor as well. Therefore, it is a waste of time to exactly maximize $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ at first few BCA iterations. In order to reduce the excessive runtime of a full GR, we propose the BCA1SG algorithm which runs only one iteration of the GR algorithm when updating Λ. The key difference between our method and the BCA algorithm is that BCA runs the GR algorithm until convergence when updating Λ while we only run one iteration.

A building block of the BCA1SG algorithm is a subroutine which will be introduced in Section 3.1. Before formally introducing the subroutine, we first briefly discuss how it works. The subroutine first partitions the constraints on Λ into two groups: those to be treated as active and those to be treated as inactive. As defined in Section 2.1, a constraint is called active if the equality holds. The constraints partitioned as inactive are preserved, while those partitioned as active are replaced by corresponding equality constraints. For example, if the constraint ${\mathit{a}}_1^T\mathbf{\Lambda }⩾b_1$ is partitioned as active, then it is replaced by ${\mathit{a}}_1^T\mathbf{\Lambda }=b_1$. Under these newly defined constraints, the subroutine then solves the maximization problem. Because the feasible set defined by these new constraints is a subset of the feasible set of problem (3), the solution of problem (3) may fall outside this subset. Hence the solution given by the subroutine may not be the solution of problem (3). However, a nice result which will be shown later is that the solution given by the subroutine satisfies all the KKT conditions for problem (3) except the dual feasibility. Therefore, to complete the BCA1SG algorithm, we further develop a procedure that ensures the dual feasibility in Section 3.2.

3.1. The Subroutine

Given initial values ${\mathit{\beta }}^{(0)}$, ${\mathbf{\Lambda }}^{(0)}$ and an initial index set of active constraints ${𝓐}^{(0)}$, the subroutine alternates between maximizing $\ell$ w.r.t. β exactly and Λ inexactly until the stopping criterion is satisfied. At the kth iteration of the subroutine, denote ${\mathit{\beta }}^{(k)}$, ${\mathbf{\Lambda }}^{(k)}$ as the parameter values, W^(k) the user-specified positive definite matrix, and ${𝓐}^{(k)}$ the index set. The subroutine is described in Algorithm 1, where the detailed procedures to conduct the backtracking line search in line 6 are given in the Supporting Information.

Algorithm 1.

The Subroutine

Input: ${\mathit{\beta }}^{(0)}$, ${\mathbf{\Lambda }}^{(0)}$, ${𝓐}^{(0)}$.
Output: $\tilde{\mathit{\beta }}$, $\tilde{\mathbf{\Lambda }}$, $\widetilde{𝓐}$, $\tilde{W}$.
1:	Initialization: k ← 0.
2:	while stopping criterion is not satisfied do
3:	Fix ${\mathbf{\Lambda }}^{(k)}$, and update ${\mathit{\beta }}^{(k+1)}=\underset{\mathit{\beta }\in \text{Θ}}{\arg \max }\ell \left(\mathit{\beta },{\mathbf{\Lambda }}^{(k)}\right)$.
4:	Evaluate ${\mathit{g}}^{(k)}={\nabla }_{\mathbf{\Lambda }}\ell \left({\beta }^{(k+1)},{\mathbf{\Lambda }}^{(k)}\right),W^{(k)}$ and ${\tilde{\mathit{g}}}^{(k)}={\left(W^{(k)}\right)}^{-1}{\mathit{g}}^{(k)}$.
5:	Compute the projection $${\mathit{d}}^{(k)}=\left[I-{\left(W^{(k)}\right)}^{-1}{A}_{{𝓐}^{(k)}}^T{\left\{A_{{𝓐}^{(k)}}{\left(W^{(k)}\right)}^{-1}{A}_{{𝓐}^{(k)}}^T\right\}}^{-1}{A}_{{𝓐}^{(k)}}\right]{\tilde{\mathit{g}}}^{(k)}.$$ (4)
6:	Determine the step length ${\rho }^{(k)}$ along ${\mathit{d}}^{(k)}$ through an inexact backtracking line search.
7:	Update ${\mathbf{\Lambda }}^{(k+1)}={\mathbf{\Lambda }}^{(k)}+{\rho }^{(k)}{\mathit{d}}^{(k)}$, and ${𝓐}^{(k+1)}=\left\{i:{\mathit{a}}_i^T{\mathbf{\Lambda }}^{(k+1)}=b_i\right\}$.
8:	$k\leftarrow k+1$.
9:	end while
10:	return ${\mathit{\beta }}^{(k)}$, ${\mathbf{\Lambda }}^{(k)}$, ${𝓐}^{(k)}$, $W^{(k)}$.

A good property of the subroutine which provides much convenience for discussing the convergence is that the sequence of the index sets $\left\{{𝓐}^{(k)}\right\}$ will eventually become invariant. The following lemma summarizes this property.

Lemma 1:

There exists an N > 0 such that ${\left\{{𝓐}^{(k)}\right\}}_{k>N}$ is a constant sequence.

Based on this result, we then discuss convergence properties of the sequence $\left\{\right({\mathit{\beta }}^{(k)},{\mathbf{\Lambda }}^{(k)}\left)\right\}$ generated by the subroutine. To establish the convergence result, the following assumptions are needed.

• (C1)The superlevel set $𝓢=\left\{(\mathit{\beta },\mathbf{\Lambda }):\ell (\mathit{\beta },\mathbf{\Lambda })⩾\ell \left({\mathit{\beta }}^{(0)},{\mathbf{\Lambda }}^{(0)}\right)\right\}$ is compact.
• (C2)For any $\mathit{\beta }\in \text{Θ}$, the function $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is concave.
• (C3)For any feasible Λ, the function $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ has a unique maximizer.
• (C4)There exist $0<L<U$ such that the eigenvalues of W^(k) are bounded in the interval [L, U] for all k.
• (C5)The set of stationary points of the following constrained optimization problem is discrete.

$$\begin{array}{l}\text{maximize}\ell (\mathit{\beta },\mathbf{\Lambda }),\\ \text{subject to}\mathit{\beta }\in \text{Θ},\\ {\mathit{a}}_i^T\mathbf{\Lambda }=b_i,i\in \widetilde{𝓐},\\ {\mathit{a}}_i^T\mathbf{\Lambda }⩾b_i,i\notin \widetilde{𝓐},\end{array}$$ (5)

where $\widetilde{𝓐}$ is the limit of $\left\{{𝓐}^{(k)},k⩾1\right\}$.

Condition (C1) is a commonly adopted assumption in optimization, e.g., the EM algorithm (Wu, 1983) and the coordinate descent (Tseng, 2001; Cassioli et al., 2013). Condition (C2) is satisfied by semi-parametric models such as the models for panel count data (Wellner and Zhang, 2007), interval-censored data (Huang and Wellner, 1997) and degradation data (Wang and Xu, 2010). A detailed proof of the concavity of $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ in these three semi-parametric models is given in the Supporting Information. Condition (C3) is required to guarantee the identifiablility of the corresponding parametric models, and it is satisfied in most semi-parametric models. If W^(k) is specified as the negative of the Hessian matrix, i.e., $W^{(k)}=-{\nabla }_{\mathbf{\Lambda }}^2\ell \left({\mathit{\beta }}^{(k+1)},{\mathbf{\Lambda }}^{(k)}\right)$, then condition (C4) can be replaced by the condition that $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is twice continuously differentiable and ${\mathbf{\nabla }}_{\mathbf{\Lambda }}^2\ell (\mathit{\beta },\mathbf{\Lambda })$ is negative definite for any feasible β. This condition holds for semi-parametric models, such as models for panel count data and degradation data. Condition (C5) is needed to prove that the sequence of iterates is convergent. A similar discrete assumption is also adopted in Wu (1983).

Theorem 1:

Suppose that conditions (C1)-(C5) hold. Then $\underset{k\to \infty }{\lim }{\mathit{d}}^{(k)}=0$ and the sequence $\left\{\left({\mathit{\beta }}^{(k)},{\mathbf{\Lambda }}^{(k)}\right)\right\}$ generated by the subroutine converges to a stationary point of problem (5).

Theorem 1 reveals that the output of the subroutine $(\tilde{\mathit{\beta }},\tilde{\mathbf{\Lambda }})$ is a stationary point of problem (5). However, whether it is a good solution for problem (3), which is of primary interest, is still unknown. In the next theorem, we show that $(\tilde{\mathit{\beta }},\tilde{\mathbf{\Lambda }})$ satisfies the stationarity in the KKT conditions for problem (3). To establish this result, we need two more assumptions as follows.

• (C6)For any feasible Λ, the maximizer of $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is an interior point of $\text{Θ}$.
• (C7)${\nabla }_{\mathit{\beta }}\ell (\mathit{\beta },\mathbf{\Lambda })$ is a continuous function in $(\mathit{\beta },\mathbf{\Lambda })$.

Condition (C6) holds in semi-parametric models when the parameter space $\text{Θ}$ for the parametric component β is open. Condition (C7) is mild and easily fulfilled. See the semiparametric models in Wellner and Zhang (2007), Ye and Chen (2014), and Zeng et al. (2016) for examples.

Theorem 2:

Suppose conditions (C1)-(C7) hold. Then $(\tilde{\mathit{\beta }},\tilde{\mathbf{\Lambda }})$ satisfies the stationarity in the KKT conditions for problem (3). The Lagrange multipliers associated with constraints on β are all zeros, and the Lagrange multiplier vector associated with constraints on Λ is

$$\tilde{\mathit{\lambda }}=-I_{\widetilde{𝓐}}{\left(A_{\widetilde{𝓐}}{\tilde{W}}^{-1}{A}_{\widetilde{𝓐}}^T\right)}^{-1}{A}_{\widetilde{𝓐}}{\tilde{W}}^{-1}{\nabla }_{\mathbf{\Lambda }}\ell (\tilde{\mathit{\beta }},\tilde{\mathbf{\Lambda }}).$$

Theorem 2 implies that the output of the subroutine satisfies the stationarity in the KKT conditions for problem (3). Besides, the primal feasibility obviously holds. Because the Lagrange multipliers associated with constraints on β and inactive constraints on Λ are all zeros, the complementary slackness is also satisfied. Therefore, it suffices to check whether $\tilde{\mathit{\lambda }}$ satisfies the dual feasibility in the next section.

3.2. The BCA1SG Algorithm

As noted, the Lagrange multipliers $\tilde{\mathit{\lambda }}$ associated with constraints on Λ may not be dual feasible. To improve on this, the BCA1SG algorithm alternates between the subroutine and the procedure that checks the dual feasibility until the dual feasibility holds.

Denote the Kth output of the subroutine as ${\tilde{\mathit{\beta }}}^{(k)}$, ${\tilde{\mathbf{\Lambda }}}^{(k)}$, ${\widetilde{𝓐}}^{(k)}$, ${\tilde{W}}^{(k)}$. Then the dual feasibility is checked through computing the Lagrange multiplier vector

$${\tilde{\mathit{\lambda }}}^{(K)}=-I_{{\widetilde{𝓐}}^{(K)}}{\left(A_{{\widetilde{𝓐}}^{(K)}}{\left({\tilde{W}}^{(K)}\right)}^{-1}{A}_{{\widetilde{𝓐}}^{(K)}}^T\right)}^{-1}{A}_{{\widetilde{𝓐}}^{(K)}}{\left({\tilde{W}}^{(K)}\right)}^{-1}{\nabla }_{\mathbf{\Lambda }}\ell \left({\tilde{\mathit{\beta }}}^{(K)},{\tilde{\mathbf{\Lambda }}}^{(K)}\right).$$ (6)

If $\min \left({\tilde{\mathit{\lambda }}}^{(K)}\right)⩾0$, the algorithm terminates and (${\tilde{\mathit{\beta }}}^{(K)}$, ${\tilde{\mathbf{\Lambda }}}^{(K)}$, ${\tilde{\mathit{\lambda }}}^{(K)}$) satisfies all the KKT conditions. Otherwise, drop the index corresponding to the smallest elements of ${\tilde{\mathit{\lambda }}}^{(K)}$ from ${\widetilde{𝓐}}^{(K)}$, and return ${\tilde{\mathit{\beta }}}^{(K)}$, ${\tilde{\mathbf{\Lambda }}}^{(K)}$, ${\widetilde{𝓐}}^{(K)}$, as the input for the subroutine to conduct the next iteration.

Write the subroutine in a functional form as $(\tilde{\mathit{\beta }},\tilde{\mathbf{\Lambda }},\widetilde{𝓐},\tilde{W})=\text{SUBROUTINE}(\beta ,\mathbf{\Lambda },𝓐)$, where ($\mathit{\beta }$, $\mathbf{\Lambda }$, $𝓐$) and ($\tilde{\mathit{\beta }}$, $\tilde{\mathbf{\Lambda }}$, $\widetilde{𝓐}$, $\tilde{W}$) are the input and output, respectively. The proposed BCA1SG algorithm is described in Algorithm 2.

In the following, we investigate when the BCA1SG algorithm terminates. According to Sard’s theorem for real-analytic functions (Souček and Souček, 1972), the sequence $\left\{\ell \left({\tilde{\mathit{\beta }}}^{(K)},{\tilde{\mathbf{\Lambda }}}^{(K)}\right)\right\}$ contains only finite distinct elements if $\ell (\mathit{\beta },\mathbf{\Lambda })$ is real-analytic. On the other hand, the algorithm strictly increases the objective function in each iteration. As a result, the BCA1SG algorithm always terminates in finite iterations. The following theorem summarizes this property.

Theorem 3:

Suppose conditions (C1)-(C7) hold and the function $\ell (\mathit{\beta },\mathbf{\Lambda })$ is real-analytic on an open set containing $\{(\mathit{\beta },\mathbf{\Lambda });\mathit{\beta }\in \text{Θ},A\mathbf{\Lambda }⩾\mathit{b}\}$. Then the BCA1SG algorithm is able to find the solution satisfying KKT conditions for problem (3) within finite iterations.

Theorem 3 ensures that there exists an N > 0 such that $\min \left({\tilde{\mathit{\lambda }}}^{(N)}\right)⩾0$, so the BCA1SG

Algorithm 2.

BCA1SG Algorithm

Input: ${\tilde{\mathit{\beta }}}^{(0)}$, ${\tilde{\mathbf{\Lambda }}}^{(0)}$, ${\widetilde{𝓐}}^{(0)}$.
1:	Initialization: K ← 0.
2:	$\left({\tilde{\mathit{\beta }}}^{(K+1)},{\tilde{\mathbf{\Lambda }}}^{(K+1)},{\widetilde{𝓐}}^{(K+1)},{\tilde{W}}^{(K+1)}\right)\leftarrow \text{SUBROUTINE}\left({\tilde{\mathit{\beta }}}^{(K)},{\tilde{\mathbf{\Lambda }}}^{(K)},{\widetilde{𝓐}}^{(K)}\right)$.
3:	Compute $\tilde{\mathit{\lambda }}(K+1)$ according to (6).
4:	if $\min \left({\tilde{\mathit{\lambda }}}^{(K+1)}\right)⩾0$ then
5:	Terminate the algorithm and output (${\tilde{\mathit{\beta }}}^{(K+1)}$, ${\tilde{\mathbf{\Lambda }}}^{(K+1)}$, ${\tilde{\mathit{\lambda }}}^{(K+1)}$).
6:	else
7:	Drop the index corresponding to the smallest elements of ${\tilde{\mathit{\lambda }}}^{(K+1)}$ from ${\widetilde{𝓐}}^{(K+1)}$.
8:	Let $K\leftarrow K+1$ and go back to step 2.
9:	end if
Output: (${\mathit{\beta }}^{\ast }$, ${\mathbf{\Lambda }}^{\ast }$, ${\lambda }^{\ast }$) that satisfies the KKT conditions.

algorithm always terminates within finite iterations. The importance of this theorem lies in the fact that the algorithm would not be trapped in an endless loop.

In addition to nice theoretical properties, the BCA1SG algorithm has many advantages in the computational perspective. For example, it avoids wasting too much time on a poor estimate of β, so the efficiency is significantly improved and the algorithm is less sensitive to initial values. Besides, very few numerical solutions are needed in this computational framework because almost all procedures in Algorithms 1 and 2 have closed forms except the maximization of $\mathit{\beta }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$, which depends on the specific form of log-likelihood functions. Therefore, the algorithm is efficient and easy to implement.

4. Applications of the BCA1SG Algorithm

This section demonstrates the applications of the BCA1SG algorithm to semi-parametric models for panel count data, interval-censored survival data and degradation data. For all the three models, the non-parametric component $\text{Λ}(\cdot )$ is estimated using a nondecreasing step function which only has jumps at $\left\{T_0<T_1<\cdots <T_m\right\}$, where T₀ = 0 and $\text{Λ}(0)=0$ by convention. For panel count data and degradation data, $\left\{T_1,\dots ,T_m\right\}$ represents the set of all distinct observation times; for “case k” interval-censored survival data, $\left\{T_1,\dots ,T_m\right\}$ stands for the set of all distinct examination (or monitoring) times. Since $\text{Λ}(\cdot )$ is only identifiable at $\left\{T_1,\dots ,T_m\right\}$, it suffices to estimate the m-dimensional vector ${\left(\text{Λ}\left(T_1\right),\dots \text{Λ}\left(T_m\right)\right)}^T$ subject to the monotone constraint $\text{Λ}\left(T_1\right)⩽\cdots ⩽\text{Λ}\left(T_m\right)$. For ease of computation, we transform ${\left(\text{Λ}\left(T_1\right),\dots \text{Λ}\left(T_m\right)\right)}^T$ to ${\left({\text{ΔΛ}}_1,\dots ,{\text{ΔΛ}}_m\right)}^T$, where ${\text{ΔΛ}}_l=\text{Λ}\left(T_l\right)-\text{Λ}\left(T_{l-1}\right)(l=1,\dots ,m)$. Then the monotone constraint becomes

$$I\mathbf{\Lambda }⩾\mathbf{0},$$

where I is an m × m identity matrix and $\mathbf{\Lambda }={\left({\text{ΔΛ}}_1,\dots ,{\text{ΔΛ}}_m\right)}^T$.

When implementing BCA1SG, we need to specify W^(k) for the subroutine. If ${\mathbf{\nabla }}_{\mathbf{\Lambda }}^2\ell (\mathit{\beta },\mathbf{\Lambda })$ is negative definite, a natural choice which usually leads to convergence in very few iterations is $-{\mathbf{\nabla }}_{\mathbf{\Lambda }}^2\ell \left({\mathit{\beta }}^{(k+1)},{\mathbf{\Lambda }}^{(k)}\right)$. When Λ is high-dimensional, however, computing the Hessian and its inverse consumes much time. As with the literature (Jongbloed, 1998; Zhang and Jamshidian, 2004), we let W^(k) be the diagonal matrix −D^(k) that discards the second-order mixed derivatives of the Hessian, i.e., ${\left[D^{(k)}\right]}_{ll}={\left[{\nabla }_{\mathbf{\Lambda }}^2\ell \left({\mathit{\beta }}^{(k+1)},{\mathbf{\Lambda }}^{(k)}\right)\right]}_{ll}(l=1,\dots ,m)$. If ${\mathbf{\nabla }}_{\mathbf{\Lambda }}^2\ell (\mathit{\beta },\mathbf{\Lambda })$ is only negative semi-definite, D^(k) may not be invertible. In such cases, we can adopt the Levenberg-Marquadt adjustment (Pan, 1999) by setting $W^{(k)}=-D^{(k)}+{ϵ}_{0}I$, where ${ϵ}_0$ is a suitably chosen small number. With such diagonal choices of W^(k), the projection in the subroutine, and the Lagrange mulitipliers in the main algorithm have neat expressions, as shown in the following properties.

Property 1:

When W^(k) is diagonal and Λ is subject to non-negativity constraints only, the projection d^(k) in the kth iteration of the subroutine can be obtained as follows

$$d_i^{(k)}=\{\begin{array}{cc}{\tilde{g}}_i^{(k)},& \mathit{\text{if}}i\notin {𝓐}^{(k)},\\ 0,& \mathit{\text{if}}i\in {𝓐}^{(k)},\end{array}$$

where $d_i^{(k)}$ and ${\tilde{g}}_i^{(k)}$ are the ith component of ${\mathit{d}}^{(k)}$ and ${\tilde{\mathit{g}}}^{(k)}$, respectively.

Property 2:

When ${\tilde{W}}^{(K)}$ is diagonal and Λ is subject to non-negativity constraints only, the Lagrange multipliers in the Kth iteration of the BCA1SG algorithm can be obtained as follows

$${\tilde{\lambda }}_i^{(K)}=\{\begin{array}{ll}-\frac{\partial \ell \left({\tilde{\mathit{\beta }}}^{(K)},{\tilde{\mathbf{\Lambda }}}^{(K)}\right)}{\partial {\text{ΔΛ}}_i},& \mathit{\text{if}}i\in {\widetilde{𝓐}}^{(K)},\\ 0,& \mathit{\text{if}}i\notin {\widetilde{𝓐}}^{(K)},\end{array}$$

where ${\tilde{\lambda }}_i^{(K)}$ is the ith component of ${\tilde{\mathit{\lambda }}}^{(K)}$.

Based on the above discussion, the BCA1SG algorithm can be readily applied to semiparametric models for panel count data, interval-censored survival data, and degradation data. Implementation details for the three semi-parametric models are given in the Supporting Information.

5. Simulation Study

Comprehensive simulation is used to examine the performance of the proposed algorithm on the semi-parametric models for panel count data, interval-censored survival data, and degradation data. The results on the model for panel count data are presented in this section, and the results on the other two models can be found in the Supporting Information. In the simulation, the BCA1SG subroutine as well as the other algorithms considered in this section are terminated when the infinity norm of the difference between successive iterates is less than the threshold . In BCA1SG, we set $W^{(k)}=-D^{(k)}+{10}^{-8}I$. The simulations are repeated 1,000 times in all the three subsections. All the computations are performed on a server with 32 cores (Intel Xeon CPU 2.60 GHz) using R.

5.1. Nonhomogeneous Poisson Process

Consider a nonhomogeneous Poisson process (NHPP) $\{\mathbb{N}(t),t⩾0\}$ with mean cumulative function $E(\mathbb{N}(t)\mid \mathit{Z})=e^{{\mathit{\beta }}^T\mathit{Z}}\text{Λ}(t)$, where $\mathit{Z}\in {ℝ}^d$ is the covariate vector, β is the regression coefficient, and $\text{Λ}(t)$ is a nondecreasing baseline mean cumulative function. There are three covariates, Z₁ ∼ Uniform(0, 1), $Z_2\sim N(0,1)$ and Z₃ ∼ Bernouli(0.5). The true values of the baseline mean cumulative function and the regression coefficients are $\text{Λ}(t)\equiv t^{3/4}$ and $\mathit{\beta }\equiv {\left({\beta }_1,{\beta }_2,{\beta }_3\right)}^T={(0.5,0.5,1.5)}^T$, respectively. The observation process is independent of the counting process $\{\mathbb{N}(t),t⩾0\}$ of interest after conditioning on the covariate Z. This observation scheme falls into the second scenario (“random sampling”) of the four observation scenarios listed in Section 3 of Gruger et al. (1991). Specifically, the number of observation times is randomly generated from {26,...,35}, and the observation times are randomly chosen from {1,...,100}. We examine two sample sizes n = 50,100. Both BCA1SG and the doubly iterative algorithm in Wellner and Zhang (2007) are used to compute the semi-parametric MLE. As suggested by Wellner and Zhang (2007), we first use the maximum pseudo likelihood estimates as initial values. The simulation results based on the 1,000 Monte Carlo replications are summarized in Figure 1 and Table 1. As can be seen, BCA1SG is faster and the estimates are more accurate. It is interesting to see that the runtime of BCA1SG decreases when the sample size increases, because the maximum pseudo likelihood estimates get closer to the MLE. To examine the sensitivity to the initial values, we rerun the simulation by setting $\left({\beta }_1^{(0)},{\beta }_2^{(0)},{\beta }_3^{(0)}\right)=(0,0,0),{\text{ΔΛ}}_l^{(0)}=r\left(T_l-T_{l-1}\right)(l=1,\dots ,m)$ where r ∼ Uniform(1, 5). The results are displayed in Table 1. Again, BCA1SG produces more accurate estimates, and it is insensitive to the initial values. On the other hand, the doubly iterative algorithm becomes very time-consuming, which agrees with the observations in Wellner and Zhang (2007).

Figure 1.

Pointwise biases (left panel) and MSEs (right panel) of the ML estimate of Λ(t) under difference sample sizes for the NHPP model with MPLE as initial value and $ϵ={10}^{-6}$.

Table 1.

The number of iterations, time cost in seconds, averaged ${\Vert \widehat{\text{Λ}}(t)-\text{Λ}(t)\Vert }_{\infty }$, and RMSE of $\widehat{\mathit{\beta }}$ for the NHPP model under different settings of initial values when $ϵ={10}^{-6}$, $\left({\beta }_1,{\beta }_2,{\beta }_3\right)=(0.5,0.5,1.5)$ and $\text{Λ}(t)=t^{3/4}$. The standard errors are shown in parentheses

		n = 50		n = 100
		BCA1SG	DIA	BCA1SG	DIA
MPLE as initial value	Iteration	160.9 (64.0)	225.2 (613.6)	81.6 (31.7)	122.2 (243.2)
	Time cost	10.6 (5.0)	87.4 (112.6)	9.7 (4.0)	100.6 (130.7)
	${\Vert \widehat{\text{Λ}}(t)-\text{Λ}(t)\Vert }_{\infty }$	1.16 (0.73)	1.25 (0.78)	0.82 (0.50)	0.90 (0.58)
	RMSE of ${\widehat{\beta }}_1\times {10}^2$	48.09	49.28	32.21	34.46
	RMSE of ${\widehat{\beta }}_2\times {10}^2$	14.44	14.46	9.69	9.69
	RMSE of ${\widehat{\beta }}_3\times {10}^2$	34.24	35.48	24.25	25.05
Random initial value	Iteration	254.7 (84.8)	470.9 (800.8)	150.7 (35.1)	296.7 (294.7)
	Time cost	19.5 (7.9)	4170.6 (847.3)	20.9 (6.4)	6463.7 (752.0)
	${\Vert \widehat{\text{Λ}}(t)-\text{Λ}(t)\Vert }_{\infty }$	1.16 (0.73)	1.28 (0.79)	0.82 (0.50)	0.96 (0.61)
	RMSE of ${\widehat{\beta }}_1\times {10}^2$	48.10	49.87	32.22	35.45
	RMSE of ${\widehat{\beta }}_2\times {10}^2$	14.44	14.51	9.69	9.71
	RMSE of ${\widehat{\beta }}_3\times {10}^2$	34.22	35.33	24.24	25.97

5.2. NHPP with Gamma Frailties

Consider the counting process $\{\mathbb{N}(t),t⩾0\}$ with conditional mean $E(\mathbb{N}(t)\mid \gamma ,\mathit{Z})=\gamma e^{{\mathit{\beta }}^T\mathit{Z}}\text{Λ}(t)$, where γ ∼ Gamma(ν, 1/ν). Without loss of generality, let ν = 1. Other settings including the covariates, the regression coefficient β, the baseline mean function Λ(t), and the observation schemes are the same as those in the last subsection. Two sample sizes n = 50,100 are considered. Both BCA1SG and the doubly iterative algorithm in Wellner and Zhang (2007) are used to fit the synthetic data. As for initial values, we set $\left({\beta }_1^{(0)},{\beta }_2^{(0)},{\beta }_3^{(0)}\right)=(0,0,0)$, ν⁽⁰⁾ ∼ Uniform(0, 10), and ${\text{ΔΛ}}_l^{(0)}=r\left(T_l-T_{l-1}\right)(l=1,\dots ,m)$ with r ∼ Uniform(1, 5) for both methods. The simulation results based on the 1,000 Monte Carlo replications are presented in Table 2. The conclusions are similar to those in Section 5.1. That is, BCA1SG is much faster, and the associated estimates of Λ and β are more accurate.

Table 2.

Under random initial values, the number of iterations, time cost in seconds, averaged ${\Vert \widehat{\text{Λ}}(t)-\text{Λ}(t)\Vert }_{\infty }$, and RMSEs of $\widehat{\nu }$ and $\widehat{\mathit{\beta }}$ for the NHPP model with gamma frailties when $ϵ={10}^{-6}$, $\nu =1,\left({\beta }_1,{\beta }_2,{\beta }_3\right)=(0.5,0.5,1.5)$ and $\text{Λ}(t)=t^{3/4}$. The standard errors are shown in parentheses

	n = 50		n = 100
	BCA1SG	DIA	BCA1SG	DIA
Iteration	2059.4 (935.8)	15.9 (5.6)	1812.0 (512.7)	12.7 (2.5)
Time cost	89.5 (43.2)	21608.4 (11267.3)	181.3 (54.2)	45913.6 (17793.2)
${\Vert \widehat{\text{Λ}}(t)-\text{Λ}(t)\Vert }_{\infty }$	10.13 (9.60)	49.49 (23.93)	6.90 (5.92)	46.65 (15.02)
RMSE of $\widehat{\nu }\times {10}^2$	22.15	26.53	18.86	24.60
RMSE of ${\widehat{\beta }}_1\times {10}^2$	59.32	119.90	42.53	114.99
RMSE of ${\widehat{\beta }}_2\times {10}^2$	16.50	19.39	11.84	12.00
RMSE of ${\widehat{\beta }}_3\times {10}^2$	35.72	47.35	24.75	37.97

6. Real Data Analysis

6.1. Panel count data

Chiou et al. (2018) did a semi-parametric analysis to the skin cancer data collected in the clinical trial conducted at the University of Wisconsin Comprehensive Cancer Center. The study recruited 290 patients experiencing skin cancer and randomly allocated them to two groups, a treatment group with oral difluoromethylornithine (DFMO) at a daily dose of 0.5 gram/m² and a placebo group. These patients were followed for 3–5 years, and the follow-up times of different patients differ. In each follow-up visit the newly developed skin tumors since the last examination were counted and removed, and then the treatment continued. Information about the treatment, age, gender and number of tumors at the beginning of the study for each patient are available. The dataset “skiTum” in R package spef including 100 patients is used in our analysis. We model the treatment effect of DFMO through the semi-parametric model

$$E(\mathbb{N}(t)\mid Z)=\text{Λ}(t)\exp \left({\beta }_1{Z}_1+{\beta }_2{Z}_2+{\beta }_3{Z}_3+{\beta }_4{Z}_4\right),$$

where $\mathbb{N}(t)$ is the cumulative counts of tumors observed from the beginning to time t. Z₁ is a dummy variable that takes one when the age at enrollment is larger than 65. Z₂, Z₃ are two indicators for the gender and treatment. Z₄ represents the number of tumors at enrollment.

We compute the semi-parametric MLE using both BCA1SG and the doubly iterative algorithm in Wellner and Zhang (2007). Initial values are ${\mathit{\beta }}^{(0)}=(0,0,0,0)$, and ${\text{ΔΛ}}_l^{(0)}=T_l-T_{l-1}(l=1,\dots ,m)$. Two convergence thresholds $ϵ={10}^{-4}$ and 10⁻⁵ are used. The estimation results are presented in Table 3, and the estimates of the baseline mean function are provided in the Supporting Information. From Table 3, we can see that the doubly iterative algorithm is much more time-consuming than BCA1SG, especially when the convergence threshold is small. When $ϵ={10}^{-4}$, the doubly iterative algorithm stops immaturely, making the estimates of regression coefficients far from the true MLE.

Table 3.

Number of iterations, time cost in seconds, log-likelihood and estimates of the regression coefficients when fitting the skin cancer data under different convergence thresholds $ϵ$

$ϵ$	Algorithm	Iteration	Time cost	Log-likelihood	${\widehat{\beta }}_1$	${\widehat{\beta }}_2$	${\widehat{\beta }}_3$	${\widehat{\beta }}_4$
10−4	BCA1SG	549	20.9	272.945	0.175	0.177	0.141	0.123
	DIA	17	602.9	269.106	0.103	0.018	0.029	0.114
10−5	BCA1SG	803	31.9	272.946	0.175	0.177	0.141	0.123
	DIA	38	2150.8	272.877	0.166	0.155	0.126	0.122

6.2. Interval-censored survival data

Gomez et al. (2000) reported HIV infection data collected in a study conducted by the Germans Trias i Pujol Hospital among 972 intravenous drug users. During the study patients recurrently went to the hospital to take HIV antibody tests. The following information was collected from each patient: the age at enrollment; gender; the date of first usage of intravenous drugs; the date of last negative HIV antibody test and the date of first positive antibody test. As a result, the elapsed time between intravenous drug initiation and HIV infection is interval-censored. In line with Gomez et al. (2000), we focus on the 767 patients who started drug usage after 1980 and consider the following Cox model

$$\text{Λ}(t\mid Z)=\text{Λ}(t)\exp \left({\beta }_1{Z}_1+{\beta }_2{Z}_2\right),$$

where Λ(t) is the baseline cumulative hazard function for the elapsed time and Z₁ and Z₂ represent gender and the age at enrollment, respectively. We compute the semi-parametric MLE using the proposed BCA1SG algorithm, the EM algorithm in Zeng et al. (2016), and the doubly iterative algorithm in Huang and Wellner (1997) where the non-parametric component is updated through the iterative convex minorant algorithm. Initial values are ${\mathit{\beta }}^{(0)}=(0,0)$, ${\text{ΔΛ}}_l^{(0)}=1/m(l=1,\dots ,m)$ and two thresholds ϵ = 10⁻⁴ and 10⁻⁵ are used. Table 4 presents the results from the three algorithms, and the estimates of the baseline cumulative hazard function are provided in the Supporting Information. As can be seen, the estimates given by the three algorithms are similar and the log-likelihood given by BCA1SG is slightly larger than those by the EM and the doubly iterative algorithm under both convergence thresholds. The EM and the doubly iterative algorithm take much longer time when we decrease from 10⁻⁴ to 10⁻⁵ while BCA1SG is much more efficient. Another interesting finding is that the doubly iterative algorithm is much more efficient than the EM algorithm in this example. The reason is twofold. First, based on our simulation experience, the ICM algorithm embedded in the doubly iterative algorithm is quite efficient if the true maximizer of $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ contains many zeros. In our case, the true ML estimate of the 111-dimensional vector Λ contains 88 zeros. Second, the observed censoring intervals are wide in this HIV data, leading to a high missing rate and thus a slow convergence for the EM algorithm. Specifically, the average length of the observed intervals (except the right-censored observations) is 67 months, which is quite large considering that the observation times range from 1 month to 178 months. Therefore, the doubly iterative algorithm is much faster than the EM algorithm in this case.

Table 4.

Number of iterations, time cost in seconds, log-likelihood and estimates of the regression coefficients when fitting the HIV infection data under different convergence thresholds $ϵ$

$ϵ$	Algorithm	Iteration	Time cost	Log-likelihood	${\widehat{\beta }}_1\times {10}^2$	${\widehat{\beta }}_2\times {10}^2$
10−4	BCA1SG	137	5.8	−604.20976	23.18	−1.37
	DIA	123	31.4	−604.21429	23.35	−1.26
	EM	442	220.1	−604.34055	23.20	−1.36
10−5	BCA1SG	197	9.0	−604.20936	23.20	−1.37
	DIA	153	94.9	−604.20944	23.20	−1.36
	EM	1245	607.7	−604.21550	23.20	−1.37

To gauge the impact of interval censoring, we analyzed the HIV infection data set in a “naive” way with standard Cox model. We simply treat the interval-censored data as the standard right-censored data using the method of midpoint imputation (Lee et al., 2011). Specifically, if a subject is censored in an interval with finite endpoints, we treat it as an exact non-censored observation and naively set the occurrence time as the midpoint of the censoring interval; if a subject is right-censored, we leave it unchanged. In this way, the resulting data are standard survival data with right censoring. We apply the Cox regression model to fit the right-censored data with midpoint imputation, where the parametric component β is estimated through the method of partial likelihood, and the non-parametric component $\text{Λ}(\cdot )$ is estimated by the Breslow estimator. The estimate for the parametric component β based on right-censored data with midpoint imputation is average length of the observed intervals $\widehat{\mathit{\beta }}={(0.1845,-0.0116)}^T$, and the estimates for the non-parametric component $\text{Λ}(\cdot )$ are shown in Figure 2.

Figure 2.

Estimates of the baseline cumulative hazard function when fitting the right-censored HIV infection data with midpoint imputation by the partial likelihood method, as well as when fitting the original interval-censored HIV infection data by the ML method (through the BCA1SG algorithm).

For the comparison purpose, we also present the estimation results for the original interval-censored data given by the ML method (through the BCA1SG algorithm) in Figure 2. As can be seen, the results for the right-censored data with midpoint imputation and the interval-censored data are quite different. This implies that it is risky to use a “naive” standard Cox analysis, which may lead to incorrect estimates for both β and $\text{Λ}(\cdot )$. Hence it is necessary to consider the interval censoring in this real example.

7. Discussion

This paper investigated the computation of MLE in semi-parametric models with shape constraints on the non-parametric component. A new computational framework was proposed and its convergence properties were investigated. Unlike the original block coordinate ascent algorithm (Tseng, 2001; Wellner and Zhang, 2007), the BCA1SG algorithm does not waste time and resources on maximizing $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ exactly for a poor estimate of β, so it is insensitive to initial values. Similar to Newton’s method, the proposed algorithm takes advantage of the information contained in the matrix W, which is often specified as $-{\mathbf{\nabla }}_{\mathbf{\text{Λ}}}^2\ell (\mathit{\beta },\mathbf{\Lambda })$ when $\mathbf{\Lambda }\mapsto \ell (\mathit{\beta },\mathbf{\Lambda })$ is concave. As a result, the convergence speed is significantly improved. Without the proposed algorithm, resampling methods such as bootstrap and jackknife for semi-parametric models are prohibitive when Λ is high-dimensional. The BCA1SG algorithm makes these resampling methods feasible as it significantly improves the computational efficiency. Therefore, it plays an important role in constructing confidence intervals for semi-parametric models through the bootstrap procedure.

A topic for future research is to further improve the computational efficiency by adaptively changing the number of GR iterations when updating Λ. For example, the number can be small at the beginning and gradually increases as the algorithm proceeds.

Data Availability Statement

The data sets analyzed in this paper are publicly available. The skin cancer data in Section 6.1, and the HIV infection data in Section 6.2 can be obtained from Chiou et al. (2018) and Gomez et al. (2000), respectively.