Semiparametric Frailty Models for Clustered Failure Time Data

Summary

We consider frailty models with additive semiparametric covariate effects for clustered failure time data. We propose a doubly penalized partial likelihood (DPPL) procedure to estimate the nonparametric functions using smoothing splines. We show that the DPPL estimators could be obtained from fitting an augmented working frailty model with parametric covariate effects, whereas the nonparametric functions being estimated as linear combinations of fixed and random effects, and the smoothing parameters being estimated as extra variance components. This approach allows us to conveniently estimate all model components within a unified frailty model framework. We evaluate the finite sample performance of the proposed method via a simulation study, and apply the method to analyze data from a study of sexually transmitted infections (STI).

1. Introduction

Frailty models are commonly used to model correlated failure time data. There is an extensive literature on parametric regression for covariate effects in frailty models (Therneau, Grambsch, and Pankratz 2003). For example, McGilchrist and Aisbett (1991) proposed a penalized partial likelihood approach for parameter estimation in a Gaussian frailty model setting. Following Breslow and Clayton (1993), Ripatti and Palgram (2000) used the Laplace approximation for the integrated likelihood of Gaussian frailty models. Murphy (1995) and Parner (1998) studied the asymptotic properties of Gamma frailty models. To accommodate complicated covariate effects and their unknown functional forms, we propose frailty models with some covariates modeled nonparametrically and some covariates modeled parametrically. To our knowledge, little work has been done on frailty models with semiparametric covariate effects for correlated failure time data.

This research is motivated by a study of sexually transmitted infections (STI). Evidence-based recommendations about the beginning age and frequency of STI screening require an accurate quantification of time from sexual debut to first infection (Meyers et al. 2008). In adolescent women, concurrent infections with multiple organisms are not uncommon, with Chlamydia trachomatis, Neisseria gonorrhoeae and Trichomonas vaginalis being the most frequently detected pathogenes. Once sexually active, young women are at risk of infection via sexual behavior. Evaluation of risk associated with STI acquisition is important as it provides clinically useful markers for more targeted STI screening. Since concurrent and repeated infections with multiple organisms within the same subject are correlated, it is desirable to use frailty models to examine factors related to the timing of STI following the onset of sexual activity.

The number of sex partners is often thought to be indicative of STI risk. However, the assumption of a linear effect for the number of partners is nonetheless questionable. Intuitively, while a smaller number of partners typically indicates lower risk, a very large number of partners may not present proportionally higher risk because of increased sexual experience and prophylactic activities. Indeed, a closer examination of the fourth order polynomial effects for the lifetime number of sex partners revealed that the coeffcients of all polynomial terms were significant at level 0.05. Hence it is desirable to model the effect of the number of sex partners nonparametrically while using the frailty to account for within-subject correlation.

For independent survival data, spline and kernel methods have been used in nonparametric regression. For example, O’Sullivan (1988) estimated the nonparametric covariate functions by smoothing splines. Hastie and Tibshirani (1990) proposed an alternative algorithm for fitting smoothing splines. Gray (1992, 1994) argued for the use of the penalized spline method. Local kernel methods were proposed by Tibshirani and Hastie (1987). Fan et al. (1997) studied the local kernel method and related asymptotic properties. More recently, Duchateau et al. (2004) studied frailty models with a single nonparametric covariate function using a smoothing spline. Cai et al. (2007, 2008) and Yu and Lin (2008) discussed a marginal likelihood estimation associated with local kernel methods for correlated survival data.

Du and Ma (2010) proposed a fully nonparametric hazard model with frailty log h(t, u; b) = η(t, u) + z^T b, with a fully nonparametric function of time and covariates η(t, u), where u is a vector of covaraites and b is a frailty. Our proposed semiparametric model takes the form $\lambda (t;{\mathit{x}}_{\mathit{i}},{\mathit{w}}_{\mathit{i}},{\mathit{z}}_{\mathit{i}},\mathit{b})={\lambda }_0\left(t\right)exp\left\{{\Sigma }_{j=1}^{p_1}{\theta }_j\right(x_{ij})+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_{\mathit{i}}^{\mathit{T}}b\}$. It is different from Du and Ma’s model in the following two aspects. (i) Modeling structure: our model distangles the baseline hazard and covariates in the same spirit as the Cox model, and models covariate effects using a semiparametric additive function which allows for both parametric and nonparametric covariate effects, with extensions to multiple covariates. Although Du and Ma’s nonparametric model takes a general functional space, models defined in a more restrictive functional space, such as semiparametric additive frailty models, are often much easier to interpret in biomedical applications. Better estimation effciency associated with the parametric component also increases its appeal compared to the fully nonparametric approach. Further, multi-dimensional nonparametric functions are often diffcult to fit. (ii) Parameter estimation: Our approach connects the semiparametric spline Cox model with a frailty model, which is parallel to the connection between mixed models and spline models, as recognized by Green (1984) and Lin and Zhang (1999). As a result, the nonparametric functions can be easily estimated as linear combinations of fixed and random effects. The smoothing parameters can be easily estimated as extra variance components which are naturally imbedded in the frailty model with a parametric covariate function. In contrary to our work, Du and Ma (2010) does not build a connection between the frailty model and spline Cox model and does not take an advantage of the variance components method for estimating the smoothing parameter. Instead, they used a cross-validation method for estimating the smoothing parameter which is computationally more intensive in general.

This article is structured as follows. In Section 2, we present our modeling structure. In Section 3, we discuss the DPPL method for estimation and inference of the semiparametric functions. In Section 4, we describe the estimation of variance components and smoothing parameters. In Section 5, we conduct a simulation study to evaluate the finite sample performance of the method. In Section 6, we illustrate the use of our model by analyzing data from an STI epidemiologic study. We conclude the paper in Section 7 with a few remarks about the proposed method.

2. Semiparametric Frailty Models

To assess the linear and potentially nonlinear effects of covariates in a frailty model setting, we introduce a general frailty model (1) with semiparametric additive covariate functions for clustered, hierarchial, and spatial data. We refer to this model as the semiparametric frailty model for simplicity.

For the ith observation, let $T_i^{\ast }$ denote the underlying failure time, and C_i denote the censoring time for i = 1, ⋯ , n, where n is the total number of observations. We write the observed time as $T_i=min(T_i^{\ast },C_i)$, C_i). Let δ_i be a censoring indicator. The covariate vector x_i has p₁ components and w_i has p₂ components. The covariates z_i are assumed to be associated with random effects b = {b₁, ⋯ , b_q}^T. Conditioning on the random effects b, we assume that survival times T_is are independent, and underlying failure time $T_i^{\ast }$ is independent of censoring time C_i. We further assume that the censoring times are independent of the random effects b.

We present in (1) a semiparametric frailty model with a general correlation structure. The model can be used to model clustered, hierarchical and spatial survival data

$$\lambda (t;{\mathit{x}}_{\mathit{i}},{\mathit{w}}_{\mathit{i}},{\mathit{z}}_{\mathit{i}},\mathit{b})={\lambda }_0\left(t\right)exp\left\{\sum _{j=1}^{p_1}{\theta }_j\right(x_{ij})+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_{\mathit{i}}^{\mathit{T}}\mathit{b}\},$$ (1)

where λ₀(t) is an unspecified baseline hazard, θ_j(x_j) is a nonparametric covariate function for the covariate x_j, w is the covariate vector whose effect is modeled linearly, random effects b ~ MVN(0, D(ν)), with a full rank covariance matrix D(ν), and ν is a vector of variance components. Examples of the choices of covariance structures for clustered, hierarchical and spatial survival data can be found, e.g., in Breslow and Clayton (1993).

We develop below an estimation and inference procedure for the general semiparametric frailty model (1), of which the model for clustered survival data is a special case. The integrated likelihood of (θ, γ, ν) for the observed data under the general additive frailty model (1) is

$$\begin{array}{cc}\hfill & L\left\{{\lambda }_0\right(t),\theta ,\gamma ,\nu \}\hfill \\ \hfill & =\int {\prod }_{i=1}^n\lambda {(t_i;{\mathit{x}}_{\mathit{i}},{\mathit{w}}_{\mathit{i}},{\mathit{z}}_{\mathit{i}},\mathit{b})}^{{\delta }_i}exp\{-\Lambda (t_i;{\mathit{x}}_{\mathit{i}},{\mathit{w}}_{\mathit{i}},{\mathit{z}}_{\mathit{i}},\mathit{b}\left)\right\}f\{\mathit{b},\mathit{D}(\nu \left)\right\}d\mathit{b}\hfill \\ \hfill & =\frac{1}{{\mid \mathit{D}\left(\nu \right)\mid }^{1∕2}}\int \prod _{i=1}^n{\left\{{\lambda }_0\right(t_i\left)e^{{\Sigma }_{j=i}^{p1}{\theta }_j\left(x_{ij}\right)+{\mathit{w}}_i^T\gamma +{\mathit{z}}_i^T\mathit{b}}\right\}}^{{\delta }_i}\hfill \\ \hfill & \times exp\{-{\Lambda }_0(t_i\left)e^{{\Sigma }_{j=i}^{p1}{\theta }_j\left(x_{ij}\right)+{\mathit{w}}_i^T\gamma +{\mathit{z}}_i^T\mathit{b}}\right\}{e}^{-\frac{1}{2}{\mathit{b}}^T\mathit{D}{\left(\nu \right)}^{-1}\mathit{b}}d\mathit{b},\hfill \end{array}$$ (2)

where b ~ MV N(0, D(ν)). Since the θ_j(·) are nonparametric functions, we discuss in the next section their estimation using smoothing splines.

3. Estimation of the Semiparametric Functions Using Smoothing Splines

3.1 Smoothing Spline Estimation

In this section, we discuss estimation and inference of model (1) using smoothing splines. We assume the nonparametric covariate functions θ_j(·) to be smooth and twice-differentiable. Smoothing spline estimation of the θ_j(·) can proceed by maximizing the penalized integrated loglikelihood

$$\ell \left\{{\lambda }_0\right(t),\theta ,\gamma ,\nu )\}+\sum _{j=1}^{p_1}\frac{1}{2{\tau }_j}\int {\theta }_j^{\left(2\right)}{\left(x\right)}^{2}dx,$$ (3)

where ℓ(·) = logL(·) and L(·) is defined in (2), τ = (τ₁, ⋯ , τ_p1) is a vector of smoothing parameters controlling the goodness of fit and the smoothness of the curves.

Let ${\theta }_j={\left\{{\theta }_j\right(x_{1j}^0),\cdots ,{\theta }_j(x_{r_{j}j}^0\left)\right\}}^T$, where $x_{kj}^0(k=1,\cdots ,r_j)$ are the ordered r_j distinct covariate values for the jth covariate x_j. We assume ${\Sigma }_{k=1}^{r_j}{\theta }_j\left(x_{kj}^0\right)=0$ to ensure identifiability similar to Lin and Zhang (1999). Let N_j be an n by r_j indicator matrix such that {θ_j(x_1j), ⋯ , θ_j (x_nj)}^T = N_jθ_j. The conditional hazard of the ith observation given the random effects b can then be rewritten as

$$\begin{array}{cc}\hfill & \lambda (t;{\mathit{x}}_i,{\mathit{w}}_i,{\mathit{z}}_i,\mathit{b})={\lambda }_0\left(t\right)\hfill \\ \hfill & \times exp\{{\mathit{N}}_{i1}^T{\theta }_1+\cdots +{\mathit{N}}_{i{p}_1}^T{\theta }_{p_1}+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_i^T\mathit{b}\},\hfill \end{array}$$ (4)

where N_ij^T is the ith row of the indicator matrix N_j. Using the results of O’Sullivan (1988) and noting equation (3) is a continuous function of the θ_j(·), one can show that, for given values of λ₀(t), τ and ν, the maximizer of the integrated penalized loglikelihood (3) is a vector of natural cubic smoothing spline estimators of the θ_j(·), and (3) can be equivalently written as

$$\ell \left\{{\lambda }_0\right(t),\theta ,\gamma ,\nu )\}+\sum _{j=1}^{p_1}\frac{1}{2{\tau }_j}{\theta }_{\mathit{j}}^{\mathit{T}}{\mathit{K}}_{\mathit{j}}{\theta }_{\mathit{j}},$$ (5)

where K_j is the smoothing spline penalty matrix of the covariate x_j. See Section 2.2 of Green and Silverman (1994) for more details of the proof of solution to be smoothing spline.

Since the integrated loglikelihood (2) does not have a closed form expression, following Breslow and Clayton (1993) and Ripatti and Palmgren (2000), we write (2) as ∫ exp{−S(b)}db, and apply the Laplace approximation to (2). Using calculations similar to expression (3) and profiling out the baseline hazard in a similar way to Appendix B of Ripatti and Palmgren (2000), one can show that, given (τ, ν), smoothing spline estimators of the θ_j(·) and the parametric regression coeffcient γ can be obtained by jointly maximizing following equation with respect to θ, γ, b.

$$\begin{array}{cc}\hfill & {\ell }_D({\theta }_1,\cdots ,{\theta }_{p_1},\gamma ,\mathit{b};\nu ,\tau )\approx \hfill \\ \hfill & \sum _{i=1}^n{\delta }_i\left[\left\{\sum _{j=1}^{p_1}{\mathit{N}}_{ij}^T{\theta }_j+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_i^T\mathit{b}\right\}-log\left\{\sum _{l\in R(t_{i)}}{e}^{{\Sigma }_{j=1}^{p_1}{\mathit{N}}_{lj}\theta +{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_l^T\mathit{b}}\right\}\right]\hfill \\ \hfill & -\frac{1}{2}{\mathit{b}}^T{\mathit{D}}^{-1}\mathit{b}-\sum _{j=1}^{p_1}\frac{1}{2{\tau }_j}{\theta }_j^T{\mathit{K}}_j{\theta }_j.\hfill \end{array}$$ (6)

Given ν, τ, the DPPL is a partial likelihood with quadratic penalty terms which is concave and has a unique maximizer (up to a constant) as a function of the θ_j and γ. Given ${\widehat{\theta }}_j$ estimated at the design points, the smoothing spline estimators of θ_j(x) at any point x can be obtained by interpolating the splines using these estimated function values (see Section 2.4.1 of Green and Silverman 1994).

3.2 The Frailty Model Representation

In this subsection, we discuss the connection between the DPPL (6) for model (1) and the penalized likelihood of traditional frailty models with parametric covariate functions. Following a similar spirit of Lin and Zhang (1999), one can show that there is a one-to-one transformation between θ_j and ${({\beta }_j,{\mathit{a}}_j^T)}^T$, i.e., ${\theta }_j={\mathit{x}}_j^0{\beta }_j+{\mathit{B}}_j^T{\mathit{a}}_j$. Here ${{\mathit{x}}_j^0=(x_{1j}^0-{\stackrel{‒}{x}}_j^0,\cdots ,x_{r_{j}j}^0-{\stackrel{‒}{x}}_j^0)}^T$, ${\stackrel{‒}{x}}_j^0={\Sigma }_{i=1}^{r_j}{x}_{ij}^0∕r_j$, ${\mathit{B}}_j={\mathit{L}}_j{\left({\mathit{L}}_j^T{\mathit{L}}_j\right)}^{-1}$ and ${\mathit{L}}_j{\mathit{L}}_j^T={\mathit{K}}_j$ (see Green, P. J. 1987 for detail). One therefore can easily show that this new parameterization yields ${\theta }_j^T{\mathit{K}}_j{\theta }_j={\mathit{a}}_j^T{\mathit{a}}_j$.

Applying this transformation to (6), we see that the DPPL is equal to

$${\ell }_D(\beta ,\gamma ,\mathit{a},\mathit{b};\nu ,\tau )=\sum _{i=1}^n{\delta }_i\left[\sum _{j=1}^{p_1}{\mathit{N}}_{ij}^T({\mathit{x}}_j^0{\beta }_j+{\mathit{B}}_j{\mathit{a}}_j)+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_i^T\mathit{b}-log\left\{\sum _{l\in R\left(t_i\right)}{e}^{{\Sigma }_{j=1}^{p_1}{\mathit{N}}_{lj}({\mathbf{x}}_j^0{\beta }_j+{\mathit{B}}_j{\mathit{a}}_j)+{\mathit{w}}_{\mathit{l}}^{\mathit{T}}\gamma +{\mathit{z}}_l^T\mathit{b}}\right\}\right]-\frac{1}{2}{\mathit{b}}^T{\mathit{D}}^{-1}\mathit{b}-\frac{1}{2}{\mathit{a}}^T{\Psi }^{-1}\mathit{a},$$ (7)

where $\mathit{a}={({\mathit{a}}_1^T,\cdots ,{\mathit{a}}_p^T)}^T$, β = (β₁, ⋯ , β_p)^T, and Ψ = diag(τ₁I_r1×r1, ⋯ , τ_pI_rp×rp).

A comparison of (7) with equation (4) in Ripatti et al. (2000) shows that the two penalized likelihoods take the same form. Hence the DPPL smoothing spline estimators of {θ₁(·), ⋯, θ_p(·)} and the regression coeffcients γ of the semiparametric frailty model (1) can be obtained by fitting the following augmented working frailty model with parametric covariate effects using the penalized likelihood approach of Ripatti, et al. (2000),

$${\lambda }_i(t;{\mathit{x}}_i,{\mathit{w}}_i,{\mathit{a}}_i,{\mathit{z}}_i,\mathit{b})={\lambda }_0\left(t\right)exp\left\{\sum _{j=1}^{p_1}{\mathit{N}}_{ij}{\mathit{x}}_j^0{\beta }_j+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +\sum _{j=1}^{p_1}{\mathit{N}}_{ij}{\mathit{B}}_j{\mathit{a}}_j+{\mathit{z}}_i^T\mathit{b}\right\},$$ (8)

where β and γ are vectors of fixed effects, and a and b are independent random effects distributed as MV N(0,Ψ) and MV N(0, D(ν)), respectively.

After obtaining the fixed and random effect estimators $\widehat{\gamma }$, ${\widehat{\gamma }}_j$, and ${\widehat{\mathrm{a}}}_{\mathrm{j}}$ by maximizing , the DPPL smoothing spline estimators of θ_j can be calculated as ${\widehat{\theta }}_j={\mathit{x}}_j^0{\widehat{\beta }}_j+{\mathit{B}}_j{\widehat{\mathit{a}}}_j$, which is a linear combination of fixed and random effect estimators. For inference, we need to estimate the variances of ${\widehat{\theta }}_j$ and $\widehat{\gamma }$. Ripatti et al. (2000) proposed using the inverse of the minus second partial derivative of penalized partial likelihood to estimate covariance matrix of coefficient estimators. Their simulation results showed that the estimators works well. To make inference of ${\widehat{\theta }}_j$, we take a similar approach. The covariance of ${\widehat{\theta }}_j$ can then be estimated by the transformation, $\widehat{COV}\left({\widehat{\theta }}_j\right)=({\mathit{x}}_j^0,{\mathit{B}}_j)\widehat{COV}({\widehat{\beta }}_j,{{\widehat{\mathit{a}}}_j}^T){({\mathit{x}}_j^0,{\mathit{B}}_{\mathit{j}})}^T$.

3.3 Frailty Models with Stratified Baseline Hazards

In the context of the STI data example, infections with different organisms tend to have distinct population prevalence rates and natural histories. We therefore extend model (1) to accommodate stratified baseline hazards as follows

$$\begin{array}{cc}\hfill & \lambda (t;{\mathit{x}}_i,{\mathit{w}}_i,{\mathit{z}}_i,\mathit{b})={\lambda }_{0j}\left(t\right)\hfill \\ \hfill & \times exp\left\{{\theta }_1\right(x_{i1})+\cdots +{\theta }_p(x_{i{p}_1})+{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\mathit{z}}_{ij}^{T}b\},\hfill \end{array}$$ (9)

where λ_0j(t) is the baseline hazard for the jth stratum (organism), j = 1, ⋯ , J. One can write a DPPL similar to (6) with different at risk sets R_ij={All observations with respect to jth organism at risk at time t_ij}. Estimation and inference for model (9) then follow a scheme similar to those with a common baseline hazard as presented in Section 3.2.

4. Inference on Smoothing Parameters and Variance Components

We assume in Section 3 that the smoothing parameters τ and the variance components ν are known. In practice, they need to be estimated from the data. Motivated by the working parametric frailty model (8), we propose to treat the smoothing parameter τ_j(j = 1, ⋯ , p) as extra variance components. We estimate τ and ν simultaneously as variance components by maximizing the profile likelihood (10) in the working parametric frailty model (8) as follows:

$$\begin{array}{cc}\hfill & l_D\left(\beta \right({\tau }_{\mathbf{j}},\nu ),{\tau }_{\mathbf{j}},\nu )=\frac{1}{2}log\mid \tilde{\mathbf{D}}\left(\theta \right)\mid -\frac{1}{2}\mathbf{log}\mid {\mathbf{S}}^{″}(\mathit{a},\mathit{b})\mid \hfill \\ \hfill & \frac{1}{2}(\mathit{a},\mathit{b})\tilde{\mathbf{D}}{({\tau }_{\mathbf{j}},\nu )}^{-1}{(\mathit{a},\mathit{b})}^{\mathbf{T}},\hfill \end{array}$$ (10)

where $\tilde{D}=\left(\begin{array}{cc}\hfill \Psi \hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill \mathbf{D}\left(\nu \right)\hfill \end{array}\right)$. Following Ripatti and Palgram (2000), The estimating equation of variance components can be derived and simplified from the profile likelihood as follows:

$$-\frac{1}{2}\left[tr\left({\tilde{\mathbf{D}}}^{-1}\frac{\partial \tilde{\mathit{D}}}{\partial (\tau ,\nu )}\right)+\mathbf{tr}\left({\mathbf{S}}^{″}{(\widehat{\mathit{a}},\widehat{\mathit{b}})}^{-1}\frac{\partial {\tilde{\mathit{D}}}^{-1}}{\partial (\tau ,\nu )}\right)-(\widehat{\mathit{a}},\widehat{\mathit{b}}){\tilde{\mathbf{D}}}^{-1}\frac{\partial \tilde{\mathbf{D}}}{\partial (\tau ,\nu )}{\tilde{\mathbf{D}}}^{-1}{(\widehat{\mathit{a}},\widehat{\mathit{b}})}^{\mathbf{T}}\right]=0$$

These equations are in the same spirit as equation (8) of Ripatti and Palmgren (2000) and equation (6.21) of Duchateau et al. (2008) by noting that $\frac{\partial {\tilde{D}}^{-1}}{\partial (\tau ,\nu )}={\tilde{\mathbf{D}}}^{-1}\frac{\partial \tilde{\mathbf{D}}}{\partial (\tau ,\nu )}{\tilde{\mathbf{D}}}^{-1}$.

The estimating equations can be simplified as:

$$\widehat{\tau }=\frac{{\widehat{\mathit{a}}}_j^{\prime }{\widehat{\mathit{a}}}_j+tr\left\{{[{\mathbf{S}}^{″}(\widehat{\mathit{a}},\widehat{\mathit{b}}){}_{{\mathit{a}}_j{\mathit{a}}_j}]}^{-1}\right\}}{r_j-2}$$ (11)

and

$$\widehat{\nu }=\frac{{\widehat{\mathit{b}}}^{\prime }\widehat{\mathit{b}}+tr\left\{{\left[{\mathbf{S}}^{″}{(\widehat{\mathit{a}},\widehat{\mathit{b}})}_{\mathit{b}\mathit{b}}\right]}^{-1}\right\}}{n_c},$$ (12)

where ${\widehat{\mathit{a}}}_j$ is the maximizer of (7); ${\mathit{S}}^{″}(\widehat{\mathit{a}},\widehat{\mathit{b}})a_j{a}_j$ is the (r_j − 2) × (r_j − 2) diagonal block submatrix of ${\mathbf{S}}^{″}(\widehat{\mathit{a}},\widehat{\mathit{b}})$ corresponding to a_j; and tr{} is the trace value of the matrix, and n_c is the number of clusters. Here ${\mathbf{S}}^{″}(\widehat{\mathit{a}},\widehat{\mathit{b}})$ is calculated by plugging in the corresponding estimators. The estimated value $\widehat{\mathit{a}}$,$\widehat{\mathit{b}}$, D are all functions of τ, ν. The estimates of τ, ν are obtained by iterating equations (11) and (12). The general expression of S″(·) is given in the Appendix I. The baseline hazard is estimated using Breslow’s estimator. An alternative to the S″ is using ${\tilde{\mathbf{S}}}^{″}$, the version profiling out the the baseline hazard function.

The variance of $\widehat{\nu }$ for a shared frailty model can be estimated by using

$$var\left(\widehat{\nu }\right)=2{\widehat{\nu }}^2{\left[n_c+\frac{1}{{\widehat{\nu }}^2}tr\left\{{\left[{\mathbf{S}}^{″}{(\widehat{\mathit{a}},\widehat{b})}_{bb}\right]}^{-1}{\left[{\mathbf{S}}^{″}{(\widehat{\mathit{a}},\widehat{b})}_{\mathit{bb}}\right]}^{-1}\right\}-\frac{2}{\widehat{\nu }}tr\left\{{\left[{\mathbf{S}}^{″}{(\widehat{\mathit{a}},\widehat{b})}_{\mathit{bb}}\right]}^{-1}\right\}\right]}^{-1}$$

Note that this estimating equation is derived from the Fisher information matrix for τ, ν and has already accounted for the variability due to the estimation of τ. Appendix II summarizes the estimation procedure for all the model components. Sample code for implementing the procedure, along with a sample data set, have been posted on www.biostat.iupui.edu/yuz/Code. The code can be adapted to conveniently fit the semiparametric additive models for data organized in the manner described in the introduction.txt file and the main function file dpplest.R.

5. Simulation Studies

In this section, we evaluate finite sample performance of the proposed method through a simulation study. We consider the following hazard model

$$\lambda (t_{ij};x_1,x_2,w_1,b_i)=exp\left\{{\theta }_1\right(x_1)+{\theta }_2(x_2)+w_1\gamma +b_i\},$$ (13)

where

$${\theta }_1\left(x\right)=\{2\times beta(x∕10,8,8)+beta(x∕10,5,5\left)\right\}∕9,$$

$${\theta }_2\left(x\right)=\{6\times beta(x∕10,30,17)+4\times beta(x∕10,3,11\left)\right\}∕40,$$

and beta(·) is the density function of the BETA distribution, and the b_i are independent random effects following N(0, ν).

The true value of regression coeffcient γ was set as 0.5. Covariate w₁ was generated as a binary random variable with an equal probability to be 0 or 1, x₁ was generated as a cluster-level covariate taking values from 100 equally spaced knots in [0, 10], x_ij1 = (i mod 100)/10, and x₂ was generated as

$$x_{2ij}=trun\left\{\right(i+5)∕6\}∕10+10∕5\times (j-1),$$

where trun{·} denotes the truncation operator. The variance component was set as ν = 0.25, 0.5. Censoring time followed an exponential distribution with rate 0.4 and the maximum followup times was set to be 5. The censoring percentage was about 18%. One hundred data sets were generated. In each data set, the number of clusters was 120 with 5 observations per cluster. For each simulated data set, we applied the DPPL method to estimate {θ₁(x₁), θ₂(x₂), γ, ν}.

Table 1 gives the averages of the estimates of the regression coeffcient γ and the variance component ν. For both settings of ν = 0.25 and ν = 0.5, the estimates of γ are very close to the true values. The estimated standard errors of ${\widehat{\gamma }}_j$ are close to their empirical counterparts. The estimates of the variance component ν are also very close to the true value. Their SE estimates are slightly smaller than the empirical ones.

Table 1. Estimators of regression coefficient coefficient and variance component in the simulation studies based on 100 runs.

Parm.	True	Average	Est. SE	Emp. SE	95% CP
			ν = 0.25
γ	0.5	0.5081	0.1016	0.0923	98%
ν	0.25	0.2445	0.0648	0.0757	90%
			ν = 0.5
γ	0.5	0.5057	0.1052	0.1003	97%
ν	0.5	0.4884	0.0973	0.1129	93%

Parm: parameter

CP: coverage probability

Average: average of the estimates

Est. SE: Estimated SE

Emp. SE: Monte-Carlo estimated SE

Fig. 1 depicts the performance of the DPPL spline estimates of the two smooth functions θ₁(x₁) and θ₂(x₂) when the variance component ν = 0.5. Figures 1 (a) and (d) show that the averages of the estimated DPPL smoothing spline estimates of θ₁(x₁) and θ₂(x₂) are close to the true values. There are some biases in the estimated curves when the curvatures of the functions are high, e.g. at the second peak of θ₂(x₂). Fig. 1(b) and (e) show that the estimated SEs of ${\widehat{\theta }}_1\left(x_1\right)$ and ${\widehat{\theta }}_2\left(x_2\right)$ are close to their empirical counterparts except for the regions where the curvatures of the functions are high. Figure 1(c) and (f) calculate the coverage probabilities of the point-wise estimated 95% confidence intervals of ${\widehat{\theta }}_1\left(x_1\right)$ and ${\widehat{\theta }}_2\left(x_2\right)$. They are close to the nominal value 95% except for the peaks. The results are similar when ν = 0.25 and are not shown here. Our simulation study shows that the proposed DPPL method works well in finite samples in estimating the nonparametric functions and the variance components.

Figure 1.

Simulation results for the nonparametric covariate function θ₁(x) and θ₂(x): (a) Average of the DPPL spline estimates of ${\widehat{\theta }}_1\left(x\right)$, dotted and true θ₁(x), solid; (b) Standard errors: estimated, dotted and empirical, solid; (c) Empirical coverage probability: the mean coverage probability is 0.901; (d) Average of the DPPL spline estimates of ${\widehat{\theta }}_2\left(x\right)$, dotted and true θ₂(x), solid, (e) Standard errors of θ₂(x): estimated, dotted and empirical, solid; (f) Empirical coverage probability of θ₂(x): the mean coverage probability is 0.875

We also perform simulations with a larger censoring percentage and a smaller number of events. In the simulation with about 66% censoring (200 events, results not shown), the bias of the parametric regression coeffcients and the nonparametric function are only slightly larger than when there is 18% censoring. The empirical coverage probability for the parametric regression coeffcient is close to 95%. The average coverage probability of the nonparametric function is 86.7% due to the somewhat larger bias at the peak of the curve resulted from a smaller number of events. Performance improves when the sample size increases.

6. Application to the Sexually Transmitted Infection Data

The primary mode of transmission of STI is sexual contact. In the United States, much of the disease burden associated with STI is on women and young people. For example, it is estimated that subjects between 15 and 24 years of age account for about half of new STI cases each year, although they only represent a quarter of the population (Weinstock, Berman, and Cates 2004). STI screening is motivated by the disproportionate morbidity among adolescent women, including pelvic inflammatory disease, ectopic pregnancy, tubal infertility, preterm birth, and increased susceptibility to human immunodeficiency virus infection (CDC 2007; Cates and Wasserheit 1991; Paavonen et al. 2008; and Fleming et al. 1999). Despite the consensus on the need of screening, potentially useful behavioral markers for selected screening have not been adequately assessed because of limited epidemiological data (USPSTF 2007). Among the behavioral markers, the most relevant to the screening practice is the number of sexual partners. An improved understanding of the effect of the number of partners will help us to identify individuals who are at increased STI risk for selective screening.

A recent study provides an opportunity to examine the effect of sexual partners on the time between sexual debut and the first STI. Briefly, young women between ages 14 and 17 years, attending one of three adolescent medicine clinics were eligible for enrollment regardless of prior sexual activity and STI diagnosis. At enrollment, participants were interviewed for their lifetime and recent (past two months) sexual behaviors, as well as the age of sexual debut (first sex) and the number of sex partners. Cervical and vaginal specimens were collected by a research nurse practitioner for testing of C trachomatis, N gonorrhoeae and T vaginalis. Infected participants were treated at the visit or shortly thereafter. The same procedures were repeated every three months. Information about infections prior to study enrollment or outside of the study venues was extracted from participants’ electronic medical records. Detailed study protocol was described elsewhere (Tu et al. 2009). If a participant was sexually active at enrollment, age of first sex was ascertained from enrollment interview; for those who became sexually active during the course of follow-up, age of first sex was determined from quarterly interviews.

There were 387 adolescent women enrolled into the study. The enrolled subjects were followed up to 8.2 years. On average, study participants reported 3 partners in their lifetime at the time of enrollment (median=2,; range 0-28). We focused here on the effects of age of sexual debut, number of unprotected sexual intercourse during the last two months, and the baseline life time number of sexual partners reported at enrollment. The outcome of interest is time from first coitus to the first infection with each of the three organisms C. trachomatis, N. gonorrhoeae and T. vaginalis. Among 387 women, 26% of participants were censored for C. trachomatis infection, 51% are censored for N. gonorrhoeae infection, and 47% are censored for T. vaginalis infection by the end of follow up.

Times from first coitus to the initial STI with each organism within the same individual are correlated due to the common sexual behavior and physiological environment within the same subject. To model the correlation of the three types of infections of the same subject, we used a stratified frailty model with a common random intercept in (9) by assuming z_ij = 1 and b_is are identical and independently distributed as a normal distribution with mean zero and an unknown variance ν. Our preliminary study shows that the lifetime number of partners has a polynomial effect up to 4th order. Therefore, it is desirable to fit a model with a nonparametric effect of lifetime number of partners. To accommodate such a nonlinear effect, we introduced a nonparametric component for the lifetime number of sexual partners. Parametric effects were used for age at first intercourse, ethnicity (white), and number of unprotected sex events in past two months.

Specifically, we considered the following model:

$${\lambda }_{ij}\left(t\right)={\lambda }_{0j}\left(t\right)exp\left\{\theta \right(N\_PARTNERS)+{\gamma }_1\times RACE+{\gamma }_2\times AGE+{\gamma }_3\times N\_SEX+b_i\},$$ (14)

where N_PARTNERS is the lifetime number of partners ascertained at enrollment, AGE is the self-reported age at the first sex, N_SEX is the number of unprotected sex in last two months, θ(·) is an unknown smooth function and b_i is the subject-specific random effect following N(0, ν). Preliminary analysis shows a similar effect of race, age, and N_SEX on risk of STI with different organisms. We therefore assume the covariate effects are the same for the three types of infections for model simplicity. As demonstrated in Fig. 2, while there was a generally positive association between the number of partners and STI acquisition, the risk was highest for subjects with four to six partners, which appeared to be the range that the number of partners effect became significantly different from those without sex partners. This intriguing observation, though previously not reported in literature, is perhaps not entirely surprising. For example, one could speculate that adolescent women with relatively fewer (less than four) partners had lower risk because they are usually younger and are seeing younger male partners, who are unlikely to be sources of infection pathogens. On the other hand, women who had a larger number of sexual partners (more than seven) are likely to be more mature and more cognizant of the STI risk thus using more prophylactics. This may explain the apparent lack of linear increase of STI risk in women with larger number of sexual partners. Additionally, we could not rule out the possibility that these women are more experienced in partner selection. As suggested by the associate editor, we also analyzed the data using an approach similar to that of Du and Ma (2010), using the function sshzd in the R package gss. Initially, we tried the model log λ_i(t, N_PARTNERS, AGE, N_SEX) = g(t, N_PARTNERS, AGE, N_SEX) + b_i with a fully nonparametric regression function g(·) and a random effect b_i, as in Du and Ma, but we had numerical diffculties fitting this model. Hence, in an effort to provide a best comparison between the Du-Ma’s fully nonparametric approach and our semiparametric additive model approach, we fit a fully nonparametric model of the form log λ_i(t, N_PARTNERS, AGE, N_SEX) = g(t, N_PARTNERS, AGE, N_SEX) without the random effect b_i. Both analysis point to a generally positive association between number of partners and infection risk. And both models show an attenuation of infection risk when partner number is greater than five (plot not shown). Therefore, the more parsimonious semiparametric model has adequately captured the trend demonstrated by the full nonparametric model.

Figure 2.

Smoothing spline estimate of the number of lifetime partner effect using model (14) in the sexually transmitted infection study

The estimated effects of the covariates are summarized in Table 2. The STI infection risk for subjects with a higher number of unprotected sex in past two months was higher than those with fewer unprotected sex events although the effect did not reach a level of statistical significance (p-value=0.89). White adolescents tended to have a lower (0.436) STI risk than black teens (p-value<0.001). Teens having later sexual debut at higher age had higher infection rates than others having sexual debut at younger age (p-value<0.001). Again, this seemingly paradoxical observation may reflect the STI risk level presented by the male partners. Although the current study is unable to fully disseminate the reasons behind the observations due to the lack of male partner data, the findings nonetheless highlight the complexity of the STI risk in adolescents and the need for a more careful examination of the behavioral markers for STI screening.

Table 2. Regression Coefficient estimates from fitting the semiparametric frailty model (Time to sexually transmitted infection study.

Covariate	Race (white)	Age	N_Sex
Estimate	−0.831	0.326	0.0008
SE	(0.240)	(0.047)	(0.0057)
p-value	<0.001	<0.001	0.89

Age: Age at first sex.

7. Discussion

In this paper, we proposed a semiparametric frailty model for examination of semiparametric covariate effects in correlated survival data. Nonparametric functions are estimated using smoothing splines. Since the observed likelihood involves integration with no closed form, the Laplace method is used to approximate the likelihood. We developed the DPPL method to estimate all of the model components, including the semiparametric covariate effects, the the variance components, and the smoothing parameter within a unified framework using an augmented working frailty model with parametric covariate effects. With the proposed approach, the smoothing spline estimators of the nonparametric functions are obtained as a linear combinations of fixed effects and random effects, and the smoothing parameters are treated as extra variance components.

Alternatively, one may use the Gaussian quadratures or the MCMC-type method to integrate out the cluster-level random effects to reduce bias in some parameters. However, these calculations are computationally intensive. In addition, one still has to estimate the smoothing parameter. If REML is used to estimate the smoothing parameter, one also needs to deal with high dimensional integration resulted from the random effects associated with the smoothing spline estimator. And these numerical integration methods for the REML likelihood are not feasible. Additionally, inferences for model parameters using numerical integration remain challenging. A key advantage of the DPPL is that estimation and inference of all the model components are easily obtained within a parametric frailty model unified framework. Future research is needed to pursue these numerical integration methods and compare their performance with the DPPL method. For future research, we will consider bias correction similar to what was used by Lin and Breslow (1996) to improve the performance of the DPPL method.

For the STI application, our analysis showed a nonlinear effect of the number of partners on STI acquisition in adolescent women. This finding has painted a more nuanced picture of the partner effect: when the cumulative lifetime number of partners is relatively small, i.e., fewer than five, infection risk clearly increases with the number of partners; but when the number of partners is relatively large, infection risk no longer has proportional increase with the number of partners, possibly due to the increased prophylactic behaviors and immunological maturity in young women with more sexual experience (Ethier and Orr, 2007). As discussed in previous sections, while such a finding is not scientifically unexpected, the complexity of partner effect appears to point to the need of an algorithm to more effectively differentiate the STI risk for screening purposes. In search of meaningful STI screening indicators, the proposed semiparametric frailty model offers an indispensable statistical tool with the necessary modeling flexibility to assess the effects of potential screening variables.

Appendix I: The Derivatives of the Integrant of the Integrated Likelihood of the General Frailty Model

Consider the general frailty model ${\lambda }_i\left(t\right)={\lambda }_0\left(t\right)exp\{{\mathit{x}}_i^T\beta +{\mathit{w}}_i^T\gamma +{\tilde{\mathit{z}}}_i^T\tilde{\mathit{b}}\}$, where β is a vector of regression coeffcients and and ${\tilde{\mathit{z}}}_i$ is the covariate vector associated with the random effect $\tilde{\mathit{b}}$, $\tilde{\mathit{b}}$ is a vector of random effects following $N(0,\tilde{\mathit{D}})$ which may include b, a for the additive frailty model (2). Write the integrated loglikelihood as $\int exp\{-S(\tilde{\mathit{b}}\left)\right\}d\tilde{\mathit{b}}$, where

$$\mathbf{S}\left(\tilde{\mathit{b}}\right)=\sum _{\mathbf{i}=1}^{\mathbf{n}}\left[-{\delta }_{\mathbf{i}}\left\{\mathbf{log}\right({\lambda }_0\left({\mathbf{u}}_{\mathbf{i}}\right))+\sum _{\mathrm{j}=1}^{\mathrm{P}}{\mathit{x}}_{\mathbf{ij}}^{\mathbf{T}}\beta +{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\tilde{\mathit{z}}}_{\mathbf{i}}^{\mathbf{T}}\tilde{\mathit{b}}\}+{\Lambda }_0\left(u_i\right)e^{{\Sigma }_{j=1}^p{\mathit{x}}_{ij}^T\beta +{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\tilde{\mathit{z}}}_i^T\tilde{\mathit{b}}}\right]+\frac{1}{2}{\tilde{\mathit{b}}}^T{\tilde{\mathit{D}}}^{-1}\tilde{\mathit{b}}$$

$${\mathbf{S}}^{\prime }\left(\tilde{\mathit{b}}\right)=\sum _{i=1}^n\left[-{\delta }_i{\tilde{\mathit{z}}}_i+{\Lambda }_0\left(u_i\right)e^{{\mathit{x}}_i^T\beta +{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\tilde{\mathit{z}}}_i^T\tilde{\mathit{b}}}{\tilde{\mathit{z}}}_i\right]+{\tilde{\mathit{D}}}^{-1}\tilde{\mathit{b}}$$

$${\mathbf{S}}^{″}\left(\tilde{\mathit{b}}\right)=\sum _{i=1}^n{\Lambda }_0\left(u_i\right)e^{{\Sigma }_{j=1}^p{\mathit{x}}_{ij}\beta +{\mathit{w}}_{\mathit{i}}^{\mathit{T}}\gamma +{\tilde{\mathit{z}}}_i^T\tilde{\mathit{b}}}{\tilde{\mathit{z}}}_i{\tilde{\mathit{z}}}_i^T+{\tilde{\mathit{D}}}^{-1}$$

Appendix II: Algorithm for estimation

• Generate the new covariates ${\mathit{x}}_j^0$, B_j, and indicator matrix N_j for fitting the augment working frailty model (8).

• For fixed variance component ν and smoothing parameters τ, maximize the (7) with respect to β_j, a_j, γ, b using a Newton-Raphson algorithm.

• For estimate β_j, a_j, γ, b obtained from step (2), calculate the smoothing parameters and variance components using equation (12,12).

• Iterate between step (2) and (3) until convergence.