Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Sep 30.
Published in final edited form as: Stat Med. 2010 Sep 30;29(22):2310–2324. doi: 10.1002/sim.3977

Latent class joint model of ovarian function suppression and DFS for premenopausal breast cancer patients

Jenny J Zhang a,*,, Molin Wang b
PMCID: PMC3786368  NIHMSID: NIHMS509851  PMID: 20552577

Abstract

Breast cancer is the leading cancer in women of reproductive age; more than a quarter of women diagnosed with breast cancer in the US are premenopausal. A common adjuvant treatment for this patient population is chemotherapy, which has been shown to cause premature menopause and infertility with serious consequences to quality of life. Luteinizing-hormone-releasing hormone (LHRH) agonists, which induce temporary ovarian function suppression (OFS), has been shown to be a useful alternative to chemotherapy in the adjuvant setting for estrogen-receptor-positive breast cancer patients. LHRH agonists have the potential to preserve fertility after treatment, thus, reducing the negative effects on a patient’s reproductive health. However, little is known about the association between a patient’s underlying degree of OFS and disease-free survival (DFS) after receiving LHRH agonists. Specifically, we are interested in whether patients with lower underlying degrees of OFS (i.e. higher estrogen production) after taking LHRH agonists are at a higher risk for late breast cancer events. In this paper, we propose a latent class joint model (LCJM) to analyze a data set from International Breast Cancer Study Group (IBCSG) Trial VIII to investigate the association between OFS and DFS. Analysis of this data set is challenging due to the fact that the main outcome of interest, OFS, is unobservable and the available surrogates for this latent variable involve masked event and cured proportions. We employ a likelihood approach and the EM algorithm to obtain parameter estimates and present results from the IBCSG data analysis.

Keywords: joint modeling, latent class model, EM algorithm, masked event, cured proportion, breast cancer

1. Introduction

Breast cancer is the most common cancer and the second leading cause of cancer deaths (after lung cancer) among women in the US. About 25 per cent of all women diagnosed with breast cancer are premenopausal, making breast cancer the leading cancer in women of reproductive age [1]. Adjuvant therapy is often given to premenopausal breast cancer patients to prevent disease recurrence or progression after surgery. One common form of such adjuvant treatment is chemotherapy, which often causes premature ovarian failure leading to infertility in many young women with breast cancer [2]. In addition to its effects on fertility, the premature menopause induced by chemotherapy also causes negative side effects such as hot flashes, osteoporosis, and sexual dysfunction, all of which contribute to poorer patient quality of life and perception of health [3]. Thus, fertility preservation and the effects of adjuvant treatments on reproductive health are playing increasingly more important roles in the treatment decision-making process of young women diagnosed with breast cancer.

Much recent attention has been focused on another form of adjuvant therapy which induces temporary ovarian function suppression (OFS), luteinizing-hormone-releasing hormone (LHRH) agonists. LHRH agonists greatly reduce the production of oestrogen, achieved through LHRH receptor downregulation. Once off treatment, most patients resume their ovarian function unless they entered menopause during treatment. It has been shown that LHRH agonists are a useful alternative to chemotherapy for estrogen-receptor (ER)-positive patients in this population due to the reversibility and tolerability of its effects [4]. ER-positive patients have oestrogen receptors on their breast cancer cells. When oestrogen binds to those receptors, the cancer cells are stimulated to grow and replicate, leading to disease progression or recurrence. It follows that ER-positive breast cancer patients are expected to respond to treatments that block/reduce oestrogen, which is the mechanism of LHRH agonists.

LHRH agonists give young women with breast cancer an increased chance of preserving their fertility and reproductive health after adjuvant therapy. It is unclear though whether patients with lower underlying degrees of OFS after taking LHRH agonists may be at a higher risk for late breast cancer events. Patients’ underlying degrees of OFS after treatment will differ depending on their individual characteristics, and those with lower degrees may have a higher chance of preserving their reproductive health. However, if having lower degrees of OFS puts those patients at a higher risk for disease recurrence, then the appeal of the treatment would be greatly reduced. Thus, there is interest in the association between a premenopausal patient’s underlying degree of OFS and her disease-free survival (DFS) after LHRH agonists. The availability of such information could affect how this patient population is treated in the adjuvant setting, especially those patients who still wish to become pregnant after treatment. In this paper, we present a novel approach for the analysis of data from International Breast Cancer Study Group (IBCSG) Trial VIII to investigate the extent to which OFS is associated with DFS in premenopausal breast cancer patients.

As OFS is not easily quantifiable or measurable, it is natural to consider a latent class variable to represent a patient’s underlying degree of OFS. As part of the study, IBCSG Trial VIII carefully collected monthly menses data during the first 36 months and then every 6 months thereafter; details are given in [5]. As a common indicator of OFS is the cessation of menses, we propose to obtain information about latent class membership for OFS through a set of time-to-event surrogates extracted from the menses data. Specifically, the menses data contain information about the times to treatment-induced amenorrhea (TIA), menopause, and recovery of menses; TIA is defined to be a cessation of menses that is induced by adjuvant therapy. The process of TIA, however, is complicated by the fact that menopause may also occur. As both TIA and menopause result in an observed cessation of menses (the failure), the event leading to the failure is masked unless the patient recovers her menses after treatment. Although all patients are expected to experience TIA after taking LHRH agonists, we allow our method to be more general by including a cure rate for TIA, which would equal 1 if all patients experienced TIA. Furthermore, not all patients who experience TIA will recover their menses after treatment, thus, we also assume the existence of a cured proportion with respect to recovery. No cure rate is assumed for menopause since all patients are expected to experience menopause eventually. To clarify, we use the term ‘cured’ in reference to a subset of patients who are assumed to be unsusceptible to some event. Thus, analysis of this IBCSG data set poses several statistical challenges, including latent variables, masked event, and cured proportions.

As our main interest is in the association of OFS and DFS, we propose to jointly model the latent class variable and DFS. Joint modeling has garnered much interest in recent years. Initial work was done mostly in the area of longitudinal studies to characterize the association between the longitudinal and event time processes (e.g. [612]). More recently, joint models have also been applied to social science data, where the interest is in the association between a latent variable and time-to-event data (e.g. [1317]); we refer to such models as latent class joint models. Specifically, the Cox model is extended to include a latent class variable as a predictor of time-to-event while using information about latent class membership from a latent class regression model.

In this paper, we propose a latent class joint model (LCJM) to investigate the association between a premenopausal breast cancer patient’s underlying degree of OFS and DFS using the IBCSG Trial VIII data, where information on class membership is obtained indirectly through a set of menses-related, time-to-event surrogates involving masked event and cured proportions. In Section 2, we describe the IBCSG Trial VIII data in greater detail. In Sections 3 and 4, we present our LCJM and detail the likelihood formation and parameter estimation via the EM algorithm, respectively. In Section 5, we present the results from fitting the proposed model to the IBCSG Trial VIII data, and close with some discussion in Section 6.

2. IBCSG Trial VIII

In IBCSG Trial VIII [18], premenopausal, node-negative breast cancer patients were randomized to four different treatment arms and stratified according to ER status, whether radiotherapy was planned after surgery, and institution. Patients were randomized to I: no adjuvant therapy, II: goserelin for 24 months, III: six cycles of CMF (chemotherapy), and IV: six cycles of CMF followed by goserelin for 18 months, where goserelin is an LHRH agonist. The primary endpoint was DFS, defined to be the time from the date of randomization to any recurrent disease, appearance of a secondary primary cancer, or death, whichever occurred first. The no adjuvant therapy arm was discontinued less than two years into the study, and the median follow-up of the trial was 7 years. As our interest lies in the association between OFS and DFS after LHRH agonists, we will focus on the goserelin arm of IBCSG Trial VIII.

The following clinically justified assumptions are made concerning the three time-to-event surrogates used to determine latent class membership (i.e. times to TIA, menopause, and recovery). We assume that TIA cannot occur after treatment end (U) and recovery of menses can only occur after U, whereas menopause and DFS can occur any time. We further assume that observing recovery of menses would successfully unmask the event leading to cessation before U as TIA. Otherwise, the observed cessation of menses before U could be due to either TIA or menopause. To specify a more general model, we assume the possible existence of patients who are unsusceptible (i.e. cured) to TIA; as TIA can only occur before U, if no failure is observed before U, then the subject can be assumed to be cured with respect to TIA. In our IBCSG trial application, however, we expect TIA to occur in all patients taking LHRH agonists. Moreover, menopause may also occur after TIA, with or without recovery first, and recovery after menopause is not possible.

3. The model

Figure 1 is a schematic of our LCJM, which consists of three components: (i) latent class membership; (ii) menses-related, time-to-event surrogates for the latent class; and (iii) hazard for DFS. Components (i) and (ii) are what is usually referred to as the latent class model (LCM). We propose to use Weibull proportional hazards (PH) models for (ii) and a semiparametric PH model for (iii).

Figure 1.

Figure 1

LCJM schematic.

Let n and K denote the total number of subjects and latent classes, respectively, and cik be the indicator that subject i belongs to class k for i = 1,…, n and k = 1,…, K. We adopt the generalized logit [19] to model the relationship between latent class (i.e. degree of OFS) and the (p × 1) vector of covariates, Wi:

pik=P(cik=1|wi)=exp(wiηk)s=1Kexp(wiηs), (1)

where pik is the probability that subject i belongs to class k and ηk is a vector of regression parameters for class k with ηK = 0. Note that all vectors are column vectors and a prime, ′, denotes the transpose.

For each class, we model the vector of menses-related, time-to-event surrogates, Y = (T1, T2, T3)′, where T1 is time to TIA with support [0, U], T2 is time to menopause with support [0, ∞), and T3 is time to recovery of menses with support (U, ∞). Recall that U is treatment end. Let fTm (tm|cik = 1) and FTm (tm|cik = 1) denote the conditional probability and cumulative distribution functions of Tm, m = 1, 2, 3, respectively. We assume that the time-to-event surrogates are mutually independent conditional on class membership, that is

fY(yi|cik=1)=m=13fTm(tmi|cik=1). (2)

Let τ1i and τ2i denote the cure indicators for TIA and recovery, respectively (0 = cured; 1 = uncured) for subject i, and π1ki = P1i = 1|cik = 1) and π2ki = P2i = 1|cik = 1). Note that τ2i = 1 implies τ1i = 1 since recovery is undefined when the patient is cured with respect to TIA. Letting STm (․) denote the survivor function for Tm, we adopt the following model specifications for T1 and T3:

ST1(t1i|cik=1)=π1kiST1(t1i|τ1i=1,cik=1)+(1π1ki),

and

ST3(t3i|cik=1)=π2kiST3(t3i|τ2i=1,cik=1)+(1π2ki).

As TIA cannot occur after treatment end (U), we model the conditional distribution of time to TIA, T1, using a truncated Weibull PH model with hazard

λT1(t1i|t1iui,τ1i=1,cik=1)={a1d1(a1t1i)d11exp[(a1t1i)d1]exp[(a1t1i)d1]exp[(a1ui)d1]}exp(αk),

where the (K × 1) parameter vector α contains the effect of the class variable on the hazard of TIA and (a1, d1) are the Weibull parameters. No truncation is needed for time to menopause, T2, or time to recovery, T3, and we model the distribution of T2 and the conditional distribution of T3 using Weibull PH models with respective hazards

λT2(t2i|cik=1)a2d2(a2t2i)d21exp(βk),

and

λT3(t3i|τ2i=1,cik=1)=a3d3(a3t3i)d31exp(γk),

where β and γ are the (K × 1) parameter vectors containing the effects of the class variable on the hazard of menopause and recovery of menses, respectively, and (a2, d2) and (a3, d3) are the respective Weibull parameters. For identifiability, αK = βK = γK = 0, where K is the total number of latent classes. Note that the hazard for T3 is conditional on being uncured with respect to recovery, and since we assume that recovery can only occur after U, λT3 (t3i2i = 1, cik = 1) is implicitly bounded below by U.

We assume conditional independence between Wi and Yi given class, i.e. fY(yi |cik = 1, wi) = fY(yi | cik = 1), which implies that the latent classes (i.e. degree of OFS) serve as a summary of the information in the menses-related, time-to-event surrogates. This assumption is an important feature of the LCM and simplifies both the modeling and estimation procedures. One way to investigate this assumption is to add the variables in Wi one at a time for each of the time-to-event surrogates. More specifically, let the class-specific hazards depend on a potentially biasing covariate, ϖi, such that λTm (․|ci, ϖi) = λ0m(․)exp(ρmϖi + ϕm,ci), where ci takes one of the values 1,…, K denoting latent class, and ρm and ϕm,ci are surrogate-specific parameters (i.e. for Tm, m = 1, 2, 3). Then, perform likelihood ratio tests (LRTs) comparing this extended model with the original LCM. If the LRTs are non-significant, then the assumption holds. A similar method was used in [15].

Figure 2 defines the five possible observed data configurations for IBCSG Trial VIII, denoted by φi = 1,…, 5 for the ith subject. Let Xi be the time to cessation of menses (if a cessation is observed) and ζi be the indicator of the event leading to failure (0 = TIA; 1 = menopause). If Xi is observed before treatment end (U) and cannot be identified as caused by either TIA or menopause (i.e. recovery is not observed), then τ1i, τ2i, and ζi are all unknown and we define φi = 1. If Xi is observed before U and recovery is also observed, then Xi is unmasked as TIA. It follows directly that τ1i = τ2i = 1 and ζi = 0, and we define φi based on whether (φi = 3) or not (φi = 2) menopause is observed after recovery. If Xi is not observed before U, then τ1i = 0 and we define φi according to whether (φi = 5) or not (φi = 4) menopause is observed after U. We assume that all censoring of subjects occur after U, which is the case for IBCSG Trial VIII.

Figure 2.

Figure 2

Possible observed data configurations for IBCSG Trial VIII.

Finally, we propose a PH model conditional on latent class membership and the vector of covariates, Zi, with unspecified baseline hazard, λ0*(), for DFS time T*. The hazard at time ti* for a DFS event is modeled as:

λT*(ti*|cik=1,zi)=λ0*(ti*)exp(κZi+νk), (3)

where ν is the (K × 1) parameter vector containing the effect of the class variable on the hazard of DFS and κ is a vector of regression parameters corresponding to zi. Letting Gi denote the censoring time for DFS, the observed variables are Vi=min(Ti*,Gi) and Δi=1(Ti*Gi). Under the assumption of noninformative censoring, the distribution of (Vi, Δi) is proportional to

{λ0*(υi)exp(κzi+νk)}δiexp{Λ0*(υi)exp(κzi+νk)},

where Λ0*() is the cumulative baseline hazard for DFS.

For a fixed number of latent classes, K, let Θ={π1,π2,η,a1,d1,α,a2,d2,β,a3,d3,γ,λ0*(),κ,ν} denote the complete parameter vector, where π1, π2, α, β, γ, and ν are all (K × 1) vectors and η is a (K × p) matrix.

4. Estimation

4.1. Parameter estimation

We employ a likelihood approach to estimate the parameters, Θ, for a fixed number of latent classes, K. As direct maximization of the observed likelihood is difficult, we employ the Expectation-Maximization (EM) algorithm [20] to estimate Θ. The complete data likelihood is

Lcomp(Θ|P)=r=15i=1npikf(Pi|Θ,φi=r){λ0*(υi)exp(κzi+νk)}δi×exp{Λ0*(υi)exp(κzi+νk)},

where pik is given in (1), Pi = {Yi, φi, Wi, Zi, τ1i, τ2i, ζi, ξi, ϑi, ci}, ξi = 1(T2i < U), ϑi = 1(T2i > Gi), P = {Pi, Vi, Δi, i = 1,…, n}, and φi indicates one of the five possible observed data configurations defined in Figure 2. We assume that, given the degree of OFS, there is no additional effect of the time-to-event surrogates on DFS.

We carefully consider each data configuration, f(Pi |Θ, φi = r), r = 1,…, 5, given in Appendix A. Lcomp(Θ|P) can be factored into seven pieces, each containing a distinct set of parameters in Θ, such that Lcomp(Θ|P)=Lc1(π1|P)Lc2(π2|P)Lc3(a1,d1,α|P)Lc4(a2,d2,β|P)×Lc5(a3,d3,γ|P)Lc6(η|P)Lc7(λ0*(),κ,ν|P), where

Lc1(π1|P)=i:φi=1π1ki{τ1iζi+(1ζi)}(1π1ki)(1τ1i)ζii:φi{2,3}π1kii:φi{4,5}(1π1ki),Lc2(π2|P)=i:φi=1π2kiτ2i(1ζi)(1π2ki)(1τ2i)(1ζi)i:φi{2,3}π2ki,Lc3(a1,d1,α|P)=i:φi=1ST1(xi|t1iui,τ1i=1,cik=1)τ1iζifT1(xi|t1iui,τ1i=1,cik=1)(1ζi)×i:φi{2,3}fT1(t1i|t1iui,τ1i=1,cik=1),Lc4(a2,d2,β|P)=i:φi=1fT2(xi|cik=1)ζifT2(t2i|cik=1)(1ζi)i:φi{2,4}ST2(gi|cik=1)i:φi{3,5}fT2(t2i|cik=1),Lc5(a3,d3,γ|P)=i:φi=1ST3(t2iui|τ2i=1,cik=1)τ2i(1ζi)(1ξi)(1ϑi)ST3(giui|τ2i=1,cik=1)τ2i(1ζi)ϑi×i:φi{2,3}fT3(t3i|τ2i=1,cik=1),Lc6(η|P)i=1n{exp(wiηk)s=1Kexp(wiηs)},

and Lc7(λ0*(),κ,ν|P)

=i=1n{λ0*(υi)exp(κzi+νk)}δiexp{Λ0*(υi)exp(κzi+νk)},=j=1Ji=1n{λ0*(t(j)*)exp(κzi+νk)}δi1(Vi=t(j)*)exp{λ0*(t(j)*)exp(κzi+νk)}1(Vit(j)*),

where t(1)*<<t(J)* denotes the J distinct DFS times. For Lc7(․), we treat DFS time as discrete to estimate the baseline hazard, λ0*(), at each event time.

The E-step of the EM algorithm computes the conditional expectation of the complete data log-likelihood, lcomp(Θ|P)= log{Lcomp(Θ|P)}, with respect to the latent variables, H = {τ1i, τ2i, ζi, ξi, ϑi, ci, i =1,…, n}, given the observed data, O = {Xi, φi, Vi, δi, Wi, Zi, i = 1,…, n}, and the latest updated parameter estimates, Θ(L). For ease of presentation, we let ωik[l,φi], l = 1,…, 11, denote the needed conditional expectations for φi in the E-step, which are defined and derived in Appendix B.

The M-step updates the estimate of Θ by maximizing the expected complete data log-likelihood,

E[lcomp(Θ|P,Θ(L)]=E[lc1(π1|P,Θ(L))]+E[lc2(π2|P,Θ(L))]+E[lc3(a1,d1,α|P,Θ(L))]+E[lc4(a2,d2,β|P,Θ(L))]+E[lc5(a3,d3,γ|P,Θ(L))]+E[lc6(η|P,Θ(L))]+E[lc7(λ0*(),κ,ν|P,Θ(L))],

with respect to Θ for fixed ωik[l,φi]. Detailed formulation of E[lcomp(Θ|P, Θ(L))] is given in Appendix C.

Closed-form solutions can be obtained for π1, π2, and λ0*(), respectively, in the M-step as

π̂1k=i:φi=1(ωik[3,1]+ωik[4,1])+i:φi{2,3}ωik[1,{2,3}]i:φi=1(ωik[3,1]+ωik[4,1]+ωik[5,1])+i:φi1ωik[1,{2,3,4,5}],π̂2k=i:φi=1ωik[6,1]+i:φi{2,3}ωik[1,{2,3}]i:φi=1(ωik[6,1]+ωik[7,1])+i:φi{2,3}ωik[1,{2,3}],

and

λ̂0*(t*)=i=1nδi1(Vi=t*)iRjexp(κzi)k=1Kωik[1,{all}]exp(νk),

where Rj is the risk set at t(j)*,ωik[1,{2,3}]=ωik[1,2]+ωik[1,3],ωik[1,{2,3,4,5}]=ωik[1,2]+ωik[1,3]+ωik[1,4]+ωik[1,5], and ωik[1,{all}]=ωik[1,1]+ωik[1,2]++ωik[1,5]. The remaining parameters are estimated by maximizing the corresponding expected complete data log-likelihoods using a quasi-Newton method.

4.2. Variance estimation

Variance estimates for Θ can be approximated using the following result from [21]:

2lcomp(Θ|O)Θ2=H2lcomp(Θ|P)Θ2P(H|O,Θ)dHvar{lcomp(Θ|P)Θ}, (4)

where the integration is with respect to the missing data, H. Let bf denote the multiple imputations from P(H|O, Θ̂), which can be generated based on the conditional expectations in Appendix B. We can then approximate the first and second terms on the right-hand side of (4) for F =10000 imputations by

1Ff=1F2lcomp(Θ|O,bf)Θ2,

and

1Ff=1F{lcomp(Θ|O,bf)Θ}2{1Ff=1Flcomp(Θ|O,bf)Θ}2,

respectively [22].

5. Results from IBCSG Trial VIII

Using the algorithm described in Section 4, we fit the proposed LCJM to the goserelin arm of IBCSG Trial VIII. There are a total of 292 patients eligible for analysis; 152 have masked events before treatment end (φ = 1) of which 46 had a DFS event, and 140 are unmasked as TIA (φ∈ {2, 3}) of which 33 had a DFS event. All patients experienced cessation before treatment end, thus, there are no patients in data configurations φ = 4 or 5. To avoid speculations concerning the influence of DFS information on determining the latent classes (i.e. degree of OFS), we establish the LCM before incorporating the DFS data [15]. The LCM models the relationship between the time-to-event surrogates, covariates Wi, and latent class membership through (1) and (2). Thus, we fit the LCM first to determine the classes/degrees of OFS, then fit the LCJM incorporating DFS. We consider the continuous covariate age at study entry (centered at the mean of 45 years), the binary covariate ER status (1 = positive; 0 = negative), and their interaction. Given the strong biological rationale behind the pharmacodynamics of LHRH agonists to support that goserelin is expected to induce TIA in all patients, we fix π1 to be 1 for all models fit to the IBCSG data.

We employ the Bayesian information criterion (BIC) to determine whether the 2-class or 3-class LCM is better; the BIC imposes a relatively larger penalty on the number of parameters than other criteria, e.g. Akaike’s information criterion (AIC). The BIC values for the 2-class and 3-class models, ignoring covariates, are 505.730 and 972.200, respectively, which strongly suggests that the 2-class model is better. Neither ER status nor its interaction with age was found to be significant predictors of latent class membership, which is expected since a patient’s underlying degree of OFS should not be related to whether or not her tumors have hormone receptors. Also as expected, age at entry is significantly associated with degree of OFS and is thus included as a predictor in the LCM.

We also investigated the conditional independence assumption, fY(yi | ci, wi) = fY(yi | ci), by adding the variable age at entry to each of the time-to-event surrogates as described in Section 3. The likelihood ratio test (LRT) statistic (p-value) on 3 degrees of freedom comparing this extended model with the original LCM fitted above is 7.09 (p = 0.069), which supports our conditional independence assumption and implies that our latent class variable (OFS) serves as a summary of the time-to-event surrogates (times to TIA, menopause, and recovery).

Given the above results, all LCJMs considered will include age at entry as a predictor of class membership in the latent class part. However, different predictors will be considered for the DFS part of the model (i.e. OFS, age, ER, and age–ER interaction). As our main interest is in the association between OFS and DFS, we keep OFS in the model regardless of its significance and perform variable selection on the remaining covariates. The LRT statistic on two degrees of freedom for the joint model with DFS predictors OFS, age, ER, and age–ER interaction vs OFS and ER only is 1.367 (p = 0.459). Thus, the final joint model includes only OFS and ER status in the DFS part. Table I gives the parameter estimates for the final joint model. Note that the estimates of π2 and the latent class regression parameters for the joint model are very similar to those obtained from the LCM, i.e. not including DFS (results not shown).

Table I.

LCJM estimates.

Class 1 Class 2
Proportion of patients in each class (k) 0.393 0.607
Probability of experiencing TIA (π̂1) 1 (fixed*) 1 (fixed*)
Probability of experiencing recovery (π̂2) 0.217 (0.067) 0.839 (0.033)
Latent class membership (class 1 vs class 2):
Estimate (p-value)
Intercept (η̂11) −1.554 (0.009)
Mean-centered age (η̂12) 0.662 (<0.001)
DFS
Estimate (p-value) Hazard ratio
ER-status (κ̂1) −0.298 (0.064) 0.732
class 1 vs class 2 (ν̂1) −0.086 (0.700) 0.918
*

As LHRH agonists are expected to induce TIA in all patients, we fix π1 to be 1.

Figure 3 shows the estimated survival curves for TIA, menopause, and recovery of menses by class. Note the different time-scales, where times to TIA and menopause are both from study entry whereas time to recover is from treatment end, which is 2 years for the goserelin arm. From Figure 3 and the parameter estimates in the latent class membership model in Table I, we can identify the defining characteristics of patients belonging to class 1 and class 2. Class 1 patients are older and have a shorter time to menopause than class 2 patients. Both classes have similar time to TIA, which is expected given the pharmacodynamics of LHRH agonists. Moreover, almost 80 per cent (i.e. (1 − π̂21) × 100) of patients in class 1 never recover their menses (i.e. they are cured with respect to recovery or, in other words, they entered menopause before recovery) compared to about 16 per cent (i.e. (1 − π̂22) × 100) in class 2. Conditional on being uncured/susceptible to recovery, class 1 patients take a longer time to recover their menses than class 2. These defining characteristics lead us to label the underlying degree of OFS of class 1 patients as ‘high’ and of class 2 patients as ‘low’.

Figure 3.

Figure 3

Estimated survival curves for TIA, menopause, and recovery of menses.

Given these latent class labels, the parameter estimate for latent class in the DFS part of the LCJM implies a non-significant 8 per cent decrease in the risk of DFS for patients with a ‘high’ degree of OFS compared to those with a ‘low’ degree of OFS. This result implies that lower degrees of OFS, or higher circulating levels of estrogen, do not significantly increase a patient’s risk of late breast cancer events after LHRH agonists. Moreover, the parameter estimate for ER suggests that having an ER-positive tumor decreases the patient’s hazard of DFS by 25 per cent compared to having an ER-negative tumor, holding class constant (p = 0.064). As mentioned in Section 1, since goserelin is an LHRH agonist, we expect better results for ER-positive patients. Similar trends were reported in the IBCSG Trial VIII clinical paper [17]. These results further support that LHRH agonists may be a useful alternative to chemotherapy for premenopausal, ER-positive breast cancer patients, especially those who are concerned about their reproductive health and fertility after treatment.

The first row of Figure 4 shows the estimated DFS curves by class, holding ER status constant, whereas the second row shows the estimated DFS curves by ER status, holding class constant. Futhermore, we can estimate the proportion of patients in class k by k=(1/n)i=1npik. About 40 per cent of patients are in the ‘high’ OFS class and 60 per cent are in the ‘low’ OFS class. Each class also has some unambiguously assigned patients since the maximum value of ik for both classes is very close to one. Similar to [15], the proportional hazards assumption can be checked graphically by simulating the latent class variables from the conditional distribution given the observed variables and the final estimates in Table I. In Figure 5, we plot the logarithm of the estimated integrated hazard for the two classes averaged over 50 simulations. We see that there is no evidence against the proportional hazards assumption between classes since the distance between the two curves appears to be independent of time.

Figure 4.

Figure 4

Estimated DFS curves by class and ER status.

Figure 5.

Figure 5

Proportional hazards assumption.

6. Discussion

Joint modeling has become more widely used due to its ability to use data more efficiently, resulting in less biased estimates. In this paper, we proposed a latent class joint model to assess the association between a premenopausal breast cancer patient’s underlying degree of OFS and DFS after adjuvant treatment with an LHRH agonist. Our joint model obtains information about latent class membership indirectly through a set of menses-related, time-to-event surrogates and accommodates the statistical challenges of masked event and cured proportions present in those surrogates.

As pointed out by a referee, the computational stability of the estimates in our LCJM is a very important issue. In practice, local convergence of an algorithm is usually sufficient when the reliability of the results is checked by starting the algorithm from several different starting values. In our LCJM analysis, we tried three potential starting values for each parameter; we randomly sampled 30 sets of starting values from the 317 possible combinations of the 17 parameters in our model, and then obtained estimates for each sampled set of starting values. The potential starting values for each parameter chosen included both positive and negative values and had relatively broad numeric ranges as well. The reliability of the estimates were checked for each sampled set of starting values. Out of 30 sets of starting values simulated for the final LCJM, only two sets gave results that were slightly inconsistent computationally, but not in terms of parameter interpretations; zero sets gave inconsistent answers to whether OFS is associated with DFS. In addition, given the complexity of our model, we recommend using starting values based on some estimates from simpler models (e.g. the LCM) and also to utilize any available biological knowledge/rationale.

The proposed method is dependent on the presence of a reasonable number of unmasked failures. In the IBCSG study, a large amount of missing/masked data could make the likelihood maximization difficult. About 50 per cent of the cessations are unmasked in the goserelin arm and the resulting estimates were quite stable. In the case where there are too few unmasked cessations, a simpler model could be fit where TIA and menopause are not distinguished, i.e. times to TIA and menopause could be collapsed into the more general ‘time to cessation’ outcome. However, this would reduce the amount of information used to determine latent class membership unless other surrogate measurements for degree of OFS are available. For IBCSG Trial VIII, whose primary objective was to study how premenopausal, node-negative breast cancer patients respond to combination therapy (arm IV), no other surrogate measures were recorded, which is a limitation of the data set for this analysis.

Acknowledgements

The authors thank the patients, physicians, nurses, and data managers who participated in the International Breast Cancer Study Group (IBCSG) Trial VIII. They further acknowledge support from the United States National Cancer Institute (CA-75362) and the United States National Institute of Health Cancer Training Grant (Jing J. Zhang). They express their gratitude to Richard Gelber, Robert Gray, and Ann Partridge for their assistance throughout. They also thank the editor and referee for their insightful comments and suggestions.

Appendix A: Formulations of likelihood contribution for each data configuration: f(pi|Θ, φi =r), r = 1,…, 5 (Section 4.1)

f(Pi|Θ,φi=1)=[fT2(xi|cik=1)(1π1ki)(1τ1i)π1kiτ1iST1(xi|t1iui,τ1i=1,cik=1)τ1i]ζi×[fT1(xi|t1iui,τ1i=1,cik=1)π1ki(1π2ki)(1τ2i)π2kiτ2i×fT2(t2i|cik=1)ST3(t2iui|τ2i=1,cik=1)τ2i(1ξi)(1ϑi)ST3(giui|τ2i=1,cik=1)τ2iϑi](1ζi)f(Pi|Θ,φi=2)=fT1(t1i|t1iui,τ1i=1,cik)π1kifT3(t3i|τ2i=1,cik)π2kiST2(gi|cik=1),f(Pi|Θ,φi=3)=fT1(t1i|t1iui,τ1i=1,cik)π1kifT3(t3i|τ2i=1,cik=1)π2kifT2(t2i|cik=1),f(Pi|Θ,φi=4)=(1π1ki)ST2(gi|cik=1),

and

f(Pi|Θ,φi=5)=(1π1ki)fT2(t2i|cik=1).

Appendix B: Definition and derivation of conditional expectations in the E-step (Section 4.1)

For notational simplicity, we let ST1 (․|ci) = ST1 (․|t1iui, τ1i = 1, cik = 1), fT1 (․|ci) = fT1 (․|t1iui, τ1i = 1, cik = 1), ST3 (․|ci) = ST3 (․|τ2i = 1, cik = 1), fT3 (․|ci) = fT3 (․|τ2i = 1, cik = 1), and ST*(ti*|ci,zi)=ST*(ti*|cik=1,zi). Recall that ωik[l,φi] denotes the conditional expectations for φi, l = 1,…, 11. Expectations containing integrals are evaluated using numerical integration.

ωik[1,1]=P(Cik=1|xi,T*>ti*,wi,zi)=Akik=1KAki,whereAki=P(Cik=1|wi)ST*(ti*|ci,zi)[(1π1ki)fT2(xi|ci)+π1kiπ2kiST1(xi|ci)fT2(xi|ci)+π1kiπ2kifT1(xi|ci){FT2(ui|ci)FT2(xi|ci)}+π1kiπ2kifT1(xi|ci)uigifT2(t2|ci)ST3(t2ui|ci)dt2+π1kiπ2kifT1(xi|ci)ST2(gi|ci)ST3(giui|ci)+π1ki(1π2ki)ST1(xi|ci)fT2(xi|ci)+π1ki(1π2ki)fT1(xi|ci){FT2(ui|ci)FT2(xi|ci)}+π1ki(1π2ki)fT1(xi|ci){FT2(gi|ci)FT2(ui|ci)}+π1ki(1π2ki)fT1(xi|ci)ST2(gi|ci)],ωik[1,2]=P(Cik=1|τ2i=1,ζi=0,T1=xi,T2>gi,T3=t3i,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)ST2(gi|ci)π2kifT3(t3i|ci)k=1KP(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)ST2(gi|ci)π2kifT3(t3i|ci),ωik[1,3]=P(Cik=1|τ2i=1,ζi=0,T1=xi,T2=t2i,T3=t3i,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)fT2(t2i|ci)π2kifT3(t3i|ci)k=1KP(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)fT2(t2i|ci)π2kifT3(t3i|ci),ωik[1,4]=P(Cik=1|τ1i=0,T2>gi,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)(1π1ki)ST2(gi|ci)k=1KP(Cik=1|wi)ST*(ti*|ci,zi)(1π1ki)ST2(gi|ci),ωik[1,5]=P(Cik=1|τ1i=0,T2=t2i,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)(1π1ki)fT2(t2i|ci)k=1KP(Cik=1|wi)ST*(ti*|ci,zi)(1π1ki)fT2(t2i|ci),ωik[2,1]=P(Cik=1|O,Θ(L))E(ζi|cik=1,O,Θ(L))=P(Cik=1,ζi=1|xi,T*>ti*,wi,zi)=ωik[4,1]+ωik[5,1],

where

ωik[4,1]=P(Cik=1|O,Θ(L))E(τ1iζi|cik=1,O,Θ(L))=P(Cik=1,τ1i=1,ζi=1|xi,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)π1kiST1(xi|ci)fT2(xi|ci)k=1KAki,

and

ωik[5,1]=P(Cik=1|O,Θ(L))E((1τ1i)ζi|cik=1,O,Θ(L))=P(Cik=1,τ1i=0,ζi=1|xi,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)(1π1ki)fT2(xi|ci)k=1KAki.ωik[3,1]=P(Cik=1|O,Θ(L))E((1ζi)|cik=1,O,Θ(L))=P(Cik=1,ζi=0|xi,T*>ti*,wi,zi)=Bik=1KAki,

where

Bi=P(Cik=1|wi)ST*(ti*|ci,zi){π1ki(1π2ki)fT1(xi|ci)ST2(gi|ci)+π1kiπ2kifT1(xi|ci)×[{FT2(ui|ci)FT2(xi|ci)}+uigifT2(t2|ci)ST3(t2ui|ci)dt2+ST2(gi|ci)ST3(giui|ci)]}.ωik[6,1]=P(Cik=1|O,Θ(L))E(τ2i(1ζi)|cik=1,O,Θ(L))=P(Cik=1,τ2i=1,ζi=0|xi,T*>ti*,wi,zi)=Dik=1KAki,

where

Di=P(Cik=1|wi)ST*(ti*|ci,zi)π1kiπ2kifT1(xi|ci)×[{FT2(ui|ci)FT2(xi|ci)}+uigifT2(t2|ci)ST3(t2ui|ci)dt2+ST2(gi|ci)ST3(giui|ci)].ωik[7,1]=P(Cik=1|O,Θ(L))E((1τ2i)(1ζi)|cik=1,O,Θ(L))=P(Cik=1,τ2i=0,ζi=0|xi,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)ST2(gi|ci)(1π2ki)k=1KAki,ωik[8,1]=P(Cik=1|O,Θ(L))E(τ2i(1ζi)ϑi|cik=1,O,Θ(L))=P(Cik=1,τ2i=1,ζi=0,ϑi=1|xi,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)ST2(gi|ci)π2kiST3(giui|ci)k=1KAki,ωik[9,1]=P(Cik=1|O,Θ(L))E(τ2i(1ζi)(1ξi)(1ϑi)|cik=1,O,Θ(L))=P(ci,τ2i=1,ζi=0,ξi=0,ϑi=0|xi,T*>ti*,wi,zi)=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(xi|ci)π2kiuigifT2(t2|ci)ST3(t2ui|ci)dt2k=1KAki,ωik[10,1]=P(Cik=1|O,Θ(L))E((1ζi)logfT2(t2|ci)|cik=1,O,Θ(L))=P(Cik=1,ζi=0|O,Θ(L))E(logfT2(t2|ci)|cik=1,ζi=0,O,Θ(L))=ωik[3,1]xilogfT2(t2|ci)fT2(t2|ci,ζi=0)dt2=ωik[3,1]xilogfT2(t2|ci)M1i+M2i+M3ixiuiM1idt2+uigiM2idt2+giM3idt2dt2=ωik[3,1][xiuilogfT2(t2|ci)M1idt2+uigilogfT2(t2|ci)M2idt2+gilogfT2(t2|ci)M3idt2xiuiM1idt2+uigiM2idt2+giM3idt2],

where

M1i=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(t1i|ci)fT2(t2|ci),M2i=M1i{(1π2ki)+π2kiST3(t2ui|ci)},

and

M3i=M1i{(1π2ki)+π2kiST3(giui|ci)}.

It is straightforward to show that

xiuilogfT2(t2|ci)M1idt2+uigilogfT2(t2|ci)M2idt2+gilogfT2(t2|ci)M3idt2=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(t1i|ci)[xiuilogfT2(t2|ci)fT2(t2|ci)dt2+(1π2ki)uigilogfT2(t2|ci)fT2(t2|ci)dt2+π2kiuigilogfT2(t2|ci)ST3(t2ui|ci)fT2(t2|ci)dt2+{(1π2ki)+π2kiST3(giui|ci)}gilogfT2(t2|ci)fT2(t2|ci)dt2],

and

xiuiM1idt2+uigiM2idt2+giM3idt2=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(t1i|ci)[{FT2(ui|ci)FT2(xi|ci)}+(1π2ki){FT2(gi|ci)FT2(ui|ci)}+π2kiuigiST3(t2ui|ci)fT2(t2|ci)dt2+{(1π2ki)+π2ciST3(giui|ci)}{1FT2(gi|ci)}]=P(Cik=1|wi)ST*(ti*|ci,zi)π1kifT1(t1i|ci)[ST2(xi|ci)π2ki{ST2(ui|ci)ST3(giui|ci)ST2(gi|ci)}+π2kiuigiST3(t2ui|ci)fT2(t2|ci)dt2].ωik[11,1]=P(Cik=1|O,Θ(L))E(τ2i(1ζi)(1ξi)(1ϑi)log{ST3(t2i|ci)}|cik=1,O,Θ(L))=P(Cik=1,τ2i=1,ζi=0,ξi=0,ϑi=0|O,Θ(L))×E(log{ST3(t2iui|ci)}|cik=1,τ2i=1,ζi=0,ξi=0,ϑi=0,O,Θ(L))=ωik[9,1]uigilog{ST3(t2ui|ci)}fT2(t2|cik=1,τ2i=1,ζi=0,ξi=0,ϑi=0)dt2=ωik[9,1]uigilog{ST3(t2ui|ci)}fT2(t2|ci)ST3(t2ui|ci)uigifT2(t2|ci)ST3(t2ui|ci)dt2dt2=ωik[9,1][uigilog{ST3(t2ui|ci)}ST3(t2ui|ci)fT2(t2|ci)dt2uigiST3(t2ui|ci)fT2(t2|ci)dt2].

Appendix C: Detailed formulation of E[lc(Θ|P, Θ(L))] in the M-step (Section 4.1)

E[lc(Θ|P,Θ(L))]=E[lc1(π1|P,Θ(L))]+E[lc2(π2|P,Θ(L))]+E[lc3(a1,d1,α|P,Θ(L))]+E[lc4(a2,d2,β|P,Θ(L))]+E[lc5(a3,d3,γ|P,Θ(L))]+E[lc6(η|P,Θ(L))]+E[lc7(λ0*(),κ,ν|P,Θ(L))], where

E[lc1(π1|P,Θ(L))]=i:φi=1{k=1K(ωik[3,1]+ωik[4,1])log(π1k)+k=1Kωik[5,1]log(1π1k)}+i:φi{2,3}k=1Kωik[1,{2,3}]log(π1k)+i:φi{4,5}k=1Kωik[1,{4,5}]log(1π1k),E[lc2(π2|P,Θ(L))]=i:φi=1{k=1Kωik[6,1]log(π2k)+k=1Kωik[7,1]log(1π2k)}+i:φi{2,3}k=1Kωik[1,{2,3}]log(π2k),E[lc3(a1,d1,α|P,Θ(L))]=i:φi=1{k=1Kωik[4,1]log{ST1(xi|ci)}+k=1Kωik[3,1]log{fT1(xi|ci)}}+i:φi{2,3}k=1Kωik[1,{2,3}]log{fT1(t1i|ci)},E[lc4(a2,d2,β|P,Θ(L))]=i:φi=1{k=1Kωik[2,1]log{fT2(xi|ci)}+k=1Kωik[10,1]}+i:φi{2,4}k=1Kωik[1,{2,4}]log{ST2(gi|ci)}+i:φi{3,5}k=1Kωik[1,{3,5}]log{fT2(t2i|ci)},E[lc5(a3,d3,γ|P,Θ(L))]=i:φi=1{k=1Kωik[11,1]+k=1Kωik[8,1]log{ST3(giui|ci)}}+i:φi{2,3}k=1Kωik[1,{2,3}]log{fT3(t3i|ci)},E[lc6(η|P,Θ(L))]=i=1n[k=1Kωik[1,{all}]wiηklog{k=1Kexp(wiηk)}],

and E[lc7(λ0*(),κ,ν|P,Θ(L))]

=j=1Ji=1nδi1(Vi=t(j)*){logλ0*t(j)*)+κzi+k=1Kωik[1,{all}]νk}λ0*(t(j)*)iRjexp(κzi)k=1Kωik[1,{all}]exp(νk),

where Rj is the risk set at time t(j)*,ωik[1,{g,h}]=ωik[1,f]+ωik[1,g] for g, h ∈ {1, 2, 3, 4, 5}, and ωik[1,{all}]=ωik[1,1]+ωik[1,2]++ωik[1,5].

References

  • 1.Goodwin PJ, Ennis M, Pritchard KI, Trudeau M, Hood N. Risk of menopause during the first year after breast cancer diagnosis. Journal of Clinical Oncology. 1999;17:2365–2370. doi: 10.1200/JCO.1999.17.8.2365. [DOI] [PubMed] [Google Scholar]
  • 2.Bines J, Oleske DM, Cobleigh MA. Ovarian function in premenopausal women treated with adjuvant chemotherapy for breast cancer. Journal of Clinical Oncology. 1996;14:1718–1729. doi: 10.1200/JCO.1996.14.5.1718. [DOI] [PubMed] [Google Scholar]
  • 3.Ganz PA, Greendale GA, Petersen L, Kahn B, Bower JE. Breast cancer in young women: reproductive and late health effects of treatment. Journal of Clinical Oncology. 2003;21:4184–4193. doi: 10.1200/JCO.2003.04.196. [DOI] [PubMed] [Google Scholar]
  • 4.Wilcken N, Stockler M. Ovarian suppression for early breast cancer. The Lancet. 2007;369:1668–1670. doi: 10.1016/S0140-6736(07)60758-2. [DOI] [PubMed] [Google Scholar]
  • 5.Szwarc SE, Bonetti M. Modelling menstrual status during and after adjuvant treatment for breast cancer. Statistics in Medicine. 2006;25:3534–3547. doi: 10.1002/sim.2445. [DOI] [PubMed] [Google Scholar]
  • 6.Tsiatis AA, DeGruttola V, Wulfsohn MS. Modeling the relationship of survival to longitudinal data measured with error: application to survival and CD4 counts in patients with AIDS. Journal of the American Statistical Association. 1995;90:27–37. [Google Scholar]
  • 7.Wulfsohn MS, Tsiatis AA. A joint model for survival and longitudinal data measured with error. Biometrics. 1997;53:330–339. [PubMed] [Google Scholar]
  • 8.Tsiatis AA, Davidian M. A semiparametric estimator for the proportional hazards model with longitudinal covariates measured with error. Biometrika. 2001;88:447–458. doi: 10.1093/biostatistics/3.4.511. [DOI] [PubMed] [Google Scholar]
  • 9.Song X, Davidian M, Tsiatis AA. A semiparametric likelihood approach to joint modeling of longitudinal and time-to-event data. Biometrics. 2002;58:742–753. doi: 10.1111/j.0006-341x.2002.00742.x. [DOI] [PubMed] [Google Scholar]
  • 10.Law NJ, Taylor JMG, Sandler H. The joint modeling of a longitudinal disease progression marker and the failure time process in the presence of cure. Biostatistics. 2002;3:547–563. doi: 10.1093/biostatistics/3.4.547. [DOI] [PubMed] [Google Scholar]
  • 11.Tsiatis AA, Davidian M. Joint modeling of longitudinal and time-to-event data: an overview. Statistica Sinica. 2004;14:809–834. [Google Scholar]
  • 12.Yu M, Law NJ, Taylor JMG, Sandler H. Joint longitudinal-survival-cure models and their application to prostate cancer. Statistica Sinica. 2004;14:835–862. [Google Scholar]
  • 13.Huang W, Zeger SL, Anthony JC, Garrett ES. Latent variable model for joint analysis of multiple repeated measures and bivariate event times. Journal of the American Statistical Association. 2001;96:906–914. [Google Scholar]
  • 14.Lin H, Turnbull BW, McCulloch CE, Slate EH. Latent class models for joint analysis of longitudinal biomarker and event process data: application to longitudinal prostate-specific antigen readings and prostate cancer. Journal of the American Statistical Association. 2002;97:53–65. [Google Scholar]
  • 15.Larsen K. Joint analysis of time-to-event and multiple binary indicators of latent classes. Biometrics. 2004;60:85–92. doi: 10.1111/j.0006-341X.2004.00141.x. [DOI] [PubMed] [Google Scholar]
  • 16.Larsen K. The Cox proportional hazards model with a continuous latent variable measured by multiple binary indicators. Biometrics. 2005;61:1049–1055. doi: 10.1111/j.1541-0420.2005.00374.x. [DOI] [PubMed] [Google Scholar]
  • 17.Proust-Lima C, Joly P, Dartigues JF, Jacqmin-Gadda H. Joint modeling of multivariate longitudinal outcomes and a time-to-event: a nonlinear latent class approach. Computational Statistics and Data Analysis. 2009;53:1142–1154. [Google Scholar]
  • 18.Castiglione-Gertsch M, O’Neill A, Price KN, Goldhirsch A, Coates AS, Colleoni M, Nasi ML, Bonetti M, Gelber RD on behalf of the International Breast Cancer Study Group. Adjuvant chemotherapy followed by goserelin versus either modality alone for premenopausal lymph node-negative breast cancer: a randomized trial. Journal of the National Cancer Institute. 2003;95:1833–1846. doi: 10.1093/jnci/djg119. [DOI] [PubMed] [Google Scholar]
  • 19.Agresti A. Analysis of Categorical Data. New York: Wiley; 1984. [Google Scholar]
  • 20.Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statlstical Society, Series B. 1977;39:1–22. [Google Scholar]
  • 21.Louis TA. Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society, Series B. 1982;44:226–233. [Google Scholar]
  • 22.Tanner MA. Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions. 3rd edn. New York: Springer; 1996. [Google Scholar]

RESOURCES