Abstract
Patients receiving radical prostatectomy are at risk of metastasis or prostate cancer related death, and often need repeated clinical evaluations to determine whether additional adjuvant or salvage therapies are needed. Since the prostate cancer is a slowly progressing disease, and these additional therapies come with significant side effects, it is important for clinical decision making purposes to estimate a patient’s risk of cancer metastasis, in the presence of a competing risk by death, under the hypothetical condition that the patient does not receive any additional therapy. In observational studies, patients may receive additional therapy by choice; the time to metastasis without any therapy is often a potential outcome and not always observed. We study the competing risks model of Fine and Gray (1999) with adjustment for treatment choice by inverse probability censoring weighting (IPCW). The model can be fit using standard software for partial likelihood with double IPCW weights. The proposed methodology is used in a prostate cancer study to predict the post-prostatectomy cumulative incidence probability of cancer metastasis without additional adjuvant or salvage therapies.
Keywords: Causal inference, Fine-Gray model, Inverse probability censoring weighting (IPCW), Landmark analysis, Potential outcomes, Prediction model
1 Introduction
Prostate cancer is the most common noncutaneous cancer in men. Although the risk of death is relatively mild compared with other more aggressive cancers, 33,270 men are expected to die from this disease in the U.S. in 2011 (Siegel, et al, 2011). This is a disease with long and complicated natural history (Albertsen, Hanley, and Fine, 2005). Most of the time, the disease is discovered while the tumor is still confined within the prostate gland, called “localized prostate cancer”. If left untreated, the tumor will usually grow at a slow progression rate with no or little symptoms for a long time and eventually may spread outside the prostate, causing metastasis. Localized prostate cancer has only moderate risk of death, but once it reaches metastasis the tumor could evolve into more aggressive form of cancer in other parts of the body, and risk of death or other serious complications are significantly elevated. The actual rate of progression and the risk of death vary, and could be a ected by patient characteristics, genetic, and environmental factors. The disease is easier to treat while it is at the localized stage. Radical prostatectomy, the surgical removal of the prostate, can curatively remove the tumor tissue. However, after prostatectomy, there is a chance that the tumor may regrow and cause metastasis. Therefore, patients must undergo periodic clinical evaluations during which biomarkers such as the prostate specific antigen (PSA) are measured to determine whether there is tumor regrowth and whether additional adjuvant or salvage therapies (denoted by “therapy” or “additional therapy” henceforth for brevity when there is no ambiguity) are needed. Elevated PSA is a sign of disease progression. Since these therapies also carry unpleasant side effects, patients and physicians must weigh both benefits and harms before deciding whether or when to start the therapy.
In this paper, we focus on a medical decision making problem among patients receiving radical prostatectomy. These patients undergo repeated PSA testings to monitor surgical failure, tumor regrowth and metastasis. On the one hand, rising PSA is generally associated with increasing prostate tumor activity and increasing risk of metastasis, but their relationship is far from clear-cut; on the other hand, adjuvant or salvage therapies can often suppress the elevation of PSA, but they all come with significant side-effects that adversely a ect the patient’s quality of life. Therefore, a question often raised during patient counseling is: “what is the risk of metastasis, if the patient chooses to avoid these therapies?” The answer is helpful to both the patients and physician in their management of the disease. This is a prediction problem. For the purpose of illustrating the methodology, we assume without loss of generality that the time of prediction (the landmark time, Houwelingen and Putter 2012) is right after the prostatectomy, and call it the time of baseline. All predictors are measured at baseline. It will be shown below that the proposed methodology is applicable to any landmark time after prostatectomy.
To find an answer to the question above, we studied a real data set that includes 4,290 patients undergoing prostatectomy for clinically localized prostate cancer between 1987 and 2008. The length of follow-up ranged between 1 and 242 months, with median at 51 months. Each patient had a series of PSA measurements until metastasis, death without metastasis, or initiation of adjuvant or salvage therapy without metastasis. Six hundred and forty-three patients started therapy at some time during follow-up; among them, the time from prostatectomy to the therapy ranged between 1 and 216 months (median 21). At the end of the follow-up, 148 patients had metastasis and 261 died without metastasis.
The risk of metastasis can be quantified statistically by its cumulative incidence function (Kalbfleisch and Prentice, 2002). The outcome of the model is a terminal event of either metastasis or death without metastasis. If none of the patients in our data set received the therapy, the solution to the aforementioned problem is straightforward: we can build a Fine-Gray competing risks regression model (Fine and Gray, 1999) for the cumulative incidence function of metastasis, using baseline predictors. Death is a competing risk to metastasis. We do not treat death as a censoring event (Shariat et al, 2009), because death is a part of the natural history of prostate cancer and it does not make sense to predict the hypothetical probability of progression assuming the patient does not die of another cause first (Gooley et al, 1999). Note that strictly speaking, metastasis and death are actually semi-competing risks (Fine, Jiang, Chappell, 2001), i.e., death precludes the observation of metastasis, but not the other way around. However, we are not interested in the gap time between metastasis and subsequent death because metastasis is a key event of interest to both patient and physician, and the gap time is no longer solely about the risk of prostate cancer. We also do not consider a composite event of either metastasis or death (Schellhammer et al, 1997), because prostate cancer is a slowly progressing disease with moderate risk of mortality, and many patients died of other causes before they reach metastasis. The predictors for metastasis and death without metastasis may be different.
Initiation of adjuvant or salvage therapy can be viewed as an event that censors the terminal event of metastasis or death. Although both therapy and death are competing risks to metastasis, we treat them differently. We treat death as another type of the terminal event, acknowledging the reality that patients may die without metastasis; we treat therapy as a dependent censoring event because we want to predict the cumulative incidence probability of metastasis under the counterfactual condition that the patient does not receive any adjuvant or salvage therapy. Hence, we will not consider a Fine-Gray model with therapy as a third terminal event type, in addition to metastasis and death. Patients receiving the therapies are different from those not, even if they all have identical baseline variables. In general, those with higher PSA levels or faster rate of PSA increase during the follow-up will have a higher chance of receiving the therapy. Since the time to terminal event and time to therapy are correlated given the predictors at baseline, therapy dependently censors the terminal event. Therefore, fitting a Fine-Gray model on the subset of patients without adjuvant or salvage therapy may produce a biased result, if the research purpose is to estimate the cumulative incidence function of metastasis with adjustment for competing risks by death, under the counterfactual situation that adjuvant or salvage therapies are always withheld from the patients. Similarly, including therapy as a time-dependent covariate (i.e., it equals 0 before initiating the therapy and 1 after that) together with baseline predictors in the Fine-Gray model for metastasis may also produce a biased result for that purpose. Adding follow-up PSA as another time-dependent covariate to the model would not help either, because the goal of the research is to build a prediction model only with baseline predictors measured at the landmark time (e.g., right after prostatectomy), and the longitudinal PSA trajectory is not available at the time of prediction.
The research question is a prediction problem under the counterfactual condition that no patients receive adjuvant or salvage therapy. Hence, it has a connection to causal inference methodology. The cumulative incidence probability of metastasis without any therapy can only be learned from similar patients in the training data set who did not receive therapy in reality. In the terminology of causal inference (Holland, 1986), every patient in the data set may have an infinite number of potential times to the terminal event, depending on whether or when they receive the therapy; but in reality, we can only observe one potential time to event outcome for each patient, i.e., the one as a result of his actual choice. To make an unbiased prediction, we need to reconstruct the potential time to event outcomes as if all the similar patients in the data set (i.e., with identical baseline predictors) had not received additional therapies, and form the prediction by aggregating their potential outcomes. As we will see below, such reconstruction is done by appropriately weighting therapy-free patients in the data set with similar baseline predictors.
The consideration above suggests that we should build a Fine-Gray model of metastasis, with death as a competing risk event, and at the same time adjust for dependent censoring due to the initiation of therapy. The Fine-Gray model directly models the cumulative incidence function of metastasis, which is an appropriate measure of the absolute risk specifically attributable to cancer. To deal with the issue of dependent censoring by therapy (or, the potential time to event outcome), we further develop the inverse probability of censoring weighting (IPCW) technique (Robins, 1993) by incorporating the IPCW weights into the Fine-Gray competing risks model. IPCW has been widely used in adjusting for dependent censoring, dealing with informative missingness, or estimating the causal e ect of interventions (Robins, Hernán, Brumback, 2000; Robins and Finkelstein, 2000; Tsiatis, 2006; Matsuyama and Yamaguchi, 2008; Schaubel and Wei, 2011). To our knowledge, there has been no publication that combined IPCW with the Fine-Gray model for an event of interest (e.g., metastasis) with both competing risk (e.g., death without metastasis) and dependent censoring (e.g, therapy). Our development leads to a partial likelihood with double IPCW weights, and the model can be fit by modifying standard software for partial likelihood. The proposed approach is presented in Section 2. Sections 3 illustrates the method by analyzing a prostate cancer data set. Section 4 is the simulation. Section 5 contains discussion and direction for future research.
2 Proposed Model and Method
Suppose there are n independent patients in the registry, indexed by i = 1, 2, …, n. The time of prediction is denoted by t0. For simplicity of discussion, we assume that t0 = 0, i.e., that the prediction is made right after the surgery. The prediction can also be made at any later landmark time during the follow-up (t0 > 0), and the methodology in this paper is still applicable to patients who were at risk beyond the landmark time by treating the landmark time as the new baseline. The research goal is to estimate, for a new patient who received prostatectomy, the cumulative incidence function of metastasis up to time τ0, if the patient chooses not to receive the additional therapy. Since prostate cancer is a slowly progressing disease, τ0 is often set at 5 or 10 years. Let the observed data of the i-th patient be {Zi, Vi, Ti, δi, Xi, Δi}. Zi is a vector of baseline covariates. Vi = (Vi 1, … Vini)T is a vector of PSA values, measured at time 0 < ti1 < … < tini during the follow-up. Ti is the time when the subject started the additional therapy or the time of the last follow-up, whichever is earlier. δi is a therapy indicator, which equals 1 when the subject started therapy at time Ti, and equals to 0 otherwise. Xi is the time to metastasis, death or right censoring due to the end of follow-up, whichever is earlier. Δi = 0 indicates the patient was right censored, 1 if there was metastasis, and 2 if the patient died. Obviously, Ti ≤ Xi, and equality holds if and only if δi = 0.
If no patient in the registry had received additional therapy, i.e., δi = 0 and Ti = Xi for all patients, then the cumulative incidence function of time to metastasis (Δi = 1), expressed as a function of baseline covariates Zi, can be modeled using the Fine-Gray model (Fine and Gray, 1999). All-cause death is a competing risk event here. Let Di be the time to metastasis or death, whichever is earlier, and Ci be the censoring time, i.e., Xi = Di ∧ Ci ≐ min(Di, Ci). Let the cumulative incidence function of metastasis be F(t; Zi) = Pr(Di ≤ t, Δi = 1|Zi). This is a sub-distribution relative to Pr(Di ≤ t|Zi), the distribution of event time Di. Fine and Gray (1999) define the sub-distribution hazard by
Although the sub-distribution hazard and its probability distribution F(t; Zi) have a relationship similar to that between the hazard and distribution functions of a univariate time to event variable, the risk set associated with the sub-distribution hazard is unnatural in the sense that those who failed due to causes other than the one being modeled are always kept in the risk set as if they were still at risk. However, under a proportional hazards specification for this sub-distribution hazard,
(h0(t) is the baseline sub-distribution hazard function) the cumulative incidence function takes the following simple form:
| (1) |
Therefore, Fine-Gray model enables us to directly model the cumulative incidence function as a function of covariates. This is very useful for the risk prediction problem studied in this paper (Kattan et al, 2003).
Denote G(t) = Pr(Ci ≥ t), the survival distribution of the censoring time. For the application in this paper, we assume that Ci is independent of Di and Zi. Hence the Kaplan-Meier estimator of G(t), , can be derived from . In situations where Ci is conditionally independent of Di given Zi, we may define G(t) = Pr(Ci ≥ t|Zi), and model it with a Cox regression model of {Xi, I(Δi = 0)} on Zi; the subsequent development is the same. Let Ni(t) = I(Di ≤ t, Δi = 1), Yi(t) = 1−Ni(t−), ri(t) = I(Ci ≥ Di∧t). Note that Ni(t) and Yi(t) are both computable when ri(t) = 1; while ri(t)Ni(t) and ri(t)Yi(t) reduce to 0 when ri(t) = 0, regardless of whether Ni(t) and Yi(t) are computable. At any time t, Fine and Gray define the time-dependent weight as
and propose a modified partial likelihood score equation to estimate β consistently:
| (2) |
The cumulative baseline sub-distribution hazard is estimated by an analog of Breslow’s estimator for Cox proportional hazard model:
Therefore, a consistent estimate of the cumulative incidence function can be obtained from (1) by replacing the unknown parameters by their estimates.
If some patients in the registry received additional adjuvant or salvage therapy, i.e., δi = 1 and Ti < Xi, we can think of their time to metastasis or death Xi as being censored by Ti. Since both Xi and Ti are related to prostate cancer progression after the surgery, this is dependent censoring. The research goal is to study the time to metastasis when there is no dependent censoring by additional therapy. The motivation of the proposed approach is described as follows. The set of patients who were not on additional therapy and still at risk of metastasis or death (δi = 0 and Xi > t) changes over time. Since the censoring by additional therapy is dependent, those who dropped out of this set due to receiving additional therapy more likely had increasing signs of cancer regrowth, such as rising PSA levels. Therefore, as time goes on, those remaining in the risk set without additional therapy are a selective subset of the original patient sample, and their risk of metastasis does not represent the risk of original patient sample had they all chosen not to have additional therapy. From a causal inference perspective, each patient has an infinite number of potential time to metastasis outcomes, depending on whether and when he might have started the additional therapy, but we can only observe the potential outcome without the additional therapy in those who actually did not receive it. From a missing data point of view, at any time during the follow-up, the instantaneous risk of metastasis (sub-distribution hazard) without additional therapy can be estimated from those who stayed in the risk set, but is missing from those who dropped out of the risk set. However, if we know the probability of missingness, we are able to estimate the distribution of the missing data by appropriately weighting the observed data from those in the risk set with the inverse of the probability of being observed. This is the intuition behind the inverse probability of censoring weighting (IPCW) method (Robins, 1993). Robins and Finkelstein (2000) illustrated the use of IPCW when the outcome is univariate time to event and there is a dependent censoring event. In this paper, we apply their general approach to adjust for dependent censoring by additional therapy when the outcome is time to metastasis with competing risks by death.
Let the observed therapy time be and the therapy indicator be , where is a conceptual quantity representing the time to therapy if the patient did not die or have metastasis or censoring. Denote by the history of a vector of time-dependent covariate processes up to time t. The IPCW method relies on the following fundamental assumption (“no unmeasured confounding”; Robins, 1997):
| (3) |
This assumption states that the cause-specific hazard for time to therapy at time t can be modeled by the baseline covariates Zi and covariate history , and it does not depend on , the possibly unobserved therapy time in the future. This assumption is satisfied in the prostate cancer problem we are studying, because is the time when the decision of starting additional therapy was made, and the decision making only depended on the clinical information accumulated up to that time. In this paper, we specify the hazard in (3) with a Cox model:
where λ0(t) is the baseline hazard function. Since this is a cause-specific hazard model, metastasis and death are treated as censoring events. All time-independent and time-dependent covariates a ecting the decision making process must be included in this model. See section 3 for more discussion of this in the context of the data application. In the application of Section 3, is a function of the observed time-dependent variable Vi. This model can be fit using standard software for Cox model. Denote the estimator of the baseline hazard by and the regression coefficients estimator by and . An estimator of the conditional probability that subject i did not have any additional therapy by time t given Zi and is:
This quantity can only be calculated for t ≤ Ti; when t > Ti, it is undefined because the time-dependent covariate is available only up to Ti.
Robins and Finkelstein (2000) suggested using the stabilized IPCW weight. For that purpose we fit an additional Cox model with only the baseline covariates:
The model produces an estimator of the probability of not having the therapy by time t as:
The stabilized IPCW weight for subject i at time t is defined by:
| (4) |
When t > Ti, is undefined; but because I(t ≤ Ti) = 0 in such case, the IPCW weight is still computable and equals 0. The term in (4) is not necessary to ensure consistent estimation of β0, but it serves as a stabilizing term to reduce the chance that might get too large as a result of small and causes numerical instability.
Following the construction of IPCW weighted partial likelihood score in Robins and Finkelstein (2000), we have the following score equation for consistently estimating β:
| (5) |
where
is the product of two IPCW weights and ( is a form of IPCW weight; See Fine and Gray (1999)), and we call it the double IPCW weight. The cumulative baseline sub-distribution hazard is estimated by an analog of Breslow’s estimator for Cox proportional hazard model:
Equation (5) is the score of weighted partial likelihood, and the computation can be easily implemented using standard software for partial likelihood. We used the coxph() function of R (R Development Core Team, 2011) with a time-dependent specification of the covariate history and time to events, as well as the weight argument. The R code is available from the authors upon request. Following Robins, Hernán, Brumback (2000), robust variance estimator must be used to account for the uncertainty in estimated IPCW weights. This can be conveniently implemented in R using coxph() with the robust option. Brumback et al (2004) also recommended the use of bootstrapping, by re-sampling the subjects with replacement and re-estimating the weights. However, for the purpose of this paper, which is to build a prediction model, we are primarily interested in the point estimator, because prognostic uncertainty (e.g., confidence interval of a predicted probability) is largely irrelevant in clinical decision making (Kattan, 2011), although we studied variance estimation in Sections 3 and 4 as part of the methodological development.
3 Data Analysis
We applied the proposed method to the data set introduced in Section 1. The descriptive statistics of the data set are presented in Table 1. We first develop a Cox proportional hazard model for time to additional therapy. The covariate Z include eight baseline covariates: age, baseline PSA, year of surgery, the sum of two Gleason scores at surgery, and four binary variables: extracapsular extension (ECE), lymph node involvement (LN), surgical margins (Marg) and seminal vesical invasion (SV). Including year of surgery is often not desirable for a prediction model for future use. We included this variable here for investigational purpose, as it was hypothesized that the risk of metastasis without additional therapy may have decreased over the past few decades (Stephenson et al, 2005). Rising PSA level is the most important trigger for additional therapy. Generally, higher PSA level and faster rate of increase suggest disease progression and presumed need for adjuvant or salvage therapy.
Table 1.
Descriptive statistics of the data set, by adjuvant therapy
| Covariate | no additional therapy (n = 3647) |
additional therapy (n = 643) |
|---|---|---|
| Age | 60.5 (6.7) | 61.2 (6.8) |
| log baseline PSA | 1.79 (0.59) | 2.27 (0.74) |
| Year of surgery | ||
| 1987-1989 | 81 (2.2%) | 57 (8.9%) |
| 1990-1999 | 1092 (29.9%) | 282 (43.9%) |
| 2000-2008 | 2474 (67.8%) | 304 (47.3%) |
| Gleason score sum | 6.6 (0.6) | 7.1 (0.8) |
| ECE | 980 (27%) | 465 (72.4%) |
| Marg | 862 (23.6%) | 390 (60.7%) |
| SV | 168 (4.6%) | 223 (34.7%) |
| LN event |
36 (1%) | 80 (12.4%) |
| censored | 3396 (93.1%) | 485 (75.4%) |
| death | 204 (5.6%) | 57 (8.9%) |
| metastasis | 47 (1.3%) | 101 (15.7%) |
To model the decision process, we consider two time-dependent covariates, both of which are functions of the observed PSA. One is the log PSA level (log transformed PSA is often used in prostate cancer research to reduce skewness). Since the log PSA was measured intermittently, its value in between two consecutive measurements is imputed by linear interpolation. The other one is the PSA doubling time, which is the natural log of 2 (0.693) divided by the slope of the relationship between the log PSA and time of PSA measurement for each patient (Pound et al, 1999); if the slope is negative, the doubling time is typically set to be infinity. We calculate the slope from the linear interpolation of the most recent two log PSA measurements. For the proposed method to give unbiased results, we need the “no unmeasured confounding” assumption in the sense that all covariates a ecting the decision to start additional therapy must be included in . As noted in Robins and Finkelstein (2000), in reality we would not expect the assumption of “no unmeasured confounding” to hold exactly, but we must include enough important ones so that the IPCW adjustment reduce, even if not totally eliminate, the bias due to dependent censoring. Latini et al (2007) studied a number of risk factors predicting active prostate cancer treatment, and found that PSA and anxiety are the only two factors that are significantly associated with treatment decision. Of the two, PSA plays a dominating role and could influence the anxiety level. In Section 4, we performed simulation to assess the sensitivity of results to omission of anxiety from the model.
Table 2 presents the estimated regression coefficients from the Cox model fit. Younger age, higher baseline PSA, higher Gleason score sum at baseline, extracapsular extension and lymph node involvement are associated with increased probability of receiving additional therapy. Surgical margins and seminal vesical invasion are not in Table 2 as they are used to stratify the baseline hazard and satisfy the proportional hazard assumption. Since the PSA doubling time is skewed as a result of the skewness of estimated log PSA slope, we categorize it with cut-o s at 3, 6, 12, 24, and 36 months. Overall, longer PSA doubling time is associated with less chance of receiving the additional therapy. The only exception is the reference level (0, 3], which appears to be associated with lower likelihood of additional therapy. This level corresponds to very large slopes, which may have been an artifact due to the measurement error in log PSA, because the slope was determined by connecting the two adjacent log PSA with a straight line. However, correcting the measurement error using techniques such as joint modeling of the longitudinal log PSA and time to additional therapy (Part IV, Fitzmaurice et al, 2009) is not desirable for this application, because the physician and patient usually make the decision using the observed log PSA without considering the concept of measurement error in PSA. In addition, the progression rate of PSA is often determined by simple interpolation in clinical practice, not by complicated statistical models. Table 2 confirms the common belief that higher log PSA is associated with higher probability of additional therapy. An interaction term between log PSA and time is included in the model to meet the proportional hazard assumption. Year of surgery is significantly associated with therapy decisions; adjuvant or salvage therapies are initiated earlier over time.
Table 2.
Regression coefficients of the Cox model for time to additional therapy. HR: hazard ratio; SE: standard error.
| Covariate | log HR | SE | p-value |
|---|---|---|---|
| Age | −0.0127 | 0.00619 | 0.041 |
| log baseline PSA | 0.185 | 0.0633 | 0.004 |
| Year of surgery | 0.0900 | 0.00925 | < 0:001 |
| Gleason score sum | 0.383 | 0.0336 | < 0:001 |
| ECE | 0.694 | 0.105 | < 0:001 |
| LN | 0.269 | 0.140 | 0.055 |
| PSA doubling time (months) | |||
| [0, 3] | 0 | - | - |
| (3, 6] | 0.713 | 0.157 | < 0:001 |
| (6, 12] | 0.424 | 0.171 | 0.013 |
| (12, 24] | 0.315 | 0.190 | 0.097 |
| (24, 36] | 0.425 | 0.274 | 0.121 |
| (36, ∞) | −1.243 | 0.169 | < 0:001 |
| log PSA | 0.423 | 0.0471 | < 0:001 |
| log PSA × time | 0.0024 | 0.000692 | < 0:001 |
Next, we present results from the Fine-Gray model for the cumulative incidence probability of metastasis, under the counterfactual condition with no adjuvant or salvage therapies. Table 3 shows the regression coefficients. This model produces estimates under the counterfactual condition that none of the patients received additional therapy. Higher Gleason score, higher baseline PSA, extracapsular extension, surgical margins, seminal vesical invasion, lymph node involvement all contribute to increased cumulative incidence of metastasis. Figure 1 presents the predicted cumulative incidence function for two patients up to 10 years. Patient (a) is at higher risk with both surgical margins and seminal vesical invasion and a baseline log PSA at 1.5; Patient (b) is at lower risk without these two risk factors and with a baseline log PSA at 0.5. All other covariates are fixed: 45 years of age, operation in 2000, Gleason score sum at 7, extracapsular extension and lymph node involvement present. In addition to the proposed method, we calculated the cumulative incidence using three other methods: Method (1) “TH as CR”: fitting the Fine-Gray model and treating additional therapy as another competing risk type; Method (2) “TH excluded”: fitting the Fine-Gray model on the subset of patients who did not receive additional therapy; Method (3) “TH censored”: fitting Fine-Gray model and treating additional therapy as independent right censoring. All three approaches appear to underestimate the potential cumulative incidence of metastasis on the two hypothesized patients. The cumulative incidence function is the probability of metastasis among several competing events. By creating an additional event type, as in Method (1), we would underestimate the cumulative incidence of metastasis. Additionally, Method (1) does not produce estimates under the counterfactual condition that additional therapy is not used. Patients who received additional therapy tend to be at higher risk than others. By excluding the higher risk ones, Method (2) also underestimates the cumulative incidence of metastasis. The regression coefficients from Method (2) are reported in Table 3 for comparison. Method (3) treats time to additional therapy as independent, instead of dependent censoring, which also leads to large bias, suggesting that there may be strong correlation between time to additional therapy and time to metastasis. Interestingly, Table 3 shows that year of surgery is not significantly associated with metastasis risk if the patient does not use adjuvant or salvage therapy; but if we exclude those on therapies, the risk of metastasis without therapy appears to decrease over time. This result, combined with the result from Table 2 on the increasing trend of adjuvant or salvage therapy usage, raises questions on whether the apparent decreased risk of metastasis without additional therapy is because more higher-risk patients are being absorbed into the group receiving additional therapies. The estimated probabilities (95% confidence interval) for the high risk patient at 5 and 10 years are 0.22 (0, 0.56) and 0.42 (0, 0.86), and for the low risk patient, 0.034 (0, 0.092) and 0.065 (0, 0.18). Figure 1. The prediction is less accurate as the time lag increases. The lower limit of the confidence interval reaches 0 probably due to the small number of metastasis in the data set (Table 1). In the next section, we studied the confidence interval of the estimated probability in simulations.
Table 3.
Regression coefficients of the Fine-Gray model for the cumulative incidence of metastasis. HR: sub-distribution hazard ratio; SE: standard error.
| Proposed Model | Therapy cases excluded | |||||
|---|---|---|---|---|---|---|
|
| ||||||
| Covariate | log HR | SE | p-value | log HR | SE | p-value |
| Age | −0.0257 | 0.0237 | 0.28 | −0.0247 | 0.0241 | 0.31 |
| Year of surgery | −0.0211 | 0.0463 | 0.65 | −0.0929 | 0.0452 | 0.040 |
| Gleason score sum | 0.796 | 0.250 | 0.001 | 0.738 | 0.279 | 0.008 |
| ECE | 2.036 | 0.516 | < 0:001 | 2.071 | 0.549 | < 0:001 |
| Marg=0, SV=0 | 0 | - | - | 0 | - | - |
| Marg=1, SV=0 | 0.205 | 0.434 | 0.64 | 0.0851 | 0.424 | 0.84 |
| Marg=0, SV=1 | 1.426 | 0.425 | < 0.001 | 1.168 | 0.426 | 0.006 |
| Marg=1, SV=1 | 1.300 | 0.586 | 0.027 | 0.885 | 0.637 | 0.16 |
| LN | 0.960 | 0.568 | 0.091 | 0.413 | 0.772 | 0.59 |
| log baseline PSA | 0.600 | 0.238 | 0.012 | 0.537 | 0.297 | 0.071 |
| log baseline PSA × time | 0.00273 | 0.00386 | 0.48 | 0.000744 | 0.00357 | 0.84 |
Fig. 1.
Predicted cumulative incidence function of metastasis up to 10 years for a high risk (a) and a low risk (b) patient.
4 Simulation
We carried out a simulation study to examine the empirical performance of the proposed method. We assumed that the log-transformed PSA follows a linear trajectory over time with patient-specific intercepts and slopes:
where the baseline value X0 follows N(1.86, 0.64) and the slope follows N(0.01, 0.15). These parameters were chosen to mimic the data in Section 3. The hazard function for the time to therapy T followed the specification of a Cox model with time-dependent covariate Xt and two independent confounders Z1 and Z2, both from the standard normal distribution.
The times to metastasis and death were conditionally independently simulated from the following two exponential hazard models, respectively,
where T is the simulated time to therapy in the first step. The coefficient of −0.2 associated with the term I(t ≥ T) indicates that the hazard is reduced if the patient receives the therapy. The simulated times to death and metastasis were censored at the simulated time-to-therapy as in the prostate cancer study. An independent censoring time was simulated from the uniform distribution U[0, 10] for each subject. We varied the value of β in the model for metastasis (λ1(t)). The sample size was either 1,000 or 200, and the simulation size was 200.
The proposed method was compared with the three methods discussed in Section 3: “TH excluded”, “TH as CR”, and “TH censored”. Table 4 shows the simulation results. For the four values of the parameter β (−0.1, −0.2, −0.5, −0.8), the observed proportions of patients receiving the therapy were on average 47%, 49%, 56% and 60%, respectively; the observed average proportions of death were 31%, 28%, 18% and 12%; the observed average proportions of metastasis were 22%, 23%, 26% and 28%, respectively. For all β’s, the proposed method produces unbiased point estimators and approximately correct variance estimators in the simulation, while all the other three methods have non-negligible bias. Figure 2 presents the average estimated cumulative incidence function of metastasis and the coverage probabilities of the pointwise 95% confidence intervals. The estimated probability is nearly unbiased and the pointwise confidence interval has approximately the correct coverage.
Table 4.
Estimate of β in the simulation. Estimate: the sample average of the point estimators in the simulations; SE: the sample average of the standard error estimators; SD: the sample standard deviation of the point estimators in the simulations; CP: coverage probability (%) of the 95% confidence intervals.
| Proposed method | TH excluded | TH as CR | TH censored | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| β | Estimate (SE) | SD | CP | Estimate (SE) | SD | CP | Estimate (SE) | SD | CP | Estimate (SE) | SD | CP |
| n = 1000 | ||||||||||||
|
| ||||||||||||
| −0.1 | −0.106 (0.088) | 0.089 | 94.4 | 0.109 (0.090) | 0.088 | 35.8 | −0.064 (0.086) | 0.088 | 92.8 | −0.024 (0.087) | 0.087 | 85.8 |
| −0.2 | −0.198 (0.094) | 0.095 | 94.6 | 0.054 (0.096) | 0.095 | 24.6 | −0.144 (0.094) | 0.094 | 91.4 | −0.109 (0.093) | 0.093 | 83.8 |
| −0.5 | −0.506 (0.115) | 0.118 | 95.0 | −0.176 (0.113) | 0.118 | 20.4 | −0.422 (0.116) | 0.116 | 89.2 | −0.399 (0.113) | 0.115 | 86.6 |
| −0.8 | −0.798 (0.151) | 0.146 | 94.4 | −0.432 (0.151) | 0.146 | 26.2 | −0.705 (0.148) | 0.143 | 89.4 | −0.685 (0.148) | 0.143 | 86.8 |
|
| ||||||||||||
| n = 200 | ||||||||||||
|
| ||||||||||||
| −0.1 | −0.100 (0.198) | 0.199 | 94.6 | 0.111 (0.202) | 0.200 | 82.2 | −0.062 (0.200) | 0.198 | 94.8 | −0.021 (0.195) | 0.195 | 93.6 |
| −0.2 | −0.211 (0.225) | 0.213 | 92.2 | 0.040 (0.217) | 0.214 | 80.0 | −0.155 (0.214) | 0.210 | 93.6 | −0.122 (0.214) | 0.209 | 91.8 |
| −0.5 | −0.497 (0.280) | 0.266 | 91.2 | −0.165 (0.280) | 0.268 | 76.4 | −0.413 (0.267) | 0.260 | 93.0 | −0.392 (0.268) | 0.259 | 92.0 |
| −0.8 | −0.804 (0.334) | 0.328 | 93.6 | −0.423 (0.333) | 0.331 | 79.0 | −0.703 (0.317) | 0.322 | 94.6 | −0.680 (0.320) | 0.322 | 92.8 |
Fig. 2.
The average estimated cumulative incidence function (CIF) of metastasis and the coverage probabilities of the pointwise 95% confidence intervals under the simulation setting in Table 4 with n = 1000 and β = −0.5. Blue curve: true CIF of metastasis without adjuvant therapy; black curve: average estimated CIF; dashed lines around the black curve: average of the pointwise lower and upper confidence limits; dashed horizontal line: 0.95; red curve: coverage probabilities of the pointwise confidence intervals.
We conducted additional simulations to assess the sensitivity of the proposed method to the “no unmeasured confounding” assumption in the context of the data application. As mentioned in Section 3, we consider the e ect of omitting anxiety from the model for therapy. To better quantify the relationship between PSA and anxiety, we decompose the e ect of patient anxiety into a variable that represent a patient’s “propensity to anxiety”, and its interaction with PSA. Note that “propensity to anxiety” is different from “anxiety” itself. The “propensity to anxiety” is directly related to the patient’s personality and possibly family environment, which can be reasonably assumed to be independent of any clinical information about prostate cancer. The actual level of anxiety could be influenced by both the “propensity to anxiety” and rising PSA. In the simulation, the overall e ect of anxiety is captured by the independent e ect of propensity to anxiety (Z2) and the interaction between “propensity to anxiety” and longitudinal PSA (Z2Xt). In the misspecified model, both Z2 and Z2Xt terms are omitted.
Table 4 shows the result of sensitivity analysis with n = 200. The proposed method has some bias with the misspecified model, but the bias is quite small compared with the other three methods. The result with n = 1, 000 is similar and the bias with the proposed method is even smaller and thus omitted. We speculate that there could be at least three reasons for this result. First, the missing covariates are with the model for therapy; we use this model to calculate the IPCW weights and then, in a second step, use the weights in the Fine-Gray model to calculate the cumulative incidence probability. Hence, the missing covariates only a ect the Fine-Gray model indirectly through the weights, and their e ect may be diminished in the process. Second, the missing covariates in this problem are special because propensity to anxiety is an independent variable. Part of its variation is absorbed into the residual variation of the model, which may also diminish its systematic influence on the weights. Third, we found in our numerical calculation that the use of stabilized weights, as in equation (4), seems to help reducing bias in the IPCW weights. This is because when a confounder is omitted, it causes biases in both the numerator and denominator of the stabilized weight, and the biases cancel out to some extent in the division. Note that the simulation-based sensitivity analysis is quite limited in scope and the result may not be generalized beyond the numerical setting studied. A methodological investigation of sensitivity analysis (e.g., Brumback et al 2004) with the special structure that the omitted covariates include an independent variable and its interaction with an observed covariate is of interest but is beyond the scope of this paper.
5 Discussion
We propose an approach to estimating the cumulative incidence function of potential time to event with competing risks. The method further extends the methodology of Robins and Finkelstein (2000) to the Fine-Gray competing risks model. The key to the methodology is the double IPCW weights. The method can be easily implemented using standard software for maximizing the partial likelihood.
Prostate cancer is a disease with long progression history, moderate risk of death, significant adverse effects on quality of life. Therapies for prostate cancer often involve significant side effects. Therefore, evidence based medical decision making, properly weighting benefits and harms, is an indispensable tool in disease management. In the current clinical practice to manage prostate cancer, a strategy called “active surveillance” is often used (Thompson and Klotz, 2010). This strategy involves frequent monitoring of the cancer progression and quality of life, and making personalized treatment decisions at each clinical evaluation based on projected future outcomes under various treatment options. Like the research question studied in this paper, many such decision making problems pertain to potential outcomes, which are not always directly observable in patient registries. However, causal inference techniques have not been widely used in the field of prostate cancer research. The application in this paper is an example where this relatively new statistical tool is useful in this field of study.
The research question studied in this paper is “what is the expected risk of metastasis of a patient, if he chooses not to receive adjuvant or salvage therapy”. This question is important for medical decision making because all these therapies have unpleasant side effects, and once a patient starts the therapy, he must stay on the therapy for a long time. Since prostate cancer is a slowly progressing disease with moderate risk of adverse outcomes, naturally the patient usually would like to postpone the therapy and only have it when it is absolutely necessary. Therefore, if the risk of adverse outcomes remains low without the therapy, the patient might choose to stay on “active surveillance” and not to start the therapy right away. However, another question, not yet answered by the methodology in this paper, is also important in patient counseling: “what is the expected risk of metastasis, if the patient starts the therapy months from now” (ν ≥ 0). The case when ν = 0 indicates that the therapy is to be initiated immediately; 0 < ν < ∞ indicates postponed therapy. In the terminology introduced in Section 1, a patient may face an infinite number of potential outcomes depending on the value of ν, i.e., whether and when the adjuvant or salvage therapy starts. In this paper we studied the case when ν = ∞. Estimating the potential outcomes when 0 ≤ ν < ∞ remains an open question that calls for novel statistical methodology.
The proposed method is to estimate patient-specific cumulative incidence of metastasis. From another perspective, we can also view it as a prediction problem, i.e., prediction of the potential time to event outcome. The analytical tool proposed in this paper makes it possible to construct a nomogram for use in clinical practice (Kattan et al, 2003). For that purpose, we might need to apply the proposed methodology at a series of landmark times t0 as suggested in the first paragraph of Section 2, so that prediction can be updated over time by incorporating accumulated longitudinal information. It is also worthwhile to study whether the structured nested failure time model (Lok et al, 2004) can be used for modeling the counterfactual quantities and whether some new perspective on cause-specific residual life (Jeong and Fine, 2009) can be adopted to deal with the competing risks in this problem. In addition, further work validating the predictive performance is needed. In statistics, prediction models have been extensively studied and widely used (Hastie et al, 2003), but to our knowledge, prediction of potential outcomes has not. Suppose y is the outcome and we want to predict and is the prediction based on predictors x. The quality of prediction is often assessed in an independent testing data set as an average distance between the actual observed y and its prediction. Prediction of potential outcomes introduces new challenges because y is not always observed in either the training or the testing data set. We have an on-going project studying the issue of validating the prediction of potential outcomes.
Table 5.
Simulation on sensitivity analysis. Estimate: the sample average of the point estimators in the simulations; SE: the sample average of the standard error estimators; SD: the sample standard deviation of the point estimators in the simulations.
| Proposed method | TH excluded | TH as CR | TH censored | |||||
|---|---|---|---|---|---|---|---|---|
| β | Estimate (SE) | SD | Estimate (SE) | SD | Estimate (SE) | SD | Estimate (SE) | SD |
| n = 200, γ = −0.05 | ||||||||
|
| ||||||||
| −0.1 | −0.118 (0.200) | 0.201 | 0.111 (0.195) | 0.201 | −0.073 (0.194) | 0.197 | −0.032 (0.190) | 0.196 |
| −0.2 | −0.224 (0.228) | 0.211 | 0.034 (0.225) | 0.212 | −0.171 (0.220) | 0.209 | −0.133 (0.221) | 0.206 |
| −0.5 | −0.520 (0.274) | 0.265 | −0.185 (0.271) | 0.267 | −0.435 (0.256) | 0.259 | −0.413 (0.262) | 0.258 |
| −0.8 | −0.829 (0.359) | 0.332 | −0.453 (0.333) | 0.330 | −0.728 (0.334) | 0.324 | −0.707 (0.333) | 0.323 |
|
| ||||||||
| n = 200, γ = 0.05 | ||||||||
|
| ||||||||
| −0.1 | −0.086 (0.206) | 0.201 | 0.122 (0.207) | 0.201 | −0.042 (0.203) | 0.198 | −0.003 (0.198) | 0.196 |
| −0.2 | −0.200 (0.218) | 0.213 | 0.048 (0.219) | 0.215 | −0.140 (0.215) | 0.209 | −0.110 (0.211) | 0.208 |
| −0.5 | −0.523 (0.277) | 0.267 | −0.193 (0.280) | 0.267 | −0.442 (0.272) | 0.261 | −0.417 (0.268) | 0.260 |
| −0.8 | −0.804 (0.357) | 0.329 | −0.452 (0.344) | 0.331 | −0.712 (0.334) | 0.321 | −0.692 (0.338) | 0.322 |
|
| ||||||||
| n = 200, γ = −0.02 | ||||||||
|
| ||||||||
| −0.1 | −0.117 (0.203) | 0.201 | 0.109 (0.203) | 0.199 | −0.073 (0.197) | 0.199 | −0.033 (0.195) | 0.196 |
| −0.2 | −0.197 (0.209) | 0.214 | 0.043 (0.216) | 0.213 | −0.146 (0.209) | 0.209 | −0.112 (0.203) | 0.207 |
| −0.5 | −0.519 (0.274) | 0.270 | −0.190 (0.271) | 0.271 | −0.446 (0.266) | 0.264 | −0.418 (0.264) | 0.263 |
| −0.8 | −0.842 (0.364) | 0.329 | −0.468 (0.358) | 0.328 | −0.755 (0.347) | 0.318 | −0.726 (0.352) | 0.319 |
|
| ||||||||
| n = 200, γ = 0.02 | ||||||||
|
| ||||||||
| −0.1 | −0.090 (0.215) | 0.200 | 0.116 (0.205) | 0.198 | −0.053 (0.200) | 0.198 | −0.009 (0.201) | 0.195 |
| −0.2 | −0.211 (0.216) | 0.214 | 0.051 (0.227) | 0.217 | −0.147 (0.219) | 0.212 | −0.115 (0.213) | 0.210 |
| −0.5 | −0.524 (0.276) | 0.265 | −0.195 (0.277) | 0.265 | −0.444 (0.261) | 0.258 | −0.416 (0.265) | 0.258 |
| −0.8 | −0.825 (0.357) | 0.331 | −0.460 (0.368) | 0.334 | −0.727 (0.348) | 0.324 | −0.704 (0.350) | 0.324 |
Acknowledgments
This research is sponsored in part by grant 1R01DK090046 from the National Institutes of Health
References
- 1.Albertsen PC, Hanley JA, Fine J. 20-year outcomes following conservative management of clinically localized prostate cancer. JAMA. 2005;293(17):2095–2101. doi: 10.1001/jama.293.17.2095. [DOI] [PubMed] [Google Scholar]
- 2.Brumback BA, Hernán MA, Haneuse S, Robins JM. Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Stat Med. 2004;23:749–767. doi: 10.1002/sim.1657. [DOI] [PubMed] [Google Scholar]
- 3.Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J of the Amer Stat Assoc. 1999;94:496–509. [Google Scholar]
- 4.Fine JP, Jiang H, Chappell R. On semi-competing risks data. Biometrika. 2001;88(4):907–919. [Google Scholar]
- 5.Fitzmaurice G, Davidian M, Verbeke G, Molenberghs G, editors. Longitudinal Data Analysis. Chapman & Hall/CRC; 2009. [Google Scholar]
- 6.Gooley TA, Leisenring W, Crowley J, Storer BE. Estimation of failure probabilities in the presence of competing risks: new representations of old estimators. Stat Med. 1999;18(6):695–706. doi: 10.1002/(sici)1097-0258(19990330)18:6<695::aid-sim60>3.0.co;2-o. [DOI] [PubMed] [Google Scholar]
- 7.Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: data mining, inference, and prediction. Springer-Verlag; New York, NY: 2003. [Google Scholar]
- 8.Holland PW. Statistics and causal inference. J of the Amer Stat Assoc. 1986;81(396):945–60. [Google Scholar]
- 9.Jeong J-H, Fine JP. A note on cause-specific residual life. Biometrika. 2009;96(1):237–242. [Google Scholar]
- 10.Kalbfleisch J, Prentice R. The Statistical Analysis of Failure Time Data. 2nd ed John Wiley & Sons; Hoboken, NJ: 2002. [Google Scholar]
- 11.Kattan MW. Comparative e ectiveness: a table of expected benefits and harms. Med Decis Making. 2009;29(6):NP 3–5. doi: 10.1177/0272989X09351585. [DOI] [PubMed] [Google Scholar]
- 12.Kattan MW. Doc, what are my chances? A conversation about prognostic uncertainty. European Urology. 2011;59:224. doi: 10.1016/j.eururo.2010.10.041. [DOI] [PubMed] [Google Scholar]
- 13.Kattan MW, Heller G, Brennan MF. A competing-risks nomogram for sarcoma-specific death following local recurrence. Stat Med. 2003;22:3515–3525. doi: 10.1002/sim.1574. [DOI] [PubMed] [Google Scholar]
- 14.Koller MT, Raatz H, Steyerberg EW, Wolbers M. Competing risks and the clinical community: irrelevance or ignorance? Stat Med. 2011 doi: 10.1002/sim.4384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Latini DM, Hart SL, Knight SJ, Cowan JE, Ross PL, DuChane J, Carroll PR. The relationship between anxiety and time to treatment for patients with prostate cancer on surveillance. J Urology. 2007;178(3):826–832. doi: 10.1016/j.juro.2007.05.039. [DOI] [PubMed] [Google Scholar]
- 16.Lok J, Gill R, van der Vaart A, Robins J. Estimating the causal e ect of a time-varying treatment on time-to-event using structural nested failure time models. Stat Neerlandica. 2004;58(3):271–295. [Google Scholar]
- 17.Lunceford JK, Davidian M. Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat Med. 2004;23(19):2937–2960. doi: 10.1002/sim.1903. [DOI] [PubMed] [Google Scholar]
- 18.Matsuyama Y, Yamaguchi T. Estimation of the marginal survival time in the presence of dependent competing risks using inverse probability of censoring weighted (IPCW) methods. Pharmaceutical Statistics. 2008;7(3):202–214. doi: 10.1002/pst.290. [DOI] [PubMed] [Google Scholar]
- 19.Pound CR, Partin AW, Eisenberger MA, Chan DW, Pearson JD, Walsh PC. Natural history of progression after PSA elevation following radical prostatectomy. JAMA. 1999;281:1591–1597. doi: 10.1001/jama.281.17.1591. [DOI] [PubMed] [Google Scholar]
- 20.R Development Core Team . R: A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: 2011. ISBN 3-900051-07-0, URL http://www.R-project.org/ [Google Scholar]
- 21.Robins JM. Information recovery and bias adjustment in proportional hazards regression analysis of randomized trials using surrogate markers. Proc of the Biopharm Section, Amer Stat Assoc. 1993:24–33. [Google Scholar]
- 22.Robins JM. Causal inference from complex longitudinal data. In: Berkane M, editor. Latent Variable Modeling and Applications to Causality. Springer-Verlag; New York: 1997. pp. 69–117. Lecture Notes in Statistics (120) [Google Scholar]
- 23.Robins JM, Finkelstein D. Correcting for non-compliance and dependent censoring in an AIDS clinical trial with inverse probability of censoring weighted (IPCW) log-rank tests. Biometrics. 2000;56:779–788. doi: 10.1111/j.0006-341x.2000.00779.x. [DOI] [PubMed] [Google Scholar]
- 24.Robins JM, Hernán MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11(5):550–560. doi: 10.1097/00001648-200009000-00011. [DOI] [PubMed] [Google Scholar]
- 25.Schaubel DE, Wei G. Double inverse-weighted estimation of cumulative treatment effects under nonproportional hazards and dependent censoring. Biometrics. 2011;67:29–38. doi: 10.1111/j.1541-0420.2010.01449.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Schellhammer P, Cockett A, Boccon-Gibod L, Gospodarowicz M, Krongrad A, Thompson IM, Scardino P, Soloway M, Adolfsson J. Assessment of endpoints for clinical trials for localized prostate cancer. Urology. 1997;49(4A Suppl):27–38. doi: 10.1016/s0090-4295(99)80321-5. [DOI] [PubMed] [Google Scholar]
- 27.Siegel R, Ward E, Brawley O, Jemal A. The impact of eliminating socioeconomic and racial disparities on premature cancer deaths. CA Cancer J Clin. 2011;61:212–236. doi: 10.3322/caac.20121. [DOI] [PubMed] [Google Scholar]
- 28.Shariat SF, Kattan MW, Vickers AJ, Karakiewicz PI, Scardino PT. Critical review of prostate cancer predictive tools. Future Oncol. 2009;5(10):1555–1584. doi: 10.2217/fon.09.121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Stephenson AJ, Scardino PT, Eastham JA, Bianco FJ, Dotan ZA, DiBlasio CJ, Reuther A, Klein EA, Kattan MW. Postoperative nomogram predicting the 10-year probability of prostate cancer recurrence after radical prostatectomy. J Clin Oncol. 2005;23(28):7005–70012. doi: 10.1200/JCO.2005.01.867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Tsiatis A. Semiparametric Theory and Missing Data. Springer Science+Business Media, LLC; 2006. [Google Scholar]
- 31.Thompson IM, Klotz L. Active surveillance for prostate cancer. JAMA. 2010;304(21):2411–2412. doi: 10.1001/jama.2010.1761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.van Houwelingen HC, Putter H. Dynamic Prediction in Clinical Survival Analysis. Chapman & Hall/CRC Press; 2012. [Google Scholar]


