Comparative Effectiveness Analysis of Lumpectomy and Mastectomy for Elderly Female Breast Cancer Patients: A Deep Learning-based Big Data Analysis

Abstract

Objectives: To evaluate the comparative effectiveness of treatments, a randomized clinical trial remains the gold standard but can be challenged by a high cost, a limited sample size, an inability to fully reflect the real world, and feasibility concerns. The objective is to showcase a big data approach that takes advantage of large electronic medical record (EMR) data to emulate clinical trials. To overcome the limitations of regression analysis, a deep learning-based analysis pipeline was developed. Study Design and Setting: Lumpectomy (breast-conserving surgery) and mastectomy are the two most commonly used surgical procedures for early-stage female breast cancer patients. An emulation trial was designed using the Surveillance, Epidemiology, and End Results (SEER)-Medicare data to evaluate their relative effectiveness in overall survival. The analysis pipeline consisted of a propensity score step, a weighted survival analysis step, and a bootstrap inference step. Results: A total of 65,997 subjects were enrolled in the emulated trial, with 50,704 and 15,293 in the lumpectomy and mastectomy arms, respectively. The two surgery procedures had comparable effects in terms of overall survival (survival year change = 0.08, 95% confidence interval (CI): -0.08, 0.25) for the elderly SEER-Medicare early-stage female breast cancer patients. Conclusion: This study demonstrated the power of “mining large EMR data + deep learning-based analysis,” and the proposed analysis strategy and technique can be potentially broadly applicable. It provided convincing evidence of the comparative effectiveness of lumpectomy and mastectomy.

Introduction

To evaluate the comparative effectiveness of treatments (drugs, operation procedures, etc.), a rigorously designed and executed randomized clinical trial (RCT) remains the criterion standard. However, challenges have been well recognized. With a high cost, RCTs, although statistically sufficiently powered by design, often have limited sample sizes. It has been repeatedly observed that treatment effects seen in the real world may differ (sometimes significantly) from RCTs, which can be attributed to the higher-than-standard care, higher adherence, and other factors in RCTs. Additionally, for treatments that have been on the market and widely used, there are feasibility concerns for conducting new head-to-head comparison trials. The fast accumulation of data from registries, large electronic medical records (EMR) and insurance claims databases, such as data contained in the SEER (Surveillance, Epidemiology, and End Results) registries, Medicare, Medicaid, and Kaiser, as well as the development of data management techniques, have made it possible to mine a large amount of real-world data to complement RCTs. Comparatively, such analysis can be advantageous with a much larger sample size and hence significantly more power, a better reflection of real-world evidence, high cost-effectiveness, and limited feasibility hurdles. Among the available observational data analysis methods (that aim to complement RCT analysis), emulation has attracted special attention, with its trial-like design, potential for causal interpretations, and scalability [1]. Emulation analysis has been conducted on cancers [2-4], cardiovascular diseases [5,6], and many other diseases [7-11]. The first objective of this study is to present another showcase of the emulation analysis of large EMR data.

In RCT analysis, regression techniques (such as linear, logistic, and Cox regressions as well as t and logrank tests) remain the common practice. This is also true for emulation analysis [1]. As in the real world there is no randomized assignment, emulation analysis usually demands an additional propensity score step, which can be accomplished by logistic regression [12]. Regression analysis is easy to conduct and has lucid interpretation. On the other hand, it is often challenged by “suboptimal” estimation and prediction performance and stringent model assumptions (and hence an inability to accommodate, for example, unknown nonlinear effects) – such limitations can be overcome by deep learning, which has experienced an unparallel surge in the past few years. Deep learning has demonstrated great power in diverse fields such as engineering, business, and social science [13-15]. It has been applied in biomedical research to the analysis of epidemiological [16], omics [17], imaging [18,19], and other types of data [20,21]. Our literature review suggests that the application of deep learning in the emulation analysis of observational data has been very limited. Additionally, “classic” deep learning techniques have been criticized as “black boxes.” As such, the second objective of this study is to develop a deep learning-based emulation analysis pipeline, which may have broad applications far beyond this study. The proposed deep neural network (DNN) approaches inherit strengths from well-established statistical principles and “classic” DNNs and can be superior to both.

For early-stage breast cancer, the surgical treatment options include mastectomy (which removes the entire breast) and lumpectomy (which removes a tumor and some normal tissues surrounding it). In the past decades, studies have hinted at the equivalency of the two surgical procedures, and both have been extensively conducted in clinical practice. However, RCTs may potentially suffer from a lack of generalizability due to extensive exclusion criteria, and previous evidence and findings from RCTs were based on cohorts from the 1980s [22-25]. Since then, there have been significant developments in breast cancer detection and treatment, such as a better understanding of tumor biology and advancements in surgical techniques and adjuvant therapy. With the period effect, patient characteristics have also changed significantly over time. It is thus of interest to compare breast cancer outcomes for patients who underwent the two surgical procedures in a more recent era. Another critical limitation of the existing RCTs and some observational studies is that there was insufficient attention to the elderly patients – many of them excluded elderly women or only included a very limited number of elderly women. The undertreatment and underrepresentation of elderly patients in studies may be attributable to limited cosmetic needs, a shorter life expectancy, an increasing incidence of comorbidities, and other factors. Breast cancer disproportionately affects older women, with those over 65 having the highest age-specific probability of developing invasive diseases [26,27]. The third objective of this study is to evaluate the comparative effectiveness of mastectomy and lumpectomy for the overall survival of elderly female breast cancer patients diagnosed and treated more recently. This effort can complement the existing RCT and observational studies and directly inform breast cancer clinical practice.

Methods

Data and Study Sample

The SEER-Medicare data was analyzed in this study. Surveillance, Epidemiology, and End Results (SEER) contains 22 cancer registries in the United States. Medicare is a federally-funded health insurance program in the US, and in 2010, approximately 94% of the US population age 65 years or older was enrolled in Medicare. The SEER-Medicare data provides an ideal opportunity to study the US elderly cancer population. It has been extensively analyzed, including in emulation analysis [3,28]. Here, it is noted that the adopted mining observational data for emulation analysis strategy and deep learning-based analysis pipeline can be directly applied to other databases. Data analyzed in this study was obtained through a Data Use Agreement (DUA) by the National Cancer Institute (NCI). The study design was motivated by relevant RCTs [22-25] and observational analysis [29-32], while taking into consideration data availability in the SEER-Medicare database. The target and emulated trial designs are described in Appendix Table S1. The flowchart of emulated trial patient selection is summarized in Figure 1. Briefly, the emulated trial enrolled Medicare beneficiaries with first primary invasive intraductal or/and lobular breast cancer (International Classification of Diseases for Oncology, 3rd Edition (ICD-O-3): 8500, 8520, and 8522) who were: 1) female and 66-100 years old at diagnosis, 2) diagnosed between 1/1/07 and 12/31/17, 3) stage I-II, 4) with tumor size ≤ 5cm, 5) received lumpectomy or mastectomy within 120 days after diagnosis, and 6) had continuous enrollment in Medicare Part A and Part B, with no health maintenance organization (HMO) enrollment from one year before to one year after the cancer diagnosis or death, whichever occurred first. A subject was excluded if she 1) was reported from registries in New York (NY), Massachusetts (MA), and Idaho (ID) because of missing cancer-related information (eg, stage), 2) had missing information on month of diagnosis, gender, stage or tumor size, and 3) was reported from death certificate or autopsy.

Figure 1.

Flowchart of sample selection.

Variable Information

After reviewing the relevant observational studies, and also considering data availability, we included the following variables as potential confounders, which may have an impact on treatment initiation (choice of surgery) and outcome (overall survival): 1) baseline demographics: age, race, and marital status; 2) cancer-related variables: tumor size, morphology, stage, primary site, laterality, grade, HR status; and 3) comorbidities: Elixhauser comorbid conditions index. It is noted that human epidermal growth factor receptor 2 (HER2) was not measured for patients diagnosed prior to 2010, and, as such, it was not included in the main analysis. In addition, as only a few (<11) patients had unknown laterality or paired site/midline tumor, they were excluded from the analysis. A patient was enrolled into the lumpectomy arm if she received lumpectomy as the first surgery procedure after being diagnosed and would be censored if a subsequent mastectomy was performed. A patient was enrolled into the mastectomy arm if she received mastectomy as the first surgery procedure after being diagnosed regardless of subsequent surgical procedures. The International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM), ICD-10-CM, and the Healthcare Common Procedure Coding System (HCPCS)/Current Procedural Terminology (CPT) codes appeared in Medicare inpatient and outpatient records were adopted to define surgeries. Details are provided in Appendix A: Table S2. Time zero was set as the time of surgery. Immortal time bias, if any, is expected to be limited, as the two treatment groups were comparable and the mortality rate right after diagnosis was very low compared to the long follow-up. The overall survival information was obtained from the Medicare records. All subjects were followed to the end of 2019 or death, whichever happened first.

A DNN-based Analysis Pipeline

Under the regression framework, the emulation technique has been well developed. We refer to [1,33] for methodological developments and [2,34] for representative case studies. Additionally, some computer codes are also publicly available, for example at [35]. Briefly, to emulate the estimation of an intention-to-treat (ITT) effect, the observed treatments in medical records are used as treatment assignments. Special considerations may be needed if a patient received multiple treatments. In a well-executed RCT, different arms can be sufficiently balanced. As such, there may be no need for accounting for confounding – although it is still commonly done out of caution. In the emulation analysis of observational data, since randomization did not really happen, there is a risk of imbalance. To tackle this problem, the propensity score and inverse probability treatment (IPT) weighting techniques are usually adopted to achieve baseline balance as in an RCT. The propensity score calculation is usually built on logistic regression that includes the treatment indicator as response and baseline covariates as covariates. Usually, this analysis only includes linear effects. Based on the logistic regression results, the IPT weight can be calculated as the inverse of the propensity score for one treatment group and the inverse of one minus the propensity score for the other group. Stabilized weights with a truncation at the upper 99.5% percentile (that is, if a weight is above the 99.5% percentile, it is set as the 99.5% percentile) are usually adopted for retaining the same group size ratio and avoiding putting extreme weights on certain subjects. With such weights, a pseudo-population can be created, for which there is no (or very weak) association between the baseline confounders and received treatment. Then, survival analysis, for example, Cox regression, can be conducted using the weighted pseudo-population to estimate the causal effect under the assumption of no unmeasured confounders. Similar to logistic regression, Cox and other survival analysis models also rely on strong assumptions and linear effects. In practice, it is not uncommon to observe violations of such assumptions.

Our overall strategy is to follow the above framework but replace regression-based analysis with DNN-based analysis, which relies on weaker model assumptions and can “automatically” accommodate unspecified nonlinear relationships. The architecture of the DNNs for both propensity score and survival analysis steps is sketched in Figure 2 and consists of a sparse layer, multiple hidden layers, and an output layer. The key building components of a DNN include its loss function and estimation approach, which are presented below. Denote n as the number of independent subjects. For subject i, we observe p baseline covariates X_i = (X_i1, ... , X_ip)^T, a binary treatment indicator Z_i, and right censored survival outcome {Y_i = min(T_i,C_i), δ_i = I(T_i ≤ C_i)}. Here, T_i, C_i are the event and censoring times, respectively, and I(∙) is the indicator function.

Figure 2.

Scheme of the proposed DNN-based analysis.

Propensity score step. A feed-forward DNN is constructed with input being X_i’s after standardization for continuous variables and one-hot encoding for categorical variables and output being Z_i,’s. EMR databases contain rich information, and collecting data from such databases is almost free. As such, to avoid losing important information, “more than necessary” covariates may be included. In addition, in regression-based emulation analysis, it is also commonly observed that some covariates are not significant. For the proposed DNN, we first propose imposing group Lasso penalization to the input layer weights. In particular, the weights corresponding to one input variable are treated as a group. Group Lasso has been extensively developed under the regression framework and has the “group in or group out” selection property [36]. With this penalty, we are able to distinguish important input variables from noises, which can lead to a simpler DNN structure, fewer variables and hence more stable estimation, and more lucid interpretations. Additionally, we also propose applying ridge penalization [37] to the hidden layer weights. Here, the goal is to regularize estimation and avoid extreme values. In principle, it is possible to also apply group Lasso (or its individual counterpart Lasso). However, this may lead to an overly sparse DNN and unstable optimization.

Use α₁,_j, a vector, where j = 1, ... ,p, to denote the input layer weights corresponding to the jth input variable, and A_k1, a matrix, where k₁ = 1, ... ,K₁, to denote the weights of the k₁th hidden layer. Additionally, use A to collectively denote all weights. For subject j denote $p_i=\frac{1}{1+\exp \left(-h_{\mathrm{{\rm A}}}\left({\mathit{X}}_i\right)\right)}$ as the probability output (that is, a Sigmoid activation function is adopted for the output layer). Intuitively, h_A(X_i) corresponds to the covariate effect in logistic regression. For the input layer and hidden layers, respectively, we adopt the Tanh and Hardtanh activation functions. We adopt the logarithmic loss function (binary cross-entropy), which has connections with the logistic likelihood function, and has been popularly adopted. This has been partly motivated by the fact that, in the literature, logistic regression has been popularly adopted in propensity score estimation and has demonstrated sensible performance. The overall penalized loss function is:

$l_1=-\frac{1}{n}{\sum }_{i=1}^n\left(Z_i\times h_{\mathrm{{\rm A}}}\left({\mathit{X}}_i\right)-\log \left(1+\exp \left(h_{\mathrm{{\rm A}}}\left({\mathit{X}}_i\right)\right)\right)\right)+{\lambda }_1{\sum }_{j=1}^p{‖{\mathit{\alpha }}_{1,j}‖}_2+{\lambda }_2{\sum }_{k_1=1}^{K_1}{‖{\mathbf{{\rm A}}}_k‖}_F^2$

where λ₁ and λ₂ are tuning parameters, ║∙║₂ denotes the l₂-norm, and ║∙║_F denotes the Frobenius norm. Denote the estimate of A as  and ${\widehat{p}}_i=\frac{1}{1+\exp \left({-h}_{\widehat{\mathrm{{\rm A}}}}\left({\mathit{X}}_i\right)\right)}$. Following regression-based analysis, we calculate the inverse probability treatment weight: if subject i is in the treatment arm, ${\widehat{\mu }}_i=\frac{1}{{\widehat{p}}_i}$; otherwise, ${\widehat{\mu }}_i=\frac{1}{1-{\widehat{p}}_i}$.

Survival analysis step. In regression analysis, the Cox model has been widely used because of clear interpretation and easy implementation, although alternative models have also been extensively adopted. In deep learning, a loss function that corresponds to the negative partial likelihood function of the Cox model has been developed [38]. However, it is not as popular as its counterpart under regression. Here, we develop an alternative loss function.

For subject i = 1, ... ,n, consider weight ${\pi }_i=\frac{{\delta }_i}{S\left(Y_i\right)}$ where S(t) is the survival function for the event time of interest. In practice, we estimate S(t) using the Kaplan-Meier estimator Ŝ(t) and denote the corresponding weight as ${\widehat{\pi }}_i$. We construct a feed-forward weighted DNN with (Z_i,X_i)’s as input and Y_i’s as output. For subject i, ${\widehat{\pi }}_i$ is imposed as weight. Note that the weight calculation does not depend on the DNN construction. In addition, censored subjects have zero weights, and as such, are virtually removed from the DNN analysis. For the input layer, the Tanh activation function is adopted. For the hidden and output layers, the Rectified Linear Units (ReLU) activation function is adopted. For the jth input variable, use β₁,_j, a vector, to denote its input layer weights. For the k₂th (= 1, ... ,K₂) hidden layer, use B_k2, a matrix, to denote its collection of weights. And use B to collectively denote all network weights. For subject i denote f_B(Z_i, X_i) the network output. Following the same penalization strategy as above, we propose the loss function: $l_2=\frac{1}{{\sum }_i^{}\mathrm{I}\left\{{\delta }_i\ne 0\right\}\times {\widehat{\mu }}_i\times {\widehat{\pi }}_i}{\sum }_{\left\{i:{\delta }_i\ne 0\right\}}^{}{\widehat{\mu }}_i\times {\widehat{\pi }}_i\times {\left(Y_i-f_{\mathrm{{\rm B}}}\left(Z_i,{\mathit{X}}_i\right)\right)}^2+{\lambda }_3{\sum }_{j=1}^{p+1}{‖{\mathit{\beta }}_{1,j}‖}_2+{\lambda }_4{\sum }_{k_2=1}^{K_2}{‖{\mathbf{{\rm B}}}_{k_2}‖}_F^2$, where λ₃ and λ₄ are tuning parameters. The first term has a least squares form, which makes computation simple.

The proposed loss has been motivated by the accelerated failure time (AFT) model [39]. Compared to the Cox model, the AFT model can sometimes be preferred because of its more lucid interpretation and simpler computation. Although it has been widely adopted in regression, its DNN development has not been well pursued. This study can fill this important knowledge gap. There can be multiple estimation approaches under the AFT modeling. We adopt the weighted least squares technique [40,41], which has the lowest computational cost and can be easily combined with the IPT weighting.

Inference on treatment effect. In RCT analysis, quantification of the significance of the treatment effect (using p-value, confidence interval, etc.) can be as important as estimation. In most of the existing DNN studies, there has been a lack of attention to the effect of an individual input variable and the quantification of its significance. To fill this knowledge gap, for the proposed analysis, we propose estimating treatment effects through a permutation of the treatment indicator Z_i for each subject. That is, the population treatment effect is defined as the average increase/decrease in predicted survival time if switching Z_i = 1 to Z_i = 0 (and vice versa) for the whole cohort. For inference, we propose a weighted bootstrap technique, which has been motivated by the weighted bootstrap for regression analysis [42] and consists of the following steps:

1) Randomly generate η₁, ... , η_η from exp(1), the exponential distribution with rate = 1.

2) Conduct the weighted penalized estimation. In particular, the first term in the survival analysis objective function is revised as $\frac{1}{{\sum }_i^{}\mathrm{I}\left\{{\delta }_i\ne 0\right\}{\eta }_i\times {\widehat{\mu }}_i\times {\widehat{\pi }}_i}{\sum }_{\left\{i:{\delta }_i\ne 0\right\}}^{}{\eta }_i\times {\widehat{\mu }}_i\times {\widehat{\pi }}_i\times {\left(Y_i-f_{\mathrm{{\rm B}}}\left(Z_i,{\mathit{X}}_i\right)\right)}^2$. The penalty terms remain unchanged.

3) Repeat Steps 1) - 2), eg, 500 times. The resulted estimates are used to construct a confidence interval (CI) for the treatment effect.

Computation. The bootstrap analysis can be conducted using the same approach as without the exponential weights. For both the propensity score and survival analysis steps, we adopt state-of-the-art techniques, including input standardization, Adaptive Moment Estimation for the gradient descent algorithm, Nesterov momentum, and learning rate scheduling with the exponential decay technique. We perform a Random hyperparameter optimization search and tune network depth and size, learning rate, penalization parameters, dropout rate, exponential learning rate decay constant, and momentum, with the assistance of the Python package Optunity. For tuning parameter selection, we conduct a grid search and minimize the cross entropy in the propensity score step and the weighted mean squared error in the survival analysis step. As the sample size is much larger than the number of parameters, cross-validation is not adopted but can be easily revised. The open-source Python module PyTorch is adopted.

Sensitivity analysis. In the main analysis, HER2 was not included because of complete missingness for patients diagnosed prior to 2010. It is recognized that HER2 is an important confounder and may impact both surgical choices and overall survival. In Appendix A: S1, analysis was conducted on a revised cohort with available HER2 information (that is, patients diagnosed in or after 2010), and HER2 was included as a confounder. Additionally, it was recognized that whether to perform radiotherapy was still being debated. Also motivated by the observed high proportion of patients receiving adjuvant radiotherapy after lumpectomy, we conducted the second sensitivity analysis in Appendix A: S2 and compared the overall survival of patients receiving mastectomy without adjuvant radiation therapy versus lumpectomy with radiation therapy. The ICD-9-CM, ICD-10-CM, HCPCS/CPT codes for identifying radiation therapy are provided in Appendix Table S2. We further fit a standard Cox regression model that included all the confounders and compared the results with those using the deep learning approach.

Results

As shown in Figure 1, a total of 65,997 patients were enrolled in the emulated trial, with 50,704 in the lumpectomy arm and 15,293 in the mastectomy arm. The waiting time from diagnosis to surgery was observed to be short – over 95% of the surgeries were performed within 120 days after diagnosis. The baseline characteristics are summarized in Table 1. It was observed that before the IPT weighting, the patients treated with lumpectomy were slightly younger, more likely to be stage I and with smaller tumor size and had lower comorbidity scores.

Table 1. Patients’ Characteristics by Treatment.

	Mastectomy (n = 15,293)	Lumpectomy (n = 50,704)	Overall (n = 65,997)
Demographics
Age at diagnosis	76.1 (6.85)	75.3 (6.55)	75.5 (6.63)
Race (Non-Hispanic White)	12474 (81.6%)	43038 (84.9%)	55512 (84.1%)
Marital Status (married)	6715 (43.9%)	25048 (49.4%)	31763 (48.1%)
Year of diagnosis
2007	1571 (10.3%)	4586 (9.0%)	6157 (9.3%)
2008	1540 (10.1%)	4535 (8.9%)	6075 (9.2%)
2009	1493 (9.8%)	4504 (8.9%)	5997 (9.1%)
2010	1456 (9.5%)	4468 (8.8%)	5924 (9.0%)
2011	1472 (9.6%)	4575 (9.0%)	6047 (9.2%)
2012	1512 (9.9%)	4508 (8.9%)	6020 (9.1%)
2013	1475 (9.6%)	4493 (8.9%)	5968 (9.0%)
2014	1373 (9.0%)	4561 (9.0%)	5934 (9.0%)
2015	1244 (8.1%)	4721 (9.3%)	5965 (9.0%)
2016	1126 (7.4%)	4902 (9.7%)	6028 (9.1%)
2017	1031 (6.7%)	4851 (9.6%)	5882 (8.9%)
Tumor characteristics
Morphology
Intraductal	12194 (79.7%)	42100 (83.0%)	54294 (82.3%)
Lobular	2046 (13.4%)	5498 (10.8%)	7544 (11.4%)
Intraductal and Lobular	1053 (6.9%)	3106 (6.1%)	4159 (6.3%)
Tumor size (mm)	20.6 (11.1)	14.6 (8.74)	16.0 (9.68)
Stage (II vs ref: I)	8235 (53.8%)	14236 (28.1%)	22471 (34.0%)
Primary site
Nipple	74 (0.5%)	186 (0.4%)	260 (0.4%)
Central	1193 (7.8%)	2168 (4.3%)	3361 (5.1%)
UIQ	1682 (11.0%)	6942 (13.7%)	8624 (13.1%)
LIQ	878 (5.7%)	3203 (6.3%)	4081 (6.2%)
UOQ	4671 (30.5%)	18704 (36.9%)	23375 (35.4%)
LOQ	1133 (7.4%)	3717 (7.3%)	4850 (7.3%)
Axillary tail	38 (0.2%)	207 (0.4%)	245 (0.4%)
Overlapping lesion	3417 (22.3%)	11850 (23.4%)	15267 (23.1%)
Breast, NOS	2207 (14.4%)	3727 (7.4%)	5934 (9.0%)
Laterality
Right: origin of primary	7454 (48.7%)	25053 (49.4%)	32507 (49.3%)
Left: origin of primary	>7828 (>51.2%)	>25640 (>50.6%)	>33479 (>50.7%)
Other or unspecified	<11	<11	<11
Grade
I	3266 (21.4%)	14862 (29.3%)	18128 (27.5%)
II	7281 (47.6%)	24097 (47.5%)	31378 (47.5%)
III	4366 (28.5%)	10376 (20.5%)	14742 (22.3%)
IV	43 (0.3%)	103 (0.2%)	146 (0.2%)
Unknown	337 (2.2%)	1266 (2.5%)	1603 (2.4%)
HR status
Positive or borderline	12742 (83.3%)	44922 (88.6%)	57664 (87.4%)
Negative	2111 (13.8%)	4856 (9.6%)	6967 (10.6%)
Unknown	440 (2.9%)	926 (1.8%)	1366 (2.1%)
Elixhauser Comorbidities
Comorbidity index	9.62 (8.28)	7.47 (6.79)	7.97 (7.22)

*For a categorical variable, count (percent). For a continuous variable, mean (standard deviation).

The unadjusted Kaplan-Meier survival curves by treatment are shown in Figure 3. For the lumpectomy arm, the overall mortality rate and median survival were 22.6% and 6.0 years, respectively. For the mastectomy arm, they were 35.5% and 5.8 years, respectively. The patients who received lumpectomy were observed to have longer overall survival (p-value < 0.001 by logrank test). The proportional hazards assumption was tested, and the global Chi-squared test returned a p-value < 0.001, which justifies the adoption of a more flexible modeling technique.

Figure 3.

Unadjusted survival analysis.

In the propensity score analysis, we evaluated the impact of tuning parameters by visualizing parameter paths, which is a common technique for penalized estimation [43]. The parameter paths as a function of tuning parameter λ₁ are shown in Figure 4a. It was observed that tumor size was the only variable that was included when λ₁ was large, suggesting its higher importance. As the penalty level decreases, all variables can be included into the propensity score model. For example, race and HR status positive also have relatively larger group norms, which suggests that those variables are important factors and may be associated with the decision-making between lumpectomy and mastectomy. Figure 5a and b show the distributions of the estimated propensity scores and weights. The two treatment groups have visibly different propensities towards surgical options. As shown in Figure 5c, prior to the weighting, some of the absolute standardized mean difference (SMD) values were above the 0.1 threshold, especially for tumor size and stage. Both variables are important factors to be considered when making a choice between lumpectomy and mastectomy. All the absolute SMDs significantly reduced to be below 0.1 and even nearly 0 with the weighting, with which the two arms could be considered as balanced. For comparison, we also considered the standard logistic regression approach. The Chi-squared test for deviance returned a p-value < 0.001, which suggested unsatisfactory model fitting and hence justified the adoption of DNN.

Figure 4.

Parameter paths for the propensity score analysis (upper) and survival analysis (lower).

Figure 5.

Distributions of estimated (a) propensity score and (b) inverse probability treatment weights; and (c) absolute standard mean difference of all confounders before and after weighting.

In the survival analysis step, we also examined the parameter paths to assess the relative importance of input variables. As shown in Figure 4b, overall survival was found to be highly related to Elixhauser comorbidities and age, which remained in the model even with larger tuning parameter values (and hence higher penalty). Subsequently, tumor size and tumor grade III or IV were found to contribute to survival. Based on the obtained DNN, the predicted subject-level survival would, on average, slightly increase, if a patient would have switched from lumpectomy to mastectomy. The result is summarized in Figure 6a. With the predicted survival time for each patient (without censoring), the Kaplan-Meier survival curves were constructed and shown in Figure 6b and c. In Figure 6b, the treatment assignment was created to be random. The survival curves of the predicted survival times for the two arms almost completely overlap (p-value = 0.2 by logrank test). In this assessment, the two surgical groups are balanced for treatment assignments, mimicking the randomization process in a well-executed RCT. In comparison, in Figure 6c, a beneficial effect of lumpectomy (p-value < 0.001 by logrank test) was observed, if the treatment assignment was based on what was observed and without addressing potential confounding. This assessment mimics real-world observations where there are always some inherent differences between those receiving different surgical procedures. This may also explain why some observational studies concluded that lumpectomy was associated with improved survival [29-32]. Overall, we concluded that lumpectomy and mastectomy had similar effects on overall survival. For all subjects in the studied cohort, the average estimated increase in overall survival is 0.05 years if a patient received lumpectomy in comparison to mastectomy. With the weighted bootstrap inference, the average mean survival year change (across bootstraps) was found to be 0.08, with a standard deviation of 0.08 and 95% CI [-0.08,0.25].

Figure 6.

Analysis of predicted survival: (a) predicted subject-level improvement in survival if a patient would switch treatment; (b) survival probabilities based on the predicted survival times if the treatment assignment was random; and (c) survival probabilities with patients under the original observed treatments.

In the sensitivity analysis with respect to HER2, a similar conclusion was made. In particular, the average estimated increase in overall survival is 0.094 years if a received lumpectomy in comparison to mastectomy. With the weighted bootstrap inference, the average mean survival year change was found to be 0.14, with a standard deviation of 0.08 and 95% CI [-0.02,0.29]. In the sensitivity analysis with respect to radiotherapy, it was concluded that lumpectomy with radiotherapy was superior. The average estimated increase in overall survival is 0.30 years if a patient received lumpectomy in comparison to mastectomy. With the weighted bootstrap inference, the average mean survival year change was found to be 0.29, with a standard deviation of 0.09 and 95% CI [0.13,0.49]. It is noted that the definition of time zero changed and that the results could be subject to immortal time bias (which may result in a misleading conclusion on the beneficial treatment effect of lumpectomy with radiation therapy). As such, this conclusion should be taken with caution. With the standard Cox regression analysis, it was found that lumpectomy was slightly superior, and the hazard ratio was 0.96, with 95% CI [0.92,0.99]. As the Cox model failed model diagnostics, the conflict with the deep learning-based analysis is not surprising. Additionally, it is noted that the estimated treatment under the Cox model is in fact very small.

Discussion

This article has provided another showcase of mining large observational data, extracting valuable information, complementing RCTs, and informing clinical practice. In our data analysis, the sample size is dramatically larger than regular RCTs, making the analysis much more powerful. It is usually not easy to significantly increase sample sizes in RCTs. However, with the accumulation of data, the sample size of this study and other observational data analyses can easily keep increasing. The analyzed real-world data can be closer to real clinical practice than RCTs. With a much broader coverage and more diverse population, the findings can be more generalizable than RCTs. As acknowledged in the literature [34,44], emulation and other big data analysis of observational data can be the most useful for assessing the effectiveness of existing treatments (that have not been directly compared in RCTs) and updating comparisons (that have been done years ago on patients with different characteristics). It is noted that the proposed analysis can also be done with local data, for example, generated by the Yale New Haven Healthcare System, to better reflect local clinical practice and patient characteristics. With the promising performance of deep learning on other biomedical problems (for example, imaging and omics data analysis), it is a natural step to develop deep learning-based emulation analysis. The proposed analysis pipeline has combined the strengths of both statistical analysis and “regular” DNNs. Specifically, key building blocks of the “standard” DNN architecture have been retained. As such, the proposed analysis, similar to other DNNs, can have better estimation/prediction performance and more flexibly accommodate unspecified nonlinear relationships. It can be especially useful when the logistic, Cox, and other model assumptions are not satisfied, as in this data analysis. Here, it is noted that some recent studies have pointed out that DNNs may not be entirely model-free [45,46]. Our literature review suggested that there is still a lack of model diagnostics tools for DNNs – it will be of interest to diagnose the proposed DNNs when such tools become available. On the other hand, the proposed analysis has inherited the lucid framework of emulation analysis, with the clear propensity score and survival analysis steps. Additionally, it has incorporated penalized estimation, examination of parameter paths, and bootstrap-based inference, which are routine in regression analysis but still have not been well developed in deep learning. It is noted that deep learning is not “free.” Different from regression analysis, there is a lack of simple models. The resulting DNNs are available from the authors, however, do not have easily interpretable forms. Additionally, they do not deliver simple statistics such as hazard ratio. The proposed assessment based on the predicted survival time can largely alleviate this problem.

In data analysis, our main finding is that lumpectomy and mastectomy have comparable long-term overall survival for early-stage elderly female breast cancer patients. It is noted that the studied population differs from many of the existing studies. The conclusion is consistent with most of the existing RCTs and can complement the existing literature. For example, the National Surgical Adjuvant Breast and Bowel Project (NSABP) B-04 study [25] and the study at the National Cancer Institute in Milan [47] both suggested no difference in 20-year overall survival after the two surgical procedures. More recent observational studies, interestingly, reached different conclusions. For example, Agarwal et al. [29], Hwang et al. [30], and Wrubel et al. [32] concluded that lumpectomy was superior in terms of breast cancer-specific and overall survival compared to mastectomy. The difference in findings can be at least partly attributed to the differences in study populations (for example, different age groups and years of diagnosis). Mogal et al. [31] specially analyzed elderly patients and also concluded that lumpectomy was significantly superior in terms of overall survival. It is noted that all of the above studies adopted regression-based association analysis techniques. Most of them did not report any model diagnostics, and there could be a risk of model misspecification. Additionally, some adopted the Kaplan-Meier technique without addressing possible confounding bias.

Limitations of the proposed analysis are fully recognized. SEER-Medicare and other EMR and insurance claims databases may not have comprehensive and accurate information as RCTs. For example, to capture most of the observed treatments of interest, we had an additional inclusion criterion in our emulated trial (compared to the target trial) as having at least one year or until death enrollment in Medicare part A or B. This may limit the sample size and introduce a selection bias to our analysis. For the main data analysis, unmeasured and possibly relevant confounders may include HER2 status, lymphovascular invasion, extracapsular invasion, and size of nodal metastases, which may have an impact on both surgical choices and prognosis. This is the practice paid for collecting data for a large population without specific targets as in RCTs. To partially address this limitation, we conducted a sensitivity analysis with respect to HER2 status. The proposed analysis may suffer the same limitations as the existing emulation techniques. For example, it demands a proper definition of treatment assignment and time zero. Otherwise, it may lead to falsely defined treatment effects and succumb to immortal time bias. In the main analysis, we omitted the subsequent chemotherapy, radiotherapy, or endocrine therapy before and after surgery, as our main interest was in the two surgical procedures. As discussed in [29], the indication for systemic chemotherapy is comparable for both surgical groups, thereby minimizing a disproportionate impact of chemotherapy. To partially mitigate this limitation, we additionally conducted a sensitivity analysis with respect to adjuvant radiation therapy performed after surgery. However, including neoadjuvant and adjuvant therapies in the analysis may pose a challenge to the proper definition of treatment and choice of time zero. Future research will be needed to address the corresponding immortal time bias – otherwise, we may not be able to perform a valid emulation analysis (but more of an observation analysis informing association). In the analysis, data from a time window much wider than regular RCTs was pooled. The analysis can be repeated for shorter time windows to examine potential temporal variations. It is also recognized that adopting DNN (and other complex modelings) may have the risk of overfitting. In this study, this risk can be largely mitigated via penalization, the relatively simple structure of the DNNs, and the large sample size.

Conclusion

This study has further advanced big data analysis for biomedicine by developing a deep learning-based emulation analysis pipeline for mining large observational data and complementing RCTs. Equally importantly, the analysis of SEER-Medicare data has led to the conclusion that lumpectomy and mastectomy have similar effects on overall survival for elderly SEER-Medicare early-stage female breast cancer patients. This finding can directly inform breast cancer clinical practice.

Glossary

EMR

electronic medical record

SEER

Surveillance, Epidemiology, and End Results

RCT

randomized clinical trial

DNN

deep neural network

ICD-O-3

International Classification of Diseases for Oncology, 3rd Edition

ICD-9/10-CM

International Classification of Diseases, Ninth/Tenth Revision, Clinical Modification

HCPCS

Healthcare Common procedure Coding System

CPT

Current Procedural Terminology

IPT

inverse probability treatment

AFT

accelerated failure time

ReLU

Rectified Linear Units

UOQ

upper outer quadrant

UIQ

upper inner quadrant

LOQ

lower outer quadrant

LIQ

lower inner quadrant

SMD

standardized mean difference

Appendix A.

yjbm_96_3_327_appendix.pdf ^{(166.9KB, pdf)}

Funding

This study was partly supported by NIH HL161691 and CA276790.

Author Contributions

Conceptualization: JW, YH, and SM; methodology: JW, SZ, and SM; software: SZ and JW; validation: JW and SM; formal analysis: JW; investigation: JW, YH, and SM; data curation: JW; writing—original draft preparation, all; writing—review and editing, all; supervision: SM. All authors have read and agreed to the published version of the manuscript.