Abstract
Background:
Several prognostic models have been proposed and demonstrated to be predictive of survival outcomes in breast cancer. In the present article, we assessed whether three of these models are comparable at an individual level.
Methods:
We used a large data set (n=965) of women with hormone receptor-positive and HER2-negative early breast cancer from the public data set of the METABRIC (Molecular Taxonomy of Breast Cancer International Consortium) study. We compared the overall performance of three validated web-based models: Adjuvant!, CancerMath.net and PREDICT, and we assessed concordance of these models in 10-year survival prediction.
Results:
Discrimination performances of the three calculators to predict 10-year survival were similar for the Adjuvant! Model, 0.74 (95% CI 0.71–0.77) for the Cancermath.net model and 0.72 (95% CI 0.69–0.75) for the PREDICT model). Calibration performances, assessed graphically, were satisfactory. Predictions were concordant and stable in the subgroup, with a predicted survival higher than 90% with a median score dispersion at 0.08 (range 0.06–0.10). Dispersion, however, reached 30% for the subgroups with a predicted survival between 10 and 50%.
Conclusion:
This study revealed that the three web-based predictors equally perform well at the population level, but exhibit a high degree of discordance in the intermediate and poor prognosis groups.
Keywords: breast cancer, survival, adjuvant treatment, risk assessment, predictive models
In breast cancer management, the decision to give adjuvant treatments depends on several factors such as survival benefits and complication risk. Software tools have been developed to aid this decision-making process. They estimate the probability of cancer-related mortality and relapse and calculate the percentage benefit of adjuvant treatments.
The web-based software program Adjuvant online! (www.adjuvantonline.com) (Ravdin, 1996) was developed from the American National Cancer Institute SEER registry (Surveillance, Epidemiology, End-Results data and estimates). The data represented ∼10% of all breast cancer patients in the United States aged between 36–59 years and treated between 1988 and 1992. It calculates the risk of relapse and 10-year cancer-related mortality with and without adjuvant therapy post cancer surgery. The outcomes are calculated using patient age, tumour size, tumour grade, oestrogen receptor (ER) status and node involvement. This model has been externally validated in different populations (Olivotto et al, 2005; Campbell et al, 2009; Mook et al, 2009; Schmidt et al, 2009; Paridaens et al, 2010; Yao-Lung et al, 2012) and is widely used by practitioners (Oakman et al, 2010). There are, however, limitations as accuracy is diminished in high-risk groups where prognosis may be overestimated (young age, high-grade and HER2-positive patients) (Hajage et al, 2011; Bhoo-Pathy et al, 2012).
In 2009, the SNAP (size, nodes, and prognostic factors) method (Chen et al, 2009) (www.CancerMath.net) was developed from 1352 breast cancer patients treated at the University of Southern California/Van Nuys Breast Center between 1966 and 2006. It predicts the risk of death for the first 15 years after diagnosis and the impact of various adjuvant therapies. Survival rates are calculated using tumour size, lymph node status, ER/progesteron receptor/HER2 status, tumour grade and histological type. To date, one unit has published its validation of SNAP(Michaelson et al, 2011). They found that SNAP provided accurate estimates of risk of death when compared against large patient data sets.
A third web-based prediction tool, PREDICT, was described in 2010 (www.predict.nhs.uk), which estimates 5- and 10-year breast cancer survival and the benefits of hormonal therapy, chemotherapy and trastuzumab (Wishart et al, 2010). The model was derived from the cancer registry information on 5694 breast cancer patients treated in East Anglia from 1999 to 2003 and used factors such as patient age, tumour size, tumour grade, number of positive nodes, ER status, Ki67 status and mode of detection. An external data set of 5468 patients from the West Midlands Cancer Intelligence Unit was used for validation (Wishart et al, 2010).
A recent review on six prediction tools (including Adjuvant!, CancerMath.net and PREDICT) and 14 biomolecular features of breast cancer found that most models were inaccurate in particular patient subgroups (Engelhardt et al, 2014). To date, there is limited data on the concordance of several prediction models. In addition, further validation of multiple models with several multinational large patient data sets is needed.
The aim of this study was to validate and to determine the concordance of three web-based prediction tools Adjuvant!, PREDICT and CancerMath.net against a patient data set consisting of ER-positive and HER2-negative breast cancer.
Patients and Methods
Patients
METABRIC data set. We retrieved 965 ER-positive and HER2-negative cases from the METABRIC study (Molecular Taxonomy of Breast Cancer International Consortium) (Curtis et al, 2012). This data set contains over 2000 clinically annotated primary fresh frozen breast cancer specimens and a subset of normals from tumour banks in the United States and Canada. Details about the METABRIC cohort have been published by Curtis et al in 2012. This public data set accessible from the web (METABRIC Data for Use in Independent Research—syn1688369, 2014).
To automatically apply the models to the data set, we used the HTML code from the web pages of the calculators (Predict and Cancermath.net models) and we generated a partition tree for Adjuvant! (the information that the Adjuvant! algorithm uses to estimates the risk are binned into categories and the output is fixed in the number of possibilities).
Some parameters could be not available in the data set, as the chemotherapy generation (required for the PREDICT and the Adjuvant! models). We considered that all the patients did receive a second generation, as the data were older than 10 years.
Statistical analysis
Epidemiological and clinical continuous variables were compared using Student's t-test (parametric) and the Mann–Whitney test (non-parametric). Categorical variables were compared using the χ2-test or Fisher's exact test where appropriate, as appropriate. P-values <0.05 was deemed statistically significant. Continuous variables data are presented as mean ±s.d. or median and IQR or min/max value.
As the three calculators provide 10-year overall survival estimates, we decided to compare the models using this end point. The performances for the three web calculators were quantified using calibration and discrimination. Calibration quantifies the concordance between observed frequencies and predicted probabilities. Calibration curves graphically depict the relationship between observed outcome frequencies and predicted probabilities by the models. Well-calibrated models have an intercept a=0 and a slope b=1 (ideal predictions are plotted on a line that crosses the origin). Therefore, a sensible measure of calibration (the unreliability index) is a likelihood ratio statistic testing the null hypothesis that a=0 and b=1. Of note, the unreliability [U]-statistic is satisfactory when the P-value is not significant.
Discrimination (i.e., whether the relative ranking of individual predictions was in the correct order) was quantified in both populations with the C-Index (corresponding to a measure of the area under curve for censored data) (Pencina and D'Agostino, 2004).
The Brier score was also calculated, corresponding for a given model, to the expected squared difference between patient status and predicted probability. The lower the Brier score of a model is, the better the predictive performance is.
To assess the models' concordance, we first compared the three predicted survival values using paired t-test analysis, ANOVA measure and correlation coefficient. We then calculated the maximal dispersion corresponding to the difference between the highest and the lowest prediction for the same patient. We also calculated differences between the mean of the three predictions and each predictor. Finally, we defined three risk groups: low-, intermediate- and high-risk groups. These groups were constituted using the average risk predicted by the three models, then cut into tercile.
The R 2.15.2 software (R Development Core Team (2012), http://cran.r-project.org/) was used for data analysis, using the Hmisc, rms, and pROC libraries.
Results
We included 965 patients from the METABRIC study. Patients' clinical and pathological characteristics are shown in Table 1. The overall mean age at diagnosis was 63.8 years. Overall, the majority of tumours were invasive ductal carcinoma (77.6%), >T1 stage (54.7%) and Elston Ellis grade 2 (46.5%). Sixty-one per cent of patients received adjuvant hormonal therapy, and only 2.1% had a chemotherapy (4% of patients received both).
Table 1. Patients characteristic.
| Overall population | High-risk group | Intermediate-risk group | Low-risk group | P-value | |
|---|---|---|---|---|---|
| N=965 | N=93 | N=364 | N=508 | ||
| Age (years) mean (s.d.) | 63.8 (12.1) | 73.9 (13.6) | 68.9 (10.5) | 58.3 (9.92) | <0.0001 |
| <40 | 23 (2.4) | 3 (3.2) | 9 (2.5) | 11 (2.2) | 0.6 |
| ⩾40 | 942 (97.6) | 90 (96.8) | 355 (97.5) | 497 (97.8) | |
| Tumour size (mm) mean (s.d.) N (%) | 25.9 (13.5) | 39.4 (20.5) | 30.1 (12.7) | 20.4 (8.91) | <0.0001 |
| ⩽20 | 437 (45.3) | 6 (6.5) | 97 (26.6) | 334 (65.7) | <0.0001 |
| >20 | 528 (54.7) | 87 (93.5) | 267 (73.4) | 174 (34.3) | |
|
Histology N (%) | |||||
| Ductal | 738 (77.6) | 65 (69.9) | 300 (82.4) | 373 (73.4) | 0.002 |
| Lobular | 87 (9.0) | 15 (16.1) | 28 (7.7) | 44 (8.7) | |
| Others | 126 (13.1) | 11 (11.8) | 33 (9.1) | 82 (16.1) | |
| Missing | 14 (1.5) | 2 (2.2) | 3 (0.8) | 9 (1.8) | |
|
Grade N (%) | |||||
| 1 | 93 (9.6) | 4 (4.3) | 19 (5.2) | 70 (13.8) | <0.0001 |
| 2 | 449 (46.5) | 22 (23.7) | 151 (41.5) | 276 (54.3) | |
| 3 | 373 (38.7) | 59 (63.4) | 176 (48.4) | 138 (27.2) | |
| Missing | 50 (5.2) | 8 (8.6) | 18 (4.9) | 24 (4.7) | |
|
Treatment N (%) | |||||
| Chemotherapy | 20 (2.1) | 2 (2.2) | 15 (4.1) | 3 (0.6) | <0.0001 |
| Hormonal therapy | 589 (61) | 69 (74.2) | 281 (77.2) | 239 (47.0) | |
| Both | 39 (4) | 17 (18.3) | 11 (3.0) | 11 (2.2) | |
| None | 317 (32.8) | 5 (5.4) | 57 (15.7) | 255 (50.2) | |
|
Lymph node N (%) | |||||
| Negative | 539 (55.9) | 2 (2.2) | 111 (30.5) | 426 (83.9) | <0.0001 |
| Positive | 426 (44.1) | 91 (97.8) | 253 (69.5) | 82 (16.1) | |
Abbreviations: HRT=hormonal therapy; LN=lymph node.
P-value for the comparison of the three risk groups.
Median follow-up was 169 months.
The percentage survival rates at 1, 5 and 10 years were 98%, 75% and 53%, respectively (Figure 1).
Figure 1.
Survival curve according to Kaplan–Meier.
Performance of the three scores in our population
The discrimination performances of the three web-based calculators to predict 10-year survival, measured by the C-Index, were 0.67 (95% CI 0.63–0.70) for Adjuvant!, 0.74 (95% CI 0.71–0.77) for Cancermath.net and 0.72 (95% CI 0.69–0.75) for PREDICT. The Brier scores were 0.22, 0.23 and 0.21, respectively, for the PREDICT, the Adjuvant! and the CancerMath.net models, showing the good adequacy of each models. Mean 10-year predictions was not different between the three models (0.62, 0.62 and 0.64, respectively, for Adjuvant!, PREDICT and CancerMath.net, P=0.07). However, in the subgroups, these predictions were significantly different (P<0.001, respectively, for the high-, the intermediate- and the low-risk group). The correlation coefficients between models were 0.5 (95% CI 0.50–0.59, P<0.001) for the correlation between the Adjuvant! and the CancerMath.net models, 0.82 (95% CI 0.79–0.83, P<0.001) for the correlation between the CancerMath.net and the PREDICT models and 0.85 (95% CI 0.83–0.87, P>0.001) for the correlation between the PREDICT and the Adjuvant! models.
The average difference between the predicted probabilities and observed survival at 10 years was 9.0% for Adjuvant! (95% CI 6.0–12.0 U-Index 0.05, P<0.001), 10.7% U-Index 0.06, P<0.001) for Cancermath.net (95% CI 0.8–18, U-Index 0.06, P<0.001) and 8.0% for PREDICT (95% CI 5.0–11.0, U-Index 0.05, P<0.001) (Figure 2).
Figure 2.
Models calibrations with confidence intervals.
Median score dispersion was 13% (range 0.2–68%) for the overall population. The median dispersion of the high-risk group (as defined by tercile) was 28% (range 3–62%), whereas the low-risk group had a 9% (0.2–19%) median dispersion, and the intermediate group, a 18% (1.1–68%). Inside the group, there was a median score dispersion of 8% (range 6–10%) for the subgroup with a predicted survival higher than 90%, and 24 (range 1–68%) to 32% (range 6–52%) for the subgroups with a predicted survival between 10 and 50% (Figure 3). Differences between the mean of the three predictions and each predictor are represented in Figure 4. If the three models were zero mean, major dispersion were observed for the intermediate- and the high-risk groups.
Figure 3.
Median dispersion of the three models' prediction and number of events according to the risk groups.
Figure 4.
Repartition of the deviation to the mean for each model, according to the risk groups.
Discussion
Prognosis prediction tools have been developed for personalised breast cancer treatment, allowing more informed decisions about adjuvant therapy. Using an independent data set, this study demonstrated excellent overall performance of each model to predict 10-year survival in patients with ER-positive HER2-negative breast tumours.
However, the reliability of prediction tools has limitations. We demonstrated that three web-based prediction tools were accurate only for patients with excellent prognosis (death events rate of 4–6%). However, the differences between the three tools' predictions reached 30% for some subgroup of intermediate prognosis. The group with poor prognosis showed high discrepancies too, with dispersion of 28%. This highlights the limitations of current prediction models, showing significant discrepancies especially in high-risk populations. This can have potential consequences for the individual patient if their survival risk for breast cancer is underestimated.
Previous validation studies of web-based prediction tools have demonstrated this problem. Version 5.0 of Adjuvant! was validated in a Canadian population of about 4000 patients. It found significant overestimation of survival in patients younger than 35 years and in case of lymphovascular invasion (2), Cancermath.net has also been validated showing greatest accuracy in patients with highest survival rates (12).
Another concern regarding prediction tools is the question whether different geographical population groups can be applied to them. Mook et al (2009) compared actual overall and breast-cancer-specific survival for 5380 European patients with early breast cancer to the American data set of Adjuvant! The authors concluded that Adjuvant! was accurate in the Netherland population and that Adjuvant! could be applied to European populations. There was, however, within the young age (<40 years) subgroup an overestimation of overall survival by 4.2% and BCSS by 4.7%. Similar overestimations of survival were found by Hajage et al (2011) in patients with poor prognostic factors (young age, high-grade and HER2-positive patients). Other studies have confirmed similar findings (Campbell et al, 2009; Paridaens et al, 2010; Bhoo-Pathy et al, 2012; Yao-Lung et al, 2012; Jung et al, 2013).
There is limited evidence in the literature comparing several prediction models with large study populations containing different nationalities with geographical variation. One study did compare PREDICT with Adjuvant! showing similar outcome estimates (14). Our study compared three web-based models with two large patient data sets representing three countries (France, United Kingdom and Canada).
Our results are consistent with previous studies showing that the greatest discrepancy in risk prediction between the three models occurs in high-risk and some intermediate-risk patients.
To our knowledge, our study is the first to compare the results of three validated web-based predictors against large and international data. We have demonstrated that predictions are consistent across different nationalities for patients with good prognosis.
Our study has limitations. First, we decided to compare the Adjuvant! model, the ‘gold standard', with two recent models. These models are not routinely used and have only few external validation studies. However, we showed no major difference in estimates, and the models dispersion did not come from a systematic error from one model (Figure 4). Some other scores exist to predict survival in breast cancer (Schmidt et al, 2009; Campbell et al, 2010; Jankowitz et al, 2011; Barton et al, 2012). Our goal was to highlight difficulty to predict survival whatever the model used. For homogeneity, we decided to evaluate only web-based models, but other models may provide more accurate information. However, in a recent review, new models using clinical and biological factors also found problems with prediction accuracy in high-risk patient groups (Engelhardt et al, 2014). Some data were missing to complete the models, as the chemotherapy generation, required in the PREDICT and the Adjuvant! models. Then we had to approximate and we chose second generation for all the patients, considering that the data were older than 10 years. If it could be a bias for the real prediction of the outcome, we believe that using the same systematic approximation does not modify the comparison of the three models. Moreover, it represents only 6% of the patients.
Given such limitations, we must remember that web-based models are just one tool, and should not be the only factor used for decision-making for patients. Discrepancies in prediction in high-risk groups must be recognised when considering therapeutic decisions for adjuvant treatments.
However, regarding to the cost and to adverse events of adjuvant therapies, reliable tools must be developed. Genomic signatures are one option to explore, although they often suffer from the same limitations. A good risk-prediction model could incorporate genetic profiles and clinical characteristics. We think that specific predictors are needed in high-risk population, such as young patients, high-grade, HER2-positive or triple-negative tumours.
In conclusion, this study revealed that the three web-based predictors equally perform well at the population level, but exhibit a high degree of discordance in some intermediate-risk group and in the high-risk groups. Consequently, an accurate prediction model in the ER-positive HER2-negative breast cancer subgroup is only available for ‘good prognosis' patients.
Footnotes
This work is published under the standard license to publish agreement. After 12 months the work will become freely available and the license terms will switch to a Creative Commons Attribution-NonCommercial-Share Alike 4.0 Unported License.
References
- Barton S, Zabaglo L, A'Hern R, Turner N, Ferguson T, O'Neill S, Hills M, Smith I, Dowsett M. Assessment of the contribution of the IHC4+C score to decision making in clinical practice in early breast cancer. Br J Cancer. 2012;106:1760–1765. doi: 10.1038/bjc.2012.166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bhoo-Pathy N, Yip C-H, Hartman M, Saxena N, Taib NA, Ho G-F, Looi L-M, Bulgiba AM, van der Graaf Y, Verkooijen HM. Adjuvant! Online is overoptimistic in predicting survival of Asian breast cancer patients. Eur J Cancer. 2012;48:982–989. doi: 10.1016/j.ejca.2012.01.034. [DOI] [PubMed] [Google Scholar]
- Campbell HE, Gray AM, Harris AL, Briggs AH, Taylor MA. Estimation and external validation of a new prognostic model for predicting recurrence-free survival for early breast cancer patients in the UK. Br J Cancer. 2010;103:776–786. doi: 10.1038/sj.bjc.6605863. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell HE, Taylor MA, Harris AL, Gray AM. An investigation into the performance of the Adjuvant! Online prognostic programme in early breast cancer for a cohort of patients in the United Kingdom. Br J Cancer. 2009;101:1074–1084. doi: 10.1038/sj.bjc.6605283. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen LL, Nolan ME, Silverstein MJ, Mihm MC, Jr, Sober AJ, Tanabe KK, Smith BL, Younger J, Michaelson JS. The impact of primary tumor size, lymph node status, and other prognostic factors on the risk of cancer death. Cancer. 2009;115:5071–5083. doi: 10.1002/cncr.24565. [DOI] [PubMed] [Google Scholar]
- Curtis C, Shah SP, Chin S-F, Turashvili G, Rueda OM, Dunning MJ, Speed D, Lynch AG, Samarajiwa S, Yuan Y, Gräf S, Ha G, Haffari G, Bashashati A, Russell R, McKinney S, METABRIC Group. Langerød A, Green A, Provenzano E, Wishart G, Pinder S, Watson P, Markowetz F, Murphy L, Ellis I, Purushotham A, Børresen-Dale A-L, Brenton JD, Tavaré S, Caldas C, Aparicio S. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature. 2012;486:346–352. doi: 10.1038/nature10983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Engelhardt EG, Garvelink MM, de Haes JHCJM, van der Hoeven JJM, Smets EMA, Pieterse AH, Stiggelbout AM. Predicting and communicating the risk of recurrence and death in women with early-stage breast cancer: a systematic review of risk prediction models. J Clin Oncol. 2014;32:238–250. doi: 10.1200/JCO.2013.50.3417. [DOI] [PubMed] [Google Scholar]
- Hajage D, de Rycke Y, Bollet M, Savignoni A, Caly M, Pierga J-Y, Horlings HM, Van de Vijver MJ, Vincent-Salomon A, Sigal-Zafrani B, Senechal C, Asselain B, Sastre X, Reyal F. External validation of Adjuvant! Online breast cancer prognosis tool. Prioritising recommendations for improvement. PLoS One. 2011;6:e27446. doi: 10.1371/journal.pone.0027446. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jankowitz RC, Cooper K, Erlander MG, Ma X-J, Kesty NC, Li H, Chivukula M, Brufsky A. Prognostic utility of the breast cancer index and comparison to Adjuvant! Online in a clinical case series of early breast cancer. Breast Cancer Res. 2011;13:R98. doi: 10.1186/bcr3038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jung M, Choi EH, Nam CM, Rha SY, Jeung HC, Lee SH, Yang WI, Roh JK, Chung HC. Application of the adjuvant! Online model to Korean breast cancer patients: an assessment of prognostic accuracy and development of an alternative prognostic tool. Ann Surg Oncol. 2013;20:2615–2624. doi: 10.1245/s10434-013-2956-z. [DOI] [PubMed] [Google Scholar]
- METABRIC Data for Use in Independent Research - syn1688369 2014 ) http://www.synapse.org/#!Synapse:syn1688369 (accessed 16 August 2014).
- Michaelson JS, Chen LL, Bush D, Fong A, Smith B, Younger J. Improved web-based calculators for predicting breast carcinoma outcomes. Breast Cancer Res Treat. 2011;128:827–835. doi: 10.1007/s10549-011-1366-9. [DOI] [PubMed] [Google Scholar]
- Mook S, Schmidt MK, Rutgers EJ, van de Velde AO, Visser O, Rutgers SM, Armstrong N, van't Veer LJ, Ravdin PM. Calibration and discriminatory accuracy of prognosis calculation for breast cancer with the online Adjuvant! program: a hospital-based retrospective cohort study. Lancet Oncol. 2009;10:1070–1076. doi: 10.1016/S1470-2045(09)70254-2. [DOI] [PubMed] [Google Scholar]
- Oakman C, Santarpia L, Di Leo A. Breast cancer assessment tools and optimizing adjuvant therapy. Nat Rev Clin Oncol. 2010;7:725–732. doi: 10.1038/nrclinonc.2010.170. [DOI] [PubMed] [Google Scholar]
- Olivotto IA, Bajdik CD, Ravdin PM, Speers CH, Coldman AJ, Norris BD, Davis GJ, Chia SK, Gelmon KA. Population-based validation of the prognostic model ADJUVANT! for early breast cancer. J Clin Oncol. 2005;23:2716–2725. doi: 10.1200/JCO.2005.06.178. [DOI] [PubMed] [Google Scholar]
- Paridaens RJ, Gelber S, Cole BF, Gelber RD, Thürlimann B, Price KN, Holmberg SB, Crivellari D, Coates AS, Goldhirsch A. Adjuvant! Online estimation of chemotherapy effectiveness when added to ovarian function suppression plus tamoxifen for premenopausal women with estrogen-receptor-positive breast cancer. Breast Cancer Res Treat. 2010;123:303–310. doi: 10.1007/s10549-010-0794-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23:2109–2123. doi: 10.1002/sim.1802. [DOI] [PubMed] [Google Scholar]
- Ravdin PM. A computer program to assist in making breast cancer adjuvant therapy decisions. Semin Oncol. 1996;23:43–50. [PubMed] [Google Scholar]
- R Development Core Team (2012R: A Language and Environment for Statistical Computing R Foundation for Statistical Computing: Vienna, Austria; Available at http://www.R-project.org . [Google Scholar]
- Schmidt M, Victor A, Bratzel D, Boehm D, Cotarelo C, Lebrecht A, Siggelkow W, Hengstler JG, Elsässer A, Gehrmann M, Lehr H-A, Koelbl H, von Minckwitz G, Harbeck N, Thomssen C. Long-term outcome prediction by clinicopathological risk classification algorithms in node-negative breast cancer—comparison between Adjuvant!, St Gallen, and a novel risk algorithm used in the prospective randomized Node-Negative-Breast Cancer-3 (NNBC-3) trial. Ann Oncol. 2009;20:258–264. doi: 10.1093/annonc/mdn590. [DOI] [PubMed] [Google Scholar]
- Wishart GC, Azzato EM, Greenberg DC, Rashbass J, Kearins O, Lawrence G, Caldas C, Pharoah PDP. PREDICT: a new UK prognostic model that predicts survival following surgery for invasive breast cancer. Breast Cancer Res. 2010;12:R1. doi: 10.1186/bcr2464. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yao-Lung K, Dar-Ren C, Tsai-Wang C. Accuracy validation of adjuvant! online in Taiwanese breast cancer patients—a 10-year analysis. BMC Med Inform Decis Mak. 2012;12:108. doi: 10.1186/1472-6947-12-108. [DOI] [PMC free article] [PubMed] [Google Scholar]