ABSTRACT
Background
A previously published failure‐type specific risk model showed good performance in the original cohort. Aim: to validate the model and separate patients with high‐ and low‐risk loco‐regional failure (LRF). Goal: to identify patients potentially suitable for treatment intensification trials.
Methods
Validation data: 756 patients from two institutions (different countries). Predictive performance was evaluated by Brier scores and AUCs. Discriminatory performance was compared to Union for International Cancer Control (UICC) staging (versions 7 and 8).
Results
The model's 3‐year AUC for LRF was 65%, significantly better than UICC7 staging, but no significant difference to UICC8. Model‐based risk stratification and UICC8 both identified high‐risk patient groups with 3‐year LRF ≈30%. The population mean was 18%.
Conclusions
The model performed well on a group level. UICC8 staging performed equally well. Although developed for the endpoint of OS, an improvement from UICC version 7 to version 8 was evident also for the prediction of LRF.
Keywords: competing risks, head and neck cancer, loco‐regional failure, UICC, validation
1. Introduction
The treatment burden of (chemo‐)radiotherapy with or without preceding surgery for head and neck squamous cell carcinoma (HNSCC) is high. Despite this, the overall loco‐regional failure (LRF) rate is around 20% [1]. Further treatment intensification to improve the outcome may increase the treatment burden even further, and patients participating in such trials would therefore optimally be high‐risk patients only [2]. At the other end of the risk spectrum, treatment de‐escalation studies for good prognosis subgroups exist side‐by‐side with treatment intensification efforts [3]. A remaining challenge is how to best identify the right patients for the right trial, which is the focus of the current study.
In 2017, the Union for International Cancer Control (UICC) staging was updated from version 7 [4] to version 8 [5], mainly to address the difference in prognosis between human papilloma virus (HPV) positive oropharynx squamous cell carcinoma (OPSCC) and other HNSCCs. The staging system was developed with overall survival (OS) as an endpoint. However, when considering a specific experimental intervention—escalation or de‐escalation of local or systemic therapy—in a patient, there is also a need for estimates of failure‐type specific probabilities such as risk of LRF and distant metastasis (DM). For this purpose, our group has previously developed a failure‐type specific model for selecting HNSCC patients for experimental treatments [6]. It uses known risk factors such as smoking, age, tumor subsite, stage, and HPV status (full list of endpoints and predictors see Materials and methods) to estimate individual risk profiles. The model was shown to perform well—significantly better than UICC staging (version 8)—in predicting the outcome of the patients in the original cohort.
However—for any risk model to be useful, a thorough validation process is necessary. For the model to add value, the performance of the risk model should also be compared to already existing models and/or other easily accessible clinical variables [7, 8]. Therefore, the aim of the current study was twofold; to validate the previously published failure‐type specific risk model in an independent set of patients, and to again compare its predictive performance to that of UICC staging. Additionally, trial inclusion was simulated for different strategies of risk stratification.
2. Materials and Methods
2.1. The Failure‐Type Specific Risk Model
The failure‐type specific risk model [6] was originally developed in a patient cohort of 560 HNSCC patients treated with intensity‐modulated radiotherapy (IMRT) at Rigshospitalet, Copenhagen between 2005 and 2012 (original cohort). The model was a combination of three cause specific cox models:
Hazard of LRF. Prediction variables: smoking status, T stage, N stage, tumor subsite (p16+ oropharynx as separate subsite), and gross tumor volume (GTV). Stratification variable: candidate for cisplatin (yes/no).
Hazard of DM. Prediction variables: smoking status, T stage, N stage, p16+ oropharynx (yes/no), and GTV. Stratification variable: candidate for cisplatin (yes/no).
Hazard of death with no evidence of disease (NED). Prediction variables: smoking status, age, p16+ oropharynx (yes/no), and WHO performance status. Stratification variable: candidate for cisplatin (yes/no).
Individual risks of LRF, DM, and death NED as the first event were estimated by combining the three cause‐specific models, that is, by using competing risk analysis [9]. The predictive performance of the model in the original cohort was 69%–73%. The model is available as an online app (https://katrin.shinyapps.io/HNSCCmodel/) and further details are given in the original publication [6].
2.2. Validation Data
A cohort of 886 HNSCC patients was used for the validation. It was a combination of temporal validation data (648 patients treated at Rigshospitalet, Copenhagen, Denmark, 2013–2017) and external validation data (238 patients treated at Sahlgrenska University Hospital, Gothenburg, Sweden, 2017–2019), following the definitions of Altman et al. [10]. All patients were treated with curatively intended intensity‐modulated radiotherapy (IMRT/VMAT). Patients treated in Copenhagen were clinically evaluated 8 weeks after the end of radiotherapy, every 6 months in years 1–2 and annually in years 3–5. Patients treated in Gothenburg were clinically evaluated at 8 weeks and radiologically evaluated at 12 weeks after radiotherapy (CT or MR as standard, PET/CT for selected cases). Thereafter they were clinically evaluated every 3 months in years 1–2 and every 6 months in years 3–5. Information on first failure (LRF, DM, or death NED) and prediction variables from medical records and treatment planning systems were specified for each patient, using the same definitions as specified in the original article. Simultaneous LRF and DM were coded as DM. For p16+ oropharynx patients where T‐ and N stages were given according to UICC version 7, retrospective re‐staging to UICC version 8 was performed. Re‐staging from UICC version 8 to version 7 was not performed for p16+ oropharynx patients. For the other tumor subsites, including p16‐ oropharynx, UICC 7 was assumed to be the same as UICC 8. The GTV was contoured separately for T‐ and N‐site lesions in Gothenburg, thus the sum was used for the current analysis. To provide risk estimates, the model required “yes” or “no” for the stratification variable “candidates for cisplatin.” There were patients in Gothenburg receiving cetuximab instead of cisplatin; these were retrospectively coded as candidates for cisplatin. The cut‐off date for the last follow‐up (LFU) was set at August 22, 2023, unless there was a prior known loss of follow‐up.
2.3. Validation Methods and Predictive Performance
The model was applied to the validation cohort and individual risks of LRF, DM, and death NED were calculated. The expected 3‐year risks according to the model were compared to the observed risks (Aalen–Johansen method) by calibration plots and Brier score [11]. A time‐dependent concordance index (AUC) [12] was used to evaluate the model's discriminatory ability, together with risk group stratified cumulative incidence plots. In a sensitivity analysis, the model was separately applied to the Copenhagen and Gothenburg data, to test whether the combination of the two cohorts for validation was reasonable (Supporting Information). The predictive performance of the failure‐type specific model was compared to the null model (model without prediction variables) and to that of UICC staging (version 7 and 8) by a paired Wald test. Statistical analysis was performed in R (https://www.R‐project.org/).
2.4. Strategies for Risk Stratification
Three different strategies for identifying high‐risk patients were compared: (a) Patients in the 4th quartile of expected 3‐year LRF risk according to the failure‐type specific model, (b) Patients with UICC stage IV, version 8, and (c) Patients with UICC stage IV, version 7. The observed 3‐year LRF risks were reported for the high‐risk patient groups identified by the different strategies, along with the proportion of patients fulfilling the criteria.
3. Results
The patient characteristics of the original and validation cohorts are displayed in Table 1. The main differences were that patients of the original cohort had higher N stage, higher (worse) performance status, and larger GTVs than the validation cohorts. The Gothenburg cohort had a higher fraction of p16+ oropharynx patients, and more nonsmokers compared to the Copenhagen cohort. The Gothenburg cohort also had a higher fraction of patients receiving chemotherapy or cetuximab (Table S1). Fifty nine patients were excluded from the analysis because of nonexisting GTV and 71 patients had missing data on one or more of the other prediction variables (Table 1). The validation was thus performed on 756 patients: 533 from Copenhagen and 223 from Gothenburg. The median potential follow‐up time [13] was 8.8 years for the Copenhagen cohort and 4.8 years for the Gothenburg cohort. All analysis in the current section has been performed on the full validation cohort, while separate analyses for the Copenhagen and Gothenburg cohorts are available in the (Figures S1, S2, S3, and Tables S2 and S3).
TABLE 1.
Patient and tumor characteristics for the original and validation cohorts. p‐values are given for comparison between the groups, calculated by T‐test or chi‐squared test, as relevant.
| Variable | Level | Original (n = 560) | Validation (n = 886) | p |
|---|---|---|---|---|
| Age | Mean (SD) | 60.4 (9.3) | 63 (9.0) | < 1e‐04 |
| Tumor Subsite | Oropharynx, p16 positive | 179 (32.0%) | 328 (38.1%) | |
| Cavum oris | 51 (9.1%) | 118 (13.7%) | ||
| Oropharynx, p16 negative | 104 (18.6%) | 123 (14.3%) | ||
| Hypopharynx | 93 (16.6%) | 86 (10.0%) | ||
| Larynx | 133 (23.8%) | 206 (23.9%) | < 1e‐04 | |
| Missing | 0 | 25 | ||
| Tobacco | Never/previous smoker | 260 (46.4%) | 527 (60.9%) | |
| Current smoker | 300 (53.6%) | 338 (39.1%) | < 1e‐04 | |
| Missing | 0 | 21 | ||
| T stage | T1 | 70 (12.5%) | 171 (19.3%) | |
| T2 | 221 (39.5%) | 376 (42.5%) | ||
| T3 | 180 (32.1%) | 181 (20.5%) | ||
| T4 | 89 (15.9%) | 156 (17.6%) | < 1e‐04 | |
| Missing | 0 | 2 | ||
| N stage | N0 | 78 (13.9%) | 305 (34.5%) | |
| N2, N2c‐N3 | 248 (44.3%) | 189 (21.4%) | ||
| N1‐N2b | 234 (41.8%) | 391 (44.2%) | < 1e‐04 | |
| Missing | 0 | 1 | ||
| PS | 0 | 351 (62.7%) | 578 (67.7%) | 0.14 |
| 1 | 162 (28.9%) | 209 (24.5%) | ||
| 2–3 | 47 (8.4%) | 67 (7.8%) | ||
| Missing | 0 | 32 | ||
| Cisplatin | No cisplatin | 266 (47.5%) | 434 (49.8%) | |
| Cisplatin | 294 (52.5%) | 408 (46.8%) | ||
| Cetuximab | 0 (0.0%) | 30 (3.4%) | < 1e‐04 | |
| Missing | 0 | 14 | ||
| GTV | Mean (SD) | 59.5 (76) | 34.5 (42.2) | < 1e‐04 |
| Missing | 0 | 59 |
The cumulative incidence of events over the first 5 years after radiotherapy can be seen in Figure 1. The rate of LRF was significantly lower in the validation cohort (Gray's test, p = 0.002), while there was no significant difference in DM (p = 0.06) nor death NED (p = 0.70). The 3‐year observed failure rates in the validation cohort were 18.7% (95% CI: 15.9%–21.5%) for LRF, 7.7% (5.8%–9.6%) for DM, and 12.1% (9.7%–14.4%) for Death NED.
FIGURE 1.
Stacked Aalen–Johansen plots of cumulative incidence of loco‐regional failure, distant metastasis (DM), and death with no evidence of disease (NED). [Color figure can be viewed at wileyonlinelibrary.com]
3.1. Predictive Performance
The validation cohort was split into four quartiles of 3‐year risk of LRF, DM, and death NED risk, respectively, and the mean observed versus predicted risks were displayed in calibration plots (Figure 2). The LRF risk quartile intervals were: 1st quartile: ≤ 7%, 2nd quartile: 7%–13%, 3rd quartile: 13%–25%, and 4th quartile: > 25% (risk intervals for the other failure‐types see Figure 2 legend). For the prediction of LRF and death NED, the plots showed increasing observed risk with increasing predicted risk. The calibration plot for DM showed a smaller range of risk estimates and less separation between the risk quartile groups. The Brier score was not significantly different from the null model (assigning the group mean) for LRF nor DM (Table 2). The 3‐year AUC was 0.65 for LRF, 0.66 for DM, and 0.72 for death NED, significantly better than the reference “coin‐toss” model of 50% risk for all failure‐types, significantly better than UICC 7 staging for LRF and Death NED, but not significantly better than UICC 8 staging (Table 2).
FIGURE 2.
Calibration plot for observed vs. predicted 3‐year risk of (a) loco‐regional failure, (b) distant metastasis, and (c) death with no evidence of disease (NED). The mean risks are displayed for the quartiles of predicted risk: For loco‐regional failure: Q1: ≤ 7%, q2: 7%–13%, q3: 13%–25%, and q4: > 25%. For distant metastasis: Q1: ≤ 3%, q2: 3%–6%, q3: 6%–10% and q4: > 10%. For death NED: Q1: ≤ 4%, q2: 4%–11%, q3: 11%–15% and q4: > 15%.
TABLE 2.
3‐year predictive performance of the failure‐type specific model in the validation cohort. Brier scores were compared to the null model, that is, when all patients were assigned the group mean risk estimate. AUCs for the failure‐type specific model, UICC 8 and UICC 7 were all compared to a reference “coin‐toss” model of 50% risk as well as to each other. Note that lower values are better for the Brier score and higher are better for the AUC.
| Brier score | AUC | ||||
|---|---|---|---|---|---|
| Reference (null) model | Failure‐type specific model | UICC7 only model | UICC8 only model | Failure‐type specific model | |
| LRF | 0.15 (0.13–0.17) | 0.15 (0.13–0.17) | 53% (48–58) | 63% (59–68) b , c | 65% (61–70) b , c |
| DM | 0.07 (0.05–0.09) | 0.07 (0.05–0.09) | 62% (57–67) b | 63% (58–68) b | 66% (61–71) b |
| Death NED | 0.11 (0.09–0.12) | 0.10 (0.08–0.12) a | 57% (52–61) b | 68% (63–72) b , c | 72% (66–77) b , c |
Marker for significance: Brier score is significantly different from the null model.
Marker for significance: AUC is significantly different from 50%.
Marker for significance: AUC is significantly different from UICC 7. There were no significant differences between the failure‐type‐specific model and UICC 8.
3.2. Simulation of Trial Inclusion
Three strategies for identifying patients with high risk of LRF with standard therapy were tested. Cumulative incidence plots for LRF showed better separation of risk groups for (a) model‐based risk stratification and (b) UICC 8 than for (c) UICC 7. UICC 7 showed a lower separation as well as a skewed distribution of patients toward the high‐risk groups (Figure 3). If UICC 8 (stage IV) was used as an inclusion criterion for a hypothesized trial of loco‐regional treatment intensification, 31% of the patients would have been offered inclusion. The corresponding numbers for the failure‐type specific model (4th quartile) and UICC 7 (stage IV) would have been 25% and 63%, respectively. Model‐based risk stratification and UICC 8 would offer inclusion to patients with similar observed rates of LRF (Table 3). Three HPV‐positive oropharynx patients would have been defined as high‐risk patients according to the model‐based risk stratification, whereas UICC 8 stage IV by definition could not include any HPV‐positive OPSCC patients (only DM at diagnosis would give stage IV).
FIGURE 3.
Observed loco‐regional failure (LRF) rates for patients stratified by (a) multi‐endpoint risk estimates, (b) UICC 8 stage, and (c) UICC 7 stage. [Color figure can be viewed at wileyonlinelibrary.com]
TABLE 3.
Observed 3‐year loco‐regional failure (LRF) rates for different strategies for identifying high‐risk patients. Note that UICC 8 stage IV by definition cannot include p16+ OPSCC patients.
| Risk group definition strategy | Group | Fraction of patients | Observed LRF (95% CI) |
|---|---|---|---|
| Failure‐type specific model (LRF risk estimate) | High risk: 4th quartile (> 25% risk) | 25% | 30% (23%–36%) |
| Low risk: 1st‐3rd quartile (0%–25% risk) | 75% | 15% (12%–18%) | |
| Stage, UICC 8 | Stage IV | 31% | 28% (22%–34%) |
| Stage I‐III | 69% | 15% (12%–18%) | |
| Stage, UICC 7 | Stage IV | 60% | 21% (17%–26%) |
| Stage I‐III | 40% | 18% (14%–23%) |
4. Discussion
A prognostic model for failure‐type specific outcomes would be highly valuable. It could guide the clinicians in which patients to enroll in what trial, as the range of trial intensity for HNSCC goes all the way from de‐escalation to treatment intensification. The increasing proportion of patients with HPV‐positive OPSCC [14, 15] have a better prognosis and fewer co‐morbidities than other HNSCC patients [16] and are therefore subjects of treatment de‐escalation studies. There are, however, still HPV‐positive OPSCC patients with other unfortunate prognostic factors, such as heavy smokers, etc. These may not be suitable for dose de‐escalation. Optimally, a failure‐type specific prognostic model would take all prognostic factors into account and provide a more individual risk profile. Exactly this is what was attempted in the model evaluated in the current study. The model was developed following the TRIPOD guidelines [8] and with an experienced team of oncologists, physicists, and a statistician specialized in prognostic modeling. A predefined set of known prognostic factors was chosen to avoid overfitting, and a high bar was set for the inclusion of additional factors [6].
Despite the efforts to avoid overfitting and build a robust failure‐type specific model, the predictive performance should be expected to drop when the model is tested in a dataset independent of the one it was developed in [17]. As such, the AUCs reported here (65%–70%, compared to 69%–73% in the original cohort) are not discouraging. A different case mix in the validation set (Table 1) should not affect the performance of a clinically robust model [10]. However, the validation results and statistical measures of predictive performance should additionally be seen in the light of a specific clinical context to enable evaluation of the model's clinical validity [10]—in this case, to identify a group of patients that would potentially be suitable for treatment intensification trials. While risk prediction on an individual level may require a higher performance for the model calibration, group‐level risk stratification sets the focus on discriminatory performance. The ability of the failure‐type specific model to separate patients with low‐ and high risk of LRF on a group level was good, as is evident from Figures 2 and 3 and Table 3. The failure‐type specific model performed better than UICC version 7 (Figure 3 and Table 3), but the updated UICC version 8 provided similar results as our model in terms of AUC and ability to stratify patients based on predicted risk of LRF (Table 2 and Figure 3). Our data thus highlights the improvement in UICC staging from version 7 to 8 to extend beyond the prediction of OS [18] to an improvement also in the prediction of LRF. This improvement thus makes it harder to outperform UICC staging, and we hypothesize that the development of many models to predict LRF in HNSCC has previously been based on the limitations of UICC version 7—limitations that are evident in the current analysis.
A strength of the current study is the comparison to UICC staging. In a review of multivariable prediction models, Collins et al. [7] found that for 79% of externally validated models, no comparison to existing models/clinical variables was performed. The comparison in the current study importantly showed that our proposed risk model and UICC 8 stage IV could be used as inclusion criteria in a hypothesized trial of treatment intensification with similar accuracy (Table 3). The patients included would with both strategies have an expected LRF rate of approximately 30%, which is a good trial enrichment compared to the population average of 18%. Risk stratification based on UICC 8 is simple as it provides readily available staging. The failure‐type specific model, on the other hand, provides more degrees of freedom, also enabling HPV‐positive OPSCC patients to be defined as high‐risk patients if combined with negative risk factors. In future studies, it may be worthwhile to separately study OPSCC patients with negative risk factors to gain more knowledge on their potential gain from treatment intensification. Using the 4th quartile to identify high‐risk patients set a limit for predicted LRF risk of > 25%. This seems like a reasonable number but may be further discussed and optimized, as the supplementary analysis of Gothenburg and Copenhagen data separately shows that an absolute risk limit would give different results in the two cohorts (Table S3). A data‐driven optimization of the cut‐off value could potentially have resulted in a better performance but was decided against to minimize the risk of overfitting.
Although the failure‐type specific model performed well, it did fail to outperform staging according to UICC 8. A challenge of using competing risks in modeling is that not only the LRF risk model must fit, but also the models for the competing events (DM and Death NED). This may have influenced the results, but likely to a limited extent as the other events showed equal or higher predictive performance as LRF (Table 2). Another potential problem is overfitting because of the data‐driven inclusion of predictors. This also seems unlikely to have had a large impact, since the prediction variables were well‐known clinical factors, except for the addition of tumor volume (GTV). Overfitting was contradicted in a test where a model without GTV (referred to as the “basis model” in the original article) was applied to the validation set, with a similar prediction performance as the full multi‐endpoint model (data not shown). It seems more likely that the suggested model lacks one or more predictors important for patient outcome. The search for such predictors is ongoing. Hopefully, the joint efforts of the head and neck cancer research community will in the future result in new and robust predictors to enable more individualized risk stratification.
In conclusion, the failure‐type specific risk model performed well in the validation cohort in terms of LRF risk stratification on a group level. Staging according to UICC version 8 performed equally well, however, HPV‐positive OPSCC patients cannot be defined as high‐risk in UICC 8 (stage IV). Both strategies seem equally well suited for identifying patients suitable for inclusion in treatment intensification trials. Although developed for the endpoint of OS, the improvement from UICC version 7 to version 8 was also evident for the failure‐type‐specific prediction of LRF.
Ethics Statement
Access to patient and treatment data has been approved by The Danish Patient Safety Authority (ref.no 3‐3013‐3142/1).
Supporting information
Data S1.
Acknowledgments
This study was funded by Varian Medical Systems.
Funding: This study was supported by Varian Medical Systems.
This study was presented at ESTRO 2024 in Glasgow, Scotland.
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
References
- 1. Grégoire V., Langendijk J. A., and Nuyts S., “Advances in Radiotherapy for Head and Neck Cancer,” Journal of Clinical Oncology 33, no. 29 (2015): 3277–3284, 10.1200/JCO.2015.61.2994. [DOI] [PubMed] [Google Scholar]
- 2. Al‐Mamgani A., Kessels R., Gouw Z. A. R., et al., “Adaptive FDG‐PET/CT Guided Dose Escalation in Head and Neck Squamous Cell Carcinoma: Late Toxicity and Oncologic Outcomes (The ADMIRE Study),” Clinical and Translational Radiation Oncology 43 (2023): 100676, 10.1016/j.ctro.2023.100676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Andreassen C. N., Eriksen J. G., Jensen K., et al., “IMRT‐Biomarkers for Dose Escalation, Dose de‐Escalation and Personalized Medicine in Radiotherapy for Head and Neck Cancer,” Oral Oncology 86 (2018): 91–99, 10.1016/j.oraloncology.2018.09.001. [DOI] [PubMed] [Google Scholar]
- 4. Edge S. B., Byrd D. R., Compton C. C., Fritz A. G., Greene F. L., and Trotti A., AJCC Cancer Staging Manual, vol. 7AMA (New York: Springer, 2010). [Google Scholar]
- 5. Amin M. B., Edge S. B., Greene F. L., et al., AJCC Cancer Staging Manual, vol. 1024 (Springer, 2017). [Google Scholar]
- 6. Håkansson K., Rasmussen J. H., Rasmussen G. B., et al., “A Failure‐Type Specific Risk Prediction Tool for Selection of Head‐And‐Neck Cancer Patients for Experimental Treatments,” Oral Oncology 74 (2017): 77–82, 10.1016/j.oraloncology.2017.09.018. [DOI] [PubMed] [Google Scholar]
- 7. Collins G. S., De Groot J. A., Dutton S., et al., “External Validation of Multivariable Prediction Models: A Systematic Review of Methodological Conduct and Reporting,” BMC Medical Research Methodology 14, no. 1 (2014): 1–11, 10.1186/1471-2288-14-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Moons K. G. M., Altman D. G., Reitsma J. B., et al., “Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and Elaboration,” Annals of Internal Medicine 162, no. 1 (2015): W1–W73, 10.7326/M14-0698. [DOI] [PubMed] [Google Scholar]
- 9. Benichou J. and Gail M. H., “Estimates of Absolute Cause‐Specific Risk in Cohort Studies,” Biometrics 46, no. 3 (1990): 813–826. [PubMed] [Google Scholar]
- 10. Altman D. G. and Royston P., “What Do We Mean by Validating a Prognostic Model?,” Statistics in Medicine 19, no. 4 (2000): 453–473, 10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5. [DOI] [PubMed] [Google Scholar]
- 11. Gerds T. A., Scheike Thomas H., and Andersen P. A., “Absolute Risk Regression for Competing Risks: Interpratation, Link Functions and Prediction,” Statistics in Medicine 31, no. 29 (2012): 3921–3930, 10.1002/aur.1474.Replication. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Blanche P., Dartigues J. F., and Jacqmin‐Gadda H., “Estimating and Comparing Time‐Dependent Areas Under Receiver Operating Characteristic Curves for Censored Event Times With Competing Risks,” Statistics in Medicine 32, no. 30 (2013): 5381–5397, 10.1002/sim.5958. [DOI] [PubMed] [Google Scholar]
- 13. Schemper M. and Smith T. L., “A Note on Quantifying Follow‐Up in Studies of Failure Time,” Controlled Clinical Trials 17, no. 4 (1996): 343–346, 10.1016/0197-2456(96)00075-X. [DOI] [PubMed] [Google Scholar]
- 14. Chaturvedi A. K., Engels E. A., Pfeiffer R. M., et al., “Human Papillomavirus and Rising Oropharyngeal Cancer Incidence in the United States,” Journal of Clinical Oncology 29, no. 32 (2011): 4294–4301, 10.1200/JCO.2011.36.4596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Zamani M., Grønhøj C., Jensen D. H., et al., “The Current Epidemic of HPV‐Associated Oropharyngeal Cancer an 18‐Year Danish Population‐Based Study With 2,169 Patients the Current Epidemic of HPV‐Associated Oropharyngeal Cancer: An 18‐Year Danish Population‐Based Study With 2,169 Patients,” European Journal of Cancer 134 (2022): 52–59, 10.1016/j.ejca.2020.04.027. [DOI] [PubMed] [Google Scholar]
- 16. Lydiatt W., Patel S., O'Sullivan B., et al., “Head and Neck Cancers‐Major Changes in the American Joint Committe on Cancer Eighth Edition Cancer Staging Manual,” CA: A Cancer Journal for Clinicians 67, no. 2 (2017): 122–137, 10.3322/caac.21389. [DOI] [PubMed] [Google Scholar]
- 17. Harrell F. E., Lee K. L., and Mark D. B., “Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors,” Tutorials in Biostatistics, Statistical Methods in Clinical Studies 1 (2005): 223–249, 10.1002/0470023678.ch2b(i). [DOI] [Google Scholar]
- 18. O'Sullivan B., Huang S. H., Su J., et al., “Development and Validation of a Staging System for HPV‐Related Oropharyngeal Cancer by the International Collaboration on Oropharyngeal Cancer Network for Staging (ICON‐S): A Multicentre Cohort Study,” Lancet Oncology 17, no. 4 (2016): 440–451, 10.1016/S1470-2045(15)00560-4. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data S1.
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.