Abstract
Background:
Metabolic response assessment for oropharyngeal squamous cell carcinoma (OPSCC) aids in identifying locoregional persistence/recurrence (LRR). The Hopkins Criteria are a standardized qualitative response assessment system using posttreatment FDG-PET/CT.
Methods:
We conducted a retrospective cohort study of patients with node-positive OPSCC treated with definitive (chemo)radiotherapy. We assessed Hopkins Criteria performance for LRR, then developed and validated a competing-risks model.
Results:
Between 2004 and 2018, 259 patients were included with median follow-up of 43 months. The Hopkins Criteria sensitivity, specificity, negative predictive value, and accuracy were 68%, 88%, 95%, and 85%. The 36-month cumulative incidence of LRR was greater with positive scores (45% vs. 5%, HR 12.60, p < 0.001). PET/CTs performed ≤10 weeks after radiotherapy were associated with a four-fold increase in pathologically negative biopsies/surgeries (36% vs. 9%, p = 0.03). The AUC for LRR was 0.89 using a model integrating the Hopkins score.
Conclusions:
The Hopkins Criteria predict LRR with high accuracy for OPSCC response assessment.
Keywords: Hopkins criteria, locoregional recurrence, oropharynx cancer, PET/CT, response assessment
1 |. INTRODUCTION
Surveillance after definitive chemoradiotherapy for oropharyngeal squamous cell carcinoma (OPSCC) is critical to confirm clinical complete response, as approximately 10% and 25% of patients with HPV-positive and HPV-negative tumors will develop locoregional persistence/recurrence (LRR), respectively.1 In regions with access to PET/CT, posttreatment PET/CT-guided surveillance is the standard of care, as it offers high negative predictive value for recurrence and spares most patients from neck dissections.2,3 While many studies have demonstrated the diagnostic and prognostic utility of pre-treatment, mid-treatment, and posttreatment PET/CT, there is a lack of consensus reporting standards to promote consistent posttreatment management and surveillance.4–8 Many studies assessing the performance of posttreatment PET/CT have not used standardized reporting criteria, making it difficult to compare results across studies and integrate PET-guided management into clinical practice.9,10
The Hopkins Criteria are a standardized qualitative system for assessing postradiotherapy PET/CT response for head and neck cancers.11–14 These Criteria have high inter-rater reproducibility and have been externally validated for overall survival (OS) and progression-free survival (PFS).11–14 It remains uncertain whether these Criteria are also independently prognostic for LRR or whether they have similar performance across all subsets of OPSCC. Accurate prognostication for LRR could guide standardized locoregional management, such as selecting patients for salvage neck dissection, more frequent examinations/imaging, or minimizing unnecessary procedures among patients who have been cured. We hypothesized that the Hopkins Criteria are prognostic for LRR independent of traditional clinicopathologic prognosticators, and would have robust performance across p16/HPV status, PET/CT timing/quality, and treatment regimens. Thereafter, we developed a predictive model for LRR and assessed whether the Hopkins Criteria were independently prognostic when controlling for standard clinicopathologic risk factors.
2 |. MATERIALS AND METHODS
2.1 |. Study design, patient population, and treatment procedure
We conducted an institutional review board-approved retrospective cohort study of consecutive adult patients with newly diagnosed biopsy-proven AJCC 7th edition cT1–4N1–3M0 oropharyngeal squamous cell carcinoma (OPSCC) treated with definitive (chemo) radiotherapy. Patients were eligible if they completed a pre-treatment PET/CT and a postradiotherapy PET/CT between 5 and 24 weeks after treatment, consistent with the original Hopkins Criteria validation.11 Patients treated between 2004 and 2018 were included in order to allow sufficient follow-up for recurrence. Exclusion criteria were: node-negative at diagnosis or no gross adenopathy at radiotherapy, distant metastasis at diagnosis, prior head and neck radiotherapy, treatment for recurrence, unknown primary site, lack of postradiotherapy PET/CT within 24 weeks of completing radiotherapy, locoregional recurrence prior to first postradiotherapy PET/CT, and completion of <64 Gy EQD2α/β=10 (for the most commonly used regimen of 70 Gy in 33 fractions, patients receiving <30 fractions would be excluded by this criterion).
Tumors were categorized as HPV-positive by the presence of at least 70% nuclear p16 immunohistochemical staining or positive high-risk HPV in situ hybridization. All included patients underwent definitive radiotherapy with or without induction/concurrent chemotherapy as indicated (Supplementary Methods).
2.2 |. PET/CT technique, follow-up, and Hopkins score assessment
All 18F-FDG-PET/CTs were conducted either in the standard diagnostic position or in the treatment position with molded foam cushion (AccuForm, Medtec, Orange City, IA) and thermoplastic mask (Aquaplast, WFR/Aquaplast Corp., Wyckoff, NJ). A variety of PET/CT instruments were in use during the study period (Supplementary Table 1). All included patients completed a PET/CT prior to initial treatment, which was available for review at the time of Hopkins scoring. For patients who received induction chemotherapy, this PET/CT occurred prior to initial chemotherapy to permit uniform assessment of gross disease extent. Patients fasted for at least 6 h prior to imaging, and then 10–18 mCi FDG were injected after confirming blood glucose levels <180 mg/dl. Image acquisition began after a 45–60 min tracer uptake period. All studies were attenuation-corrected by CT acquired in axial or helical mode. For patients who had completed whole-body staging prior to initial PET/CT, image acquisition was limited to the head, neck, and thorax. Otherwise, all PET/CTs spanned the top of the head to the mid-thighs.
Patients were typically assessed for treatment response by physical examination and endoscopy 4–8 weeks after completing radiotherapy. All patients completed a postradiotherapy diagnostic PET/CT within 24 weeks of completing treatment, at which time an examination and endoscopy were also typically performed. The timing of this exam and any additional imaging (CT, MRI) was at the discretion of the treatment team, but was typically 12–16 weeks after radiotherapy. Biopsy or serial imaging/examination were performed for patients with nodal progression or residual/progressive primary disease by examination or imaging, followed by salvage surgery if indicated. Thereafter, patients were followed with a head and neck evaluation every 1–3 months for year one, every 2–6 months for year 2, every 4–8 months for years 3–5, and then annually.3 Subsequent surveillance imaging of the head, neck, or chest was not routinely performed.
A single board-certified nuclear medicine physician with subspecialty head and neck expertise retrospectively scored all posttreatment PET/CTs according to the Hopkins Criteria described by Marcus et al.11 This reader was blinded to each patient’s clinical outcome (recurrence, death, subsequent therapy). We did not score or compare posttreatment imaging using alternative scoring systems (e.g., NI-RADS, Porceddu). Given high inter-reader agreement previously described, studies were not scored by multiple readers.11,12 To replicate real-world use of this system, the data available to the scoring physician were: (1) Brief oncologic history prior to PET/CT acquisition (clinical stage, primary tumor laterality, dates of recent treatment with definitive (chemo)radiotherapy); (2) The pre-treatment PET/CT images and diagnostic report; (3) The posttreatment PET/CT images. All unfused image series were available for review in MIM (MIM Software Inc., Cleveland, Ohio). The left neck, right neck, and primary tumor were scored on the five-point qualitative scale defined by Marcus et al., with the internal jugular vein (IJV) 18F-FDG uptake taken as reference for the background blood pool (Supplementary Methods). The overall score was the highest score among the primary tumor, left neck, and right neck. As recommended by Marcus et al., overall scores 1–3 were considered negative for persistent/recurrent tumor. We also investigated a ternary system of negative (1–2), equivocal (3), and positive (4–5), analogous to other qualitative systems.14
2.3 |. ecurrence definitions and study outcomes
The primary outcome was the time to locoregional recurrence/persistence (LRR); death was considered a competing risk. To avoid introducing a temporal bias secondary to differing fractionation schemes or patient adherence/tolerance to radiotherapy, recurrences and survival were calculated from the date of initial radiotherapy. Date of last oncologic follow-up was defined as the last clinic visit which included transoral physical examination (and generally also nasopharyngolaryngoscopy). Noninformative censoring was performed at loss to oncologic follow-up. Local and regional recurrence/persistence were, respectively, defined as pathologically proven or clinically overt tumor at the primary site or regional nodes (levels I-V, retropharyngeal/retrostyloid). Patients with locoregional recurrence/persistence who did not undergo repeat biopsy had gross disease evident on both PET/CT and examination/endoscopy, had persistent disease at the time of last follow-up/death, and enrolled in hospice or proceeded immediately to palliative chemotherapy. There was a multidisciplinary consensus that this gross disease represented locoregional recurrence/persistence. The presence of either a local or regional recurrence was defined as LRR. Any patient with residual pathologically proven tumor by biopsy or salvage surgery was considered to have LRR; however, patients with LRR before posttreatment PET/CT were excluded as described above (three patients). All other recurrences confirmed by biopsy or clinically overt diagnostic imaging (e.g., biopsy declined due to performance status) were considered distant metastases. Secondary outcomes included progression-free survival (PFS), overall survival (OS), and the cumulative incidence of distant metastasis (DM).
2.4 |. Statistical analysis
Medians with interquartile ranges [IQR] were reported for continuous variables, whereas proportions were reported for categorical variables. Categorical data were compared with Fisher exact tests. Hopkins Criteria performance were calculated using contingency tables and reported with Wilson score binomial 95% confidence intervals. The Kaplan–Meier method was used to estimate progression-free and overall survival, with differences compared via log-rank tests. The unadjusted cumulative incidences of LRR and DM were estimated based on the subdistribution method with death as a competing risk.15 Differences in cumulative incidences were compared via Gray’s test.15 We compared performance of the Hopkins Criteria and rates of biopsies/surgeries among patients who underwent PET/CTs ≤10 weeks, 10–14 weeks, and >14–24 weeks after completion of radiotherapy. Additional sensitivity analyses were conducted in cohort subsets, including a subset of patients with available pre- and mid-treatment metabolic tumor volume (MTV) measures.8,16 These MTV measures included: SUV maximum [MTVSUVmax], tumor volume at 50% SUV Maximum [MTVSUV50%], tumor volume at SUV of 2.0 [MTVSUV2.0], and MTV total lesion glycolysis [MTVTLG]. We assessed whether pre- or mid-treatment MTV measures could improve upon the prognostic accuracy of posttreatment Hopkins Scores. These MTV measures were collected as previously described and are further detailed in the supplementary methods.8
We used all data from the cohort to develop a risk prediction model for LRR using a competing-risk framework with regularization for covariate selection (Supplementary Methods).17–22 The independent prognostic value of the Hopkins score was assessed in cross-validated multivariable competing risk regressions for LRR (Supplementary Methods). Hazard ratios for the Hopkins score were similar among variable selection approaches. Statistical comparisons were not adjusted for family-wise type I error. Data analyses were performed using R 4.0.4 with the riskRegression, crrstep, and crrp packages.18,19,21 This study was reported in accordance with STARD and TRIPOD guidelines.23,24
3 |. RESULTS
3.1 |. Patient, tumor, and treatment characteristics
Between 2004 and 2018, 259 patients with newly diagnosed OPSCC were eligible for inclusion, while a separate 181 were excluded, most commonly due to lack of posttreatment PET/CT within 24 weeks of radiotherapy (76 patients, 42%), node-negative at diagnosis or lack of gross adenopathy at radiotherapy (46, 25%), and/or distant metastasis at diagnosis (5, 3%). All reasons for exclusion are described in Supplementary Figure 1. The reasons for lack of postradiotherapy PET/CT within 24 weeks were: loss to follow-up prior to PET/CT (46%), other imaging performed (39%), patient illness/death (12%), or PET/CT not ordered (3%). Most patients were men (90%), 47% had never smoked, 86% had p16/HPV-positive tumors, and 39% had AJCC 7th edition T3–4 tumors (Table 1). The AJCC 8th edition group stage distribution was: 110 (I, 43%), 63 (II, 24%), 53 (III, 20%), 29 (IVA, 11%), and 4 (IVB, 2%).
TABLE 1.
Patient, tumor, and treatment characteristics
| Characteristic | All patients | Hopkins score 1–3 (negative) | Hopkins score 4–5 (positive) |
|---|---|---|---|
| No. of patients | 259 | 209 (81) | 50 (19) |
| Age at start of radiotherapy | 61 [55–68] | 61 [55–67] | 63 [58–69] |
| Male | 233 (90) | 187 (89) | 46 (92) |
| Race | |||
| White | 205 (79) | 165 (79) | 40 (80) |
| East and South Asian | 16 (6) | 13 (6) | 3 (6) |
| Black | 7 (3) | 4 (2) | 3 (6) |
| Native Hawaiian/Pacific Islander | 2 (1) | 2 (1) | 0 (0) |
| Other/not reported | 29 (11) | 25 (12) | 4 (8) |
| Smoking status | |||
| Never smoker | 121 (47) | 97 (46) | 24 (48) |
| Former smoker | 124 (48) | 102 (49) | 22 (44) |
| Current smoker | 14 (5) | 10 (5) | 4 (8) |
| Smoking pack-years | |||
| 0 | 121 (47) | 97 (46) | 24 (48) |
| >0–10 | 37 (14) | 33 (16) | 4 (8) |
| >10–20 | 14 (5) | 10 (5) | 4 (8) |
| >20 | 46 (18) | 37 (18) | 9 (18) |
| Unknown | 41 (16) | 32 (15) | 9 (18) |
| ECOG performance status | 1 [0–3] | 0 [0–3] | 1 [0–3] |
| 0–1 | 237 (92) | 192 (92) | 45 (90) |
| 2–3 | 22 (8) | 17 (8) | 5 (10) |
| Tumor location | |||
| Base of tongue | 140 (54) | 106 (51) | 34 (68) |
| Palatine tonsil | 102 (39) | 89 (43) | 13 (26) |
| Soft palate/posterior wall/multiple sites | 17 (7) | 14 (7) | 3 (6) |
| p16/HPV positive | 222 (86) | 181 (87) | 41 (82) |
| AJCC 7 clinical tumor stage | |||
| T1 | 56 (22) | 51 (24) | 5 (10) |
| T2 | 102 (39) | 86 (41) | 16 (32) |
| T3 | 46 (18) | 35 (17) | 11 (22) |
| T4a | 50 (19) | 33 (16) | 17 (34) |
| T4b | 5 (2) | 4 (2) | 1 (2) |
| AJCC 7 clinical nodal stage | |||
| N1 | 24 (9) | 20 (10) | 4 (8) |
| N2a | 18 (7) | 17 (8) | 1 (2) |
| N2b | 140 (54) | 114 (55) | 26 (52) |
| N2c | 63 (24) | 49 (23) | 14 (28) |
| N3 | 14 (6) | 9 (4) | 5 (10) |
| Posterior triangle nodes involved | 58 (22) | 36 (17) | 22 (44) |
| Lowest nodal station | |||
| II/VA upper | 50 (19) | 41 (20) | 9 (18) |
| III/VA lower | 146 (56) | 119 (57) | 27 (54) |
| IV/VB | 63 (24) | 49 (23) | 14 (28) |
| Contoured GTVp + GTVn (cc) | 40.9 [26.0–65.5] | 37.3 [25.2–61.0] | 50.1 [34.5–78.7] |
| Delivered radiotherapy dose (Gy)a | 70.0 [65.7–71.4] | 70.0 [65.7–71.4] | 70.0 [66.0–70.0] |
| Induction chemotherapy | 28 (11) | 21 (10) | 7 (14) |
| Concurrent chemotherapy | 254 (98) | 204 (98) | 50 (100) |
| Cisplatin | 121 (47) | 105 (50) | 16 (32) |
| Cetuximab | 107 (41) | 81 (39) | 26 (52) |
| Platinum/Cetuximab combinationb | 14 (5) | 9 (4) | 5 (10) |
| Other platinum combinationc | 12 (5) | 9 (4) | 3 (6) |
| None | 5 (2) | 5 (2) | 0 (0) |
| Oncologic follow-up (months)d | 43 [27–72] | 47 [29–73] | 31 [15–65] |
Note: Values are reported as median [interquartile range] or number (percent). Except for treatment characteristics, all values are reported at the time of initial diagnosis and work-up.
Abbreviations: AJCC, American Joint Committee on Cancer; ECOG, Eastern Cooperative Oncology Group; GTVp, primary gross tumor volume; GTVn, nodal gross tumor volume; HPV, human papillomavirus.
Patients were treated with a planned dose of 70 Gy in 33 fractions, 70 Gy in 35 fractions, or 66 Gy in 30 fractions. Seven patients received one or two fewer fractions than planned due to toxicity/hospitalization/refusal.
Cisplatin administered for initial cycle followed by cetuximab due to toxicity/intolerance.
Carboplatin-based regimens.
Oncologic follow-up including transoral physical exam, generally with nasopharyngolaryngoscopy.
Nearly all (98%) patients received concurrent chemotherapy (weekly or q3 week cisplatin: 47%; cetuximab: 41%), while 11% received induction chemotherapy. Five patients (2%) completed 32/33 2.12 Gy fractions, while one patient (<1%) completed 31/33 fractions.
3.2 |. Posttreatment procedures, PET/CT timing, and Hopkins score distribution
Median oncologic follow-up for recurrence was 43.0 months. After radiotherapy, 41 patients (16%) underwent at least one biopsy or operative procedure of either the primary site or regional lymph nodes, which was more common among patients with Hopkins scores 4–5 (Supplementary Table 2). The initial posttreatment PET/CT was conducted at a median of 13.2 weeks after completing radiotherapy (Table 2). Among the 259 included patients, 11 (4%), 147 (57%), and 101 (39%) patients completed this PET/CT 6–10 weeks, >10–14 weeks, and >14–24 weeks after radiotherapy. The distribution of Hopkins scores at the primary site, right neck, and left neck are provided in Table 2. Fifty patients (19%) had overall scores of 4–5, while the most common overall score was 3 (120 patients, 46%).
TABLE 2.
Distribution of first follow-up PET/CT timing, expert-assessed quality, and Hopkins scores
| Characteristic | All patients | Locoregional persistence/recurrence | No locoregional persistence/recurrence |
|---|---|---|---|
| No. of patients | 259 | 34 (13) | 225 (87) |
| Duration from radiotherapy to first PET/CT (weeks) | 13.2 [6.4–24.0] | 13.2 [7.6–22.4] | 13.2 [6.4–24.0] |
| 6–10 weeks | 11 (4) | 2 (6) | 9 (4) |
| >10–14 weeks | 147 (57) | 20 (59) | 127 (56) |
| >14–24 weeks | 101 (39) | 12 (35) | 89 (40) |
| Expert-assessed PET/CT quality | |||
| Fair (suboptimal field of view, patient motion, CT quality, etc.) | 115 (44) | 19 (56) | 96 (43) |
| Good | 132 (51) | 14 (41) | 118 (52) |
| Excellent (ideal examination) | 12 (5) | 1 (3) | 11 (5) |
| Hopkins scores | |||
| Overall (highest) | |||
| 1 | 40 (15) | 2 (6) | 38 (17) |
| 2 | 49 (19) | 5 (15) | 44 (20) |
| 3 | 120 (46) | 4 (12) | 116 (51) |
| 4 | 34 (13) | 9 (26) | 25 (11) |
| 5 | 16 (6) | 14 (41) | 2 (1) |
| Primary site | |||
| 1 | 54 (21) | 6 (18) | 48 (21) |
| 2 | 45 (17) | 4 (12) | 41 (18) |
| 3 | 128 (49) | 9 (26) | 119 (53) |
| 4 | 24 (9) | 9 (26) | 15 (7) |
| 5 | 8 (3) | 6 (18) | 2 (1) |
| Left neck | |||
| 1 | 190 (73) | 21 (62) | 169 (75) |
| 2 | 51 (20) | 6 (18) | 45 (20) |
| 3 | 2 (1) | 0 (0) | 2 (1) |
| 4 | 11 (4) | 2 (6) | 9 (4) |
| 5 | 5 (2) | 5 (15) | 0 (0) |
| Right neck | |||
| 1 | 197 (76) | 19 (56) | 178 (79) |
| 2 | 44 (17) | 9 (26) | 35 (16) |
| 3 | 4 (2) | 0 (0) | 4 (2) |
| 4 | 8 (3) | 0 (0) | 8 (4) |
| 5 | 6 (2) | 6 (18) | 0 (0) |
Note: Values are reported as median (range) or number (percent). The left neck, right neck, and primary tumor were scored on the five-point qualitative scale defined by Marcus et al., with the internal jugular vein (IJV) 18F-FDG uptake taken as reference for the background blood pool (Supplementary Methods). The overall score was the highest score among the primary tumor, left neck, and right neck.
3.3 |. Survival, recurrence, and performance of the Hopkins criteria
Thirty-four patients (13%) ultimately developed locoregional persistence/recurrence (primary site: 24 patients; regional nodes: 16 patients), while 33 (13%) developed distant metastasis and 44 (17%) have died. Three patients with locoregional recurrence/persistence who did not undergo repeat biopsy had gross disease evident on both PET/CT and examination/endoscopy, had persistent disease at the time of last follow-up/death, and enrolled in hospice or proceeded immediately to palliative chemotherapy. Additional details regarding these three patients are provided in the Supplementary Results.
Among all patients, the sensitivity, specificity, overall accuracy, positive predictive value, and negative predictive value of the Hopkins Criteria (4–5 vs. 1–3) for LRR were: 68% (95% CI: 51–81%), 88% (95% CI: 83–92%), 85% (95% CI: 81–89%), 46% (95% CI: 33–60%), and 95% (95% CI: 91–97%) (Table 3). Overall accuracy was considerably lower if Hopkins score 3 was considered positive rather than negative (42% vs. 85%).
TABLE 3.
Hopkins score performance for locoregional persistence/recurrence
| Characteristic | All patients | p16/HPV positive |
p16/HPV negative/untested |
|||
|---|---|---|---|---|---|---|
| Locoregional persistence/recurrences | 34 (13) | 21 (9) | 13 (35) | |||
| Cutoff: 4–5 vs. 1–3a | ||||||
| Sensitivity | 67.6% [50.8–80.9%] | 81.0% [60.0–92.3%] | 46.2% [23.2–70.9%] | |||
| Specificity | 88.0% [83.1–91.6%] | 88.1% [82.9–91.8%] | 87.5% [69.0–95.7%] | |||
| Overall accuracy | 85.3% [80.5–89.1%] | 87.4% [82.4–91.1%] | 73.0% [57.0–84.6%] | |||
| Positive predictive value | 46.0% [33.0–59.6%] | 41.5% [27.8–56.6%] | 66.7% [35.4–87.9%] | |||
| Negative predictive value | 94.7% [90.8–97.0%] | 97.7% [94.7–99.0%] | 75.2% [69.5–80.1%] | |||
| Cutoff: 3–5 vs. 1–2 | ||||||
| Sensitivity | 79.4% [63.2–89.7%] | 85.7% [65.4–95.0%] | 69.2% [42.4–87.3%] | |||
| Specificity | 36.4% [30.4–42.9%] | 36.3% [30.0–43.2%] | 37.5% [21.2–57.3%] | |||
| Overall accuracy | 42.1% [36.2–48.2%] | 41.0% [34.7–47.6%] | 48.6% [33.5–64.1%] | |||
| Positive predictive value | 15.9% [11.2–22.1%] | 12.3% [7.9–18.7%] | 37.5% [21.2–57.3%] | |||
| Negative predictive value | 92.1% [84.6–96.1%] | 96.1% [89.0–98.7%] | 69.2% [42.4–87.3%] | |||
| Duration from radiotherapy to first PET/CT (Weeks), Cutoff: 4–5 vs. 1–3 | ||||||
| 6–10 weeks | ||||||
| Sensitivity | 50.0% [9.5–90.6%] | No Recurrences | 50.0% [9.5–90.6%] | |||
| Specificity | 77.8% [45.3–93.7%] | 75.0% [40.9–92.9%] | 100.0% [20.7–100.0%] | |||
| Overall accuracy | 72.7% [43.4–90.3%] | 75.0% [40.9–92.9%] | 66.7% [20.8–93.9%] | |||
| Positive predictive value | 33.3% [6.2–79.2%] | 0.0% [0.0–65.8%] | 100.0% [20.7–100.0%] | |||
| Negative predictive value | 87.5% [52.9–97.8%] | 100.0% [61.0–100.0%] | 50.0% [9.5–90.6%] | |||
| Proportion of patients undergoing negative biopsy or surgical procedureb | 36.4% | 50.0% | 0.0% | |||
| Proportion of biopsies/surgeries negativec | 66.7% | 100.0% | None Performed | |||
| >10–14 weeks | ||||||
| Sensitivity | 65.0% [43.3–81.9%] | 75.0% [46.77–91.1%] | 50.0% [21.5–78.5%] | |||
| Specificity | 88.2% [81.4–92.7%] | 89.6% [82.64–93.9%] | 75.0% [46.8–91.1%] | |||
| Overall accuracy | 85.0% [78.4–89.9%] | 88.2% [81.43–92.7%] | 65.0% [43.3–81.9%] | |||
| Positive predictive value | 46.4% [29.5–64.2%] | 42.9% [24.47–63.5%] | 57.1% [25.1–84.2%] | |||
| Negative predictive value | 94.1% [88.4–97.1%] | 97.2% [92.01–99.0%] | 69.2% [42.4–87.3%] | |||
| Proportion of patients undergoing negative biopsy or surgical procedure | 8.8% | 7.9% | 15.0% | |||
| Proportion of biopsies/surgeries negative | 65.0% | 71.4% | 50.0% | |||
| >14–24 weeks | ||||||
| Sensitivity | 75.0% [46.8–91.1%] | 88.9% [56.5–98.0%] | 33.3% [6.2–79.2%] | |||
| Specificity | 88.8% [80.5–93.8%] | 87.2% [77.98–92.9%] | 100.0% [74.1–100.0%] | |||
| Overall accuracy | 87.1% [79.2–92.3%] | 87.4% [78.76–92.8%] | 85.7% [60.1–96.0%] | |||
| Positive predictive value | 47.4% [27.3–68.3%] | 44.4% [24.56–66.3%] | 100.0% [20.7–100.0%] | |||
| Negative predictive value | 96.3% [89.8–98.8%] | 98.6% [92.24–99.7%] | 84.6% [57.8–95.7%] | |||
| Proportion of patients undergoing negative biopsy or surgical procedure | 8.9% | 15.0% | 14.3% | |||
| Proportion of biopsies/surgeries negative | 60.0% | 63.6% | 50.0% | |||
Note: Values are reported as number (percent) or proportion [95% Wilson confidence interval].
Overall Hopkins scores of 4 or 5 were considered positive, whereas scores of 1–3 were negative.
Proportion of total patients in subgroup that underwent at least one biopsy or surgical procedure which was pathologically negative for viable malignant cells.
Proportion of biopsies and surgical procedures in subgroup that were negative for viable malignant cells.
The 36-month cumulative incidence of LRR was greater among patients with Hopkins scores 4–5 vs. 1–3 (45% vs. 5%, p < 0.001, Table 4 and Figure 1). Scores 4–5 were similarly prognostic for 36-month OS (69% vs. 94%, p = 0.010), PFS (55% vs. 87%, p < 0.001), and DM (22% vs. 10%, p = 0.02). LRR, DM, PFS, and OS did not significantly differ in pairwise comparisons between patients with scores of 3 vs. 1–2.
TABLE 4.
Crude and cumulative incidences of recurrence and death
| Characteristic | All patients | Hopkins score 1–3 (Negative) | Hopkins score 4–5 (Positive) | p value* |
|---|---|---|---|---|
| No. of patients | 259 | 209 (81) | 50 (19) | – |
| Locoregional persistence/recurrence | 34 (13) | 11 (5) | 23 (46) | <0.001 |
| Primary tumor Persistence/recurrence | 24 (9) | 7 (3) | 17 (34) | <0.001 |
| Regional nodal persistence/recurrence | 16 (6) | 5 (2) | 11 (22) | <0.001 |
| Distant metastasis | 33 (13) | 22 (11) | 11 (22) | 0.035 |
| Death | 44 (17) | 29 (14) | 15 (30) | 0.010 |
| Overall survival (95% CI) | 0.001 | |||
| 12 months | 96.5% (94.2–98.7%) | 98.1% (96.2–99.9%) | 89.9% (81.5–98.3%) | |
| 24 months | 92.3% (89.0–95.6%) | 97.1% (94.7–99.4%) | 72.3% (59.5–85.2%) | |
| 36 months | 89.3% (85.3–93.3%) | 94.0% (90.6–97.5%) | 69.4% (55.9–83.0%) | |
| Progression-free survival (95% CI) | <0.001 | |||
| 12 months | 87.1% (83.0–91.2%) | 92.7% (89.1–96.2%) | 63.9% (50.6–77.3%) | |
| 24 months | 81.6% (76.8–86.4%) | 88.0% (83.5–92.5%) | 55.4% (41.5–69.4%) | |
| 36 months | 80.5% (75.5–85.5%) | 86.6% (81.7–91.4%) | 55.4% (41.5–69.4%) | |
| Locoregional persistence/recurrence (95% CI) | <0.001 | |||
| 12 months | 8.6% (5.2–12.1%) | 2.5% (0.3–4.6%) | 34.1% (20.9–47.3%) | |
| 24 months | 12.0% (7.9–16.0%) | 4.0% (1.3–6.7%) | 44.7% (30.7–58.6%) | |
| 36 months | 12.6% (8.4–16.7%) | 4.8% (1.7–7.9%) | 44.7% (30.7–58.6%) | |
| Distant metastasis (95% CI) | 0.020 | |||
| 12 months | 6.7% (3.6–9.7%) | 3.9% (1.3–6.6%) | 18.1% (7.4–28.8%) | |
| 24 months | 10.5% (6.7–14.3%) | 7.6% (3.9–11.4%) | 22.4% (10.7–34.0%) | |
| 36 months | 12.1% (7.9–16.3%) | 9.6% (5.4–13.9%) | 22.4% (10.7–34.0%) |
Note: Actuarial estimates are reported with 95% confidence intervals (95% CI) for overall survival or the cumulative incidences of recurrence, with death as a competing risk.
p values are provided for fisher exact tests (crude incidences), log-rank tests (overall survival), or Gray’s tests (recurrence estimates).
FIGURE 1.
Survival and recurrence among all included patients grouped by positive (4–5), equivocal (3), or negative (1–2) overall Hopkins scores. (A) Overall survival, (B) progression‐free survival, (C) cumulative incidence of locoregional progression; and (D) cumulative incidence of distant metastasis.
We assessed the impact of posttreatment PET/CT timing and p16/HPV status on Hopkins Criteria performance (Table 3). Sensitivity, specificity, and overall accuracy were similar for studies performed >10–14 weeks and >14–24 weeks after radiotherapy; however, studies performed 6–10 weeks after radiotherapy had empirically lower sensitivity, specificity, and overall accuracy. Full details for the subset of patients undergoing early posttreatment PET/CT are provided in Supplementary Table 3. Sensitivity was highest approximately 16 weeks after radiotherapy, while specificity after 8 weeks was similar (Supplementary Figure 3).
Twenty-four patients who did not develop LRR (11%) underwent at least one negative biopsy or operative procedure. A significantly greater proportion of patients who completed early PET/CTs subsequently underwent negative biopsies/surgeries (≤10 weeks: 36% of patients; >10–14 weeks: 9%; >14–24 weeks: 9%, p = 0.03). The underlying incidence of LRR (18% vs. 14% vs. 12%, p = 0.81) and the proportion of biopsies/surgeries that were pathologically negative (67% vs. 65% vs. 60%, p = 0.73) did not significantly differ among these timing groups. No patient experienced primary site recurrence after a pathologically negative primary site biopsy/surgery, and only one patient experienced a neck recurrence after a pathologically negative neck biopsy.
The distribution of PET/CT quality and Hopkins Scores by pre- and posttreatment PET/CT scanner are provided in Supplementary Table 1. There was no significant difference in Hopkins Score overall accuracy by posttreatment PET/CT instrument (p = 0.12) or by pre-/posttreatment expert-assessed PET/CT quality (Supplementary Table 4). The Hopkins subscore at the primary site and neck correlated well with primary and neck recurrences, respectively. For example, among the 23/34 LRRs which were positive by the overall Hopkins score, only one patient had a primary and neck recurrence which was falsely negative by the primary site subscore (3), but not the neck subscore (5).
3.4 |. Sensitivity analyses
We explored generalizability of the Hopkins score in subset sensitivity analyses (Figure 2). Positive Hopkins scores remained significantly associated with LRR across assessed subsets. For example, the Hopkins score appeared generalizable to the subset of 28 patients receiving induction chemotherapy (HR 15.7) vs. no induction chemotherapy (HR 11.4, pint = 0.64). This effect size was greater in p16/HPV-positive versus -negative tumors (HR 23.1 vs. 6.21, pint = 0.08) and later vs. earlier posttreatment PET/CTs (HR 18.1 vs. 4.8, pint = 0.08).
FIGURE 2.
(A) Forest plot of the natural logarithm of hazard ratios and 95% confidence intervals for subset analyses assessing locoregional recurrence (LRR) with positive (4–5) or negative (1–3) Hopkins scores. High values indicate a greater risk of LRR with positive relative to negative Hopkins scores in each subset. When ln(Hazard Ratio) is 0, the hazard ratio is 1. (B) Cumulative incidence of locoregional progression among patients with low (0–33%), intermediate (34–67%), and high (>67%) predicted risk by LASSO variable‐selected competing‐risks regression (empiric estimates: 4.4%, 45.0%, and 93.3%; p < 0.001 between each group). Low, intermediate, and high groups are defined by regression risk estimates ≤6.53, >6.53–8.36, and >8.36). Risk estimates can be calculated from coefficients provided in Table 5. For example, a 60 year‐old former smoker with a 150 cc p16‐positive tumor treated with concurrent cetuximab and a posttreatment PET/CT Hopkins score of 4 would be in the second risk tertile (7.17 = 0.07*60[age] + 0.05[former smoker] − 1.16[p16‐positive] + 0.006*150[GTVp+GTVn cc] + 0.65[cetuximab] + 2.53[Hopkins positive].
We separately assessed the Hopkins Criteria performance by p16/HPV status (Table 3). Sensitivity (81% vs. 46%, p = 0.035) and overall accuracy (87% vs. 73%, p = 0.022) were significantly higher for p16/HPV-positive tumors (Supplementary Figures 4–5).
In a subset of 50 patients with detailed pre-treatment and mid-treatment metabolic tumor volume (MTV) measures (MTVSUVmax, MTV50%SUVmax, MTVSUV=2.0, MTVTLG), pre-treatment MTV measures did not significantly distinguish true positives from false negatives, true negatives from false positives, true positives from false positives, or true negatives from false negatives (Supplementary Table 5).8 However, mid-treatment MTVTLG (total lesion glycolysis) did significantly distinguish true positives from false positives (26.93 vs. 7.71, p = 0.047).
3.5 |. Multivariable competing-risks model
Models with different covariate selection approaches yielded similar AUC (LASSO: 0.89 [95% CI 0.84–0.95]; AIC: 0.89 [95% CI 0.82–0.96]; BIC: 0.86 [95% CI 0.77–0.94]) and similarly low Brier scores indicating low prediction error (LASSO: 0.09 [95% CI 0.04–0.12]; AIC: 0.07 [95% CI 0.04–0.10]; BIC: 0.08 [95% CI 0.05–0.11]). Calibration plots indicated slightly better calibration for the LASSO model, whereas the BIC model was more parsimonious. Patient age, smoking status, p16/HPV status, pre-treatment contoured gross tumor volume, and the use of concurrent cetuximab each had prognostic value in at least one of the three variable selection approaches. AUC (0.79 [95% CI 0.71–0.88]) was lower with a model including only dichotomized Hopkins score (4–5 vs. 1–3).
Given that the LASSO model offered improved calibration with similar prediction accuracy and error, it was selected as the final model (Table 5, Supplementary Figure 6). The observed 36-month cumulative incidence of LRR with low (0–33%), intermediate (34–67%), and high (67–100%) predicted risk for the LASSO model were 4.4%, 45.0%, and 93.3% (p < 0.001 between each partition, Figure 2). Individualized risk estimates can be calculated from coefficients in Table 5 and as described in Figure 2.
TABLE 5.
Final multivariable competing risks regression model for locoregional persistence/recurrence
| Characteristic | Univariable p value | Multivariable model selected by LASSO |
|||
|---|---|---|---|---|---|
| Coefficient | HR | 95% CI | p value | ||
| Hopkins score: 4–5 vs. 1–3 | <0.001 | 2.53 | 12.60 | 5.87–27.20 | <0.001 |
| Agea | <0.001 | 0.07 | 1.07 | 1.02–1.12 | 0.005 |
| Male | 0.96 | – | – | – | – |
| Smoking status | 0.006 | ||||
| Never smoker | [reference] | [reference] | 1.00 | [reference] | |
| Former smoker | 0.007 | 0.05 | 1.05 | 0.49–2.26 | |
| Current smoker | 0.004 | 1.43 | 4.20 | 1.63–10.80 | |
| ECOG performance status | |||||
| 0–1 | [reference] | – | – | – | – |
| 2–3 | 0.63 | – | – | – | – |
| p16/HPV positive | <0.001 | −1.16 | 0.32 | 0.14–0.71 | 0.006 |
| AJCC 7 clinical tumor stage | |||||
| T1 | [reference] | – | – | – | – |
| T2 | 0.57 | – | – | – | – |
| T3 | 0.40 | – | – | – | – |
| T4a/T4b | 0.31 | – | – | – | – |
| AJCC 7 clinical nodal stage | |||||
| N1–N2a | [reference] | – | – | – | – |
| N2b | 0.73 | – | – | – | – |
| N2c | 0.51 | – | – | – | – |
| N3 | 0.68 | – | – | – | – |
| Contoured GTVp+GTVn (cc)b | 0.01 | 0.006 | 1.01 | 1.00–1.01 | 0.005 |
| Induction chemotherapy | 0.49 | – | – | – | – |
| Concurrent chemotherapy | <0.001 | ||||
| Platinum-based | [reference] | [reference] | 1.00 | [reference] | – |
| Cetuximab | 0.01 | 0.65 | 1.91 | 0.90–4.08 | – |
| None | 0.00c | −11.87 | 0.00c | 0.00–0.00c | – |
Note: All patients included in model fitting.
Abbreviations: CI, confidence interval; GTV, gross tumor volume; HPV, human papillomavirus; HR, hazard ratio.
Per 1-year increase.
Per one unit (cc) increase.
Unstable confidence interval and p values due to zero events and few patients (n = 5) not receiving chemotherapy.
4 |. DISCUSSION
Since the adoption of routine PET/CT-guided surveillance for OPSCC, standardized posttreatment assessment is critical to identify patients who may require salvage therapies while simultaneously minimizing unnecessary procedures among patients who have been cured. The Hopkins Criteria are perhaps the most rigorous and best-validated qualitative system for head and neck squamous cell carcinoma (HNSCC) response assessment. Supporting our hypothesis, positive Hopkins scores were significantly associated with LRR after adjusting for confounding prognosticators. Equivocal (3) scores conferred similar prognosis as clearly negative (1–2) scores, and Hopkins Criteria performance was significantly reduced among patients with p16/HPV-negative tumors and early (≤10 weeks) PET/CTs.
Residual nodal abnormalities are common within the first 3 months after definitive radiotherapy, but a minority of these patients have persistent/recurrent disease.7,25 Following the PET-NECK randomized trial that established PET/CT-guided surveillance as standard, several studies have sought to standardize PET/CT-guided surveillance for HNSCC.2 Marcus et al. originally described the simple qualitative Hopkins Criteria and their performance for a cohort of 214 patients with HNSCC.11 Overall inter-reader agreement was 0.69–0.79, and the sensitivity (68%), specificity (92%), and overall accuracy (87%) were nearly identical to the present study (68%, 88%, and 85%) despite differing populations and endpoints (PFS vs. LRR). In 2017, Kendi et al. conducted an external validation for PFS in a cohort of 69 patients.12 Overall inter-reader agreement was excellent (0.91), while sensitivity (67%) and specificity (87%) were again similar to our findings.12 Notably, neither study validated the Criteria for LRR, DM, and PFS separately, and did not incorporate clinicopathologic risk factors (smoking, p16 status, chemotherapy) that can improve recurrence prognostication.26 We observed that combining these known prognosticators with the Hopkins score further improves recurrence prediction with high discrimination.
Urban et al. recently reported results from a similar study of 556 patients with OPSCC using an analogous dichotomized qualitative system.27 Complete metabolic response was defined as any focal moderate/intense FDG uptake at 12 weeks. Importantly, the authors demonstrated that PET/CT specificity was inferior for PET/CTs performed within 12 weeks of radiotherapy, generally consistent with our findings and a prior meta-analysis of studies not using standardized scoring systems.9 This decrease in specificity is the rationale for current society guidelines that recommend initial PET/CT after at least 12 weeks.3,28,29 In contrast to earlier studies,30 Urban et al. also demonstrated that PPV was inferior for p16/HPV-negative tumors, thought this may relate to differential event rates. In our study, we build upon this finding and observe that sensitivity and overall accuracy for LRR are significantly inferior for p16/HPV-negative tumors.
Several other posttreatment qualitative scoring have been described. Zhong et al. recently compared the NI-RADS, Porceddu, Hopkins, and Deauville systems.14 Each of these 3–5 point systems distinguishes complete, equivocal, and incomplete metabolic response with or without reference tissues (liver, internal jugular vein, mediastinal blood pool). In this series, the Hopkins Criteria scored the fewest number of cases as indeterminate (score 3) but had slightly reduced NPV relative to the Porceddu and Deauville systems (88% vs. 91–93%). Overall accuracy for the primary tumor and neck were similar across systems.
Beyond external validation of the Hopkins Criteria for LRR, an additional important finding in our study is the equivalent long-term recurrence and survival between patients with negative (1–2) and equivocal (3) Hopkins Scores. This has been previously observed for overall survival in a series of 236 patients scored with a three-level qualitative system.10
Most series studying the Hopkins Criteria have dichotomized scores into a binary system of positive (4–5) vs. negative (1–3). Alternatively, one might consider an alternative binary system (positive [3–5] vs. negative [1, 2]) or a ternary system of positive (4–5), equivocal (3), and negative (1–2). For example, certain lymphoma clinical trials consider Deauville 3 scores as positive.31We find no evidence that equivocal (3) Hopkins scores confer any worse prognosis than clearly negative (1–2) scores (Supplementary Figure 2). This is highly relevant, as patients with equivocal scores constituted the largest group (46%). This finding supports scores 1–3 being considered negative when dichotomizing studies for management decisions.
We also find that early (≤10 weeks) PET/CTs not only have reduced diagnostic performance, but that this may translate to an increased rate of pathologically negative biopsies/surgeries despite similar LRR event rates. One might reasonably expect early PET/CTs to have low specificity secondary to posttreatment inflammation/mucositis, but we also importantly observe that sensitivity is decreased with early (50%, ≤10 weeks) vs. late (69%, >10 weeks) PET/CTs. This further reinforces current recommendations for obtaining PET/CTs at least 12 weeks after radiotherapy, and suggests that alternative diagnostic approaches (e.g., biopsy) should be considered among patients with high suspicion for early locoregional persistence/recurrence.
Although the Hopkins Criteria have reasonable performance, sensitivity and positive predictive value are relatively low (40–70%),6,32 and it would be desirable to triage positive PET/CTs with additional quantitative prognosticators. Wray et al. studied the added value of CT-based nodal size and morphologic criteria, but observed no additional discriminatory value with CT features added to the Hopkins Criteria.13 Other studies have also failed to demonstrate added utility in posttreatment PET/CT SUVmax.5,32 Given the very low incidence of LRR after a negative PET/CT for p16/HPV-positive tumors (36-month: 2.6%), further imaging to identify these false negatives is unlikely to be cost-effective.
Our study has several limitations. First, the retrospective institutional design introduces measurement and selection biases. Given the long study period, advances in imaging, pathology, radiotherapy, chemotherapy, and supportive care could limit generalizability. A single board-certified nuclear medicine physician with head/neck subspecialty experience assigned Hopkins scores; while high inter-rater agreement has been previously described, our findings may not be generalizable to all PET/CT readers. Given the epidemiologic rise of p16/HPV-positive oropharynx cancer during the study period, PET/CT quality could confound performance for HPV/p16-negative tumors, although we observed no such evidence in unadjusted or multivariable analyses. Our findings are also not generalizable to cT1–4 N0 tumors which were excluded. We assessed for differential Hopkins Criteria performance across multiple subsets, and generally observed stable performance with the exception of p16/HPV-negative tumors and early PET/CTs, which were both relatively small cohorts (n = 37, n = 11). Additional limitations include a low event rate, lack of comparison against alternative semi-quantitative scoring systems, and lack of an external validation dataset (necessitating internal validation with cross-validation). Strengths of this study include the large sample size, assessment of individual recurrence patterns, cross-validated multivariable competing-risk modeling to reliably ascertain the Criteria’s independent prognostic value, and sensitivity analyses to study the Criteria’s robustness in performance.
5 |. CONCLUSION
There is a lack of consensus reporting standards for response assessment after definitive radiotherapy for OPSCC. The Hopkins Criteria are an easily implemented robust standard developed for head and neck cancers that appear to offer high accuracy for locoregional recurrence. This system’s performance may be reduced with early PET/CTs and for p16/HPV-negative tumors, but is otherwise robust across PET/CT instruments, quality, patient presentations, and treatment regimens. These data support current recommendations for initial PET/CT at least 12 weeks after radiotherapy to limit unnecessary procedures. An internally validated model integrating both clinicopathologic risk factors and the Hopkins Criteria improves prognostication relative to either approach alone.
Supplementary Material
Footnotes
CONFLICT OF INTEREST
The authors declare no conflict of interest.
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1.Asheer J, Jensen JS, Grønhøj C, Jakobsen KK, von Buchwald C. Rate of locoregional recurrence among patients with oropharyngeal squamous cell carcinoma with known HPV status: a systematic review. Acta Oncol 2020;59(9):1131–1136. doi: 10.1080/0284186X.2020.1759822 [DOI] [PubMed] [Google Scholar]
- 2.Mehanna H, Wong WL, McConkey CC, et al. PET-CT surveillance versus neck dissection in advanced head and neck cancer. N Engl J Med 2016;374(15):1444–1454. doi: 10.1056/NEJMoa1514493 [DOI] [PubMed] [Google Scholar]
- 3.NCCN Clinical practice guidelines in oncology. Head and neck cancers. Version 1.2021. 2020. https://www.nccn.org/professionals/physician_gls/pdf/head-and-neck.pdf [DOI] [PubMed]
- 4.Lowe VJ, Duan F, Subramaniam RM, et al. Multicenter trial of [18F]fluorodeoxyglucose positron emission tomography/computed tomography staging of head and neck cancer and negative predictive value and surgical impact in the N0 neck: results from ACRIN 6685. J Clin Oncol off J Am Soc Clin Oncol 2019;37(20):1704–1712. doi: 10.1200/JCO.18.01182 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Moeller BJ, Rana V, Cannon BA, et al. Prospective risk-adjusted [18F]Fluorodeoxyglucose positron emission tomography and computed tomography assessment of radiation response in head and neck cancer. J Clin Oncol off J Am Soc Clin Oncol 2009;27(15):2509–2515. doi: 10.1200/JCO.2008.19.3300 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Yao M, Smith RB, Graham MM, et al. The role of FDG PET in management of neck metastasis from head-and-neck cancer after definitive radiation treatment. Int J Radiat Oncol Biol Phys 2005;63(4):991–999. doi: 10.1016/j.ijrobp.2005.03.066 [DOI] [PubMed] [Google Scholar]
- 7.Porceddu SV, Pryor DI, Burmeister E, et al. Results of a prospective study of positron emission tomography-directed management of residual nodal abnormalities in node-positive head and neck cancer after definitive radiotherapy with or without systemic therapy. Head Neck. 2011;33(12):1675–1682. doi: 10.1002/hed.21655 [DOI] [PubMed] [Google Scholar]
- 8.Pollom EL, Song J, Durkee BY, et al. Prognostic value of midtreatment FDG-PET in oropharyngeal cancer. Head Neck. 2016;38(10):1472–1478. doi: 10.1002/hed.24454 [DOI] [PubMed] [Google Scholar]
- 9.Isles MG, McConkey C, Mehanna HM. A systematic review and meta-analysis of the role of positron emission tomography in the follow up of head and neck squamous cell carcinoma following radiotherapy or chemoradiotherapy. Clin Otolaryngol 2008;33(3):210–222. doi: 10.1111/j.1749-4486.2008.01688.x [DOI] [PubMed] [Google Scholar]
- 10.Zhou S, Rulach R, Hendry F, et al. Positron emission tomography-computed tomography surveillance after (chemo) radiotherapy in advanced head and neck squamous cell cancer: beyond the PET-NECK protocol. Clin Oncol R Coll Radiol 2020;32(10):665–673. doi: 10.1016/j.clon.2020.05.018 [DOI] [PubMed] [Google Scholar]
- 11.Marcus C, Ciarallo A, Tahari AK, et al. Head and neck PET/-CT: therapy response interpretation criteria (Hopkins criteria)-interreader reliability, accuracy, and survival outcomes. J Nucl Med 2014;55(9):1411–1416. doi: 10.2967/jnumed.113.136796 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kendi AT, Brandon D, Switchenko J, et al. Head and neck PET/CT therapy response interpretation criteria (Hopkins criteria) - external validation study. Am J Nucl Med Mol Imaging. 2017;7(4):174–180. [PMC free article] [PubMed] [Google Scholar]
- 13.Wray R, Sheikhbahaei S, Marcus C, et al. Therapy response assessment and patient outcomes in head and neck squamous cell carcinoma: FDG PET Hopkins criteria versus residual neck node size and morphologic features. AJR Am J Roentgenol 2016;207(3):641–647. doi: 10.2214/AJR.15.15730 [DOI] [PubMed] [Google Scholar]
- 14.Zhong J, Sundersingh M, Dyker K, et al. Post-treatment FDG PET-CT in head and neck carcinoma: comparative analysis of 4 qualitative interpretative criteria in a large patient cohort. Sci Rep 2020;10(1):4086. doi: 10.1038/s41598-020-60739-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gray RJ. A class of K-sample tests for comparing the cumulative incidence of a competing risk. Ann Stat 1988;16(3):1141–1154. [Google Scholar]
- 16.La TH, Filion EJ, Turnbull BB, et al. Metabolic tumor volume predicts for recurrence and death in head-and-neck cancer. Int J Radiat Oncol Biol Phys 2009;74(5):1335–1341. doi: 10.1016/j.ijrobp.2008.10.060 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Fu Z, Parikh CR, Zhou B. Penalized variable selection in competing risks regression. Lifetime Data Anal 2017;23(3):353–376. doi: 10.1007/s10985-016-9362-3 [DOI] [PubMed] [Google Scholar]
- 18.Fu Z Crrp: penalized variable selection in competing risks regression; 2015. https://CRAN.R-project.org/package=crrp [DOI] [PubMed]
- 19.Kuk K and Varadhan R. Crrstep: stepwise covariate selection for the fine & gray competing risks regression model. 2015. https://CRAN.R-project.org/package=crrstep
- 20.Kuk D, Varadhan R. Model selection in competing risks regression. Stat Med 2013;32(18):3077–3088. doi: 10.1002/sim.5762 [DOI] [PubMed] [Google Scholar]
- 21.Gerds TA, Blanche P, Mortensen R, et al. RiskRegression: risk regression models and prediction scores for survival analysis with competing risks.; 2020. https://CRAN.R-project.org/package=riskRegression
- 22.Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc 1999;94(446): 496–509. doi: 10.1080/01621459.1999.10474144 [DOI] [Google Scholar]
- 23.Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Radiology. 2015;277(3):826–832. doi: 10.1148/radiol.2015151516 [DOI] [PubMed] [Google Scholar]
- 24.Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162(1):55–63. doi: 10.7326/M14-0697 [DOI] [PubMed] [Google Scholar]
- 25.Wong ET, Huang SH, O’Sullivan B, et al. Head and neck imaging surveillance strategy for HPV-positive oropharyngeal carcinoma following definitive (chemo)radiotherapy. Radiother Oncol 2021;157:255–262. doi: 10.1016/j.radonc.2021.02.005 [DOI] [PubMed] [Google Scholar]
- 26.Ang KK, Harris J, Wheeler R, et al. Human papillomavirus and survival of patients with oropharyngeal cancer. N Engl J Med 2010;363(1):24–35. doi: 10.1056/NEJMoa0912217 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Urban R, Godoy T, Olson R, et al. FDG-PET/CT scan assessment of response 12 weeks post radical radiotherapy in oropharynx head and neck cancer: the impact of p16 status. Radiother Oncol 2020;148:14–20. doi: 10.1016/j.radonc.2020.03.032 [DOI] [PubMed] [Google Scholar]
- 28.Machiels JP, René Leemans C, Golusinski W, et al. Squamous cell carcinoma of the oral cavity, larynx, oropharynx and hypo-pharynx: EHNS-ESMO-ESTRO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol 2020;31(11): 1462–1475. doi: 10.1016/j.annonc.2020.07.011 [DOI] [PubMed] [Google Scholar]
- 29.Lewis-Jones H, Colley S, Gibson D. Imaging in head and neck cancer: United Kingdom National Multidisciplinary Guidelines. J Laryngol Otol 2016;130(S2):S28–S31. doi: 10.1017/S0022215116000396 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Awan MJ, Lavertu P, Zender C, et al. Post-treatment PET/CT and p16 status for predicting treatment outcomes in locally advanced head and neck cancer after definitive radiation. Eur J Nucl Med Mol Imaging. 2017;44(6):988–997. doi: 10.1007/s00259-016-3612-1 [DOI] [PubMed] [Google Scholar]
- 31.Radford J, Illidge T, Counsell N, et al. Results of a trial of PET-directed therapy for early-stage Hodgkin’s lymphoma. N Engl J Med 2015;372(17):1598–1607. doi: 10.1056/NEJMoa1408648 [DOI] [PubMed] [Google Scholar]
- 32.Vainshtein JM, Spector ME, Stenmark MH, et al. Reliability of post-chemoradiotherapy F-18-FDG PET/CT for prediction of locoregional failure in human papillomavirus-associated oropharyngeal cancer. Oral Oncol 2014;50(3):234–239. doi: 10.1016/j.oraloncology.2013.12.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.