Summary
Oral cavity squamous cell carcinoma (OCSCC) is a prevalent surgically treated subset of head and neck cancer with frequent recurrence and poor survival. Immunotherapy has demonstrated efficacy in recurrent/metastatic head and neck cancer. However, whether antitumor responses could be fostered by neoadjuvant presurgical immunotherapy remains unclear. Using a Simon’s two-stage design, we present results of a single-arm phase-II trial where 12 patients with stage II-IVA OCSCC received 3 to 4 biweekly doses of 3 mg/kg nivolumab followed by definitive surgical resection with curative intent. Presurgical nivolumab therapy in this cohort shows an overall response rate of 33% (n = 4 patients; 95% CI: 12%–53%). With a median follow up of 2.23 years, 10 out of 12 treated patients remain alive. Neoadjuvant nivolumab is safe, well-tolerated, and is not associated with delays in definitive surgical treatment in this study. This work demonstrates feasibility and safety for incorporation of nivolumab in the neoadjuvant setting for OCSCC (ClinicalTrials.gov: NCT03021993).
Keywords: neoadjuvant, PD-1 therapy, nivolumab, neoadjuvant immunotherapy, oral cavity cancer, head and neck cancer
Graphical abstract
Highlights
-
•
Presurgical nivolumab in patients with resectable oral cancer has an ORR of 33%
-
•
10 out of 12 treated patients remain alive with a median follow up of 2.23 years
-
•
Neoadjuvant nivolumab is safe, well-tolerated, and not associated with treatment delays
-
•
This shows feasibility of nivolumab in the neoadjuvant setting for OCSCC (NCT03021993)
Oral cavity squamous cell carcinoma is a prevalent subset of head and neck cancer with high recurrence rates and poor survival outcomes despite complex multimodality therapy. The current study by Knochelmann et al. demonstrates that presurgical nivolumab therapy has an overall response rate of 33% in this patient population.
Introduction
Oral cavity squamous cell carcinoma (OCSCC) is a tobacco-related head and neck cancer that accounts for 350,000 new cancer diagnoses and 170,000 fatalities worldwide.1 Treatment of OCSCC often requires complex, multimodality therapy of surgical resection followed by post-operative radiation with the addition of platinum-based chemotherapy for patients at high risk of failure.2,3 Despite these comprehensive strategies (and in contrast to less aggressive human papillomavirus [HPV]-related oropharyngeal cancers), OCSCC outcomes are poor, and the disease recurs in 25%–50% of patients.4,5 Due to these poor outcomes, different treatment regimens have been considered, including neoadjuvant chemotherapy. Unfortunately, meta-analysis of induction chemotherapy prior to definitive surgical or radiation therapy revealed only a modest reduction in distant metastases and no improvement in locoregional control or overall survival—therefore, it is not considered to be the standard of care.6
Recent advances have revealed that immunotherapies, like programmed death 1 (PD-1) blockade, demonstrate benefits in some cohorts of patients over traditional chemotherapy in recurrent and metastatic head and neck cancer.7,8 Therefore, while neoadjuvant chemotherapies prior to surgery have not demonstrated significant improvement in outcomes, perhaps neoadjuvant immunotherapy could be beneficial for some patients. Head and neck squamous cell carcinoma (HNSCC) demonstrates among the highest immune infiltration of all solid tumors,9,10 has a high tumor mutational burden,11,12 and expresses elevated levels of PD-1 and its primary ligand PD-L1 within the malignancy.13 It is therefore appealing to expand the clinical indications for PD-1 blockade into larger subsets of head and neck cancer patients, including patients with resectable disease, at an earlier point in disease progression.
Neoadjuvant immunotherapy for surgically resectable OCSCC is expected to reduce tumor burden and generate anti-tumor immunity, thereby improving long-term clinical outcomes.14 Success of this strategy is predicted based on the emerging results of presurgical PD-1 inhibition in similarly immunogenic tumors such as melanoma15 and lung cancer.16 A single pre-operative dose and adjuvant pembrolizumab for high-risk HPV-unrelated HNSCC is associated with a pathologic tumor response (pTR) >10% in 44% of patients, with pTR defined as the proportion of the resection bed with tumor necrosis, keratinous debris, and giant cells/histocytes.17 A subsequent study investigating neoadjuvant nivolumab or nivolumab plus ipilimumab reports volumetric response rates of 50% or 53%, respectively, with volumetric response defined as any reduction in tumor size (which may increase the response rate).18 Neither trial compares clinical or radiographic tumor response to pathologic response.17,18 Other preliminary trials show evidence of pathological response to neoadjuvant pembrolizumab,19 with additional cycles of immunotherapy prior to surgery associated with better responses.20 Therefore, these early reports concur on the potential of neoadjuvant immunotherapy to benefit at least some HNSCC patients.
We designed a Simon two-stage, phase-II, single-arm clinical trial (Figure 1) of neoadjuvant nivolumab (anti-PD-1 monoclonal antibody) before surgical resection in stage II-IVA OCSCC. Upon enrollment, patients received nivolumab every two weeks for a total of three doses prior to interval radiographic evaluation. In the event of disease progression, patients received definitive surgical resection. If stable disease or response was observed, patients received a 4th dose of nivolumab followed by definitive surgical resection (see STAR Methods). This trial incorporated a pathology-enhanced RECIST with a primary endpoint of objective response rate (ORR) defined as pathologic complete response + pathologic partial response (>30% reduction in tumor size).21,22 Results from this trial reported herein corroborate emerging literature demonstrating the potential of neoadjuvant immunotherapy to instill response and survival benefit in subgroups of HNSCC patients and provided longitudinal tissues for in-depth molecular analysis to nominate mechanisms of response, resistance, and post-surgical recurrence.23
Results
Patient characteristics
Beginning April 2017 through April 2020, 14 patients enrolled and received at least 1 dose of nivolumab. Two additional patients were enrolled but came off the study prior to receiving any nivolumab. For inclusion in the efficacy analysis, patients were specified to receive at least 2 doses of nivolumab (see Protocol section 11.1.2 in Methods S1). One (patient 8) experienced treatment delay due to hospitalization for dehydration between his 1st and 2nd dose of nivolumab that resulted in significant disease progression; therefore, he was removed from the trial and proceeded directly to surgical resection. This patient was not included in the study analysis per trial protocol. One (patient 11) received 4 doses of nivolumab and demonstrated a clinical response, but this patient was lost to follow-up prior to surgery and was not included in the analysis. One patient (patient 16) received only 2 doses of nivolumab prior to surgery, due to research restrictions during the COVID-19 outbreak, to prevent any delay to surgery. This patient was included in the analysis per trial protocol. All remaining patients received either 3 or 4 doses of nivolumab and curative-intent surgical resection. We did not observe substantial differences between eventual responders and non-responders in their baseline characteristics (Table 1).
Table 1.
All patients (n = 12) | ||
---|---|---|
Age, years | mean ± SD | 64 ± 8.0 |
median (range) | 62 (48–78) | |
Gender (%) | male | 5 (42) |
female | 7 (58) | |
Smoking status (%) | current | 6 (50) |
former | 3 (25) | |
never | 3 (25) | |
Alcohol use (drinks/week) | mean ± SD | 13.2 ± 27.4 |
median (range) | 5.5 (0–98) | |
ECOG status | 0 | 5 (42) |
1 | 7 (58) | |
T stage (%) | T2 | 3 (25) |
T3 | 3 (25) | |
T4a | 6 (50) | |
N stage (%) | N0 | 4 (33) |
N1 | 4 (33) | |
N2b | 2 (16) | |
N2c | 2 (16) | |
Clinical stage (%) | II | 3 (25) |
III | 1 (8) | |
IVA | 8 (67) | |
Tumor size (greatest dimension), cm | mean ± SD | 3.6 ± 1.1 |
median (range) | 3.1 (2.1–5.4) | |
Histologic grade | well | 1 (8) |
mod to well | 1 (8) | |
moderate | 5 (42) | |
mod to poor | 3 (25) | |
poor | 1 (8) | |
spindle | 1 (8) |
Safety and feasibility
As per trial protocol, adverse events were monitored on patients throughout treatment and up to at least 100 days following the final dose of treatment. Nivolumab was well-tolerated with no grade 4 adverse events (AEs) (Table 2). One grade 3 AE of pain was reported in patient 16, which was resolved and deemed unrelated to treatment. 12 patients experienced a total of 67 grade 1 to 2 AEs, which included fatigue, nausea, diarrhea, oral pain and non-oral pain, rash/psoriasis, myalgia, constipation, cough, creatinine increase, dyspnea, back spasm, and hypertension (Table 2; Table S1). The most common AEs reported included constipation (n = 7; 58%), oral pain (n = 5; 42%), non-oral pain (n = 5; 42%), and weight loss (n = 4; 33%). Of AEs “possibly” or “definitely” related to treatment, fatigue, myalgia, nausea, diarrhea, and rash were each reported in 17% of patients (n = 2) (Table 2). At least one AE “possibly” or “definitely” related to treatment was experienced in 100% of responders (n = 4), 75% of non-responders (n = 3), and 25% of patients with stable disease (n = 1). There were no dose reductions, study withdrawals, or delays in definitive surgery due to AEs.24 Treatment interruption was required in only 1 patient (patient 16) due to COVID-19 shutdowns, though this patient was still able to receive 2 doses of nivolumab. All other patients (11 of 12) received either 3 or 4 doses according to the study design (Figure S1). Median (range) time from 1st nivolumab dose to surgery was 40 days (38–41) for those receiving three doses (n = 6) and 52 days (47–55) for those receiving four doses (n = 5), which were all within the predefined ranges to avoid delay of surgical resection. Time from 1st nivolumab dose to surgery was 27 days for the 1 patient receiving 2 doses. Negative margins (1 close) were achieved in all patients.
Table 2.
All events |
“Definitely” or “possibly” related to nivolumab |
|||
---|---|---|---|---|
No. (%), n = 12 patients | ||||
Characteristic | Grades 1 and 2 | Grades 3 and 4 | Grades 1 and 2 | Grades 3 and 4 |
Constipation | 7 (58) | 0 (0) | 1 (8) | 0 (0) |
Pain (non-oral) | 5 (42) | 1 (8) | 1 (8) | 0 (0) |
Oral pain | 5 (42) | 0 (0) | 0 (0) | 0 (0) |
Weight loss | 4 (33) | 0 (0) | 0 (0) | 0 (0) |
Fatigue | 3 (25) | 0 (0) | 2 (17) | 0 (0) |
Myalgia | 2 (17) | 0 (0) | 2 (17) | 0 (0) |
Oral mucositis/dry mouth/dysphagia | 2 (17) | 0 (0) | 0 (0) | 0 (0) |
Nausea | 2 (17) | 0 (0) | 2 (17) | 0 (0) |
Diarrhea | 2 (17) | 0 (0) | 2 (17) | 0 (0) |
Hypokalemia | 2 (17) | 0 (0) | 0 (0) | 0 (0) |
Painful swelling | 2 (17) | 0 (0) | 0 (0) | 0 (0) |
Loss of appetite | 2 (17) | 0 (0) | 0 (0) | 0 (0) |
Rash | 2 (17) | 0 (0) | 2 (17) | 0 (0) |
Muscle stiffness | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Numbness of tongue | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Cough | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Neck stiffness | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Actinic keratosis | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Creatinine increase | 1 (8) | 0 (0) | 1 (8) | 0 (0) |
Dyspnea | 1 (8) | 0 (0) | 1 (8) | 0 (0) |
Thrush | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Papilloma | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
ALT increase | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Back spasms | 1 (8) | 0 (0) | 1 (8) | 0 (0) |
Hypertension | 1 (8) | 0 (0) | 1 (8) | 0 (0) |
Surgical site infection | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Anxiety | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Hearing impairment | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Hypomagnesemia | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Hypercalcemia | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Facial edema | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
T8 compression deformity | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Insomnia | 1 (8) | 0 (0) | 0 (0) | 0 (0) |
Efficacy
Of the 12 enrolled patients eligible for efficacy analysis, 4 had >30% response (33.3%, 95% CI: 12%–53%), 4 had stable disease (33.3%), and 4 had progression of their disease (33.3%) resulting in an ORR of 33.3% (95% CI: 12%–53%) (Figure 2). There were no complete pathologic responses.
Based on the trial design, clinical response on radiographic imaging following the 3rd dose of nivolumab was used to determine which patients received a 4th dose prior to surgery. We sought to evaluate the validity of this design. Neither initial tumor size (Figures S1A and S1B) nor tumor size following dose 3 (Figures S1C and S1D) correlated with ultimate pathologic response to treatment. However, changes in tumor size from enrollment to interval imaging after 3 doses of nivolumab (“clinical response”) were associated with ultimate response to treatment (“surgical-pathologic response”) (Figure S2A). That is, all patients that eventually had a pathologic response demonstrated clinical reduction in tumor size on interval scan, and those with pathologic stable or progressive disease had no change or increased tumor burden on interval scan (Figure S2A). A positive correlation between clinical response on interval presurgical imaging and final pathologic response was identified (Figure S2B; r = 0.745; p = 0.011). Note that patient 16, who received only 2 doses, did not have an interval scan due to the impact of COVID-19 on clinical research at our institution and thus was excluded from this specific analysis.
At median follow-up of 2.23 years (0.43–3.32), 10 of 12 patients were alive (thus a median overall survival [OS] was not reached) (Figure 3A), and 7 patients remained recurrence-free. All responders or patients with stable disease remain alive, while 2 of 4 progressors have died of disease (Figure 3B). Median disease-free survival was 3.2 years overall (Figure 3C). By response category, median disease-free survival was 1.9 years, not reached, and 2.1 years for responders, stable disease, and non-responders, respectively (Figure 3D). 9 (75%) patients underwent adjuvant radiation (radiation was recommended but refused by patients 2, 14, and 16). 2 patients (17%) had the addition of concurrent adjuvant chemotherapy for extra-nodal extension (patients 4 and 12). Median (range) time to start of adjuvant treatment following surgery was 41 days (14–48). 2 patients (22%) had delays in commencement of adjuvant radiation treatment beyond 6 weeks,25 due to patient preference.
Post-surgical recurrences
Up to a data cutoff of February 23, 2021, there was a total of 8 recurrences in 5 patients (Figure 2; Table S1). Patient 1 developed regional metastasis in the contralateral neck 5 months after surgical resection; this recurrence was treated with neck dissection and chemoradiation. This patient had not received contralateral neck dissection during her on-trial surgery, and the contralateral neck was not included in the post-operative radiation field. This same patient subsequently developed distant metastasis to the lung 1 year after on-trial surgery that was treated with stereotactic radiation therapy. This patient then developed local and regional recurrence and was treated with maxillectomy and lateral pharyngectomy. Patient 2 developed a local recurrence 3.29 years after enrollment and underwent a subtotal glossectomy. Patient 4 developed regional metastasis to the contralateral neck 9 months after on-trial surgery, despite having received post-operative platinum-based chemotherapy and radiation (including the contralateral neck). This patient was treated with neck dissection and adjuvant chemoradiation but subsequently developed a 2nd large surgical site recurrence and passed away 12 months after on-trial surgery. Patient 6 had a first local recurrence 2 years after enrollment and underwent a composite resection with segmental mandibulectomy followed by adjuvant re-irradiation. This patient developed 2nd local recurrence 2 months following completion of adjuvant re-irradiation and received chemotherapy given the short disease-free interval; this patient has responded well to chemotherapy and is currently alive with evidence of disease. Finally, patient 7 developed local recurrence at the posterior margin of the surgical site in the oropharynx 7 months after on-trial surgery; this recurrence was treated with chemoradiation. On-trial surgery was noted to have a close margin at this site (0.3 cm) on surgical pathology, and this patient is now deceased from disease.
Immunologic evaluation
In a responder, decrease in tumor size on clinical and radiographic images was associated with marked lymphoid infiltration on hematoxylin and eosin staining of the surgical specimen (Figure 4A), compared to tumor progression and muted lymphoid infiltration (Figure 4B) in a non-responder. To further evaluate the immune infiltration in responding versus non-responding patients, we performed multiplexed immunofluorescence of post-treatment surgical specimens (Figure 5). A representative interface of tumor tissue, identified by cytokeratin expression, is presented from the most responsive tumor (from patient 9) (Figures 5A and 5B) and the most progressive tumor (from patient 4) (Figures 5C and 5D). In the responding tumor, minimal PD-L1 staining was noted to coincide with the remaining cytokeratin areas, while robust CD4+ and CD8+ infiltrates were observed in the surrounding tissue with scant FoxP3+ expression in the infiltrative cells (Figure 5B). In contrast, in the progressive tumor, PD-L1 markedly overlapped with cytokeratin across approximately 50% of the whole-slide tissue (Figure 5C). While CD4+ and CD8+ T cell infiltrates were present in the surrounding tissue, relatively few of these lymphocytes overlapped with the PD-L1/cytokeratin staining, where a more exuberant FoxP3+CD4+ population was present (Figure 5D).
Discussion
We report here a phase-II trial of neoadjuvant nivolumab for OCSCC, which yielded a meaningful ORR of 33%. Treatment was well-tolerated, with only one grade 3 AE, which resolved, and no grade 4 AEs. There were no delays in definitive surgery and therefore no deviations from the current standard of care. This identified response rate surpassed the total number of responders (n = 2) required to consider this a positive trial based on a priori statistical considerations of the Simon two-stage design. Due to the initial trial design, limited tissue availability prior to treatment precluded robust evaluation of PD-L1 expression as a predictive biomarker of response to therapy. Follow-up analysis revealed that partial response and stable disease appear to correlate with an improved overall survival. Although these findings need to be interpretated with caution, they corroborate results from previous immunotherapeutic window of opportunity trials.17,18 Our work provides additional rationale to execute larger randomized clinical trials to this disease setting, such as the ongoing phase-III trial evaluating pembrolizumab as a neoadjuvant/adjuvant agent relative to no neoadjuvant therapy and adjuvant standard of care radiotherapy/chemotherapy (ClinicalTrials.gov: NCT03765918). This trial also generated longitudinal tissues that have been analyzed deeply to provide molecular and immune insights into the reported response, relapse, and survival patterns.23
Importantly, clinical response as determined by re-imaging at week 4 to 5 (after three doses of nivolumab) predicted patients that ultimately had a pathologic response. This technique of comparing radiographic response to pathologic is used frequently in other solid tumors including breast and colon cancer, where neoadjuvant therapy is the standard of care.22,26 The ability of radiography to predict pathologic response in these OCSCC patients was unique compared to analogous immunotherapy studies in other solid tumors,16,27 where change in tumor size on interval imaging was inconsistent with the final pathological response. Theoretically, this incongruity can be attributed to transient and unpredictable immune infiltration and inflammation of the tumor, so-called “pseudoprogression.” However, reported rates of pseudoprogression are generally lower in head and neck cancer,27 which is corroborated by our findings.
Multiplexed imaging of the post-treatment surgical specimens in our best responding patient (patient 9) relative to the worst progressing patient (patient 4) revealed striking differences at the tumor/immune interface. Both patients demonstrated significant infiltration of CD4+ and CD8+ T cells, suggesting that the presence of immune cells themselves was not a limiting factor for response. However, the responding patient’s post-treatment surgical tumor sample showed minimal overlap of PD-L1 staining with tumor cells, in contrast to high intensity PD-L1 staining coinciding with tumor marker staining in the non-responder. While CD4+ and CD8+ T cells overlapped with tumors in the responder, relatively fewer immune cells but more T regulatory cells (FoxP3+CD4+) were present in cytokeratin+ areas in the non-responder, who went on to relapse and die of disease. Future investigations in a comprehensive cohort will be important to validate and understand these histologic differences as mechanisms of response or relapse.
Notably, 5 patients recurred, including 2 responders and 3 non-responders, which resulted in a 41% relapse rate in our patient population over the total follow-up period. Our relapse rate at 1 year following enrollment was 3 out of 12 patients, or 25%, which is comparable to a prior report of a 16.7% incidence of relapse after 1 year in high-risk patients treated with neoadjuvant pembrolizumab.17 Our patients who are deceased since the commencement of the study were not responsive to neoadjuvant anti-PD-1 therapy. These findings highlight the urgent need to understand mechanisms of response to anti-PD-1 therapy as well as biomarkers that will identify patients likely to benefit in order to most appropriately incorporate immunotherapy into treatment plans.
Two of the recurring patients developed regional lymph node disease in the contralateral neck. The decision to treat the contralateral neck up front (with surgery or radiation) is controversial in OCSCC, and practices vary by institution. Reported rates of recurrence in the untreated contralateral neck are ∼5% and even lower when this neck is treated.28 A past concern with pre-surgical treatments was how to approach the surgical margins in responders, i.e., whether the margins should be planned around the pre- or post-therapy tumor. Though all margins were negative in this study, there was 1 close margin which associated with a local recurrence in 1 patient. Cancer cells likely persist at the invasive margin even in responders. Therefore, until this is explicitly investigated, the most prudent strategy should be to plan resection based on the margins of the untreated tumor, relying on training and experience to make appropriate intra-operative decisions regarding the extent of resection. Although the current sample size is small, our recurrence rate warrants close monitoring of these metrics moving forward. Recurrences in the clinical trial setting here, where all patients were discontinued on nivolumab after in-trial surgery, cannot be considered true acquired resistance. Thus, in addition to the aforementioned issue of surgical margins, another possible contributor to clinical relapse may be inadequate CD8+ T cell rejuvenation by a limited number of neoadjuvant nivolumab doses or inadequate functional persistency of tumor-specific/cytotoxic T cells due to exhaustion. The former scenario suggests the value of adjuvant anti-PD-1 therapy, whereas the latter implies combinatorial neoadjuvant immunotherapy to thwart or minimize T cell exhaustion.
Prior trials of neoadjuvant chemotherapy in patients with head and neck cancer have failed to show survival benefits.6 The findings reported here, showing significant rates of anti-tumor activity via PD-1 blockade in the neoadjuvant setting, warrant consideration and evaluation of neoadjuvant immunotherapy for improving surgical outcomes long term. Immunotherapy is increasingly being combined in innovative ways with traditional treatment modalities including surgery and radiation, and we provide key evidence of activity for PD-1 immune-checkpoint blockade prior to surgery specifically for patients with oral cavity cancer. This therapeutic approach demonstrates an encouraging 33% ORR and is a safe and feasible modality in the overall management plan. Future investigations to determine predictive biomarkers of response are critical to stratify patients likely to benefit from neoadjuvant anti-PD-1 therapy.
Limitations of study
First, the small number of patients limits the broad applicability of our findings and their long-term implications for similar patients. Additionally, the single-arm design of our study precludes control group comparison. We enrolled a total number of 16 patients but were able to evaluate only 12 patients for efficacy endpoints.
A further limitation is that patient 8, a rapid progressor after only a single dose of nivolumab, was excluded from the primary trial analysis following the original trial protocol. At the time of study registration, there was concern that patients may come off the study early because neoadjuvant therapy is not the standard of care for oral cancer. It was hypothesized at that time that a single dose of nivolumab would not be able to demonstrate a meaningful response, as there were no published studies addressing this. Therefore, the inclusion of this patient in the response outcome would be considered a statistical ad hoc analysis which detracts from the prospective nature of the study.
STAR★Methods
Key resources table
REAGENT or RESOURCE | SOURCE | IDENTIFIER |
---|---|---|
Antibodies | ||
CD4 (SP35) | Cell Marque | Cat# 104R-1: RRID: AB_1516770 |
CD8 (SP16) | Cell Marque | Cat# 108R-1: RRID:AB_2892088 |
Foxp3 (236A/E7) | Abcam | Cat# ab20034: RRID:AB_445284 |
PDL1 (E1L3N) | Cell Signaling | Cat# 13684: RRID:AB_2687655 |
cytokeratin (AE1/AE3) | Dako | Cat# GA05361-2: RRID:AB_2892089 |
Reagent | ||
EDTA buffer (pH 9) | Agilent/Dako | Cat# S2367 |
Citrate buffer (pH 6) | Roche | Cat# 980-223 |
ProLong Gold Antifade Reagent | ThermoFisher | Cat# P36934 |
Opal 480 | Akoya Biosciences | Cat# FP1500001KT: |
Opal 520 | Akoya Biosciences | Cat# FP1487001KT |
Opal 570 | Akoya Biosciences | Cat# FP1488001KT |
Opal 690 | Akoya Biosciences | Cat# FP1497001KT |
Opal 780 | Akoya Biosciences | Cat# FP1501001KT |
Software | ||
Prism | https://www.graphpad.com/scientific-software/prism/ | v9 |
inForm | https://www.akoyabio.com/phenoptics/software/inform-tissue-finder/ | v2.4.10 |
Instruments | ||
Vectra® Polaris Automated Imaging System | Akoya Biosciences | N/A |
Ventana Discovery Ultra Automated Research Stainer | Roche | N/A |
Resource availability
Lead contact
Further information and requests should be directed to the lead contact, David Neskey (neskey@musc.edu).
Materials availability
This study did not generate new unique reagents.
Experimental model and subject details
Patients
Eligible patients were 18-years old or older and had newly diagnosed histologically proven HPV-negative OCSCC. There could be no evidence of distant metastasis at enrollment. See Table 1 for specific patient demographic information. Primary tumors were required to be American Joint Committee on Cancer (AJCC) 7th Edition T stage 2-4a to ensure response to therapy could be accurately assessed clinically and radiographically. Patients were required to be Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1. Key exclusion criteria were T1 or unresectable tumor, prior non-surgical treatment (immunotherapy, chemotherapy including cetuximab, or radiation therapy), and active autoimmune disorder or infectious disease. Study data were collected and managed using REDCap electronic data capture tools hosted at the Medical University of South Carolina.29 REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.
Study design
Conducted at a single institution, this was a single-arm, investigator-initiated study (NCT03021993, Figure 1). A Simon two-stage design was employed,30 with an initial recruitment of 9 patients in stage one and an additional 8 patients in stage 2 (see Study Protocol in Methods S1). Enrolled patients underwent baseline clinical and radiographic evaluation and initial pathologic diagnosis within 14 days of study registration. Clinical evaluation included standard flash or endoscopic photography of the primary tumor. Radiographic analysis included computed tomography (CT) of the head and neck. Radiographic tumor size was defined as the greatest cross-sectional dimension of the tumor on the enrollment imaging study and post-treatment size was the greatest cross-sectional dimension of the tumor on surgical pathology.
Upon enrollment, patients received a 3 mg/kg dose of nivolumab intravenously every two weeks for a total of three doses prior to interval radiographic evaluation between study days 28-35. In the event of disease progression (any increase in tumor burden or symptom progression), patients were taken for definitive surgical resection between days 36-42. In the event of stable disease (no change in tumor burden or symptoms) or response (reduction in tumor burden and symptoms), patients received a 4th dose of nivolumab on day 43+/−1 followed by definitive surgical resection on day 50-56. This dosing schedule was chosen to fit within the standard of care timing between diagnosis and surgical resection to prevent delays in care.
Tumor responses on interval imaging were evaluated by averaging measurements of two radiologists blinded to the patient history and each other’s measurements (SS and MVS) based on Response Evaluation Criteria in Solid Tumors (RECIST) criteria version 1.1.21 Tumor pathologic analysis was performed with standard hematoxylin and eosin (H&E) staining. Resection of the primary tumor and draining lymph nodes along with recommendation for adjuvant radiotherapy or chemotherapy were based on National Comprehensive Cancer Network (NCCN) guidelines.31 All patients were monitored for adverse events (AE) according to the National Cancer Institute Common Terminology Criteria for Adverse Events version 4.0 (2010).
The primary endpoint was objective response rate defined as pathologic complete response + pathologic partial response.21 Partial response was defined as > 30% reduction in maximum tumor diameter in the surgical specimen compared to the single greatest tumor dimension on pretreatment radiographic measurement.22 Secondary endpoints included safety and feasibility. Efficacy analysis was confined to patients who received at least two doses of nivolumab was due to concern that patients would come off study independent of tumor progression or toxicity but because neoadjuvant therapy is not the standard of care for oral cavity squamous cell carcinoma. For more details regarding study design see full clinical trial protocol (Methods S1).
Study oversight
Institutional review board and protocol review committee approvals at the Medical University of South Carolina were obtained prior to initiation of the study. The authors attest to their sole ownership of the trial design, data analysis, and drafting of the manuscript. The study was supported (with drug and funding) by Bristol-Myers Squibb; they played no role in the collection or analysis of the results.
Method details
Tumor histology and multiplex immunohistochemistry
Hematoxylin and eosin (H&E) stained slides were scanned using the Vectra® Polaris Automated Imaging System at 20X (Akoya Biosciences, Marlborough, MA). These slides were assessed for tumor type/differentiation and for cellularity by a surgical pathologist (MSR), and images were captured using Phenochart whole slide contextual viewer software in regions of interest (Akoya Biosciences, Marlborough, MA).
For multiplex immunohistochemistry, unstained slides with 4-5 μm sections of FFPE tissue were deparaffinized and stained using the Roche Ventana Discovery Ultra Automated Research Stainer (Roche Diagnostics, Indianapolis, IN). Heat-induced epitope retrieval was performed in EDTA buffer pH 9 (Cat# S2367 Agilent/Dako Santa Clara, CA) for 32 min at 100°C, and endogenous peroxidase was blocked with a hydrogen peroxide solution after incubation of the first primary antibody. Optimized multiplex immunofluorescence was performed using the OPAL multiplexing method based on Tyramide Signal Amplification. Antibodies used included CD4 (Cell Marque, clone SP35, 1:100), CD8 (Cell Marque, clone SP16, 1:300), FOXP3 (Abcam, clone 236A/E7, 1:600), PD-L1 (Cell Signaling Technologies, clone E1L3N, 1:200), and pan-cytokeratin (Dako, clone AE1/AE3, 1:100), as detailed in the Key Resources table. Fluorescence signals were generated using the following Akoya OPAL TSA fluorophores: OPAL 480, OPAL 520, OPAL 570, OPAL 690, and OPAL 780 (Akoya Biosciences, Marlborough, MA), and nuclei were visualized with DAPI counterstaining. Slides were incubated in citrate buffer pH 6 (Cell Conditioning Solution (CC2) Cat. #980-223, Roche Diagnostics) between each sequential antibody staining step at 90°C to remove the previous primary and secondary antibody complexes. Multiplex-stained slides were mounted with ProLong Gold Antifade Reagent (Cat. # P36934, ThermoFisher) and scanned at a 20x magnification using the Vectra® Polaris Automated Imaging System (Akoya Biosciences, Marlborough, MA). Whole slide scans were reviewed, and images were captured using Phenochart whole slide contextual viewer software in regions of interest (Akoya Biosciences, Marlborough, MA). Spectral unmixing and elimination of autofluorescence was performed using the inForm® Software v2.4.10 (Akoya Biosciences, Marlborough, MA). Captured images were exported in TIFF format.
Quantification and statistical analysis
The full study used a Simon two-stage design with a null hypothesis of 2% for overall response rate and an alternative of 25%. With an alpha of 4% and power of 91%, the null hypothesis was rejected if two or more of 17 patients responded. Response rates were estimated with a 95% confidence interval with statistical inference for shortened phase II study based on the Simon two-stage design.32 A single patient responding in the first 9 was considered sufficient evidence to progress into stage 2 in the absence of untoward AEs. The rate of grade 3-4 AEs was predicted to be < 5%, and the trial would have been stopped if there was strong evidence (a likelihood ratio > 4 based on the binomial distribution) that the rate of grade 3-4 AEs was 25% or higher. Reported p values are two-sided and significant if < 0.05. Continuous variables are compared between response categories using the Kruskal-Wallis test. Continuous bivariate relationships were determined using Spearman’s rank correlation.
Additional resources
Description:https://clinicaltrials.gov/ct2/show/NCT03021993.
Acknowledgments
The authors would like to thank the patients and their families who generously agreed to participate, as well as the numerous providers and support staff who aided in the execution of this trial including Tiffany Wall, DNP; Caitlin Mengler, NP; and Marybeth Chalk, NP. Specific gratitude is extended to the clinical trials office of Hollings Cancer Center: Brittanie Weinerman, Kate Anderton, and Kristina W. Godwin, MHA CCRP. We would also like to thank the members of the Paulos and Hammerbacher labs for the many discussions and assistance with immunology studies: Connor Dwyer, Aubrey Smith, Guillermo Rangel Rivera, Jeff Hammerbacher, Bulent Arman Aksoy, Pinar Aksoy, Eric Czech, and Elinor Gottschalk. This work was supported in part by the Biostatistics and Clinical Trials Office Shared Resource and the Translational Science Laboratory Shared Resource, Hollings Cancer Center, and Medical University of South Carolina (P30 CA138313). Funding included NCI F30 243307, NIH T32 GM08716 and T32 DE017551, and American Head and Neck Society (H.M.K.); Melanoma Research Alliance Dermatology Fellowship and Jonsson Comprehensive Cancer Center post-doctoral fellowship (S.L.); NCI R01 CA208514 and CA175061 (C.M.P.); SC Clinical and Translational Research Institute UL1TR001450, NIDCR K08 DE26542, and Bristol Myers Squibb CA209-831 (D.M.N.); NCI R01 CA176111A1, R21 CA255837-01, and P01 CA168585 and V Foundation for Cancer Research (R.S.L.); and NIDCR R21 029592 (D.M.N. and C.M.P.).
Author contributions
Conceptualization, H.M.K., J.D. Horton, C.M.P., and D.M.N.; methodology, E.G.-M., K.A., and D.M.N.; formal analysis, H.M.K., J.D. Horton, K.A., E.G.-M., C.M.P., and D.M.N.; investigation, H.M.K., J.D. Horton, S.L., J.M.K., M.S.R., S.H.L., Y.X., E.M.G., E.J.L., J.D. Hornig, J.S., S.S., M.V.S., E.C.O., C.D.T., M.J.R., and T.A.D.; resources, E.C.O., C.D.T., and M.J.R.; data curation, H.M.K., J.D. Horton, M.M.W., M.S.R., S.S., M.V.S., E.C.O., C.D.T., and M.J.R.; writing – original draft, H.M.K., J.D. Horton, C.M.P., and D.M.N.; writing – review & editing, all authors; visualization, H.M.K., J.D. Horton, E.C.O., C.M.P., and D.M.N.; supervision, H.M.K., J.M.W., M.R.I.Y., M.P.R., T.A.D., R.S.L., C.M.P., and D.M.N.; funding acquisition, H.M.K., C.M.P., and D.M.N.
Declaration of interests
C.M.P. is the co-founder of Ares Immunotherapy. R.S.L. received research or clinical trial support from Merck, Pfizer, BMS, and OncoSec. D.M.N. received research or clinical trial support from BMS. M.R.I.Y. received research or clinical trial support Merck and BMS.
Inclusion and diversity
We worked to ensure gender balance in the recruitment of human subjects. We worked to ensure ethnic or other types of diversity in the recruitment of human subjects. We worked to ensure that the study questionnaires were prepared in an inclusive way. While citing references scientifically relevant for this work, we also actively worked to promote gender balance in our reference list. The author list of this paper includes contributors from the location where the research was conducted who participated in the data collection, design, analysis, and/or interpretation of the work.
Published: October 19, 2021
Footnotes
Supplemental information can be found online at https://doi.org/10.1016/j.xcrm.2021.100426.
Supplemental information
Data and code availability
All data reported in this paper will be shared by the lead contact upon request. This paper does not report original code. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.
References
- 1.Bray F., Ferlay J., Soerjomataram I., Siegel R.L., Torre L.A., Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 2018;68:394–424. doi: 10.3322/caac.21492. [DOI] [PubMed] [Google Scholar]
- 2.Bernier J., Domenge C., Ozsahin M., Matuszewska K., Lefèbvre J.L., Greiner R.H., Giralt J., Maingon P., Rolland F., Bolla M., European Organization for Research and Treatment of Cancer Trial 22931 Postoperative irradiation with or without concomitant chemotherapy for locally advanced head and neck cancer. N. Engl. J. Med. 2004;350:1945–1952. doi: 10.1056/NEJMoa032641. [DOI] [PubMed] [Google Scholar]
- 3.Cooper J.S., Pajak T.F., Forastiere A.A., Jacobs J., Campbell B.H., Saxman S.B., Kish J.A., Kim H.E., Cmelak A.J., Rotman M., Radiation Therapy Oncology Group 9501/Intergroup Postoperative concurrent radiotherapy and chemotherapy for high-risk squamous-cell carcinoma of the head and neck. N. Engl. J. Med. 2004;350:1937–1944. doi: 10.1056/NEJMoa032646. [DOI] [PubMed] [Google Scholar]
- 4.Koo B.S., Lim Y.C., Lee J.S., Choi E.C. Recurrence and salvage treatment of squamous cell carcinoma of the oral cavity. Oral Oncol. 2006;42:789–794. doi: 10.1016/j.oraloncology.2005.11.016. [DOI] [PubMed] [Google Scholar]
- 5.Kademani D., Bell R.B., Bagheri S., Holmgren E., Dierks E., Potter B., Homer L. Prognostic factors in intraoral squamous cell carcinoma: the influence of histologic grade. J. Oral. Maxillofac. Surg. 2005;63:1599–1605. doi: 10.1016/j.joms.2005.07.011. [DOI] [PubMed] [Google Scholar]
- 6.Ma J., Liu Y., Huang X.L., Zhang Z.Y., Myers J.N., Neskey D.M., Zhong L.P. Induction chemotherapy decreases the rate of distant metastasis in patients with head and neck squamous cell carcinoma but does not improve survival or locoregional control: a meta-analysis. Oral Oncol. 2012;48:1076–1084. doi: 10.1016/j.oraloncology.2012.06.014. [DOI] [PubMed] [Google Scholar]
- 7.Ferris R.L., Blumenschein G., Jr., Fayette J., Guigay J., Colevas A.D., Licitra L., Harrington K., Kasper S., Vokes E.E., Even C. Nivolumab for Recurrent Squamous-Cell Carcinoma of the Head and Neck. N. Engl. J. Med. 2016;375:1856–1867. doi: 10.1056/NEJMoa1602252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Seiwert T.Y., Burtness B., Mehra R., Weiss J., Berger R., Eder J.P., Heath K., McClanahan T., Lunceford J., Gause C. Safety and clinical activity of pembrolizumab for treatment of recurrent or metastatic squamous cell carcinoma of the head and neck (KEYNOTE-012): an open-label, multicentre, phase 1b trial. Lancet Oncol. 2016;17:956–965. doi: 10.1016/S1470-2045(16)30066-3. [DOI] [PubMed] [Google Scholar]
- 9.Mandal R., Şenbabaoğlu Y., Desrichard A., Havel J.J., Dalin M.G., Riaz N., Lee K.W., Ganly I., Hakimi A.A., Chan T.A., Morris L.G. The head and neck cancer immune landscape and its immunotherapeutic implications. JCI Insight. 2016;1:e89829. doi: 10.1172/jci.insight.89829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Martincorena I., Campbell P.J. Somatic mutation in cancer and normal cells. Science. 2015;349:1483–1489. doi: 10.1126/science.aab4082. [DOI] [PubMed] [Google Scholar]
- 11.Cancer Genome Atlas N., Cancer Genome Atlas Network Comprehensive genomic characterization of head and neck squamous cell carcinomas. Nature. 2015;517:576–582. doi: 10.1038/nature14129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Stransky N., Egloff A.M., Tward A.D., Kostic A.D., Cibulskis K., Sivachenko A., Kryukov G.V., Lawrence M.S., Sougnez C., McKenna A. The mutational landscape of head and neck squamous cell carcinoma. Science. 2011;333:1157–1160. doi: 10.1126/science.1208130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Malm I.J., Bruno T.C., Fu J., Zeng Q., Taube J.M., Westra W., Pardoll D., Drake C.G., Kim Y.J. Expression profile and in vitro blockade of programmed death-1 in human papillomavirus-negative head and neck squamous cell carcinoma. Head Neck. 2015;37:1088–1095. doi: 10.1002/hed.23706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hanna G.J., Adkins D.R., Zolkind P., Uppaluri R. Rationale for neoadjuvant immunotherapy in head and neck squamous cell carcinoma. Oral Oncol. 2017;73:65–69. doi: 10.1016/j.oraloncology.2017.08.008. [DOI] [PubMed] [Google Scholar]
- 15.Amaria R.N., Reddy S.M., Tawbi H.A., Davies M.A., Ross M.I., Glitza I.C., Cormier J.N., Lewis C., Hwu W.J., Hanna E. Neoadjuvant immune checkpoint blockade in high-risk resectable melanoma. Nat. Med. 2018;24:1649–1654. doi: 10.1038/s41591-018-0197-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Forde P.M., Chaft J.E., Smith K.N., Anagnostou V., Cottrell T.R., Hellmann M.D., Zahurak M., Yang S.C., Jones D.R., Broderick S. Neoadjuvant PD-1 Blockade in Resectable Lung Cancer. N. Engl. J. Med. 2018;378:1976–1986. doi: 10.1056/NEJMoa1716078. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Uppaluri R., Campbell K.M., Egloff A.M., Zolkind P., Skidmore Z.L., Nussenbaum B., Paniello R.C., Rich J.T., Jackson R., Pipkorn P. Neoadjuvant and Adjuvant Pembrolizumab in Resectable Locally Advanced, Human Papillomavirus-Unrelated Head and Neck Cancer: A Multicenter, Phase II Trial. Clin. Cancer Res. 2020;26:5140–5152. doi: 10.1158/1078-0432.CCR-20-1695. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Schoenfeld J.D., Hanna G.J., Jo V.Y., Rawal B., Chen Y.H., Catalano P.S., Lako A., Ciantra Z., Weirather J.L., Criscitiello S. Neoadjuvant Nivolumab or Nivolumab Plus Ipilimumab in Untreated Oral Cavity Squamous Cell Carcinoma: A Phase 2 Open-Label Randomized Clinical Trial. JAMA Oncol. 2020;6:1563–1570. doi: 10.1001/jamaoncol.2020.2955. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wise-Draper T.M., Takiar V., Mierzwa M.L., Casper K., Palackdharry S., Worden F.P., Old M.O., Dunlap N.E., Kaczmar J.M., Patil Y. Association of pathological response to neoadjuvant pembrolizumab with tumor PD-L1 expression and high disease-free survival (DFS) in patients with resectable, local-regionally advanced, head and neck squamous cell carcinoma (HNSCC) J. Clin. Oncol. 2021;39:6006. [Google Scholar]
- 20.Uppaluri R., Chernock R., Mansour M., Jackson R., Rich J., Pipkorn P., Paniello R.C., Puram S., Zevallos J.P., Annino D.J. Enhanced pathologic tumor response with two cycles of neoadjuvant pembrolizumab in surgically resectable, locally advanced HPV-negative head and neck squamous cell carcinoma (HNSCC) J. Clin. Oncol. 2021;39:6008. [Google Scholar]
- 21.Eisenhauer E.A., Therasse P., Bogaerts J., Schwartz L.H., Sargent D., Ford R., Dancey J., Arbuck S., Gwyther S., Mooney M. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1) Eur. J. Cancer. 2009;45:228–247. doi: 10.1016/j.ejca.2008.10.026. [DOI] [PubMed] [Google Scholar]
- 22.Arredondo J., González I., Baixauli J., Martínez P., Rodríguez J., Pastor C., Ribelles M.J., Sola J.J., Hernández-Lizoain J.L. Tumor response assessment in locally advanced colon cancer after neoadjuvant chemotherapy. J. Gastrointest. Oncol. 2014;5:104–111. doi: 10.3978/j.issn.2078-6891.2014.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Liu S., Knochelmann H.M., Lomeli S.H., Hong A., Richardson M., Yang Z., Lim R.J., Wang Y., Dumitras C., Timmers C. Response and recurrence correlates in individuals treated with neoadjuvant anti-PD-1 therapy for resectable oral cavity squamous cell carcinoma. Cell Rep Med. 2021;2:100411-1–100411-12. doi: 10.1016/j.xcrm.2021.100411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Graboyes E.M., Kompelli A.R., Neskey D.M., Brennan E., Nguyen S., Sterba K.R., Warren G.W., Hughes-Halbert C., Nussenbaum B., Day T.A. Association of Treatment Delays With Survival for Patients With Head and Neck Cancer: A Systematic Review. JAMA Otolaryngol. Head Neck Surg. 2019;145:166–177. doi: 10.1001/jamaoto.2018.2716. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Graboyes E.M., Garrett-Mayer E., Ellis M.A., Sharma A.K., Wahlquist A.E., Lentsch E.J., Nussenbaum B., Day T.A. Effect of time to initiation of postoperative radiation therapy on survival in surgically managed head and neck cancer. Cancer. 2017;123:4841–4850. doi: 10.1002/cncr.30939. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kitajima K., Miyoshi Y., Yamano T., Odawara S., Higuchi T., Yamakado K. Assessment of tumor response to neoadjuvant chemotherapy in patients with breast cancer using MRI and FDG-PET/CT-RECIST 1.1 vs. PERCIST 1.0. Nagoya J. Med. Sci. 2018;80:183–197. doi: 10.18999/nagjms.80.2.183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Chiou V.L., Burotto M. Pseudoprogression and Immune-Related Response in Solid Tumors. J. Clin. Oncol. 2015;33:3541–3543. doi: 10.1200/JCO.2015.61.6870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Feng Z., Niu L.X., Yuan Y., Peng X., Guo C.B. Risk factors and treatment of contralateral neck recurrence for unilateral oral squamous cell carcinoma: a retrospective study of 1482 cases. Oral Oncol. 2014;50:1081–1088. doi: 10.1016/j.oraloncology.2014.08.003. [DOI] [PubMed] [Google Scholar]
- 29.Harris P.A., Taylor R., Thielke R., Payne J., Gonzalez N., Conde J.G. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Simon R. Optimal two-stage designs for phase II clinical trials. Control. Clin. Trials. 1989;10:1–10. doi: 10.1016/0197-2456(89)90015-9. [DOI] [PubMed] [Google Scholar]
- 31.Colevas A.D., Yom S.S., Pfister D.G., Spencer S., Adelstein D., Adkins D., Brizel D.M., Burtness B., Busse P.M., Caudell J.J. NCCN Guidelines Insights: Head and Neck Cancers, Version 1.2018. J. Natl. Compr. Canc. Netw. 2018;16:479–490. doi: 10.6004/jnccn.2018.0026. [DOI] [PubMed] [Google Scholar]
- 32.Zhao J., Yu M., Feng X.P. Statistical inference for extended or shortened phase II studies based on Simon’s two-stage designs. BMC Med. Res. Methodol. 2015;15:48. doi: 10.1186/s12874-015-0039-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All data reported in this paper will be shared by the lead contact upon request. This paper does not report original code. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.