Reliability of quantitative F-18 FDG PET/CT imaging biomarkers for classifying early response to chemoradiotherapy in patients with locally advanced non-small cell lung cancer

Abstract

Purpose of the Report:

We evaluated reliability of F-18 FDG PET imaging biomarkers to classify early response status across observers, scanners, and reconstruction algorithms in support of biologically adaptive radiation therapy for locally advanced non-small cell lung cancer.

Materials and Methods:

Thirty-one patients with unresectable locally advanced NSCLC were prospectively enrolled on a phase II trial (NCT02773238) and underwent F-18 FDG PET on GE-DSTE or GE-DMI PET/CT systems at baseline and during the third week external beam radiation therapy regimens. All PET scans were reconstructed using OSEM; GE-DMI scans were also reconstructed with BSREM-TOF. Primary tumors were contoured by three observers using semi-automatic gradient-based segmentation. SUV_max, SUV_mean, SUV_peak, MTV, and TLG were correlated with mid-therapy multidisciplinary clinical response assessment. Dice similarity of contours and response classification AUCs were evaluated across observers, scanners, and reconstruction algorithms. LASSO logistic regression models were trained on DSTE PET patient data and independently tested on DMI PET patient data.

Results:

Inter-observer variability of PET contours was low for both OSEM and BSREM-TOF reconstructions; intra-observer variability between reconstructions was slightly higher. ΔSUV_peak was the most robust response predictor across observers and image reconstructions. LASSO models consistently selected ΔSUV_peak and ΔMTV as response predictors. Response classification models achieved high cross-validated performance on the DSTE cohort and more variable testing performance on the DMI cohort.

Conclusion:

The variability of FDG PET lesion contours and imaging biomarkers was relatively low across observers, scanners, and reconstructions. Objective mid-treatment PET response assessment may lead to improved precision of biologically adaptive radiation therapy.

Introduction

Early treatment response assessment is critical to guide treatment management decisions within personalized oncology to guide treatment management decisions. Early identification of an inadequate response to therapy may lead to improved outcomes by providing earlier opportunities to terminate ineffective therapies, especially those with toxicities and/or high cost, and transition to potentially more efficacious treatments. In the case of a partial response, there may be an opportunity to augment the current therapy by increasing the dose or adding an additional therapeutic agent. The ability to intervene earlier in the course of therapy is particularly important in patients with locally advanced non-small cell lung cancer (LA-NSCLC). Worldwide, lung cancer has the highest incidence of any cancer and is also the leading cause of cancer deaths ¹. Despite being only the second most common cancer diagnosed in both females and males in the United States, it is responsible for the greatest number of cancer deaths with a five-year overall survival rate of only 19% for all stages combined and 57% for localized disease ².

Radiation therapy is one of the primary treatment options for patients with LA-NSCLC. However, current standard treatment regimens result in limited tumor control with local failures in up to 50% of patients and a 5-year overall survival (OS) of 10–20% ^3–6. Mortality for patients with intra-thoracic disease recurrence is similar to that for patients with distant metastatic disease ^7–9, indicating that lack of local tumor control correlates with worse survival. This is further supported by data from the CHART trial, which demonstrated that a 9% increase in local control (from 20% to 29%) more than doubled the median survival from 10 months to 28 months ¹⁰. Multiple dose-escalation trials have demonstrated improvement in both local control rates and overall survival at 5 years when the radiation dose is increased above the 60 Gy administered with conventional fractionation ^{5, 11}. However, both appropriate patient and target volume selection is paramount as the RTOG 0617 trial showed that a uniform dose increase to 74 Gy across the entire tumor volume in unselected patients actually results in poorer local control and overall survival compared to the standard 60 Gy dose ¹². Therefore, strategies are needed to identify patients at high risk of local failure who will benefit from dose escalation as well as the specific area(s) of tumor to target with therapy intensification.

PET/CT is being utilized more frequently for imaging-based response assessments, as it can supplement anatomical characterization with a functional assessment of tumor biology, which is important as changes in tumor size may lag behind changes in tumor physiology, especially early in therapy ^{13, 14}. Specifically, F-18 fluorodeoxyglucose (F-18 FDG) PET/CT has been adopted for early response assessment during chemoradiotherapy in LA-NSCLC ^{15, 16}. Local recurrences after radiation treatment tend to occur inside tumor regions demonstrating high F-18 FDG avidity on mid-treatment PET scans and as such, high tumor metabolic activity midway through fractionated radiation therapy is associated with poorer clinical outcomes ^17–19. Additionally, there is high spatial correlation between F-18 FDG-avidity and areas of viable tumor allowing high-risk disease to be both identified and targeted ^{20, 21}. Previous studies have utilized F-18 FDG PET/CT to define targets for dose escalation, however, have not selected patients for dose escalation based upon local failure risk ^{17, 22}. Patients at high risk for local failure can be identified as early as 2–3 weeks into therapy using quantitative changes in tumor maximum standardized uptake values (SUV_max) and total lesion glycolysis (TLG), which are also predictive of clinical outcome ^{15, 23}. Earlier response assessment allows for the implementation of dose escalation earlier in the therapy course, possibly leading to improvement of cancer control and clinical outcome.

Response assessment utilizing molecular imaging in research settings can be complex, incorporating multiple parameters and analyses which typically extend well beyond the traditional clinical image-based assessment (i.e. SUV_max). When translating analysis methods to the clinical setting, processes will need to be streamlined to minimize the amount of time and effort required to avoid disruption of the clinical workflow. Ideally, imaging-based metrics derived from research studies will be universally applicable and will be consistent between sites irrespective of scanner manufacturer/model, reconstruction algorithm, analysis software package, observer, etc.

FLARE-RT is an ongoing phase II clinical trial evaluating the utility of mid-treatment F-18 FDG PET/CT tumor response to select non-responding patients for dose escalation while utilizing functional lung avoidance to mitigate toxicity ^24–26. The following work represents a sub-study within this clinical trial with the goal of developing a feasible decision-support system to augment clinical treatment response assessments early in the course of therapy. Here we evaluate the variability of semi-automatically delineated F-18 FDG PET lesion contours and the subsequently derived biomarkers of SUV_max, SUV_mean, SUV_peak, metabolic tumor volume (MTV), and TLG between multiple observers, PET/CT scanners, and reconstruction algorithms. We then determine the agreement and reliability of these various PET-derived metrics with the mid-therapy multidisciplinary clinical response status. Identifying robust imaging biomarkers forms the basis for streamlined objective PET-based response assessment to guide biologically adaptive radiation therapy of LA-NSCLC.

Materials and Methods

Patient eligibility/selection

The study population consisted of patients with pathologically-proven locally advanced unresectable Stage IIB-IIIB non-small cell lung cancer as defined by the 7th edition of the AJCC cancer staging manual ²⁷ who were referred for radiotherapy and concurrent chemotherapy with curative intent as part of the FLARE RT phase II clinical trial (http://clinicaltrials.gov/ Identifier: NCT02773238) at the University of Washington Medical Center and Seattle Cancer Care Alliance at the Fred Hutchinson Cancer Research Center (FIGURE 1). The study was approved by the local institutional review board (CCIRB 9599) and complied with the Health Insurance Portability and Accountability Act (HIPPA). All patients signed an informed consent form. All study procedures were conducted according to US and international standards of Good Clinical Practice (FDA Title 21 part 312 and International Conference on Harmonization guidelines), applicable government regulations, and institutional research policies and procedures.

FIGURE 1:

Flow chart indicating the study workflow from data acquisition through analysis.

Patients enrolled on the FLARE RT trial received a baseline F-18 FDG PET/CT (PET_pre) followed by fractionated x-ray volumetrically modulated arc therapy / intensity modulated radiation therapy, or alternatively proton pencil beam scanning external beam radiation therapy (EBRT) with concurrent platinum-based doublet chemotherapy. A short-interval F-18 FDG PET/CT (PET_mid) was acquired during week 3 of EBRT, after patients received 24 Gy nominal radiation dose. A multidisciplinary clinical response assessment, combining radiographic interpretation, clinical examination, and quantitative measurement was performed after the mid-treatment PET/CT in order to classify patients as responders or non-responders to therapy. Patients who were prospectively classified as responders continued to receive a standard dose of 60 Gy. Patients prospectively classified as non-responders underwent subsequent EBRT dose escalation of 74 Gy total to unresponsive sites of disease ²⁵.

The study population was derived from the first 36 consecutive patients enrolled on the FLARE-RT clinical trial from June 10^th, 2016 to March 1^st, 2019. One patient withdrew from the trial before baseline imaging was performed. Two additional patients were withdrawn from the trial as they were no longer eligible after being upstaged on the baseline PET/CT imaging. A total of 33 patients underwent both PET_pre and PET_mid imaging. One patient was excluded from this study due to the loss of the raw data from PET_pre prior to the creation of the harmonized OSEM reconstruction. One final patient was excluded due to the presence of F-18 FDG-avid post obstructive consolidation and superimposed infection/inflammation immediately adjacent to the primary tumor which precluded adequate reliable identification of the primary disease site on PET and CT imaging. A total of 31 patients were included in this analysis (TABLE 1).

TABLE 1:

Patient Characteristics

		Total (n = 31)		Responders (n = 18)		Non-responders (n = 13)		P-value	DSTE (n = 20)		DMI (n = 11)		P-value
Age	Median, range (years)	62	50–78	65	50–74	63	52–78	0.629*	62	50–75	68	58–78	0.043 *
Gender	Female	16	51.6%	10	55.6%	6	46.2%	0.722†	13	65.0%	3	27.3%	0.066†
	Male	15	48.4%	8	44.4%	7	53.8%		7	35.0%	8	72.7%
Stage	IIB	2	6.5%	2	11.1%	0	0.0%	0.577‡	2	10.0%	0	0.0%	0.345‡
	IIIA	16	51.6%	8	44.4%	8	61.5%		9	45.0%	7	63.6%
	IIIB	10	32.3%	6	33.3%	4	30.8%		6	30.0%	4	36.4%
	Recurrent disease	3	9.7%	2	11.1%	1	7.7%		3	15.0%	0	0.0%
Radiation Therapy	X-rays	14	45.2%	7	38.9%	7	53.8%	0.481†	10	50.0%	4	36.4%	0.707†
	Protons	17	54.8%	11	61.1%	6	46.2%		10	50.0%	7	63.6%
Chemotherapy	Carboplatin/paclitaxel	19	61.3%	10	55.6%	9	69.2%	0.527‡	11	55.0%	8	72.7%	0.276‡
	Cisplatin/etoposide	8	25.8%	6	33.3%	2	15.4%		5	25.0%	3	27.3%
	Pemetrexed-based	4	12.9%	2	11.1%	2	15.4%		4	20.0%	0	0.0%

A P-value ≤ 0.05 is considered to be significant.

^*Mann-Whitney U Test

^†Fisher’s Exact Test

^‡Chi-Square Test

PET/CT imaging

Patients underwent both a baseline and mid-treatment F-18 FDG PET/CT scan on either a Discovery STE (DSTE) or a Discovery MI (DMI) PET/CT camera (GE Healthcare, Waukesha, WI, USA) with CT-based attenuation correction (FIGURE 1). A median dose of 374.4 MBq (10.1 mCi) with a range of 8.0–14.6 mCi (296.7–540.9 MBq) FDG was administered intravenously followed by a 60-minute uptake period before PET acquisition. PET emission data were acquired in 3D mode from the base of skull to midthighs for 5 minutes per bed position on the DSTE scanner (n = 19). Time per bed position was 4 (n = 6) or 2.5 (n = 6) minutes per bed position on the DMI. PET data acquired on the DSTE were reconstructed using an ordered subset expectation maximization (OSEM) algorithm with 34 subsets and 2 iterations onto a 5.47× 5.47 × 3.27 mm³ voxel grid with post-reconstruction Gaussian filtration with a filter cut off of 6 mm. PET data acquired on the DMI were reconstructed using the same OSEM algorithm used on the DSTE as well as with a block sequential regularized expectation maximization reconstruction algorithm incorporating time-of-flight (BSREM-TOF) marketed as “Q.Clear^” (GE Healthcare, Waukesha, WI, USA) with β-value of 350, with all DMI images sampled onto a 2.73 × 2.73 × 2.80 mm³ voxel grids. Imaging data sets included the ‘harmonized OSEM’ reconstructions from both the DSTE and DMI scanners as well as the ‘DMI OSEM’ and ‘DMI BSREM-TOF’ reconstructions.

Lesion contouring

Primary tumor and contiguous involved nodes were contoured on PET_pre and PET_mid images independently by 3 observers (KPH, HMT, SRB) using the MIM 6.8.8™ PET Edge semi-automatic segmentation tool (MIM Software, Cleveland, OH) (FIGURE 1). This tool utilizes gradient-based segmentation, which has been shown to correlate more accurately with lesion volume than fixed threshold segmentation methods ²⁸. Immediately adjacent satellite nodules and contiguously involved lymph nodes were included in the primary lesion contour if they could not be easily distinguished on the PET imaging. Dice indices for inter-observer variability of the contours for each patient, reconstruction, and time-point were calculated by multiplying the volume of intersection between the lesion contours from the observers (A-C) by the number of observers and dividing by the sum total volume of observer A-C contours (InterObserver Dice = [3A ∩ B ∩ C]/[A + B + C]). Dice indices for inter-reconstruction/intra-observer variability were calculated by dividing twice the volume of intersection between the OSEM (O) and BSREM-TOF (T) contours by the sum total volume of the OSEM and BSREM-TOF contours for each patient imaged on the DMI, observer, and time-point (IntraObserver Dice = 2[O ∩ T]/[O + T]).

Quantitative PET metric extraction

All standardized uptake value (SUV) measurements were normalized to body weight. Metrics derived from the PET image data included the SUV_max, SUV_mean, SUV_peak, MTV, and TLG (FIGURE 1). Each metric was extracted from each primary lesion contour on both PET_pre and PET_mid for each reconstruction and observer using MIM 6.8.8™. SUV_peak was determined by measuring FDG uptake in a 1.0 cm³ sphere centered upon the voxel with the highest uptake (SUV_max). In instances in which the lesion contour was smaller than 1.0 cm³ or if the voxel with the highest uptake was less than 0.62 cm (the radius of a 1.0 cm³ sphere) from the edge of the lesion contour in any direction, adjacent non-tumor tissue was included within the 1.0 cm³ sphere for the SUV_peak measurement.

Univariate analysis

Statistical analysis comparing the patient characteristics of age, gender, cancer stage, radiation therapy modality, and concurrent chemotherapy regimen in patients classified as responders vs. those classified as non-responders as well as patients imaged on the DSTE vs. those imaged on the DMI was performed using Origin 2018b (OriginLab Corp., Wellesley Hills, MA, USA) (FIGURE 1). Comparison of patient age was evaluated using a Mann-Whitney U test, comparison of gender and radiation therapy modality were evaluated with a Fisher’s Exact test, and comparison of cancer stage and chemotherapy regimen were evaluated using a Chi-Square test.

Univariate statistical analyses for the PET-derived metrics of SUV_max, SUV_mean, SUV_peak, MTV, and TLG on PET_pre as well as the change in the five PET-derived metrics between PET_pre and PET_mid (ΔSUV_max, ΔSUV_mean, ΔSUV_peak, ΔMTV, and ΔTLG) for each observer and reconstruction were also performed using Origin 2018b (FIGURE 1). Differences in inter-observer variability were evaluated using a Friedman ANOVA while differences in inter-reconstruction/intra-observer variability were evaluated using a Wilcoxon Signed Ranks test. Additionally, a receiver operating characteristics (ROC) curve analysis was performed for each metric, reconstruction, and observer to determine the area under the ROC curve (AUC) and the cutoff value with the highest Youden index as well as its associated sensitivity and specificity.

Multivariate analysis

Multivariate statistical analyses were performed using Origin 2018b (OriginLab Corp., Wellesley Hills, MA, USA) and Orange (University of Ljubljana, Ljubljana, Slovenia). The multivariate analysis utilized a LASSO logistic regression with PET data from the subset of patients imaged on the DSTE PET/CT scanner being used for training with subsequent testing performed on the subset of data acquired on the DMI PET/CT scanner (FIGURE 1). PET_pre metrics and change in metrics between PET_pre and PET_mid were first ranked and filtered based on ANOVA to minimize information loss prior to multivariate model building. The multivariate LASSO logistic regression model performance was estimated both on internal cross validations of the DSTE cohort and external validation on the independent DMI cohort. Internal cross-validation consisted of 100 random duplications of 66%/34% training/testing subsets stratified for PET response status to ensure consistent proportionality, from which ensemble performance of testing subsets was calculated. External validation performance was estimated by fixing the trained LASSO logistic regression coefficients and applying the trained model to the DMI data. This process was repeated for each observer and image reconstruction (harmonized OSEM, DMI BSREM-TOF) combination.

Results

Patient population

A total of 31 patients were included in this analysis of the prospective FLARE trial (TABLE 1). The initial twenty patients (64.5%) were imaged on a DSTE PET/CT scanner (with only an OSEM reconstruction) prior to the installation of a new DMI PET/CT scanner at our institution. The remaining 11 patients (35.5%) were imaged on the DMI with both OSEM and BSREM-TOF reconstructions (FIGURE 2). A multidisciplinary mid-treatment clinical response assessment was performed midway through the radiotherapy treatment course and 18 (58.0%) patients were clinically classified as responders while 13 (42.0%) were classified clinically as non-responders (FIGURE 3). In the subset of patients imaged on the DMI, five (45.5%) were classified as responders and six (54.5%) as non-responders. There was no statistically significant difference between responders and non-responders in terms of age, gender, cancer stage, radiotherapy type, and chemotherapy regimen. When comparing the patients imaged on the DSTE versus those imaged on the DMI, there was a statistically significant difference only in age with a median age of 62 years on the DSTE and 58 years on the DMI (p-value = 0.043). A difference in gender approached, but did not reach, statistical significance.

FIGURE 2: Example images from DMI OSEM and BSREM-TOF reconstructions.

Single transaxial, sagittal, and coronal slices from PET_pre OSEM (A) and BSREM-TOF (B) reconstructions acquired on the DMI for patient FLARE 025. These images provide a visual comparison between the two reconstruction algorithms. Intensity-scale bars on the right indicate absolute activity in SUVs with an upper limit saturation threshold of 10.

FIGURE 3: Examples of a response and a non-response to therapy as defined by the multidisciplinary clinical mid-treatment response assessment.

Single transaxial, sagittal, and coronal slices of OSEM reconstructions acquired on the DSTE. A, B: A clinically classified responder (FLARE 008) demonstrates a ΔSUV_peak decrease of 60% and a ΔMTV decrease of 56% between PET_pre (A) and PET_mid (B). C, D: A clinically classified non-responder (FLARE 010) demonstrates a ΔSUV_peak decrease of 23% and a ΔMTV decrease of 20% between PET_pre (C) and PET_mid (D). Intensity-scale bars on the right indicate absolute activity in SUVs with an upper limit saturation threshold of 10.

Contour variability

All primary lesions were independently contoured in three-dimensions for each patient, reconstruction, and time-point by three observers using the same software-based semi-automated lesion contouring tool (FIGURE 4). Inter-observer variability was determined for both the harmonized OSEM (n=31) and DMI BSREM-TOF (n=11) reconstructions while inter-reconstruction variability was assessed via the intra-observer variability between the DMI OSEM and DMI BSREM-TOF reconstructions (n=11). The inter-observer variability of contours on the harmonized OSEM reconstructions was relatively low with median Dice coefficients of 0.95 on PET_pre and 0.93 on PET_mid (TABLE 2). The median Dice coefficients for the inter-observer variability on the BSREM-TOF reconstructions were similar at 0.92 on PET_pre and 0.93 on PET_mid. Inter-reconstruction/intra-observer variability between DMI OSEM and DMI BSREM-TOF reconstructions (n=11) was somewhat higher with median Dice coefficients ranging from 0.84–0.88 on PET_pre and 0.79–0.89 on PET_mid.

FIGURE 4: Examples of lower and higher inter-observer and intra-observer variability in lesion contouring.

Single transaxial, sagittal, and coronal slices of OSEM and BSREM-TOF PET_pre reconstructions acquired on the DMI. A, B: A patient (FLARE 034) with inter-observer dice indices of 0.97 on the OSEM (A) and 0.98 on BSREM-TOF (B) reconstructions. Inter-reconstruction/intra-observer dice indices are 0.95 for observer 1 (yellow), 0.96 for observer 2 (red), and 0.95 for observer 3 (blue). C, D: A patient (FLARE 025) with inter-observer dice indices of 0.74 on the OSEM (c) and 0.88 on BSREM-TOF (D) reconstructions. Inter-reconstruction/intra-observer dice indices are 0.85 for observer 1 (yellow), 0.85 for observer 2 (red), and 0.88 for observer 3 (blue). Intensity-scale bars on the right indicate absolute activity in SUVs with an upper limit saturation threshold of 10.

TABLE 2:

Lesion Contour Variability

Variability Assessed	Reconstruction / Observer	PETpre		PETmid
		Median Dice Index	Interquartile Range	Median Dice Index	Interquartile Range
Inter-observer	Harmonized OSEM	0.95	0.89–0.98	0.93	0.81–0.97
	DMI BSREM-TOF	0.92	0.88–0.97	0.93	0.71–0.95
Inter-reconstruction	Observer 1	0.84	0.81–0.95	0.82	0.75–0.94
	Observer 2	0.86	0.83–0.95	0.89	0.81–0.92
	Observer 3	0.88	0.84–0.95	0.79	0.70–0.92

PET-derived metrics

The metrics of SUV_max, SUV_mean, SUV_peak, MTV, and TLG were extracted from each lesion contour on both PET_pre and PET_mid for each reconstruction and observer. Inter-observer variability was low with no statistically significant difference for all observers on both the harmonized OSEM (n = 31) and BSREM-TOF (n = 11) reconstructions for all five PET-derived metrics on both PET_pre (TABLE 3) and PET_mid (SUPPLEMENTAL TABLE 1). The variability between reconstructions was higher with a statistically significant difference in intra-observer variability for SUV_max and SUV_peak for all three observers on both PET_pre and PET_mid as well as MTV and TLG for a single observer on PET_mid only.

TABLE 3:

Variability in PET-Derived Metrics at Baseline (PET_pre)

Metric	Variability Assessed	Reconstruction	Observer 1		Observer 2		Observer 3		P-value
			Median	Interquartile Range	Median	Interquartile Range	Median	Interquartile Range
SUVmax	Inter-observer	Harmonized OSEM	11.53	8.71–19.29	11.53	8.71–19.29	11.53	8.71–19.29	1.000*
		DMI BSREM-TOF	14.30	8.55–20.93	14.30	8.55–20.93	14.30	8.55–20.93	1.000*
	Inter-reconstruction	DMI OSEM	13.17	7.62–17.83	13.17	7.62–17.83	13.17	7.62–17.83
		DMI BSREM-TOF	14.30	8.55–20.93	14.30	8.55–20.93	14.30	8.55–20.93
			P-value	0.001 †	P-value	0.001 †	P-value	0.001 †
SUVmean	Inter-observer	Harmonized OSEM	5.98	4.58–8.75	5.95	4.62–8.87	6.13	4.73–8.81	0.171*
		DMI BSREM-TOF	6.42	4.72–8.20	6.05	4.71–8.29	6.33	4.72–8.66	0.494*
	Inter-reconstruction	DMI OSEM	5.33	4.52–8.67	5.65	4.52–8.87	5.49	4.44–8.68
		DMI BSREM-TOF	6.42	4.72–8.20	6.05	4.71–8.29	6.33	4.72–8.66
			P-value	0.814†	P-value	0.577†	P-value	0.781†
SUVpeak	Inter-observer	Harmonized OSEM	9.62	7.10–14.34	9.62	7.10–14.33	9.62	7.09–14.33	0.631*
		DMI BSREM-TOF	10.13	7.22–14.05	10.13	7.22–14.06	10.13	7.22–14.06	0.815*
	Inter-reconstruction	DMI OSEM	9.10	6.72–14.34	9.11	6.71–14.33	9.10	6.71–14.33
		DMI BSREM-TOF	10.13	7.22–14.05	10.13	7.22–14.06	10.13	7.22–14.06
			P-value	0.007 †	P-value	0.032 †	P-value	0.007 †
MTV	Inter-observer	Harmonized OSEM	28.30	9.73–100.05	28.53	8.38–109.61	28.52	9.43–100.14	0.124*
		DMI BSREM-TOF	17.67	8.92–114.62	37.77	9.83–113.86	37.92	9.89–113.36	0.529*
	Inter-reconstruction	DMI OSEM	36.00	8.40–100.05	31.02	8.38–109.61	35.70	7.99–109.27
		DMI BSREM-TOF	17.67	8.92–114.62	37.77	9.83–113.86	37.92	9.89–113.36
			P-value	0.320†	P-value	0.320†	P-value	0.067†
TLG	Inter-observer	Harmonized OSEM	216.41	46.98–753.42	196.73	44.43–755.10	200.90	44.76–750.42	0.078*
		DMI BSREM-TOF	83.30	45.73–759.89	90.42	51.35–762.09	94.46	49.79–761.66	0.761*
	Inter-reconstruction	DMI OSEM	118.72	46.98–753.42	79.83	45.00–755.10	90.98	44.76–750.42
		DMI BSREM-TOF	83.30	45.73–759.89	90.42	51.35–762.09	94.46	49.79–761.66
			P-value	0.365†	P-value	0.083†	P-value	0.175†

A P-value ≤ 0.05 is considered to be significant.

^*Friedman ANOVA

^†Wilcoxon Signed Ranks Test

Additionally, the relative change between PET_pre and PET_mid was calculated for each metric (ΔSUV_max, ΔSUV_mean, ΔSUV_peak, ΔMTV, and ΔTLG). Again, the inter-observer variability was low for both the harmonized OSEM and DMI BSREM-TOF with no statistically significant difference for all five Δmetrics for all three observers (TABLE 4). The intra-observer/inter-reconstruction variability was modestly higher with a statistically significant difference for ΔSUV_peak for all three observers and a statistically significant difference in ΔTLG for a single observer (p-value of 0.014 vs. 0.278 for each of the other two observers).

TABLE 4:

Variability in the Change in PET-Derived Metrics Between PET_pre and PET_mid

Metric	Variability Assessed	Reconstruction	Observer 1		Observer 2		Observer 3
			Median*	Interquartile Range*	Median*	Interquartile Range*	Median*	Interquartile Range*	P-value
ΔSUVmax	Inter-observer	Harmonized OSEM	28.26%	18.82–38.47%	28.26%	18.82–38.47%	28.26%	18.82–38.47%	0.976†
		DMI BSREM-TOF	25.87%	20.49–49.36%	25.87%	20.49–49.36%	25.87%	20.49–49.36%	1.000†
	Inter-reconstruction	DMI OSEM	28.26%	21.40–42.63%	28.26%	21.40–42.63%	28.26%	21.40–42.63%
		DMI BSREM-TOF	25.87%	20.49–49.36%	25.87%	20.49–49.36%	25.87%	20.49–49.36%
			P-value	0.520‡	P-value	0.520‡	P-value	0.520‡
ΔSUVmean	Inter-observer	Harmonized OSEM	26.65%	15.65–38.98%	28.75%	15.91–38.24%	27.08%	16.68–38.04%	0.657†
		DMI BSREM-TOF	24.62%	14.39–41.18%	19.02%	12.38–43.91%	25.29%	12.50–51.29%	0.761†
	Inter-reconstruction	DMI OSEM	25.52%	12.74–48.84%	28.75%	18.26–31.31%	26.47%	12.84–49.55%
		DMI BSREM-TOF	24.62%	14.39–41.18%	19.02%	12.38–43.91%	25.29%	12.50–51.29%
			P-value	1.000‡	P-value	1.000‡	P-value	0.765‡
ΔSUVpeak	Inter-observer	Harmonized OSEM	30.73%	22.36–42.77%	30.64%	22.36–42.77%	30.73%	22.36–42.77%	0.508†
		DMI BSREM-TOF	28.34%	18.15–41.09%	28.44%	18.15–40.26%	28.34%	23.53–41.20%	0.441†
	Inter-reconstruction	DMI OSEM	28.73%	24.39–44.07%	28.73%	24.39–44.10%	28.73%	24.39–44.03%
		DMI BSREM-TOF	28.34%	18.15–41.09%	28.44%	18.15–40.26%	28.34%	23.53–41.20%
			P-value	0.014‡	P-value	0.001‡	P-value	0.014‡
ΔMTV	Inter-observer	Harmonized OSEM	26.74%	4.82–42.67%	31.05%	12.65–39.07%	31.94%	13.51–46.13%	0.159†
		DMI BSREM-TOF	21.94%	5.09–46.37%	20.97%	(−)4.53–44.23%	17.52%	(−)5.22–50.38%	0.913†
	Inter-reconstruction	DMI OSEM	18.36%	(−)9.88–50.10%	24.51%	1.48–59.95%	20.41%	13.51–46.13%
		DMI BSREM-TOF	21.94%	5.09–46.37%	20.97%	(−)4.53–44.23%	17.52%	(−)5.22–50.38%
			P-value	0.577‡	P-value	0.520‡	P-value	0.054‡
ΔTLG	Inter-observer	Harmonized OSEM	49.67%	23.55–63.41%	49.13%	25.03–60.77%	50.57%	33.72–67.14%	0.067†
		DMI BSREM-TOF	43.17%	28.72–70.30%	38.09%	31.24–70.55%	40.94%	25.63–66.53%	0.695†
	Inter-reconstruction	DMI OSEM	41.81%	35.38–66.59%	46.55%	29.88–68.01%	43.72%	34.72–67.14%
		DMI BSREM-TOF	43.17%	28.72–70.30%	38.09%	31.24–70.55%	40.94%	25.63–66.53%
			P-value	0.278‡	P-value	0.278‡	P-value	0.014 ‡

A P-value ≤ 0.05 is considered to be significant.

^*percentage decrease from PET_pre to PET_mid

^† Friedman ANOVA

^‡Wilcoxon Signed Ranks Test

Univariate Analysis

A univariate analysis was then performed to determine the correlation of each PET-derived metric with the prospectively determined mid-treatment multidisciplinary clinical response assessment. Overall, the PET-derived metrics at individual time points (i.e., PET_pre or PET_mid) were not strongly associated with the clinical response status with AUC p-values ranging from 0.098–0.107 and 0.105–0.113 on the harmonized OSEM reconstructions on PET_pre and PET_mid, respectively. Similar results were observed on the subset of patients imaged on the DMI using the BSREM-TOF reconstruction with AUC p-values ranging from 0.126–0.184 and 0.153–0.194 on PET_pre and PET_mid, respectively compared to 0.126–0.177 and 0.143–0.188 on the PET_pre and PET_mid DMI OSEM reconstructions, respectively.

Relative changes in imaging biomarkers between PET_pre and PET_mid were more strongly associated with response status than imaging biomarkers at individual time points (TABLE 5). ΔSUV_peak was the most robust response predictor across observers and image reconstructions with AUCs ranging from 0.936–1.000 (p-values: <0.001–0.006) and a Youden cutoff of a 29.5–31.2% decrease yielding a sensitivity of 100% and a specificity range of 83–100%. The second-most robust predictor of response was ΔSUV_max with AUCs ranging from 0.897–1.000 (p-values: <0.001–0.011) and a Youden cutoff of a 28.5–35.3% decrease yielding a sensitivity of 100% and a specificity range of 72.2–100%.

TABLE 5:

PET-Derived Metrics ROC Analysis

Metric	Reconstruction	Observer 1				Observer 2				Observer 3
		Youden Cutoff*	Sensitivity	Specificity	AUC	Youden Cutoff*	Sensitivity	Specificity	AUC	Youden Cutoff*	Sensitivity	Specificity	AUC
ΔSUVmax	Harmonized OSEM	35.3%	100.0%	72.2%	0.897	35.3%	100.0%	72.2%	0.897	35.3%	100.0%	72.2%	0.897
	DMI BSREM-TOF	28.5%	100.0%	80.0%	0.967	28.5%	100.0%	80.0%	0.967	28.5%	100.0%	80.0%	0.967
	DMI OSEM	29.4%	100.0%	100.0%	1.000	29.4%	100.0%	100.0%	1.000	29.4%	100.0%	100.0%	1.000
ΔSUVmean	Harmonized OSEM	31.5%	100.0%	72.2%	0.889	23.3%	76.9%	83.3%	0.850	31.0%	100.0%	66.7%	0.876
	DMI BSREM-TOF	32.8%	100.0%	80.0%	0.867	37.6%	100.0%	60.0%	0.733	30.6%	100.0%	80.0%	0.900
	DMI OSEM	37.9%	100.0%	80.0%	0.933	30.8%	100.0%	60.0%	0.900	32.9%	100.0%	80.0%	0.900
ΔSUVpeak	Harmonized OSEM	31.2%	100.0%	83.3%	0.936	31.2%	100.0%	83.3%	0.936	31.2%	100.0%	83.3%	0.936
	DMI BSREM-TOF	30.6%	100.0%	100.0%	1.000	29.5%	100.0%	100.0%	1.000	30.7%	100.0%	100.0%	1.000
	DMI OSEM	30.9%	100.0%	100.0%	1.000	30.9%	100.0%	100.0%	1.000	30.9%	100.0%	100.0%	1.000
ΔMTV	Harmonized OSEM	26.2%	76.9%	72.2%	0.697	23.7%	76.9%	88.9%	0.786	20.9%	76.9%	77.8%	0.774
	DMI BSREM-TOF	31.4%	66.7%	60.0%	0.567	40.1%	83.3%	60.0%	0.600	24.0%	66.7%	60.0%	0.533
	DMI OSEM	24.9%	66.7%	60.0%	0.467	23.7%	66.7%	80.0%	0.633	19.7%	66.7%	80.0%	0.667
ΔTLG	Harmonized OSEM	56.6%	100.0%	66.7%	0.825	52.1%	100.0%	72.2%	0.880	54.4%	100.0%	72.2%	0.889
	DMI BSREM-TOF	60.6%	100.0%	60.0%	0.767	59.0%	100.0%	60.0%	0.733	56.9%	100.0%	60.0%	0.700
	DMI OSEM	60.9%	100.0%	60.0%	0.667	59.5%	100.0%	60.0%	0.733	60.0%	100.0%	60.0%	0.800

^*Percentage decrease from PET_pre to PET_mid

Multivariate Analysis

Following dimensionality reduction, only changes in metrics between PET_pre and PET_mid were retained for model building. Multivariate LASSO logistic regression consistently selected ΔSUV_peak and ΔMTV as response predictors in the final model. Multivariate response classification models achieved high internally cross-validated performance (AUC median 0.91, range 0.88–0.92) in the DSTE cohort and more variable externally validated testing performance in the DMI cohort when using harmonized OSEM (AUC median 0.87, range 0.73–0.93) or BSREM-TOF reconstructions (AUC median 0.86, range 0.75–0.92).

Discussion

Our study reveals that the variability of semi-automatically delineated F-18 FDG PET lesion contours and the subsequently derived biomarkers of SUV_max, SUV_mean, SUV_peak, MTV, and TLG is relatively low across multiple observers, PET/CT scanners, and reconstruction algorithms. The relative change in PET-derived metrics between baseline and mid-therapy scans is more accurate at predicting response status than absolute values at either time point, which has also been demonstrated recently for predicting overall survival ²⁹. Our analysis indicates that ΔSUV_peak is the most robust predictor of early treatment response for all observers and reconstructions with a Youden cutoff decrease of 29.5–31.2%. This correlates well with the PERCIST 1.0 response assessment criteria, which define a partial metabolic response as a ≥30% decrease in SUL_peak, the lean body mass-corrected SUV ³⁰. One notable difference between our study and the PERCIST 1.0 criteria is that we measured F-18 FDG uptake normalized to total body weight/mass (SUV) instead of lean body mass (SUL), as this is the clinical reporting standard at our institution. As normal adipose tissue demonstrates minimal F-18 FDG uptake, many consider SUL to be more accurate across patients, especially when calculating the absolute change between time points. However, given minimal change in body weight between the baseline and mid-therapy scans, SUV and SUL will yield similar relative percent changes even when their absolute values differ. This may explain why our experimentally determined Youden cutoff for relative change in SUV_peak matches the PERCIST 1.0 criteria so closely, as well as provide some insight into our observation that the relative changes in the imaging-based biomarkers were stronger predictors of response than the absolute values.

Our multivariate analysis selected both ΔSUV_peak and ΔMTV as the best predictors of response when assessed at mid-therapy, suggesting that volumetric assessment provides complementary information to measures of tumor metabolic activity. In fact, multiple studies utilizing mid-radiotherapy F-18 FDG PET/CT in NSCLC have demonstrated increased prognostic value of F-18 FDG PET-derived tumor volumetric data. One retrospective study found that a decrease in MTV of >15% between the mid- and post-treatment scans was associated with a better overall survival ³¹. Another demonstrated that higher absolute values of both MTV and TLG on mid-therapy scans were associated with higher risk of local recurrence whereas the ΔSUV_max between the baseline and mid-treatment scans correlated with the risk of developing regional or distant disease ¹⁶. Pooled data from four prospective studies determined that changes in the volumetric measures of ΔMTV and ΔTLG between baseline and mid-RT PET/CT studies were associated with overall survival when using a univariate classification, however, only ΔTLG reached statistical significance on multivariate analysis ²⁹.

PET/CT imaging was performed at two clinical sites equipped with identical PET/CT scanners using identical imaging protocols and reconstruction parameters. Both sites installed new PET/CT systems that offered a more sophisticated BSREM reconstruction algorithm and time-of-flight (TOF) capability in addition to OSEM after the trial had accrued several patients. BSREM is a Bayesian penalized likelihood reconstruction algorithm which subtracts noise with each iteration to allow reiteration to continue until there is near convergence, resulting in improved count recovery and more accurate SUV_max measurements compared to OSEM, especially in small lesions ³². PET data acquired on the new systems were reconstructed with a “harmonized” OSEM using identical parameters to those used on the earlier studies as well as a BSREM reconstruction algorithm with TOF. To ensure consistency, all patients had both their pre- and mid-therapy scans acquired on the same scanner. A recent study by Messerli et al. analyzing single imaging time point FDG PET restaging scans found a statistically significant difference in the SUV_max extracted from OSEM-TOF and BSREM-TOF reconstructions on NSCLC with SUV_max measured on BSREM reconstructions being up to 34.8% higher depending on the β-value used and the tumor histology ³³. Our data also demonstrate a statistically significant difference in absolute SUV_max between reconstructions; however, there was no statistically significant difference in ΔSUV_max between OSEM and BSREM-TOF reconstructions. As the relative change is a ratio, any differences in absolute values between different reconstructions are mitigated, provided the same reconstruction is used for both time points.

It is important to note that this study has its limitations. First of all, the parent clinical trial is still actively accruing patients, so our study population is relatively small with 31 evaluable patients. The limited number of patients is further split between two different PET/CT systems However, despite the small sample size our results held up to robust statistical analyses. Another caveat is that our gold standard for early treatment response classification was a prospectively determined multidisciplinary clinical treatment response assessment midway through the six-week radiotherapy rather than the overall post-treatment response or long-term follow-up. At the time of submission, the FLARE RT trial is currently open and accruing patients. We cannot yet determine the concordance of the early response assessments with the post-treatment response assessments at the completion of therapy or long-term progression-free and/or overall survival. We plan to continue this work by including analyses of the post-therapy F-18 FDG PET studies in the future. The inclusion of post-treatment PET data will allow us to correlate the early mid-treatment response with the post-treatment response and will allow us to compare the relative change between baseline, mid-therapy, and post-therapy F-18 FDG PET studies.

Treatment response assessments utilizing PET/CT can provide a functional assessment of tumor biology in addition to an anatomical characterization of tumor size ^{13, 14}. However, in order to successfully integrate research-derived analysis methods into the standard clinical response assessment, analysis protocols will need to be streamlined to minimize their impact on the clinical workflow. Additionally, image-based metrics need to be universally applicable so that they will yield consistent results between sites irrespective of scanner technology, reconstruction algorithm, analysis software package, observer, etc.; and harmonization initiatives are underway in Europe ³⁴, Japan ³⁵, and the United States ^{36, 37}. Here we demonstrate that semi-automatically derived F-18 FDG PET imaging biomarkers can be used to consistently classify response status across multiple observers, PET/CT scanners with dissimilar detector technology, and reconstruction algorithms in support of biologically adaptive radiation therapy for LA-NSCLC. Importantly, this early response assessment was performed during the course of radiotherapy. Unlike the traditional post-therapy response assessment, an accurate short-interval on-therapy response assessment such as this will provide an opportunity to intervene early to boost radiation dose to sites of non-responsive disease during therapy to maximize the chance of adequate local control leading to improved outcomes.

Conclusion

The variability of semi-automatically delineated F-18 FDG PET lesion contours and the subsequently derived biomarkers of SUV_max, SUV_mean, SUV_peak, MTV, and TLG is relatively low across multiple observers, PET/CT scanners, and reconstruction algorithms. The relative change in SUV_peak on interval PET/CT imaging yields reliable early response classification across observers, scanners, and reconstruction algorithms for patients with LA-NSCLC undergoing concurrent chemoradiotherapy, with threshold response that aligns with PERCIST 1.0 criteria. Multivariate models consistently selected the combination of ΔSUV_peak and ΔMTV as providing the best early treatment response classification. When applied to early response assessment, these findings may improve the precision of biologically adaptive radiation therapy. This approach could be utilized to develop a standardized early response assessment in LA-NSCLC patients receiving concurrent chemoradiotherapy and potentially extended to other cohorts receiving alternate therapies as well as in the setting of other FDG-avid malignancies.