RETRACTED ARTICLE: Inter- and intraobserver agreement of the quantitative assessment of [^99mTc]-labelled anti-programmed death-ligand 1 (PD-L1) SPECT/CT in non-small cell lung cancer

Abstract

Purpose

Checkpoint inhibition therapy using monoclonal antibodies against programmed cell death protein 1 (PD-1) or its ligand (PD-L1) is now standard management of non-small cell lung cancer (NSCLC). PD-L1 expression is a validated and approved prognostic and predictive biomarker for anti-PD-1/PD-L1 therapy. Technetium-99 m [^99mTc]-labelled anti-PD-L1 single-domain antibody (NM-01) SPECT/CT quantification correlates with PD-L1 expression in NSCLC, presenting an opportunity for non-invasive assessment. The aim of this study was to determine the inter- and intraobserver agreement of the quantitative assessment of [^99mTc]NM-01 SPECT/CT in NSCLC.

Methods

[^99mTc]NM-01 SPECT/CT studies of 21 consecutive NSCLC participants imaged for the evaluation of PD-L1 expression were analysed. Three independent observers measured maximum counts in a tumour region of interest (ROI_max) of primary lung, metastatic lesions and normal tissue references of both 1 and 2 h post-injection (n = 42) anonymised studies using a manual technique. Intraclass correlation coefficients (ICC) were calculated, and Bland–Altman plot analysis was performed to determine inter- and intraobserver agreement.

Results

Intraclass correlation of primary lung tumour-to-blood pool (T:BP; ICC 0.83, 95% CI 0.73–0.90) and lymph node metastasis-to-blood pool (LN:BP; ICC 0.87, 0.81–0.92) measures of [^99mTc]NM-01 uptake was good to excellent between observers. Freehand ROI_max of T (ICC 0.94), LN (ICC 0.97), liver (ICC 0.97) and BP (ICC 0.90) reference tissues also demonstrated excellent interobserver agreement. ROI_max scoring of healthy lung demonstrated moderate to excellent interobserver agreement (ICC 0.84) and improved when measured consistently at the level of the aortic arch (ICC 0.89). Manual ROI_max re-scoring of T, LN, T:BP and LN:BP using [^99mTc]NM-01 SPECT/CT following a 42-day interval was consistent with excellent intraobserver agreement (ICC range 0.95–0.97).

Conclusion

Good to excellent inter- and intraobserver agreement of the quantitative assessment of [^99mTc]NM-01 SPECT/CT in NSCLC was demonstrated in this study, including T:BP which has been shown to correlate with PD-L1 status. [^99mTc]NM-01 SPECT/CT has the potential to reliably and non-invasively assess PD-L1 expression.

Clinical trial registration

ClinicalTrials.gov identifier no. NCT02978196. Registered 30th November 2016.

Background

Lung cancer is the most commonly diagnosed cancer globally and a leading cause of mortality with over 1.7 million deaths in 2018 alone [1]. Therapeutic molecular-targeting agents have resulted in significant improvements in progression-free and overall survival in advanced non-small cell lung cancer (NSCLC); however, targetable genetic aberrations represent only a small proportion of cases [2]. The introduction of monoclonal antibodies targeting immune checkpoint molecules including programmed cell death protein 1 (PD-1) and its ligand (PD-L1) has revolutionised the treatment paradigm of NSCLC. An important mechanism of immune escape involves the upregulation of co-inhibitory molecule PD-L1 by tumour cells, which on interaction with PD-1, expressed by effector T cells, lead to their dysfunction. Anti-PD-1/PD-L1 therapy improves median overall survival in advanced NSCLC in both first- and second-line settings compared to standard cytotoxic chemotherapy, with durable responses seen in around 20% [3–6].

PD-L1 expression determined by immunohistochemistry (IHC) is a widely validated biomarker correlating with anti-PD-1/PD-L1 therapeutic response and survival [4–7]. Despite this correlation, up to 10% of patients deemed ‘non-expressers’ by IHC respond to anti-PD-1/PD-L1 therapy [4]. Heterogeneity of PD-L1 expression both within and between tumours is well reported, as are changes over time particularly following exposure to anti-cancer therapies [8, 9]. Considering that multiple or serial biopsies are impractical and associated with increased risk to individual patients, this temporospatial heterogeneity presents a particular challenge as needle biopsy only samples a small area of the tumour. Additionally, there are multiple PD-L1 assays available which may assess PD-L1 expression on tumour or infiltrative immune cells alone or in combination [10]. Considering a potential for false negative results with IHC and the limitations described, non-invasive imaging techniques present a potential solution and opportunity to improve the predictive value of PD-L1 assessment.

NM-01 is a camelid single-domain antibody against PD-L1 that when radiolabelled with technetium-99 m ([^99mTc]) can be detected by single-photon emission computed tomography (SPECT). Recently, we have reported results from a first-in-human study of [^99mTc]NM-01 that demonstrated both safety and acceptable dosimetry in the first 16 recruited participants with NSCLC [11]. SPECT/computed tomography (CT) scans were obtained 1 and 2 h following [^99mTc]NM-01 injection with primary tumour-to-blood pool ratio (T:BP) assessment correlating with PD-L1 expression determined by IHC. Additionally, uptake was demonstrated in nodal and bone metastases with heterogeneity of expression in 30% of cases. This novel single-domain antibody presents an opportunity for the non-invasive total tumoural assessment of PD-L1 that could help clinicians better stratify patients to receive the most appropriate anti-cancer therapy at the right time in their disease course. Our hypothesis was that quantitative measurement of PD-L1 expression using [^99mTc]NM-01 SPECT/CT is consistent and reproducible between and within observers. The aim of this study was to determine the reproducibility of and agreement between experienced and less experienced observers within a cohort of patients with NSCLC.

Methods

Participants aged between 18 and 75 years with histologically confirmed, untreated NSCLC and an Eastern Cooperative Oncology Group (ECOG) performance score of 1 or less were eligible to participate and undergo [^99mTc]NM-01 SPECT/CT. Exclusion criteria included pregnant or lactating females, severe infection and inability to provide biopsy sample for assessment of PD-L1. The study was registered with ClinicalTrials.gov identifier no. NCT02978196. Ethics approval was obtained from Shanghai General Hospital Ethics Committee (approval no. 2016KY220), and all enrolled participants provided written informed consent [11].

SPECT/CT protocol

SPECT/CT examinations were performed on a GE Discovery NM670 SPECT/CT scanner (GE Healthcare; NY, USA). Participants were administered an intravenous bolus of [^99mTc]NM-01 (3.8–8.4 MBq/kg) equivalent to 100 μg (n = 18; 1.65 ± 0.46 μg/kg; range 1.19–2.11 μg/kg) and (9.1–10.4 MBq/kg) equivalent to 400 μg (n = 3; 5.81 ± 0.25 μg/kg; range 5.56–6.06 μg/kg). Participants were asked to drink 300–500 mL water post-injection and void bladder prior to imaging. Following an uptake time of 60 min, a low-dose CT was performed for anatomical correlation and attenuation correction. SPECT imaging, focusing on primary tumour (thorax) and site(s) of suspected metastases, was performed with the patient supine at 1 and 2 h post-injection at 10 cm/slice/min. Scans were performed as previously described using low-energy high-resolution collimators with a ± 10% energy window centred around 140 keV in a 64 × 64 matrix for tomographic images [11]. A 10% energy window centred at 120 keV was also used for tomographic image acquisition for scatter correction. SPECT was performed over 360° in 60 frames per rotation with 20-s acquisition per frame. Images were reconstructed using OSEM iterative reconstruction (2 iterations, 10 subsets) at a matrix size of 128 × 128 using scatter correction.

Image analysis

Images were reviewed by three independent observers blinded to patient details and each other’s assessments using Hermes GOLD™ (Hermes Medical Solutions; Stockholm, Sweden). The observers included one nuclear medicine physician, one nuclear medicine clinical fellow in training and one oncology clinical fellow PhD student with 30, 3 and 1 years of experience in nuclear medicine image analysis, respectively. Regions of interest including primary tumour and metastatic lesions, including lymph nodes and normal tissue references (lung, liver and blood pool), were identified with CT correlation. Using a freehand manual technique, the maximum count for regions of interest (ROI_max) was recorded from 1- and 2-h SPECT images (n = 42) for each patient. ROI_max was chosen as ROI_mean could be affected by differences in the manual segmentation and is more likely to be affected by the partial volume effect. In addition, the method using ROI_max was previously shown to correlate with IHC [11]. Freehand ROI_max was recorded for normal lung in the right upper lobe (or contralateral upper lobe if pathology present) for calculation of tumour-to-lung (T:L) ratio and for blood pool within the aortic arch for calculation of tumour-to-blood pool (T:BP) ratio. To evaluate if rule-based approaches improved consistency of scoring of normal tissue references, ROI_max was also recorded using a standardised 3-cm-diameter sphere for normal lung at the level of the aortic arch and carina, and the liver at the level of the gastroesophageal junction (GOJ) on axial view. Examples of image analysis are provided in Fig. 1. To determine intraobserver agreement, the two independent observers with least experience (one nuclear medicine and one oncology clinical fellow) repeated their calculations for all measured regions blind to their initial measurements following a 42-day period.

Fig. 1.

Image analysis using ROI_max scoring of [^99mTc]NM-01 SPECT/CT of: primary left lower lobe tumour, IHC PD-L1 65% (a), freehand; unaffected lung tissue freehand (b) and using a 3-cm sphere at level of the aortic arch (c); blood pool reference tissue (d); liver reference tissue freehand (e) and using a 3-cm sphere at the axial level of the gastroesophageal junction (f)

Statistical analysis

Intraclass correlation coefficient (ICC) is a reliability index that represents both the degree of correlation but also the agreement between measurements. A full description of their application and formulae is described in the literature [12]. ICC and their 95% confidence intervals (CIs) were calculated using a two-way random consistent model, to determine interobserver agreement between all three observers. ICC and their 95% CI were calculated using a two-way mixed effects absolute agreement model, to determine intraobserver agreement for two observers. ICC values range from 0 to 1, where the values less than 0.5 indicate poor agreement, 0.5–0.75 moderate, 0.75–0.9 good, and greater than 0.9, i.e. close to 1, represent excellent agreement [12]. As the ICC obtained is an estimated value of the true ICC, the levels of agreements are defined by their 95% confidence intervals. Bland–Altman plots and their 95% limits of agreement were used to determine the agreement between observers and their repeat measurements for logarithm-transformed T:BP and LN:BP scores. Linear regression of Bland–Altman plots was performed to determine the β coefficient of the mean difference and demonstrate any proportional bias (where p < 0.05 is significant). Statistical analysis was performed using IBM SPSS Statistics for Windows, version 26.0 (Armonk, NY: IBM Corp.).

Results

Participant characteristics

Participants were recruited to the study between March 2018 and April 2019 (n = 21). The median age was 65 years (range 36–75 years); all were of Asian ethnicity. All had a histologically confirmed diagnosis of NSCLC (adenocarcinoma n = 10, squamous cell carcinoma n = 11) with 9 of 21 participants having metastatic disease. A full summary of participant characteristics is provided in Table 1.

Table 1.

Participant demographics

Patient no.	Age (yrs)	Sex	ECOG score	Tumour histology	TNM staging	Primary tumour size (mm)	Disease site(s)	PD-L1 expression (%)	Administered radioactivity (MBq/kg)
1	49	M	1	Adenocarcinoma	T4N3M1	37 × 27	RUL, multiple mediastinal LNs, renal	–	4.84
2	75	M	1	Squamous cell carcinoma	T3N3M1	44 × 48	LLL, multiple mediastinal LNs and chest wall	20	6.79
3	75	M	1	Squamous cell carcinoma	T2bN3M0	55 × 46	LLL, localised LNs	0	7.50
4	65	M	0	Adenocarcinoma	T2bN3M1	48 × 42	LUL, bilateral lung and bone	0	9.12
5	57	M	0	Squamous cell carcinoma	T2N2M0	32 × 35	RUL, multiple mediastinal LNs	55	10.38
6	65	M	0	Squamous cell carcinoma	T4N3M0	30 × 58	RUL, multiple mediastinal LNs	3	9.63
7	75	F	0	Adenocarcinoma	T4N3M1	38 × 28	RUL, multiple bilateral lung	–	4.81
8	52	F	0	Squamous cell carcinoma	T2aN0M0	33 × 23	LUL	0	7.25
9	36	F	1	Adenocarcinoma	T2aN2M1	45 × 35	LLL, multiple mediastinal LNs and multiple bone	1	7.59
10	47	F	0	Adenocarcinoma	T3N1M0	42 × 35	LUL, localised LNs	50	6.56
11	51	M	0	Squamous cell carcinoma	T2aN3M0	47 × 35	LLL, mediastinal LNs	2	3.77
12	72	M	1	Adenocarcinoma	T2bN3M1	47 × 35	LLL, multiple mediastinal LNs	–	6.54
13	55	M	0	Squamous cell carcinoma	T4N0M1c	71 × 78	LUL, liver	85	8.41
14	69	M	0	Squamous cell carcinoma	T3N1M0	20 × 28	LLL, mediastinal LNs	10	6.59
15	71	F	1	Squamous cell carcinoma	T4N1M1a	78 × 95	LUL, mediastinal and distant LNs	–	6.02
16	60	M	0	Adenocarcinoma	T4N3M1a	93 × 75	RUL, multiple bilateral mediastinal LNs, chest wall, renal	2	5.58
17	70	M	0	Adenocarcinoma	T3N1M0	66 × 44	LLL, mediastinal LNs	65	5.33
18	41	F	0	Squamous cell carcinoma	T3N2M1	66 × 52	RLL, mediastinal LNs, lung	2	7.14
19	69	M	1	Squamous cell carcinoma	T2N2M1	35 × 90	LLL, mediastinal LNs, bone	–	6.78
20	68	M	1	Adenocarcinoma	T2NXM1	49 × 35	LLL, bilateral mediastinal and distant LNs	0	5.84
21	48	M	1	Adenocarcinoma	T1N3M0	40 × 30	Right hilar, multiple mediastinal LNs	0	4.56

ECOG Eastern Cooperative Oncology Group Performance score, – denotes an inconclusive result, LLL left lower lobe, LNs lymph nodes, LUL left upper lobe, RLL right lower lobe, RUL right upper lobe

Interobserver agreement

There was excellent agreement of manual freehand ROI_max scoring between all three observers of primary lung tumour (T; ICC 0.94; 95% CI 0.9–0.97), lymph node metastases (LN; ICC 0.97; 0.95–0.98) and blood pool healthy reference tissue (BP; ICC 0.9; 0.84–0.94) using [^99mTc]NM-01 SPECT/CT (Table 2). T:BP (ICC 0.83; 0.73–0.90) and LN:BP (ICC 0.87; 0.81–0.92) ratios, which provide a quantitative measure of [^99mTc]NM-01 uptake for primary lung tumour and lymph node metastases on SPECT/CT, respectively, both demonstrated good interobserver agreement. Bland–Altman plot analysis demonstrated interobserver agreement with no proportional bias on linear regression for T:BP scores (Fig. 2). Bland–Altman analysis for LN:BP scores (Fig. 2) did, however, demonstrate proportional bias for observer B compared with both observer A (β = 0.11, p = 0.047) and observer C (β = -0.17, p = 0.02). There was acceptable agreement and no proportional bias for LN:BP scores between observers A and C (β = 0.06, p = 0.448).

Table 2.

Interobserver agreement

SPECT	Observer A	Observer B	Observer C	ICC (95% CI)	ICC level of agreement
	ROImax (mean ± SD)	ROImax (mean ± SD)	ROImax (mean ± SD)
Malignant lesion(s) ROI
Primary lung tumour (T)	548 ± 150	560 ± 154	568 ± 157	0.94 (0.90–0.97)	Good to excellent
Lymph node metastasis (LN)	459 ± 167	461 ± 167	448 ± 163	0.97 (0.95–0.98)	Excellent
Healthy reference tissue
Blood pool (BP)	260 ± 80	295 ± 80	270 ± 83	0.90 (0.84–0.94)	Good to excellent
Lung (freehand)	249 ± 146	310 ± 157	251 ± 88	0.84 (0.75–0.90)	Moderate to excellent
Lung (AA)	279 ± 113	279 ± 108	250 ± 94	0.89 (0.82–0.93)	Good to excellent
Lung (C)	307 ± 121	286 ± 129	288 ± 128	0.88 (0.81–0.93)	Good to excellent
Liver (freehand)	1121 ± 274	1262 ± 288	1194 ± 274	0.97 (0.95–0.98)	Excellent
Liver (GOJ)	1116 ± 264	1185 ± 270	1219 ± 283	0.95 (0.92–0.97)	Excellent

	ROImax ratio (mean ± SD)	ROImax ratio	ROImax ratio (mean ± SD)	ICC (95% CI)	ICC level of agreement
		(mean ± SD)
Ratios
T:BP	2.29 ± 0.98	1.99 ± 0.67	2.20 ± 0.77	0.83 (0.73–0.90)	Moderate to good
T:L (freehand)	2.56 ± 1.18	2.01 ± 0.71	2.40 ± 0.86	0.79 (0.68–0.88)	Moderate to good
T:L (AA)	2.10 ± 0.75	2.17 ± 0.86	2.41 ± 0.90	0.85 (0.77–0.91)	Good to excellent
T:L (C)	1.92 ± 0.71	2.20 ± 0.92	2.18 ± 0.88	0.80 (0.69–0.88)	Moderate to good
LN:BP	1.86 ± 0.65	1.65 ± 0.51	1.76 ± 0.66	0.87 (0.81–0.92)	Good to excellent

Malignant lesion and healthy tissue reference measurements (ROI_max; mean ± SD) and their ratios of all three observers with intraclass correlation coefficient (ICC), their 95% confidence interval (CI) and descriptive ICC level of agreement

AA aortic arch, BP blood pool, C carina, CI confidence interval, ICC intraclass correlation coefficient, GOJ gastroesophageal junction, L lung, LN lymph node metastasis, ROI region of interest, T primary lung tumour

Fig. 2.

Interobserver Bland–Altman level of agreement plots for log₁₀ T:BP (a–c) and log₁₀ LN:BP (d–f) scores. Upper and lower 95% limits of agreement represented by dashed lines. Solid horizontal lines represent between-observer mean difference. a T:BP scores observer A versus B (β = 0.13, p = 0.117); b T:BP scores observer A versus C (β = 0.07, p = 0.375); c T:BP scores observer B versus C (β = -0.06, p = 0.410); d LN:BP scores observer A versus B (β = 0.11, p = 0.047); e LN:BP scores observer A versus C (β = -0.06, p = 0.448); f LN:BP scores observer B versus C (β = -0.17, p = 0.020)

Freehand ROI_max scoring of non-affected lung background reference tissue demonstrated moderate to excellent interobserver agreement (ICC 0.84; 0.75–0.90). The ICC was improved with good to excellent agreement when either rule-based approach was applied, measuring ROI_max at the level of the aortic arch (ICC 0.89; 0.82–0.93) or the carina (ICC 0.88; 0.81–0.93). Calculated T:L ratios, when measuring healthy lung ROI_max at the level of the aortic arch, were also improved to good to excellent (ICC 0.85; 0.77–0.91) compared to moderate to excellent agreement demonstrated with freehand (ICC 0.79; 0.68–0.88) and carina rule-based (ICC 0.80; 0.69–0.88) approaches.

Excellent interobserver agreement (ICC 0.97; 0.95–0.98) was also demonstrated of freehand ROI_max scores for healthy reference tissue liver. Applying a consistent rule-based approach to score the liver at the level of the gastroesophageal junction did not improve agreement further (ICC 0.95; 0.92–0.97).

Using a T:BP score of ≥ 2.32 to represent a PD-L1 of ≥ 1%, the interobserver mean sensitivity was 61% and specificity 73% for this cohort (Table 3). Discrepant cases were reviewed, and a consensus was made between the three observers defining the T:BP as either < or ≥ 2.32 (Table 4). Five cases with PD-L1 expression between 1 and 10% on IHC remained discordant, four of which were considered negative PD-L1 by T:BP score of [^99mTc]NM-01 SPECT/CT but positive (≥ 1%) by IHC.

Table 3.

Summary of PD-L1 assessments made by T:BP using ≥ 2.32 as definition of positive result by [^99mTc]NM-01 SPECT/CT and ≥ 1% by IHC, along with interobserver mean sensitivity and specificity

	T:BP ≥ 2.32 and PD-L1 ≥ 1% (n)	T:BP < 2.32 and PD-L1 < 1% (n)	T:BP ≥ 2.32 and PD-L1 < 1% (n)	T:BP < 2.32 and PD-L1 ≥ 1% (n)
Observer A	8	3	2	3
Observer B	5	4	1	6
Observer C	7	4	1	4
Mean sensitivity	61%
Mean specificity	73%

Table 4.

Discrepant cases with individual observer and consensus 2-h T:BP scores (positive ≥ 2.32). PD-L1 tumour proportion score (TPS) ≥ 1% considered positive by immunohistochemistry (IHC)

Patient no.	PD-L1 expression by IHC		Observer A		Observer B		Observer C		Consensus
	TPS (%)	PD-L1 Assessment	T:BP	PD-L1 Assessment	T:BP	PD-L1 Assessment	T:BP	PD-L1 Assessment	T:BP	PD-L1 Assessment	Concordance with IHC
6	3	+	1.39	−	1.79	−	2.18	−	< 2.32	−	Discordant
9	1	+	1.93	−	2.00	−	1.97	−	< 2.32	−	Discordant
11	2	+	2.06	−	2.37	+	1.99	−	< 2.32	−	Discordant
14	10	+	1.74	−	1.98	−	1.57	−	< 2.32	−	Discordant
16	2	+	2.21	−	2.47	+	2.77	+	≥ 2.32	+	Concordant
17	65	+	2.29	−	2.70	+	3.10	+	≥ 2.32	+	Concordant
20	0	−	2.19	−	3.56	+	2.19	−	< 2.32	−	Concordant
21	0	−	3.41	+	6.69	+	4.27	+	≥ 2.32	+	Discordant

Positive ( +), negative (−)

Intraobserver agreement

Manual ROI_max scoring of primary lung tumour, lymph node metastases and blood pool reference tissue using [^99mTc]NM-01 SPECT/CT following a 42-day interval was consistent for the two observers analysed (Table 5). The intraobserver ICC for primary lung tumour ROI_max scores for observer B (ICC 0.96; 95% CI 0.93–0.98) and observer C (ICC 0.95; 0.91–0.97) demonstrated excellent agreement. Scoring of lymph node metastases also demonstrated excellent agreement (observer B ICC 0.97; observer C ICC 0.97, see Table 5 for 95% CIs). The intraobserver ICC for freehand ROI_max scores for reference tissue blood pool (observer B ICC 0.98; observer C ICC 0.97) confirmed excellent agreement. Excellent intraobserver agreement of both T:BP and LN:BP ratios for both observer B (ICC 0.96 and 0.95, respectively) and observer C (ICC 0.95 and 0.95) were also demonstrated. Bland–Altman plot analysis demonstrated intraobserver agreement with no proportional bias on linear regression for both T:BP and LN:BP scores (Fig. 3).

Table 5.

Intraobserver agreement. Malignant lesion and healthy tissue reference measurements (ROI_max or ratio; mean ± SD) and their ratios, of observer B and C from two timepoints, with intraclass correlation coefficient (ICC), its 95% confidence interval (CI) and descriptive ICC level of agreement

SPECT	Observer B				Observer C
	1	2	ICC (95% CI)	ICC level of agreement	1	2	ICC (95% CI)	ICC level of agreement
	ROImax (mean ± SD)	ROImax (mean ± SD)			ROImax (mean ± SD)	ROImax (mean ± SD)
Malignant lesion(s) ROI
Primary lung tumour (T)	560 ± 154	552 ± 146	0.96 (0.93–0.98)	Excellent	568 ± 157	560 ± 156	0.95 (0.91–0.97)	Excellent
Lymph node metastasis (LN)	461 ± 167	454 ± 177	0.97 (0.94–0.98)	Excellent	448 ± 163	460 ± 166	0.97 (0.95–0.98)	Excellent
Healthy reference tissue
Blood pool (BP)	295 ± 80	289 ± 82	0.98 (0.96–0.99)	Excellent	270 ± 83	272 ± 83	0.97 (0.94–0.98)	Excellent
Lung (freehand)	310 ± 157	321 ± 121	0.87 (0.77–0.92)	Good to excellent	251 ± 88	246 ± 97	0.91 (0.84–0.95)	Good to excellent
Lung (AA)	279 ± 108	266 ± 112	0.94 (0.89–0.97)	Good to excellent	250 ± 94	258 ± 105	0.94 (0.88–0.96)	Good to excellent
Lung (C)	286 ± 129	291 ± 127	0.94 (0.9–0.97)	Good to excellent	289 ± 128	302 ± 121	0.96 (0.92–0.98)	Excellent
Liver (freehand)	1262 ± 288	1288 ± 308	0.98 (0.95–0.99)	Excellent	1194 ± 274	1192 ± 269	0.99 (0.99–1.00)	Excellent
Liver (GOJ)	1185 ± 270	1176 ± 257	0.97 (0.94–0.98)	Excellent	1219 ± 283	1223 ± 283	0.98 (0.97–0.99)	Excellent

	ROImax ratio (mean ± SD)	ROImax ratio (mean ± SD)	ICC (95% CI)	ICC level of agreement	ROImax ratio (mean ± SD)	ROImax ratio (mean ± SD)	ICC (95% CI)	ICC level of agreement
Ratios
T:BP	1.99 ± 0.67	2.01 ± 0.65	0.96 (0.92–0.98)	Excellent	2.20 ± 0.77	2.17 ± 0.81	0.95 (0.90–0.97)	Good to excellent
T:L (freehand)	2.01 ± 0.71	1.86 ± 0.66	0.84 (0.69–0.92)	Moderate to excellent	2.40 ± 0.86	2.42 ± 0.87	0.92 (0.85–0.96)	Good to excellent
T:L (AA)	2.17 ± 0.86	2.24 ± 0.70	0.84 (0.72–0.91)	Moderate to excellent	2.41 ± 0.90	2.32 ± 0.85	0.91 (0.83–0.95)	Good to excellent
T:L (C)	2.20 ± 0.92	2.08 ± 0.72	0.85 (0.74–0.92)	Moderate to excellent	2.18 ± 0.88	1.99 ± 0.68	0.87 (0.71–0.94)	Moderate to excellent
LN:BP	1.65 ± 0.51	1.64 ± 0.58	0.95 (0.91–0.97)	Excellent	1.76 ± 0.66	1.77 ± 0.60	0.95 (0.91–0.97)	Excellent

AA aortic arch, BP blood pool, C carina, CI confidence interval, ICC intraclass correlation coefficient, GOJ gastroesophageal junction, L lung, LN lymph node metastasis, ROI region of interest, T primary lung tumour

Fig. 3.

Intraobserver Bland–Altman level of agreement plots for log₁₀ T:BP (a, b) and log₁₀ LN:BP (c, d) scores. Upper and lower 95% limits of agreement represented by dashed lines. Solid horizontal lines represent between-timepoints mean difference. a T:BP scores observer B, time 1 versus time 2 (β = 0.01, p = 0.781); b T:BP scores observer C, time 1 versus time 2 (β = -0.04, p = 0.462); c LN:BP scores observer B, time 1 versus time 2 (β = -0.08, p = 0.183); d LN:BP scores observer C, time 1 versus time 2 (β = 0.09, p = 0.080)

The intraobserver ICC for freehand ROI_max scores for healthy lung (observer B ICC 0.87; observer C ICC 0.91) and liver (observer B ICC 0.98; observer C ICC 0.99) demonstrated good to excellent agreement. A trend towards improved intraobserver agreement with rule-based approaches for healthy lung scoring was demonstrated, but no overall difference in the level of agreement was seen. Calculated T:L ratios demonstrated good or excellent intraobserver agreement (ICCs 0.84 to 0.92) irrespective of the healthy lung tissue scoring applied.

Discussion

Our study demonstrates that the quantitative assessment of [^99mTc]NM-01 using SPECT/CT is both reliable and reproducible within and between independent observers. Interobserver agreement was demonstrated for both T:BP (ICC 0.83) and LN:BP (ICC 0.87). In addition, excellent intraobserver agreement was shown (T:BP ICC 0.95–0.96; LN:BP ICC 0.95). This provides further evidence that [^99mTc]NM-01 has significant potential and clinical utility as a diagnostic agent for the measurement of PD-L1. Non-invasive assessment of PD-L1 is an attractive possibility considering the dynamic nature and heterogeneity of its expression. [^99mTc]NM-01 uptake measured by T:BP on SPECT/CT has already been shown to correlate with PD-L1 expression measured by IHC (r = 0.68, p = 0.014) [11]. This study, which confirms good to excellent inter- and intraobserver agreement of the quantitative assessment of [^99mTc]NM-01 SPECT/CT, supports its potential to provide reliable assessment of PD-L1 expression. It remains unclear whether temporal changes in PD-L1 expression and response assessment using [^99mTc]NM-01 SPECT/CT following anti-PD-1/PD-L1 therapy will be demonstrated and of clinical utility. This will be further explored in a phase II clinical trial, PECan [NCT04436406], which will also compare changes in PD-L1 expression and response to parameters on [¹⁸F]FDG PET/CT in both NSCLC and malignant melanoma.

This study is the first to assess the agreement of SPECT/CT in measuring PD-L1 expression in cancer. Several other radionuclides are currently being developed specifically for imaging the PD-1/PD-L1 axis. ¹⁸F-BMS-986192 (¹⁸Fluor-labelled anti-PD-L1 Adnectin) uptake on positron emission tomography (PET) has been shown to correlate with PD-L1 expression in NSCLC, as has ⁸⁹Zirconium-nivolumab for PD-1 expression, both in early phase clinical trials [13]. In both cases, inter- and intra-tumoural heterogeneity was demonstrated, consistent with the findings described in the early phase trial of [^99mTc]NM-01 SPECT/CT. An important characteristic of [^99mTc]NM-01 is that it is a small (14.3 kDa) antigen-binding fragment radiotracer with rapid blood clearance, with optimal SPECT/CT imaging performed at just 2 h following administration. As [^99mTc]NM-01 does not directly block the PD-L1 binding site, it does not interfere with the PD-1/PD-L1 axis and thus has the potential to assess whole-body PD-L1 status before, during and after anti-PD-L1 therapy. Whilst PET/CT provides a higher degree of spatial resolution, there are some notable benefits to SPECT/CT imaging. [^99mTc] radioisotope and SPECT imaging are both more widely available and relatively inexpensive. Concerns regarding the non-standardised quantification techniques for SPECT/CT may not be fully justified if quantification techniques are reproducible and reliable. Applying simple rules to ROI_max scoring may improve both inter- and intraobserver agreement, as demonstrated in this study where applying a set 3-cm sphere to score the unaffected lung at the level of the aortic arch improved the interobserver ICC. Whilst we did not show any significant improvement in agreement applying a similar rule to the liver, both inter- and intraobserver ICC remained excellent, suggesting that simple rule-based approaches may be used to standardise and simplify image interpretation without significant impact on quantification.

There are some limitations to this study. Firstly, it is limited by its sample size; nevertheless, the relatively narrow confidence intervals suggest a good estimate of the agreement. Despite good to excellent interobserver agreement, the mean sensitivity and specificity were relatively poor with some discrepant cases resulting in a PD-L1 assessment determined by T:BP of [^99mTc]NM-01 discordant with that found on IHC. This is not unexpected considering that heterogeneity of PD-L1 measured by IHC is widely reported in the literature and was demonstrated on [^99mTc]NM-01 assessment in our previous study [11]. In addition, the cut-off value of T:BP ≥ 2.32 correlating with a PD-L1 of ≥ 1% on IHC was determined on a small sample size and requires further validation in larger cohorts [11]. It is also important to note that the patient cohort was relatively heterogenous with regards to tumour staging. Due to the low number of measurable extra-nodal (lung and bone) metastases in the cohort (n = 8), statistical analysis using ICC of the quantitative assessment of [^99mTc]NM-01 in these lesions was not possible. With further understanding of the relationship between PD-L1 expression by IHC and [^99mTc]NM-01 SPECT/CT, it may be possible for both quantitative (as described in this study) and qualitative assessments to be made by observers blind to IHC PD-L1 expression, and their agreement evaluated. SPECT is a highly sensitive imaging modality but has relatively poor resolution; further optimisation with iterative reconstruction methods along with CT attenuation and scatter corrections have the potential to further improve and standardise quantification [14]. Novel SPECT reconstruction techniques that enable standardised quantification will be employed in forthcoming PECan and PELICAN studies [EudraCT 2020-002809-26] to further investigate and validate [^99mTc]NM-01 SPECT/CT clinically. This would also enable quantitative comparison with other PD-L1 PET radionuclides, for example the aforementioned ¹⁸F-BMS-986192 [13].

Conclusion

Overall, good to excellent inter- and intraobserver agreement of the quantitative assessment of [^99mTc]NM-01 SPECT/CT in NSCLC was demonstrated in this study. With correlation between PD-L1 expression determined by [^99mTc]NM-01 SPECT/CT and by immunohistochemistry previously demonstrated, there is considerable potential for [^99mTc]NM-01 SPECT/CT to reliably assess PD-L1 expression, with further analysis in subsequent clinical trials now being conducted.

Abbreviations

BP

Blood pool

CI

Confidence interval

CT

Computed tomography

ECOG

Eastern Cooperative Oncology Group

[¹⁸F]FDG

2-Deoxy-2-[fluorine-18]fluoro-D-glucose

GOJ

Gastroesophageal junction

ICC

Intraclass correlation coefficient

IHC

Immunohistochemistry

L

Lung

LN

Lymph node

LN:BP

Lymph node-to-blood pool ratio

NSCLC

Non-small cell lung cancer

PD-1

Programmed cell death protein 1

PD-L1

Programmed death-ligand 1

PET

Positron emission tomography

ROI_max

Maximum region of interest

SPECT

Single-photon emission computed tomography

T

Tumour

T:BP

Primary tumour-to-blood pool ratio

[^99mTc]

Technetium-99 m

Authors’ contributions

DJH, GC, VG and GJRC were responsible for study concept and design, data analysis and interpretation, writing, reviewing and editing the manuscript. DJH, GC and GJRC were responsible for observer rating. DJH and GJRC made the final decision to approve and submit the final manuscript. All authors read and approved the final manuscript.

Funding

This research was supported by NanoMab Technology Limited. The authors acknowledge financial support from the Wellcome Trust EPSRC Centre for Medical Engineering at King’s College London (WT203148/Z/16/Z), and UK Research & Innovation London Medical Imaging and Artificial Intelligence Centre.

Availability of data and materials

The data sets used and analysed during the current study are available from the corresponding author on reasonable request.

Code availability

Not applicable.

Ethics approval and consent to participate

Institutional ethics approval for this study was obtained from Shanghai General Hospital Ethics Committee (2016KY220). Participants enrolled in the study provided written informed consent to participate.

Consent for publication

Not applicable.

Competing interests

DJH has received honoraria from Novartis and research funding from NanoMab Technology Limited. GC provides consultancy for NanoMab Technology Limited. VG has received research support from Siemens Healthcare. GJRC has received research support from NanoMab Technology Ltd., Theragnostics Ltd. and Serac Healthcare Ltd. and provides consultancy for GE Healthcare and NanoMab Technology Ltd.