Interobserver and Intraobserver agreement of PD-L1 scoring in Head and Neck Squamous Cell Carcinoma (HSCC), Urothelial Carcinoma (UC), and Breast Carcinoma (BC)

Abstract

Aims:

Programmed death-ligand 1 (PD-L1) expression by tumour cells (TC) is a mechanism for tumour immune escape through downregulation of antitumour T cell responses and is a target for immunotherapy. PD-L1 status as a predictor of treatment response has led to the development of multiple biomarkers with different reference cut offs. We assessed pathologist consistency in evaluating PD-L1 immunopositivity by examining the interobserver and intraobserver agreement using various antibody clones and different cancer types.

Methods and results:

PD-L1 expression in TC and immune cells (IC) was manually scored in 27 HSCC, 30 UC, and 30 BC using three commercial clones (SP263, SP142, 22C3) and one platform-independent test (E1L3N). For interobserver agreement, PD-L1 status was evaluated blindly by three pathologists. For intraobserver agreement, PD-L1 expression was re-evaluated following a wash-out period. Intraclass correlation coefficient (ICC), overall percentage agreement (OPA) and κ values were calculated.

Using clinical algorithms, the percentage of PD-L1 positive cases in HSCC, BC, and UC were 15–81%, 47–67%, and 7–43% respectively. The percentage of PD-L1 positive cases relied heavily on the algorithm/cutoff values used. Almost perfect interobserver agreement was achieved using SP263 and E1L3N in HSCC, 22C3, SP142, and E1L3N in BC, and 22C3 in UC. The SP142 clone in UC and HSCC showed moderate agreement and was associated with lower ICC and decreased intraobserver concordance.

Conclusions:

Excellent interobserver and intraobserver agreement can be achieved using SP263, 22C3 and E1L3N, whereas PD-L1 scoring using SP142 clone is associated with a higher level of subjectivity.

INTRODUCTION

One hallmark of cancer is its ability to evade the human immune system (so called immune evasion). Programmed death-ligand 1 (PD-L1) is a transmembrane protein expressed on immune cells (IC) and tumour cells (TC), whereas programmed cell death (PD)-1 is expressed on IC and its expression can be induced following T cell activation. Upon binding to the receptor PD-1 in IC, the overexpressed PD-L1 in TC leads to functional inactivation of T cells and downregulation of T-cell anti-tumoural activity, resulting in immune evasion ^1–5. In recent years, PD-L1 and PD-1 have emerged as a key checkpoint that can be manipulated with inhibitory monoclonal antibodies in various cancer types. Multiple prospective randomized clinical trials have shown promising results of PD-1/PD-L1 immune checkpoint inhibitors in a variety of cancer types, including a subset of patients with urothelial carcinoma (UC) ^5–8, breast carcinoma (BC) ^{9, 10}, and head and neck squamous cell carcinoma (HSCC) ^11–14, leading to the Food and Drug Administration (FDA) approval of several anti-PD-1/PD-L1 agents.^{13, 15, 16}.

The implementation of PD-1/PD-L1 inhibitor immunotherapy, a costly treatment modality with non-negligible side effects, has triggered a search for predictive biomarkers that may aid in the identification of patients most likely to derive clinically meaningful benefit. Immunohistochemistry (IHC) of PD-L1 has emerged as one such predictive biomarker and has become a mandatory test to determine patient eligibility in certain clinical settings, e.g. first-line anti-PD-1/PD-L1 immunotherapy for advanced non-small cell lung cancer (NSCLC) ¹⁷.

Two major diagnostic challenges one faces in testing and interpreting PD-L1 IHC are the presence of multiple commercially-available diagnostic platforms, and variation in thresholds for PD-L1 immunopositivity based on both the antibody clones and the cancer type being assessed. To date, at least five PD-L1 antibody clones, 22C3, SP263, SP142, 28–8, and 73–10 are commercially available, each marketed as a companion or complementary diagnostic test for a different anti-PD-1/PD-L1 agent ¹⁷. Each antibody is associated with a different criterion for positivity in different cancer types ^5–14. Therefore, given the complexity of PD-L1 interpretation, it is important to determine whether pathologists can provide comparable and consistent PD-L1 scoring in various clinical settings.

In this study, we aimed to investigate the inter-observer and intra-observer agreement in interpreting PD-L1 immunopositivity in three tumour types (UC, BC, and HSCC) using four antibody clones (22C3, SP142, SP263, and a laboratory developed test employing clone E1L3N).

MATERIALS AND METHODS

Case selection

This work was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre (187–2016 and 359–2017). A retrospective search of the laboratory information system (Sunquest CoPath, Sunquest Information Systems, Tucson, AZ, USA) was performed to identify surgical resection specimens of UC between 1999 and 2015 (n=30), HSCC between 2000 and 2017 (n=27), and BC from 2016 to 2017 (n=30). A total of 87 cases were included in the study.

All cases were reviewed by a fellowship-trained pathologist (MRD for UC, BX for HSCC, and ES for BC) to confirm the diagnosis.

TMA construction

Triplicate 1-mm core TMAs were constructed using a TMA instrument (Beecher Instruments, Silver Springs, MD, USA) as previously described¹⁸. Sections from the TMA blocks were stained with hematoxylin and eosin (H&E) to confirm the adequacy of tumour sampling and to facilitate orientation during the interpretation of the PD-L1 staining.

Immunohistochemistry (IHC)

Serial unstained slides (4 microns) were prepared from each block for subsequent immunohistochemistry (IHC) with the following PD-L1 antibody clones: SP142 (Ventana medical systems, Tucson, AZ, USA), SP263 (Ventana medical systems, Tucson, AZ, USA), 22C3 pharmDX (Agilent technologies, Santa Clara, CA, USA) and E1L3N (Cell Signaling Technologies, Danvers, MA, USA). Sections were stained with the SP142 and SP263 antibody clones on the Ventana Benchmark Ultra automated staining platform, E1L3N on Leica Bond-III Autostainer (Leica Biosystems, Buffalo Grove, IL, USA), and with the 22C3 clone utilized the DAKO EnVision FLEX system on a DAKO Autostainer Link 48 system (Agilent technologies, Santa Clara, CA, USA) according to the manufacturer’s instructions.

Scoring of PD-L1 immunostain in tumour cells (TC) and immune cells (IC) and determination of PD-L1 immunopositivity

TC staining was defined as either partial or complete membranous staining of any intensity. IC staining was defined as cytoplasmic or membranous staining of any intensity in lymphocytes and macrophages. Only tumour-infiltrating ICs and the IC infiltrate surrounding the tumour within the 1-mm TMA core (i.e. peritumoural stroma) were included in IC scoring. To account for possible staining heterogeneity, TC and IC scores from triplicate TMA cores were each averaged to obtain one estimated percentage of positive TCs or ICs for each tumour specimen. The combined positive score (CPS) was defined as the number of PD-L1 positive TCs and ICs divided by total number of TCs X 100 (Figure 1).

Figure 1.

(A) A representative example of PD-L1 scoring. TC: tumour cells; IC: immune cells, HSCC: head and neck squamous cell carcinoma. (B) Interobserver agreement of PD-L1 evaluation using various antibody clones in HSCC, breast carcinoma (BC), and urothelial carcinoma (UC). As the cut-off value of E1L3N clone was not established, the threshold for 22C3, SP263, and SP142 antibody clones were applied.

To determine PD-L1 immunopositivity, the recommended cutoff values are summarized in Table 1. These thresholds were established according to clinical response to the associated immune checkpoint inhibitor in various clinical trials for UC ^{8, 19–21}, HSCC ^{11, 22–25}, and BC ^{9, 10}.

Table 1.

PD-L1 antibody clones and recommended algorithms to -identify PD-L1 immunopositivity determined in clinical trial settings.

PD-L1 antibody Clones	UC	HSCC	BC
22C3	CPS ≥ 10	CPS ≥ 1	CPS ≥ 1
SP142	IC ≥ 5%	IC ≥ 5%	IC ≥ 1%
SP263	TC ≥ 25% or IC ≥ 25%	TC ≥ 25%	NA

UC: urothelial carcinoma, HSCC: head and neck squamous cell carcinoma, BC: breast carcinoma, NA: not available.

In brief, PD-L1 22C3 was considered positive if CPS ≥ 10% in UC or ≥ 1% in HSCC or BC; PD-L1 SP142 was deemed positive if IC ≥ 5% in UC and HSCC, or IC ≥ 1% in BC; whereas SP263 was determined as positive when ≥25% TCs or ICs were stained in UC, or ≥25% of TCs were stained in HSCC. The threshold of PD-L1 positivity for SP263 clone in BC has not been determined by clinical trials to date. As no standard cutoff was determined for E1L3N clone, a lab developed test, we applied all available cut-off thresholds to this specific clone.

Statistics, interobserver agreement and intraoberver agreement

All statistical analyses were performed using the SPSS software, version 24.0 (IBM, Armonk, NY, USA). To assess the interobserver agreement of PD-L1 scoring, each participating pathologist (BX, ES, and MD) evaluate the TMAs independently and blindly to produce scores for the percentage of positive TCs and ICs. PD-L1 positivity was subsequently determined based on the average percentage using the algorithms stated in Table 1. The interobserver agreement was assessed using Fleiss’ kappa statistics. A κ value of 0.01–0.20 indicated slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement and 0.81–0.99 near-perfect agreement. The overall percentage agreement (OPA) was calculated, and the raw percentage of positive TCs and ICs were compared using inter-class correlation coefficient (ICC) among the three scorers.

To evaluate intraoberver agreement, each pathologist re-evaluated the TMA (BX for HSCC, ES for BC, and MD for UC) following a washout period of at least 1 month and the pre-washout and post-washout scores were compared.

RESULTS

PD-L1 immunoexpression in HSCC, BC, and UC

The frequency of PD-L1 immunopositivity determined using clinical algorithms in UC, HSCC and BC is summarized in Figure 1. In brief, PD-L1 immunopositivity was seen in 15% to 81% of HSCC, 47% to 67% of BC, and 7% to 43% of UC. The variability of PD-L1 immunopositivity within each tumour type was in part due to variation of PD-L1 cut-off thresholds. For example, in HSCC, when a high cut-off value of TC≥25% was used (SP263 clone), the PD-L1 immunopositivity was 19–22%. In contrast, when a lower threshold e.g. CPS≥1 was used (22C3 clone), the PD-L1 positivity was 67–78%.

Interobserver agreement of PD-L1 evaluation

Figure 1 summarizes the interobserver concordance among evaluators to determining PD-L1 positivity using clinical algorithms. Overall, an almost perfect interobserver agreement was achieved using the following antibody clones, SP263 (κ=0.836, OPA=93%) shown to preferably staining IC cut-off (κ=0.856, OPA=85%) in HSCC, all 3 clones (22C3, SP142, and E1L3N) in BC (κ ranged from 0.815 to 0.956, OPA ranged from 91% to 98%), and 22C3 clone in UC (κ=0.808, OPA=96%), whereas the remaining clones shows moderate to substantial agreement (κ ranged from 0.457 to 0.775).

We further evaluated the interobserver concordance using various cut-offs (TC: 1%, 5%, 25%; IC: 1%, 5%, 25%, CPS 1 and CPS 10) across all cancer types. The κ values are listed in Table 2. Overall, SP142 clone appeared to generate poorer interobserver agreement with a κ ranging from 0.326 (fair agreement) to 0.688 (substantial agreement) compared with the other 3 clones in which a substantial to perfect concordance was achieved (range of κ values: 22C3 0.644 to 0.821, SP263 0.681 to 0.896, E1L3N 0.640 to 1.000).

Table 2.

Interobserver agreement expressed as κ values using different cut-off across all three cancer types.

	TC 1%	TC5%	TC 25%	IC 1%	IC 5%	IC 25%	CPS 1	CPS 10		Agreement
22C3	0.723	0.779	0.821	0.760	0.644	0.729	0.775	0.777		Perfect
SP263	0.728	0.896	0.862	0.712	0.681	0.650	0.750	0.724		Almost perfect
SP142	0.601	0.655	0.326	0.688	0.582	0.560	0.688	0.754		Substantial
E1L3N	0.659	0.807	1.000	0.818	0.707	0.640	0.831	0.676		Moderate
										Fair

In addition, the raw percentage of positive TCs and ICs were compared among 3 evaluators using - ICC (Figure 2 and Table 3). The TC% and IC% scoring had good reliability using 22C3, SP263, and E1L3N clones with ICC ranging from 0.834 to 0.983. In contrast, SP142 clone generated only moderate agreement on TC% and IC% scoring (ICC = 0.694 and 0.689 respectively).

Figure 2.

Interobserver agreement and intraclass correlation coefficient (ICC) on raw positive percentage of tumor cells (TC) and immune cells (IC).

Table 3.

Interclass correlation coefficient and 95% confidence interval (CI) to evaluate positive percentage of TC and IC.

	TC	95% CI	IC	95% CI
22C3	0.922	0.894–0.944	0.834	0.779–0.879
SP263	0.983	0.976–0.988	0.851	0.795–0.895
SP142	0.694	0.606–0.770	0.689	0.600–0.766
E1L3N	0.983	0.976–0.988	0.892	0.855–0.946

Intraobserver concordance of PD-L1 interpretation

The intraobserver consistency of PD-L1 scoring was evaluated by comparing the reading before and after a washout period of at least 1 month. The results are shown in Figure 3. When using an algorithm evaluating TC% only (SP263 and E1L3N clone SP263 cutoff), a perfect agreement (κ=1.000) was reached. The intraobserver agreement using IC-only algorithms (SP142 and E1L3N SP142 cutoff in all cancer types) appeared to be lowest, resulting in a moderate to substantial agreement (κ=0.696 and 0.500 for HSCC, 0.798 and 0.861 for BC, 0.423 and 0.659 for UC), whereas intraobserver concordance using thresholds incorporating both IC and TC (SP263 in UC and 22C3 in all cancer types) fell in the middle with a κ value ranging from 0.600 to 1.000.

Figure 3.

Intraobserver agreement of PD-L1 interpretation. Shaded squares represent cases that are interpreted as PD-L1 positive.

DISCUSSION

PD-L1 immunoexpression in TCs and/or ICs is a useful clinical biomarker to select patients that may benefit the most from immune checkpoint inhibitors. Evaluation of PD-L1 IHC has been included as either companion or supplementary tests in multiple anti-PD1/anti-PD-L1 clinical trials of various cancers ^{5–14, 17}. Therefore, it becomes increasingly important for practicing pathologists to provide reliable and consistent interpretation of PD-L1 IHC across different tumour types, as it is of paramount importance in determining the eligibility for anti-PD-L1/anti-PD1 therapy, and to predict treatment benefits.

Using various clinical algorithms determined in clinical trials as detailed in Table 1, we reported a frequency of immunopositivity in 15 to 81% of HSCC, 47 to 67% of BC, and 10% to 43% of UC. Although there are previous reports that PD-L1 immunopositivity may be affected by tumour and/or therapy-associated factors, e.g. tumour type and subtypes ²⁶, tumour stage/metastasis status ²⁷, and prior therapies received ²⁸, the variation in frequency of PD-L1 immunopositivity reported in this study related solely to the difference of cutoff values. The lowest rate of positivity was observed when an algorithm of 25% of TC (SP263, HSCC) or 25% of TC or IC (SP263, UC) were utilized, whereas the highest frequency of PD-L1 positivity was seen using a threshold of CPS≥1 (22C3 in HSCC and BC) and IC≥1 (SP142 in BC). Clearly, the threshold of PD-L1 immunopositivity has a significant impact on patients’ selection and eligibility for anti-PD-L1/anti-PD1 therapy.

To date, only a limited number of publications have studied the consistency and accuracy of PD-L1 evaluation by investigating the interobserver agreement on PD-L1 scoring among pathologists. Most studies focused exclusively on NSCLC, while only two studies included HSCC, one included UC, and none - BC ^29–39. Table 4 provides a literature review of these studies. Overall, it appears that a reliable PD-L1 reading can be achieved using various antibody clones and TC cutoff values. The reported κ ranges from 0.58 to 0.96. Two studies have reported a poor interobserver agreement when evaluating IC in NSCLC with a κ value of < 0.3 ^{31, 35}. It is postulated that the poor inter-observer concordance among pathologists in scoring IC in NSCLC may be secondary to a lack of experiences in evaluating IC as most of the PD-L1 clinical algorithms in NSCLC do not include routine evaluation of IC ³⁵. Indeed, in UC and HSCC in which evaluation of IC is incorporated into the PD-L1 evaluation algorithm, we have previously shown that a cut off value of ≥5% IC using SP142 clone resulted in comparable inter-observer concordance with a kappa value of 0.757 ³⁸. Similarly, in this study, we have reported an overall consistent PD-L1 interpretation using 22C3, SP263, and E1L3N clone when applying universal cutoff on TC, IC, or CPS score. The κ values using TC alone ranged from 0.659 to 1.000, IC alone from 0.640 to 0.818, and CPS from 0.676 to 0.831.

Table 4.

Literature review: interobserver and intraobserver agreement in PD-L1 scoring

Ref.	Study cohort	Antibody clone	Threshold for PD-L1 positivity	Agreement studied	Number of evaluators	Kappa (κ)	OPA
33	60 NSCLC	22C3	1% TC or 50% TC	Interobserver Intraobserver	10	0.68 (1% TC) 0.58 (50% TC) NA	84.2% (1% TC) 91.3% (50% TC) 89.7% (1% TC) 91.3% (50% TC)
37	198 NSCLC	SP263 22C3	1% TC and 50% TC	Interobserver	2	0.73 (SP263) 0.77 (22C3)	NA
29	81 NSCLC 100 HSCC	SP263	25% TC	Interobserver Intraobserver	3	NA NA	96.7% (NSCLC) 96.3% (HSCC) 96.3% (NSCLC) 94.3% (HSCC)
30	55 NSCLC	28–8 22C3 SP142 SP263	TC: <1%, 1–4%, 5–9%, 10–24%, 25–49%, and/or ≥50%	Interobserver	7	0.78–0.92 (28–8) 0.71–0.95 (22C3) 0.75–0.91 (SP263) 0.81–0.96 (SP142)	NA
31	30 NSCLC	E1L3N 28–8 22C3 SP142 SP263	TC or IC: ≥1%, ≥5%, ≥10% or ≥50%	Interobserver	2	TC: 0.73–0.79 (E1L3N) 0.63–0.79 (28–8) 0.66–0.79 (22C3) 0.63–0.80 (SP142) 0.59–0.78 (SP263) IC: 0.34–0.55 (E1L3N) 0.12–0.24 (28–8) 0.14–0.28 (22C3) 0.15–0.25 (SP142) 0.12–0.19 (SP263)	NA
39	46 NSCLC	E1L3N	TC: ≥1%, ≥50% IC: ≥1%, ≥10%	Interobserver	2	NA	79% (TC 1%) 98% (TC 50%) 75% (IC 1%) 77% (IC 10%)
34	90 NSCLC	28–8 22C3 SP142 E1L3N	TC: ≥1%, ≥50%	Interobserver	13	≥50%: 0.749 ≥1%: 0.537	NA
35	71 NSCLC	22C3 28–8 SP142 SP263	TC: 1%, 5%, 10%, 25%, 50%, 80% IC: categorical as IC1, IC2, IC33	Interobserver	24	TC: >0.7 (slightly diminished at 1% and 80%) IC: 0.08–0.27	NA
38	58 UC 27 HSCC	SP263 SP142	SP263: ≥25% TC or IC SP142: ≥5% IC	Interobserver	3	0.702 (SP263) 0.757 (SP142)	93% (SP263) 94% (SP142)

UC: urothelial carcinoma, PC: prostatic carcinoma, HSCC: head and neck squamous cell carcinoma, SGC: salivary gland carcinoma, NSCLC: non-small cell lung carcinoma, TC: tumor cells, IC: immune cells.

Among the four antibody clones tested, the antibody that seems to generate lower interobserver agreement is the SP142 clone in evaluating TC alone or IC alone, with a κ value of 0.326 to 0.655 when evaluating TC, and 0.560 to 0.688 when evaluating IC. Similarly, the ICC is lower for SP142 clone compared with the other 3 clones when evaluation raw positive TC or IC percentage, being 0.689 to 0.694 compared with 0.834 to 0.983 of ICC from the other 3 antibody clones. SP142 clone has been previously shown to preferably stain IC with a significantly lower percentage of TC staining in NSCLC, UC, and HSCC ^{30, 34, 35}. As the evaluators have remained the same for all antibody clones, it is likely that the unique staining properties of the SP142 clone is the cause of variable PD-L1 readout. Overall, we have shown a moderate to substantial interobserver agreement in evaluating PD-L1 using SP263, 22C3 and E1L3N clones, and a moderate concordance using SP142 clones, supporting the notion that a reliable and reproducible PD-L1 readout can be achieved using various antibody clones in UC, HSCC, and BC.

Our study is one of the three studies to date that have investigated intra-observer consistency in evaluating PD-L1 scoring. The other two studies focused on NSCLC and/or HSCC and showed a reasonable intra-observer agreement with a κ value of 0.73–0.77 and an OPA of 90–91% ^{29, 33}. Herein, we report a moderate to perfect interobserver agreement (κ ranges from 0.423 to 1.000) after a washout period of at least 1 month. Once again, SP142 appears to have lower intra-observer constancy compared with the other three clones.

Several mechanisms can be proposed to improve the performance and consistency of pathologists in PD-L1 scoring. First, a unified scoring and reporting system across multiple antibody clones, staining platforms, and cancer types may greatly reduce the complexity of the task leading to more consistent results. However, this requires standardization and large-scale validation studies. Second, formal training on PD-L1 interpretation may be useful in the initial phase of standardization to unify the criteria each pathologist uses. All three participating pathologists had at least one formal training session on PD-L1 interpretation. Third, PD-L1 as a predictive biomarker, similar to ER, PR, and Her2 in breast cancer, should require testing sites to participate in external quality assessment program in addition to routine discussion of challenging/borderline cases at consensus meeting to lead to improved and consistent performance. Lastly, we have previously shown that a small proportion of tumors may show PD-L1 staining heterogenicity and discrepancy between small (biopsy) material and surgical resection specimens ³⁸. Therefore, it may be prudent to repeat PD-L1 IHC in resection specimens in borderline and/or negative cases.

Our study has several potential limitations. We have utilized TMA methodology. Although each tumour had triplicate cores, the PD-L1 scoring from the cores may not represent the PD-L1 readout from the whole section. However, several studies, including our own, have previously shown that the PD-L1 IHC results from whole sections and small materials (e.g. biopsy or TMA) are largely comparable ^{35, 38}. Therefore, TMA can serve as an affordable alternative in evaluating inter- and intra-observer agreement for PD-L1 IHC in research setting. Although the study included a reasonable number of cases, being 87, there were only 27 to 30 cases per cancer type that were evaluated by three pathologists. We also did not address the impacts of preanalytical parameters on PD-L1 IHC in this study, e.g. inter-run and inter-instrument reproducibility. As a submission of IHC application to the FDA usually requires a validation set of 60 to 100 cases as well as inter-run and inter-instrument reproductivity ⁴⁰, future larger-scale multi-institutional studies are needed to establish the reliability of PD-L1 scoring in a broader setting and to address the impacts of various pre-analytic parameters on PD-L1 IHC.

CONCLUSIONS

In summary, we have reported that the clinical cutoff values had a significant impact on the percentage of tumours determined as PD-L1 positive in UC, BC, and HSCC. A substantial interobserver and intraobserver agreement can be achieve in evaluating TC, IC, or CPS using SP263, 22C3, and E1L3N clones, whereas PD-L1 evaluation using SP142 clone appears to be more variable.