Abstract
Programmed Death 1 Ligand 1 (PD-L1) Immunohistochemistry (IHC) is the key FDA-approved predictive marker to identify responders to anti-PD1 axis drugs. Multiple PD-L1 IHC assays with various antibodies and cut-points have been used in clinical trials across tumor types. Comparative performance characteristics of these assays have been extensively studied qualitatively but not quantitatively. Here we evaluate the use of a standardized PD-L1 Index TMA to objectively determine agreement between antibody assays for PD-L1 applying quantitative digital image analysis. Using a specially constructed Index Tissue Microarray (TMA) containing a panel of 10 isogenic cell lines in triplicate, we tested identical but independently grown batches of isogenic cells to prove Index TMAs can be produced in large quantities and hence serve as a standardization tool. Then the Index TMAs were evaluated using quantitative immunofluorescence (QIF) to validate the TMA itself and also to compare antibodies including E1L3N, SP142 and SP263. Next, an inter-laboratory and inter-assay comparison of 5 PD-L1 chromogenic IHC assays (US Food and Drug Administration (FDA) approved and lab developed test (LDT)) were performed at 12 sites around the USA. As previously reported, the SP142-FDA assay failed to detect low levels of PD-L1 in cell lines distinguished by the other 4 assays. The assays for 22C3-FDA, 28–8-FDA, SP263-FDA and E1L3N-LDT were highly similar across sites and all laboratories showed a high consistency over time for all assays using this Index TMA. In conclusion, we were able to objectively quantify PD-L1 expression on a standardized Index TMA using digital image analysis and we confirmed previous subjective assessments of these assays, but now in a multi-institutional setting. We envision commercial use of this Index TMA or similar smaller version as a useful standardization mechanism to compare results between institutions and to identify abnormalities while running routine clinical samples.
Introduction
Immune checkpoint blockade inhibitors have changed the landscape of cancer treatment in the last decade, demonstrating unprecedented clinical success in several tumor types1–3. Simultaneously, PD-L1 (Programmed Cell Death 1 Ligand 1) expression has been identified as a predictive diagnostic marker to select patients that may benefit from anti-PD-1 (Programmed Cell Death 1) axis agents such as nivolumab, pembrolizumab, atezolizumab, and durvalumab4–7. With each drug, a unique proprietary diagnostic test has been developed. Currently, there are multiple qualitative PD-L1 assays, involving various antibodies, to assess the expression of PD-L1 by immunohistochemistry (IHC) using chromogenic methods. The US Food and Drug Administration (FDA) approved some of them as either companion and/or complementary diagnostic tests for specific drugs and cancers. The PD-L1 IHC 22C3 PharmDx kit (Agilent Technologies Inc.) is the only companion diagnostic test approved by FDA for pembrolizumab in non-small cell lung cancer (NSCLC), whereas the Ventana PD-L1 SP142 Assay® (Roche Ventana Medical Systems, Inc.) is a diagnostic test for atezolizumab that the FDA approved as a complementary assay in NSCLC and as a companion assay in patients with urothelial cancer. Additionally, the range of cut points for defining positive cases and cell type expression (tumor or immune cells) is widely variable across clinical trials.
In the past years, there has been a significant effort to compare the performance characteristics of these PD-L1 IHC assays in patient samples8–15. Recently, Tsao et al16 published the results of the second phase of the Blue Print study, confirming the interchangeability of 22C3, 28–8, and SP263 assays and the lower sensitivity of SP142 assay in lung cancer, after PD-L1 scoring by 25 experienced pathologists. Moreover, concordance between PD-L1 IHC assays has been assessed by correlating levels of protein detected by the corresponding assay and the level of PD-L1 mRNA assessed by RNAscope assay9. Although there is a high concordance between trained pathologists for PD-L1 scoring in tumor cells8, 10–12, 16, there is a poor reliability when PD-L1 is evaluated in immune cells or at low PD-L1 scores16, 17.
The assessment of PD-L1 in all previous studies has been on the entire assay, combining the subjective interpretation with the level of expression and the localization. It is possible that these variables are confounding, and variable levels of expression may not be discernable when combined with both localization and subjective interpretation. Despite the increasing number of publications on this topic, there is very little work done on the analytic assessment of expression separated from the interpretation. Here, we propose the use of a standardized Index TMA (Tissue Microarray) to objectively compare PD-L1 IHC assays using quantitative image analysis. To achieve this goal, an Index TMA with isogenic cell lines expressing PD-L1 spanning a predetermined dynamic range was built and quantitatively evaluated for PD-L1 expression using both chromogenic and fluorescent immunohistochemistry methods. In addition, the Index TMA was tested independently across multiple institutions that used diverse PD-L1 chromogenic IHC assays, both FDA-approved and LDT (Laboratory Developed Test) to compare the assays and to illustrate the utility of analytic standardization between institutions.
Materials and Methods
Index TMA construction
A panel of 15 isogenic cell lines expressing various amounts of PD-L1 was sourced from Horizon Dx including production and growth of cell lines. After review, 10 lines were selected, and an Index TMA was constructed according to the map in figure 1. Three independent batches of cell lines were cultured, each approximately 2 months apart, to produce three independent batches of formalin-fixed, paraffin-embedded (FFPE)) cell pellets blocks for batch to batch concordance and correlation assessment.
Figure 1. Map of the Index TMA for standardization of PD-L1 measurement.
A) Schematic map including the cell line code, where the color gradient corresponds to the level of PD-L1 expression (black = high expression; white = no expression). B) Representative image of standardized TMA stained with 22C3 IHC assay at low magnification, including three spots at high magnification of negative spot (“1”, cell line 8063), low expresser (“2”, cell line 11221), and high expresser (“3”, cell line 11185). Bar = 200µm.
PD-L1 IHC assays
PD-L1 expression was evaluated by quantitative immunofluorescence (QIF) and chromogenic IHC using five monoclonal antibodies (Suppl. Table 1), including both LDT and FDA-approved assays. For QIF, clones E1L3N (#13684, Cell Signaling Technology, Inc.), SP142 (#M4420, SpringBio), and SP263 (#790–4905, Ventana Medical Systems, Inc.) were assessed. For chromogenic IHC, automated systems were used for different clones using our own protocol for the LDT E1L3N (#13684, Cell Signaling Technology, Inc.) on multiple platforms, and protocols specified by corresponding manufacturer per the FDA labeling for 22C3 (#SK006, Dako) and 28–8 (#SK005, Dako) with the Dako Autostainer Link 48 Instrument (Dako). Similarly, on label protocols were used for FDA-approved assays; SP263 (#740–4907) and SP142 (#740–4859) both from Ventana Medical Systems, Inc. on the Benchmark Ultra (Ventana Medical Systems, Inc.). For the multi-institutional comparison, twelve 5-µm sections per PD-L1 assay were cut from a block of Index TMA at Yale University and sent to 12 institutions for staining weekly during 6 consecutive weeks, running 2 slides per week with their clinical workload using the assay of choice for each institution.
Quantitative immunofluorescence
TMA slides were stained using a protocol previously described by our laboratory10. Primary antibodies for PD-L1 E1L3N and SP142 were incubated overnight at 4ºC and clone SP263 was incubated for 20 minutes at 37ºC. An optimized final concentration for each antibody was used10, except for SP263 which was only available from the company as a pre-diluted reagent and hence not optimizable (Suppl. Table 1). Then, slides were incubated in rabbit EnVision reagent (K4009, Dako) for 1 hr at RT and Cy5-Tyramide (SAT705A001EA, Perkin Elmer) was used to amplify target signal. Finally, TMAs were stained with 1:250 4,6-diamidino-2-phenylindole (DAPI) for 10 minutes at RT and mounted with Prolong Gold antifade mounting reagent (P36394, Life Technologies). Image analysis was performed using AQUA method of QIF (NavigateBP), which generates a score by dividing the sum of target pixel intensities by the area of the molecularly designated compartment. Scores are automatically normalized to lamp hours, bit depth and CC intensity of the microscope to account for day to day variation.
Digital image analysis on chromogenic IHC
After chromogenic staining, as described above, slides were scanned on the Aperio ScanScope XT platform. Then, PD-L1 expression on the cell lines was quantified using the open-source software QuPath18 (Queen’s University of Belfast, Northern Ireland). Briefly, once images were loaded, cores were automatically selected through the TMA annotation module, and later classified by the module “positive cell detection”. An optimized algorithm was used for cell segmentation based on the size of the nucleus and cell expansion, and for DAB intensity quantification of PD-L1 expression for all antibodies looking at the “cell: DAB OD mean” score compartment, among the possible options (nucleus, cytoplasm, cell). The settings were adjusted to avoid false positive detection. Results were shown as percentage of PD-L1+ cells or as optical density (OD) of the chromogenic staining divided by mm2.
Statistical Analysis
All datasets were analyzed and plotted using GraphPad Prism v7.0 software for Windows (GraphPad Software, Inc.). Bar graphs were used to plot average of QIF scores and percentage of PD-L1+ cells and OD/mm2 for DAB staining quantified by QuPath, per cell line from for each assay. Due to the complexity of this work, we define the following terms and concepts related to standardization of measurement: reproducibility, a component of the precision, determines the agreement among results obtained from testing the same substance by using the same test protocol under repeatable conditions (different operators, locations and time); correlation, a statistical relationship to quantify the strength of association between two variables; and concordance, the proportion of the outcomes of a specific test which are identical to an agreed upon reference19–21. Comparison between blocks, antibodies/assays and laboratories was assessed for correlation using a linear regression coefficient (R2) and Bland-Altman plots22 were used to assess correlation and concordance, respectively, between PD-L1 assays. Levey-Jennings plots were used to evaluate the quality and the consistency of measurement over time for every laboratory and assay, using both the average of all cell lines or each cell line.
Results
Validation of the Index TMA as a standardized PD-L1 assay
To validate the Index TMA, we quantitatively showed the dynamic range of PD-L1 expression on the TMA and then assess the correlation of Index TMA production. Using the Index TMA described above (Fig. 1 and Suppl. Fig. 1), we first measured PD-L1 by quantitative immunofluorescence using different antibodies with previously optimized conditions10 (Suppl. Table 1). Lack of staining in cell lines 8063 and 11198 with all antibodies confirmed them as negative controls, as none of them expressed detectable levels of protein by Western Blot and only 11198 showed very low levels of PD-L1 mRNA (HorizonDx, unpublished data). The highest levels of PD-L1 were detected on cell lines 11228, 11190 and 11185 by all antibodies. The rest of the clones expressed intermediate amounts of PD-L1 protein in different proportions. As previously described10, 11, 16, the cellular location of PD-L1 was membranous with all the antibodies in all the cell lines. To quantitatively determine the amount of PD-L1 in each line on the TMA, we performed QIF with each antibody (Fig. 2) showing the full dynamic range of the assay and the relative expression levels determined by each antibody.
Figure 2. PD-L1 distribution on Index TMA using different PD-L1 antibodies by quantitative immunofluorescence.
Bar graphs show mean ± SD (Standard Deviation) of QIF scores for E1L3N, SP142 and SP263 antibodies used in 3 independent experiments with 3 different blocks.
Correlation between three independent blocks was assessed by both QIF and chromogenic assays using linear regression (Table 1). For QIF analysis, the correlation between blocks was high for antibodies SP142 (R2=0.983–0.995) and E1L3N (R2=0.834–0.857), but somewhat lower for SP263 (R2=0.655–0.962) most likely due to our inability to optimize the titration for this antibody. Chromogenic assessment utilized QuPath analysis as described above and showed results comparable to the QIF results (Table 1), showing the ability to construct highly reproducible Index TMAs over time using isogenic cell lines.
Table 1. Linear regression analaysis of anti-PD-L1 antibodies in 3 independent batches of Index TMA as determined by QIF and by DAB staining.
(3 independent experiments per batch/block and antibody).
| QIF |
SP142 LDT |
SP263 LDT |
E1L3N LDT |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AQUA score | Block pairs | 1 – 2 | 2 – 3 | 1 – 3 | 1 – 2 | 2 – 3 | 1 – 3 | 1 – 2 | 2 – 3 | 1 – 3 | |||
| Slope | 0.876 | 1.156 | 1.025 | 1.043 | 1.073 | 1.23 | 1.344 | 0.517 | 0.759 | ||||
| R2 | 0.983 | 0.988 | 0.995 | 0.655 | 0.962 | 0.761 | 0.857 | 0.834 | 0.853 | ||||
|
DAB |
22C3 FDA |
28–8 FDA |
E1L3N LDT |
||||||||||
| % PD-L1(+) cells | Block pairs | 1 – 2 | 2 – 3 | 1 – 3 | 1 – 2 | 2 – 3 | 1 – 3 | 1 – 2 | 2 – 3 | 1 – 3 | |||
| Slope | 0.878 | 0.969 | 0.857 | 0.439 | 2.11 | 1.02 | 0.573 | 1.269 | 0.792 | ||||
| R2 | 0.864 | 0.991 | 0.869 | 0.877 | 0.917 | 0.972 | 0.885 | 0.927 | 0.956 | ||||
Comparison of PD-L1 antibodies by QIF and DAB staining using the Index TMA
Correlation and concordance amongst the antibodies or assays were analyzed by linear regression and Bland-Altman plots, respectively. For QIF quantification (Fig. 3A), an average of 27 spots per cell line were evaluated per PD-L1 antibody (3 spots per cell line/3 independent experiments/3 different blocks or batches/per antibody). Correlation of QIF scores obtained with antibodies E1L3N, SP142 and SP263 under LDT conditions was extremely high by linear regression, with coefficients greater than 0.96 similar to that previously described10. Additionally, the concordance was high as confirmed by Bland-Altman analysis.
Figure 3. Validation of Index TMA with cell lines using 3 clones of anti-PD-L1 antibody by QIF.
A) Linear regression and Bland-Altman plots for PD-L1 antibodies (SP142, SP263, E1L3N) tested by an immunofluorescence protocol developed at the lab (LDT) and previously published10. Data represent the average of 3 independent experiments with 3 different blocks per antibody used. B) Linear regression and Bland-Altman plots for PD-L1 DAB IHC assays (22C3 FDA, 28–8 FDA, E1L3N LDT, SP263 FDA, SP142 FDA) performed by our CLIA-certified laboratory. Data represent the mean of % PD-L1+ cells per cell line, measured by QuPath platform, of 2 TMAs per run for each week during 10 consecutive weeks (n=20 slides per assay). For A) and B): R2 = coefficient of determination; for Bland-Altman diagrams, continued line represents bias and dotted lines represent upper and lower 95% limits of agreement.
Regarding the chromogenic IHC assays, we tested 5 PD-L1 IHC assays (22C3 FDA, 28–8 FDA, SP263 FDA, SP142 FDA, E1L3N LDT) by running 2 slides weekly over 10 consecutive weeks in our CLIA-certified research laboratory (Fig. 3B). The chromogenic IHC assays also showed comparably high degree of agreement between 22C3 FDA, 28–8 FDA and E1L3N LDT assays by Bland-Altman analysis, whereas the concordance between SP142 and SP263 FDA assays was equally poor when compared to the 3 other assays and themselves (Fig. 3B, Table 2), identifying some of the intermediate expressers as outliers. On the other hand, linear regressions showed the highest correlation between 22C3 FDA, 28–8 FDA and E1L3N LDT assays (R2=0.943–0.993) and a slightly lower correlation between SP142 FDA and the above three assays than SP263 FDA and the above three assays (R2=0.823–0.871) (Fig. 3B). After comparing SP142 and SP263 FDA assays, linear regression showed a high correlation (R2=0.992).
Table 2. High Degree of Agreement Achieved for PD-L1 DAB IHC assays by our CLIA-certified laboratory based on Bland-Altman graphs.
(2 slides weekly during 10 consecutive weeks per assay).
| 22C3 FDA | 28–8 FDA | SP142 FDA | SP263 FDA | E1L3N LDT | |
|---|---|---|---|---|---|
| 22C3 FDA | YES | NO | NO | YES | |
| 28–8 FDA | NO | NO | YES | ||
| SP142 FDA | NO | NO | |||
| SP263 FDA | NO |
Comparison of PD-L1 chromogenic IHC assays among multiple institutions using the Index TMA
The vision for use of the Index TMA is to be used as a tool for calibration for development of standardization of assays, as shown above, and also to enable methodology transfer and facilitate standardization of assays between institutions, and even more importantly between clinical trials and diagnostic IHC laboratories that will be performing the predictive IHC assays for patient care. To assess the efficiency of this Index TMA for methodology transfer and comparison of analytical performance between different laboratories, the Index cell line TMA was tested independently in 12 institutions across the country, including hospitals and clinical laboratories, using diverse PD-L1 IHC assays and platforms (Table 3). Although they weren’t run in parallel with clinical samples as the slides for the multi-institutional study, we included the results for SP142 FDA assay from our CLIA-certified research laboratory to increase statistical power for this assay. The chromogenic assays were evaluated both by intensity (OD/mm2) and percentage of positive cells (counting cells above the detection threshold for the assay). As shown in figure 4, summarizing 240 slides from 12 institutions and measuring both by % PD-L1+ cells or OD/mm2, all PD-L1 assays performed essentially identically for the very high expressers (cell lines 11228, 11190, 11185) and for the negative cases (cell lines 8063, 11198). Although, in the negative clones 28–8 and E1L3N show some measurable background signal. Confirming previous qualitative results11, 16, the SP142 FDA assay showed lower sensitivity failing to detect low levels of PD-L1 in the intermediate cell lines (cell lines 11186–11187) measurable by the other 4 assays (Suppl. Fig. 2). Similar conclusions may be drawn when assessing intensity, although the background signal is more easily illustrated in the non-expressing cases, showing that even optimized assays show low levels of signal inherent in the IHC technique.
Table 3.
Assays and platforms tested in the multi-institutional comparison of the PD-L1 standardization TMA.
| Assay | 22C3 FDA | 28–8 FDA | SP263 FDA | SP142 FDA | E1L3N LDT |
|---|---|---|---|---|---|
| LAB01 | X | X | X | ||
| LAB02 | X | X | X | ||
| LAB03 | X | ||||
| LAB04 | X | X | |||
| LAB05 | X | X | X | ||
| LAB06 | X | ||||
| LAB07 | X | ||||
| LAB08 | X | X | |||
| LAB09 | X | ||||
| LAB10 | X | ||||
| LAB11 | X | ||||
| LAB12 | X |
Figure 4. PD-L1 distribution on Index TMA using different PD-L1 DAB IHC assays by multiple institutions.
Bar graphs show mean ± SD, as percentage of PD-L1+ cells (A) or OD/mm2 (B) measured for each cell line including all the slides (2 per run) run during 6 consecutive weeks by all the laboratories that ran the specific PD-L1 assay: 22C3 FDA: 6 labs; 28–8 FDA: 4 labs; SP263 FDA: 3 labs; SP142 FDA: 2 labs (including the CLIA-certified laboratory that participated in the validation of the Index TMA); E1L3N LDT: 6 labs. The open-source platform QuPath was used to quantify DAB staining. OD=Optical Density for DAB.
Having an assay perform with the same sensitivity and specificity is critical for a companion diagnostic test. Figure 5 shows cross institutional comparisons for each assay. To compare the assays from each site to each other we have used regressions (Suppl. Fig. 3, Table 4).
Figure 5. Comparison between laboratories for each PD-L1 assay on Index TMA.
All bar graphs show mean ± SD as percentage of PD-L1+ cells for all the runs performed by each laboratory and PD-L1 chromogenic assay: 22C3 FDA (A), 28–8 FDA (B), E1L3N LDT (C), SP263 FDA (D), and SP142 FDA (E). #00#: CLIA-certified laboratory.
Table 4.
Linear regression analysis of 5 PD-L1 IHC assays between institutions using the Index TMA.
| 22C3 FDA Assay | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| % PD-L1+ cells | Lab pairs | 01–02* | 01–04 | 01–05 | 01–09 | 01–12 | 02–04* | 02–05* | 02–09* |
| Slope | 1.553 | 1.001 | 1.013 | 0.996 | 0.999 | 0.548 | 0.625 | 0.432 | |
| R2 | 0.953 | 0.999 | 0.999 | 0.994 | 0.999 | 0.943 | 0.935 | 0.843 | |
| % PD-L1+ cells | Lab pairs | 02–12* | 04–05 | 04–09 | 04–12 | 05–09 | 05–12 | 09–12 | |
| Slope | 0.647 | 1.011 | 0.996 | 0.997 | 0.983 | 0.986 | 0.996 | ||
| R2 | 0.962 | 0.999 | 0.997 | 0.998 | 0.994 | 0.999 | 0.991 | ||
| 28–8 FDA Assay | |||||||||
| % PD-L1+ cells | Lab pairs | 01–02* | 01–04 | 01–05* | 02–04* | 02–05 | 04–05* | ||
| Slope | 1.321 | 0.989 | 1.418 | 0.598 | 0.998 | 1.725 | |||
| R2 | 0.977 | 0.996 | 0.983 | 0.959 | 0.997 | 0.967 | |||
| E1L3N LDT Assay | |||||||||
| % PD-L1+ cells | Lab pairs | 01–02 | 01–03 | 01–05* | 01–06* | 01–07* | 02–03 | 02–05* | 02–06* |
| Slope | 1.021 | 1.005 | 0.534 | 0.826 | 0.622 | 0.983 | 0.432 | 0.641 | |
| R2 | 0.983 | 0.983 | 0.631 | 0.864 | 0.804 | 0.999 | 0.766 | 0.964 | |
| % PD-L1+ cells | Lab pairs | 02–07* | 03–05* | 03–06* | 03–07* | 05–06 | 05–07 | 06–07 | |
| Slope | 0.492 | 0.416 | 0.628 | 0.481 | 1.208 | 1.237 | 1.021 | ||
| R2 | 0.932 | 0.715 | 0.934 | 0.896 | 0.991 | 0.993 | 0.996 | ||
| SP263 FDA Assay | |||||||||
| % PD-L1+ cells | Lab pairs | 08–10 | 08–11 | 10–11 | |||||
| Slope | 0.975 | 0.993 | 1.015 | ||||||
| R2 | 0.995 | 0.999 | 0.998 | ||||||
| SP142 FDA Assay | |||||||||
| % PD-L1+ cells | Lab pairs | 08–00 | |||||||
| Slope | 1.001 | ||||||||
| R2 | 0.998 | ||||||||
: saturated values of highest expressers of PD-L1 (cell clones 11228, 11190, 11185) were not included, only assessed cell clones with expression in the linear range.
Finally, another use for the Index TMA is to assure consistency of measurements over time. We assessed this for each of the collaborating institutions across a period of 6 weeks (Fig. 6, Suppl. Fig. 4–13). As shown by the Levey-Jennings plots for assays from collaborating labs, all PD-L1 assays and laboratories showed values between the ± 2SD limits, leading to approximately 10% false rejections when 2 levels of control are being analyzed23.
Figure 6. Levey-Jennings plots of PD-L1 IHC chromogenic assays among all the laboratories during 6 consecutive weeks.
Each dot represents the mean of % PD-L1+ cells on all cell lines of 2 TMAs per run for each week for 22C3 FDA, 28–8 FDA, E1L3N LDT, SP263 FDA, SP142 FDA assays. “Mean”: total average of all cell lines in all experiments, SD: Standard Deviation of “Mean”, where ±1/2/3SD corresponds to “Mean” ±1/2/3SD.
Discussion
PD-L1 expression by IHC is used in clinical trials to predict response to anti-PD1/PD-L1 therapies in a variety of cancers4–7. There are 4 PD-L1 IHC assays approved by the FDA, either as companion or complementary diagnostic tests, with different cut points and score systems associated with each assay. In evaluation of these assays, the FDA required proof of quality, reproducibility, and sensitivity for prediction, but was less stringent on analytic sensitivity. Several studies have compared the performance of many PD-L1 assays available by qualitative analysis on tumor samples8–15, with variable results. In this study, we used an Index TMA with isogenic cell lines expressing PD-L1 in a dynamic range to quantitatively compare PD-L1 IHC assays using two immunohistochemistry methods and digital image analysis. This approach allowed us to quantitatively assess analytic sensitivity, concordance and correlation of the assays, although the use of isogenic cell lines, not clinical samples, precludes assessment of predictive accuracy24. After confirming regression of PD-L1 levels between 3 independent blocks using diverse antibodies and two immunohistochemical techniques, we observed a high correlation and concordance among the antibodies by QIF by linear regressions and Bland-Altman plots, respectively, confirming previous results10. The Index TMA also allowed us to assess the analytic sensitivity of each FDA-approved assay. We find that three of the four FDA-approved assays, and an LDT assay, have comparable analytic sensitivity, but the SP142 assay is less sensitive, failing to detect low level expression that is detected by the other four assays. This observation supports previous qualitative studies11, 16 and two comparative studies using a commercial cell line TMA, also showing reduced PD-L1 staining with SP142 assay on moderate expressers compared to 22C3, 28–8 and SP263 assays10, 25.
The use of this Index array allowed the comparison of the PD-L1 assays across both time and institution. The assessment of data from 12 independent institutions revealed that the similarity of the pair 22C3/SP263 and 28–8/E1L3N was the highest (Fig. 4). However, the staining patterns were extremely similar for the negative and the saturated cases for all the DAB IHC assays. Most of the studies suggest the method of detection as the principal parameter for discrepancy among assays rather than the antibody itself10, 25, 26. The 12 institutions study also showed good measurement consistency over time at every institution and good correlation by linear regression between institutions doing the same assay. This suggests that the assays themselves should not lead to discordant readings amongst pathologists or, with the exception of the SP142 assay, between assays from different vendors.
There are several limitations to this study. Perhaps the most significant is that this study only tests the analytic sensitivity of the assays or antibodies in the context of isogenic cell lines. It is possible that levels of PD-L1 in actual tumors may be different or that the protein may have post-translational modifications that are not seen in cell lines. While this concern is recognized, the testing of PD-L1 in actual cancers has been done in many other studies and is not the goal of this work. A second weakness is the use of TMAs. In clinical usage, all of these assays are done on full histologic sections. As such, this data must be taken in the context of its usage as a control, not as an evaluation of issues related to assay performance in patient tissue. Finally, another weakness of this work is the absence of quantitative information from patients with outcome data. While we explore the dynamic range in a model system, we cannot be sure we have replicated the dynamic range seen in patient tumors that have been treated with immunotherapy.
In conclusion, the results generated with this Index TMA are concordant and correlated between independently constructed blocks as assessed by two immunohistochemical methods suggesting this sort of control can be manufactured and used over a period of time. We also suggest that Index TMAs can be a useful tool to compare results from multiple PD-L1 IHC assays and institutions and is also useful as an internal control to identify abnormal runs in a routine clinical laboratory.
Supplementary Material
Acknowledgements:
We would like to thank Lori Charette and the team at the Yale Pathology Tissue Service for production of the high-quality sections used in this study. We also thank Dr. Emina Torlakovic for critical review of the manuscript. This work was supported by a research agreement from Astra Zeneca and support for the Yale SPORE in Lung Cancer (P50- CA196530) and the Yale Cancer Center (P30- CA016359).
Footnotes
Conflict of interest: Dr. Cougot is employee of Horizon Discovery; Dr. Sholl received honoraria from Foghorn Therapeutics and Astra Zeneca and travel support from Bristol Myers Squibb, and she is a consultant for LOXO Oncology; Dr. Qiu is a US Pathology Advisory Board member for Merck & Co. Inc.; Dr. Mino-Kenudson is a consultant for Merrimack Pharmaceuticals and H3 Biomedicine; Dr. Walker is employee of Astra Zeneca; Dr. Beasley has been a prior consultant for Merck and Genentech; Dr. Yaziji is at the Speakers Bureau for the HER2 program and is a consultant for Genentech; in the past 12 months, Dr. Rimm has served as a consultant for advisor to Astra Zeneca, Agendia, Agilent, Biocept, BMS, Cell Signaling Technology, Cepheid, Daiichi Sankyo, GSK, Merck, Konica/Minolta, PerkinElmer/Akoya, and Ultivue. The remaining authors declare that they have no conflict of interest.
Supplementary information is available at Laboratory Investigation’s website.
References
- 1.Brahmer JR, Tykodi SS, Chow LQ, et al. Safety and activity of anti-PD-L1 antibody in patients with advanced cancer. N Engl J Med 2012;366(26):2455–2465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Topalian SL, Hodi FS, Brahmer JR, et al. Safety, activity, and immune correlates of anti-PD-1 antibody in cancer. N Engl J Med 2012;366(26):2443–2454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Herbst RS, Soria JC, Kowanetz M, et al. Predictive correlates of response to the anti-PD-L1 antibody MPDL3280A in cancer patients. Nature 2014;515(7528):563–567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Herbst RS, Baas P, Kim DW, et al. Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet 2016;387(10027):1540–1550. [DOI] [PubMed] [Google Scholar]
- 5.Horn L, Spigel DR, Vokes EE, et al. Nivolumab Versus Docetaxel in Previously Treated Patients With Advanced Non-Small-Cell Lung Cancer: Two-Year Outcomes From Two Randomized, Open-Label, Phase III Trials (CheckMate 017 and CheckMate 057). J Clin Oncol 2017;35(35):3924–3933. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.McDermott DF, Sosman JA, Sznol M, et al. Atezolizumab, an Anti-Programmed Death-Ligand 1 Antibody, in Metastatic Renal Cell Carcinoma: Long-Term Safety, Clinical Activity, and Immune Correlates From a Phase Ia Study. J Clin Oncol 2016;34(8):833–842. [DOI] [PubMed] [Google Scholar]
- 7.Antonia SJ, Villegas A, Daniel D, et al. Durvalumab after Chemoradiotherapy in Stage III Non-Small-Cell Lung Cancer. N Engl J Med 2017;377(20):1919–1929. [DOI] [PubMed] [Google Scholar]
- 8.Hirsch FR, McElhinny A, Stanforth D, et al. PD-L1 Immunohistochemistry Assays for Lung Cancer: Results from Phase 1 of the Blueprint PD-L1 IHC Assay Comparison Project. J Thorac Oncol 2017;12(2):208–222. [DOI] [PubMed] [Google Scholar]
- 9.Tretiakova M, Fulton R, Kocherginsky M, et al. Concordance study of PD-L1 expression in primary and metastatic bladder carcinomas: comparison of four commonly used antibodies and RNA expression. Mod Pathol 2018;31(4):623–632. [DOI] [PubMed] [Google Scholar]
- 10.Gaule P, Smithy JW, Toki M, et al. A Quantitative Comparison of Antibodies to Programmed Cell Death 1 Ligand 1. JAMA Oncol 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rimm DL, Han G, Taube JM, et al. A Prospective, Multi-institutional, Pathologist-Based Assessment of 4 Immunohistochemistry Assays for PD-L1 Expression in Non-Small Cell Lung Cancer. JAMA Oncol 2017;3(8):1051–1058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Batenchuk C, Albitar M, Zerba K, et al. A real-world, comparative study of FDA-approved diagnostic assays PD-L1 IHC 28–8 and 22C3 in lung cancer and other malignancies. J Clin Pathol 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fujimoto D, Sato Y, Uehara K, et al. Predictive Performance of Four Programmed Cell Death Ligand 1 Assay Systems on Nivolumab Response in Previously Treated Patients with Non-Small Cell Lung Cancer. J Thorac Oncol 2018;13(3):377–386. [DOI] [PubMed] [Google Scholar]
- 14.Hendry S, Byrne DJ, Wright GM, et al. Comparison of Four PD-L1 Immunohistochemical Assays in Lung Cancer. J Thorac Oncol 2018;13(3):367–376. [DOI] [PubMed] [Google Scholar]
- 15.Ratcliffe MJ, Sharpe A, Midha A, et al. Agreement between Programmed Cell Death Ligand-1 Diagnostic Assays across Multiple Protein Expression Cutoffs in Non-Small Cell Lung Cancer. Clin Cancer Res 2017;23(14):3585–3591. [DOI] [PubMed] [Google Scholar]
- 16.Tsao MS, Kerr KM, Kockx M, et al. PD-L1 Immunohistochemistry Comparability Study in Real-Life Clinical Samples: Results of Blueprint Phase 2 Project. J Thorac Oncol 2018;13(9):1302–1311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Brunnstrom H, Johansson A, Westbom-Fremer S, et al. PD-L1 immunohistochemistry in clinical diagnostics of lung cancer: inter-pathologist variability is higher than assay variability. Mod Pathol 2017;30(10):1411–1421. [DOI] [PubMed] [Google Scholar]
- 18.Bankhead P, Loughrey MB, Fernandez JA, et al. QuPath: Open source software for digital pathology image analysis. Sci Rep 2017;7(1):16878. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Ferrario D, Brustio R, Hartung T. Glossary of reference terms for alternative test methods and their validation. ALTEX 2014;31(3):319–335. [DOI] [PubMed] [Google Scholar]
- 20.Fitzgibbons PL, Bradley LA, Fatheree LA, et al. Principles of analytic validation of immunohistochemical assays: Guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Arch Pathol Lab Med 2014;138(11):1432–1443. [DOI] [PubMed] [Google Scholar]
- 21.Francis DP, Coats AJ, Gibson DG. How high can a correlation coefficient be? Effects of limited reproducibility of common cardiological measures. Int J Cardiol 1999;69(2):185–189. [DOI] [PubMed] [Google Scholar]
- 22.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1(8476):307–310. [PubMed] [Google Scholar]
- 23.Westgard JO. Statistical quality control procedures. Clin Lab Med 2013;33(1):111–124. [DOI] [PubMed] [Google Scholar]
- 24.Howat WJ, Lewis A, Jones P, et al. Antibody validation of immunohistochemistry for biomarker discovery: recommendations of a consortium of academic and pharmaceutical based histopathology researchers. Methods 2014;70(1):34–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Scheel AH, Baenfer G, Baretton G, et al. Interlaboratory concordance of PD-L1 immunohistochemistry for non-small-cell lung cancer. Histopathology 2018;72(3):449–459. [DOI] [PubMed] [Google Scholar]
- 26.Schats KA, Van Vre EA, De Schepper S, et al. Validated programmed cell death ligand 1 immunohistochemistry assays (E1L3N and SP142) reveal similar immune cell staining patterns in melanoma when using the same sensitive detection system. Histopathology 2017;70(2):253–263. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.