Performance comparison of a flow cytometry immunoassay for intracellular cytokine staining and the QuantiFERON® SARS-CoV-2 test for detection and quantification of SARS-CoV-2-Spike-reactive-IFN-γ-producing T cells after COVID-19 vaccination

Abstract

Purpose

We compared the performance of an in-house-developed flow cytometry assay for intracellular cytokine staining (FC-ICS) and a commercially-available cytokine release assay (the QuantiFERON® SARS-CoV-2 Test [QF]) for detection and quantification of SARS-CoV-2-Spike (S)-reactive-IFN-γ-producing T cells after COVID-19 vaccination.

Patients and methods

The sample included 141 individuals (all male; median age, 42 years; 20–72) who had been fully vaccinated with the Comirnaty® COVID-19 vaccine (at a median of 114 days; 34–145). Prior to vaccination, 91 were categorized as being SARS-CoV-2-naïve and 50 as SARS-CoV-2-experienced. A whole blood-based FC-ICS using 15-mer overlapping peptides encompassing the entire SARS-CoV-2 S protein was used for enumeration of virus-specific IFN-γ-producing CD4⁺ and CD8⁺ T cells. The QF test (Ag1 for CD4⁺ T cells and Ag2 for CD4⁺ and CD8⁺ T cells in combination) was carried out following the manufacturer’s instructions.

Results

The FC-ICS and the QF assays returned significantly discordant qualitative results in both the entire cohort (P<0.001 with QF Ag1 and QF Ag2) and in SARS-CoV-2-naïve participants alone (P=0.005 and P=0.01, respectively). Discrepant results mostly involved FC-ICS positive/QF negative specimens. Overall, no correlation was found either between SARS-CoV-2 IFN-γ- CD4⁺ T-cell frequencies and IFN-γ levels measured in the QF Ag1 tube (P=0.78) or between the sum of SARS-CoV-2 IFN-γ CD4⁺ and CD8⁺ T-cell frequencies and IFN-γ levels quantified in the QF Ag2 tube.

Conclusion

The data suggest a greater sensitivity for the FC-ICS assay than the QF test, and urge caution when comparing SARS-CoV-2 T-cell immune responses assessed using different analytical platforms.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10096-022-04422-7.

Introduction

Natural SARS-CoV-2 infection in humans elicits robust functional T-cell responses, which target a wide array of epitopes within all structural and several non-structural viral proteins. These are thought to contribute crucial protection against severe clinical presentations of COVID-19 (see [1] for review), as supported by data obtained in non-human primate experimental models [2, 3]. In turn, licensed COVID-19 vaccines strongly induce Th-1-skewed spike (S)-reactive T-cell responses [1] which may overcome evasion by SARS-CoV-2 variants of concern [4, 5]. Peripheral blood (PB) levels of SARS-CoV-2-reactive T cells are considered a surrogate for tissue and organ virus-specific T-cell responses, although this assumption is a matter of debate. Use of whole blood as the matrix for carrying out T cell immunoassays offers a number of advantages over peripheral blood mononuclear cells (PBMCs), most notably simplicity, shorter turnaround times, and the possibility of automation. A wide range of either in-house or commercialized tests using whole blood as the starting material, including cytokine release assays (CRA) and flow cytometry for intracellular cytokine (FC-ICS)-based assays have been developed for measuring SARS-CoV-2 T-cell responses [6–19], yet it is uncertain how results provided by these assays correlate, which hampers direct comparison of studies employing different platforms. In the current study, we compared the performance of an in-house-developed FC-ICS based upon an homologous assay designed to enumerate Cytomegalovirus-specific T cells [20, 21], and a commercially-available CRA, the QuantiFERON® SARS-CoV-2 Test (QF), for detection and quantification of SARS-CoV-2-Spike-reactive-IFN-γ-producing T cells after COVID-19 vaccination.

Patients and methods

Patients and specimens

The current prospective study enrolled 141 adult individuals (all male; median age, 42 years; range, 20–72) who had received the full dose of the Comirnaty® COVID-19 vaccine (at a median of 114 days; range, 34–145) and agreed to participate in the study. A single specimen was analyzed per participant. The study was approved by the Research Ethics Committee of Fundación de Investigación del Hospital General Universitario de Valencia (FIHGUV). Informed consent was obtained from all participants.

Flow cytometry SARS-CoV-2 Spike immunoassay

Detection and enumeration of SARS-CoV-2-S-reactive IFNγ-producing-CD8⁺ and CD4⁺ T cells were performed by a flow cytometry immunoassay for intracellular cytokine staining (BD Fastimmune, Becton Dickinson and Company, BD Biosciences, San Jose, CA) as previously described [14, 15]. Briefly, heparinized whole blood (0.5 ml) was simultaneously stimulated for 6 h with two sets of 15-mer overlapping peptides (11-mer overlap) encompassing the SARS-CoV-2 Spike (S) glycoprotein (S1, 158 peptides and S2, 157 peptides) at a concentration of 1 μg/ml per peptide, in the presence of 1 μg/ml of costimulatory monoclonal antibodies (mAbs) to CD28 and CD49d. Peptide mixes were obtained from JPT Peptide Technologies GmbH (Berlin, Germany). Samples mock-stimulated with phosphate-buffered saline (PBS)/dimethyl sulfoxide and costimulatory antibodies were run in parallel. Brefeldin A (10 μg/ml) was added for the last 4 h of incubation. Blood was then lysed (BD FACS lysing solution) and frozen at −80°C until tested. On the day of testing, stimulated blood was thawed at 37°C, washed, permeabilized (BD permeabilizing solution), and stained with a combination of labelled mAbs (anti-IFNγ-FITC, anti-CD4-APC-H7, anti-CD8-PerCP-Cy5.5, and anti-CD3-APC) for 1 h at room temperature. Appropriate positive (phytohemagglutinin) and isotype controls were used. Cells were then washed, resuspended in 200 μL of 1% paraformaldehyde in PBS, and analyzed within 2 h on a LSR Fortessa flow cytometer (BD Biosciences Immunocytometry Systems, San Jose, CA) using the FlowJo v-10 software (BD Biosciences). CD3⁺/CD8⁺ and CD3⁺/CD4⁺ events were gated and then analyzed for IFN-γ production (Supplementary Fig. 1). All data were corrected for background IFN-γ production (FITC-labelled isotype control antibody or negative control, whichever yielded a higher background level) and expressed as the number of SARS-CoV-2-reactive IFN-γ-producing CD4⁺ or CD8⁺ T cells relative to the absolute number of CD4⁺ and CD8⁺ T cells, respectively, ×100 (%). Any frequency value of SARS-CoV-2-reactive IFN-γ-producing CD4⁺ or CD8⁺ T cells after background substraction was considered a positive (detectable) result and used for analysis purposes.

QuantiFERON® SARS-CoV-2 immunoassay

The QuantiFERON® SARS-CoV-2 assay (QF) was performed as previously reported [16] following the manufacturer’s recommendations. QF Starter Set Blood Collection Tubes (BCT) consist of two tubes: Ag1 (T-cell epitopes within the receptor binding domain of S1) and Ag2 (T-cell epitopes within S1 and S2), which allow detection of IFNγ production by SARS-CoV-2-S CD4⁺ T cells, and both CD4⁺ and CD8⁺ T cells, respectively. The control set tubes include negative (Nil) and positive (Mitogen) controls. Tubes were incubated at 37°C for 16–24 h, then centrifuged for 15 min at 2500 g to separate the plasmas, which were assayed within one week of specimen collection. Next, IFN-γ ELISA was performed in a Dynex DS2® automated ELISA system. IFNγ values (IU/ml) for CD4⁺, CD4⁺/CD8⁺ were obtained by subtracting the Nil value from the raw data. Since no interpretative criteria are provided by the manufacturer, we chose to categorize the specimens as reactive if they returned any positive value (>0 IU/ml) after background substraction.

Electrochemiluminescent SARS-CoV-2 nucleoprotein immunoassay

SARS-CoV-2-nucleocapsid (N)-reactive total antibodies were measured by Roche Elecsys® Anti-SARS-CoV-2 N assays (Roche Diagnostics, Pleasanton, CA, USA), following the manufacturer’s instructions.

Statistical methods

Positive and negative percent agreement (PPA and NPA, respectively) between immunoassays were calculated using a diagnostic 2×2 test, assuming 100% specificity for both tests. The McNemar test was used to assess differences in the proportions of qualitative results (positive/negative) returned by comparison assays. Differences between medians were compared using the Mann–Whitney U test. Spearman’s rank test was used to assess the correlation between continuous variables. Two-sided exact P values were reported. A P value <0.05 was considered statistically significant. The analyses were performed using SPSS version 20.0 (SPSS, Chicago, IL, USA).

Results

Baseline characteristics of participants

Two patient characteristics are pertinent to the current study. First, 91 out of the 141 participants (65%) were categorized as SARS-CoV-2-naïve and 50 (35%) as SARS-CoV-2-experienced, based on the absence or presence of SARS-CoV-2 N-reactive antibodies and historical records; and second, 6 of the 141 participants were HIV-1 positive.

Agreement between qualitative results provided by the immunoassays

We first compared the rate of detectable SARS-CoV-2-S IFN-γ CD4⁺ T-cell responses obtained in participants using the FC-ICS and QF Ag1 tests. The data are shown in Table 1. FC-ICS returned more positive results than QF Ag1 (134 vs. 120). Both assays yielded significantly discordant results (McNemar, P=0.004), with an overall agreement of 83%. PPA across assays was acceptable (83%), whereas NPA was strikingly low (7%). When considering SARS-CoV-2-naïve and experienced participants separately for analyses, qualitative results were significantly divergent for the former (McNemar, P=0.005; overall agreement, PPA and NPA: 86%, 86%, and 0%), yet not for the latter (McNemar, P=0.54; overall agreement, PPA, and NPA: 78%, 77%, and 15%). Overall, the frequency of SARS-CoV-2 IFN-γ CD4⁺ T cells was higher (P=0.02) in participants testing positive than in those returning negative results by QF Ag1 (Fig. 1A). This was also observed when SARS-CoV-2-naïve and experienced participants were analyzed separately, although statistical significance was not reached.

Fig. 1.

SARS-CoV-2-Spike-reactive-IFN-γ-producing T cells as measured by flow cytometry in participants testing either positive or negative by the QuantiFERON® SARS-CoV-2 Test by SARS-CoV-2 infection status prior to full vaccination with the Comirnaty® COVID-19 vaccine. Frequencies of SARS-CoV-2-Spike-reactive-IFN-γ-producing CD4⁺ T cells (A) and the sum of SARS-CoV-2-Spike-reactive-IFN-γ-producing CD4⁺ and CD8⁺ T cells (B). P values (Mann-Whitney U test) are shown for comparisons

We next compared qualitative results provided by FC-ICS and QF Ag2. For this analysis, SARS-CoV-2 S-reactive IFN-γ CD4⁺ and CD8⁺ T by FC-ICS were considered in combination: any given specimen yielding a positive result for either T-cell subset was categorized as positive. All 141 participants had detectable SARS-CoV-2-S-reactive IFN-γ CD4⁺, CD8⁺ T cells, or both, whereas only 129 tested positive in the QF Ag2 tube; thus, NPA across assays could not be assessed. Again, results returned by comparison assays differed significantly when taking into consideration for analysis either all participants (McNemar, P=0.0005) or SARS-CoV-2-naïve alone (P=0.01), but not SARS-CoV-2-experienced participants (McNemar, P=0.13). Overall median frequency of SARS-CoV-2-S IFN-γ T cells was higher in participants testing positive than negative in the QF Ag2 test (Fig. 1B), although the difference did not reach statistical significance (P=0.30).

Correlation between quantitative results returned by the comparison immunoassays

Overall, no correlation was found between SARS-CoV-2 IFN-γ CD4⁺ T-cell frequencies and IFN-γ levels measured in the QF Ag1 tube (Fig. 2A), nor between the frequency sum of SARS-CoV-2-S IFN-γ CD4⁺ and CD8⁺ T cells and IFN-γ levels quantified in the QF Ag2 tube (Fig. 2B).

Fig. 2.

Correlation between frequencies of SARS-CoV-2-Spike-reactive-IFN-γ-producing T cells as measured by flow cytometry and IFN-γ levels (IU/ml) measured in participants. Frequencies of SARS-CoV-2-Spike-reactive-IFN-γ-producing CD4⁺ T cells (A) and the sum of SARS-CoV-2-Spike-reactive-IFN-γ-producing CD4⁺ T and CD8⁺ T cells (B) relative to IFN-γ levels (IU/ml) measured in the QuantiFERON® SARS-CoV-2 Ag 1 and Ag 2 tubes, respectively. Rho and P values (Spearman rank test) are shown

Discussion

Cytokine release assays (CRA) employing whole blood instead of PBMCs are gaining ground for assessment of functional SARS-CoV-2-reactive T-cell responses due to their ease of use, and notably because they allow a comprehensive functional evaluation of primed T cells; moreover, as noted by Tan and colleagues [22], testing whole blood may more closely mimic in vivo conditions than assessing purified PBMC cells. Among commercially-available whole blood SARS-CoV-2 CRA, use of the QuantiFERON® platform is supported by extensive experience gathered in other clinical settings, such as diagnosis of latent tuberculosis [23] and inference of protection against active CMV infection and disease in transplant recipients [24]. The QF SARS-CoV-2 test has been used to evaluate post-vaccination T-cell responses in different group populations including seemingly healthy individuals, elderly nursing home residents, and immunosuppressed patients [13, 17–19]. We developed a whole blood FC-ICS assay for enumeration of SARS-CoV-2-S-reactive IFNγ-producing CD4⁺ and CD8⁺ T cells [14, 15] based upon a protocol we have been using for years to monitor CMV-specific T-cell responses in transplant recipients [20, 21]. This CMV FC-ICS assay was found to be more sensitive than the QuantiFERON-CMV assay, but both provided quantitative results showing good correlation (Rho=0.7) [21]; furthermore, the QF-CMV assay allowed us to estimate total and polyfunctional CMV-specific IFN-γ-producing CD8⁺ T-cell responses in specimens testing positive by both methods [21]. In this context, here we aimed at comparing the performance of our SARS-CoV-2 FC-ICS assay and the QF CRA in a cohort of individuals fully vaccinated with the Comirnaty® COVID-19 vaccine. We were aware that a strict analytical comparison across the assays is not straightforward due to differences involving the stimulating antigen (presumed) and the test design, yet elucidation of how these tests do compare from a qualitative and quantitative standpoints may have potential clinical and epidemiological interest. Since no consensus reference test exists for resolving discrepancies across tests, we assumed for both assays that all positive results were true ones (100% specificity). In addition, in the absence of manufacturer established criteria for interpretation of QF SARS-CoV-2 assay results, we chose to categorize the specimens as positive whichever quantity of IFN-γ was measured (after background substraction). Overall, the data revealed a substantial lack of agreement between qualitative results returned by the two assays regardless of SARS-CoV-2 infection status of participants prior to full vaccination, owing mainly to the outperformance of FC-ICS against QF-SARS-CoV-2 Ag1 and Ag2 in returning positive results. Nonetheless, PPA across assays was acceptable. The data suggested that the FC-ICS assay may be more sensitive than the QF test for detection of SARS-CoV-2-S-targeted T-cell responses. Setting a higher threshold for QF-SARS-CoV-2 positivity (i.e.≥0.15 IU/ml), as previously suggested [16, 18, 19], would have yielded a much lower positive results rate. From a quantitative standpoint, our data revealed a lack of correlation between SARS-CoV-2 IFN-γ CD4⁺ and CD4⁺ plus CD8⁺ T-cell frequencies enumerated by the FC-ICS assay and IFN-γ levels measured by the QF-SARS-CoV-2 Ag1 and Ag2 tubes, respectively, which was certainly unexpected. These discrepancies may be partly due to the different nature of the S antigen employed in the assays. In the FC-ICS assay, we used two libraries of 15 mer peptides with 11 amino acid overlaps spanning the entire S protein sequence, mixes which are known to strongly stimulate CD4⁺ T cells equivalent to those elicited with whole recombinant protein, and CD8⁺ T-cell responses at much higher level than whole proteins, regardless of HLA-haplotype specificities. The precise nature of the SARS-CoV-2 antigens in the QF Ag tubes is not disclosed by the manufacturer, but they are presumably immunogenic peptides of different lengths mapping within S protein; in this regard, it remains to be elucidated whether Ag1 and Ag2 peptide pools in the QF SARS-CoV-2 assay may be less fitted to bind to all HLA-I and HLA-II specificities than overlapping 15 mer peptides. The striking dissimilarity across results provided by the FC-ICS and QF immunoassays in our cohort warrants further investigation.

A few studies have compared CRA with other immunoassay platforms using PBMCs for quantitation of SARS-CoV-2 functional T-cell responses [6, 7], but to our knowledge, none has involved CRA and whole blood-based FC-ICS assay. In these studies [6, 7] including both vaccinated and unvaccinated individuals, the sensitivity of a CRA, based on stimulation of whole blood with overlapping peptides encompassing the entire S sequence and subsequent measurement of IFN-γ-and IL-2 by ELISA in plasma, was found comparable to both IFN-γ and IL-2 ELISPOT assays and also to a TCR-driven activation-induced marker (AIM) flow cytometry assay, carried out using cryopreserved PBMCs and identical S peptide libraries. However, concentrations of IFN-γ- and IL-2 correlated modestly with the frequency of reactive T cells in the ELISPOT assay (Rho=0.44 and Rho=0.57, respectively) and poorly (Rho <0.27) with the frequencies of AIM⁺ CD4⁺ and CD8⁺ T cells.

The current study has two major limitations. First, a relatively limited number of samples were subjected to analysis, in particular specimens retrieving negative results by either or both assays, although this was not unexpected as all participants had been vaccinated within 5 months before testing. In this regard, further studies should involve group populations with lower positive pre-test probability. Second, the FC-ICS assay used lacks standardization, and additionally, the possibility of misclassified SARS-CoV-2 infection status in some participants cannot be entirely ruled out. Despite the above, our data underscore that caution should be taken when comparing SARS-CoV-2 T-cell immune responses assessed using different platforms for analyses, and highlight an urgent need to develop quantitative and qualitative reference standards for calibration of T-cell immunoassays.

Author contribution

NT, EG, MM-N, EA, DN, IT: methodology, analysis of data, validation, review, and editing. CG and DN: conceptualization and supervision. DN: writing the original draft. All authors reviewed the final version of the manuscript.

Data availability

The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethical approval

The current study was approved by the Research Ethics Committee of La Fundación de Investigación del Hospital General Universitario de Valencia.

Informed consent

Informed consent was obtained from all participants.

Consent for publication

Not applicable.

Conflict of interest

The authors declare no competing interests.