Validation of an interferon-gamma enzyme-linked immunosorbent spot assay to evaluate cell-mediated immunity against severe acute respiratory syndrome coronavirus 2

ABSTRACT

The COVID-19 pandemic forced the rapid development of methods to measure humoral and cellular immunity against SARS-CoV-2. The lack of a global standardized protocol and the high variability of intra- and inter-assay precision of the T-cell response made it difficult to compare T-cell assay results with those of other laboratories. The interferon-gamma enzyme-linked immunosorbent spot (IFN-γ ELISpot) assay for immunogenicity evaluation was validated using naturally infected donor peripheral blood mononuclear cells, a commercially available IFN-γ ELISpot kit, and a SARS-CoV-2 specific peptide pool. Depending on anti-CD3 and peptide pool stimulation, the mean coefficients of variation (CVs) of the intra-assay precision were 19.0% and 13.4%, respectively. The mean CVs of the inter-assay precision were 26.1% and 25.4%, and the mean CVs for reproducibility were 6.7% and 15.9%, respectively. Linearity with an R-squared value between 0.98 and 0.99 was established, and the mean CVs between the lots were 17.6% and 6.6%, depending on the anti-CD3 and peptide pool stimulation, respectively. The limit of detection was 11 spot-forming counts per well. Taken together, we demonstrated that the IFN-γ ELISpot assay is feasible for evaluating SARS-CoV-2-specific cell-mediated immune function through validation based on standard operating procedures.

Introduction

Since the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) outbreak in 2019 and the subsequent global pandemic and variant reinfections, a comprehensive assessment of humoral and cellular immune responses was conducted for COVID-19-naïve, -recovered, and vaccinated individuals.^1–3 Despite the reduced neutralizing antibody activities of the wild-type COVID-19 vaccines to the COVID-19 variants, the vaccines reduced the severity and hospitalization rate of COVID-19 variant infected individuals.⁴ Furthermore, the longer persistence of SARS-CoV-2-specific T cells compared with that of the neutralizing antibody may provide prolonged protection against SARS-CoV-2.⁵ In addition to neutralizing antibody titers, the T cell response has become a main concern in evaluating SARS-CoV-2 immunity. The enzyme-linked immunosorbent spot (ELISpot) assay, intracellular cytokine staining assay, activation induced marker assay, cytokine secretion assays, and peptide-MHC tetramer staining assay have been used to evaluate the T cell response induced by COVID-19 infection or vaccination.⁶ Of these, the ELISpot assay is one of the most widely used assay to measure cell-mediated immunity in COVID-19 research.^2,7

The IFN-γ ELISpot assay is widely used to detect cellular immune responses in individuals vaccinated or infected with several pathogens, including HIV, tuberculosis, and influenza.^8–10 Compared with serological studies, relatively more factors, such as the viability of peripheral blood mononuclear cells (PBMCs), the type of stimulant antigens, concentration of the stimulant, ELISpot kit, and incubation time affect the ELISpot assay, which makes it difficult to standardize or validate the assay. There have been some recent efforts to validate ELISpot assays for influenza and SARS-CoV-2.^1,3,11 In their study, the sensitivity and specificity of the IFN-γ ELISpot was greater than 80.0% and 72.0%, respectively, and showed linearity with an R-value greater than 0.94. The %CV value for the intra-assay was approximately 21.0% and the range of %CV for the inter-assay was 12.0%–41.0%. Nonetheless, there is no global standard protocol for the ELISpot assay and how to set the acceptance criteria for detection limits, precision, linearity, and accuracy are optional for each laboratory.

In this study, the IFN-γ ELISpot assay was validated according to the standard operating protocol (SOP) prepared in our laboratory to evaluate SARS-CoV-2-specific T-cell immune responses using PBMCs from individuals who have recovered from COVID-19 infection.

Materials and methods

Participants

To validate the IFN-γ ELISpot assay, 29 PBMC samples from 27 donors were selected from the YUMC-COVID-R02 study cohorts (Yeungnam University Medical Center Institutional Review Board, IRB No. YUMC-2020-04-009), a longitudinal immunogenicity evaluation study of individuals who recovered from the first outbreak of COVID-19 in Korea.¹² To determine the sensitivity, specificity, and accuracy of the IFN-γ ELISpot assay, 30 COVID-19-recovered individuals (IRB No. YUMC-2020-04-009) and 15 pre-pandemic participants (IRB No. YUMC-2018-09-045) were evaluated. COVID-19-recovered participants showed evidence of a positive RT-PCR test for SARS-CoV-2 previously and consented to the secondary use of their samples for research studies. Detailed donor information is summarized in Supplementary materials Table 1 and 2.

Table 1.

Determination of LOD.

Sample no. (n=29)	Duplicate mean SFCs/well	Duplicate mean SFCs (log10 transformed)
1	0.5	-0.301
2	0.6	-0.222
3	0.2	-0.699
4	8.3	0.919
5	2.1	0.322
6	1.2	0.079
7	0.4	-0.398
8	0.9	-0.046
9	1.1	0.041
10	2.3	0.362
11	3.6	0.556
12	3.6	0.556
13	15.2*	1.182*
14	6.1	0.785
15	8.4	0.924
16	2.7	0.431
17	1.6	0.204
18	1.6	0.204
19	4.9	0.690
20	1.8	0.255
21	1.5	0.176
22	0.0	-1.000
23	0.3	-0.523
24	5.8	0.763
25	3.8	0.580
26	1.0	0.000
27	0.0	-1.000
28	4.0	0.602
29	1.0	0.000
Mean		0.152
Std. deviation		0.535
Mean + 1.645×Std.Dev**		1.032
LOD (Anti-log of above)		10.8

SFCs: spot-forming counts; LOD: limit of detection.

* This value was excluded from the LOD calculation as an outlier.

** This value represents the 95th percentile upper distribution limit.

Table 2.

Sensitivity, specificity, and accuracy of the IFN-γ ELISpot.

ELISpot assay	real-time PCR test			Sensitivity (%, 95%CI)	Specificity (%, 95%CI)	Accuracy (%, 95%CI)
	Positive	Negative	Total
Positive	23	1	24	76.7 (57.7–90.1)	93.3 (68.1–99.8)	82.2 (68.0–92.0)
Negative	7	14	21
Total	30	15	45

PBMC isolation

PBMCs were isolated based on the YUMC SOP for the ELISpot assay (YUMC_A_17). Briefly, PBMCs were isolated by isopycnic centrifugation using a Sepmate™ tube (STEMCELL Technologies Inc., 86415, 86450), based on the manufacturer’s instructions and frozen at a concentration of 5–10 million cells/mL. The PBMCs were isolated within 6 h of whole blood collection and stored in liquid nitrogen until use.

Stimulation conditions

The overlapping peptide pool of the wild-type SARS-CoV-2 membrane (M), nucleocapsid (N), and spike (S) proteins (SARS-CoV-2 peptide pool) were used as PBMC stimulants at a final concentration of 0.16 µg/mL each (Miltenyi Biotec, 130-126-703, 130-126-698, 130-126-701), in a preliminary concentration-dependent study (data not shown). The anti-CD3 monoclonal antibody (Anti-CD3, Mabtech AB, 3605-1-1000) was used as a positive control. The background conditions included PBMCs cultured in medium only. Each condition was tested in duplicate and the mean responses were reported.

IFN-γ ELISpot assay

SOP for the IFN-γ ELISpot was established in our laboratory based on the manufacturer’s protocol for the human IFN-gamma ELISpot PRO kit (Mabtech AB, 3420-2AST-2). Pre-coated plates were rehydrated with 200 µL/well DPBS and blocked with 200 µL of culture medium [Roswell Park Memorial Institute (RPMI)1640 containing 10% FBS] at room temperature (RT) for at least 30 min. While blocking, the PBMC vials were thawed, centrifuged at 1,400 rpm for 10 min, resuspended, and washed with DPBS with 2% FBS. The PBMCs were stained using the Muse count and viability kit (Luminex Corporation, MCH100102) counted using an automated cell counting analyzer (Muse™ cell analyzer, EMD Millipore Corporation, 0500–3115). PBMCs with viability ≥ 89% were used in the ELISpot assay. The PBMC samples were diluted to a concentration of 2.5 × 10⁶ live PBMCs/mL in serum-free RPMI 1640 medium (Lonza, BE12-702F). The blocking solution was removed from the assay plates, 50 µL of the stimulant (anti-CD3 or SARS-CoV-2 peptide pool) was added to each well, and 100 μL of the PBMCs were added to each well at a final cell density of 2.5 × 10⁵/well. The plates were incubated overnight (18–24 h) in an incubator at 37°C in 5% CO₂. After incubation, the cell suspensions were removed and the plates were washed with DPBS. Next, 100 µL of detection antibody was added and the plates were incubated for 2 h at RT. After washing, substrate (100 µL, 0.22 μm filtered) was added to each well for 12 min to visualize the spots. Spot development was stopped using tap water wash and the plates were air-dried overnight at RT in the dark. The spots were enumerated within 3 days after development using an automated spot counter (ImmunoSpot® S6, Cellular Technology Limited, S6VRS12), followed by manual verification of the identified spots. The parameters for counting are presented in the Supplementary material Table 3.

Statistical analysis

Statistical analyses were performed using GraphPad Prism version 9.5.1 software (GraphPad Software). The sensitivity, specificity, and accuracy were measured using MedCalc Software version 20 (MedCalc Software). The number of cells that responded specifically to the antigen was expressed as spot-forming counts (SFCs) and calculated by subtracting the negative control (PBMCs cultured in medium only) spot count from the antigen-stimulated spot count. Statistical outliers were removed using the interquartile range (IQR) rule: Q3–Q1 was multiplied by 1.5 and either added to Q3 or subtracted from Q1. Any number above or below these limits was considered a statistical outlier. To determine the limit of detection (LOD), the mean mock response data collected from 29 PBMC samples were used and the Shapiro – Wilk test was used to evaluate the normal distribution of the data. The upper 95% distribution limit for the LOD was calculated as follows: the antilog of the [grand mean + (1.645 × standard deviation [SD]) of log10 transformed values]. For the intra and inter assay assessments, the percent coefficient of variation (%CV) was calculated from the SD divided by the mean for a given dataset and multiplied by 100. The coefficients of correlation (r²) and regression curves were obtained using linear regression calculations of the paired values. The acceptance criterion was set at the mean 30% CV for replicate measurements as described in white paper.¹³

Results

Determination of the minimum PBMC functional requirements for the IFN-γ ELISpot assay

Anti-CD3 induces IFN-γ secretion in T cells. Thus, a minimum anti-CD3 response serves as a quality control for PBMC responses. To determine the minimum anti-CD3 response, the anti-CD3 response distribution from 87 values from the linearity, intra- and inter-precision, and lot-to-lot variation experiments was determined. The anti-CD3 responses ranged from 175.5 to 1,240.5 SFCs/well, but were not normally distributed by the Shapiro – Wilk test (Figure 1). Thus, the minimum anti-CD3 response was empirically set at the lower 1st percentile (175.5 SFCs/well). The limit was selected to achieve a ∼1% failure rate for the assay based on the guidance of the US Food and Drug Administration (FDA) for immunogenicity testing of therapeutic protein products.¹⁴

Figure 1.

Determination of minimum PBMC functionality required for the ELISpot assay. The distribution of 87 anti-CD3 responses was used to establish minimum PBMC functionality, which was empirically set to the lower 1st percentile.

Abbreviations: PBMC, peripheral blood mononuclear cell; SFCs, spot-forming counts.

Determination of the limit of detection (LOD)

The LOD is the lowest detectable analyte level at which positive and nonspecific responses can be reliably distinguished. To determine the LOD, we analyzed the background signals of 29 mock-stimulated (medium only) PBMCs based on linearity, precision, and repeatability experiments. The data from one donor were considered statistical outliers (interquartile range method) and excluded from the analysis. Data from the remaining 28 PBMC samples were transformed into log10, which resulted in a normal distribution, using the Shapiro – Wilk test. The LOD was derived as the 95th percentile, anti-log transformed, and rounded to the nearest whole number, resulting in 11 SFCs per well (44 SFCs per million PBMCs) (Table 1).

Assay sensitivity, specificity, and accuracy

We observed the sensitivity, specificity, and accuracy of the IFN-γ ELISpot assay compared with the viral nucleic acid test results, which is the gold standard for the confirmative diagnosis of SARS-CoV-2 infection. Using PBMCs from 30 COVID-19 convalescent donors and 15 pre-pandemic donors, the sensitivity, specificity, and accuracy of the IFN-γ ELISpot assay for the SARS-CoV-2 specific peptide pool were 76.7, 93.3, and 82.2%, respectively (Table 2).

Linearity

To determine the linearity range, PBMCs were seeded at various densities in plate. The assay was performed in duplicate at five cell densities (ranging from 31,250 to 500,000 cells/well) using PBMCs from two moderate-to-high responders to the anti-CD3 and SARS-CoV-2 peptide pools. The linear range was defined as the range of cell densities yielding a linear regression correlation coefficient (r²) >0.90. Cell densities with responses less than the LOD were excluded from the analysis. The linear ranges were 31,250–500,000 cells/well for anti-CD3 and 125,000–500,000 cells/well for the SARS-CoV-2 peptide pool for both PBMC samples (Figure 2, Table 3). SFCs were directly proportional to the number of PBMCs, with a linear relationship of r² >0.98 for the anti-CD3 and SARS-CoV-2 peptide pool for both donors.

Figure 2.

Determination of linearity. Cell densities are plotted on the x-axis, and the mean SFCs/well response, expressed as Log₁₀ transformed values, are plotted on the y-axis. The linear range (yellow area) is defined as the range of cell densities yielding a linear regression correlation coefficient (R²) >0.90 and the responses to the anti-CD3 (a) and SARS-CoV-2 peptide pools (b) were >LOD. Correlation coefficients are shown for each donor, with each symbol ( and ) representing a different donor.

Abbreviations: PBMC, peripheral blood mononuclear cell; SFCs, spot-forming counts; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; LOD, limit of detection.

Table 3.

Validation of linearity.

Donor No.	Stimulant	Coefficient of correlation (r2)	Linear range (cells/well)
1	Anti-CD3	0.9863	31,250-500,000
	SARS-CoV-2 peptide pool	0.9966	125,000-500,000
2	Anti-CD3	0.9868	31,250-500,000
	SARS-CoV-2 peptide pool	0.9897	125,000-500,000

SARS-CoV-2: Severe acute respiratory syndrome coronavirus 2.

Intra-assay precision

To assess the variability between replicate wells, the IFN-γ spot forming responses induced by the anti-CD3 and the SARS-CoV-2 peptide pool were measured in five donor samples. Each sample was tested five times in duplicate in a single run, which resulted in 10 measurements per sample. The mean responses from each of the five donor samples ranged from 213.2 to 718.5 SFCs/well and from 24.8 to 118.5 SFCs/well after stimulation with the anti-CD3 and SARS-CoV-2 peptide pools, respectively (Table 4, Figure 3a). When stimulated with anti-CD3, the %CV range was 11.8%–24.9% (mean, 19.0%). When stimulated with the SARS-CoV-2 peptide pool, the %CV range was 5.8%–20.4% (mean, 13.4%) (Table 4, Figure 3a). The overall intra-assay mean %CV was 16.2% (Table 4, Figure 3a). These results met the acceptance criteria (CV ≤ 30%), indicating that the intra-assay variability was acceptable.

Table 4.

Validation of intra-assay precision.

PBMC stimulant	Donor No.	Quintuplicates mean SFCs/well	%CV	Mean of %CV	Overall Mean of %CV
Anti-CD3	1	213.2	11.8	19.0	16.2
	2	270.6	22.8
	3	609.7	24.9
	4	718.5	12.9
	5	216.2	22.4
SARS-CoV-2 peptide pool	1	36.0	20.4	13.4
	2	24.8	5.8
	3	51.9	18.8
	4	118.5	16.0
	5	59.2	6.0

SFCs: spot-forming counts; %CV: percent coefficient of variation; SARS-CoV-2: Severe acute respiratory syndrome coronavirus 2.

Figure 3.

Distribution of %CV based on the SFCs of intra- and inter-assays.

The distribution the mean %CV values for intra-assay (a), inter-day (b), and inter-operator (c) validation. Open circles represent the mean %CV values calculated by mean SFCs for replicate measurements when stimulated with peptide pools. Black circles represent the mean %CV values calculated by mean SFCs when stimulated with anti-CD3. The red dotted line indicates the overall mean %CV and the blue dotted line indicates the acceptable criteria (30%).

Abbreviations: %CV, percent coefficients of variation; SFCs, spot-forming counts.

Inter-assay precision and reproducibility

For inter-assay precision and reproducibility, five donor samples were tested in duplicate by two operators on three different days, which resulted in 12 measurements per sample. The inter-day %CV range was 13.6%–42.9% and 21.0%–31.7% for the anti-CD3 and the SARS-CoV-2 peptide pools, respectively (Table 5, Figure 3b). The inter-day %CV means were 26.1% and 25.4% for the anti-CD3 and SARS-CoV-2 peptide pools, respectively (Table 5, Figure 3b). The overall inter-day mean %CV was 25.8%, which met the acceptance criteria (CV ≤ 30%) (Table 5, Figure 3b). Reproducibility was determined by the mean responses obtained from the same samples analyzed by the two operators. The reproducibility range was 1.2%–8.9% and 10.5%–21.6% for the anti-CD3 and the SARS-CoV-2 peptide pool, respectively (Table 5, Figure 3c). The %CV means were 6.7% and 15.9% for the anti-CD3 and SARS-CoV-2 peptide pool, respectively (Table 5, Figure 3c). The overall mean %CV was 11.3% for reproducibility, which met the acceptance criteria (CV ≤ 30%) (Table 5, Figure 3c). The results demonstrate the consistency and precision of the IFN-γ ELISpot assay for the anti-CD3 and SARS-CoV-2 peptide pools.

Table 5.

Validation of inter-assay precision and reproducibility.

PBMC stimulant	Donor No.	Operator	Inter-day (Inter-assay)			Inter-operator (Reproducibility)
			Mean SFCs/well	%CV	Mean of %CV	Triplicates mean SFCs/well	%CV	Mean of %CV
Anti-CD3	1	A	788.3	13.6	26.1	781.8 794.8	1.2	6.7
		B
	2	A	408.0	42.9		431.0 385.0	8.0
		B
	3	A	533.3	18.2		562.5 504.2	7.7
		B
	4	A	742.6	16.4		783.2 702.0	7.7
		B
	5	A	418.4	39.5		444.7 392.2	8.9
		B
SARS-CoV-2 peptide pool	1	A	88.0	24.4	25.4	96.3 79.7	13.4	15.9
		B
	2	A	51.5	21.0		55.3 47.7	10.5
		B
	3	A	38.2	22.9		44.0 32.3	21.6
		B
	4	A	71.9	31.7		79.2 64.7	14.3
		B
	5	A	26.0	26.9		29.7 22.3	19.9
		B
Overall mean of %CV					25.8			11.3

SFCs: spot-forming counts; %CV: percent coefficient of variation; SARS-CoV-2: Severe acute respiratory syndrome coronavirus 2.

Lot-to-Lot variation

To confirm whether there was a difference between the lots used in the ELISpot kit, two different lots were tested three times in duplicate using five donor PBMCs. The mean SFCs, variance (%CV), and mean %CV values between the lots for each donor were obtained. When stimulated with anti-CD3, the %CV range was 12.1%–20.5%, and the %CV mean was 17.6%, within the acceptance criteria of ≤ 30% (Table 6). When stimulated with the SARS-CoV-2 peptide pool, the %CV range was 1.0%–12.3%, and the %CV mean was 6.6%. The overall mean %CV was 12.1%, which was far below the acceptance criterion of ≤ 30% (Table 6).

Table 6.

Validation of lot-to-lot variation of ELISpot kit.

PBMC stimulant	Donor No.	Lot No. of kit	Triplicates mean SFCs/well	%CV	Mean of %CV	Overall mean of %CV
Anti-CD3	1	1	265.0 207.0	17.4	17.6	12.1
		2
	2	1	298.8 228.8	18.8
		2
	3	1	648.5 493.8	19.1
		2
	4	1	1167.7 983.2	12.1
		2
	5	1	270.7 202.0	20.5
		2
SARS-CoV-2 peptide pool	1	1	82.3 73.7	7.9	6.6
		2
	2	1	65.8 55.3	12.3
		2
	3	1	98.0 96.7	1.0
		2
	4	1	37.0 41.8	8.7
		2
	5	1	76.0 79.2	2.9
		2

SFCs: spot-forming counts; %CV: percent coefficient of variation; SARS-CoV-2: Severe acute respiratory syndrome coronavirus 2.

Discussion

There is an increasing need for cell-mediated immunity assays to evaluate the immune response induced by infection or vaccination. The ELISpot assay is no longer limited to research or exploratory purposes as it has become a widely used cell-mediated immunoassay for preclinical and clinical analyses.¹³ Currently, ELISpot analysis has been used for COVID-19 vaccine development and evaluation²; however, the diversity of materials and protocols used in each laboratory has made it difficult to establish a standard protocol and compare the results. Furthermore, several factors, such as PBMC viability, frequency of memory T cells in aliquot vials of PBMC, and technical skill of experimenters influence the variability of the ELISpot assay. Therefore, the variability of the intra- and inter-assay precision is challenging for the ELISpot assay as a tool for evaluating immunity, and defining the performance criteria remains incomplete. To evaluate T cell responses against SARS-CoV-2, we developed an ELISpot assay SOP and validated the assay based on industry white papers, consortium guidelines, FDA guidance documents, and publications.^11–13–19

The ELISpot assay SOP was developed through a preliminary test of several conditions that may contribute to the variability of the assay before validation. This included the following: 1) the overall procedure to maintain the quality of the PBMCs, such as PBMC isolation, handling, and storage; 2) the conditions for the ELISpot reaction, including cell density, culture medium, type of stimulant, and antigen stimulation time; 3) the interpreting the results, range of the ELISpot counts, and system suitability.

To ensure the quality of the PBMCs applicable for the ELISpot assay, two criteria were established regarding PBMC functionality: cell viability and PC (anti-CD3) response. First, PBMCs with ≥ 89% cell viability were subjected to the ELISpot assay based on our previous experience and the reference.²⁰ Second, the minimal PC response was determined as 177.5 SFCs/well when calculated for 87 samples as the lower 1st percentile, according to the FDA guidelines.¹⁴ This value was similar to that of our previous data, in which we empirically set the lower PC limit at 200 SFCs/well for PBMCs with a viability of 89% or higher (data not shown).

The ELISpot assay is very sensitive. Therefore, setting the detection limit is important. In a study by Barabas et al., a cutoff of 10 spot-forming units per well with 200,000 PBMCs was established for the human IFN-γ ELISpot assay to cytomegalovirus infections using 45 healthy donors.²¹ Other studies have also indicated that IFN-γ ELISpot LODs range from 12 to 15 spots per well in both human and nonhuman primates.^1,22,23 These LOD results are similar to those of our results (11 SFCs/well) calculated using 28 convalescent PBMC background signals. Because we established the LOD using PBMCs of subjects recovered from natural infection, it is appropriate for clinical studies.

The sensitivity, specificity, and accuracy of the IFN-γ ELISpot assay were 76.7%, 93.3% and 82.2%, respectively. The blood samples were collected at 22–165 days (median 80.5 days) after initial symptom onset, which indicates the possibility of waning T-cell immunity in some donor; thus, the sensitivity may be relatively low. Lin et al. evaluated the IFN-γ ELISpot by combining S, M, and N peptide pools. The sensitivity was 59.7%–79.9%, the specificity was 82.1%–96.4%, and the accuracy was 63.8%–81.9%.¹ It showed a wide range depending on the combination of peptides and the time of blood collection following diagnosis. T cell responses specific to the SARS-CoV-2 spike proteins, N, and M were reported in individuals not exposed to SARS-CoV-2, but resulting from other common cold coronavirus infections.^24–26 Braun et al. found that 35% of healthy donors in Germany developed CD4+ T-cell responses to a pool of SARS-CoV-2 spike protein peptides.²⁴ Le Bert et al. detected responses to N and nonstructural proteins in at least 30% of healthy donors in Singapore.²⁵ Grifoni et al. found that 40%–60% of CD4+ T cells in prepandemic samples responded to SARS-CoV-2 peptides.²⁶ When we used 15 prepandemic PBMCs, 6.7% of the donors were positive for the ELISpot assay. Our previous study also demonstrated that 21.3% of COVID-19-naïve individuals who had neither COVID-19 symptoms nor were diagnosed with COVID-19 during April 2020 ~ May 2020 had T-cell responses to the SARS-CoV-2 peptide pool in their PBMCs (data not shown). This indicates a limitation in confirming specificity because of the cross-reactivity observed with previous common cold coronavirus infections. Because most individuals have prior immunity to SARS-CoV-2 by natural infection or vaccination, spot count fold-changes for pre- and postvaccination samples are more important than sensitivity and specificity when evaluating vaccine immunogenicity.

In cell-based immunoassays, optimal cell-to-cell contact is required for sufficient stimulation. Therefore, confirming the appropriate number of PBMCs within a linear range is necessary. In the present study, the cell numbers in the linear range were 31,250–500,000 cells/well for the anti-CD3 pool and 125,000–500,000 cells/well for the SARS-CoV-2 peptide pool, with r² >0.98. Therefore, 250,000 cells/well were considered suitable for the IFN-γ ELISpot assay against SARS-CoV-2.

Compared with the small molecule assay, a higher CV is acceptable for cell-based immune assays, such as the ELISpot assay.^13,14 The laboratories set the acceptance criteria for ELISpot validation between 20% and 40%.^11,22,23 The guidelines published in the 2020 and 2022 white papers suggested that the intermediate %CV must be ≤ 40%.^13,15 In the present study, the acceptance criterion for validation was set as a mean %CV ≤ 30% by referring to the GCC white paper and Clinical and Laboratory Standards Institute document, which in consistent with several previous reports.^13,22,23,27 The results of IFN-γ ELISpot assay validation indicated that the mean intra- and inter-assay precision, and reproducibility for the anti-CD3 and SARS-CoV-2 peptide pool were all within the acceptance criteria.

Recent studies indicate that lot-to-lot variation significantly impacts the accuracy, precision, and overall performance of immunoassays.²⁸ In the present study, the lot-to-lot mean %CVs were lower than 20% in both the anti-CD3 and SARS-CoV-2 peptide pools. Although the intra-assay was performed on a single plate and the lot-to-lot variation assay was performed with two different lot plates, the precision of the lot-to-lot variation was lower compared with that of the intra-assay with the SARS-CoV-2 peptide pool. These unanticipated results may be caused by the variation in memory cell frequencies in the PBMC aliquot vials. For the lot-to-lot assay, one vial of PBMCs was sufficient for each donor; however, the intra-assay was repeated five times for each donor and more than two vials of PBMCs were used. Smith et al. reported that several aliquots from one donor represent a diverse readout.²⁹ Although there were no harmonized acceptance criteria, the %CV values for lot-to-lot variation in the present study were mostly below 20%. This confirmed the robustness of the IFN-γ ELISpot assay by showing low variation values, even when different ELISpot kit batches were analyzed.

This study had several limitations. First, the sample size was small (two donors) in the linearity validation. Second, we had difficulty obtaining sufficient numbers of PBMCs from donors, which limited the number of repeat measurements. Third, there were no validation results for low immune response because the test was conducted on subjects with an intermediate or high immune response. However, empirically, SARS-CoV-2-specific T-cell immune responses were sufficiently detectable even at low responses below 20 and above the LOD value (11 SFCs/well).

Overall, our validation data suggest that the IFN-γ ELISpot assay, when performed with a well-prepared SOP, can reliably analyze the cell-mediated response to SARS-CoV-2. This study provides a foundation to support the standardization of the ELISpot assay. Further research is need to confirm its routine use as a validated method for assessing vaccine efficacy in individuals vaccinated against SARS-CoV-2.

Biographies

Yunhwa Kim, currently works in Department of the Microbiology, College of Medicine, Yeungnam University in Daegu. Her research interests focus on the host immune responses toward viral infection and vaccination. She has a special interest in infections and vaccinations caused by SARS-CoV-2 and Varicella-Zoster Virus, and has extensive experience analyzing T cell and antibody responses in clinical trials.

Hosun Park is a professor at the Department of Microbiology, College of Medicine, Yeungnam University in Daegu, Republic of Korea. She graduated from the College of Medicine, Hanyang University, and obtained her MD and PhD. She is interested in the pathogenesis and immunity of viral diseases. Her main research area is the evaluation of viral vaccine immunogenicity. She has developed and validated several immunogenicity methods to evaluate varicella and zoster vaccines, and a COVID-19 vaccine. She is currently the director of the clinical trial center at Yeungnam University Medical Center and runs an Immunogenicity Evaluation Laboratory.

Funding Statement

This work was supported by the Ministry of Food and Drug Safety under Grant [No. 21172MFDS181] and GC BioPharma corporate.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Credit authorship contribution statement

Yunhwa Kim: methodology, formal analysis, validation, visualization, writing of the original draft. Eun-Jeong Jang: methodology, formal analysis, resources, validation Ji-Young Hwang: resources, formal analysis; Kyung-Min Lee: resources, formal analysis. Sivilay Xayaheuang: resources, formal analysis. Seok-Tae Choi: resources, formal analysis; Hosun Park: conceptualization, investigation, resources, funding acquisition, and writing. All the authors have read and agreed to the published version of the manuscript.

Declaration of generative AI in scientific writing

No AI technology was used in the scientific writing.

Data availability statement

Data supporting the findings of this study are available from the corresponding author, H.P, upon reasonable request.

Institutional review board statement

This study was conducted in accordance with the guidelines of the Declaration of Helsinki and was approved by the Institutional Review Board of Yeungnam University Medical Center (IRB No. YUMC-2018-09-045, YUMC-2020-04-009).

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2024.2428516