Comparison of an In-house and a Commercial RD1-based ELISPOT-IFN-γ Assay for the Diagnosis of Mycobacterium tuberculosis Infection

Abstract

Objective: To compare a RD1-based in-house ELISPOT-interferon-γ (IFN-γ) assay with a commercial (T-SPOT.TB™) assay for the diagnosis of Mycobacterium tuberculosis (TB) infection and the efficacy of the tuberculin skin test (TST) and ELISPOT assay in detecting latent TB infection (LTBI).

Design: Eighty-six subjects (65 household contacts of contagious TB-infected patients, 13 subjects with active or previous TB infection, and 8 with suspected TB infection) were consecutively recruited in the context of a surveillance program.

Methods: Enrolled subjects underwent the Mantoux TST and two different ELISPOT-IFN-γ assays: an in-house assay using a pool of selected M. tuberculosis peptides (MTP) and the commercial T-SPOT.TB assay.

Results: The in-house and commercial ELISPOT-IFN-γ assays showed almost complete concordance (99%) in diagnosing acute or LTBI.When comparing the efficacy of the TST with the in-house ELISPOT assay in detecting TB infection, a small agreement was observed (k=0.344, P<0.0001): 36% of the subjects with a positive TST were ELISPOT-MTP negative and 12% with a negative TST were ELISPOT-MTP positive. Furthermore, 78% of the ELISPOT-MTP negative individuals were ELISPOT- Bacillus Calmette-Guérin (BCG) positive, most of whom had received BCG vaccination.

Conclusion: Our in-house ELISPOT assay based on a restricted pool of highly selected peptides is equivalent to the commercial T-SPOT.TB assay, is cheaper and is probably not confounded, unlike the TST, by BCG vaccination in our setting

The global resurgence of Mycobacterium tuberculosis (TB) infection, supported by the human immunodeficiency virus pandemic and emerging multi-drug resistance, underlines the need for new diagnostic and prevention measures and strategies.¹ Moreover, in industrialized countries, the growing numbers of immigrants from countries with endemic TB infection make it essential to be continuously vigilant for active and latent TB infection (LTBI).

The tuberculin skin test (TST) used to be the only available means of diagnosing LTBI, but its specificity was limited because the purified protein derivative used as antigen broadly cross-reacts with antigens derived from a number of mycobacterial species, including strains of Bacillus Calmette-Guérin (BCG) vaccine and nontuberculous mycobacteria (NTM).² This has led to the development of new in vitro tests that are based on detecting the interferon-γ (IFN-γ) released by activated T lymphocytes (T cells),³ and whose antigens are a number of secretory proteins coded by the RD1 genes of the M. tuberculosis complex but are absent from the majority of environmental isolates, including BCG strains.⁴ Two of these proteins, early secretory antigenic target 6 (ESAT-6) and culture filtrate protein 10 (CFP-10), and synthetic overlapping peptides corresponding to the full length of each were initially tested and elicited a strong T-cell response in subjects with active TB infection or LTBI.⁵

We have recently validated an in-house ELISPOT-IFN-γ assay based on a pool of highly immunogenic peptides derived from ESAT-6 and CFP-10 proteins that encompass multiple epitopes⁶ in subjects classified as having TB infection. The test proved to be highly specific (88%) and sensitive (93%). In order to validate it further, we compared our in-house assay with the commercial T-SPOT.TB^™ assay⁷ in a group of subjects with documented acute TB infection or who are at high risk of developing it. We also compared its efficacy in detecting LTBI against the TST.

Material and Methods

Participants

Eighty-six subjects were consecutively recruited within the context of a surveillance program at Villa Marelli Institute, Milan, Italy, Lombardy’s Regional Reference Center for Tuberculosis, between October 2004 and January 2005. All subjects underwent previous informed consent, the Mantoux TST, our in-house ELISPOT-IFN-γ assay, and the commercial T-SPOT.TB assay (Oxford Immunotec, Oxford, UK; currently approved for use in Europe and awaiting US Food and Drug Administration approval).^6–8 The origins and characteristics of the patients and their history of BCG vaccination (documented by charts in 15 individuals and proved by a scar in 23 individuals) are shown in table 1 ▶. All subjects were classified for TB infection on the basis of the 5-point scale of the American Thoracic Society (ATS).⁹ Sixty-five (76%) individuals were household contacts of contagious (sputum smear-positive) TB-infected patients and were at high risk for LTBI. Of the 86 subjects, 15 had no evidence of previous infection and had a negative reaction to TST (ATS class 1), 50 had a suspected TB infection and a positive TST reaction without history of disease (ATS class 2), 6 had culture-confirmed active pulmonary TB infection (ATS class 3), 7 had a history of TB episodes but showed no signs of clinically active TB infection (ATS class 4), and 8 were awaiting diagnosis (ATS class 5).

Table 1.

Characteristics of the enrolled subjects.

Origin	Number	M/F	Age range (years)	Household contacts	BCG vaccinated	TST positive
Africa
Libya, Senegal	2	2/0	20–29	2	2	2
Asia
Philippines	5	2/3	13–51	3	5	4
Central and South America
Ecuador, Peru	26	13/13	8–69	19	22	23
Eastern Europe
Rumania, Russia, Ukraine	7	3/4	0.5–60	6	5	7
Western Europe
Italy	46	26/20	2–93	35	4	33
Total	86	46/40	0.5–93	65	38	69

M/F, males/females; BCG, Bacillus Calmette-Guérin; TST, tuberculin skin test.

They all underwent a standard TST in which 5 tuberculin units (0.1 ml) of purified protein derivative were administered intradermally, and the size of induration was evaluated after 48 to 72 hours by highly trained staff. In the case of the household contacts, the TST reaction was considered positive when induration was ≥5 mm (≥10 mm for the other classes).²

Ex Vivo ELISPOT-IFN-γ Assay

Peripheral blood mononuclear cells (PBMCs) were obtained from each subject by means of blood centrifugation over a density gradient, divided into two aliquots, and were tested under double-blind conditions in separate laboratories.

The first laboratory (San Raffaele Scientific Institute) used the in-house ELISPOT assay as previously described.⁶ Briefly, the PBMCs (2 x 10⁵ cells/well) were seeded in duplicate in 96-well plates (MAIPS4510, Millipore, Bedford, MA) precoated with an anti-IFN-γ capture monoclonal antibody (mAb) (clone B-B1, Diaclone, Besançon, France). A pool of six >70% purified, 20 amino acid-long synthetic M. tuberculosis peptides (MTP) (Primm Srl, Milan, Italy) derived from the sequences of the ESAT-6 and CFP-10 proteins were used at a final concentration of 2 μg/ml/peptide to detect a specific response (ESAT-6 protein peptide_71–90 was added to the original pool of five peptides). Heat-inactivated and sonicated whole-cell BCG (Organon Teknika, Boxtel, The Netherlands) at a dilution of 1:10 was used as the control antigen. The PBMCs, in medium alone or stimulated with 5 μg/ml of phytohemagglutinin (Sigma, St. Louis, MO), were respectively used as negative and positive controls. After overnight incubation at 37°C, 5% CO₂ and washing, the cells were incubated with biotinylated anti-IFN-γ detection mAb (clone B-G1, Diaclone) for 4 hours followed by streptavidin-alkaline phosphatase conjugate (Amersham Pharmacia, Freiburg, Germany) for 1 hour and then by the chromogenic substrate (nitroblue tetrazolium-5-bromo-4-chloro-3-indolylphosphate [BCIP]) for 10 to 15 minutes, after which the plates were washed with tap water. All the comparisons between the TST and the different antigens used for assessing TB positivity were made on the data obtained using the in-house assay.

The second laboratory (Villa Marelli Institute) used the commercial T-SPOT.TB assay in accordance with the procedures recommended by the manufacturer. The PBMCs were seeded at 2.5 x 10⁵ cells/well in single-well plates,⁷ and two separate pools of overlapping peptides spanning the full length of ESAT-6 and CFP-10 proteins were used to detect a specific response (International Application published under the terms of the Patent Cooperation Treaty WO 02/054072 A2, 11 July 2002), together with negative controls (cells alone) and positive controls (cells plus phytohemagglutinin). The individual spots were counted using an automated image analysis system, ELISPOT reader (AID-GmbH, Strassberg, Germany).

The responses to the in-house assay were considered positive if the test wells contained a higher mean number of spot-forming cells (SFC) than the mean number plus two standard deviations in the negative control wells and when the number of SFCs per million PBMCs in the stimulated wells (minus the values of the negative control wells) was ≥20 (in our experience with PBMCs of healthy subjects, the mean number of SFC per million PBMCs plus two standard deviation was ≤20 in negative control wells).⁶ The responses to the commercial assay were considered positive when the number of spots in the test wells (against at least one of the two tested antigens) was ≥6 times the number of spots in the negative control wells or, if the negative control had more than 6 SFCs, at least twice that number. The discrepancy between the cutoff for the two tests could depend on the different reagents and materials used in each assay (e.g., capture and detection mAbs, different concentrations of the reagents, type of plates) and their optimization.

Statistical Analysis

The data were analyzed using SAS software (version 8.2, SAS Institute, Cary, NC). A two-sided alpha level of 0.05 was used to determine statistical significance in all the analyses. The differences in the mean number of MTP-specific SFCs between the subjects in TB infection classes 1 through 5 (1 vs. 2 vs. 3+4 vs. 5) were compared using the Kruskal-Wallis test. McNemar’s test of symmetry for paired data was used to test the equality of the off-diagonal frequencies obtained when comparing two different methods of assessing TB positivity (TST, ELISPOT-MTP, ELISPOT-BCG). The agreement of the different tests was assessed by estimating Cohen’s Kappa (k) statistic and the corresponding 95% confidence interval. According to Fleiss, k values of >0.75 suggest excellent agreement beyond that determined by chance, values of >0.4 and <0.75 indicate fair to good agreement, and values of <0.4 indicate poor agreement.¹⁰ The receiver operating characteristics curve was calculated to measure the ability of the TST to correctly classify the subjects who were positive and negative to ELISPOT-MTP (measured on the basis of the area under the curve), and in order to identify the optimal TST cutoff point that minimized classification errors.

Results

In-house and Commercial ELISPOT-IFN-γ Assay

Sixty-nine of the 86 subjects (80%) showed a positive TST reaction (figure 1 ▶ and table 1 ▶): 33 from Italy, where the incidence of TB infection is low and BCG vaccination is not compulsory (only four had a BCG scar or documentation) and 36 from countries in which TB infection and NTM are endemic and vaccination is extensively used at least at birth (34 had a BCG scar or documentation). Forty-six (53%) showed a positive response to the in-house ELISPOT assay and 45 (52%) to the commercial assay (99% concordance: k=0.977, 95% CI 0.931–1.022, P=ns). In one subject, the in-house assay was borderline positive (20 SFCs per million PBMCs), and the commercial assay was negative. The responses to ESAT-6 or CFP-10 (T-SPOT.TB) and to the MTP pool (in-house assay) are shown in figure 2 ▶ in terms of the number of SFCs per million PBMCs.

Figure 1.

TST, ELISPOT-MTP and ELISPOT-BCG results. TST, tuberculin skin test; ELISPOT-MTP, in-house ELISPOT-IFN-γ assay using a pool of M. tuberculosis specific peptides (MTP) as antigen; ELISPOT-BCG, in-house ELISPOT assay using BCG as antigen; LTBI = latent TB infection.

Figure 2.

Box and whisker plot of SFCs per million PBMCs in relation to TB classification and the antigens used. The boxes indicate the median, 25^th and 75^th quartiles, and minimum and maximum values. The ESAT-6 and CFP-10 peptide pools were used in the commercial assay. The MTP pool was used in the in-house assay as specific M. tuberculosis antigens. Class 1 n=15, class 2 n=50, class 3+4 n=13, and class 5 n=8.

Analysis of the subjects by TB classification revealed significant between-class differences in the mean number of SFCs (P=0.0001 for ESAT-6, P=0.0007 for CFP-10 and P=0.0003 for MTP), with the highest values being observed in classes 3+4 and 5. Of note, after 1 year from recruitment, two class 5 contacts (one from Italy and one from Ecuador) developed an active TB infection (both were ELISPOT-MTP positive, while one had a TST reaction =5 mm and the second a TST reaction =21 mm); five contacts (all Italian) did not develop an active TB infection (all had a TST reaction >10 mm, three were ELISPOT-MTP positive, and one ELISPOT-MTP negative); and one individual was not a contact but developed a lymphadenitis due to a M. avium complex infection with a positive (borderline) ELISPOT-MTP response and a TST reaction <5 mm.

Frequency of ELISPOT-MTP Responses in Relation to TST Reactivity and ELISPOT-BCG Responses

Forty-four of the 69 TST-positive subjects (64%) showed a positive response to the in-house ELISPOT-MTP assay, as did two of the 17 TST-negative subjects. The remaining 15 TST-negative subjects (88%) were ELISPOT-MTP negative (figure 1 ▶). It is worth noting that the 25 subjects (36%) who were TST positive and ELISPOT-MTP negative had probably received a BCG vaccination or were infected with NTM (11 of 38 subjects with a BCG scar or documentation did not respond to ELISPOT-MTP, whereas all but one responded to ELISPOT-BCG). The agreement between the ELISPOT-MTP assay and TST reactivity was quite poor (k=0.344, 95% CI 0.175–0.513), and the test of symmetry was highly significant (P<0.0001).

Because a TST cutoff ≥5 mm for individuals with a history of previous BCG vaccination could be questionable, we tried to correlate the frequencies of ELISPOT-MTP responses in relation to a TST cutoff ≥10 mm. In this case, 19 subjects changed from positive to negative, thus making the TST positive and ELISPOT-MTP negative responses 14 of 50 (28%), instead of the initial 25 of 69 (36%).

Sixty-seven of the 70 subjects tested (96%) responded to ELISPOT-BCG. These included all the 30 subjects who were ELISPOT-MTP positive, whereas 31 of 40 ELISPOT-MTP negative subjects (78%) were positive to ELISPOT-BCG, thus corroborating the idea that most of them may have been vaccinated or infected with NTM (figure 1 ▶). The agreement between ELISPOT-MTP and ELISPOT-BCG was poor (k = 0.09, 95% CI 0.009–0.190), and the test of symmetry was highly significant (P<0.0001).

Correlation of ELISPOT-MTP Responses with TST Induration

When the subjects were divided into subgroups on the basis of whether they responded positively to TST, there was a significant difference between them in terms of the number of SFCs per million PBMCs in the ELISPOT-MTP assay (TST negative: median=0, q1=0, q3=15; TST positive: median=67, q1=5, q3= 423; P<0.0001). There was also a significant difference in TST induration (expressed in mm) between the subjects with a negative or positive ELISPOT-MTP response (MTP negative: median=6.5; q1=0, q3=13.5; MTP positive: median=16, q1=11, q3=20; P<0.0001; figure 3 ▶).

Figure 3.

Box and whisker plot of TST induration (in mm) in relation to ELISPOT-MTP response. The boxes indicate the median, 25^th and 75^th quartiles, and minimum and maximum values.

The analysis of the ability of TST induration to discriminate negative and positive ELISPOT-MTP responses revealed an area under the curve of 81% (95% CI 76–90%, P<0.0001) and showed that the best cutoff point was 9 mm.

Considering that BCG vaccination may contribute to the poor specificity of TST,² we calculated the frequencies of ELISPOT-MTP responses in relation to TST reactivity in BCG vaccinated individuals (table 2 ▶). Depending on the TST cutoff, the subjects were divided into three groups, TST <5 mm, TST ≥5 and <10 mm, and TST ≥10 mm. Only one subject with a negative TST reaction was also negative to the ELISPOT-MTP assay, while 8 of the 34 individuals (24%) with a TST reaction ≥10 mm were ELISPOT-MTP negative, confirming the lower sensitivity of TST compared to the ELISPOT assay.

Table 2.

Frequencies of ELISPOT-MTP responses in relation to TST reactivity in BCG vaccinated subjects.

	Number (%) of responses that were ELISPOT-MTP
TST result	Negative	Positive
<5 mm	1 (100%)	0
5–10 mm	2 (67%)	1 (33%)
≥ 10 mm	8 (24%)	26 (76%)

Discussion

The results of this study show nearly complete concordance (99%) between our in-house ELISPOT-IFN-γ assay and the commercial T-SPOT.TB assay in terms of the diagnosis of TB infection. Moreover, our assay was more accurate than the TST in identifying subjects with suspected LTBI (the close contacts of people with contagious TB infection).

The in-house assay is based on a restricted and highly selected pool of synthetic peptides containing multiple epitopes derived from ESAT-6 and CFP-10 proteins.^5,6,11,12 The possibility of false-negative results and the consequent underestimate of infected individuals therefore cannot be excluded, mainly because of human leukocyte antigen restrictions and the limited number of peptides.¹³ However, the very high concordance between the results of our assay and the commercial assay based on two separate pools of peptides spanning the full length of the same proteins⁷ confirms that our selected peptides are sufficient to detect acute TB infection and may be useful for identifying subjects with suspected latent infection. Furthermore, the calculated cost of approximately $6.36 per patient for disposables and reagents is almost 70% less than the current Italian costs of the commercial assay ($44.50 per patient).

TST is currently the only test routinely used to diagnose LTBI in people living in developing countries where TB and NTM are endemic and BCG vaccination is common practice.¹⁴ However, as its specificity is low because it cross-reacts with BCG vaccine strains and NTM,^2,9 new and specific diagnostic and prevention strategies are essential. It is difficult to imagine the large-scale application of ELISPOT methodology in TB-endemic countries where resources are limited, because it does not distinguish acute and latent infection. However, in industrialized countries with large immigrant populations, it could be useful for screening purposes, especially in the case of the close contacts of individuals with acute TB infection.^15,16 ELISPOT assays may also allow the more sensitive testing of healthcare workers by discriminating truly infected people from those who have received BCG-vaccinations or who show nonspecific TST reactions (e.g., hypersensitivity to heterologous proteins).¹⁷

Seventy-six percent of the subjects enrolled in this study were household contacts of contagious TB infected patients and nearly half of them came from developing countries (85% had a detectable BCG scar or documentation confirming previous vaccination, although it must be noted that not all vaccinated individuals develop a detectable scar).¹⁸ Thirty-seven of the 38 subjects with a detectable scar had a positive (≥5 mm) TST reaction, but 10 showed a negative ELISPOT-MTP response. Using a cutoff ≥10 mm (because a TST cutoff ≥5 mm for contacts who have previously received BCG vaccination could be questionable), 8 of 34 subjects with a positive TST reaction were still negative to ELISPOT-MTP, thus confirming the greater specificity of MTP and stressing the higher sensitivity of the ELISPOT assay compared to TST. As contacts with a positive TST reaction currently undergo preventive chemoprophylaxis,^2,19 our assay offers a real advantage by better targeting interventions to truly infected individuals and helping to avoid unnecessary treatment.

More generally, the agreement between TST and our in-house ELISPOT assay was quite poor, as 36% of the TST-positive subjects did not respond to MTP. Moreover, the sensitivity of our assay was confirmed by the fact that 100% of the subjects positive to ELISPOT-MTP were also positive to ELISPOT-BCG. Conversely, 78% of the MTP-negative subjects were ELISPOT-BCG positive, which is in line with the high incidence of BCG-vaccinated subjects among contacts (some of whom could also be infected with NTM).

The ability of TST to discriminate positive and negative ELISPOT-MTP responses was tested by means of the receiver operating characteristic curves, which showed a close relationship between the latter and the size of TST-induced induration. The area under the curve was 81%, and the best cutoff point was about 9 mm, suggesting that the conventional 10 mm threshold (and not the 5 mm threshold currently used for contacts) should be considered a positive reaction in subjects at high risk of TB infection.^2,19

As previously reported,⁶ evaluation of single-cell IFN-γ production (expressed as the number of SFCs) showed a close relationship with the TB classification with the highest values being found in classes 3+4 and 5. Moreover, as also previously reported,^20,21 the number of SFCs was significantly different between the subjects with positive and negative TST reactions. These results seem to show a close correlation between exposure to M. tuberculosis and the quantitative response to ELISPOT-MTP and, although larger-scale longitudinal studies are required to confirm this finding, it is possible that it could be used to discriminate between acute and LTBI.

Conclusion

Our reduced but highly selected pool of peptides from ESAT-6 and CFP-10 proteins is sufficient to detect subjects with acute TB infection. Our assay is equivalent to the commercial T-SPOT.TB assay, is cheaper, and it is probably not confounded, unlike the TST, by BCG vaccination in our setting. Thus, RD1-based ELISPOT assays are more accurate than TST in identifying LTBI as recently reported^20–23 and may help to avoid unnecessary treatments.

Acknowledgments

The authors wish to thank the following public health nurses for collecting the patients’ histories and carefully reading all the TSTs: Mrs. G. Brambilla, Mrs. M. Oddone and Mrs. P. Seno, and Oxford Immunotec for kindly providing free T-SPOT.TB kits.