Comparison Between Screening and Confirmatory Serological Assays in Blood Donors in a Region of South Italy

Abstract

Background

Screening assays are needed in order to guarantee safety of donated blood, but a significant number of safe donations are removed from blood supply because of reactive screening results. It is important to evaluate the positive predictive value (PPV) of screening assays in order to modulate confirmatory algorithm and implement an adequate counseling.

Methods

An analysis of 17,912 blood donations has been conducted at Transfusion Medicine at Second University Naples, Italy, in 2009–2012. Serological screening for syphilis, hepatitis B virus (HBV), hepatitis C virus (HCV), and human immunodeficiency virus (HIV) was performed by ARCHITECT (Abbott Diagnostics, Wiesbaden, Germany); repeatedly reactive (RR) samples were checked by respective confirmatory tests. The relationship between sample/cutoff and confirmed seropositivity were analyzed.

Results

RR rates were low as expected in blood donors: 0.47% for syphilis, 0.42% for HBV, 0.50% for HCV, and 0.15% for HIV. The specificity on RR + gray zone (GZ) was 99.67%, 99.79%, 99.77%, and 99.88%, respectively; due to the low prevalence, PPV value was 30.6% for syphilis, 50.7% for HBV, 42.2% for HCV, and 18.5% for HIV. These values increased substantially reaching a plateau of 89.3% for syphilis, 94.6% for HBV, 85.7% for HCV, and 100% for HIV at the threshold established by receiver operating characteristics curve analysis.

Conclusions

Supplemental testing on samples with high signal by screening assays seems to add little information. GZ settings and confirmatory testing for positive screening results should be designed taking in account several factors, including difference in the natural history among blood‐borne infections, the characteristics of first‐ and second‐level tests, and, when available, the results of nucleic acid amplification testing.

INTRODUCTION

Blood transfusion is an essential component of health care that worldwide saves millions of lives each year. Lack of access to sufficient quantities of safe blood may compromise the obvious right of patients to health care. One of the most prominent factors in ensuring safe blood samples is to have a national program for donor selection, recruitment, retention, and education to minimize donations from donors who might transmit diseases to recipients 1.

Currently, blood transfusion centers in Italy use serological methods for routine screening of blood donation for transfusion‐transmissible infections that include Treponema pallidum (syphilis), hepatitis B virus (HBV), hepatitis C virus (HCV) and human immunodeficiency viruses 1 and 2 (HIV 1/2) as per mandatory law 2, 3, 4. The selection of appropriate assays is a critical part of the screening program 1 and currently enzyme immunoassays (EIA) and chemiluminescent immunoassays (CMIA) are the most frequently employed screening technologies.

All assays should have a high level both of sensitivity and specificity 5, 6, 7. In particular, increase in test sensitivity of transfusion‐transmitted infections is highly desirable to ensure recipient safety. However, in a low prevalence population such as blood donors viral screening assays have relatively low positive predictive values (PPVs) and biological false‐positive results in blood donors are problematic due to both component loss and donor management issues 8. Then, there is a need for a more standardized approach to the screening of blood donors with the aim of minimizing the number of biological false‐positive screening test results. Since biological false‐positives occur for a variety of reasons, confirmatory tests are necessary. The practice recommended by the assay manufacturers and health authorities with regard to a positive screening test results is to repeat the serological screening twice by the same assay and to proceed with a confirmatory test for repeatedly reactive (RR) samples 2, 3, 9, 10, 11.

The aim of our study was to analyze the seroprevalence of transfusion‐transmissible infectious agents on voluntary blood donors in the period 2009–2012 and verify the specificity and accuracy of CMIA screening test followed by confirmatory assays, with the additional results of nucleic acid amplification testing (NAT). In addition, we aimed to calculate the PPV of the screening assays according to sample/cutoff (S/CO) ratio in order to minimize the number of false‐positives and set up an appropriate algorithm for confirmatory testing and donor counseling.

MATERIALS AND METHODS

We carried out a retrospective analysis of blood donor data recorded between 2009 and 2012 at the Immunohematology, Transfusion Medicine and Immunology of the Second University of Naples, Italy. The study was conducted on 17,912 voluntary blood donors, including both first‐time donors and repeated donors, all apparently healthy subjects of both sexes, aged 25–60 (average: 42.5 ± 24.7). Demographic information on the donors (age and gender) was accessed from the standard blood donor questionnaire form of the Central Blood Transfusion Service (CBTS) recorded by health professionals from the blood donors. Samples were tested with confidentiality and identified by the sample code number given during the sample collection. Ethical approval to conduct the study was obtained from CBTS.

Primary Screening and Testing Algorithms

Blood samples were tested on the ARCHITECT platform for syphilis, HBsAg, anti‐HCV and HIV 1–2 antigen, and antibodies using the respective CMIA assays (Abbott Diagnostics, Wiesbaden, Germany). The precision of each CMIA was evaluated by using the commercial negative and positive control reagents recommended by manufacturer. Each internal control was run twice for each session analysis and at least one external quality control sample was included on each run. All assays being qualitative, S/CO results ≥1.00 are considered as initial reactive (IR). In our setting and to maximize the donation safety, samples yielding S/CO ratios between 0.70 and 0.99 are scored as gray zone (GZ) and subjected to the same protocol of IR that includes repeatedly testing in duplicate by CMIA after centrifugation followed by a confirmatory assay on RR samples. The confirmatory tests used were as follows: for syphilis, the T. pallidum hemoagglutination assay (TPHA; Omega Diagnostics, UK), for HBsAg, the HBsAg qualitative II neutralization assay (Abbott), and for HCV and HIV, two specific immunoblot assays (INNO‐LIA, Innogenetics, Ghent, Belgium). Furthermore, all blood donations were screened also by NAT for HBV‐DNA, HCV‐RNA, and HIV‐RNA by the TaqScreen method on the Cobas s201 system (Roche Molecular Systems, Branchburg, NJ): the assay has been performed on mini pools of six samples each and has a nominal sensitivity of <20 international units per milliliter (IU/mL).

Data Analysis

All data were entered in an Excel spreadsheet and analyzed by commercial statistical software (Analyse‐it, Analyse Ltd., Birmingham, UK). The statistical analysis included an ROC (receiver operating characteristics) curve analysis on the cumulative results based on the S/CO values in order to evaluate the overall accuracy and the optimal threshold value for a confirmed positivity. A separate evaluation of the PPV and specificity of the screening assays at different threshold values ranging from GZ to high‐level reactivity was subsequently performed.

RESULTS

Syphilis

We observed 85 RR (0.47%), of whom 10 were GZ (0.06%) and 75 over the assay cutoff (0.41%). Supplemental testing by TPHA confirmed the CMIA reactivity in 26 cases (30.6% of RR), none of whom was GZ reactive. By ROC curve analysis (Fig. 1A), the area under the curve (AUC) was 0.94 (95% confidence interval [CI] 0.92–0.96) and the best threshold was at 4.35 S/CO, yielding a sensitivity of 92.3%, a specificity of 93.2%, and an overall accuracy of 86.0%. The ensuing specificity for all RR results according to the current screening policies, for all results above the cutoff, and for all results above the ROC curve threshold were 99.67% (95% CI 99.50–99.75), 99.73% (95% CI 99.65–99.80), and 99.98% (95% C.I. 99.96–100), respectively, and the PPV increased from 30.6% by the current algorithm to 89.3% when the ROC threshold was applied (Table 1).

Figure 1.

ROC curve analysis for the S/CO ratios of CMIA screening assays for syphilis (A), HBsAg (B), and anti‐HCV (C). The arrows and the numbers indicate the threshold values with the highest overall accuracy. HIV results are not shown because of the very low number (five) of true‐positive results.

Table 1.

Results of Screening and Confirmatory Testing for Syphilis, HBsAg, Anti‐HCV, HIV, and Correlation With NAT Results on 17,912 Blood Donations

	Classification	N	%	IND	FP	TP	TN	Specificity (%)	95% CI	PPV (%)	NAT+ (%)
Syphilis	S/CO 0.70–0.99	10	0.06	n.a.	10	0	17,902			0
	S/CO 1.00–4.35	47	0.26	n.a.	46	1	17,865			2.1
	S/CO >4.35	28	0.16	n.a.	3	25	17,884	99.98	99.96–100%	89.3
	S/CO >1.00	75	0.41	n.a.	49	26	17,837	99.73	99.65–99.80%	34.7
	Total RR	85	0.47	n.a.	59	26	17,827	99.67	99.50–99.75%	30.6
HBsAg	S/CO 0.70–0.99	27	0.15	n.a.	27	0	17,885			0	0
	S/CO 1–100	11	0.06	n.a.	8	3	17,901			27.3	0
	S/CO > 100	37	0.21	n.a.	2	35	17,875	99.99	99.97–100%	94.6	78.4
	S/CO > 1.00	48	0.27	n.a.	10	38	17,864	99.94	99.91–99.98%	79.2	56.2
	Total RR	75	0.42	n.a.	37	38	17,837	99.79	99.73–99.86%	50.7	36.0
Anti‐HCV	S/CO 0.70–0.99	19	0.11	2	16	1	17,893			5.9	0
	S/CO 1.00–2.50	39	0.22	13	21	5	17,873			19.2	0
	S/CO >2.50	31	0.17	3	4	24	17,881	99.98	99.96–100%	85.7	29.0
	S/CO >1.00	70	0.39	16	25	29	17,842	99.86	99.81–99.91%	53.7	12.9
	Total RR	89	0.50	18	41	30	17,823	99.77	99.70–99.84%	42.2	10.1
HIV	S/CO 0.70–0.99	9	0.05	0	9	0	17,903			0	0
	S/CO 1–70	14	0.08	0	13	1	17,898			7.1	7.1
	S/CO > 70	4	0.02	0	0	4	17,908	100	99.95–100%	100	100
	S/CO > 1.00	18	0.10	0	13	5	17,894	99.93	99.89–99.97%	27.8	27.8
	Total RR	27	0.15	0	22	5	17,885	99.88	99.83–99.93%	18.5	18.2

S/CO, sample to cutoff ratio; RR, repeatedly reactive; N, number; IND, indeterminate; FP, false‐positive; TP, true‐positive; TN, true‐negative; 95% CI, 95% confidence interval; PPV, positive predictive value; NAT, nucleic acid amplification testing; n.a., not available.

HBsAg

Seventy‐five RR were recorded (0.42%), of whom 27 were GZ (0.15%) and 48 over the assay cutoff (0.27%). The HBsAg neutralization assay confirmed the CMIA reactivity in 38 cases (50.7% of RR), none of whom was GZ reactive. By ROC curve analysis (Fig. 1B), the AUC was 0.96 (95% CI 0.92–1.00) and the best threshold was at 100 S/CO, yielding a sensitivity of 94.7%, a specificity of 94.6%, and an overall accuracy of 89.6%. The ensuing specificity for all RR results according to the current screening policies, for all results above the cutoff, and for all results above the ROC curve threshold were 99.79% (95% CI 99.73–99.86), 99.94% (95% CI 99.91–99.98), and 99.99% (95% CI 99.97–100), respectively, and the PPV increased from 50.7% by the current algorithm to 94.6% when the ROC threshold was applied. NAT testing for HBV‐DNA yielded a positive result in 29 cases, none of whom were GZ, corresponding to 56.2% of all samples over the cutoff and 78.4% of samples with a S/CO value ≥100 (Table 1).

Anti‐HCV

The highest number of RR specimens was recorded for anti‐HCV (89 or 0.50%), of whom 19 were GZ (0.11%) and 70 over the assay cutoff (0.39%). The immunoblot assay confirmed the CMIA reactivity in 30 cases (33.7% of RR), one of whom was GZ reactive (S/CO = 0.99). Eighteen specimens were indeterminate by immunoblot and thence excluded from ROC curve analysis and from the calculations of specificity and PPV. By ROC curve analysis (Fig. 1C), the AUC was 0.86 (95% C.I. 0.78–0.95) and the best threshold was at 2.50 S/CO, yielding a sensitivity of 80.0%, a specificity of 90.2%, and an overall accuracy of 72.2%. The ensuing specificity for all RR results according to the current screening policies, for all results above the cutoff, and for all results above the ROC curve threshold were 99.77% (95% CI 99.70–99.84), 99.86% (95 CI 99.81–99.91), and 99.98% (95% CI 99.96–100), respectively, and the PPV increased from 42.2% by the current algorithm to 85.7% when the ROC threshold was applied. By NAT testing, the presence of HCV‐RNA was confirmed only in nine specimens (10.1%), all with S/CO values over the ROC‐established threshold (Table 1).

HIV

Twenty‐seven specimens (0.15%) were RR of whom 9 were GZ (0.05%) and 18 over the assay cutoff (0.10%). The immunoblot assay confirmed the CMIA reactivity in five cases that were also positive for HIV‐RNA and showed high S/CO levels (≥69). The ROC curve is not shown because its reliability is questionable due to the very low number of positive samples; results indicated an AUC of 1.00 (95% CI not available) and the best threshold at 69.0 S/CO, yielding both a sensitivity and a specificity of 100%. The specificity for all RR results according to the current screening policies, for all results above the cutoff, and for all results above the ROC curve threshold were 99.88% (95% CI 99.83–99.93), 99.93% (95 CI 99.89–99.97), and 100% (95% CI 99.95–100), respectively, and the PPV increased from 18.5% by the current algorithm to 100% when the ROC threshold was applied (Table 1).

An overall summary of the cumulative screening results obtained by the current algorithm is depicted in Figure 2.

Figure 2.

Laboratory algorithm for syphilis, HBsAg, anti‐HCV, and HIV testing on 17,912 blood donors.

DISCUSSION

Our data confirm the low prevalence of transfusion‐transmitted infections in Italian blood donors as previously reported 4, 12, 13. As a consequence, like in all other populations with a low prevalence of syphilis, HBV, HCV, and HIV infections, the PPV of the screening assays is poor and a relevant number of uninfected donations are excluded based on a reactive serological result probably linked to a biological false positivity (BFP) with the additional temporary or permanent deferral of blood donors resulting in overall drop in potentially available donors 8, 10.

These results cannot be ascribed to a poor performance of the serological screening assays, as reported in our experience where CMIA screening assays were compared with the results of the confirmatory tests that are mandatory according to the current law 2, 3. From our data the reproducibility was quite good, since the average coefficients of variation of S/CO values after repeat testing ranged from 3.45% for HBsAg to 9.61% for HIV (data not shown). The specificity was also in line with our expectations, as the rates for samples equaling or exceeding the preset cutoff value were equal or better than declared in the respective package inserts (99.73% vs. 99.67% for syphilis, 99.94% vs. 99.91% for HBsAg, 99.86% vs. 99.77% for anti‐HCV, 99.93% vs. 99.89% for HIV).

The usefulness of a confirmatory algorithm was confirmed by the low PPV of each screening assay. In an attempt to ensure a higher PPV, higher thresholds for the S/CO values were considered. This approach has been employed in many previous studies on anti‐HCV, and most of them have indeed demonstrated the relationship between high S/CO values and serological confirmation of anti‐HCV positivity 9, 14, 15, 16, 17 and even with HCV viremia 14, 18. Thus, a higher S/CO ratio for anti‐HCV is deemed to guarantee a higher PPV, but the data on the other screening assays are less common 10, 11, 19, and we believe that ours is one of the first systematic attempts in this direction, although it has been carried out on a relatively small number of blood donations.

Our results indicate that it is possible to define S/CO values for each ARCHITECT assay with a PPV that nears 100%, as already demonstrated by Kiely et al. 10 and by Acar et al. 19 for other assay systems. However, if these S/CO values were used to discriminate confirmed positive samples from BFP without confirmatory testing, a substantial percentage of specimens would be incorrectly classified. The misclassification rate is not the same across assays: it would be highest for anti‐HCV, whose AUC was the poorest (0.86) and lowest for HIV, though the very limited number of positive samples in our study (five) does not allow a definite conclusion. A separate issue is the serological confirmation of T. pallidum antibodies: while no specific rules are in place, we decided to use a second treponemal assay based on a different technology from the screening test, but this approach may have guaranteed a good specificity to the detriment of sensitivity, as agglutination assays for syphilis have shown false negative results 20.

For the three viral infections, another factor to be considered is the availability of screening results for HBV, HCV, and HIV nucleic acid testing. Molecular assays have become integrated into the routine practice of blood donors for the management of viral infections. NAT screening has been shown to reduce the residual risk of infection transmission by detecting potentially infectious HBV, HCV, and HIV during the window period before seroconversion and also in the late phases of infection and in breakthrough infections 21, 22, 23, 24, and allows also a better appraisal of the donor status, especially for HCV infection. From our data, we have observed NAT positivity rates, that is, active infections, of 0.16% for HBV, 0.05% for HCV, and 0.03% for HIV, only on samples with a high S/CO ratio by serological assays and never on GZ specimens. While NAT is currently not available in all blood transfusion settings in Europe or other major countries, such as India or China, the alternate approach for the donations that are initially reactive by serological assays will still be to carry out supplemental testing.

Our results indicate that, in agreement with previous experiences, confirmatory testing could be limited only to samples yielding an S/CO result lower than a value predetermined as having a PPV of 95% 9, 15, 25, while donations with higher S/CO would be deemed as confirmed positive without additional testing. However, the risk remains that BFP donations could yield a high S/CO ratio, as we observed in two of our cases for anti‐HCV, because even for the assay that gave the better analytical results (HBsAg) there is an overlap between BFP and confirmed positive samples. Alternatively, a secondary serological screening assay may be used on RR specimens for HCV and confirmatory testing can be carried out only on samples that are RR by both assays.

In conclusion, the ARCHITECT assays are specific and suitable as screening tests. From an operative standpoint, a 10% GZ around the cutoff value shall guarantee an adequate level of sensitivity without compromising the specificity. Confirmatory assays can be discontinued on strongly reactive specimens or when NAT screening results are available, while in settings not yet adopting NAT a serological confirmation is useful for a better counseling of blood donors.