Clinical Performance Evaluation of Four Automated Chemiluminescence Immunoassays for Hepatitis C Virus Antibody Detection

Abstract

Various automated chemiluminescence immunoassay (CLIA) analyzers for the detection of antibodies to hepatitis C virus (HCV) are now commercially available in clinical laboratories and are replacing conventional enzyme immunoassays. We investigated the performance of four anti-HCV CLIAs (the Architect Anti-HCV assay on the Architect i2000 system, the Vitros Anti-HCV assay on the Vitros ECiQ Immunodiagnostic System, the Access HCV Ab PLUS assay on the UniCel DxI 800 analyzer, and the newly developed Elecsys Anti-HCV assay on the Cobas e 411 analyzer). The total percent coefficient of variation values of imprecision were 3.5 to 5.7% with positive control materials and 7.2 to 10.2% with negative control materials. The agreement between the results of the Elecsys, Architect, Vitros, and Access CLIAs ranged from 94.5 to 98.1%. The clinical sensitivity of all CLIAs was 100%. Each CLIA showed excellent reproducibility and clinical sensitivity. The Elecsys, Architect, Vitros, and Access CLIAs showed clinical specificities of 98.2, 98.8, 96.5, and 98.2%.

Hepatitis C virus (HCV), first identified in 1989, is an enveloped positive-strand RNA virus classified in the Hepacivirus genus in the family Flaviviridae (6). The HCV genome is about 9.5 kb in length and encodes 3,011- to 3,033-amino-acid polypeptides in structural and nonstructural regions (20). The structural region contains the core protein and two envelope proteins (E1 and E2), and nonstructural proteins have been assigned protease (NS2, NS3, and NS4A), helicase (NS3), and RNA-dependent RNA polymerase (NS5B) (21) functions.

The first commercially available anti-HCV enzyme immunoassay (EIA) used a single HCV recombinant antigen derived from the nonstructural NS4 protein designated c100-3 (19). The sensitivity of this first-generation EIA was low for a high-prevalence population (approximately 80%) and showed a high false-positive rate (up to 70%) in a low-prevalence blood donor group (13). Therefore, a second-generation EIA was developed and approved for use by the Food and Drug Administration (FDA) in 1992 (3). The second-generation EIA, which contained additional HCV antigens from the core (c22-3) and NS3 (c33c) proteins, showed increased sensitivity and specificity and shortened the average seroconversion period from 16 to 10 weeks (1, 3, 13, 18). The third-generation EIA, which added a fourth antigen (NS5), showed significantly improved performance, particularly for high-risk patients (2, 8). However, a residual risk still exists due to the seroconversion period of approximately 56 days, and high false-positive rates were not resolved (12). The Centers for Disease Control and Prevention (CDC) recommended that an anti-HCV screening test positive result be verified by a more specific supplemental assay such as recombinant immunoblot or nucleic acid test (5). To facilitate the use of the supplemental assay, the revised guideline included an option for reflex supplemental testing based on signal-to-cutoff (s/co) ratios (4).

Today, automated chemiluminescence immunoassay (CLIA) analyzers are widely used, particularly in high-volume clinical laboratories. These instruments offer excellent precision and reliability, high-speed throughput, random access, and the technical simplicity of full automation. CLIA showed significantly improved specificity, a greater positive predictive value, and a similar sensitivity compared to those of EIA for detecting anti-HCV antibodies (10, 15). Although automated CLIAs are gradually replacing the EIA, there are no published studies on the comparative evaluation of automated CLIAs (10, 15, 16, 22, 27). We compared the performance of currently marketed anti-HCV automated CLIAs under routine conditions of a hospital laboratory.

MATERIALS AND METHODS

Assay systems.

Four automated CLIAs were compared, the Elecsys Anti-HCV assay on the Cobas e 411 analyzer (Roche Diagnostics, Mannheim, Germany), the Architect Anti-HCV assay on the Architect i2000 system (Abbott Laboratories, Abbott Park, IL), the Vitros Anti-HCV assay on the Vitros ECiQ Immunodiagnostic System (Ortho-Clinical Diagnostics, Raritan, NJ), and the Access HCV Ab PLUS assay (Bio-Rad Laboratories, Redmond, WA) on the UniCel DxI 800 analyzer (Beckman-Coulter, Fullerton, CA). The characteristics of the four reagents and the technical specifications of each instrument are summarized in Table 1.

TABLE 1.

Characteristics of four automated anti-HCV antibody assays

Reagent	Analyzer	Manufacturer	HCV antigen				Assay principle	Solid phase	Labeled substance	Sample vol (μl)	Time of reaction (min)	Gray zoneb
			Core	NS3	NS4	NS5
Elecsys Anti-HCV	Cobas e 411	Roche Diagnostics	Present	Present	Present	Absent	ECLIAa	Magnetic particle	Ruthenium complex	40	18	0.9-1.0
Architect Anti-HCV	Architect i2000	Abbott Laboratories	Present (HCr43)	Present (HCr43)	Present (c100-3)	Absent	CLIA	Paramagnetic particle	Acridinium	20	29	Not indicated
Vitros Anti-HCV	Vitros ECiQ	Ortho-Clinical Diagnostics	Present (c22-3)	Present (c200)	Present (c200)	Present	CLIA	Well	Luminol derivative	20	56	0.9-1.0
Access HCV Ab PLUS	UniCel DxI 800	Bio-Rad Laboratories/Beckman-Coulter	Present	Present	Present	Absent	CLIA	Paramagnetic particle	Lumi-Phos 530	25	55	0.9-1.0

^aECLIA, electrochemiluminescence immunoassay.

^bShown is the s/co ratio of the gray zone requiring retest according to the manufacturer.

Precision tests of four CLIAs.

The reproducibility of each CLIA was determined by using a modified form of the EP5-A2 protocol of the Clinical and Laboratory Standards Institute (7). Experiments were performed twice a day for 10 days in duplicate with negative and positive quality control materials recommended by each manufacturer.

CLIA screening.

A total of 400 consecutive unselected fresh serum samples sent daily to our laboratory underwent anti-HCV testing with four CLIAs. We prospectively collected HCV-positive samples screened by Genedia HCV ELISA 3.0 (Greencross Life Science, Seoul, Korea) from December 2007 to April 2008. These sera, which had been stored frozen at −20°C, were then tested with the four CLIAs and for HCV RNA. However, there were limitations in sample volume; the Vitros and Access assays were performed with only 127 and 140 samples, respectively. The four anti-HCV CLIAs were carried out according to the manufacturers' instructions.

Confirmation of results.

Samples showing positive results from any CLIA were investigated by medical record review and confirmatory testing. The medical record review involved the examination of any clinical or laboratory evidence of chronic HCV infection. These samples were investigated for the presence of clinical conditions that could interfere with the accuracy of CLIA results such as autoimmune disease, dialysis, pregnancy, bacterial infection, and rheumatoid factor.

Measurements of HCV RNA were performed with samples with undetermined HCV status by medical record review and any discrepancy in the CLIA results. HCV RNA was detected by the COBAS Amplicor HCV 2.0 qualitative assay and the COBAS TaqMan HCV assay (both from Roche Diagnostics). In the COBAS Amplicor HCV 2.0 assay, the viral genome was extracted with the HCV Specimen Preparation Kit; details of the assay have been described elsewhere (25). The lower detection limit of the COBAS Amplicor HCV 2.0 assay is 50 IU/ml. In the COBAS TaqMan HCV assay, the viral genome was extracted by automated COBAS AmpliPrep instruments and HCV RNA was amplified and detected with the COBAS TaqMan Analyzer (24). The lower detection limit of the COBAS TaqMan HCV assay is 15 IU/ml.

Recombinant immunoblot assays (RIBA) were additionally performed with samples with HCV RNA-negative results by using LG HCD Confirm (LG Life Sciences, Seoul, Korea) according to the manufacturer's instruction.

RESULTS

Precision.

The precision of each CLIA was evaluated by using the commercial negative and positive control materials recommended by each manufacturer. Each control was run twice a day for 10 days in duplicate (40 runs in 10 days). Precision results for each CLIA are shown in Table 2. Intra-assay and total percent coefficient of variation values for positive controls ranged from 2.8 to 4.7% and 3.5 to 5.7%, respectively. Due to low mean values for the negative control, intra-assay and total percent coefficient of variation values for negative controls ranged from 5.5 to 9.1% and 7.2% to 10.2%, respectively.

TABLE 2.

Intra-assay and total assay precision results of each CLIA

Reagent and sample	Grand mean (s/co)	SDa (%CV)b
		Intra-assay	Total
Elecsys
NCc	0.134	0.011 (7.8)	0.014 (10.2)
PCd	9.58	0.451 (4.7)	0.543 (5.7)
Architect
NC	0.074	0.007 (9.1)	0.006 (8.3)
PC	3.21	0.115 (3.6)	0.129 (4.0)
Vitros
NC	0.232	0.015 (6.5)	0.023 (9.9)
PC	5.53	0.173 (3.1)	0.191 (3.5)
Access
NC	0.097	0.005 (5.5)	0.007 (7.2)
PC	3.06	0.084 (2.8)	0.138 (4.5)

^aSD, standard deviation of s/co ratio.

^bCV, coefficient of variation of s/co ratio.

^cNC, negative control material.

^dPC, positive control material.

Comparison of CLIA results.

The overall correlation of the results of the four CLIAs is shown in Table 3. The agreement between the results of the Elecsys, Architect, and Vitros CLIAs ranged from 97.1 to 98.1%. The agreement between the results of the Access CLIA and the other CLIAs ranged from 94.5 to 97.0%.

TABLE 3.

Results of comparison of each anti-HCV CLIA for a clinical specimen

Reagent and result	No. of samples negative, no. positive or % agreement between two CLIAs (no. of identical results/total)
	Elecsys (n = 585)	Architect (n = 585)	Vitros (n = 527)	Access (n = 325)
Elecsys
Negative		425, 10	413, 11	168, 14
Positive		7, 143	3, 100	4, 139
Architect	97.1 (568/585)
Negative			414, 8	167, 13
Positive			2, 103	5, 140
Vitros	97.3 (513/527)	98.1 (517/527)
Negative				165, 1
Positive				7, 94
Access	94.5 (307/325)	94.5 (307/325)	97.0 (259/267)
Negative
Positive

There was a limitation on sample volume; the Access CLIA could not be performed with every sample, and the percent agreement ranged from 94.5 to 97.0%, slightly lower than that of the others. Among 267 samples assayed by all of the CLIAs (Table 4), the overall concordance rate was 94.0% (251/267). Sixteen samples showing discrepant results were confirmed negative by either the COBAS Amplicor HCV 2.0 qualitative assay or the COBAS TaqMan HCV assay for detection of HCV RNA. Furthermore, an additional confirmatory RIBA was performed with 14 of 16 HCV RNA-negative samples; the remaining 2 samples could not be confirmed by RIBA due to lack of sample volume. Ten samples were RIBA negative, and four were RIBA indeterminate.

TABLE 4.

Comparison of 267 samples assayed by four CLIA methods

No. (%) of samplesc	CLIA result				RIBA result(s) (n)
	Elecsys	Architect	Vitros	Access
160 (59.9)	Negative	Negative	Negative	Negative	NTa (160)
91 (34.1)	Positive	Positive	Positive	Positive	NT (91)
4 (1.5)	Negative	Negative	Positive	Negative	Negative (3), INDb (1)
3 (1.1)	Positive	Negative	Negative	Negative	Negative (2), NT (1)
2 (0.7)	Negative	Positive	Negative	Negative	Negative (1), NT (1)
2 (0.7)	Negative	Positive	Positive	Negative	Negative (1), IND (1)
1 (0.4)	Negative	Negative	Negative	Positive	Negative (1)
1 (0.4)	Negative	Negative	Positive	Positive	Negative (1)
1 (0.4)	Negative	Positive	Positive	Positive	IND (1)
1 (0.4)	Positive	Negative	Positive	Positive	Negative (1)
1 (0.4)	Positive	Positive	Positive	Negative	IND (1)

^aNT, not tested.

^bIND, indeterminate result.

^cThe total number of samples was 267, and the overall concordance rate was 94.0%.

Clinical specificity.

To assess the specificity of each CLIA, we confirmed HCV infectious status with the following algorithm. Among 267 samples assayed by all CLIAs, 160 showing negative results in all CLIAs were categorized as “screening test negative” and did not undergo any supplemental testing as recommended by the CDC (4). Among 107 samples showing positive results in any of the CLIAs, 54 were further investigated based on clinical data and medical record review as described above due to insufficient sample volume as a result of the consumption of large volumes during comparison experiments and repeat tests. Therefore, 30 samples were further categorized as “clinically confirmed positive” and 24 in which HCV status could not be confirmed were excluded from the specificity analysis.

The other 53 samples were tested for HCV RNA. If HCV RNA test were negative, a further confirmatory RIBA was performed. Any samples showing indeterminate HCV RNA or RIBA results were excluded from the specificity analysis. The specificity of each CLIA ranged from 96.5 to 98.8% (Table 5). There were no false-negative results from any CLIA, making the sensitivity of each assay 100% in our experiment.

TABLE 5.

Specificity of each CLIA on samples confirmed by additional confirmatory test or retrospective medical record review

Test result(s) (no. of samples)	No. of samples tested by:
	Elecsys		Architect		Vitros		Access
	Positive	Negative	Positive	Negative	Positive	Negative	Positive	Negative
Screening test negative (160)	0	160	0	160	0	160	0	160
Clinically confirmed positive (30)	30	0	30	0	30	0	30	0
HCV RNA positive (35)	35	0	35	0	35	0	35	0
HCV RNA indeterminate (2)a	2	0	2	0	2	0	2	0
HCV RNA negative, RIBA negative (10)	3	7	2	8	6	4	3	7
HCV RNA negative, RIBA indeterminate (4)a	1	3	3	1	4	0	1	3

^aIndeterminate results of the HCV RNA test and RIBA were excluded from the analysis. Clinical specificity: Elecsys, 98.2%; Architect, 98.8%; Vitros, 96.5%; Access, 98.2%.

Detection of HCV RNA in relation to CLIA s/co ratios.

We analyzed the s/co ratio result of samples confirmed by HCV RNA and RIBA or confirmed by retrospective medical record review (clinically confirmed positive). The number of cases with HCV infection increased in relation to the s/co ratio (Table 6). In the Vitros assay, only 1 (5.9%) of 17 cases with an s/co ratio of <8.0 had HCV infection. In the Architect and Access assays, 1 (7.1%) of 14 and 1 (5.9%) of 17 cases with an s/co ratio of <3.0 had HCV infection, respectively. However, in the Elecsys, we could not precisely identify the s/co ratio predictive of HCV infection negativity for more than 95% of the samples due to the paucity of samples with a low s/co ratio.

TABLE 6.

HCV infection status in relation to CLIA s/co ratio

CLIA and s/co ratio	No. of cases in each group	No. (%) of cases with HCV infection
Elecsys
1-200	13	5 (38.5)
200-400	24	21 (87.5)
400-1,000	56	55 (98.2)
>1,000	13	13 (100)
Total	106	94 (88.7)
Architect
1.0-3.0	14	1 (7.1)
3.0-10.0	12	7 (58.3)
10.0-15.0	68	68 (100)
>15.0	18	18 (100)
Total	112	94 (83.9)
Vitros
1.0-8.0	17	1 (5.9)
8.0-20.0	6	4 (66.7)
20.0-30.0	56	56 (100)
>30.0	7	7 (100)
Total	86	68 (79.1)
Access
1.0-3.0	17	1 (5.9)
3.0-10.0	13	9 (69.2)
10.0-15.0	75	74 (98.7)
>15.0	7	7 (100)
Total	112	91 (81.3)

The guidelines for laboratory testing and result reporting of antibodies to HCV from the CDC (4) include an option for reflex supplemental testing based on screening test-positive s/co ratios. Use of s/co ratios could minimize the amount of supplemental testing that needs to be performed while improving the reliability of reported test results. In short, screening test-positive results are classified as having high s/co ratios if their ratios are at or above a predetermined value that predicts a supplemental test (HCV RNA or RIBA) positive result for ≥95% of the samples tested. However, only the Vitros Anti-HCV assay has been approved by the FDA and an s/co ratio of 8.0 was set as the screening test positive value to determine the need for reflex supplemental tests. With our data, we set the cutoff s/co ratios as follows: Elecsys assay, ≥200 (89 [95.7%] of 93); Architect assay, ≥3 (93 [94.9%] of 98); Vitros assay, ≥7.0 (67 [95.7%] of 70); Access assay, ≥3 (90 [94.7%] of 95).

DISCUSSION

Since anti-HCV EIAs using recombinant antigen were introduced in 1990, they have been widely used for clinical diagnosis and screening of asymptomatic persons. With the development of newer generations of EIAs, sensitivity and specificity were greatly improved (8). For HCV screening of a population of blood donors, the most sensitive test should be chosen to avoid false-negative results. On the contrary, for screening of patients to be treated, false-positive results should be avoided. The CDC has recommended performing reflex supplemental testing for low s/co ratios of screening test positive results to resolve false-positive results (4).

Recently, various assay formats of anti-HCV CLIAs have been developed, and they offer the great advantages of improved precision, reliability, technical simplicity, short turnaround time, high-speed throughput, and full automation, particularly for high-volume hospital laboratories. Furthermore, CLIAs have improved specificity and a greater positive predictive value than conventional EIAs and result in fewer low-positive samples that require confirmatory testing (10). In the present study, we assessed the clinical performance of four automated CLIAs available in our laboratory to help in selecting the anti-HCV CLIA format and instruments. Among the four systems, only the Vitros assay contains the NS5 antigen of HCV and the Elecsys uses a ruthenium complex as a labeled substance for EIA.

The percent agreement among the results of the four CLIAs ranged from 94.5 to 98.1%. The Access assay showed a lower concordance rate than the others because of the smaller number of negative samples tested. Previous reports on the performance evaluation of anti-HCV CLIAs usually compared the results of a CLIA with those of an EIA. Therefore, there are no available data to compare to our results. In addition, the Elecsys Anti-HCV and Access HCV Ab PLUS assays have never been studied. The best correlation (98.1%) was found between the results of the Architect and Vitros assays. The HCr43 and c100-3 HCV antigens in the Architect assay are known to be prepared under contract agreement by Ortho Diagnostic Systems and the Chiron Corporation. This may explain why the best correlation was found between the results of the Architect and Vitros assays.

Among 257 samples tested by the four CLIAs, discrepant results were obtained with 16. These 16 samples were confirmed negative for HCV RNA. The number of false-positive results based on HCV RNA results were 5 (1.9%), 6 (2.2%), 10 (3.7%), and 4 (1.5%) by the Elecsys, Architect, Vitros, and Access assays, respectively. The United Kingdom Health Protection Agency has recommended the use of a second anti-HCV antibody EIA for the confirmation of positive samples in the National Standard Method Minimum Testing Algorithm for the investigation of HCV infection (14). This algorithm is more cost effective and could reduce the number of samples in the problematic immunoblot-indeterminate group when following the CDC guidelines (17, 26). Among the discordant samples, the number of cases in which only one of the four CLIAs gave a positive result was 10 (63%). If we choose two CLIA formats for screening and confirmation, false-positive reactions are dramatically decreased, theoretically, for certain combinations of CLIAs.

We investigated the clinical specificity of four CLIAs. The clinical specificity was 98.8% in the Architect assay, 98.2% in the Elecsys and Access assays, and 96.5% in the Vitros assay. The major difference among the four CLIAs is the inclusion of NS5 in the Vitros Anti-HCV assay. However, it is not clear that the addition of recombinant HCV NS5 protein in the assay format may be responsible for nonspecific reactivity (11, 27). Based on our data, the absence of NS5 protein in the assay format seems to improve the specificity of the anti-HCV CLIA.

The 2003 CDC guidelines for laboratory testing and result reporting of antibody to hepatitis C virus (4) require the use of a screening assay with high sensitivity and, for samples with low s/co ratios, confirmation by a recombinant immunoblot or PCR test. Among the available anti-HCV CLIAs on the market, only the Vitros assay has been approved by the FDA, and an s/co ratio of 8.0 was set as the screening test positive value to determining the need for reflex supplemental tests. Oethinger et al. (23) reported that >99% of the samples with very low s/co ratios tested by the Ortho Vitros anti-HCV assay had no evidence of HCV infection. Furthermore, they decided to report the results of samples with low s/co ratios between 1 and 5 as “borderline,” with the recommendation that follow-up testing be performed when HCV infection continues to be suspected. We investigated s/co ratios of four CLIAs according to HCV infection status. As mentioned above, HCV infection status was determined by HCV RNA and evaluation of clinical data. For the Vitros assay, we could set a cutoff s/co ratio of 7.0 to predict a supplemental test positive result for more than 95% of the samples, similar to that of 8.0 assigned by the CDC (4). Similarly, a cutoff s/co ratio of 3.0 could be used for the Architect and Access assays and a cutoff s/co ratio of 200 could be used for the Elecsys assay.

According to the European Union standards (9), anti-HCV assays were required to have a sensitivity and a specificity of 100% and >99.5% for market approval, respectively. In our study, four anti-HCV CLIAs showed an excellent sensitivity of 100% and a good concordance rate. The clinical specificity varied from 96.5 to 98.8%, and the Elecsys, Architect, and Access assays showed specificities of >98%.