Field-based performance of three pre-market rapid hepatitis C virus antibody assays in STAHR (Study to Assess Hepatitis C Risk) among young adults who inject drugs in San Diego, CA

Abstract

Background

Approximately 4.1 million Americans are estimated to have been infected with hepatitis C virus (HCV), 45–85% of whom are unaware of their infection. Persons who inject drugs (PWID) account for 55.8% of all persons with HCV antibody (anti-HCV) in the U.S. PWID have limited access to healthcare and are infrequently tested for anti-HCV using conventional laboratory assays.

Objective

To evaluate performance characteristics (sensitivity and specificity) of three, pre-market rapid point-of-care tests (one oral fluid and two finger-stick assays) from two manufacturers (Chembio and MedMira) in settings providing services to young adult PWID in San Diego, CA.

Study design

Behavioral risk assessment surveys and testing for HCV were conducted among persons who reported injection drug use (IDU) within the past 6 months as part of the Study to Assess Hepatitis C Risk (STAHR) among PWID aged 18–40 years in 2009–2010. Sensitivity and specificity of the rapid anti-HCV assays were evaluated among STAHR participants, using two commonly used testing algorithms.

Results

Variability in sensitivity (76.6–97.1%) and specificity (99.0–100.0%) was found across assays. The highest sensitivity achieved for the Chembio finger-stick blood, Chembio oral fluid and MedMira finger-stick blood tests was 97.1%, 85.4% and 80.0% respectively; the highest specificity was 99.0%, 100.0% and 100.0%, respectively. In multivariate analysis false negative anti-HCV results were associated with female sex for the MedMira blood assay.

Conclusions

Sensitive anti-HCV rapid assays are appropriate and feasible for high-prevalence, high-risk populations such as young PWID.

1. Background

The Centers for Disease Control and Prevention (CDC) estimates that 4.1 million Americans have been infected with the hepatitis C virus (HCV) of whom 75–80% are chronically infected.³ CDC recommends routine HCV antibody (anti-HCV) testing for persons at risk of infection,¹⁷ yet recent studies have estimated that 45–85% of HCV-infected persons are unaware of their status.²¹ Knowledge of HCV status is a prerequisite for persons to make health-promoting behavior changes and treatment decisions.

Persons with a history of injection drug use (IDU) account for 55.8% of all anti-HCV persons in the U.S.²⁶ and CDC estimates that 48% of acute hepatitis C cases in the U.S in 2007 were attributable to IDU.⁶ IDU among young adults has been increasing since the 1990s⁶ and studies of young persons who inject drugs (PWID) have found anti-HCV prevalence of 30–70% among PWID depending on frequency and duration of IDU.²

Many young PWID are unaware of their HCV status,⁹ although persons who receive drug treatment or syringe exchange program services are more likely to be aware than those who do not receive these services.⁷ PWID have limited access to healthcare and are infrequently tested for anti-HCV using conventional laboratory assays.²⁴ In addition, results from conventional assays are less likely to be received by PWID than are point-of-care (POC) results.⁹ Rapid assays enable specimens to be collected and tested at the POC and do not require a follow-up visit to receive antibody test results. Just as rapid HIV testing increases the likelihood that PWID receive HIV test results^11,13,15,20 and improves access to care and health outcomes,^10,11 rapid anti-HCV assays could benefit young PWID similarly.

2. Objectives

The purpose of the current study was to assess the performance of three recently developed rapid anti-HCV assays that had undergone both a laboratory-based validation¹⁸ and a field evaluation.¹⁹

3. Study design

3.1. STAHR

Data were drawn from the Study To Assess Hepatitis C Risk (STAHR)⁴ which was conducted in San Diego, CA in 2009–2010. STAHR was designed primarily to test three recruitment methods of PWID (aged 18–40 years) for the study of hepatitis C. Finger-stick blood and oral fluid from consenting participants was tested using rapid anti-HCV assays, conventional anti-HCV assays and HCV nucleic acid tests (NAT). Because the rapid tests were not approved for diagnostic use by the FDA at the time of data collected, results from the conventional anti-HCV and HCV NAT were provided to participants.

3.2. Anti-HCV rapid assays

The Chembio DPP^™ HCV test (Chembio Diagnostic Systems, Inc., Medford, NY) and the Multiplo^™ Rapid HIV/HCV Antibody Test (MedMira Laboratories, Inc., Halifax, Nova Scotia, Canada) were evaluated. The two manufacturers responded to a Federal Register Notice in 2009 announcing collaboration for the evaluation of rapid HIV and anti-HCV assays. The rapid assays are single use, disposable chamber, in vitro, qualitative, immune-chromatographic assays to detect anti-HCV which provide visual results in less than 40 min. More detailed descriptions of each assay can be found elsewhere.¹⁸

3.3. Rapid assay performance

The manufacturers provided documentation, instruction manuals, and onsite training for performing the assays and interpreting the results. Finger-stick blood and/or oral specimens were collected. An assay was interpreted as invalid if the control line was missing or broken, as non-reactive if a control line was present (regardless of intensity) with no corresponding test line, and reactive with an unbroken control and test line.

3.4. Reference assays

All specimens in the evaluation panel were tested for anti-HCV by the AxSYM anti-HCV Microparticle Enzyme Immunoassay (Abbott Laboratories, Abbott Park, IL) (MEIA). A third generation recombinant immunoblot assay (RIBA: The Chiron RIBA HCV 3.0 SIA; Novartis Vaccines & Diagnostics, Inc., Emeryville, CA) was used to confirm antibody positivity for reactive specimens with a signal to cut off (s/co) ratio below the CDC-recommended threshold and has the effect of increasing the sensitivity.¹ Nucleic acid testing (NAT) was conducted on all specimens in the evaluation panel to test for viremia (see Technical Appendix).

3.5. Specimen panel

PWID were recruited from a syringe exchange program, a community-based organization focused on HIV prevention, and the broader community via targeted outreach. A total of 409 specimens were obtained (114 anti-HCV positive, 295 anti-HCV negative) and tested by the three assays: the Chembio blood assay, the Chembio oral assay and the MedMira blood assay.

3.6. Data analysis

Performance of each anti-HCV rapid assay was evaluated in comparison to the results from the two conventional reference methods most commonly used in public laboratories.²⁶ Sensitivity and specificity were assessed by comparing the results of the rapid assays first to the results of the MEIA only (screening assay [SA] reference method) and second, to the results of the CDC-recommended HCV testing algorithm (CDC reference method)¹ (see Technical Appendix).

3.7. Sensitivity and specificity

Sensitivity was defined as the number of positive specimens detected by the rapid assay divided by the total number of reference assay positive specimens. Specificity was defined as the number of negative specimens identified by the rapid assay divided by the total number of the reference assay negative specimens. Confidence intervals for sensitivity and specificity were calculated using the Wilson Score method.^23,27 Multivariate logistic regression was used to determine whether discordant rapid anti-HCV results, conventional anti-HCV results, RIBA results and HCV NAT results were predicted by selected demographic variables (sex, age, race/ethnicity, injection duration and HIV RNA status) using both reference methods. Data analysis was performed using PASW 18.0 (Chicago, IL).

4. Results

Of 566 STAHR participants, 409 (72.3%) provided informed consent and completed rapid testing for this sub-study; 15 persons refused the rapid test. Invalid rapid assay results [Chembio blood (n = 2), Chembio oral (n = 7) and MedMira (n = 33)] occurred for all three assays resulting in a smaller analytic sample. On average, participants were aged 29.2 years (range aged 18–40 years; standard deviation = 6.2), and the majority were White (51.1%) or Hispanic (29.3%) and male (74.1%) (Table 1).

Table 1.

HCV status of selected demographics by reference method.

	N	SA (N = 409)		CDC (N = 407)
		% HCV+	OR (CI95)	% HCV+	OR (CI95)
Total	409	27.9%		26.3%
Sex
Male	303	29.4%	<ref>	27.8%	<ref>
Female	102	24.5%	0.78 (0.47–1.31)	22.8%	0.77 (0.45–1.33)
Age (yrs)
18–23	85	16.5%	<ref>	12.9%	<ref>
24–29	120	21.7%	1.40 (0.68–2.89)	20.8%	1.77 (0.82–3.83)
30–34	95	36.8%	2.96 (1.46–6.01)	35.8%	3.75 (1.75–8.02)
35–40	104	37.5%	3.04 (1.52–6.11)	36.3%	3.83 (1.81–8.12)
Race/ethnicity
Hispanic	120	31.7%	<ref>	30.0%	<ref>
AA/black	25	4.0%	0.09 (0.01–0.69)	4.0%	0.10 (0.01–0.75)
White	209	27.8%	0.83 (0.51–1.35)	26.3%	0.83 (0.51–1.37)
Other	23	17.4%	0.45 (0.15–1.43)	17.4%	0.49 (0.16–1.55)
HIV status
Negative	380	25.3%	<ref>	23.9%	<ref>
Positive	15	46.7%	2.59 (0.92–7.33)	33.3%	1.59 (0.53–4.77)
Injection duration (yrs)
0–6	201	16.4%	<ref>	14.4%	<ref>
7–13	122	36.1%	2.87 (1.70–4.86)	35.2%	3.23 (1.88–5.55)
14–20	59	39.0%	3.25 (1.71–6.19)	37.9%	3.63 (1.87–7.02)
21–31	22	63.6%	8.91 (3.46–22.93)	61.9%	9.64 (3.67–25.29)

Note: not all demographics were available for all participants; SA: screening assay method; CDC: CDC algorithm method.

Compared to the SA method, the sensitivity for the Chembio blood assay was 92.8% and specificity was 99.0%; sensitivity for the Chembio oral fluid assay was 81.8% and specificity was 100.0%; and sensitivity for the MedMira blood assay was 76.6% and specificity was 100.0%. Sensitivities increased using the CDC reference method across assays, although not significantly (Table 2).

Table 2.

Performance characteristics of rapid HCV test by assay and reference method.

	Sensitivity (95% CI)	Specificity (95% CI)	TP	FP	FN	TN	Invalid
SA reference (N = 409)
Chembio blood	92.8% (86.4–96.3%)	99.0% (97.0–99.7%)	103	3	8	290	2
Chembio oral	81.8% (73.6–87.9%)	100.0% (98.7–100.0%)	90	0	20	289	7
MedMira	76.6% (67.8–83.6%)	100.0% (98.6–100.0%)	82	0	25	269	33
CDC reference (N = 407)
Chembio blood	97.1% (91.9–99.0%)	99.0% (97.1–99.7%)	101	3	3	295	2
Chembio oral	85.4% (77.4–91.0%)	100.0% (98.7–100.0%)	88	0	15	294	7
MedMira	80.0% (71.1–86.7%)	100.0% (98.6–100.0%)	80	0	20	274	33

Note: TP: true positive; FP: false positive; FN: false negative; TN: true negative; Invalid: invalid rapid result (excluded from analysis).

Discordant results were analyzed to determine if an association existed for selected demographics. False negative rapid anti-HCV results were associated with female sex for the MedMira blood assay (aOR = 3.07; 95% CI, 1.12–8.39; p = 0.03) using the CDC reference method. Discordant results also were compared to HCV NAT results. Of the 409 specimens, two (0.49%) were HCV NAT positive but anti-HCV negative and rapid anti-HCV negative by all three rapid assays. Twenty-five (6.1%) specimens were SA anti-HCV positive and HCV NAT negative (Table 3). Five of the 25 specimens with low s/co ratios were identified as negative by RIBA and also identified as rapid anti-HCV negative by all three assays.

Table 3.

Discordant results (SA method positive/NAT negative).

Rapid test result			Confirmatory test result
CBB	CBO	MMB	SA s/co	RIBA	HCV NAT
Pos	Pos	Pos	79.86	NA	Neg
Pos	Pos	Neg	14.64	NA	Neg
Pos	Pos	Pos	16.29	NA	Neg
Pos	Pos	Neg	21.52	NA	Neg
Pos	Pos	Pos	21.66	NA	Neg
Pos	Pos	Pos	48.30	NA	Neg
Pos	NA	Pos	29.37	NA	Neg
Pos	Pos	Pos	39.12	NA	Neg
Pos	Pos	Pos	91.53	NA	Neg
Pos	Pos	Pos	92.67	NA	Neg
Pos	Pos	Pos	15.06	NA	Neg
Pos	Pos	Neg	19.58	NA	Neg
Pos	Pos	Neg	26.96	NA	Neg
Neg	Neg	Neg	13.32	NA	Neg
Pos	Neg	Invalid	94.26	NA	Neg
Neg	Neg	Neg	2.38	Neg	Neg
Neg	Neg	Neg	1.23	Neg	Neg
Neg	Neg	Neg	1.55	Neg	Neg
Neg	Neg	Neg	2.42	Neg	Neg
Neg	Neg	Neg	1.24	Neg	Neg
Pos	Neg	Neg	7.78	Pos	Neg
Pos	Neg	Neg	3.95	Pos	Neg
Pos	Neg	Neg	3.29	Pos	Neg
Pos	Neg	Pos	6.22	Pos	Neg
Pos	Pos	Pos	2.30	Indeterminate	Neg

CBB: Chembio blood; CBO: Chembio oral; MMB: MedMira blood; Pos: positive; Neg: negative; NA: not available.

Of the 11 RIBAs that were conducted, two were indeterminate and therefore excluded from the CDC reference analysis. The indeterminate specimens were both HIV-positive and the rapid assays identified the specimens as anti-HCV positive. Of the 15 total HIV-positive specimens, 7 (46.7%) were SA anti-HCV positive and 5 (33.3%) were CDC anti-HCV positive. All three rapid tests concurred with the SA method positive results. Chembio blood had one false positive among the HIV mono-infected specimens.

5. Discussion

This field evaluation of three pre-market rapid assays found considerable variability in sensitivity across assays. Sensitivity ranged from 76.6% to 97.1%, which is similar to the findings of previous studies (sensitivity 78.9–97.8%).^18,19 Sensitivities of the Chembio oral fluid and MedMira blood assays were similar, while the sensitivity of the Chembio blood assay was significantly higher than the MedMira blood assay. The Chembio and MedMira assays specificity ranged from 99.0% to 100.0%, which is higher than in previous studies (80.0–99.8%).^18,19

These results are similar to the findings of recent CDC laboratory¹⁸ and field¹⁹ evaluations of POC assays, although as might be expected, sensitivity in field use was slightly lower than in laboratory use. These differences in rapid assay sensitivity between laboratory and field settings are similar to those found for HIV rapid assays.^5,16 These differences may be due to the use of recently collected whole blood in the field versus stored serum in the laboratory, differences in technical expertise of those performing the assays, and possible contamination in field settings versus a more sterile laboratory setting.

Comparing sensitivities and specificities of rapid assays requires operational and interpretable assays. In this study, as well as in previous studies, MedMira had more invalid rapid assay results than Chembio oral and blood combined (9% versus <2%, respectively), resulting in lower utility of the assay. Anecdotal reports from testers suggested that the MedMira assay was more difficult to perform and interpret than the Chembio assays.

As expected, using the CDC reference method resulted in somewhat higher sensitivity for all assays than using the SA reference method. Two of the eleven (18%) RIBA results in this study were indeterminate, which is similar to a laboratory evaluation and a field study conducted by CDC where two of 10 (20%)¹⁸ and 4 of 25 (16%) RIBA results,¹⁹ respectively were indeterminate. One of the RIBA indeterminate results was rapid test reactive and NAT positive. The other RIBA indeterminate result was rapid test reactive and NAT negative.

The HIV prevalence of 3.7% (n = 15) among all rapid assay evaluation participants was similar to that found among PWID in previous studies.²⁵ HIV-HCV co-infection was low (ranging from 30.7% to 33.3% among persons with HIV across reference methods) in this sample as compared with other PWID studies (80–89%).^7,21,22 Some of this may be attributable to age as anti-HCV is less prevalent among younger populations, resulting in lower co-infection rates.²¹

Discordant results (SA positive and NAT negative (N = 25)) could be spontaneous clearers, however follow-up testing would be required to confirm. Female sex was a significant predictor of false negative anti-HCV rapid test results for the MedMira assay using the CDC reference method but not for the other assays. The association of false results identified in this study was different from the associations found in other field settings (HIV and race/ethnicity).¹⁹ Studies with larger sample sizes would be needed for further analysis.

Using anti-HCV rapid assays, PWID would receive their results at the POC on the same day, increasing the likelihood that they could be provided with prevention counseling messages and referrals for follow up. In this study, only 53.3% of persons tested in this study received their conventional anti-HCV test results and on average received them 27.5 days after testing, results which are comparable to those seen in other studies.⁸ Rapid anti-HCV assays which utilize oral fluid or capillary blood have the benefit of not requiring a phlebotomist and can be provided at POC, expanding testing options for PWID.^12,14 Rapid anti-HCV assays could be administered in syringe exchange programs (both through store fronts and mobile units), methadone maintenance treatment programs, and other programs that provide direct services to persons who inject drugs.

6. Conclusion

We found further evidence that sensitive rapid anti-HCV assays can be useful for the detection of anti-HCV among persons at risk for HCV, such as PWID who can be reached through specific social service settings, such as syringe exchange programs. Of the three assays evaluated, the Chembio blood rapid assay demonstrated acceptable sensitivity and specificity, and was comparable to conventional assays currently in use.

While this study is not generalizable beyond our study population, some limitations from previous studies were addressed. For example, in the previous field study¹⁹ testing one specimen with multiple rapid assays and NAT was not feasible and in the laboratory study¹⁸ the rapid assays were not tested in the field, while in this study we were able to gather and analyze such data, enabling direct comparison of assay performance in the field where rapid assays would most likely be utilized.

Rapid anti-HCV tests will require further evaluation. Demonstration projects should be conducted to evaluate the impact of integrating rapid anti-HCV testing into protocols used at HIV testing sites. CDC should evaluate the HCV testing algorithm given that the RIBA confirmatory test is expensive and added limited information regarding antibody status. Finally, standardized educational materials and guidance for post-test counseling for anti-HCV rapid assay positive persons need to be developed and evaluated.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jcv.2012.04.003.