Laboratory Evaluation of Three Rapid Diagnostic Tests for Dual Detection of HIV and Treponema pallidum Antibodies

Abstract

The performance of three research-use-only, dual HIV and syphilis rapid diagnostic tests (RDTs) was evaluated for 150 patient serum samples and compared to reference HIV and Treponema pallidum antibody detection methods. The RDTs performed comparably, with sensitivities of 93 to 99% and specificities of 97 to 100%. The kappa statistic between the RDTs was 0.95.

TEXT

In the United States, HIV and syphilis (caused by Treponema pallidum) coinfection is increasingly common, with an estimated median HIV seroprevalence in men with syphilis of 27.5% and in women with syphilis of 12.4% (1). Men who have sex with men (MSM) have particularly high rates of HIV and syphilis coinfection, documented to be 47 to 72% in some areas of the United State (2–6). Coinfection with syphilis can increase the transmission of HIV by both increasing viral shedding through open ulcers (7, 8) and by increasing patient viral load (9, 10).

Reference methods used by most major clinical laboratories in the in United States for the diagnosis of HIV include enzyme immunoassays (EIAs) for the qualitative detection of antibodies to HIV-1 and HIV-2. EIA-reactive specimens are typically confirmed with an HIV-1 antibody Western blot assay. In 2014, the Centers for Disease Control and Prevention (CDC) issued new guidance for HIV diagnostic testing, which included primary testing by a combination immunoassay that detects both HIV-1 and HIV-2 antibodies and the HIV-1 p24 antigen. Specimens reactive by the screening assay undergo supplemental testing with an immunoassay that differentiates HIV-1 and HIV-2 antibodies (11). Specimens that are reactive on initial antigen/antibody combination immunoassays and nonreactive or indeterminate in the HIV-1/HIV-2 antibody differentiation immunoassay are then tested with an FDA-approved HIV-1 RNA nucleic acid test (NAT) (11). Reference methods for diagnosis of syphilis include primary screening by nontreponemal tests, such as the rapid plasma reagin (RPR), and confirmation with a treponeme-specific test, such as the T. pallidum particle agglutination (TP-PA) assay. Alternatively, many laboratories have adopted a “reverse algorithm,” whereby a T. pallidum-specific immunoassay (e.g., enzyme immunoassay) is the screening test and a nontreponemal test, such as the RPR, is performed on EIA-reactive samples to determine the stage of the disease and monitor treatment (12).

In the United States, HIV tests are also commonly administered in both clinical and nonclinical community-based organizations, through the use of the Clinical Laboratory Improvement Amendments (CLIA)-waived rapid diagnostic tests (RDTs) for HIV, which detect HIV-1 and HIV-2 antibodies. The advantage of such testing is that results are immediately available at the point of care, which provides early diagnosis of HIV infection and improved linkage to care (13–15). In contrast, FDA-approved, CLIA-waived point-of-care tests for the diagnosis of syphilis are not yet available in the United States, although such tests are available in other countries (16). At the time of this writing, one rapid T. pallidum test has obtained FDA clearance and is awaiting the CLIA waiver (Syphilis Health Check [Diagnostics Direct, Youngstown, OH]). As is the case for HIV, rapid diagnosis and treatment of syphilis is critical to reducing transmission. The availability of rapid, CLIA-waived syphilis tests will allow immediate evaluation and treatment of patients who test positive for syphilis and the potential for screening in nonmedical settings. The bulk of the syphilis epidemic in the United States is among MSM, and the largest increase in primary and secondary syphilis cases between 2009 and 2012 was in MSM aged 25 to 29 years (17). However, sexually active MSM, and in particular young MSM, do not seek HIV and syphilis screening at the frequencies recommended by the CDC. As such, the availability of CLIA-waived, rapid, dual testing has the potential to reduce both syphilis and HIV rates among this at-risk population. While evaluation of point-of-care testing with RDTs for HIV or syphilis has been performed in various settings, the use of dual RDTs for both HIV and syphilis has not been fully evaluated.

In this study, we evaluated the performance of three commercially available, research-use-only (RUO) HIV/T. pallidum antibody dual RDTs by using remnant, deidentified sera from 150 people who were previously tested by routine methods. Twenty-five specimens were obtained from the San Francisco Department of Public Health (and had been previously characterized to be positive for HIV and syphilis antibodies); HIV and syphilis testing was confirmed at UCLA prior to the start of the study. The remaining 125 serum specimens were from the UCLA Clinical Microbiology Laboratory and selected based on the results of routine HIV and syphilis serologic testing. HIV testing was performed using the Siemens Advia Centaur HIV 1/O/2 enzyme immunoassay (HIV EIA; Siemens, Tarrytown, NY); all positives were confirmed by Western blotting, using the GS HIV-1 Western blot kit (Bio-Rad, Hercules, CA). RPR testing was performed using the Macro-Vue 18-mm circle card test (Becton Dickinson, Sparks, MD). Presence of T. pallidum antibodies was confirmed by using the Serodia TP-PA test (Fujirebio Diagnostics, Inc., Malvern, PA). All specimens were stored at −70°C prior to testing with the RDTs.

The three RUO dual HIV/syphilis RDTs evaluated were the MedMira Multiplo TP/HIV test (MedMira Inc., Halifax, Nova Scotia, Canada), Standard Diagnostics (SD) Bioline HIV/Syphilis Duo test (Standard Diagnostics Inc., Gyeonggi-do, Republic of Korea), and Chembio DPP HIV-syphilis assay (ChemBio Diagnostics Inc., Medford, NY). The SD and Chembio tests are solid-phase immunochromatographic assays, whereas the MedMira test is a vertical flow qualitative immunoassay. All three assays are single-use RDTs for the dual, qualitative detection of HIV-1, HIV-2, and T. pallidum antibodies. None of the RDTs differentiates HIV-1 from HIV-2. A comparison of the RDTs evaluated in this study is presented in Table 1. All three dual RDTs evaluated in the present study can be stored at room temperature, require no laboratory equipment (other than a timer), are easy to perform, are rapid, and are relatively easy to interpret.

TABLE 1.

Comparison of the three HIV/T. pallidum antibody RDTs used in this study

Parameter	ChemBio DPP HIV-syphilis assay	SD Bioline HIV/syphilis duo	MedMira Muliplo rapid syphilis/HIV antibody test
Specimen	Whole blood, serum, or plasma	Whole blood, serum, or plasma	Whole blood, serum, or plasma
Time to detection (min)	25	20	3
Equipment	Requires a timer	Requires a timer	None
Shelf life	24 mos at room temp	24 mos at room temp	18 mos at room temp
HIV component	Recombinant HIV-1 and HIV-2 antigens (not specified)	Recombinant HIV-1 gp41, sub-O antigens	Synthetic HIV-1 gp41, gp120, and group O peptides
		Recombinant HIV-2 gp36 antigen	Synthetic HIV-2 gp36 peptide
			IgM and IgG
T. pallidum component	Recombinant antigen (not specified)	Recombinant antigen (17 kDa)	Recombinant antigens (15 kDa, 17 kDa, 47 kDa)
Method	Solid-phase immunochromatographic assay	Solid-phase immunochromatographic assay	Vertical flow immunoassay
Antibodies detected	IgM and IgG antibodies to HIV and T. pallidum antigens	IgG, IgM, and IgA antibodies to HIV and T. pallidum antigens	IgM and IgG antibodies to HIV and T. pallidum peptides

Specimens were tested by all 3 RDTs in parallel, following the manufacturers' instructions, by a trained laboratory technician blinded to the reference results. Any discernible reactivity in the RDTs, even a faint reaction, was considered positive, as recommended by the manufacturers' package inserts. The results of the RDTs for HIV were compared to those via routine testing (EIA and Western blotting). The results of the RDTs for T. pallidum were compared to the TP-PA test results. Specimens that yielded discordant or difficult-to-interpret (faint) results were repeated using all reference methods and all 3 RDTs, in parallel. Data were summarized using descriptive statistics, including sensitivity and specificity, with 95% confidence intervals (CI) calculated by using the exact binomial distribution method. The kappa statistic was used to describe concordance between the three RDTS. Statistical analyses were performed using Microsoft Excel. All protocols were approved by the UCLA Institutional Review Board.

Among 150 samples included in this study, 29 (19.3%) were negative for T. pallidum and HIV, 24 (16%) were positive for T. pallidum but negative for HIV, 35 (23.3%) were positive for HIV but negative for T. pallidum, and 62 (41.3%) were positive for both HIV and T. pallidum by the reference methods. All HIV EIA-positive results were confirmed by a positive HIV-1 Western blot assay (data not shown). RPR titers for the 86 specimens positive by the TP-PA assay ranged from not reactive (n = 28) to a 1:512 titer (mean titer of reactive specimens, 1:8).

The performance of the RDTs is listed in Table 2. Sensitivity for HIV antibody detection by the RDTs was 98 to 99% and specificity was 94 to 100%, compared to the Siemens Advia HIV EIA. Similarly, detection of T. pallidum antibodies was excellent for all three methods, ranging from 93 to 95% sensitivity and 97 to 100% specificity, compared to the TP-PA assay (Table 2). The kappa coefficient between the three RDTs was 0.95 (95% CI, 0.80 to 1.0) for the HIV component and 0.93 (95% CI, 0.78 to 1.0) for the T. pallidum component.

TABLE 2.

Laboratory performance levels of three rapid diagnostic tests for the dual detection of HIV and T. pallidum antibodies, compared to reference methods^a

RDT	HIV antibody		T. pallidum antibody
	% sensitivity (95% CI)	% specificity (95% CI)	% sensitivity (95% CI)	% specificity (95% CI)
SD Bioline	97.9 (92.0–99.6)	100 (91.5–100)	93.0 (84.8–97.1)	100 (92.9–100)
ChemBio	98.9 (93.6–99.9)	98.1 (88.6–99.9)	95.3 (87.9–98.5)	100 (92.9–100)
MedMira	97.9 (92.0–99.6)	94.2 (83.1–98.5)	94.1 (86.3–97.8)	96.9 (88.2–99.5)

^aThe reference HIV antibody method was the Siemens Advia Centaur HIV 1/O/2 enzyme immunoassay and the GS HIV-1 Western blot kit, and the T. pallidum antibody reference method was the Serodia TP-PA method.

Repeat testing did not resolve any false-negative or false-positive results observed with the RDTs. All false-negative T. pallidum antibody results (n = 7) were from HIV-positive specimens (Table 3). Two of these were from specimens with high (≥1:8) RPR results. Two false-positive T. pallidum results were observed, both only with the MedMira assay and again in HIV-positive specimens (Table 3). Four false-positive HIV results were observed; three of these were with the MedMira assay (Table 4), two of which had repeatedly faint reactions for HIV. Two specimens yielded false-negative HIV reactions (Table 4). One of these specimens was negative by all three RDTs, whereas specimen SF5 was negative by the MedMira and SD RDTs but positive by the Chembio RDT (Table 4). Faint HIV reactions were observed in 2 MedMira tests (1.3%; both false positive), 3 SD tests (2%; all true positives), and 2 ChemBio tests (1.3%; one false positive). Faint T. pallidum reactions were noted for 16 MedMira tests (10.7%; two false positives), 10 SD tests (6.7%; all true positives), and 6 ChemBio tests (4%; all true positives). Repeat testing yielded results that were similarly difficult to interpret. Overall, our evaluation showed performance by the RDTs that was comparable to the reference methods, with excellent sensitivity and specificity. Ease of use was qualitatively comparable.

TABLE 3.

Characteristics of specimens with discordant T. pallidum antibody test results by one or more RDTs for the dual detection of HIV and syphilis antibodies

^aShaded boxes indicate discordant results.

^bFaint results were considered positive.

^cNR, not reactive.

TABLE 4.

Characteristics of specimens with discordant HIV antibody results by one or more RDTs for the dual detection of HIV and syphilis antibodies

^aShaded boxes indicate discordant results.

^bFaint reactivity was considered positive.

^cNR, not reactive.

In the United States, several HIV RDTs have been approved for clinical use by the U.S. FDA since 2002. The CDC recommends a second specimen be collected and tested by an HIV 1/2 immuno-differential test for those patients with a positive HIV RDT. If positive, the diagnosis is confirmed, and if negative, additional testing with an appropriate HIV RNA NAT is recommended (11). In contrast, the clinical experience with syphilis RDTs is limited in the United States, as only one manufacturer has recently received FDA clearance for their syphilis RDT. The management and follow-up testing for patients that test positive with a syphilis RDT remains to be defined but will likely include confirmation with laboratory-based treponemal tests and nontreponemal testing for disease staging and monitoring of treatment. Nonetheless, global experience with syphilis RDTs has shown excellent results. A systematic review of multiple syphilis RDTs used in 15 studies from over 22,000 whole-blood, plasma, or fingerstick specimens showed sensitivity (median of 86%; interquartile range, 75% to 94%) and specificity (99%; interquartile range, 98% to 99%) that were comparable with nontreponemal screening tests characteristics (18). A more recent meta-analysis further reported performance levels that were estimated to be comparable to those for laboratory-based treponemal tests for these rapid treponemal tests (16).

Dual RDTs for HIV and syphilis infection have been less well evaluated in either laboratory or clinical settings. Recently, a multisite laboratory evaluation from 6 countries demonstrated excellent sensitivity and specificity of the SD Bioline HIV/Syphilis Duo RDT. The sensitivity and specificity of the HIV antibody test component (n = 2,336 specimens tested) were reported to be 99.9% and 99.7%, respectively. For the T. pallidum antibody component (n = 2,059 specimens tested), the sensitivity and specificity were 99.7% and 99.7%, respectively (19). These values are comparable to those obtained in the present study.

Limitations to our study include a relatively small number of patient specimens evaluated. Another limitation includes the use of patient serum, as opposed to fingerstick whole-blood specimens, which would be used for point-of care testing. Furthermore, the performance levels of these tests may have been higher in our study than what would be observed in the real world, as testing was performed in a controlled laboratory setting by a small number of highly skilled technicians. Future research should include field evaluations of dual HIV/syphilis rapid tests.

While RDTs are not intended to replace standard reference methods, the development of quality RDTs could have an enormous impact on public health initiatives, by providing earlier identification of patients infected with HIV and/or T. pallidum. The fact that syphilis is a cofactor in HIV transmission and HIV infection affects the clinical presentation of syphilis (9, 10), and coupled with the high rate of HIV and syphilis coinfection among MSM in the United States, it underscores the need for both HIV and syphilis testing at the point of care. Since no HIV/syphilis dual RDTs remain categorized as research use only in the United States, such testing requires use of two RDTs. The recent FDA clearance of the Syphilis Health Check RDT makes such testing now feasible. In regions of the United States where coinfection rates are high, or in areas where testing can be targeted to high-risk patient populations, the availability of such testing may not only improve detection of syphilis infection, in addition to HIV, but also prevent further transmission by immediate treatment (18).

In summary, our study is the first to evaluate the sensitivity and specificity of three commercial combination HIV and syphilis RDTs in parallel, and we demonstrated comparable performance to reference methods for all three RDTs. Further use of these RDTs in the clinical setting may more adequately determine their performance as point-of-care tests. However, until the manufacturers submit data to the FDA for clearance of these products, this testing will not be available in the United States.