Field accuracy of 4th generation rapid diagnostic tests for acute HIV-1: a systematic review

Abstract

Introduction

4th generation HIV-1 rapid diagnostic tests (RDTs) detect HIV-1 p24 antigen to screen for acute HIV-1. However, diagnostic accuracy during clinical use may be suboptimal.

Methods

Clinical sensitivity and specificity of 4th generation RDTs for acute HIV-1 were collated from field evaluation studies in adults identified by a systematic literature search.

Results

Four studies with 17,381 participants from Australia, Swaziland, the UK and Malawi were identified. All reported 0% sensitivity of the HIV-1 p24 component for acute HIV-1 diagnosis; 26 acute infections were missed. Specificity ranged from 98.3%-99.9%.

Conclusion

4th generation RDTs are currently unsuitable for the detection of acute HIV-1.

Introduction

An estimated 5-20% of HIV infections are due to transmission from recently infected individuals[1], although this depends on epidemic characteristics[2,3]. Diagnosing HIV in the weeks after an individual has acquired infection is likely to be critical to epidemic control as prompt diagnosis allows early entry into care and treatment, reducing the risk of onward transmission. Laboratory diagnosis relies on the appearance of HIV antigen (e.g. HIV-1 p24 antigen) or nucleic acid in the bloodstream prior to detectable antibodies to HIV-1/2, but testing procedures can be burdensome. Easy-to-use and accurate in-vitro diagnostics for detection of acute HIV infection that can be used at point-of-care are required to realise the potential public health benefits of early diagnosis, especially in resource-limited settings where the large majority of HIV transmissions occur.

New 4th generation point-of-care rapid diagnostic tests (RDTs) for HIV-1 detect HIV-1 p24 antigenaemia as well as antibodies to HIV-1/2. They have the potential to meet these requirements, and have been approved their sale in many settings. However the sensitivity of the p24 antigen component has been variable (2-92%) in laboratory evaluations on banked panels of specimens[4–9] leading to concerns this may translate into poor diagnostic accuracy in the field for the diagnosis of acute HIV-1[10] We therefore performed a systematic review to assess the field accuracy of 4th generation RDTs for the diagnosis of acute HIV-1 in clinical practice.

Methods

In accordance with our published protocol[11], we systematically searched Medline (using PubMed) and ISI Web of Science for studies published between 1^st January 2005 and 22^nd April 2015 that evaluated the diagnostic accuracy of 4th generation RDTs for acute HIV-1 (Supplementary Table 1: Search Strategy). Studies were eligible if they evaluated the field performance of at least one 4th generation RDT for acute HIV-1 in adults (aged >15 years). We defined field performance as the RDT being used on an unselected population, at the point of care (defined as being conducted in the presence of the patient[12]), in the setting of intended use and carried out by the intended user. Cross-sectional studies, cohort studies and randomised control trials were included. Studies that evaluated 4th generation RDTs within laboratories or retrospectively against specimen panels were excluded. We placed no restrictions on assay manufacturer, language or country.

Two reviewers (PM, EA) independently reviewed the titles and abstracts of all studies against inclusion and exclusion criteria, with discrepancies resolved by group decision with a third reviewer (MT). Two reviewers (JL, PM) independently reviewed the full text of all selected studies using a piloted data extraction form to determine final selection status. Discrepancies between reviewers were resolved by group decision with a third reviewer (MT). Participants receiving the index test were defined as true positive, false positive, false negative and true negative for the presence of a positive HIV-1 p24 antigen reaction on the 4th generation HIV RDT compared to the study-defined reference standard for acute HIV-1 infection. We recognised that there is no current consensus reference standard for the diagnosis of acute HIV infection and specified that an acceptable reference standard for the diagnosis of acute HIV-1 infection would include: at least one non-reactive antibody test result using a third generation HIV immunoassay; and at least one reactive antigen test result on a 4th generation HIV immunoassay; and/or at least one reactive result using an HIV-1 p24 antigen immunoassay. However where this was not used, we used study-reported reference standards. Details of studies excluded after full text review are shown in the supplementary appendix (Supplementary Table 2)[5–9,13–37].

Risk of bias in the included studies was independently assessed by two reviewers (JL and PM) using the QUADAS-2 tool[38] with signalling questions used across four domains (selection; index test conduct; reference standard conduct; and participant flow and timing) to assess overall risk of bias.

Study flow and results are reported in accordance with the Standards for Reporting of Diagnostic Accuracy (STARD) checklist[39]. Given the high degree of heterogeneity in study populations and conduct, we did not perform meta-analysis, but would have done so if results had indicated this to be appropriate.

Results

Of 2056 unique studies identified, four met inclusion criteria and contributed data relating to 17,381 participants to this analysis (Figure 1). Studies came from Swaziland[40], Malawi[41], Australia[42] and the United Kingdom[43] (Table 1). Three studies included community members[41], men-who-have-sex-with men[42], or injecting drug users[43] attending specialist sexually transmitted diseases or HIV testing clinics, while in the fourth, participants were recruited from a national HIV incidence survey[40]. All had a cross-sectional study design.

Figure 1. Flowchart of included and excluded studies.

Table 1. Characteristics of included studies investigating diagnostic accuracy of 4th generation HIV rapid diagnostic tests.

Study	Countries	Study population	Proportion men	Reference standard for diagnosis	True positive	False positive	False negative	True negative	Sensitivity (95% CI)	Specificity (95% CI)
Conway 2014	Australia	Adult (≥18 years old) MSM attending STD clinics in 4 suburbs	100%	Algorithm comprising of: 4th generation immunoassay (Architect HIV Ag/Ab Combo or Elecsys HIV Combi PT) – All participants Supplementary antibody immunoassay (Serodia HIV-1 Antibody particle agglutination assay or Genscreen HIV 1&2 Antibody EIA) – If index test antigen or antibody reactive HIV p24 antigen immunoassay (Genscreen HIV-1 p24 Antigen) – If index test antigen or antibody reactive HIV Western blot (HIV Western Blot 2.2, MP Diagnostics) – If index test antigen or antibody reactive	0	6	3	3181	0.00 ø	0.998 ø
Duong 2014	Swaziland	Adult (18-49 years old) community members participating in a national HIV incidence survey	39%	Algorithm comprising of: 3rd generation RDT (Uni-Gold Recombigen) – Index test antibody reactive 4th generation RDT (Determine™ HIV-1/2 Ag/Ab Combo) – Index test antigen reactive HIV viral load quantification (COBAS AmpliPrep/COBAS TaqMan) – Index test antigen reactive Pooled (with deconstruction and individual retesting if detected) HIV viral load quantification (COBAS AmpliPrep/COBAS TaqMan) – Index test antigen and antibody non-reactive	0	12	13	12,345	0.00 (0-0.23)	0.999 (0.998-0.999)
Jones 2012	United Kingdom	Adult (≥18 years old) MSM, IDUs, individuals from high prevalence settings and at-risk partners attending STD clinic in London	87%	Algorithm comprising of: 3rd generation RDT (Determine HIV-1/2) – All participants 4th generation immunoassay (Architect HIV Ag/Ab Combo) – all participants 4th generation immunoassay (VIDAS platform, assay not specified) – Index test reactive	0	3	2	980	0.00ø	0.997ø
Rosenburg 2012	Malawi	Adults (18-49 years old) attending hospital STD† and HIV testing clinics in Lilongwe	STD: 65% HTC: 52%	3rd generation RDT (Uni-Gold Recombigen, SD Bioline HIV 1/2 3.0) – All participants HIV viral load quantification (Roche Monitor) – Participants with non-reactive or discrepant HIV antibody RDT results	0	14	8	824	0.00 (0.00-0.37)	0.983 (0.972-0.991)

MSM: men who have sex with men

STI: sexually transmitted infections

NR: not reported

RDT: rapid diagnostic test

PWID: people who inject drugs

HTS: HIV testing services

^†Only patients with negative or discrepant HIV RDTs results at high risk for acute HIV infection attending STD clinic were invited to participate

^ø95% confidence intervals not reported

True positive/false positive/true negative/false negative all refer to p24 antigen strip of index test

All four studies evaluated the diagnostic accuracy of Alere Determine HIV-1/2 Ag/Ab Combo (Alere Medical Co. Ltd., Japan), hereafter described as the index test. This is a 4^th generation RDT that can detect antibody to HIV-1/2 and HIV-1 p24 antigen. Only 2 studies used reference standards as defined in our study protocol[42,43], with the other two studies variously using laboratory-performed testing algorithms for characterisation of acute HIV comprising of: a 3^rd generation RDT, a 4th generation RDT and HIV quantitative nucleic acid testing (NAT) technology (HIV viral load) for participants with a reactive HIV-1 p24 antigen index test result[40]; and a 3^rd generation RDT (all participants) and HIV viral load quantification (participants with non-reactive or discrepant antibody index test results)[41].

In all four studies, sensitivity of the HIV-1 p24 antigen component of the index test for reference standard-classified acute HIV infection was 0%, with a total of 26 acute HIV infections missed (Table 1). Specificity of HIV-1 p24 antigen component of the index test for acute HIV infection was similar across all studies [98.3%-99.9%]; a total of 35 false positive results were observed. It was possible to extract data on the accuracy of the antibody component of the index test from two of the four studies; in the first, the antibody component detected 0/3 (0%) acute HIV infections[42]; in the second, 2/8 (25%)[41]. The risk of bias in these studies as assessed across the four assessed domains was largely low or unclear. Only in one domain of one study was there a high risk of bias: in patient selection; in this study a non-consecutive, non-random cohort of patients were enrolled - only patients who were assessed as high risk for acute HIV infection[41]. The full assessment of risk of bias is shown in the supplementary appendix (Supplementary Table 3).

Discussion

This systematic review reveals unacceptably low field sensitivity in the diagnosis of acute HIV-1 of the p24 antigen component of Determine HIV 1/2 Ag/Ab Combo (Alere Medical Co. Ltd., Japan) – the only product evaluated in the included studies – with not a single true positive for acute infection detected in over 17,000 individuals from at-risk groups in sub-Saharan Africa, Australia and the UK. Across these studies, 26 reference-standard-defined acute HIV infections were missed, and 35 false positives were observed. On the basis of these findings, we recommend that Determine HIV 1/2 Ag/Ab Combo (Alere Medical Co. Ltd., Japan) is not currently used for screening for acute HIV-1 infection at the point of care.

There are a number of likely reasons for the poor test performance. The assay detects free HIV-1 p24 antigen; this has a short window of detection, before developing antibodies form immune complexes with antigen in the bloodstream and preclude binding of antigen to the test strip. This technical difficulty can be overcome with an antigen dissociation step[44], but this may be hard to implement at the point of care. Differential affinity of binding to different HIV subtypes could also contribute to poor detection of HIV-1 p24 antigen.

We excluded a number of studies as they did not meet our inclusion criteria. The majority were not field evaluations but rather evaluations in reference laboratory settings on specimen panels. Of those studies that did perform a field evaluation, but were excluded from this review, the reference and/or index test(s) were often inconsistently applied across all patients making evaluation of true sensitivity and specificity impossible. This likely represents the inherent difficulty in designing a study to assess an in vitro diagnostic for acute HIV infection, which is rare even in areas of high HIV prevalence and has a short timeframe for detection.

The lack of a consistent application of a clinical definition of acute HIV-1 infection makes it challenging to compare results across studies and compare diagnostic accuracy. The included studies defined acute HIV-1 infection using a variety of biomarkers and clinical parameters. In laboratory terms, acute HIV-1 infection has been clearly defined in terms of the Fiebig stages of acute HIV -1 infection, characterised by sequential appearance of HIV-1 nucleic acid, HIV-1 p24 antigen followed by IgM then IgG antibodies to HIV-1[45]. These stages were originally defined using 2^nd and 3^rd generation immunoassays and attempts have been made to redefine these stages in terms of more modern 3^rd and 4^th generation assays[46]. Regardless of the existence of these clear laboratory definitions, we found that clinical definitions of acute HIV-1 are inconsistently applied across studies. Internationally agreed and applied consensus definitions of acute HIV infection for use in clinical studies would go some considerable way to clarifying a confusing field.

Despite a comprehensive literature search and rigorous analysis, there are some limitations to our study: data may be reported as supplementary material, and have been missed by our search; only adults were included. Because of the small number of studies identified, and high degree of heterogeneity in study settings and populations, it was inappropriate to undertake meta-analysis, meaning that overall summary estimates for diagnostic accuracy cannot be made. All studies assessed the accuracy of only one 4th generation RDT; others are available (e.g. SD Bioline HIV Ag/Ab (Standard Diagnostics, Inc., Republic of Korea) and ImmunoComb^® II HIV 1&2 TriSpot Ag-Ab (Orgenics Ltd, Israel)). The generalisability of our results are unclear but it seems likely that the same technical difficulties will affect other 4th generation RDTs[47], and clinicians should interpret the results of HIV-1 RDT antigen testing with caution.

In conclusion, the 4th generation RDT for HIV that was the subject of this systematic review has low sensitivity for detection of acute HIV-1, and is unsuitable for this purpose. Point of care options for diagnosis of acute HIV-1 are therefore currently limited to 3^rd generation RDTs alongside careful assessment of timing of exposure and serial testing, with a continuing need for laboratory support for antigen testing or NAT where the suspicion of acute HIV infection is high. New approaches to diagnosis of acute HIV such as point of care testing with 3^rd generation RDT exhibiting best possible seroconversion sensitivity, or NAT – including the prospect of expansion of qualitative NAT platforms for use at point-of-care that have been developed for early infant diagnosis – need evaluating in the field against agreed definitions of acute HIV in order to realise the public health benefits of making this important diagnosis.