Xpert® HIV-1 Point of Care Test for neonatal diagnosis of HIV in the birth testing programme of a maternity hospital – a field evaluation study

Summary

Background

Performance of Point-of-Care testing (POCT) amongst HIV-exposed infants, compared to laboratory-based testing (LABT), may improve linkage to care. We present a field evaluation of HIV-1 POCT at birth in the context of universal LABT in a maternity hospital in Johannesburg, South Africa and describe our implementation experience.

Methods

We conducted our field evaluation study between October 2014–April 2016 at an urban public delivery facility. We aimed to sample consecutive neonates at birth for POCT (Cepheid Xpert^® HIV-1 Qualitative test) and compared results to LABT (Roche COBAS® TaqMan® HIV-1 Qualitative test) with respect to performance (sensitivity, specificity, positive and negative predictive value [PPV and NPV] and Cohen’s kappa coefficient), result return, antiretroviral treatment (ART) initiation and coverage.

Findings

Amongst 3970 infants with LABT, 57 (1.4%) tested positive, 3906 (98.5%) negative, 2 (0.1%) indeterminate, and 5 (0.1%) had an error result. 2238 (56.4%) of these infants had concurrent POCT. POCT detected all 30 HIV-infected neonates (sensitivity 100%; 95% CI: 88.4–100%) with two additional false positive results (specificity 99.9%; 95% CI: 99.7–100%). All positive and 96.2% of negative POCT results were returned compared with 88.9% of positive and 52.8% of negative LABT results. While each POCT required 90 minutes of instrument time, 2.6 hours (IQR: 2.3–3.1) elapsed between phlebotomy and result return. In days, median time of result return for POCT was one day, significantly earlier than ten days for LABT (p<0.0001). Antiretroviral treatment was initiated in 30 (100%) neonates with positive POCT compared to 24 (88.9%, p=0.10) of 27 infants who had LABT only, with initiation occurring a median of 5 days earlier in the POCT group (p<0.0001). POCT implementation required additional staff and weekend cover.

Interpretation

Compared to LABT, POCT was associated with excellent performance, improved rates of result return and reduced time to ART initiation. Resources needed to integrate POCT into a routine birth testing program require further evaluation.

Funding

The study was supported in part by the National Institutes of Health U01 HD080441 and USAID/PEPfAR.

Introduction

Infant HIV infection requires early diagnosis and antiretroviral treatment (ART) initiation before the first peak in mortality at two to three months of age.^1–3 Birth HIV polymerase chain reaction (PCR) testing for all neonates born to HIV-infected mothers was added to South Africa’s early infant diagnosis (EID) algorithm in June 2015 and later adopted as a conditional recommendation by the World Health Organisation (WHO).^4,5 Improving early diagnostic and ART coverage is challenging and may be strengthened by innovative approaches like Point of Care Testing (POCT).^6,7

The majority of births in South Africa occur at centralised maternity units while postnatal well-baby follow-up and subsequent HIV testing occurs at peripheral clinics.⁸ HIV PCR testing within the public sector, including birth testing, is centralised in nine laboratories in the country and turnaround times range from 24 hours to weeks for remote peripheral clinics. Thus after delivery, neonates and mothers are usually discharged before birth HIV results are released. In high maternal HIV prevalence settings like South Africa, where provincial neonatal HIV-exposure rates range from 17.5% to 401%,⁹ large volumes of HIV birth test results from bloods sampled at maternity units have to be returned at peripheral clinics responsible for postnatal care. This poses logistical challenges which threaten the usefulness of birth HIV testing.¹⁰ Non-negative results should prompt urgent tracing of the neonate for repeat HIV testing and appropriate care. HIV negative result return is also important and provides the opportunity to counsel mothers regarding the necessity of further testing for intrapartum and postnatal infection.

POCT offers the potential of returning the HIV PCR results to the mother before she and her neonate are discharged from the delivery unit. This reduces the need to trace neonates with non-negative results, and enables immediate initiation of ART in neonates with positive results. While there is some laboratory evaluation data available,^11–14 field evaluations of POCT platforms for EID, particularly at birth, are required prior to implementation. The Alere Q NAT test (Alere Technologies, Jena, Germany) has been shown to perform at a sensitivity of 98.5% (Jani et al, no neonates)¹⁵ and 96.5% overall (93.3% in 92 neonates, Hsiao et al; 99% in 291 neonates, Kroon et al)^16,17 with specificity >99.5% and there are few published data for Cepheid EID POCT in infants or neonates including the EID consortium which gathers data on neonates and infants.¹⁸

We conducted a field evaluation of birth POCT at a busy maternity hospital in Johannesburg, South Africa. Here we present a comparison of the performance of the Cepheid POCT with the standard of care laboratory-based testing (LABT) in neonates, heavily exposed to maternal ART. We report on implementation of POCT within a birth testing programme albeit that POCT occurred within a study setting.

Methods

Study Design

We conducted the field evaluation study at Rahima Moosa Mother and Child Hospital (RMMCH) in Johannesburg where approximately 1000 births occur per month of which ~23% are HIV-exposed.¹⁹ This urban public hospital is the sub-district’s main delivery facility and referral site for complicated cases. At the time of the study the standard intervention for prevention of mother to child transmission (PMTCT) was Option B+ i.e. life-long maternal ART for all infected women.⁴ Counsellors screened all new deliveries to identify known HIV-positive women and those newly diagnosed at delivery and LABT was offered for all exposed infants since 5 June 2014. HIV-positive women were interviewed about obstetric and ART history, disclosure, partner testing and adherenece while data on the neonate (anthropometrics, gestation, APGAR scores²⁰) and mode of delivery were collected. From 1 October 2014 through 30 April 2016, all identified HIV-positive women were invited to enrol their neonates in an observational cohort study of routine universal birth testing including this field evaluation of POCT. LABT was not dependent upon enrolment in the study.

Procedures

Neonatal whole blood was sampled by venipuncture in the post-natal ward or during neonatal admission. Cord blood was never sampled. The LABT sample was collected into a 0.5mL ethylenediaminetetra-acetic acid (EDTA) tube and sent to the national laboratory for HIV PCR testing (Roche COBAS® TaqMan® HIV-1 Qualitative Test Version 2.0, Roche Molecular Systems, Inc., Branchburg, NJ) where processing was done by routine, non-study staff. From the same blood draw, an additional identical 0.5mL whole blood sample was collected for POCT (Cepheid Xpert^® HIV-1 Qualitative assay, Cepheid, Sunnyvale, CA) for processing by study staff in a small satellite research laboratory on site.²¹ The minimum volume of blood required for this assay is 0.1mL but extra volume was drawn to allow for repeat tests on the same sample if needed. Neonates were not rebled for POCT if their initial LABT blood draw had been performed by the routine clinical staff before study consent could be obtained.

The on-site satellite laboratory housed three Cepheid instruments, each with four modules capable of running four POCTs independently. Technical support from Cepheid was readily available telephonically and on site assistance was provided for troubleshooting and instrument maintenance (~once or twice a month as required). The POCT instruments were operated by a laboratory technician or study nurse supervised by a laboratory-trained project manager. Each test required adding 0.75 mL buffer followed by 0.1mL whole blood to a cartridge, loading the cartridge into a module and running for 90 minutes. Two staff members verified results and conveyed them to the mother before discharge. Maternal admissions post-Caesarian section lasted three days while post-vaginal delivery stays ranged from six to 24 hours. Staff coverage for POCT was available on weekdays (during office hours) until September 2015 when Sunday coverage was added. Public holiday coverage was limited, particularly over December and January. The research funded study team consisted of four cousellors, one phlebotomist, two study nurses, one lab technician, two data capturers, one project manager, and a medical doctor.

POCT provided the following results: ‘HIV detected’, ‘HIV not detected’, ‘error’, ‘invalid’ and ‘no result’. The final three options were combined as ‘errors’ and analysed according to associated standardised instrument error codes. LABT provided the following results: ‘HIV positive’, ‘HIV negative’, ‘indeterminate’ (defined by laboratory cut-offs and representing either low-level viremia or false positives) and various ‘error’ results.

A positive LABT or POCT result prompted ART initiation and confirmatory testing by means of repeat LABT and an HIV viral load (VL) test (COBAS® AmpliPrep/COBAS® TaqMan® HIV-1 test, version 2.0 [Roche Molecular Systems, Inc., Branchburg, NJ]). We initiated ART on a positive POCT prior to receiving the LABT result from the concurrent blood draw only after the POCT was positive on re-run. Indeterminate LABT prompted further testing to establish HIV status. Management of infected neonates was undertaken on-site. All mothers received an appointment to collect their neonate’s LABT result within one week. If the LABT result was positive or indeterminate, a study nurse telephonically recalled the mother for an earlier visit. Results, tracing barcodes as well as the hospital’s contact details were recorded in the neonate’s patient held record with the mother’s permission. Written informed consent was obtained from all mothers and the study was approved by the Human Research Ethics Committees of the University of the Witwatersrand (M140639) and Columbia University.

Statistical Analysis

Staff recorded interview and laboratory data on paper forms that were captured electronically into REDCap.²² We calculated performance characteristics of POCT i.e. sensitivity, specificity, positive and negative predictive value (PPV and NPV) respectively and Cohen’s kappa coefficient assuming the final diagnostic outcome of the concurrent LABT was the gold standard. Rates and timing of results return and initiation of ART for infected neonates were described. The coverage of LABT and POCT was calculated as the percentage of identified HIV-exposed neonates in whom testing occurred and correlations were calculated (Spearman correlation). To compare groups for continuous variables, the Student’s T-test or Wilcoxon rank-sum tests were used, and for categorical variables the Chi-Square test or Fisher’s exact test. De-identified data were analyzed using SAS (Version 9.4, SAS Institute Inc., Cary, NC).

Role of the funding source

The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. We received a reduced price per cartridge from the manufacturer. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

Between 1 October 2014 and 30 April 2016, 18268 women delivered live births at RMMCH and 4267 (23.4%) were HIV-positive with 4336 HIV-exposed neonates delivered (Figure 1). Mothers of 4141 (95.5%) HIV-exposed neonates were offered infant birth testing. LABT uptake was high with mothers of 4112 (99.3%) consenting to testing. In 78 neonates with consent (1.9%), a test was not performed due to early neonatal death, mother departing before venesection or staff unavailability. Thus 4034 of 4336 HIV-exposed neonates born, had LABT yielding a coverage rate of 93.0%. Mothers of 3970 (98.4%) neonates with LABT agreed to study participation of which 2238 (56.4%) had concurrent POCT. Therefore POCT coverage of all HIV-exposed neonates born (n=4336) was 51.6%. Supplementary Table 1 describes the study population. Mothers of neonates who underwent POCT had marginally longer ART exposure during pregnancy (23 vs 21 weeks, p=0.010) and were more likely to have attended antenatal care (97% vs 95%, p=0.0053) but had similar rates of being on ART by delivery (93%). Neonates who underwent POCT were less likely to have required admission (5% vs. 23%, p<0.0001) and had lower rates of birthweight <2.5kg (15% vs. 25%, p<0.0001) and prematurity (20% vs. 28%, p<0.0001).

Figure 1. Flow diagram showing numbers of HIV-positive mothers identified with subsequent coverage of birth PCR testing with laboratory-based testing (LABT) and point of care testing (POCT).

†includes 67 twins and two triplets.

Temporal trends in coverage of LABT and POCT are shown in the time-series line plots in Figure 2. Coverage of both tests was reduced during year-end holiday periods. Coverage of LABT was maintained at >90%. LABT coverage during the holidays in December 2015/January 2016 was low at (87.0%) but was improved compared to the previous year (December 2014/January 2015) during which time it was 49.3%. A median of seven LABT tests were conducted each day (interquartile range [IQR]: 5–9, range: 1–16). Coverage of POCT peaked at 81.1% in February 2016 but was less than 40% at both year-ends. POCT was performed on 317 (54.8%) of the 578 study days, during which POCT coverage was 83.3%. A median of six POCT tests were conducted each day (IQR: 5–8, range: 1–13). Overall, monthly coverage of the two tests were moderately correlated (r=0.53, p=0.014).

Figure 2. Monthly coverage of birth HIV PCR testing by Laboratory-based testing (LABT) and Point of Care testing (POCT).

Coverage was defined as the percentage of HIV-exposed neonates with either LABT or POCT out of all HIV-exposed neonates identified in each month.

LABT of 3970 neonates produced 54 positive (1.4%; 95% CI: 1 .0–1.7), 3876 negative (97.6%; 95% CI: 97.2–98.1), 12 indeterminate (0.3%; 95% CI: 0.1-0.5) and 28 error (0.7%; 95% CI: 0.4–1.0) results. Confirmatory HIV testing for the 54 positive neonates demonstrated repeat positive HIV PCR results or a VL >1000 copies per mL on subsequent samples in 51 cases. In three neonates confirmation was not possible. Repeat sampling and testing of the 12 neonates with indeterminate results confirmed HIV infection in three cases, demonstrated no evidence of HIV infection in seven cases and the diagnosis remained indeterminate in two cases. Of 28 neonates with LABT errors, 23 had repeat testing all with negative results. Thus final HIV diagnostic outcome by LABT for 3970 neonates tested at birth was 57 positive (1.4%; 95% CI: 1.1–1.8), 3906 negative (98.5%; 95% CI: 98.1–98.9), two indeterminate (0.1%; 95% CI: 0.0–0.1) and five with no results due to inability to repeat the sample. POCT results for 2238 neonates tested concurrently comprised 32 positive (14%; 95% CI: 0.9–1.9), 2098 negative (94.0%; 95% CI: 92.7–94.7) and 108 error (4.8%; 95% CI: 3.9–5.7) results.

Table 1 compares the initial result of the POCT to the final diagnostic outcome on LABT. The Cohen’s kappa coefficient of 0.967 (95% CI: 0.922–1) indicates excellent agreement between the tests. Using the final diagnostic outcome on the LABT as the gold standard, the sensitivity of POCT was 100% (exact CI: 88.4–100) since all LABT-identified infected neonates were also identified by POCT. The specificity of POCT was estimated to be 99.9% (exact CI: 99.7–100) as two false positive POCT results occurred however, on repeat POCT of the same sample, results were negative. Of the 108 (4.8%) errors on the first POCT test, 103 samples were rerun yielding 94 (91.3%) negative results and nine (9.7%) persisting errors. When three of the latter samples were repeated for a third time, all tested negative. Therefore 97 of 108 initial POCT errors tested negative on repeat POCT reducing the final POCT error rate to 11 (0.5%) of 2238 tests.

Table 1.

Comparison of the initial result on Point of Care Testing (POCT) to final diagnostic outcome on Laboratory-based testing (LABT)

		LABT
		Positive	Negative	Indeterminate	Error	Total
POCT	Positive	30	2*	0	0	32
	Negative	0	2097	1†	0	2098
	Error	0	108	0	0	108
	Total	30	2207	1	0	2238
	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Kappa statistic
	1 (0.884-1)	0.999 (0.997-1)	0.938 (0.792–0.992)	0.995 (0.991–0.998)	0.967 (0.922-1)

PCR = polymerase chain reaction, CI= exact confidence interval, PPV = positive predictive value, NPV = negative predictive value.

^*Since these two false positive POCT results re-tested as negative, if the final POCT diagnosis is used sensitivity, specificity, PPV and NVP are all 1.

^†This neonate had further indeterminate and later negative results and the diagnosis remains uncertain.

The causes and temporal frequency of the 117 errors (108 on first test and nine on rerun) are summarised in Supplementary Figure 1. Mechanical/cartridge-related errors were the most frequent (41%) followed by sample volume errors (38%). Half of the sample volume errors occurred during a period when incorrect pipettes were supplied. Around half (54%) the mechanical/cartridge errors occurred in the months before modules required replacement. Most POCT (1864 [83%]) were run by the laboratory technician with an error rate of 4.3%. The remaining tests included 287 (13%) run by the project manager with an error rate of 7.0% and 87 (3.9%) tests run by nurses with an error rate of 9.2% (p=0.022).

Blood was sampled at a median of 14 hours (IQR: 8–21) after birth. The median turnaround time from phlebotomy to result release for LABT was 43 hours (IQR: 31–54) and for POCT was 2.6 hours (IQR: 2.3–3.1). Figure 3 outlines the average time for each step in the POCT process. In addition to the 90 minutes to run the POCT in the instrument, 64 minutes between phlebotomy and result release and 16 minutes for result return to the mother were required. Results were returned on the day of venesection except in 20 (1%) cases, when result returnoccurred the following morning where the mother was due for discharge the day before.

Figure 3. Median time in minutes per component between phlebotomy of neonate and Point of Care Testing (POCT) result return to mother.

IQR= interquartile range

The overall rate of POCT result return (96.3%) was significantly higher than for LABT result return (53.2%, p<0.0001). POCT results were also returned significantly earlier at a median of 1 day compared to 10 days for LABT results (p<0.0001, Table 2). For neonates who tested positive, the rate of POCT and LABT result return as well as ART initiation was similar with 100% (30/30) and 88.9% (24/27) of mothers receiving the birth test result and neonates starting ART (p=0. 10) with POCT vs. LABT respectively. However, the time to result return was significantly earlier for POCT at a median of one day in comparison to a median of 8 days for LABT results (p<0.0001). Infected neonates identified by POCT initiated ART at a significantly earlier age than infected neonates with only LABT (median one day vs. six days, p<0.0001, Table 2). Six of 24 HIV-infected neonates with LABT only initiated ART prior to their test results being returned to their mothers because of maternal illness or separation. This accounts for the discrepancies between median age at ART initiation and time to results return in the LABT only group.

Table 2.

Results return and ART initiation in neonates with Point of Care testing (POCT) and laboratory-based testing (LABT) results compared to those with LABT results only

	Total	POCT and LABT	LABT only	p
Final HIV test results, n	3970	2238	1732
Positive, n	57	30	27	0.077¶
Negative, n	3906	2207	1699
Indeterminate, n	2	1	1
No result, n	5	0	5‡
Results returned to mother, n (%)	3076 (77.5)	2155 (96.3)	921 (53.2)	<0.0001
Median age in days (IQR)	1 (1–8)	1 (0–1)	10 (9–13)	<0.0001
Positive results, n (%)	54 (94.7)	30 (100)	24 (88.9)	0.10 Ω
Median age in days (IQR)	2 (1–8)	1 (1–1)	8 (6–12)	<0.0001†
Negative results, n (%)	3021 (77.3)	2124 (96.2)	897 (52.8)	<0.0001
Median age in days (IQR)	1 (1–8)	1 (0–1)	10 (9–13)	<0.0001
ART initiated in infected neonates, n (%)	54 (94.7)	30 (100)	24 (88.9)	0.10Ω
Median age in days (IQR)	3 (1–8)	1 (1–2)	6 (5–10)	<0.0001†
Minimum, Maximum age in days	0;104	0;104	2;78

IQR=inter-quartile range, ART= antiretroviral treatment.

^¶p=0–56 when excluding indeterminate results and errors.

^‡In these five cases it was not possible to repeat the test after an initial error was encountered.

^ΩFisher Exact test

^†Wilcoxon test.

Discussion

In this field evaluation of the Cepheid Xpert® HIV-1 Qualitative Assay within a large universal birth testing program in South Africa, we demonstrated near perfect concordance between POCT and LABT in routine use. POCT facilitated result return to mothers by enabling more mothers to receive results and to receive them earlier. Active tracing ensured that infected neonates both with and without POCT received ART but POCT enabled earlier treatment. While weekday coverage of POCT was high, overall POCT coverage reached only 50% as the study was not set up to achieve full implementation.

The sensitivity of 100% and specificity of 99% demonstrated at birth is comparable to or better than that reported for Alere POCT at birth and for Cepheid POCT by the Early Infant Diagnostic Consortium.^15–18 The less than perfect specificity we observed was eliminated by an algorithm requiring a repeat test on all POCT positive results. Our two false positive POCT results tested negative on repeat testing of the same sample and on LABT. Since positive results have significant clinical implications, confirmatory testing on a separate sample is strongly recommended.

POCT results were received by 96% of mothers before discharge leading to significantly more mothers learning their child’s HIV status (96.3%) than for LABT alone (53.2%). Although not directly measured mothers generally agreed to have their neonate’s test information and results documented in their patient held record. Since we actively traced all mothers whose neonates had positive results, result return rates for positive neonates were excellent in both scenarios. All mothers of infected neonates in the POCT group received results and agreed to their neonates starting ART whereas only 88.9% in the LABT only group did. Three mothers whose infected neonates had a LABT only were discharged before results were available and never returned despite active tracing. Enhanced counselling and collection of more comprehensive contact details may mitigate this loss to follow-up. Neonates with a POCT were started on ART significantly earlier at a median of one versus six days. Whether very early initiation of ART, within 48 hours of birth, leads to better outcomes or increased risks is unknown but current clinical consensus supports the benefit of ART initiation upon diagnosis.^23–25

Further work on the psychological impact of diagnosis at birth is required. While dealing with their own diagnosis (commonly during the current pregnancy), POCT requires mothers to navigate coping with a diagnosis of their neonate at delivery. During EID, the focus of healthcare is more often on the infant than the HIV-infected mother’s health.^26,27 EID at six weeks was associated with beneficial and detrimental psychosocial effects.²⁸ Less is known about the effect of an even earlier diagnosis. In some instances maternal illness and incapacitation following delivery prevented neonatal test results being communicated to the mother before initiating neonatal ART, highlighting the vulnerability of mothers and their infected neonates.

Privacy and space to perform large volumes of POCT precluded placing the POCT instrument in close proximity to the postnatal discharge ward. Instead it was situated in a separate satellite laboratory on another floor in the hospital complex. This arrangement explains the relatively long delay between blood draw and results return to mothers. The high daily case load and staff constraints made it practical for two to three blood samples to be collected before proceeding to the POCT instrument, accounting for the median of 40 minutes between phlebotomy and samples reaching the instrument (Figure 3). The logistics of how POCT will be integrated into any clinical site needs careful consideration of these factors. In high case load settings, clinical staff in the postnatal wards may require at least one or more dedicated staff members for POCT implementation. Another potential solution, is implementation of POCT by building a “network of testers” within an institution.²⁹

The initial error rate observed with POCT (4.8%) was 7-fold higher than with LABT (0.7%), higher than the 3% reported in the WHO pre-qualification report³⁰ and lower than the 10% recently reported for the Alere Q POCT for birth testing.¹⁶ Repeating the POCT on the same sample reduced the error rate to 0.5%. Closer analysis of the POCT errors revealed many modifiable factors. These included site-specific problems including a dusty environment and power interruptions during renovations, but also manufacturer/instrument-related issues such as faulty modules and incorrect pipette supply. Achieving 3% error rates or less will require ongoing training, supervision and monitoring and engaging the manufacturer’s support.^29,30

Birth testing coverage with LABT as standard of care exceeded 90% within 23 months of implementation, however POCT coverage averaged around 50% after 19 months of implementation because of constraints imposed by the study setting. These included a finite study team unable to provide POCT outside of office hours and having to obtain informed consent prior to venesection for POCT whereas LABT only was possible without study consent. Because we opted not to rebleed neonates, this limited our ability to enrol admitted neonates. Therefore, the POCT cohort was generally comprised of healthier neonates at birth whereas sick neonates may have been prioritised for POCT if it were available outside of the study setting. Although the study setting was a barrier to assessing routine implemention of POCT at birth, the experience gained is described to pre-empt challenges that sites may face such as requiring additional resources to integrate POCT into standard of care. Other limitations include a single study site which may limit generalisability.

POCT is an accurate and useful tool for birth HIV testing that increases overall result return rates, reduces time to result and enables earlier ART initiation for HIV-infected neonates. Implementation is challenging and requires careful consideration and innovative approaches particularly for busy sites in high maternal HIV prevalence settings. Further research is required to assess the benefits of very early ART initiation in infected children to establish whether this specific consequence of POCT warrants the additional resources required to implement birth POCT. More information is needed on the psychosocial impact on mothers of very ealy diagnosis as well as cost-benefit analysis of additional resources needed for full implementation.