Abstract
The Ebola virus disease (EVD) outbreak in West Africa has highlighted an urgent need for point-of-care (POC) assays for the diagnosis of this devastating disease in resource-limited African countries. The diagnostic performance characteristics of a prototype Cepheid GeneXpert Ebola POC used to detect Ebola virus (EBOV) in stored serum and plasma samples collected from suspected EVD cases in Sierra Leone in 2014 and 2015 was evaluated. The GeneXpert Ebola POC is a self-contained single-cartridge automated system that targets the glycoprotein (GP) and nucleoprotein (NP) genes of EBOV and yields results within 90 min. Results from 281 patient samples were compared to the results of a TaqMan real-time reverse transcription-PCR (RT-PCR) targeting the polymerase gene and performed on two real-time PCR machines. Agreement between the three platforms was 100% at cycle threshold (CT) values of ≤34.99, but discordant results were noted between CT values of 35 and 45.The diagnostic sensitivity of the three platforms was 100% in 91 patient samples that were confirmed to be infectious by virus isolation. All three molecular platforms detected viral EBOV RNA in additional samples that did not contain viable EBOV. The analytical sensitivity of the GeneXpert Ebola POC for the detection of NP was higher, and comparable to that of polymerase gene detection, than that for the detection of GP when using a titrated laboratory stock of EBOV. There was no detectable cross-reactivity with other hemorrhagic fever viruses or arboviruses. The GeneXpert Ebola POC offers an easy to operate and sensitive diagnostic tool that can be used for the rapid screening of suspected EVD cases in treatment or in holding centers during EVD outbreaks.
INTRODUCTION
The unprecedented scale of Ebola virus disease (EVD) outbreaks in West Africa from 2013 to 2015 caused by the Ebola virus (EBOV) represents a dramatic expansion of case numbers and the introduction of this highly lethal disease into new geographic areas (1, 2). As of 7 October 2015, the World Health Organization reported a total of 28,421 EVD cases (confirmed, probable, and suspected) of which 11,297 (39.7%) were fatal, including 881 confirmed cases among health care workers of which 513 (58.2%) were fatal (3). The diagnostic burden of the largest EVD outbreak in the recorded history of the disease has been mostly borne by mobile laboratories, deployed throughout the affected countries by international agencies and institutes. Delays in the diagnosis of suspected EVD cases due to sample transport from remote areas to these laboratories have put additional pressure on outbreak control efforts. The use of a wide range of assays, often not clinically validated, has complicated the interpretation and consolidation of results from different laboratories. Rapid and accurate diagnostic results have a great impact on the management of suspected cases and on the tracing of contacts. The extent to which qualitative cycle threshold (CT) values from real-time reverse transcription-PCR (RT-PCR) assays correlate with the infectious states of patients is not well understood. Patients are often kept in isolation until three consecutive blood samples, collected days apart, are found negative by RT-PCR. High CT value results are often recorded in the blood of recovering patients for several days after clinical recovery (National Institute for Communicable Diseases [NICD], unpublished data). It is unclear whether these high CT values infer that the patient still represents a risk for spreading infection.
The 2013 to 2015 West African EVD outbreak is caused by the Ebola virus (species Zaire ebolavirus), one of five species in the Ebolavirus genus, which is known to cause a high fatality rate in infected humans (60% to 90%) (4). The EBOV genome is a negative-sense, single-strand RNA consisting of 18,959 nucleotides encoding seven structural proteins and one nonstructural protein. After 2 to 21 days of incubation, the disease presents initially with flu-like symptoms such as fever, malaise, and myalgia followed by vomiting, diarrhea, abdominal pain, edema, neurological signs, and hemorrhagic manifestations such as rash, petechiae, and bleeding from puncture sites (4).The nonspecific clinical presentation of viral hemorrhagic fevers (VHFs) and their high risk for nosocomial spread highlight the importance of accurate and rapid laboratory diagnosis. Diagnosis of infection by a filovirus can be achieved by the detection of antigen, the detection of virus nucleic acid, the isolation of virus, or by the detection of a virus-specific antibody response. Antigen can be detected in serum and other body fluid samples by antigen detection enzyme-linked immunosorbent assay (ELISA) (5) and indirect immunoelectron microscopy (6) and in skin biopsy specimens by immunohistochemistry (7). Detection of immunoglobulin (Ig) M and G specific to filoviruses can indicate a recent or past infection (8). Traditionally, filoviruses have been isolated successfully in vitro in African green monkey cell cultures (Vero) (9) and in suckling mice (10). In recent years, filovirus diagnostics have relied mostly on real-time RT-PCR assays using fluorogenic probes (11–19). Two prototype point-of-care assays that detect viral antigen by using lateral flow technology were recently evaluated using clinical specimens from suspected EVD cases in Sierra Leon (20, 21).
In this study, we evaluated the diagnostic performance of the prototype GeneXpert (Cepheid, Sunnyvale, CA, USA) Ebola assay in serum and plasma samples from suspected EVD cases in Sierra Leone. The results of this assay were directly compared to real-time RT-PCR targeting of the polymerase (L) gene run on two field deployable real-time PCR platforms, SmartCycler (Cepheid, Sunnyvale, CA, USA) and LightCycler Nano (Roche, Basel, Switzerland). The sensitivity of the three molecular diagnostic platforms was compared to that of virus isolation in Vero E6 cell culture.
MATERIALS AND METHODS
Analytical sensitivity.
To determine and compare the analytical sensitivity of the GeneXpert Ebola assay to that of the polymerase gene-based TaqMan RT-PCR run on two real-time platforms, a log dilution series of stock Ebola virus (SPU220/96; passage 4 Vero; 1 × 105.0 50% tissue culture infective dose [TCID50]/ml) was prepared in culture medium (Eagle's minimal essential medium [EMEM]) and tested in quadruplicate.
Analytical specificity.
To evaluate the cross-reactivity of the GeneXpert Ebola assay with selected hemorrhagic fever and arthropod borne viruses (arboviruses), stocks of the following viruses were tested: Sudan and Marburg viruses (Filoviridae); Lassa and Lujo viruses (Arenaviridae); Rift Valley fever and Crimean-Congo hemorrhagic fever viruses (Bunyaviridae); West Nile, Yellow fever, and Dengue type 1 to 4 viruses (Flaviviridae); and Chikungunya and Sindbis viruses (Alphaviridae). More detailed individual isolate information can be found in Table 1.
TABLE 1.
Family | Virus | Isolate (cell line) | Virus concn per milliliter | GeneXpert result (GP and NP target) |
---|---|---|---|---|
Filoviridae | Sudan | 276/00/6 (Passage 2 Vero) | 2.2 × 108.0 RNA copies | Negative |
Marburg | Watsa/DRC 148/99/1 (passage 2 Vero) | 1 × 105.75 TCID50 | Negative | |
Arenaviridae | Lassa | Luga L319 (passage 5 Vero) | 1 × 106.7 FFUa | Negative |
Lujo | GM serum (passage 5 Vero) | 1 × 108 FFU | Negative | |
Bunyaviridae | Rift Valley fever | 1981 V20368 (passage 1 BHK) | 1 × 106.75 TCID50 | Negative |
Crimean-Congo hemorrhagic fever | SPU 4/81 (passage 21 Vero) | 1 × 107.6 TCID50 | Negative | |
Flaviviridae | West Nile | SPU 116/89 (passage 5 Vero) | 1 × 108.25 TCID50 | Negative |
Yellow fever | A9/86 (passage 2 Vero) | 1 × 105.25 TCID50 | Negative | |
Dengue serotype 1 | Prototype TVP 2172 3/22/89 (passage 2 Vero) | 5.4 × 106.0 RNA copies | Negative | |
Dengue serotype 2 | NGC TVP 10863 7/2/2011 (passage 2 Vero) | 1.6 × 106.0 RNA copies | Negative | |
Dengue serotype 3 | H87 TVP 17541 8/10/2012 (passage 2 Vero) | 4.7 × 106.0 RNA copies | Negative | |
Dengue serotype 4 | SA216/15 (passage 1 Vero) | RNA copies unknown (CT, 17.22) | Negative | |
Alphaviridae | Chikungunya | H817 (passage 11 C6-36) | 1 × 107.5 TCID50 | Negative |
Sindbis | AR86 (passage 10 C6-36) | 1 × 108.5 TCID50 | Negative |
FFU, fluorescence focus units.
Diagnostic sensitivity.
Diagnostic sensitivity was evaluated using 281 blood specimens from suspected EVD cases submitted from August 2014 through March 2015 to the field Ebola Molecular Laboratory of the Centre for Emerging and Zoonotic Diseases of the National Institute for Communicable Diseases (NICD) in Freetown, Sierra Leone. This field laboratory was established as a part of the WHO Global Outbreak Alert and Response Network international outbreak response. Serum was separated from clotted blood and plasma from EDTA, and the resulting samples were stored at −70°C before shipment on dry ice to the NICD biosafety level 4 facility (BSL-4) in Johannesburg, South Africa for further analysis and long-term storage.
Extraction of viral RNA.
RNA was extracted from clinical and laboratory generated samples (input, 140 μl) using the QIAamp viral RNA kit (Qiagen, Hilden, Germany) per the manufacturer's instructions and as previously described (22). The final elution volume was 60 μl.
Polymerase (L) gene TaqMan real-time RT-PCR.
The assay was performed as previously described (14, 22) using the primers and probes targeting the Ebola virus L gene and 5 μl RNA as the template. An in vitro transcribed RNA copy of the L gene was used as a positive control at a known copy number. The RNA standard was prepared as previously described (23) using L-gene-specific primers (14). All runs included two negative controls as per standard PCR practice, a no-template control and an extraction negative control. Two field-deployable real-time PCR platforms were used, the SmartCycler (Cepheid) using propriety single-reaction Smart Tubes and the LightCycler Nano (Roche) using 8-well strip tubes. The SmartCycler is operated with Cepheid SmartCycler Version 2.0d software, and the LightCycler Nano is operated with software version 1.0.7. Analysis on the two platforms' software was run with default analysis settings. Cycles were as follows: reverse transcription (50°C for 30 min), denaturation (95°C for 15 min), and amplification/detection (45 cycles of 95°C for 15 s, 52°C for 25 s plus acquisition, and 72°C for 20 s). Any fluorescence detected above the threshold before 45 cycles is regarded as positive by the software and assigned a CT value. The two platforms were deployed in the NICD Ebola Mobile Laboratory Unit (EMLU) in Sierra Leone from August 2014 to run the polymerase gene TaqMan real-time RT-PCR. The SmartCycler results were used to determine RNA copy numbers in patient samples.
GeneXpert Ebola assay.
The assay was performed on the GeneXpert IV system with the Cepheid GeneXpert Dx software package using disposable prototype GeneXpert Ebola cartridges. The assay integrates sample purification, nucleic acid amplification, and the detection of target a sequence in a single automated process. In addition to targeting EBOV nucleoprotein (NP) and glycoprotein (GP) genes, the assay includes a sample adequacy control (SAC; human housekeeping gene hydroxymethylbilane synthase [HMBS]) and an internal control (IC) to ensure the adequate addition of sample and to control for PCR inhibitors. A volume of 100 μl serum or plasma (or laboratory-generated samples) was added to the lysis reagent bottle (containing guanidinium thiocyanate) in the BSL-4 laboratory. Outside biocontainment, a volume of 1 ml of the sample/lysis mix was added directly to the GeneXpert Ebola cartridge sample well and was processed on the GeneXpert IV system. The cycle threshold (CT) values for NP and GP were obtained through the GeneXpert Dx software package along with a decision on the validity of a specific test based on the two internal controls (SAC and IC). Any fluorescence detected above the threshold with either target (NP or GP) before 45 cycles is regarded as positive by the software and assigned a CT value.
Virus isolation.
Serum or plasma samples that tested positive either by one or all of the above-mentioned molecular assays were subjected to virus isolation. Samples were diluted 1:5 in tissue culture medium (EMEM) prior to inoculation. Vero E6 cells at 80% to 90% confluence in 25 cm2 flasks were overlaid with 1 ml diluted sample and were incubated for 1 h at 37°C. After removal of the inoculum, fresh EMEM containing antibiotics (penicillin/streptomycin/amphotericin B) was added (10 ml), and the flasks were incubated at 37°C for 14 days or until cytopathic effects (CPE) were observed. After incubation (or at early signs of CPE), flasks were frozen at −70°C and were subsequently thawed at room temperature. The presence or absence of replicating virus in the culture supernatants was confirmed by real-time RT-PCR. Cultures in which the virus did not replicate in the first passage were subjected to a second passage by inoculating 1 ml of undiluted supernatant from passage one followed by incubation and testing as described above.
Statistics.
The percentage of agreement was calculated between different assays and virus isolation. Diagnostic accuracy parameters for the assays at a 95% confidence interval (CI) were calculated using MedCalc version 15.8 (www.medcalc.org). The following estimates were calculated: sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Basic calculations of means and standard deviations were done in Microsoft Excel 2007. Cutoff values for determining the CT value at which a patient sample is likely to yield an isolate on Vero E6 cells at the 95% accuracy level, using different gene targets or assay platforms, were optimized using the two-graph receiver operating characteristics (TG-ROC) analysis (24–26).
Ethics statement.
Approval to conduct this study was obtained from the Office of Sierra Leone Ethics and Scientific Review Committee (version 24/03/2015) and from the Human Research Ethics Committee of the University of the Witwatersrand, South Africa (clearance certificate number M150157). Clearance for the export of samples from Sierra Leone to South Africa was granted under export permit numbers PBSL/061/02/2015 and PBSL/063/02/2015 by the Pharmacy Board of Sierra Leone. Xpert Ebola assay (Cepheid) has WHO authorization for emergency use.
RESULTS
Analytical sensitivity and specificity.
Tenfold serial dilutions of live EBOV in culture medium from 10,000 to 0.01 TCID50/ml were tested in quadruplicate using the GeneXpert Ebola assay and the TaqMan RT-PCR targeting the virus polymerase (L) gene on the SmartCycler and LightCycler Nano platforms (Table 2). The L-gene-based RT-PCR on the two platforms detected virus RNA in all four replicates at 1.0 TCID50/ml. The CT value obtained with the LightCycler Nano was consistently lower than the value obtained with the SmartCycler at the same dilution (the smallest and largest differences were 3.4 and 6.9 CT values, respectively). At 0.1 TCID50/ml the LightCycler Nano yielded fluorescence in two of the four replicates compared to one replicate with the SmartCycler. Although the GeneXpert also detected the NP target in all four replicates at 1.0 TCID50/ml, the GP target could only be detected in all replicates down to 10 TCID50/ml. An additional single replicate yielded a detectable NP target at 0.1 TCID50/ml and a GP target at 1.0 TCID50/ml, respectively. No cross-reaction was detected using the GeneXpert Ebola assay with other hemorrhagic fever viruses and arboviruses tested (Table 1).
TABLE 2.
Ebola virus 220/96 passage 4 Veroa (TCID50/ml) | Stock virus titration |
Ebola GeneXpert assay |
SmartCycler |
LightCycler Nano |
||||||
---|---|---|---|---|---|---|---|---|---|---|
RNA copies per milliliterb | Ratio RNA copies to TCID50 | Positive GP replicatesc | CT (±SD)d | Positive NP replicatesc | CT (±SD)d | Positive replicatesc | CT (±SD)d | Positive replicatesc | CT (±SD)d | |
10,000 | 7,020,300 | 702 | 4/4 | 29.03 ± 0.22 | 4/4 | 24.4 ± 0.22 | 4/4 | 25.3 ± 0.19 | 4/4 | 21.9 ± 0.13 |
1,000 | 624,120 | 624 | 4/4 | 32.68 ± 0.21 | 4/4 | 28.25 ± 0.1 | 4/4 | 28.9 ± 0.12 | 4/4 | 25.5 ± 0.55 |
100 | 31,773 | 318 | 4/4 | 36.03 ± 0.37 | 4/4 | 31.68 ± 0.21 | 4/4 | 33.4 ± 0.36 | 4/4 | 28.9 ± 0.33 |
10 | 3,660.3 | 366 | 4/4 | 39.83 ± 0.95 | 4/4 | 35.33 ± 0.1 | 4/4 | 36.7 ± 0.51 | 4/4 | 32.7 ± 0.09 |
1 | 163.52 | 163 | 1/4 | 41.50 | 4/4 | 38.2 ± 0.08 | 4/4 | 42.1 ± 1.75 | 4/4 | 35.2 ± 0.26 |
0.1 | 73.92 | 739 | 0/4 | 1/4 | 41.3 | 1/4 | 41.1 | 2/4 | 41.2; 39.5 | |
0.01 | 0 | n/ae | 0/4 | 0/4 | 0/4 | 0/4 |
Ebola virus isolate 220/96, passage four on Vero cells, at a known titer of 1 × 105 TCID50/ml was used to prepare 10-fold serial dilutions in tissue culture medium.
RNA copies calculated using the data obtained on the SmartCycler.
Each dilution of the stock virus was tested in quadruplicate. Number positive out of number tested is shown.
The average CT value is indicated at each dilution. Standard deviation is shown where 4 replicates yielded a CT value.
n/a, not available.
Diagnostic accuracy.
A direct comparison was done between the GeneXpert Ebola assay and the L gene TaqMan assay on two platforms using two different CT cutoff values. Using the L gene assay run on the SmartCycler platform as a comparator, the percentages of agreement were calculated (Table 3). A total of 122 samples were regarded as positive and 159 were regarded as negative when using a CT cutoff value of 45; when using a CT of 40, 112 samples were regarded as positive and 169 were regarded as negative. The agreement between the assays was the highest when using a CT of 45 as the cutoff. When analyzing the data separately for the two targets in the GeneXpert assay, the agreement of the GP target was lower than that with NP regardless of the cutoff value used. The agreement was highest between the L gene TaqMan assay run on the two different platforms at a 45 CT cutoff but not at a 40 CT cutoff. Using the lower cutoff of 40 CT decreased the sensitivity (from 99.18% to 97.32%) and increased the specificity (from 97.48% to 98.22%) of the GeneXpert (one or both targets) compared to that of analysis with a 45 CT cutoff but had the inverse effect when analyzing the LightCycler Nano platform results (sensitivity increased from 98.36% to 99.11% and specificity decreased from 99.37% to 94.08%).
TABLE 3.
Positive or negative qRT-PCR result (no. of samples) | GeneXpert Ebola assaya |
LightCycler Nano L gene qRT-PCRa |
||||||
---|---|---|---|---|---|---|---|---|
GP target |
NP target |
One or both targets (GP and NP) |
||||||
+ | − | + | − | + | − | + | − | |
SmartCycler L gene qRT-PCR, positive (n = 122b and n = 112c) | 107b and 97c | 15b,c | 120b and 109c | 2b and 3c | 121b and 109c | 1b and 3c | 120b and 111c | 2b and 1c |
SmartCycler L gene qRT-PCR, negative (n = 159b and n = 169c) | 2b and 0c | 157b and 169c | 3b,c | 156b and 166c | 4b and 3c | 155b and 166c | 1b and 10c | 158b and 159c |
Agreement (%) | 93.95b | 94.66c | 98.22b | 97.86c | 98.22b | 97.86c | 98.93b | 96.09%c |
+, positive; −, negative.
CT cutoff value of ≤45 used.
CT cutoff value of ≤40 used.
Samples were further categorized according to the range of CT values obtained by the L gene TaqMan assay on the SmartCycler. At CT values of ≤34.99, there was a 100% agreement between all assays and platforms (Table 4). At CT values between 35 and 39.99, the agreement decreased slightly (93.75% for all assays), but it was much lower when analyzing specifically the GP target of the GeneXpert Ebola assay (56.25%). Between CT values of 40 and 45, the agreement was 90% between the L gene TaqMan run on the SmartCycler and on the LightCycler, 100% between the GeneXpert Ebola assay and the L gene SmartCycler, and only 20% and 90% when analyzing the two targets GP and NP, respectively.
TABLE 4.
SmartCycler L gene qRT-PCR CT value range (no. of samples) | GeneXpertEbola assaya |
LightCycler Nano L gene qRT-PCRa |
||||||
---|---|---|---|---|---|---|---|---|
GP target |
NP target |
One or both targets (GP and NP) |
||||||
+ | − | + | − | + | − | + | − | |
CT < 30 (n = 81) | 81 | 0 | 81 | 0 | 81 | 0 | 81 | 0 |
100%b | 100% | 100% | 100% | |||||
CT 30–34.99 (n = 15) | 15 | 0 | 15 | 0 | 15 | 0 | 15 | 0 |
100% | 100% | 100% | 100% | |||||
CT 35–39.99 (n = 16) | 9 | 7 | 15 | 1 | 15 | 1 | 15 | 1 |
56.25% | 93.75% | 93.75% | 93.75% | |||||
CT 40–45 (n = 10) | 2 | 8 | 9 | 1 | 10 | 0 | 9 | 1 |
20.0% | 90.0% | 100% | 90.0% | |||||
No fluorescence signal above the threshold (n = 159) | 2 | 157 | 3 | 156 | 4 | 155 | 1 | 158 |
98.74% | 98.11% | 97.48% | 99.37% |
+, Positive; −, negative.
Percentage agreement between applicable assay and SmartCycler L gene qRT-PCR using a cutoff CT of 45.
Of a total of 125 serum samples subjected to virus isolation, Ebola virus could be recovered from only 91 samples after a maximum of two passages on Vero E6. The agreement of the RT-PCR results on all three platforms with virus isolation as the standard reference was analyzed (Table 5). Agreement in samples from which EBOV was isolated was 100% with all assays when using CT 45 as the cutoff. At a CT cutoff of 40, only the agreement with the GP target in the GeneXpert assay decreased to 98.9%, while the others remained 100% in samples from which EBOV was isolated. Agreement in virus-isolation-negative samples was poor and varied between 2.9% and 82.35% depending on the assay and the cutoff used.
TABLE 5.
Positive or negative virus isolation | No. with GeneXpert Ebola assaya (CT value) |
No. with LightCycler Nano L-gene qRT-PCRa (CT value) |
No. with SmartCycler L-gene qRT-PCRa (CT value) |
|||||||
---|---|---|---|---|---|---|---|---|---|---|
GP target |
NP target |
One or both targets (GP and NP) |
||||||||
+ | − | + | − | + | − | + | − | + | − | |
Virus isolation, positive (n = 91) | 91b (22.2–40.5) and 90c (22.2–36.8) | 0b and 1c (40.5) | 91b,c (15.8–38.2) | 0b,c | 91b,c (15.8–40.5) | 0b,c | 91b,c (13.2–34.76) | 0b,c | 91b,c (14.77–38.62) | 0b,c |
Virus isolation, negative (n = 34) | 17b (28.5–44.6) and 6c range (28.5–38.8) | 17b (>45) and 28c (>40) | 31b (25.1–43.8) and 20c (25.1–39.8) | 3b (>45) and 14c (>40) | 33b (25.1–44.6) and 20c (25.1–39.8) | 1b (>45) and 14c (>40) | 19b,c (25.6–39.8) | 5b,c (>40) | 30b (26.9–42.9) and 20c (26.9–39.96) | 4b (>45) and 14c (>40) |
Agreement Virus isolation, positive (%) | 100b and 98.9c | 100b,c | 100b,c | 100b,c | 100b,c | |||||
Agreement Virus isolation, negative (%) | 50.0b and 82.35c | 8.8b and 41.2c | 2.9b and 41.2c | 14.3b,c | 11.8b and 41.2c |
+, positive; −, negative.
CT cutoff value of ≤45 used.
CT cutoff value of ≤40 used.
Infectivity versus CT values.
The CT values obtained by the different assays were compared to those of the virus isolation results. Two samples yielded clearly outlying results. Virus could not be isolated from one sample (date of collection post onset is unknown) yielding the following CT values on the different assays: 26.92 on SmartCycler, 25.64 on LightCycler Nano, and 28.5 and 25.1 on GeneXpert for GP and NP, respectively. Another sample (collected on day 12 post onset) yielded live virus but the following CT values: 38.62 on SmartCycler, 34.76 on LightCycler Nano, and 40.5 and 38.2 on GeneXpert for GP and NP, respectively. When discarding these outliers, a relatively narrow intermediate CT range was established for each assay and target wherein positive and negative virus isolation results would overlap (Fig. 1). For the L gene TaqMan assay, the ranges were CT 31.28 to 33.7 and CT 28.08 to 31.5 on the SmartCycler and LightCycler Nano platforms, respectively. On the GeneXpert, the ranges were CT 34.0 to 36.8 and CT 31.2 to 32.2 with the GP and NP targets, respectively. Excluding the outliers, values outside these ranges corresponded 100% to the infectivity or noninfectivity of the samples. The range of RNA copies per milliliter that corresponded consistently to successful virus isolation was from 9.12 × 109 to 1.33 × 105 copies/ml serum (excluding the outlier) while, within the range of 1.31 × 105 to 2.42 × 104 copies/ml, virus could not be isolated from all of the samples. Cutoff CT values for determining the infectivity of patient samples with the different assays were determined by two-graph receiver operating characteristic (TG-ROC) analysis. The cutoff on the SmartCycler platform was determined as 31.06 CT (yielding a sensitivity of 95.45% and a specificity of 97.06%). The LightCycler Nano cutoff was 28.56 (yielding a sensitivity of 93.18% and a specificity of 94.12%). The GeneXpert cutoff values were 34.21 (yielding a sensitivity of 92.05% and a specificity of 94.12%) for the GP and 31.09 (yielding a sensitivity of 96.59% and a specificity of 97.06%) for the NP targets.
The day of sample collection post disease onset was known for 98 of the RT-PCR-positive samples (actual range, day 0 to 33). Samples were arranged according to day of collection post onset in the following four groups: day 0 to 4 (n = 41), 5 to 10 (n = 42), 11 to 15 (n = 8), and 16 to 33 (n = 7). Mean CT values and standard deviations on each assay and platform were calculated for the four time groups (Fig. 2). Sample groups at earlier time points after disease onset yielded the lowest average CT values, with an increase in time leading to higher CT values. The percentage of samples from which Ebola virus could be recovered was highest early after onset, after which the number decreased with time, and none of the samples in the day 16 to 33 time group containing live virus.
DISCUSSION
The magnitude of the 2013 to 2015 Ebola virus disease epidemic in West Africa has highlighted the unpreparedness of the world to respond to massive transmission of this highly dangerous pathogen. An important aspect of the control of such outbreaks is access to rapid and reliable diagnostic capacity, often required in remote and resource-constrained areas. Management of suspected cases in Ebola treatment or holding centers is heavily dependent on laboratory testing to ensure that infected patients are isolated in a timely manner and that noninfected patients are released. The availability of accurate, reliable point-of-care diagnostics would contribute greatly to the better management of infected and noninfected patients and thereby decrease the risk of unnecessary exposure of noninfected patients. Point-of-care diagnostic capacity was lacking during most of the West African outbreak, leading to complete dependence on mobile laboratories. The countrywide transmission in each of the most affected countries and the low number of laboratories performing Ebola diagnostics resulted in delays from patient submission to laboratory confirmation due to long distances, poor road infrastructure, and the lack of reliable sample transport.
Although real-time RT-PCR assays are currently widely used for Ebola virus diagnostics by reference laboratories worldwide and by all mobile laboratories deployed in West Africa, they were not intensively validated in the field against the gold standard, the virus isolation, mostly due to limited availability of clinical specimens in the past. Thus, there is no standardized molecular reference test to compare and validate new prototype assays. To our knowledge, this is the first evaluation of a POC molecular assay in direct comparison to virus isolation in clinical specimens and the largest clinical evaluation of currently in-use real-time RT-PCR targeting the L gene. To illustrate the effect of PCR equipment choice, we also evaluated the performance of the L gene real-time RT-PCR run on the Roche LightCycler Nano platform.
The analytical sensitivity and specificity of the GeneXpert Ebola assay, which targets GP and NP genes, were compared to those of a TaqMan-based reverse transcription-quantitative PCR (qRT-PCR) targeting the polymerase gene and run on the Cepheid SmartCycler and Roche LightCycler Nano platforms using a laboratory-generated virus dilution series. The GeneXpert assay did not yield any cross-reaction to the hemorrhagic fever viruses or arboviruses tested in this study. The limit of EBOV detection for all of the assays was 1.0 TCID50/ml (corresponding to 163 L gene RNA copies per milliliter or 1.94 copies per reaction), where all four replicates at this dilution were detected. At 0.1 TCID50/ml, detection was intermittent by all assays (74 L gene RNA copies per milliliter or 0.88 copies per reaction). It is important to note that detection of the GP target gene was less sensitive than detection of the NP target gene in the GeneXpert assay, with a detection limit of 1 log10 less. A similar trend has been observed previously, albeit in conventional PCR format, where the amount of DNA amplified by an RT-PCR targeting the L gene was greater than that by a GP-targeting assay on the same samples (12). Interestingly, in the same study, it was found that the detection of NP was 125-fold more sensitive than the detection of the L gene, an observation that we could not reproduce in this study with the lab-generated virus titration series. The inherent multiplex characteristic of the GeneXpert assay may explain the difference. RT-PCR protocols following the one-step principle allow detection of genomic- and antigenomic-sense (messenger) RNA (18). Although Ebola virus has a linear, nonsegmented negative-sense RNA genome, inferring an equal number of copies of each of the virus' genes per particle, it is possible that some genes are transcribed in higher numbers than others, leading to a higher number of detectable copies (including mRNA) relative to other genes during infection. The number of RNA copies is not a direct indication of the number of virus particles, as RNA copies are consistently between 3 and 4 log10 higher than PFU (18) or between 2 and 3 log10 higher than TCID50 in our study.
The GeneXpert system is designed to use whole blood as the sample input. It has been shown that there is earlier clearance of virus from serum and plasma than from whole blood (27). Testing of whole blood has an important practical and safety advantage since it does not require specimen processing. In our study, we could only evaluate and directly compare the different assays using stored serum and plasma samples from suspected EVD cases. All assays yielded high estimates of diagnostic accuracy. As expected, when using a lower CT cutoff value to characterize a sample as positive or negative, the sensitivity decreased slightly but the specificity increased. The lack of agreement between the L-gene-based assay and the GeneXpert assay at CT values of >40 may be due to the fact that the latter is more sensitive in detecting borderline concentrations of RNA. This emphasizes the importance of the careful interpretation of results yielding high CT values, even above CT values of 35, together with clinical data and the exposure history of the patient. It appears that the performance of the GeneXpert assay is highly dependent on the detection of the NP gene rather than on the GP gene. There were only two samples where the GP gene target was detected (CT, >40) but where NP was not detected, compared to 16 samples where the NP gene target was detected (actual CT range, 38.4 to 43.8) but where GP was not detected.
Arguably, a more appropriate way to directly compare performance is to look at samples grouped according to different ranges of CT values. All samples yielding CT values of ≤34.99 (on the SmartCycler) were detected by all of the assays (100% agreement), with agreement decreasing in lower positive samples. The value of detecting NP and GP in the GeneXpert assay is illustrated when analyzing the results at a CT range of 40 to 45. When detection of GP and NP is analyzed separately at this range, the agreement to the L gene assay is 20% and 90%, respectively. However, when following the principle of regarding a sample as positive with the detection of either GP or NP or both, the agreement becomes 100%. Another important analysis was to demonstrate that the assays are reliable for the detection of RNA in samples that contain live virus. All of the assays were able to detect RNA in samples that tested positive by virus isolation. As expected, all of the assays detected RNA in a number of samples that did not contain live virus. Similarly to what was reported for Marburg virus in bats (28), we found a clear correlation between CT values (RNA copies) and the ability to isolate virus from the clinical specimens. Marburg virus could only be isolated from bat tissues that yielded CT values of <35 (28). In our own study, we were also unable to isolate EBOV from the blood and tissues of experimentally infected bats with CT values of <35 (NICD, unpublished data).
A direct correlation between viral RNA levels determined by qRT-PCR and the ability to isolate EBOV suggests potentially important practical aspects in terms of identifying infectious or noninfectious samples. In our study, we identified an intermediate range where there was an overlap of CT values yielding positive virus isolation and not, and this CT range was different for the different assays. Excluding the limited outliers, all of the assays perform 100% at CT values of <35, and all are able to detect RNA in samples from which virus was isolated.
Although we do not have matching serology data available, another explanation for the detection of RNA in virus-isolation-negative samples may be that the RNA detected in the negative isolation samples represents virus that is in the process of being cleared in the form of immunocomplexes. This may be supported by the fact that the percentage of successful virus isolation decreased over time post onset (Fig. 2). Considering that, there are numerous factors that can have an effect on the outcome of a diagnostic test on a patient's clinical sample. Sample quality is an obvious factor that includes various aspects such as volume, hemolysis, cold-chain transport, and storage. Probably a more important factor is the timing of sample collection post disease onset. A sample collected too early or too late after disease onset can yield a false-negative result depending on which analyte is targeted. The duration of viremia caused by infection with different viruses differs as does the time needed to develop a detectable antibody response. For this reason, the diagnosis of a viral hemorrhagic fever should ideally not depend on a single test result (single analyte), especially in the case of excluding VHF as a diagnosis.
In this study, we showed that the prototype GeneXpert Ebola assay was highly accurate in the detection of RNA in serum and plasma samples containing live Ebola virus. The agreement between the different assays we compared was very high at low CT values and decreased with lower positive samples. Despite the sample input volume of the GeneXpert assay being only 100 μl compared to the 140 μl used for manual RNA extraction, it did not noticeably affect assay sensitivity. The prototype GeneXpert Ebola assay presents several advantages over currently available qRT-PCR protocols. The assay incorporates an automated extraction and sample addition process, making it possible to be run by minimally trained technicians within 90 min. If placed within Ebola treatment centers, the GeneXpert system would negate the need for additional biocontainment devices where patient samples first have to be inactivated, processed, and RNA extracted. This technology minimizes the possibility of human error, for example, during the extraction process, RT-PCR master mix preparation, and sample addition, by being automated. One other advantage of the assay is the usage of stable reagents. The ongoing Cepheid shelf-life testing demonstrates that cartridges used for Xpert Ebola POC are stable under room temperature for at least 6 months (Cepheid, personal communication).
Hemoglobin and lactoferrin have been identified as major inhibitors of diagnostic PCR in human blood cells (29). However, an internal control included in the Ebola Xpert POC ensures the detection of inhibitory effects from factors possibly present in patient samples.
False-negative results in patients with severe hemorrhagic fever have been noted before (30). Therefore, analytical results obtained by this and any other test should be interpreted by trained and experienced diagnosticians together with all available laboratory results and clinical, pathological, and epidemiological data to ensure an accurate diagnosis. In conclusion, the prototype GeneXpert Ebola assay represents a promising point-of-care screening tool to make rapid presumptive decisions about patient management and infection control measures.
ACKNOWLEDGMENTS
The authors would like to dedicate this work to and acknowledge all individuals who contributed and who are still contributing to the fight against the Ebola epidemic in West Africa. In particular, we thank the staff of the Ministry of Health and Sanitation of Sierra Leone, the staff of the National Institute for Communicable Diseases, the National Department of Health of South Africa, the World Health Organization, and the Global Outbreak Alert and Response Network. We also thank Lynsey Isherwood for helping in preparation of ethics applications and related project logistics.
J.T.P., P.J.V.V., I.S., and A.K. conceived and designed the study. P.J.V.V., N.S., O.C., K.K., and J.T.P. were responsible for sample processing, long-term storage, and database management in Sierra Leone. P.J.V.V., N.S., and A.G. executed the laboratory work. P.J.V.V., A.G., and J.T.P. interpreted and analyzed the data. P.J.V.V. drafted the manuscript after which it was critically reviewed by all coauthors.
Funding Statement
The NICD was supported by Cepheid by provision of the GeneXpert Ebola POC IV system, the GeneXpert Dx software package, disposable prototype GeneXpert Ebola cartridges, and funds for covering the costs of running polymerase (L) gene TaqMan real-time RT-PCR assays.
REFERENCES
- 1.WHO Ebola Response Team. 2014. Ebola virus disease in West Africa–the first 9 months of the epidemic and forward projections. N Engl J Med 371:1481–1495. doi: 10.1056/NEJMoa1411100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Carroll MW, Matthews DA, Hiscox JA, Elmore MJ, Pollakis G, Rambaut A, Hewson R, García-Dorival I, Bore JA, Koundouno R, Abdellati S, Afrough B, Aiyepada J, Akhilomen P, Asogun D, Atkinson B, Badusche M, Bah A, Bate S, Baumann J, Becker D, Becker-Ziaja B, Bocquin A, Borremans B, Bosworth A, Boettcher JP, Cannas A, Carletti F, Castilletti C, Clark S, Colavita F, Diederich S, Donatus A, Duraffour S, Ehichioya D, Ellerbrok H, Fernandez-Garcia MD, Fizet A, Fleischmann E, Gryseels S, Hermelink A, Hinzmann J, Hopf-Guevara U, Ighodalo Y, Jameson L, Kelterbaum A, Kis Z, Kloth S, Kohl C, Korva M, et al. . 2015. Temporal and spatial analysis of the 2014-2015 Ebola virus outbreak in West Africa. Nature 524:97–101. doi: 10.1038/nature14594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.World Health Organization. 2015. Ebola situation report—7 October 2015. World Health Organization, Geneva, Switzerland. [Google Scholar]
- 4.Feldmann H, Geisbert TW. 2011. Ebola haemorrhagic fever. Lancet 377:849–862. doi: 10.1016/S0140-6736(10)60667-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ksiazek TG, Rollin PE, Jahrling PB, Johnson E, Dalgard DW, Peters CJ. 1992. Enzyme immunosorbent assay for Ebola virus antigens in tissues of infected primates. J Clin Microbiol 30:947–950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Geisbert TW, Rhoderick JB, Jahrling PB. 1991. Rapid identification of Ebola virus and related filoviruses in fluid specimens using indirect immunoelectron microscopy. J Clin Pathol 44:521–522. doi: 10.1136/jcp.44.6.521. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Zaki SR, Shieh W-J, Greer PW, Goldsmith CS, Ferebee T, Katshitshi J, Tshioko FK, Bwaka MA, Swanepoel R, Calain P, Khan AS, Lloyd E, Rollin PE, Ksiazek TG, Peters CJ. 1999. A novel immunohistochemical assay for the detection of Ebola virus in skin: implications for diagnosis, spread and surveillance of Ebola haemorrhagic fever. J Infect Dis 179(Suppl):S36–S47. doi: 10.1086/514319. [DOI] [PubMed] [Google Scholar]
- 8.Ksiazek TG, West CP, Rollin PE, Jahrling PB, Peters CJ. 1999. ELISA for the detection of antibodies to Ebola viruses. J Infect Dis 179(Suppl):S192–S198. doi: 10.1086/514313. [DOI] [PubMed] [Google Scholar]
- 9.Moe JB, Lambert RD, Lupton HW. 1981. Plaque assay for Ebola virus. J Clin Microbiol 13:791–793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.van der Groen G, Jacob W, Pattyn SR. 1979. Ebola virus virulence for newborn mice. J Med Virol 4:239–240. doi: 10.1002/jmv.1890040309. [DOI] [PubMed] [Google Scholar]
- 11.Weidmann M, Mühlberger E, Hufert FT. 2004. Rapid detection protocol for filoviruses. J Clin Virol 30:94–99. doi: 10.1016/j.jcv.2003.09.004. [DOI] [PubMed] [Google Scholar]
- 12.Sanchez A, Ksiazek TG, Rollin PE, Miranda MEG, Trappier SG, Khan AS, Peters CJ, Nichol ST. 1999. Detection and molecular characterization of Ebola viruses causing disease in human and nonhuman primates. J Infect Dis 179(Suppl):S164–S169. doi: 10.1086/514282. [DOI] [PubMed] [Google Scholar]
- 13.Onyango CO, Opoka ML, Ksiazek TG, Formenty P, Ahmed A, Tukei PM, Sang RC, Ofula VO, Konongoi SL, Coldren RL, Grein T, Legros D, Bell M, De Cock KM, Bellini WJ, Towner JS, Nichol ST, Rollin PE. 2007. Laboratory diagnosis of Ebola hemorrhagic fever during an outbreak in Yambio, Sudan, 2004. J Infect Dis 196:S193–S198. doi: 10.1086/520609. [DOI] [PubMed] [Google Scholar]
- 14.Panning M, Laue T, Olschlager S, Eickmann M, Becker S, Raith S, Courbot MC, Nilsson M, Gopal R, Lundkvist A, di Caro A, Brown D, Meyer H, Lloyd G, Kummerer BM, Gunther S, Drosten C. 2007. Diagnostic reverse-transcription polymerase chain reaction kit for filoviruses based on the strain collections of all European biosafety level 4 laboratories. J Infect Dis 196:S199–S204. doi: 10.1086/520600. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Leroy EM, Baize S, Lu CY, McCormick JB, Georges AJ, Georges-Courbot MC, Lansoud-Soukate J, Fisher-Hoch SP. 2000. Diagnosis of Ebola haemorrhagic fever by RT-PCR in an epidemic setting. J Med Virol 60:463–467. doi:. [DOI] [PubMed] [Google Scholar]
- 16.Gibb TR, Norwood DA Jr, Woollen N, Henchal EA. 2001. Development and evaluation of a fluorogenic 5′ nuclease assay to detect and differentiate between Ebola virus subtypes Zaire and Sudan. J Clin Microbiol 39:4125–4130. doi: 10.1128/JCM.39.11.4125-4130.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Trombley AR, Wachter L, Garrison J, Buckley-Beason VA, Jahrling J, Hensley LE, Schoepp RJ, Norwood DA, Goba A, Fair JN, Kulesh DA. 2010. Comprehensive panel of real-time TaqMan polymerase chain reaction assays for detection and absolute quantification of filoviruses, arenaviruses and new world hantaviruses. Am J Trop Med Hyg 82:954–960. doi: 10.4269/ajtmh.2010.09-0636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Towner JS, Rollin PE, Bausch DG, Sanchez A, Crary SM, Vincent M, Lee WF, Spiropoulou CF, Ksiazek TG, Lukwiya M, Kaducu F, Downing R, Nichol ST. 2004. Rapid diagnosis of Ebola hemorrhagic fever by reverse transcription-PCR in an outbreak setting and assessment of patient viral load as a predictor of outcome. J Virol 78:4330–4341. doi: 10.1128/JVI.78.8.4330-4341.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Zhai J, Palacios G, Towner JS, Jabado O, Kapoor V, Venter M, Grolla A, Briese T, Paweska J, Swanepoel R, Feldmann H, Nichol ST, Lipkin WI. 2007. Rapid molecular strategy for filovirus detection and characterization. J Clin Microbiol 45:224–226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Broadhurst MJ, Kelly JD, Miller A, Semper A, Bailey D, Groppelli E, Simpson A, Brooks T, Hula S, Nyoni W, Sankoh AB, Kanu S, Jalloh A, Ton Q, Sarchet N, George P, Perkins MD, Wonderly B, Murray M, Pollock NR. 2015. ReEBOV antigen rapid test kit for point-of-care and laboratory-based testing for Ebola virus disease: a field validation study. Lancet 386:867–874. doi: 10.1016/S0140-6736(15)61042-X. [DOI] [PubMed] [Google Scholar]
- 21.Walker NF, Brown CS, Youkee D, Baker P, Williams N, Kalawa A, Russell K, Samba AF, Bentley N, Koroma F, King MB, Parker BE, Thompson M, Boyles T, Healey B, Kargbo B, Bash-Taqi D, Simpson AJ, Kamara A, Kamara TB, Lado M, Johnson O, Brooks T. 2015. Evaluation of a point-of-care blood test for identification of Ebola virus disease at Ebola holding units, Western Area, Sierra Leone, January to February 2015. Euro Surveill 20(12):pii=21073 http://dx.doi.org/10.2807/1560-7917.ES2015.20.12.21073. [DOI] [PubMed] [Google Scholar]
- 22.Paweska JT, Jansen van Vuren P, Masumu J, Leman PA, Grobbelaar AA, Birkhead M, Clift S, Swanepoel R, Kemp A. 2012. Virological and serological findings in Rousettus aegyptiacus experimentally inoculated with Vero cells-adapted Hogan strain of Marburg virus. PLoS One 7(9):e45479. doi: 10.1371/journal.pone.0045479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Le Roux CA, Kubo T, Grobbelaar AA, van Vuren PJ, Weyer J, Nel LH, Swanepoel R, Morita K, Paweska JT. 2009. Development and evaluation of a real-time reverse transcription-loop-mediated isothermal amplification assay for rapid detection of Rift Valley fever virus in clinical specimens. J Clin Microbiol 47:645–651. doi: 10.1128/JCM.01412-08. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Greiner M. 1995. Two-graph receiver operating characteristics (TG-ROC): a Microsoft-Excel template for the selection of cut-off values in diagnostic tests. J Immunol Methods 185:145–146. doi: 10.1016/0022-1759(95)00078-O. [DOI] [PubMed] [Google Scholar]
- 25.Greiner M. 1996. Two-graph receiver operating characteristics (TG-ROC): update version supports optimisation of cut-off values that minimize overall misclassification costs. J Immunol Methods 191:93–94. doi: 10.1016/0022-1759(96)00013-0. [DOI] [PubMed] [Google Scholar]
- 26.Greiner M, Sohr D, Göbel P. 1995. A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests. J Immunol Methods 185:123–132. doi: 10.1016/0022-1759(95)00121-P. [DOI] [PubMed] [Google Scholar]
- 27.Southern TR, Rasca LD, Albarino CG, Fey PD, Hinrichs SH, Murphy CN, Herrera VL, Sambol AR, Hill CE, Ryan EL, Kraft CS, Campbell S, Sealy TK, Schuh A, Ritchie JC, Lyon GM III, Mehta AK, Varkey JB, Ribner BS, Brantly KP, Ströher U, Iwen PC, Burd EM. 2015. Comparison of FilmArray and qRT-PCR for the detection of Zaire ebolavirus from contrived and clinical specimens. J Clin Microbiol 53:2956–2960. doi: 10.1128/JCM.01317-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Towner JS, Amman BR, Sealy TK, Carroll SAR, Comer JA, Kemp A, Swanepoel R, Paddock CD, Balinandi S, Khristova ML, Formenty PB, Albarino CG, Miller DM, Reed ZD, Kayiwa JT, Mills JN, Cannon DL, Greer PW, Byaruhanga E, Farnon EC, Atimnedi P, Okware S, Katongole-Mbidde E, Downing R, Tappero JW, Zaki SR, Ksiazek TG, Nichol ST, Rollin PE. 2009. Isolation of genetically diverse Marburg viruses from Egyptian fruit bats. PLoS Pathog 5:e1000536. doi: 10.1371/journal.ppat.1000536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Al-Soud WA, Rådström P. 2001. Purification and characterization of PCR-inhibitory components in blood cells. J Clin Microbiol 39:485–493. doi: 10.1128/JCM.39.2.485-493.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Drosten C, Panning M, Guenther S, Schmitz H. 2002. False-negative results of PCR assay with plasma of patients with severe viral hemorrhagic fever. J Clin Microbiol 40:4394–4395. doi: 10.1128/JCM.40.11.4394-4395.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]