Abstract
Background: West Nile virus (WNV) molecular detection is being conducted by a growing number of laboratories, but the degree of proficiency may vary between them. External quality control is needed.
Methods: We have conducted an international quality assurance study on WNV molecular detection. Participating laboratories tested noninfectious samples inactivated by heat and gamma irradiation. Participants received 7 coded lyophilized samples containing WNV of genetic lineages 1a, 1b, and 2 at 2600 to 18 000 000 RNA copies/mL, 3 samples containing heterologous flaviviruses, and 2 negative samples.
Results: Thirty laboratories participated. The average laboratory achieved 50% detection probability from 7762 copies/mL onward (probit analysis; 95% CI = 1174–24547 copies/mL). Lineages 1a and 1b were detected with equal efficiencies, but the lineage 2 strain (Ug37) was detected at significantly lower rates. Only 27% of participants were able to detect the 6 samples containing ≥1.8 × 104 copies/mL. Three laboratories generated false-positive results in negative samples. Six of 30 laboratories reported correct strain identification in 3 samples containing non-WNV flaviviruses. We observed a significant positive correlation between the capability of detecting non-WNV flaviviruses and detecting WNV lineage 2.
Conclusions: Most participants showed good performance in detecting lineage 1 WNV, the predominant virus in the Northern Hemisphere. The inability of some laboratories to detect even highly concentrated lineage 2 WNV downgraded the overall outcome. The lineage 2 material received through this study will provide laboratories with the necessary template for improving their assays. Such material is otherwise hard to obtain.
West Nile virus (WNV)1 is a member of the Japanese encephalitis virus group of flaviviruses, causing febrile illness and encephalitis in humans. Two genetic lineages exist (1)(2)(3). Lineage 1 has the largest area of distribution, and its recent introduction into North America has already caused more than 16 000 cases of human illness since 1999 (4)(5). WNV-Kunjin, which is enzootic in Australia, is an outlier cluster within lineage 1. Lineage 2 is restricted to Sub-Saharan Africa and Madagascar (2)(5). In Europe and the Middle East, WNV is being introduced continually by migrating birds, but it is probably also persisting in natural reservoirs (2)(6)(7).
Molecular detection of WNV is used for ecological investigation, case management, and prevention of transmission by transfusion and transplantation (3)(8)(9). Reverse transcription (RT)-PCR is the preferred tool. Reports of several molecular diagnostic assays have been published, and the first commercial test products have become available (8)(9). Performance of assays may vary considerably between laboratories, however. Much of the available evaluation data has been generated in pilot studies only. Little information is available about the relative and overall proficiency in different laboratories.
We report the results of the first international external quality assurance (EQA) study on WNV molecular detection.
Materials and Methods
participants
Thirty laboratories from 18 countries participated, including 11 European, 2 Middle Eastern, 4 North or South American, and 1 African; a complete list is given in the Acknowledgements section.
preparation of test samples
Representative WNV strains of genetic lineages 1a, 1b, and 2 and heterologous flaviviruses were obtained from Vero cell cultures 4 days after infection. Strains comprised WNV-NY99 (GenBank accession number DQ211652), WNV-Pan001 (GenBank accession number AY268132), WNV-Kunjin (M.N., personal observations), and WNV-Ug37 (10).
Supernatants were heated to 56 °C for 1 h and gamma-irradiated with 30 kGy. Infectivity was excluded by reinoculation in Vero cell cultures, with 3 subsequent passages. Test samples were generated by diluting inactivated virus stock solutions in human fresh-frozen plasma tested and confirmed to be negative for HIV-1, hepatitis B virus, hepatitis C virus, and WNV. After dilution, enriched plasma was divided into aliquots, frozen, and lyophilized. Samples from each lot were redissolved and quantified in triplicate by real-time RT-PCR (RealArt West-Nile Virus, Qiagen Diagnostics).
Three additional samples containing mixtures of heterologous flaviviruses were also included. Mixing was done to challenge the specificity of WNV assays with as many strains as possible. Sample 1 contained dengue viruses (DenV) types 1–4 (ATCC VR-344, VR-345, VR-1256, and VR-1257). Sample 2 contained yellow fever virus vaccine strain 17D and tick-borne encephalitis virus strain K23 (11). Sample 3 contained St. Louis encephalitis virus (ATCC VR-1265) and Japanese encephalitis virus (ATCC SA14-14-2). These samples were processed and inactivated with the method described for the WNV test samples. Quantification was done by published (12) and unpublished in-house real-time RT-PCR assays. The contents and concentrations of individual samples are listed in Table 1 .
Table 1.
Results by laboratory.
| Code | Strain | RNA, copies/mL1 | Laboratory | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 22 | 3 | 4 | 5 | 62 | 7 | 82 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | Total | |||||||||||||||||||||||||||||||||
| 2 | WNV-NY99 | 1.8E7 | +3 | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | – | 29 | ||||||||||||||||||||||||||||||
| 5 | WNV-PaAn001 | 5.0E6 | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | – | + | + | + | + | + | – | + | + | + | + | + | + | – | – | 26 | ||||||||||||||||||||||||||||||
| 9 | WNV-NY99 | 1.8E6 | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | – | – | + | – | 27 | ||||||||||||||||||||||||||||||
| 1 | WNV-Ug37 | 1.4E6 | + | + | + | + | + | + | + | + | + | – | – | – | – | – | – | + | – | + | – | – | – | – | – | – | – | – | – | + | + | – | 13 | ||||||||||||||||||||||||||||||
| 12 | WNV-NY99 | 1.8E5 | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | + | – | – | – | – | – | – | – | – | 22 | ||||||||||||||||||||||||||||||
| 4 | WNV-NY99 | 1.8E4 | + | + | + | + | + | + | + | + | – | + | + | + | + | + | + | + | + | – | + | + | – | + | – | – | – | – | – | – | – | – | 19 | ||||||||||||||||||||||||||||||
| 6 | WNV-Kunjin | 2.6E3 | + | – | + | + | + | – | + | + | + | + | – | – | – | – | – | – | – | + | – | + | – | + | – | – | – | – | – | – | – | – | 11 | ||||||||||||||||||||||||||||||
| 10 | SLEV/JEV4 | 2E8/2E7 | – | – | S | S | – | – | – | – | – | + | – | – | – | – | – | – | – | – | – | + | – | – | – | – | – | – | – | – | – | – | 2 | ||||||||||||||||||||||||||||||
| 11 | YFV/TBEV4 | 1E6/1E6 | – | – | – | Y | – | – | F | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | 1 | ||||||||||||||||||||||||||||||
| 3 | DenV 1–45 | 2E6–2E8 | – | – | D | D | – | – | D | D | – | – | – | – | – | – | – | – | – | D | – | – | – | – | D | – | – | – | – | + | + | – | 2 | ||||||||||||||||||||||||||||||
| 7 | Negative | 0 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | + | – | – | – | – | – | – | – | – | – | – | – | 1 | ||||||||||||||||||||||||||||||
| 8 | Negative | 0 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | 0 | ||||||||||||||||||||||||||||||
Confirmed by real-time PCR in triplicate. Samples 2, 9, 12, and 4 constitute a dilution series: their concentrations are provided as expected upon dilution factors, but they have also been confirmed by real-time RT-PCR.
These participants used commercial RT-PCR assays (Revertal, Qiagen).
+, West Nile virus detected. Viruses other than West Nile virus detected: S, St. Louis encephalitis virus; Y, yellow fever virus; D, dengue virus; F, unspecified flavivirus.
SLEV, St. Louis encephalitis virus; JEV, Japanese B encephalitis virus; TBEV, tick-borne encephalitis virus.
RNA copies in detail: DenV-1, 1.5E8/mL; DenV-2, 2.0E8/mL; DenV-3, 2.0E6/mL; DenV-4, 2.2E8/mL.
Results
Lyophilized test samples were shipped at ambient temperature to the participating laboratories. Each participant received a coded panel of 7 samples containing WNV at 2 600 to 18 000 000 RNA copies/mL, 3 samples containing heterologous flaviviruses, and 2 negative samples (Table 1 ). Participants were asked to use their routine setup of molecular assays for specific diagnosis of acute WNV infections to analyze the material but were not asked to detect or type other flaviviruses.
For each individual sample, we first determined what fraction of the 30 participating laboratories detected virus (Fig. 1 ). These fractions were plotted against the respective virus concentrations in each sample and analyzed by probit analysis (n = 30 replicate datum points per concentration). With the exception of 1 outlier sample, the positivity rate of each sample corresponded exactly with the concentration-dependent response rates calculated by a dose–response model (probit analysis, P ≤ 0.0001). When the outlier was eliminated, the hypothetical average laboratory achieved 50% detection probability from 7762 copies of RNA/mL of sample onward (95% CI, 1174–24547 copies/mL). The outlier sample was the only member of the panel belonging to WNV genetic lineage 2 (strain WNV-Ug37). For this sample, the hit rate was 43.3%, less than half the rate predicted by the regression model. In 10 laboratories, the lineage 2 sample could not be detected, although a lineage 1 sample of 10 times less virus (1.8 × 105 copies/mL) tested positive. For WNV-Kunjin, which is an outlier within lineage 1 (designated lineage 1b), a genotype bias was not observed. The WNV-Kunjin detection rate of 36.7% corresponded well with the dose–response model, because it had a low virus concentration of 2600 copies/mL (Fig. 1 ). It should be mentioned, however, that we did not make a direct comparison of detection rates for genotype 1a. In addition, no significant differences were observed between the detection rates of the North American NY99 WNV prototype strain and the WNV PaAn001 strain that was isolated from an animal in France in 2000 (2)(13). Both strains belong to lineage 1a.
Figure 1.
Probit analysis of the fractions of laboratories achieving a positive result (y axis) in relation to the virus RNA concentration in a given positive sample (x axis).
Data points represent individual samples in the proficiency test panel. Samples without legend, WNV-NY99 and WNV-PaAn001 (lineage 1a); 1b, WNV-Kunjin (lineage 1b); 2, WNV-Ug37 (lineage 2). The thick line is the regression line calculated on the basis of a Probit analysis (dose–response model); the thin lines are 95% confidence intervals. Data fit into the model with P ≤ 0.0001. Sample 2 was excluded from Probit analysis as an outlier.
To appraise the performance for each individual laboratory, we defined 2 proficiency criteria. First, the 6 samples containing 1.8 × 104 or more copies of viral RNA/mL had to be correctly detected as positive (Table 1 ). This concentration was chosen because it was well within the detection interval of published and commercial RT-PCR assays for WNV (e.g., (4)(14)(15). Second, no false-positive results were allowed for the 2 negative samples. When we applied these criteria, only 8 of 30 laboratories (27%) passed the minimum requirements for successful participation. All other laboratories failed because of lack of sensitivity, not because of false-positive results (Table 1 ). Six of the 8 successful laboratories also detected the low copy number sample, containing WNV-Kunjin at 2600 copies/mL. Two laboratories missed the proficiency criteria only because the 1.8 × 104 copies/mL lineage 1 sample was not detected. Nine laboratories failed to detect the lineage 2 sample but detected all other samples correctly. One laboratory missed only 1 of the high-titered samples, an omission that may have been caused by a handling error. The remaining laboratories missed more than 1 sample. One laboratory provided no correct result at all.
Three samples in the test panel contained heterologous flaviviruses and 2 contained no virus (Table 1 ). Five of 30 laboratories generated false-positive WNV results for 1 of these samples, with no more than 1 false-positive test result per laboratory. Four of 5 false-positive results occurred in samples containing heterologous flaviviruses. Six of 30 laboratories correctly identified heterologous flaviviruses in the 3 virus-containing samples. A significant positive correlation existed between the capability of laboratories to detect heterologous flaviviruses and their ability to detect the lineage 2 sample (t-test, P <0.022).
In one of our earlier studies, ANOVA analysis identified the use of commercial test products for severe acute respiratory syndrome coronavirus as the only technical factor that had a positive influence on laboratory performance (16). When we applied the same statistical test to the current dataset, we did not identify a significant advantage, possible because of the small number of participants using commercial assays in this study (n = 3). Nevertheless, the users of commercial products were among the best performers, and all of them detected lineage 2 (Table 1 ). No significant influence on laboratory performance was found to be associated with the use of real-time PCR, column-based RNA extraction methods (Roche, Qiagen), automated RNA preparation, or 1 popular RT-PCR protocol (15).
Discussion
Molecular detection of WNV is becoming an increasingly important task. EQA is therefore necessary. Compared with our earlier EQA studies on emerging agents such as Ebola, Lassa, Pox, and severe acute respiratory syndrome viruses (16)(17), in this study the overall diagnostic performance for WNV appears disappointing. From a technical point of view, much lower detection limits, in the range of 100-1000 copies/mL, can be achieved in RT-PCR, and some of our participants will indeed reach this concentration. The average laboratory, however, was not as efficient, and only a small number of participants passed a rather easy set of proficiency criteria, a critical problem because maximum sensitivity is required in clinical cases of encephalitis, in blood screening, and in testing reservoir components such as mosquitoes.
On the other hand, the low success rate in our study was mainly caused by limitations in detecting WNV lineage 2. Without the lineage 2 sample, 60% of participants would have passed the proficiency criteria. Because lineage 1 is far more prevalent in general and in the Northern Hemisphere in particular, it is conceivable that many participants may have designed their assays according to their own geographic location, a situation that would be acceptable in most settings. Several other laboratories may have detected lineage 2 only by broad-range flavivirus assays, as suggested by the observed correlation between non-WNV flavivirus detection and lineage 2 results. In view of recent findings on the presence of lineage 2 WNV and potentially new lineages 3 and 4 in Europe (1)(18)(19), the importance of using broad-range assays is obvious. At least on the reference laboratory level, broad-range flavivirus assays should routinely be applied in parallel with specific WNV detection.
The present study shows that EQA is adequate and necessary for identifying shortcomings in diagnostic proficiency. For rare pathogens such as WNV, EQA furthermore provides critical virus material that is required to improve and adjust diagnostic assays. The samples used in this study are available from the European Network for Diagnostics of “Imported” Viral Diseases for future reference.
Acknowledgments
The EQA was performed by the European Network for Diagnostics of “Imported” Viral Diseases, currently funded in part by the European Directorate General for Health and Consumer Affairs under the program AIDS and Other Communicable Diseases Grant No. SI2.299717(2000CVG4-26). Work of the Bernhard Nocht Institute was supported by Bundesamt für Bevölkerungsschutz und Katastrophenhilfe Grant No. BBK-F-440-00-1.
Laboratories within the following institutions took part in this study: Europe/Middle East: Robert Koch Institut, Berlin, Germany; Euroimmun AG, Lübeck, Germany; Artus GmbH, Hamburg; Germany; Bernhard Nocht Institut, Hamburg, Germany; Statens Serum Institut, Copenhagen, Denmark; Centre for Emergency Preparedness & Response, Porton Down, UK; Veterinary Laboratories Agency, Weybridge, New Haw, UK; Institut für klinische Mikrobiologie, St. Gallen, Switzerland; Spiez Laboratory, Spiez, Switzerland; Institut de Médecine Tropicale du Service de Santé des Armées, Marseille Armées, France; Unité de Biologie des Infections Virales Emergentes, Institut Pasteur, Lyon, France; Istituto Nazionale per le Malattie Infettive, Rome, Italy; Istituto Zooprolifattico Sperimentale dell’Abruzzo e Molise “G. Caporale,” Teramo, Italy; Army Medical and Veterinary Research Center, Rome, Italy; Baxter AG, Vienna, Austria; Veterinärmedizinische Universität Wien, Vienna, Austria; Medizinische Universität Wien, Vienna, Austria; National Centre for Microbiology, Majadahonda, Spain; Laboratorio Central de Veterinaria, Algente, Spain; Erasmus University Rotterdam, Rotterdam, The Netherlands; University of Ljubljana, Ljubljana, Slovenia; Central Institute of Epidemiology, Moscow, Russian Federation. Middle East: Kimron Veterinary Institute, Beit Dagan, Israel; Sheba Medical Center, Ramat-Gan, Israel; Pasteur Institute of Iran, Teheran, Iran. Americas: Focus Technologies, Inc., Cypress, CA; Instituto de Diagnostico y Referencia Epidemiologicos, Mexico City, Mexico; Caribbean Epidemiology Centre, Port of Spain, Trinidad and Tobago; Laboratorio Nacional de Salud Guatemala, Villa Nueva, Guatemala. Africa: Special Pathogens Unit National Institute for Communicable Diseases, Johannesburg, South Africa. We are grateful to G. Wengler for providing a WNV strain.
Footnotes
Nonstandard abbreviations: WNV, West Nile virus; RT-PCR, reverse transcription-PCR; EQA, external quality assurance.
References
- 1.Bakonyi T, Hubalek Z, Rudolf I, Nowotny N. Novel flavivirus or new lineage of West Nile virus, central Europe. Emerg Infect Dis 2005;11:225-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Zeller HG, Schuffenecker I. West Nile virus: an overview of its spread in Europe and the Mediterranean basin in contrast to its spread in the Americas. Eur J Clin Microbiol Infect Dis 2004;23:147-156. [DOI] [PubMed] [Google Scholar]
- 3.Hayes EB, Sejvar JJ, Zaki SR, Lanciotti RS, Bode AV, Campbell GL. Virology, pathology, and clinical manifestations of West Nile virus disease. Emerg Infect Dis 2005;11:1174-1179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Briese T, Jia XY, Huang C, Grady LJ, Lipkin WI. Identification of a Kunjin/West Nile-like flavivirus in brains of patients with New York encephalitis. Lancet 1999;354:1261-1262. [DOI] [PubMed] [Google Scholar]
- 5.Hayes EB, Komar N, Nasci RS, Montgomery SP, O’Leary DR, Campbell GL. Epidemiology and transmission dynamics of West Nile virus disease. Emerg Infect Dis 2005;11:1167-1173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Charrel RN, Brault AC, Gallian P, Lemasson JJ, Murgue B, Murri S, et al. Evolutionary relationship between Old World West Nile virus strains. Evidence for viral gene flow between Africa, the Middle East, and Europe. Virology 2003;315:381-388. [DOI] [PubMed] [Google Scholar]
- 7.Weissenbock H, Kolodziejek J, Fragner K, Kuhn R, Pfeffer M, Nowotny N. Usutu virus activity in Austria, 2001–2002. Microbes Infect 2003;5:1132-1136. [DOI] [PubMed] [Google Scholar]
- 8.Lanciotti RS. Molecular amplification assays for the detection of flaviviruses. Adv Virus Res 2003;61:67-99. [DOI] [PubMed] [Google Scholar]
- 9.Cockerill FR, III, Smith TF. Response of the clinical microbiology laboratory to emerging (new) and reemerging infectious diseases. J Clin Microbiol 2004;42:2359-2365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Castle E, Leidner U, Nowak T, Wengler G. Primary structure of the West Nile flavivirus genome region coding for all nonstructural proteins. Virology 1986;149:10-26. [DOI] [PubMed] [Google Scholar]
- 11.Niedrig M, Klockmann U, Lang W, Roeder J, Burk S, Modrow S, et al. Monoclonal antibodies directed against tick-borne encephalitis virus with neutralizing activity in vivo. Acta Virol 1994;38:141-149. [PubMed] [Google Scholar]
- 12.Drosten C, Gottig S, Schilling S, Asper M, Panning M, Schmitz H, et al. Rapid detection and quantification of RNA of Ebola and Marburg viruses, Lassa virus, Crimean-Congo hemorrhagic fever virus, Rift Valley fever virus, dengue virus, and yellow fever virus by real-time reverse transcription-PCR. J Clin Microbiol 2002;40:2323-2330. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Durand B, Chevalier V, Pouillot R, Labie J, Marendat I, Murgue B, et al. West Nile virus outbreak in horses, southern France, 2000: results of a serosurvey. Emerg Infect Dis 2002;8:777-782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Shi PY, Kauffman EB, Ren P, Felton A, Tai JH, Dupuis AP, II, et al. High-throughput detection of West Nile virus RNA. J Clin Microbiol 2001;39:1264-1271. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lanciotti RS, Kerst AJ, Nasci RS, Godsey MS, Mitchell CJ, Savage HM, et al. Rapid detection of West Nile virus from human clinical specimens, field-collected mosquitoes, and avian samples by a TaqMan reverse transcriptase-PCR assay. J Clin Microbiol 2000;38:4066-4071. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Drosten C, Doerr HW, Lim W, Stohr K, Niedrig M. First external quality assurance study on severe acute respiratory syndrome-associated coronavirus molecular detection. Emerg Infect Dis 2004;10:2200-2202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Niedrig M, Schmitz H, Becker S, Gunther S, ter Meulen J, Meyer H, et al. First international quality assurance study on the rapid detection of viral agents of bioterrorism. J Clin Microbiol 2004;42:1753-1755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Weissenbock H, Kolodziejek J, Url A, Lussy H, Rebel-Bauder B, Nowotny N. Emergence of Usutu virus, an African mosquito-borne flavivirus of the Japanese encephalitis virus group, central Europe. Emerg Infect Dis 2002;8:652-656. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Bakonyi T, Ivanics E, Erdelyi K, Ursu K, Ferenczi E, Weissenböck H, Nowotny N. Lineage 1 and 2 strains of encephalitic West Nile virus, Central Europe. Emerg Infect Dis 2006;12:618-623. [DOI] [PMC free article] [PubMed] [Google Scholar]