Skip to main content
PLOS Neglected Tropical Diseases logoLink to PLOS Neglected Tropical Diseases
. 2010 Oct 5;4(10):e833. doi: 10.1371/journal.pntd.0000833

2nd International External Quality Control Assessment for the Molecular Diagnosis of Dengue Infections

Cristina Domingo 1, Matthias Niedrig 1, Anette Teichmann 1, Marco Kaiser 1, Leonid Rumer 1, Richard G Jarman 2, Oliver Donoso-Mantke 1,*
Editor: Eva Harris3
PMCID: PMC2950135  PMID: 20957194

Abstract

Background

Currently dengue viruses (DENV) pose an increasing threat to over 2.5 billion people in over 100 tropical and sub-tropical countries worldwide. International air travel is facilitating rapid global movement of DENV, increasing the risk of severe dengue epidemics by introducing different serotypes. Accurate diagnosis is critical for early initiation of preventive measures. Different reverse transcriptase PCR (RT-PCR) methods are available, which should be evaluated and standardized. Epidemiological and laboratory-based surveillance is required to monitor and guide dengue prevention and control programmes, i.e., by mosquito control or possible vaccination (as soon as an effective and safe vaccine becomes available).

Objective

The purpose of the external quality assurance (EQA) study described is to assess the efficiency and accuracy of dengue molecular diagnosis methods applied by expert laboratories.

Study Design

A panel of 12 human plasma samples was distributed and tested for DENV-specific RNA. The panel comprised 9 samples spiked with different DENV serotypes (DENV-1 to DENV-4), including 10-fold dilution series of DENV-1 and DENV-3. Two specificity controls consisted of a sample with a pool of 4 other flaviviruses and a sample with chikungunya virus. A negative control sample was also included.

Results

Thirty-seven laboratories (from Europe, Middle East Asia, Asia, the Americas/Caribbean, and Africa) participated in this EQA study, and reports including 46 sets of results were returned. Performance among laboratories varied according to methodologies used. Only 5 (10.9%) data sets met all criteria with optimal performance, and 4 (8.7%) with acceptable performance, while 37 (80.4%) reported results showed the need for improvement regarding accomplishment of dengue molecular diagnosis. Failures were mainly due to lack of sensitivity and the presence of false positives.

Conclusions

The EQA provides information on each laboratory's efficacy of RT-PCR techniques for dengue diagnosis and indicates for most laboratories an urgent need to improve sensitivity and specificity.

Author Summary

Dengue viruses (DENV) are the most widespread arthropod-borne viruses which have shown an unexpected geographic expansion, as well as an increase in the number and severity of outbreaks in the last decades. In this context, the accurate diagnosis and reliable surveillance of dengue infections are essential. The laboratory diagnosis of dengue relies on the use of several methods detecting markers of DENV infection present in patient serum. Molecular diagnosis methods are usually rapid, sensitive, and simple when correctly standardized. Moreover, PCR-based diagnosis techniques are able to readily detect DENV during the acute phase of the disease and may assume an important role in dengue diagnosis and surveillance. Different reverse transcriptase PCR (RT-PCR) methods have been developed and are currently available and should be standardized in each laboratory to maintain high quality performance. In this work an External quality assessment (EQA) activity has been carried out to evaluate the accuracy and quality of laboratory data for the molecular diagnosis and surveillance of dengue, which involved worldwide dengue reference laboratories. In conclusion, RT-PCR techniques for dengue diagnosis applied by the participating laboratories demonstrated the need of further improvement in most laboratories.

Introduction

Dengue viruses (DENV) are transmitted by Aedes sp. mosquitoes and are members of the Flaviviridae family, genus Flavivirus. DENV comprise four antigenically distinct serotypes (DENV1-4), which are genetically quite distinct. Infection with one DENV serotype leads to lifelong protection against homologous challenge, but only brief cross-protection against heterologous infection with a different serotype[1].

The four serotypes of DENV are the causative agents of one of the most important arthropod-borne viral diseases in terms of human morbidity and mortality. Several virus and host-specific factors have been suggested to correlate with severe disease outcomes, which are mostly associated with secondary infections with a heterologous serotype[2] and/or infections with more intrinsically virulent strains of the virus[3], [4].

Epidemic dengue fever (DF) and dengue haemorrhagic fever (DHF) have emerged as a public health problem in recent decades, with the development of hyperendemicity (different DENV serotypes overlapping in an spatiotemporal manner), in urban and periurban centres of many tropical and subtropical countries. Air travel is also facilitating the rapid global movement of DENV and increasing the risk of DHF through the introduction of different serotypes into new areas. Two and a half billion people live in areas where the disease is endemic, and 50–100 million cases of DF are estimated to occur every year, including 500,000 cases of the more severe illnesses, DHF and dengue shock syndrome (DSS)[5].

The laboratory diagnosis of dengue relies on the use of several methods detecting markers of DENV infection present in patient serum. Definitive diagnosis of dengue rests on the detection of the infective virus, virus-encoded antigens, viral genomic RNA, or virus-induced antibodies. PCR-based diagnosis techniques can be more sensitive than viral culture, since they are able to readily detect the dengue viruses during the acute phase of the disease, and may assume an important role in dengue diagnosis, since detection of IgM is not possible practically until defervescence[6]. Molecular diagnosis methods are usually rapid, sensitive, and simple when correctly standardized and can be used for genome detection in human clinical samples, biopsies, autopsy tissues, or mosquitoes. However, specimens and RNA must be carefully handled to avoid RNA degradation. Further, RT-PCR is highly sensitive to amplicon contamination, and without proper controls false-positive results may occur.

Different RT-PCR methods have been developed for detecting and typing DENV, which vary somewhat in terms of the amplified gene regions of the genome, in the ways they detect RT-PCR products, and the virus-typing procedures. RT-PCR methods for detecting dengue virus-infected Aedes mosquitoes in the field may serve as an additional early warning monitoring system for predicting dengue outbreaks [7]. Moreover, since a change in serotype would be particularly important in bringing about a surge in DF/DHF cases, molecular methods are further advantageous in detecting the specific circulating serotypes both in dengue-infected patients and in natural populations of mosquitoes.

Aiming at the evaluation of the accuracy and estimation of the quality of laboratory data for the molecular diagnosis and surveillance of dengue, we have realised an external quality assurance (EQA) activity which involved worldwide dengue reference laboratories.

Materials and Methods

Participants

Laboratories in Europe (n = 22), Middle-East Asia (n = 1), Asia (n = 8), Africa (n = 1), and the Americas/Caribbean (n = 5) participated in the EQA activity: Institut für Virologie, Philips Universität Marburg, Germany; Institut für Virologie, Universitätsklinikum Bonn, Germany; Institut für Virologie, Medizinische Universität Wien, Austria; CNR Associé des Arbovirus IRBA-Antenne Marseille-IMTSSA, France; Institut Pasteur, Centre de Réference des Arbovirus, Marseille, France; Centro Nacional de Microbiología, Instituto de Salud Carlos III, Madrid, Spain; Instituto Tecnológico “La Marañosa”, Madrid, Spain; Aristotle University of Thessaloniki, School of Medicine Department of Microbiology, Greece; Erasmus Medical Centre, Rotterdam, The Netherlands; Spiez Biology Laboratory, Spiez, Switzerland; Institut für Klinische Mikrobiologie und Immunologie, St. Gallen, Switzerland; University of Geneva, Laboratory of Virology, Geneva, Switzerland; Army Medical Research Center, Rome, Italy; Department of Infectious, Parasitic and Immunomediated Diseases, Instituto Superiore di Sanità, Rome, Italy; Instituto Nazionale per le Malattie Infettive “L. Spallanzani”, Rome, Italy; Laboratory of Vector-borne Infections, Cantacuzzino Institut, Bucarest, Romania; Instituut voor Tropische Geneeskunde, Centraal Laboratorium Klinische Biologie, Antwerpen, Belgium; Haartman Institute, University of Helsinki, Finland; Statens Serum Institut, Department of Virology, Copenhagen, Denmark; “Béla Johan” National Center for Epidemiology, Budapest, Hungary; Norwegian Institute of Public Health, Oslo, Norway; National Reference Laboratory for Arboviruses, National Institute of Public Health, Ostrava, Czech Republic; National Centre for Zoonotic Viruses, Tel Hashomer; Israel; Department of Laboratory Medicine, Tan Tock Senk Hospital, Singapore; Department of Molecular Medicine, National University Hospital, Singapore; Department of Pathology, Singapore General Hospital, Singapore; Environmental Health Institute, Singapore; Hospital for Tropical Diseases, Oxford University Clinical Research Unit, Ho Chi Minh City, Vietnam; Tropical Infectious Diseases and Education Center, University Malay, Kuala-Lumpur, Malaysia; Institut Pasteur, Cambodia; Special Pathogens Unit, NICD-NHLS, Sandringham, South Africa; Caribbean Epidemiology Center (CAREC), Port of Spain, Trinidad; Laboratorio de Flavivirus, Instituto Oswaldo Cruz, FIOCRUZ, Rio de Janeiro, Brazil; Centro de Investigación en Enfermedades Infecciosas, INER, Tlalpan, Mexico; Instituto Nacional de Salud, Bogotá, Colombia; Instituto Nacional de Enfermedades Virales Humanas Dr. Julio Maiztegui, Pergamino, Argentina.

The EQA was established and coordinated by the European Network for the Diagnostics of ‘Imported’ Viral Diseases-Collaborative Laboratory Response Network (ENIVD-CLRN) which has previously coordinated EQA studies [8], [9], [10].

Specimen preparation and dispatch

Inactivated and stable DENV preparations were generated from cell culture (Vero B4) supernatants of the 4 dengue serotypes: DENV-1 VR344 (Thai 1958); DENV-2 VR345 (TH-36 strain); DENV-3 VR216 (H87 strain); and DENV-4 VR217 (H241 strain). Viral cell supernatants were inactivated by heating for 1 h at 56°C and gamma irradiation [25 kilogray (kGy)] to assure their non-infectivity.

Sample preparations were tested by an in-house real-time quantitative RT-PCR with a sensitivity threshold of 5 genome equivalents per reaction (GE)/rxn (100 GE/ml sample). Intra-assay and inter-assay variance of this method were determined as <3% and <4%, respectively. The inactivated supernatant viral load was estimated after heat inactivation and additionally after gamma irradiation. The inactivated supernatants were diluted in serum plasma to prepare a set of 9 positive samples which included 5 serial 10-fold dilution series of DENV-1 (7×105 GE/ml to 70 GE/ml), and two DENV-3 dilutions (3×104 GE/ml and 3×103 GE/ml). DENV-2 and DENV-4 inactivated supernatants were diluted with human plasma to 105 GE/ml. Two further plasma samples were prepared as specificity controls, one of them containing yellow fever virus [YF (YF-17D strain)], West Nile virus [WN (New York)], Japanese encephalitis virus [JE (strain SA-14-02)] and tick-borne encephalitis virus [TBE (strain Absettarov)], and a second one containing chikungunya virus [CHIK (strain Marseille LR 2006/684-1)], a common differential diagnosis of DF. A negative control plasma sample was also included (Table 1). Aliquots of 100 µl were number-coded, freeze dried for 24 h (Christ, AlphaI-5, Hanau, Germany) and stored at 4°C until dispatch.

Table 1. Results of the EQA for the molecular detection of DENV.

Samples no.
#2 #9 #12 #4 #14 #5 #13 #6 #10 #11 #3 #7
DENV-1 DENV-1 DENV-1 DENV-1 DENV-1 DENV-3 DENV-3 DENV-2 DENV-4 JE/YF WN/TBE CHIK Negative
Copy no. [GE/mL]
Lab. no. RT-PCR technique 7,0E+05 7,0E+04 7,5E+03 7,0E+02 7,0E+01 3,0E+04 3,0E+03 1,0E+05 1,0E+05 neg. neg. neg. Score* Classification #
8 Heminested[17] ++ ++ ++ + + ++ ++ ++ ++ ++ 22 OPTIMAL
7 TaqMan[12] ++ ++ ++ ++ (−) ++ ++ ++ ++ 22 OPTIMAL
13 SYBR-Green[18] ++ ++ ++ ++ (−) ++ ++ ++ ++ 22 OPTIMAL
17a TaqMan[14] ++ ++ ++ ++ (−) ++ ++ ++ ++ 22 OPTIMAL
12 TaqMan[19] ++ ++ ++ + ++ + (−) ++ ++ 20 IMPROVE
21 SYBR-Greena ++ ++ ++ ++ (−) ++ ++ ++ ++ (+) 20 IMPROVE
2a Nested[12] ++ ++ ++ (−) (−) ++ ++ ++ ++ 20 OPTIMAL
2b TaqMana ++ ++ ++ (−) (−) ++ (−) ++ ++ 18 IMPROVE
4b Nested[28] ++ ++ ++ (−) (−) ++ (−) ++ ++ 18 IMPROVE
28a Nestedb ++ ++ ++ (−) (−) ++ (−) ++ ++ 18 IMPROVE
15 TaqMan[20] ++ ++ (−) (−) ++ ++ (−) ++ ++ 18 IMPROVE
5 TaqMan[21] ++ ++ ++ ++ () () () ++ ++ 18 IMPROVE
20 TaqMan[21] ++ ++ ++ ++ () () () ++ ++ 18 IMPROVE
14 Nested[12] + + + + () ++ ++ ++ + 17 ACCEPTABLE
27 Nestedb + ++ ++ ++ () ++ () ++ ++ (+) 17 IMPROVE
28b TaqMana ++ ++ () () () ++ () ++ ++ 16 IMPROVE
29 TaqMan[18] ++ ++ ++ () () ++ () ++ () 16 IMPROVE
31 TaqMana + + + + + + + + + 15 ACCEPTABLE
23b TaqMana + + () () () ++ () ++ ++ 14 IMPROVE
19a Nested[14] ++ ++ () () () ++ () () ++ 14 IMPROVE
1 TaqManc ++ ++ () () () + + ++ () 14 IMPROVE
36 Nested[28] + + + () () + + + + 13 ACCEPTABLE
10 TaqMan[17] + + + () () + + + + 13 ACCEPTABLE
19b TaqMan[19] + + + () + + + + () 13 IMPROVE
25b Nested[15] ++ + () () () ++ () ++ () 12 IMPROVE
9a Nestedb ++ ++ ++ () () ++ ++ ++ () (+) 12 IMPROVE
9b TaqMana + + + + () + + + + (+) 12 IMPROVE
22 TaqMan[13] ++ ++ () () () () () () ++ 12 IMPROVE
4a TaqMana + + + () () + () + + 12 IMPROVE
30 Nested[28] () ++ () ++ ++ () () + () (+) 11 IMPROVE
17b TaqMan[19] + + () () + () + () () 11 IMPROVE
37 TaqMan[22] + + () () () + + + () 11 IMPROVE
3 SYBR-Greena + () () () () + () + + 10 IMPROVE
16 TaqManc ++ + () ++ + + + + + (+) (+) (+) 10 IMPROVE
18 TaqMan[19] + + () () () () () + + 10 IMPROVE
24 RT-PCRb + + () () () () () + + 10 IMPROVE
6 TaqManc + + () () () + () + () 10 IMPROVE
25a TaqMan[19] + + () () () + () + + 10 IMPROVE
11 Nested[15] ++ ++ () () () + () + () (+) 10 IMPROVE
35a Nested[12] ++ ++ () () () () () () () 10 IMPROVE
34 TaqMan[23] + + () () () () () () + 9 IMPROVE
23a SYBR-Green[29] + () () () () () () + () 8 IMPROVE
32 Heminested[17] ++ () () () () () () () () 8 IMPROVE
33 TaqMan[19] + + + () () () () + + 8 IMPROVE
26 Nestedb () () () () () () () () () 6 IMPROVE
35b Nested[16] () () () () () () () () + (+) 5 IMPROVE
Correct positive/total results (%) 43/46 (93.5) 41/46 (89) 23/46 (50) 14/46 (30.4) 8/46 (17.4) 32/46 (69.5) 17/46 (37) 38/46 (82.6) 32/46 (69.5) 40/46 (87) 44/46 (95.6) 44/46 (95.6)

*The score reflects the quality of the diagnostic results: ++, positive result and correctly typed (score 2 points); +, positive but no information on serotype or incorrectly typed (score 1 point); (+), false positive (score 0 points); −, correct negative result (score 2 points); (), false negative (score 0 points). Sample #14 was not evaluated for scoring.

# Classification reflects the capability to perform optimal surveillance and diagnosis activities (sensitivity up to 103 GE/ml and serotyping); acceptability for diagnosis purposes (sensitivity up to 103 GE/ml); or the need of improvement (failure to detect one or more serotypes or sensitivity below 103 GE/ml). Samples #4 and #14 were not evaluated for classification.

a In-house qRT-PCR system(s) for quantification and/or serotyping;

b In-house RT-PCR system(s);

c Commercial Artus Light Cycler test (Qiagen).

Four different sets of test aliquots were tested before distribution by the Robert Koch Institute (RKI), Berlin, Germany. After reconstitution with 100 µl of water, samples were extracted using the QIAamp viral RNA minikit (Qiagen, Hilden, Germany) according to the manufacturer's instructions. DENV genome copies present in the samples were estimated by an in-house real time PCR and serotypes were confirmed by nested RT-PCR and sequencing [11]. Additionally, the sample panel was analysed independently by one expert laboratory (USAMC-AFRIMS, Bangkok, Thailand) using an RT-PCR assay[12] and an in-house quantitative real time RT-PCR. This laboratory was blinded to sample contents before analysis.

A temperature/time-stability control of the EQA samples was additionally performed at the RKI. A set of samples were placed at room temperature for 1 month, analysed for maintenance of their properties, and compared to a set of samples stored at 4°C.

The EQA panel was distributed to all participants with full instructions. Samples were shipped at ambient temperature by post to participating laboratories. Participants were requested to resuspend the samples in 100 µl of water and to analyse the material as serum samples for DENV molecular diagnosis. They were requested to report their results, and any problems encountered, as well as to provide information on the assay details (RT-PCR method and extraction procedure) using a common form included in the documentation.

Evaluation of the results

To assure anonymous participation, an individual numerical identification code was assigned to the results sent by each laboratory. This number was followed by a letter (a, b) when different sets of results (more than one method) were sent.

The results were scored in reflection of sensitivity, specificity, and correct serotyping. Two points were given for correct dengue-positive results and correct serotyping. One point was given for correct dengue-positive results without or incorrect type determination. False-negative or false-positive results were scored with no points. Sample #14 (DENV-1; 70 GE/ml) was not considered for scoring. Quantitative data were considered only informative about the availability of this data.

Additionally, results were classified as optimal for diagnosis and surveillance (full detection up to 103 GE/ml and serotyping results), acceptable for diagnosis (full detection up to 103 GE/ml), or need of improvement (failure to detect one or more DENV serotypes or sensitivity below 103 GE/ml). Samples #4 (DENV1; 7E+02 GE/ml) and #14 (DENV1; 70 GE/ml) were not considered for classification. No false-positive results were allowed in virus-free samples.

Test results were sent out in an anonymous manner to all participants.

Statistical analysis

Data collected were entered into Microsoft Excel (Microsoft Corp., Bellingham, WA, USA) and analysed using SPSS 14.0 for Windows. To project performance of a hypothetical average laboratory, cumulative fractions of positive results reported for each test sample of the 10-fold dilution series with DENV-1 were correlated against RNA concentrations in samples and subjected to probit analysis. Results with respect to categorised variables were analysed by the Chi-square test. Parametric (independent sample t-test and ANOVA) and non-parametric (Mann-Whitney and Kruskall-Wallis) tests were used to estimate the influence of the PCR format on the performance.

Results

Sixty-seven institutions involved in laboratory diagnostics of DENV infection were invited to participate in this study. Invitees were members of the International WHO DengueNet, the ENIVD-CLRN, or national/regional dengue reference laboratories. The study was announced as an EQA study on DENV molecular diagnostic proficiency, which included certifying and publishing the results in a comparative and anonymous manner.

Thirty-seven laboratories (response rate 55.2%) from 27 countries participated in the study. A total of 46 sets of results were received (Table 1), including 9 double sets from laboratories that used 2 methods each (sets 2ab, 4ab, 9ab, 17ab, 19ab, 23ab, 25ab, 28ab, 35ab).

A variety of tests were used for screening and identification of DENV by participating laboratories; these included nested (n = 15, 32.6%)[11], [12], [13], [14], [15], [16] and hemi-nested (n = 2, 4.3%) RT-PCR [17], TaqMan (n = 25, 54.3%) [13], [14], [18], [19], [20], [21], [22], [23] and SYBR Green (n = 4, 8.7%)-based real time PCR (Table 2).

Table 2. Probit analysis of the sensitivity profile of the different RT-PCR methods reported.

Method used for RT-PCR Protocols Percentage Low sensitivity# Positive performance*50% [CI 95%] Positive performance*95% [CI 95%]
RT nested PCR [12], [14], [11], l [13], [15], [16],a 32.6% (15/46) 80% (12/15) 104 GE/ml[3×103−4.2×104 GE/ml] 4×106 GE/ml[5.7×105−3.6×108 GE/ml]
RT heminested PCR [17] 4.3% (2/46) 50% (1/2) NP NP
TaqMan real time PCR [12],[[14], [18], [14], [19][20], [21], [13],[22],[23],a; b 54.3% (25/46) 76% (19/25) 104GE/ml[2.8×103−4×104 GE/ml] 4.2×106GE/ml[5.2×105−4.4×108GE/ml]
SYBR Green real time PCR [18], [29],a 8.6% (4/46) 50% (2/4) 6.3×104GE/ml.[NP] 3×105 GE/ml[NP]

NP: Not enough data for statistical analysis; a: in-house protocol; b: Commercial kit;

*Probit analysis.

# Sensitivity below 103 GE/ml.

Data fit into the model with p<0.05.

Different RNA extraction methods were reported by the participating laboratories; QIAmp Viral RNA Minikit (QIAGEN) [n = 19, 51.4%], EZ1 Biorobot (QIAGEN) [n = 2, 5.4%], QIAmp UltraSens kit (QIAGEN) [n = 1, 2.7%], RNeasy kit (QIAGEN) [n = 1, 2.7%], QIASymphony Viral RNA extraction kit (QIAGEN) [n = 1, 2.7%], Macherey Nagel (Düren, Germany) RNA virus kit [n = 1, 2.7%], NucliSens Easy Mag (BIOMERIEUX, Nürtingen, Germany) [n = 1, 2.7%]. Unfortunately, eleven laboratories (29.7%) did not report data on the extraction procedure.

Performance varied among the laboratories according to whether the methodologies used achieved a full or acceptable result (Table 1). Only 5 out of 46 (10.9%) analyses reported met all criteria with optimal performance; 4 out of 46 (8.7%) data sets achieved acceptable results; and 37 out of 46 (80.4%) of the results reported showed that there is room for improvement regarding DENV diagnosis. In 22 cases non-optimal results were due to the failure to identify one or more DENV serotypes: DENV-2 (lab n°19a); DENV-3 (n° 5, 20, 18, 24, 33); DENV-4 (n° 29, 1, 19b, 25b, 9a, 37, 6, 11); DENV2 and DENV3 (n° 22, 34); DENV2 and DENV4 (n° 17b); DENV-3 and DENV-4 (n° 30, 23a); DENV-2, DENV-3 and DENV-4 (n° 35a, 32); DENV-1, DENV-3 and DENV-2 (n° 35b); one laboratory completely failed in the detection of positive DENV samples (n° 26). Failures due to the presence of false positives were present in 8 data sets (n° 21, 9a, 9b 27, 30, 16, 11, 35b).

The serial dilutions of DENV-1 (samples #2, #9, #12, #4, and #14) and DENV-3 (samples #5 and #13) showed a clear tendency of low sensitivity with increased dilution, this being the major reason for non-acceptable results (Table 1). Samples #4 and #14 (700 and 70 GE/ml, respectively) were not considered in the classification, assuming that a minimal detection of 7,000 GE/ml should be achieved to perform a reliable DENV molecular diagnosis. Only 47.7% (n = 22) of the entries reported positive results in detecting 7,000 GE/ml of DENV-1, and 36.3% (n = 17) of the data sets achieved a positive result in the 3,000 GE/ml DENV-3 samples.

A probit analysis was performed using positive results reported for each sample of the 10-fold DENV-1 dilution series correlating with RNA concentrations in samples (Figure 1). The data demonstrated that globally 50% of positive performance could be expected in concentrations higher than 104 GE/ml (95% confidence interval [CI] 23,110–32,000 GE/ml). A certainty of 95% was achieved for 5×104 GE/ml (95% CI 41,760–57,500 GE/ml). This pointed to the need of improvement of the laboratories' capacity for achieving positive results in lower viral loads, so the existence of false negatives in routine practice of DENV diagnosis could be minimized.

Figure 1. Probit analysis of data sets with a correct result (y axis) for DENV-1 related to viral RNA concentration in positive samples (x axis).

Figure 1

Data points represent individual samples in the panel. The thick line is the regression line calculated on the basis of a probit model (dose-response curve), and thin lines are 95% confidence intervals. Data fit into the model with p<0.003.

A probit analysis of the correct results obtained, using TaqMan-based real time PCR and RT nested PCR protocols, showed that 50% of positive performance could be expected when viral genome concentration in the sample is higher than 104 GE/ml (Table 2). 95% of certainty was achieved with TaqMan real time PCR and RT nested PCR when more than 4.2×106 GE/ml (95% CI [5.2×105–4×108]) or 4×106 GE/ml (95% CI [5.7×105–3.6×108]), respectively were present in the sample (Table 2). T-test comparing the TaqMan real time performance versus RT nested PCR and SYBR Green real time RT-PCR did not reveal any significant difference in performance for the applied methods. In general, the choice of the real-time PCR format seemed to have no effect on the success of the analysis which is mostly affected by the performance of the individual laboratories as demonstrated by the wide range of the confidence intervals.

As many as 9 laboratories (n° 2, 17, 28, 19, 9, 4, 25, 35, and 23) used two different RT-PCR techniques for the evaluation. This approach is advantageous to exclude false positives/negatives in routine diagnosis. One laboratory used two different TaqMan PCRs (17ab), one laboratory used a combination of TaqMan- and SYBR Green-based PCRs (23a,b), one laboratory used two different RT nested PCR techniques (35a,b); and six laboratories used a combined approach of TaqMan real time PCR and RT nested PCR (2a,b; 19a,b; 25a,b; 9a,b; 28a,b; 4a,b).

Only 26 data sets (56.5%) included serotyping results that are necessary for achieving optimal results and support the reliability of the techniques and the laboratories to perform both diagnosis and surveillance activities.

Further information on the number of copies of DENV strains referred to the participants was requested to estimate the experience in viral load determination in DENV diagnosis. Only 8 laboratories out of 37 (21.6%) reported quantitative results (data not shown), even though as many as 29 laboratories (78.4%) were using real time-based procedures which would have allowed obtaining this information.

Results reported from different laboratories using the same techniques differed in their final performance, probably due to differences in the individual operational procedures and not to limitations of the methods themselves (Table 1). This is the case of 6 laboratories which reported data obtained using the TaqMan PCR developed by Drosten et al[19], where a different competence was observed depending on the reporting laboratory. On the other hand, in some cases, the same limitations were observed in laboratories using a common technique, thus indicating that an improvement or up-dating of the technique might be needed. This is the case of those laboratories using a TaqMan PCR from Leparc-Gofart et al[21] which failed to detect DENV-3 (Table 1).

A stability assessment of the samples when stored at room temperature (23–25°C) for 1 month (Table 3) demonstrated that the failure in detecting the DENV RNA experienced by some laboratories was probably not due to genome degradation caused by shipment conditions.

Table 3. Stability of the lyophilized panel of samples.

#2 #9 #12 #4 #14 #5 #13 #6 #10
DEN-1 DEN-1 DEN-1 DEN-1 DEN-1 DEN-3 DEN-3 DEN-2 DEN-4
7×105 7×104 7×104 NEG NEG 2×104 1×103 6×104 1×105 GE/ml 4°C
4×104 1.4×104 1×103 90 NEG 1.5×104 2×103 7.3×104 1×105 GE/ml RT

A set of YF EQA samples were kept at room temperature (23–25°C) for 1 month. These samples were analysed in parallel with a control set of samples kept at 4°C using an in-house real-time RT-PCR to control the level of stability/degradation of the samples when sent by mail at room temperature.

Some non-optimal results were due to the presence of false positives even when a good profile of sensitivity and thus a high score was reached (n° 21, 9b). False positives were mostly found on sample #11, containing other flaviviruses like Japanese encephalitis, yellow fever, West Nile and tick-borne encephalitis virus.

Of 13 laboratories taking part in the first DENV molecular diagnosis EQA realised in 2003[9], eight laboratories participated in the present EQA. Four laboratories improved their percentage of correct results, three experienced a decrease, and 1 laboratory reached 100% of correct results in both EQAs (data not shown).

Discussion

EQA studies for laboratory molecular diagnostics of DENV help to monitor the quality and accuracy of current diagnosis and the capacity for surveillance of the participating centres, to highlight problems in particular techniques or specific laboratories, and to give advice and assurance to all the centres. This is the second DENV molecular diagnosis EQA under the auspices of the ENIVD-CLRN project[9].

Twenty-two laboratories out of 37 (59.4%) participants were located in Europe, a non-endemic area. Even though the EQA was announced widely to national and international public health institutions involved in dengue control and surveillance in endemic areas, the number of responses was not as abundant as expected. Among the reasons for that, we should consider the cost of the reagents to be consumed for the exercise and to be covered by the individual laboratories, or the unwillingness to send incorrect results. Even though, it has to be considered that laboratories in endemic countries, that may not have participated in this EQA but perform dengue molecular diagnosis routinely, usually have strong internal quality control programs, and there are numerous laboratories that do perform internal QC procedures. Participation in EQA diagnosis activities must be encouraged and supported by health institutions as they provide helpful information on the capabilities of the health system to detect and control the disease.

In the present study the performance of molecular methods for DENV diagnosis with different designs varies depending on the method and the reporting laboratory. It is difficult to assess which methods are better, since primers, enzymes, buffers, reaction conditions, genome target, and thermocyclers can all influence the reliability of a PCR[24]. However, some conclusions could be assumed from the results obtained in this EQA.

Surprisingly, a low sensitivity was appreciated in the general results of this EQA, even when reliable real time procedures were used. We selected a cut-off of ≥103 GE/ml to achieve an optimal or acceptable classification because it is considered over the above limit of detection of current RT-PCR protocols [6]. Secondly, we have also assumed this to be a suitable detection threshold for diagnosis of acute dengue according to published results on viral load titers in DF and DHF patients [25], [26], [27]. Extreme precaution should be taken when performing molecular diagnosis using methods with a limited sensitivity, so it could suppose the presence of false negatives in the diagnostic results. Likewise, in the final classification, no false positives results were allowed in virus-free samples, and the inability to detect one or more DENV serotypes was also considered non-acceptable, as this could lead to an incorrect DENV diagnosis with high impact on the clinical and public health management of DENV cases. In our study, false positives were found mainly in sample #11, which contained viral genome of different flaviviruses (Japanese encephalitis virus, tick-borne encephalitis virus, West Nile virus, and yellow fever virus). This reflects the need for further improvement on the specificity of the methodologies and/or the procedures carried out in the laboratories to avoid cross-contamination.

The sets of samples were sent by post to the participating laboratories, and it was of concern in some cases that it took long for them to reach their destination while maintained at ambient temperature. We tested the stability of the samples when stored at room temperature and demonstrated that genome degradation caused by shipment conditions was minimal.

In this study, a clear significant difference in sensitivity could not be noted when real time RT-PCR procedures were compared to RT-nested or heminested-PCR protocols, so no conclusions can be ventured in this regard. It should be said, however, that the most important factor in the reliability of the results for dengue diagnosis is the individual performance of each laboratory, as demonstrated by the different score values reached by different laboratories that used the same published protocols. This should alert to the need to revise the running protocols and processes in those laboratories without good results.

Different extraction methods have been reported, with most of the laboratories using QIAGEN kits for viral RNA extraction. Unfortunately, we did not receive this information from all laboratories, so we could not perform an accurate statistical analysis. However, in previous EQAs, the extraction procedures showed no great relevancy regarding final performance[9].

Discrepancies due to the reporting of an incorrect serotype were considered acceptable for diagnosis, since the case management would not have been affected. However, the absence of serotyping procedures or non-reliable ones diminished the possibility of performing surveillance studies. In those laboratories in non-endemic areas where dengue is diagnosed in the context of an “imported” disease (i.e. European laboratories), serotype classification of the cases could be considered unnecessary for clinical management of the patients, whereas in DENV-endemic countries it is highly desirable for reference laboratories to contribute serotype information, since established surveillance systems should monitor where DENV is transmitted, which serotypes are involved, and which type of illness is associated with these serotypes.

Quantification of DENV RNA in human plasma samples can provide more clues for performing pathogenesis studies and monitoring the progress of clinical manifestations [25], [26], [27]. The use of different quantitative real time PCRs has been reported on this EQA, and 29 different data sets were obtained using these techniques. However, the absence of quantification data in most of participating laboratories is remarkable.

In order to improve awareness and technical performance of DENV reference laboratories, we recommend the continuation of these EQA exercises.

Supporting Information

Checklist S1

STARD Checklist

(0.98 MB PDF)

Acknowledgments

The authors are grateful to J. Kraft and R. Schädler for excellent assistance during the EQA activity and U. Erikli for assistance in the preparation of the manuscript.

Footnotes

The authors have declared that no competing interests exist.

This new EQA study is part of the ENIVD-CLRN project funded by the European Centre for Disease Prevention and Control (ECDC) under the Framework Service Contract Ref. No. ECDC/2008/011. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Kurane I, Ennis FE. Immunity and immunopathology in dengue virus infections. Semin Immunol. 1992;4:121–127. [PubMed] [Google Scholar]
  • 2.Halstead SB. The Alexander D. Langmuir Lecture. The pathogenesis of dengue. Molecular epidemiology in infectious disease. Am J Epidemiol. 1981;114:632–648. doi: 10.1093/oxfordjournals.aje.a113235. [DOI] [PubMed] [Google Scholar]
  • 3.Rico-Hesse R, Harrison LM, Salas RA, Tovar D, Nisalak A, et al. Origins of dengue type 2 viruses associated with increased pathogenicity in the Americas. Virology. 1997;230:244–251. doi: 10.1006/viro.1997.8504. [DOI] [PubMed] [Google Scholar]
  • 4.McBride WJ, Bielefeldt-Ohmann H. Dengue viral infections; pathogenesis and epidemiology. Microbes Infect. 2000;2:1041–1050. doi: 10.1016/s1286-4579(00)01258-2. [DOI] [PubMed] [Google Scholar]
  • 5.W.H.O. Dengue and dengue haemorragic fever. Fact Sheet. 2002;117 [Google Scholar]
  • 6.Kao CL, King CC, Chao DY, Wu HL, Chang GJ. Laboratory diagnosis of dengue virus infection: current and future perspectives in clinical diagnosis and public health. J Microbiol Immunol Infect. 2005;38:5–16. [PubMed] [Google Scholar]
  • 7.Chow VT, Chan YC, Yong R, Lee KM, Lim LK, et al. Monitoring of dengue viruses in field-caught Aedes aegypti and Aedes albopictus mosquitoes by a type-specific polymerase chain reaction and cycle sequencing. Am J Trop Med Hyg. 1998;58:578–586. doi: 10.4269/ajtmh.1998.58.578. [DOI] [PubMed] [Google Scholar]
  • 8.Donoso Mantke O, Lemmer K, Biel SS, Groen J, Schmitz H, et al. Quality control assessment for the serological diagnosis of dengue virus infections. J Clin Virol. 2004;29:105–112. doi: 10.1016/s1386-6532(03)00110-0. [DOI] [PubMed] [Google Scholar]
  • 9.Lemmer K, Donoso Mantke O, Bae HG, Groen J, Drosten C, et al. External quality control assessment in PCR diagnostics of dengue virus infections. J Clin Virol. 2004;30:291–296. doi: 10.1016/j.jcv.2003.11.002. [DOI] [PubMed] [Google Scholar]
  • 10.Donoso Mantke O, Schmitz H, Zeller H, Heyman P, Papa A, et al. Quality assurance for the diagnostics of viral diseases to enhance the emergency preparedness in Europe. Euro Surveill. 2005;10:102–106. [PubMed] [Google Scholar]
  • 11.Domingo C, Palacios G, Jabado O, Reyes N, Niedrig M, et al. Use of a short fragment of the C-terminal E gene for detection and characterization of two new lineages of dengue virus 1 in India. J Clin Microbiol. 2006;44:1519–1529. doi: 10.1128/JCM.44.4.1519-1529.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Lanciotti RS, Calisher CH, Gubler DJ, Chang GJ, Vorndam AV. Rapid detection and typing of dengue viruses from clinical samples by using reverse transcriptase-polymerase chain reaction. J Clin Microbiol. 1992;30:545–551. doi: 10.1128/jcm.30.3.545-551.1992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Chinnawirotpisan P, Mammen MP, Jr, Nisalak A, Thaisomboonsuk B, Narupiti S, et al. Detection of concurrent infection with multiple dengue virus serotypes in Thai children by ELISA and nested RT-PCR assay. Arch Virol. 2008;153:2225–2232. doi: 10.1007/s00705-008-0249-9. [DOI] [PubMed] [Google Scholar]
  • 14.Laue T, Emmerich P, Schmitz H. Detection of dengue virus RNA in patients after primary or secondary dengue infection by using the TaqMan automated amplification system. J Clin Microbiol. 1999;37:2543–2547. doi: 10.1128/jcm.37.8.2543-2547.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.ter Meulen J, Grau M, Lenz O, Emmerich P, Schmitz H, et al. Isolation and partial characterization of dengue virus type 2 and 4 strains from dengue fever and dengue haemorrhagic fever patients from Mindanao, Republic of the Philippines. Trop Med Int Health. 2000;5:325–329. doi: 10.1046/j.1365-3156.2000.00562.x. [DOI] [PubMed] [Google Scholar]
  • 16.Sanchez-Seco MP, Rosario D, Hernandez L, Domingo C, Valdes K, et al. Detection and subtyping of dengue 1-4 and yellow fever viruses by means of a multiplex RT-nested-PCR using degenerated primers. Trop Med Int Health. 2006;11:1432–1441. doi: 10.1111/j.1365-3156.2006.01696.x. [DOI] [PubMed] [Google Scholar]
  • 17.Scaramozzino N, Crance JM, Jouan A, DeBriel DA, Stoll F, et al. Comparison of flavivirus universal primer pairs and development of a rapid, highly sensitive heminested reverse transcription-PCR assay for detection of flaviviruses targeted to a conserved region of the NS5 gene sequences. J Clin Microbiol. 2001;39:1922–1927. doi: 10.1128/JCM.39.5.1922-1927.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Johnson BW, Russell BJ, Lanciotti RS. Serotype-specific detection of dengue viruses in a fourplex real-time reverse transcriptase PCR assay. J Clin Microbiol. 2005;43:4977–4983. doi: 10.1128/JCM.43.10.4977-4983.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Drosten C, Gottig S, Schilling S, Asper M, Panning M, et al. Rapid detection and quantification of RNA of Ebola and Marburg viruses, Lassa virus, Crimean-Congo hemorrhagic fever virus, Rift Valley fever virus, dengue virus, and yellow fever virus by real-time reverse transcription-PCR. J Clin Microbiol. 2002;40:2323–2330. doi: 10.1128/JCM.40.7.2323-2330.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Ito M, Takasaki T, Yamada K, Nerome R, Tajima S, et al. Development and evaluation of fluorogenic TaqMan reverse transcriptase PCR assays for detection of dengue virus types 1 to 4. J Clin Microbiol. 2004;42:5935–5937. doi: 10.1128/JCM.42.12.5935-5937.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Leparc-Goffart I, Baragatti M, Temmam S, Tuiskunen A, Moureau G, et al. Development and validation of real-time one-step reverse transcription-PCR for the detection and typing of dengue viruses. J Clin Virol. 2009;45:61–66. doi: 10.1016/j.jcv.2009.02.010. [DOI] [PubMed] [Google Scholar]
  • 22.Gurukumar KR, Priyadarshini D, Patil JA, Bhagat A, Singh A, et al. Development of real time PCR for detection and quantitation of Dengue Viruses. Virol J. 2009;6:10. doi: 10.1186/1743-422X-6-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Dumoulin A, Marti H, Panning M, Hatz C, Hirsch HH. Pan-dengue virus detection by PCR for travelers returning from the tropics. J Clin Microbiol. 2008;46:3104–3106. doi: 10.1128/JCM.01294-08. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Raengsakulrach B, Nisalak A, Maneekarn N, Yenchitsomanus PT, Limsomwong C, et al. Comparison of four reverse transcription-polymerase chain reaction procedures for the detection of dengue virus in clinical specimens. J Virol Methods. 2002;105:219–232. doi: 10.1016/s0166-0934(02)00104-0. [DOI] [PubMed] [Google Scholar]
  • 25.Thomas L, Verlaeten O, Cabie A, Kaidomar S, Moravie V, et al. Influence of the dengue serotype, previous dengue infection, and plasma viral load on clinical presentation and outcome during a dengue-2 and dengue-4 co-epidemic. Am J Trop Med Hyg. 2008;78:990–998. [PubMed] [Google Scholar]
  • 26.Vaughn DW, Green S, Kalayanarooj S, Innis BL, Nimmannitya S, et al. Dengue viremia titer, antibody response pattern, and virus serotype correlate with disease severity. J Infect Dis. 2000;181:2–9. doi: 10.1086/315215. [DOI] [PubMed] [Google Scholar]
  • 27.Wang WK, Chao DY, Kao CL, Wu HC, Liu YC, et al. High levels of plasma dengue viral load during defervescence in patients with dengue hemorrhagic fever: implications for pathogenesis. Virology. 2003;305:330–338. doi: 10.1006/viro.2002.1704. [DOI] [PubMed] [Google Scholar]
  • 28.Domingo C, Palacios G, Niedrig M, Cabrerizo M, Jabado O, et al. A New Tool for the Diagnosis and Molecular. Surveillance of Dengue Infections in Clinical Samples. Dengue Bulletin. 2004;28:87–95. [Google Scholar]
  • 29.Lai YL, Chung YK, Tan HC, Yap HF, Yap G, et al. Cost-effective real-time reverse transcriptase PCR (RT-PCR) to screen for Dengue virus followed by rapid single-tube multiplex RT-PCR for serotyping of the virus. J Clin Microbiol. 2007;45:935–941. doi: 10.1128/JCM.01258-06. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Checklist S1

STARD Checklist

(0.98 MB PDF)


Articles from PLoS Neglected Tropical Diseases are provided here courtesy of PLOS

RESOURCES