Variable Sensitivity in Molecular Detection of Zika Virus in European Expert Laboratories: External Quality Assessment, November 2016

ABSTRACT

Zika virus (ZIKV) infections are a significant public health concern. A strong capability for ZIKV detection is an absolute requirement for adequate preparedness and response strategies and individual patient care. The objective of this study was to assess and improve the capability of European expert laboratories for molecular testing for ZIKV through an external quality assessment (EQA) scheme. Laboratories were provided a panel of 12 samples which included negative samples, samples containing African- or Asian-lineage ZIKV at various concentrations (10³ to 10⁹ copies/ml), and samples containing dengue virus, yellow fever virus, or chikungunya virus. The results were analyzed on the basis of the outcomes of testing for the samples and the extraction and detection method used. Samples with a ZIKV RNA status scored correctly by >50% of the laboratories were designated the core sample. A total of 85 panel outcomes were submitted by 50 laboratories in 31 countries. The results designated all samples as core samples. Thirty-three percent (28/85) of the panel outcomes identified all samples. Analysis at the laboratory level showed that only 40% of the laboratories (20/50), representing 45% of the countries, scored sufficiently; i.e., they had at least one test operational that scored all core samples correctly. There is a need for improvement of the molecular detection of ZIKV in 60% of the participating laboratories. While the specificity of the tests was more robust, the results of the EQA showed large variation in test sensitivity. Improvements should focus on both nucleic acid extraction and ZIKV detection methods.

INTRODUCTION

Zika virus (ZIKV) has become a significant public health concern since the outbreaks in Micronesia in 2007 (1, 2) and French Polynesia in 2013 and 2014 (3) and, in particular, since its emergence in the Americas in 2015 and 2016 (4). ZIKV infections have been linked to congenital malformations in neonates from mothers that were infected with ZIKV during pregnancy and to neurological disorders, like Guillain-Barré syndrome (GBS), in adults (5, 6). ZIKV is a positive-sense RNA virus that belongs to the genus Flavivirus, family Flaviviridae. ZIKV transmission to humans occurs via bites of infected mosquitoes, mainly those of the Aedes genus (7). Sexual transmission via infected individuals has been described, as has mother-to-fetus transmission, which results in severe adverse effects (8). ZIKV viremia is typically short-lived, but viral RNA can be detected for longer periods in some specimens, e.g., semen, urine, or placental tissue (9, 10). A high degree of cross-reactivity with other flaviviruses is seen in the serology of ZIKV infections, precluding a definitive diagnosis, which may be problematic when pregnant women are tested. For this reason, molecular detection of ZIKV is preferred for the laboratory diagnosis of ZIKV infections, despite the relatively short-lived and low-level viremia (11, 12).

Due to the high number of travelers between areas where ZIKV is endemic and Europe, importation of ZIKV to Europe through viremic individuals is not uncommon. From June 2015 to 27 January 2017, 21 European Union countries reported a total of 2,081 travel-associated cases of ZIKV infection to the European Centre for Disease Prevention and Control (ECDC). This total included 103 pregnant women (13). The capability for the timely and accurate identification of ZIKV infections is important for preparedness and the response to emerging cases of disease in Europe, enabling containment of a putative local ZIKV transmission at an early stage (14, 15). The availability of reliable tools for the diagnosis of ZIKV infections is also imperative to avoid the misdiagnosis of potential ZIKV infections in pregnant women, which can have severe implications for pregnancy follow-up. This requires that ZIKV diagnostic methods, including molecular detection, have high degrees of accuracy (i.e., high specificities and sensitivities). As part of the activities at the European Union level to facilitate the implementation of preparedness and response plans, which include the availability of sufficient capacity and capability of ZIKV diagnostic tools, the Emerging Viral Diseases Expert Laboratory Network (EVD-LabNet; https://www.evd-labnet.eu/) organized an external quality assessment (EQA) for molecular detection of ZIKV in October and November 2016.

The objectives of the EQA were to assess the capability of European expert laboratories to perform accurate ZIKV molecular diagnostics and to provide insight into points for improvement of their detection capability by analysis of the outcome of testing in relation to the sample extraction and test methods used.

RESULTS

Test systems and overall assay performance.

The participating laboratories (n = 50) submitted a total of 85 EQA panel outcomes. These were obtained with four types of molecular detection methods and were based on 18 different test systems, besides 12 not further specified, referred to as “own design” methods (Table 1). Twenty-eight percent of the panel outcomes were based on a commercial test (24/85).

TABLE 1.

Overview of RT-PCR methods used for ZIKV RNA detection in EQA

Method (readout)	Authors, manufacturer, or source, yr (reference no.)	No. of submissions
In-house conventional RT-PCR	Faye et al., 2008 (16)	1
	Balm et al., 2012 (17)	1
	Moureau et al., 2007 (18)	1
	Patel et al., 2013 (19)	1
	Own design	1
	Scaramozzino et al., 2001 (20)	3
In-house quantitative real-time RT-PCR (no. of RNA copies/ml)	Corman et al., 2016 (21)	3
Real-time RT-PCR (CT value)
In-house	Corman et al., 2016 (21)	3
	Faye et al., 2013 (22)	13
	Lanciotti et al., 2008 (2)	17
	Pyke et al., 2014 (23)	1
	Tappe et al., 2014 (24)	1
	Own design	8
	CDC Trioplex (25)	4
Commercial	Altona (26)	16
	Clonit (27)	1
	Fast Track Diagnostics (28)	1
	Genesig (29)	1
	Liferiver (30)	2
	Roche (31)	2
	Sacace Biotechnologies (32)	1
In-house RT–loop-mediated isothermal amplification (time [in minutes])	Own design	3

Table 2 provides a summary of the EQA test outcomes (n = 85) per sample. On the basis of the submitted results, all samples in the panel were designated core samples, which were defined beforehand to be those samples for which more than 50% of the participating laboratories scored a correct ZIKV status, and these samples were set by expert opinion to be the minimum for each laboratory to score correctly to achieve a sufficient analytical performance.

TABLE 2.

Summary of EQA test outcomes per sample

Sample no.	Strain or sample	No. of copies/0.2 ml	No. of tests with:			Total % of tests with correct result
			Correct result	Inconclusive result	False-positive result
4	ZIKV (Asian lineage)	5.54E+6	83	0	2	97.6
5	ZIKV (Asian lineage)	3.62E+4	67	1	17	78.8
6	ZIKV (Asian lineage)	3.62E+2	45	1	39	52.9
7	ZIKV (African lineage)	1.2E+6	80	2	3	94.1
8	ZIKV (African lineage)	8.3E+3	56	2	27	65.9
11	ZIKV (Asian lineage)	2.5E+8	85	0	0	100
1	Chikungunya virus (Asian lineage)	2.05E+5	84	1	0	98.8
2	Yellow fever virus	8.53E+6	84	1	0	98.8
3	Dengue virus type 1	5.84E+4	80	1	4	94.1
9	Negative-control plasma		84	0	1	98.8
10	Negative-control plasma		83	0	2	97.6
12	Negative-control urine		75	2	7	89.3a

^aThe total percentage of 84 tests conducted with the correct result (all others based on the total of 85 tests).

A total overview of the EQA test outcomes per sample and anonymized laboratory on the basis of the scores and methods is given in Table S1 in the supplemental material. Twenty-eight of 85 panel outcomes (33%) achieved the maximum score of 24 points, while the lowest outcome score was 14 points (n = 1, 1.1%). Analysis at the laboratory level showed that 20 laboratories (40%, n = 50) had operational at least one test that obtained the maximum score. The highest score for the laboratory with the poorest performance was 15 points. The overall analytical specificity was 95% at the laboratory level and 96% at the test level. The overall analytical sensitivity was 87% at the laboratory level and 82% at the test level.

The analytical sensitivity and specificity of the molecular detection methods in combination with the applied extraction methods were analyzed in more detail to gauge the influence of different extraction methods and different detection methods on EQA outcomes (Table S2). The extraction methods used most often were those of the MagNA Pure system (Roche Molecular Systems) and the QIAamp viral RNA minisystem (Qiagen). However, the number of test results for which both the PCR step and the method used for the extraction step were indicated was too small to be able to draw solid conclusions about the preferred methods. Furthermore, three laboratories experienced problems with clotting of the EQA material while using the QIAamp viral RNA miniextraction system (Qiagen).

DISCUSSION

Of the 50 laboratories that participated in the EVD-LabNet EQA for molecular testing for ZIKV, only 40% had the maximum score possible. At the country level, the score was 45%: there were 14 out of 31 countries with at least one laboratory with a 100% score with the core samples. This indicates a clear need for improvement of the molecular detection of ZIKV in EVD-LabNet laboratories to achieve an adequate state of preparedness and response to the threat of ZIKV transmission in Europe. The observation that only 52% of the laboratories correctly scored the sample with the lowest concentration of 1.8 × 10³ RNA copies/ml is of concern. Plasma/serum is often the sample of choice in diagnostic laboratories (45), and the ZIKV viral loads in plasma that have been described in the literature (10, 33–36) are low and variable but are about 10² RNA copies/ml. With respect to analytical specificity and sensitivity, the risk of a false-negative outcome is especially a concern in laboratories that missed more than one of the ZIKV-positive samples. However, because of the typically short viremic phase of ZIKV infections (37), some laboratories will perform molecular testing in combination with serological testing for diagnosis. In May 2016, ZIKV serology was available in 24 European Union/European Economic Area (EEA) countries (45). Nonetheless, the potential negative consequences of false-negative results for ZIKV infection are considerable, especially for pregnant women, the partners of pregnant women, or women who wish to become pregnant, due to the teratogenic nature of ZIKV (5, 38). No differences in the detection of Asian-lineage ZIKV versus African-lineage ZIKV were observed in this EQA.

Overall, the results obtained by all assay systems were variable, and there is therefore no clear evidence for a significant advantage of, e.g., commercial assays over in-house assays. The total numbers of panels tested by each method do not allow significant conclusions to be made about the specific PCR method that laboratories should be advised to use, while the possibility that the extraction methods used influenced the results cannot be excluded. Although the performance of the EQA is the outcome obtained by the combination of the specific PCR method and the extraction method used, a comparison of the different extraction methods used with a specific molecular detection method seemed to suggest that assay sensitivity can be improved with a change to the extraction method. To accurately assess solely the extraction or PCR procedures, a different EQA setup, e.g., one in which panels that consist of extracted RNA are provided to assess the PCR methodology, should be used. However, the main aim of the current EQA was to assess the capability of the diagnostic laboratories for molecular testing for ZIKV in clinical samples. This requires assessment of the whole routine procedure from sample receipt to generation of a result. Nevertheless, this EQA analysis shows the participants whether improvement is needed and provides some insight into points for improvement. Furthermore, it provides the opportunity for the provision of within-network support by linking laboratories scoring less well with laboratories that scored 100%.

Improvements to methods of molecular detection of ZIKV seem to be needed in 60% of the participating laboratories. Furthermore, only 14 out of 26 participating European Union/EEA countries scored 100% on the core sample, indicating that improvement is needed in terms of adequate surveillance for ZIKV in European countries. This is particularly of concern for countries in which the competent vectors of ZIKV, such as Aedes aegypti and, putatively, A. albopictus mosquitoes, are established (39). Finally, the wide variety of test systems in use shows a profound absence of standardization across European laboratories, complicating a consistent pan-European surveillance strategy.

In light of the improvements to molecular testing for ZIKV needed in European laboratories, it is strongly recommended that molecular testing be combined with serological testing to establish a diagnosis, especially for pregnant women. A second EQA for the molecular diagnosis of ZIKV is needed in 2018 to monitor whether improvements have been made and validate the changes made by the participating laboratories.

MATERIALS AND METHODS

Organization of EQA scheme.

The EQA was organized by Erasmus MC, Rotterdam, The Netherlands, and Aix-Marseille University, Marseille, France, in October and November 2016. All registered members of EVD-LabNet (68 laboratories as of 1 October 2016) were invited to participate. Fifty-three laboratories from 33 countries registered online, and the panel was shipped on 20 October 2016. Fifty laboratories from 31 countries had submitted EQA results by 23 November 2016. A list of the participating laboratories can be found in the Acknowledgments.

Panel composition.

The EQA test panel consisted of 12 samples, 6 ZIKV RNA-positive samples and 6 ZIKV RNA-negative samples (Table 3). The panel comprised different concentrations of two ZIKV lineages, i.e., MRS_OPY_Martinique_PaRi_2015 (representing the Asian lineage, including the current outbreak strain) (40) and MR766 (which represented ZIKV of the African lineage) (41). Specificity was assessed by adding samples positive for other arboviruses that cocirculate in regions where the Zika virus epidemic occurs. These were one alphavirus and two flaviviruses: chikungunya virus (CHIKV) strain H20235/St. Martin/2013 of the Asian lineage (42), yellow fever virus (YFV) strain BOL 88/1999 (43), and dengue virus serotype 1 (DENV-1) strain CNR 16079/2012. The freeze-dried samples were prepared from 0.2 ml human plasma or urine that was either spiked or not spiked with different dilutions of a virus culture supernatant. The virus preparation was inactivated by heat treatment (60°C for 60 min). Proper inactivation was confirmed by the absence of a cytopathic effect in Vero cells and by an undetectable increase in ZIKV RNA levels after 5 days of virus culture. The viral loads per reconstituted sample were quantified with reference to in-house ZIKV-specific synthetic RNA controls. To achieve this, an ∼500-bp region that was tagged at the 5′ end with a T7 promoter sequence (5′-TAATACGACTCACTATAGGG-3′) and that encompassed the virus-specific TaqMan-targeted sequence was amplified by reverse transcription (RT)-PCR using an Access RT-PCR kit (Promega) according to the manufacturer's instructions. Purified amplicons were transcribed using a T7 MEGAshort script kit (Ambion, Thermo Fisher Scientific), and the RNA was purified using a MEGAclear purification kit (Ambion, Thermo Fisher Scientific). The RNA concentrations were measured using a NanoDrop 1000 spectrophotometer (Thermo Scientific) and translated into copy numbers. For each sample, real-time RT-PCR of sample RNA and a range of dilutions of the corresponding T7-generated RNA standard of known quantities (100 copies to 100 million copies) was performed on a QuantStudio 12K Flex real-time PCR system using an Express One-Step SuperScript quantitative RT-PCR kit (Universal; Life Technologies). The numbers of RNA copies of the sample were determined using QuantStudio 12K Flex software (v1.2.3).

TABLE 3.

Composition of ZIKV molecular EQA panel

Parameter	Sample no.	Virus, sample type	Strain	Amt of viral RNA
				No. of copies/0.2 ml	No. of copies/ml
Sensitivity	4	ZIKV (Asian), plasma	MRS_OPY_Martinique_PaRi_2015	5.5E+6	2.8E+7
	5	ZIKV (Asian), plasma	MRS_OPY_Martinique_PaRi_2015	3.62E+4	1.8E+5
	6	ZIKV (Asian), plasma	MRS_OPY_Martinique_PaRi_2015	3.62E+2	1.8E+3
	7	ZIKV (African), plasma	MR766	1.24E+6	6.2E+6
	8	ZIKV (African), plasma	MR766	8.3E+3	4.2E+4
	11	ZIKV (Asian), urine	MRS_OPY_Martinique_PaRi_2015	2.5E+8	1.2E+9
Specificity	1	CHIKV (Asian), plasma	H20235/St. Martin/2013	2.05E+5	1.0E+6
	2	YFV, plasma	BOL 88/1999	8.53E+6	4.3E+7
	3	DENV-1, plasma	CNR 16079/2012	5.84E+4	2.9E+5
Contamination	9	Negative-control plasma
	10	Negative-control plasma
	12	Negative-control urine

Evaluation of results and EQA scoring.

The following scoring system, which was described before (44), was used: 2 points were given per sample with a correct ZIKV RNA detection result, i.e., ZIKV RNA positive or negative, resulting in a maximum score of 24 points per test conducted. Samples listed as inconclusive were given one point. Results for laboratories that tested the EQA test panel with different assays were analyzed independently. For analysis on an overall laboratory level, the test with the highest overall score for each laboratory was used. In the case of equal scores but differences in the sensitivity or specificity of two tests by one laboratory, the test with the better sensitivity was used for analysis at the laboratory level.

Core samples were defined beforehand by the management team and the scientific advisory board of EVD-LabNet as those samples for which more than 50% of the participating laboratories scored a correct ZIKV status and were set as a minimum for each laboratory to score correctly to achieve a sufficient analytical performance.