Abstract
Reporter virus neutralization test (RVNT) has been used as an alternative to the more laborious and time-demanding conventional PRNT assay for both DENV and ZIKV. However, few studies have investigated how these techniques would perform in epidemic areas with the circulation of multiple flavivirus. Here, we evaluate the performance of ZIKV and DENV Rluc RVNT and ZIKV mCh RVNT assays in comparison to the conventional PRNT assay against patient sera collected before and during ZIKV outbreak in Brazil. These samples were categorized into groups based on (1) acute and convalescent samples according to the time of disease, and (2) laboratorial diagnostic results (DENV and ZIKV RT-PCR and IgM-capture ELISA). Our results showed that DENV Rluc assay presented 100% and 78.3% sensitivity and specificity, respectively, with 93.3% accuracy, a similar performance to the traditional PRNT. ZIKV RVNT90, on the other hand, showed much better ZIKV antibody detection performance (around nine-fold higher) when compared to PRNT, with 88% clinical sensitivity. Specificity values were on average 76.8%. Even with these results, however, ZIKV RVNT90 alone was not able to reach a final diagnostic conclusion for secondary infection in human samples due to flavivirus cross reaction. As such, in regions where the flavivirus differential diagnosis represents a challenge, we suggest the establishment of a RVNT panel including other flaviviruses circulating in the region, associated with the other serological techniques such as IgM ELISA and the investigation of seroconversion, in order to help define an accurate diagnostic conclusion using serology.
Keywords: reporter virus, neutralization test, Dengue, ZIKV, flavivirus, diagnosis
1. Introduction
Zika virus (ZIKV) is a mosquito-borne flavivirus (family Flaviviridae), isolated from the blood of a febrile sentinel rhesus monkey in the Zika Forest of Uganda in 1947 [1]. In the following 60 years, ZIKV was isolated from several species of Aedes spp. mosquitos [2,3] and associated to oligoassymptomatic cases, characterized by fever, rashes, myalgia, arthralgia, conjunctivitis, gastrointestinal disturbance, and headaches [4,5]. Since 2007, outbreaks have been reported in the Southeast regions of Africa and Asia [6,7,8]. Three ZIKV lineages are recognized and related in the epidemiologic data: East African, West African, and Asian lineages [9,10,11], but only one serotype has been found [8,10].
In 2015, ZIKV Asian lineage emerged in Brazil. A dramatic increase of microcephaly and Guillain-Barré Syndrome (GBS) cases have reported and associated with ZIKV [12,13,14]. Besides mosquito vectors, ZIKV was found to be transmitted by maternofetal route, sexual intercourse, blood transfusion, and organ transplantation [12,15]. Consequently, ZIKV spread quickly throughout South America and Caribbean region and a pandemic was established. In 2016, WHO announced “Public Health Emergency of International Concern” for ZIKV outbreak [16].
In these epidemic areas, with the co-circulation of several other flaviviruses, the diagnosis to differentiate them became a huge challenge. Since viremia is rather short-lived, to confirm the ZIKV infection the antibodies detection has been used [17] by ZIKV MAC-ELISA (CDC, Atlanta) or the IgM capture ELISA for ZIKV detection (InBios, Washington, USA) [16,18]. However, cross-reactive flaviviruses antibodies can make the interpretation of serological results notoriously difficult, especially in patients with previous flavivirus infections and/or vaccinations.
Plaque reduction neutralization test (PRNT) remains the gold standard to confirm and specify flavivirus antibodies, but it is labor-intensive, expensive, and not widely available, thus limiting its use for routine diagnostics, mainly during outbreaks [18,19]. One alternative test developed was the reporter virus neutralization test (RVNT). The ZIKV mCherry report virus has been used to evaluate ZIKV vaccine efficacy in mice and monkey models [20,21,22]. Recently, ZIKV and DENV-2 Renilla luciferase (Rluc) reporter assay was validated for human samples, using sera from asymptomatic travelers who returned from ZIKV epidemic areas. This reporter assay showed equivalent results as a traditional plaque assay and improved test turnaround time, dynamic range, and diagnostic throughput [23]. In this communication, we evaluated RVNT, using samples from ZIKV and Dengue symptomatic cases from Brazil, and also validated the ZIKV mCherry report virus for human samples.
2. Results
2.1. Groups’ Categories
We selected specimens from symptomatic patients compatible with DENV and/or ZIKV infections and tested by traditional PRNT and RVNT. The median age was 35.9 years old and 60% were female non-pregnant. These samples were categorized in four groups based on: (1) time of disease after symptoms’ onset and (2) laboratorial diagnosis results. Group I (n = 20) include patients who had diagnosis for DENV by RT-PCR and with samples collected between 1 to 5 days after symptoms’ onset; Group II (n = 19) include samples positives to DENV by IgM-capture ELISA and with samples collected with more than 5 days of symptoms’ onset. The patients with diagnosis for ZIKV were included in Group III (n = 19) and Group IV (n = 17). Group III contains acute samples tested for ZIKV RT-qPCR, while Group IV includes convalescent samples positives to ZIKV by IgM-capture ELISA from patients with diagnosis confirmed in the acute phase by ZIKV RT-qPCR. The majority of DENV positive samples in Groups I and II were collected before ZIKV introduction in Brazil [24], but five of them were collected during ZIKV outbreak, showing monotypic reaction to DENV in the serological tests (H815485, H817979, H817981, H819129, and H819519). On the other hand, ZIKV positive samples (Groups III and IV) were collected only during 2015/2016 outbreak (Table 1).
Table 1.
Samples per group with time of disease and the PRNT and RVNT antibodies titers.
| Samples | PRNT90 | RVNT90 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Group | ID | # | Year | Time of Disease | ZIKV | DENV2 | Rluc DENV2 | Rluc ZIKV | mCherry ZIKV |
| H726608 | 1 | 2007 a | 7 days | Neg | 1:320 | 1:25,600 | Neg | Neg | |
| H769882 | 2 | 2010 a | 4 days | Neg | 1:20 | 1:400 | Neg | Neg | |
| H769929 | 3 | 2010 a | 4 days | Neg | Neg | Neg | Neg | Neg | |
| H770060 | 4 | 2010 a | 2 days | Neg | Neg | 1:200 | Neg | Neg | |
| H770220 | 5 | 2010 a | 3 days | Neg | Neg | Neg | Neg | Neg | |
| H773035 | 6 | 2010 a | 3 days | Neg | 1:80 | 1:1600 | Neg | Neg | |
| H773052 | 7 | 2010 a | 4 days | Neg | 1:20 | 1:12,800 | Neg | Neg | |
| H773054 | 8 | 2010 a | 3 days | Neg | 1:20 | 1:400 | Neg | Neg | |
| H773564 | 9 | 2010 a | 4 days | Neg | Neg | Neg | Neg | Neg | |
| I | H773565 | 10 | 2010 a | 4 days | Neg | 1:320 | 1:12,800 | Neg | Neg |
| H773582 | 11 | 2010 a | 4 days | Neg | Neg | Neg | Neg | Neg | |
| H774749 | 12 | 2010 a | 5 days | Neg | 1:160 | 1:3200 | Neg | Neg | |
| H775262 | 13 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H775844 | 14 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H775845 | 15 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H775848 | 16 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H775852 | 17 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H775853 | 18 | 2010 a | <5 days | Neg | 1:80 | 1:400 | Neg | Neg | |
| H775854 | 19 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H775862 | 20 | 2010 a | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H627940 | 21 | 2000 a | >5 days | Neg | 1:640 | 1:3200 | Neg | Neg | |
| H632195 | 22 | 2000 a | >5 days | Neg | 1:320 | 1:3200 | Neg | Neg | |
| H674584 | 23 | 2004 a | >5 days | Neg | 1:20 | 1:200 | Neg | Neg | |
| H738095 | 24 | 2007 a | >5 days | Neg | 1:80 | 1:400 | Neg | Neg | |
| H739187 | 25 | 2007 a | 6 days | Neg | 1:320 | 1:12,800 | Neg | Neg | |
| H739983 | 26 | 2008 a | >5 days | Neg | 1:640 | 1:6400 | Neg | Neg | |
| H787665 | 27 | 2012 a | 14 days | Neg | 1:10,240 | 1:25,600 | Neg | Neg | |
| H788930 | 28 | 2012 a | >5 days | Neg | 1:640 | 1:6400 | Neg | Neg | |
| H789010 | 29 | 2012 a | >5 days | Neg | 1:5120 | 1:25,600 | Neg | Neg | |
| II | H789197 | 30 | 2012 a | >5 days | Neg | 1:10,240 | >1:51,200 | 1:200 | 1:200 |
| H789912 | 31 | 2012 a | >5 days | Neg | 1:160 | 1:800 | Neg | Neg | |
| H789990 | 32 | 2012 a | >5 days | Neg | 1:640 | 1:12,800 | Neg | Neg | |
| H789997 | 33 | 2012 a | >5 days | Neg | 1:2560 | 1:25,600 | 1:100 | Neg | |
| H790260 | 34 | 2012 a | >5 days | Neg | 1:1280 | 1:6400 | Neg | Neg | |
| H815485 | 35 | 2015 | >5 days | Neg | 1:20 | Neg | Neg | Neg | |
| H817979 | 36 | 2015 | >5 days | Neg | 1:20 | 1:200 | Neg | Neg | |
| H817981 | 37 | 2016 | >5 days | Neg | 1:1280 | 1:12,800 | Neg | Neg | |
| H819129 | 38 | 2015 | >5 days | Neg | 1:160 | 1:1600 | Neg | Neg | |
| H819519 | 39 | 2015 | >5 days | Neg | 1:320 | 1:1600 | Neg | Neg | |
| H817986 | 40 | 2015 | <5 days | Neg | Neg | Neg | Neg | Neg | |
| H819966 | 41 | 2015 | 3 days | Neg | Neg | Neg | Neg | Neg | |
| H820771 | 42 | 2015 | 1 day | Neg | 1:320 | 1:1600 | 1:800 | 1:400 | |
| H821519 | 43 | 2015 | <5 days | Neg | 1:160 | 1:1600 | Neg | Neg | |
| H821557 | 44 | 2015 | 2 days | Neg | 1:80 | 1:800 | 1:200 | 1:100 | |
| H821585 | 45 | 2015 | 3 days | Neg | 1:20 | 1:200 | 1:100 | Neg | |
| H821735 | 46 | 2015 | <5 days | Neg | Neg | 1:400 | Neg | Neg | |
| H821956 | 47 | 2015 | 2 days | Neg | Neg | Neg | Neg | Neg | |
| III | H822137 | 48 | 2015 | 4 days | Neg | Neg | 1:100 | Neg | Neg |
| H822215 | 49 | 2015 | <5 days | Neg | 1:80 | 1:400 | Neg | Neg | |
| H822217 | 50 | 2015 | <5 days | Neg | 1:80 | 1:800 | Neg | Neg | |
| H822226 | 51 | 2015 | 3 days | Neg | Neg | Neg | Neg | Neg | |
| H822604 | 52 | 2015 | 2 days | Neg | Neg | Neg | Neg | Neg | |
| H823608 | 53 | 2015 | 4 days | Neg | Neg | Neg | Neg | Neg | |
| H823390 | 54 | 2015 | 3 days | Neg | 1:80 | 1:800 | 1:200 | Neg | |
| H823594 | 55 | 2015 | 1 day | Neg | Neg | 1:400 | Neg | Neg | |
| H824562 | 56 | 2015 | 3 days | Neg | 1:320 | 1:800 | Neg | Neg | |
| H824564 | 57 | 2015 | 1 day | Neg | Neg | Neg | Neg | Neg | |
| H824708 | 58 | 2015 b | 4 days | Neg | Neg | 1:1600 | 1:6400 | 1:6400 | |
| H816971 | 59 | 2015 | 12 days | 1:20 | 1:1280 | 1:12,800 | 1:1600 | 1:1600 | |
| H819282 | 60 | 2015 | >5 days | 1:40 | 1:5120 | 1:25,600 | 1:6400 | 1:3200 | |
| H819284 | 61 | 2015 b | >5 days | 1:80 | 1:1280 | 1:6400 | >1:51,200 | 1:3200 | |
| H819360 | 62 | 2015 | 8 days | Neg | 1:640 | 1:12,800 | 1:1600 | 1:400 | |
| H820967 | 63 | 2015 | 7 days | Neg | 1:1280 | 1:12,800 | 1:1600 | 1:800 | |
| H821489 | 64 | 2016 | >5 days | 1:320 | 1:2560 | 1:12,800 | 1:12,800 | 1:1600 | |
| H821491 | 65 | 2015 b | >5 days | Neg | 1:160 | 1:800 | 1:1600 | 1:3200 | |
| H822200 | 66 | 2015 | >5 days | Neg | 1:160 | 1:1600 | 1:1600 | 1:1600 | |
| H822413 | 67 | 2015 | >5 days | Neg | 1:1280 | 1:6400 | 1:1600 | 1:6400 | |
| VI | H822540 | 68 | 2015 | >5 days | 1:20 | 1:640 | 1:6400 | 1:1600 | Neg |
| H823176 | 69 | 2015 b | 50 days | 1:160 | 1:640 | 1:6400 | >1:51,200 | 1:12,800 | |
| H824696 | 70 | 2015 | 7 days | Neg | 1:320 | 1:1600 | 1:200 | 1:400 | |
| H825051 | 71 | 2015 | 11 days | Neg | 1:640 | 1:12,800 | 1:1600 | 1:3200 | |
| H826145 | 72 | 2016 b | >5 days | Neg | 1:20 | 1:400 | 1:1600 | 1:1600 | |
| H828107 | 73 | 2016 | >5 days | Neg | 1:1280 | 1:25,600 | 1:1600 | 1:3200 | |
| H828108 | 74 | 2016 | >5 days | Neg | 1:1280 | 1:6400 | 1:6400 | 1:6400 | |
| H829179 | 75 | 2016 | 19 days | 1:20 | 1:1280 | 1:6400 | 1:1600 | 1:3200 | |
2.2. Neutralization Tests
Eight samples from Group I probably have previous antibodies against DENV, but all of them were negative for ZIKV by PRNT90 and RVNT90. Therefore, in DENV secondary infection cases followed by other non-ZIKV flaviviruses (before ZIKV pandemic), observed in Group II, just a few samples showed unspecific antibodies against ZIKV and only in low dilutions for both assays. In Group III, despite the fact ZIKV genome was detected in theses samples, the ZIKV PRNT90 result was negative. In addition, eight samples (42.1%) were positive by both ZIKV and DENV RVNT90. These data indicate that in the scenario of a previous flavivirus infection followed by a ZIKV secondary infection, previous DENV unspecific antibodies may react with ZIKV antigen, but can only be detected in the lower dilutions (<1:1600). Later on, after 5 days post-symptoms, the titers of ZIKV antibodies increase during the secondary infection, but also do the DENV antibodies. This can be observed in Group IV. All samples were positive for DENV and ZIKV by RVNT90. Interestingly, the traditional ZIKV PRNT90 was not able to detect ZIKV antibodies in 88.2% of the samples, but using RVNT90 was possible to titer ZIKV antibodies in all of them. In addition, 23.5% (n = 4) of Group IV samples have ZIKV titer higher than DENV titer but it is not correlated to the time of disease (Table 1).
2.3. RVNT
For both DENV and ZIKV RVNT, (RLuc and mCherry), the test cut offs were kept at 90%, following PRNT criteria. However, the dilutions lower than 1:100 were defined as unspecific reactions since RVNT has been shown higher sensitivity compared to PRNT [23] due the high cross reaction observed among flavivirus. To calculate the RVNT90 assays’ sensitivity and specificity, all samples were also tested by traditional PRNT as the gold-standard test. For Dengue, DENV Rluc assay had a sensitivity and specificity equal to 100% and 78.3%, respectively, and showed 93.3% of accuracy. The ROC curve showed that DENV RVNT Rluc had a similar performance to PRNT (Figure 1A).
Figure 1.
ROC curves comparing the performance of the DENV and ZIKV RVNT assays with the conventional PRNT. (A) DENV RVNT Rluc versus PRNT; (B) ZIKV RVNT Rluc versus PRNT; (C) ZIKV RVNT mCherry versus PRNT. (D) Comparison between ZIKV RVNT assays and the ZIKV PRNT using ZIKV diagnostic results (IgM-ELISA) as query.
3. Discussion
The arbovirus circulation in Brazil has been constantly changing throughout the last decades, with successive epidemics, co-circulation of one or more serotypes of DENV [24,25], reemergence of (YFV) [26,27], and the introduction of ZIKV [24,28,29,30,31] and Chikungunya viruses [32,33]. Consequentially, an alarming increase has been noted in the number of people that have been affected. Because these viruses have similar symptoms, the laboratorial tools are essential to distinguish them. However, for flavivurus, the diagnosis is a huge challenge. A significant cross-reaction is observed using the traditional methods, such as Hemaglutition Inhibition (HI) and Mac-ELISA, leading to a high number of cases that remain without conclusion. For a long time, PRNT has been used for specific flavivirus diagnosis; however, the turnaround time to a final result can take a week. In this paper, it is shown that RVNT can efficiently replace the traditional PRNT in epidemic areas for flavivirus, reducing the result time to 3 days.
In samples from Groups I and II, the DENV RVNT was able to elucidate dengue diagnostics by detecting DENV specific neutralizing antibodies, with high sensitivity, specificity, and accuracy values, and similar performance as the PRNT (Figure 1A and Figure 2A). As for ZIKV differential diagnostics, there was low neutralization of ZIKV by unspecific antibodies, with more than 95% of samples with no cross-reaction detected between ZIKV and DENV (Figure 2B). The lack of ZIKV reactive antibodies produced in DENV infections has also been reported by Collins et al. 2017. These sera were harvested before the introduction of ZIKV in Brazil [24,34]; therefore, the lack of ZIKV exposition leads to the absence of ZIKV-specific antibodies. These data show that there is a reduced chance of ZIKV false positive results in samples from patients that have been exposed to other flavivirus but are naïve for ZIKV, suggesting that the ZIKV RVNT assay specificity is high even in regions that are highly endemic for other flavivirus such as DENV but with no ZIKV circulation (Figure 1B,C) [17,23].
Figure 2.
Neutralizing antibodies titers comparison. (A) DENV RVNT Rluc versus PRNT; (B) ZIKV RVNT Rluc and mCherry versus PRNT.
However, one should be extra careful when analyzing the ZIKV RVNT results of samples from regions where there is actually co-circulation of ZIKV and other flavivirus such as DENV, especially in ZIKV secondary infections. We observed that in samples harvested during the ZIKV epidemics (Groups III and IV), there was high cross-reactivity between ZIKV and DENV. Interestingly, the majority of these samples (11/17) showed high total antibody titers for flavivirus in the IH assay (data not shown). Taking into consideration that IgM titers from flavivirus infections can persist for a long time, ranging from 90 days for DENV [35] to as long as one year for WNV [36], it is possible that high IgM antibodies titers from previous flavivirus infections may be contributing to this cross-reactivity. We should not rule out the possibility that this cross-reactivity could also be enhanced by the higher sensitivity of the RVNT when compared to the traditional PRNT.
The samples from Group III are ZIKV acute samples (less than 5 days of symptoms’ onset) diagnosed by ZIKV RT-qPCR; thus, no detectable ZIKV neutralizing antibodies were to be expected from this sample set. However, we observed that some samples (5/19) were positive for RVNT, whereas all samples were negative for the traditional PRNT (Table 1). Given the notoriously cross-reaction between flavivirus, these positive samples may be an unspecific reaction from cross-reactive DENV antibodies, as all these five samples also were positive for the DENV RVNT. Although unlikely, we should also not rule out the possibility that due to the ZIKV Rluc RVNT higher sensitivity as compared to the traditional PRNT (Figure 1D), the RVNT could be detecting ZIKV antibodies earlier in the infection when compared to PRNT. However, to evaluate this hypothesis, a different sample set of acute and convalescent samples with no history of previous flavivirus infection is needed.
If we now look specifically at the samples from Group IV, which contains only convalescent sera with more than five days of disease onset, it is extremely hard to identify ZIKV-specific results due to the mass cross-reaction with DENV antibodies that may be occurring given the results of the DENV RVNT. Even so, if we look at each sample individually, the ZIKV antibodies titers seem to be lower than the DENV antibodies titers, especially in the initial stages of disease. One possible solution for this problem is to request another sample collected later in the convalescent phase (more than 15 days) when the ZIKV antibodies titer might be higher than the previous DENV antibodies, as observed with sample 69 (Table 1), collected 50 days after disease onset. In fact, this increase of ZIKV antibodies titer collected later has been reported in a previous study using mother and neonate samples infected by ZIKV [35,37].
One of the PRNT’s disadvantages in flavivirus epidemic areas is the need of at least two samples to define a case and in most of the cases, the neutralizing antibodies can only be detected around 14 days after disease onset. In an experimental infection with non-human primates, using ZIKV RNVT90 mCherry to evaluate the efficacy of ZIKV vaccine candidate, Shan et al., 2017 [18] showed that this assay could detect ZIKV antibodies in the later viremic period. Here, we used samples with less than 5 days of the symptoms’ onset. This period is still early to define a conclusion for human samples showing secondary infections by flavivirus. Based on these results, we noticed that immune response profile of Brazilian population has been changed after ZIKV introduction.
Since the PRNT has been widely used to confirm MAC-ELISA results and exclude other flaviviruses, it would be invaluable to establish a RVNT panel, using flaviviruses of most medical importance, including ZIKV, DENV YFV, Saint Louis virus (SLV), West Nile virus (WNV), and Rocio virus (ROCV). In the case of DENV, where the co-circulation of viral serotypes is frequent, one could either choose the most predominant serotypes [17,38] or use DENV-2 as a standard, since it shows higher amino acidic relationship among all the other DENV serotypes, based on the E protein [39]. It would be interesting to investigate more components involved in the RVNT performance, the importance of both IgM and IgG in the cross-reactivity, and the evaluation of other arbovirus by RVNT in endemic areas.
4. Materials and Methods
4.1. Specimen Selection
This study was conducted by Instituto Evandro Chagas (IEC), Ananindeua, Para State, Brazil and University of Texas Medical Branch, Galveston, Texas, US (UTMB). All sera tested in this study were collected from patients who live in North or Northeast regions of Brazil, where co-circulation of ZIKV, DENV, yellow fever virus (YFV), and other flaviviruses has been detected. Furthermore, the secondary infection for all samples were confirmed by Hemagglutination Inhibition (HI) assay [40].
Seventy-five (75) serums were previously diagnosed to Dengue or ZIKV, based on authorized assays by Brazilian Minister of Health and PAHO (protocol number 3.601.702) and following the flowchart for arboviruses/ZIKV diagnosis in IEC (Figure S1). All of them were also tested by in-house MAC-ELISA IgM to other flaviviruses, such as YFV, Saint Louis Encephalitis virus (SLEV), West Nile virus (WNV), Ilheus virus (ILHV), and Rocio Virus (ROCV), showing negative results. Early acute patient sera, collected until 5 days after onset of disease symptoms, were previously tested by RT-qPCR [17], whereas later acute phase or convalescent patient samples were collected after 5 days of the onset of symptoms and tested by IgM-capture ELISA [41]. Samples were collected between 2000 and 2016, before and after ZIKV outbreak in Brazil (Table 1).
4.2. Cells and Viruses
Vero E6 and BHK-21 cells were purchased from the American Type Culture Collection (ATCC, Bethesda, MD, USA), and maintained in a high-glucose Dulbecco′s modified Eagle′s medium (DMEM) supplemented with 10% fetal bovine serum (FBS; HyClone Laboratories, South Logan, UT) and 1% penicillin/streptomycin at 37 °C with 5% CO2. Infectious cDNA clone pFLZIKV were constructed previously and used to produce Renilla luciferase ZIKV, mCherry ZIKV (ZIKV Cambodian strain FSS13025) [20,42]. Renilla luciferase DENV-2 (strain NGC) was previously described by Zou et al., 2011. cDNA plasmids were used to in vitro transcribe genomic RNAs. Reporter ZIKV RNA transcripts were transfected into Vero cells, whereas reporter DENV-2 RNA were transfected in BHK-21 cells. To eliminate its interference with luciferase signal measurement, luciferase ZIKV and DENV-2 transfected cells were cultured in DMEM without phenol red. On day 10 (ZIKV) and 6 (DENV) post-transfection, culture fluids were collected and quantified for viral titers using an immuno-staining focus assay and plaque assay, respectively, as previously reported [42]. The three reporter genes were engineered at the beginning of the viral genome open-reading-frame, as detailed elsewhere [20,42,43]. For the standard PRNT assay, we used ZIKV Puerto Rico strain PRVABC59 and DENV-2 New Guinea (NGC) strain.
4.3. Plaque Reduction Neutralization Test
Standard PRNT was performed according to Castanha et al., 2013 [44]. Virus dilutions of Puerto Vero cells and BHK cells concentration were 2 × 105 cells/mL in a high-glucose Dulbecco′s modified Eagle′s medium (DMEM) supplemented with 10% fetal bovine serum (FBS; HyClone Laboratories, South Logan, UT, USA) and 1% penicillin/streptomycin at 37 °C with 5% CO2. In each 24-well plate, a single serum sample was tested in duplicate with DMEM 2% FBS and 1%P/S from 1:20 to 1:20480 dilution. After incubation, challenge, and infection, both for 1 h at 37 °C with 5% CO2, 500 µL of 0.8% methyl cellulose overlay was added. After 4 days the incubation, plates were fixed with 3.7% formaldehyde for 30 min and stained with 1% crystal violet for 5 min. The neutralization titers were determined by plaque quantity formed in controls wells (without antibody). The serum dilution that inhibited at least 90% of the tested virus inoculum (PRNT90) was considered the antibody titer [23,45].
4.4. Reporter Virus-Based Neutralization Assay
Reporter ZIKV and DENV-2 containing a Renilla luciferase (Rluc) gene and reporter ZIKV containing mCherry (mCh) were used to measure the neutralization titers of patient sera against ZIKV or DENV-2, according to Shan et al., 2017 [21]. Briefly, Vero cells (1.5 × 104 cells per well) were seeded into either 96-well white plate for Rluc or 96-well black plate for mCh (Corning Costar, St. Louis, MO, USA) one day prior to infection. Patient sera were initially diluted as 10-fold in a phenol red-free DMEM medium (ThermoFisher Scientific, Sugar Land, TX, USA) containing 2% FBS and 1% penicillin/streptomycin, followed by 2-fold serial dilution (1:100 to 1:51,200). Thirty microliters of each serum dilution were mixed thoroughly with 30 μL reporter ZIKV or DENV-2 and incubated at 37 °C for 1 h to form antibody-virus complexes. Afterwards, 50 μL serum-virus mixtures were inoculated onto the Vero cell monolayer (containing 50 μL phenol red-free DMEM medium with 2% FBS and 1% penicillin/streptomycin). The plate was incubated at 37 °C for 24 h (Rluc) or 48 h (mCh). The intracellular luciferase signals were measured using ViviRen substrates (Promega, Madison, WI, USA) and the mCherry fluorescence signals were measured directly on Cytation5 Cell Imaging Multi-Mode Reader (Biotek, Winooski, VT, USA) according to the manufacturer′s instructions. A medium containing the same amounts of reporter ZIKV or DENV-2 but without specimen serum was used as a non-treatment control and wells containing non-infected cells were used as non-signal controls for read background normalization for both luciferase and mCherry. Luciferase and mCherry signals from the non-treatment controls were set at 100%, whereas the signals from the non-signal controls were set at 0%. Luciferase and mCherry fluorescence signals from each diluted serum-treated samples were normalized to those from the non-treatment controls. A four-parameter sigmoidal (logistic) model in the software GraphPad Prism 7 was used to calculate the neutralization titers that suppressed 90% of the luciferase or mCherry signals of the non-treatment control RVNT90.
4.5. Statistical Analysis
The results were analyzed with the GraphPad Prism v7.02 software and Bioestat 5.3 [46]. Sensitivity, specificity, and accuracy for mCherry ZIKV, Rluc ZIKV, and Rluc DENV RVNT were calculated using the conventional PRNT90 assay as reference. In addition, sensitivity, specificity, and accuracy was calculated for mCherry ZIKV using Rluc ZIKV as reference [21]. Additionally, we calculated the ROC curve and McNemar test analysis was performed to compare neutralization techniques. All analyses used a p-value of <0.05 for statistical significance.
5. Conclusions
Finally, RVNT90 was not able to define a final diagnostic conclusion, but RVNT could be a useful tool to be included in the diagnostic algorithm of ZIKV and DENV alongside with clinical and epidemiologic data associated with MAC-ELISA results as indicative of recent infection. This could solve the cases of primary infection. As for the cases of secondary infection, a second sample collected later to evaluate seroconversion might be necessary especially in regions where the differential diagnosis represents a challenge.
Acknowledgments
We would like to kindly thank UTMB for the donation of the reporter viruses used in this study; and to the team at the IEC bio-bank and laboratory of molecular biology who dedicated part of their time to the finding and separation of the samples used here, specifically to Alice Queiroz, Daniela Rodrigues, and Gleiciane Pinheiro.
Supplementary Materials
The following are available online at https://www.mdpi.com/article/10.3390/pathogens10070840/s1, Figure S1: Arbovirus diagnosis algorithm. Table S1. Contingency table with the RVNT assays diagnostics predictive values. All RVNT assays were evaluated in comparison to the reference assay PRNT.
Author Contributions
Conceptualization, D.B.A.M., P.-Y.S., R.M.R.B., and P.F.C.V.; methodology, J.G.C.N., B.T.D.N., C.S., T.R.S., L.L.d.C., F.A.e.S., E.V.P.d.S., and D.F.H.; software, J.G.C.N, and L.M.N.C.; validation, J.G.C.N. and T.R.S.; formal analysis, J.G.C.N., R.S.S.A., L.C.M., L.M.N.C., J.O.C., D.F.H., P.F.C.V., R.M.R.B., and D.B.A.M.; investigation, A.F.M. and M.H.R.d.M,; resources, P.F.C.V., P.-Y.S.; data curation, J.G.C.N., T.R.S., C.S., L.M.N.C., D.F.H. and D.B.A.M.; writing—original draft preparation, J.G.C.N. and D.B.A.M.; writing—review and editing, J.G.C.N., B.T.D.N., and D.B.A.M.; visualization, J.G.C.N. and T.R.S.; supervision, P.F.C.V., R.M.R.B., P.-Y.S. and D.B.A.M.; project administration, P.-Y.S. and D.B.A.M.; funding acquisition, P.F.C.V. and P.-Y.S. All authors have read and agreed to the published version of the manuscript.
Funding
P.F.C.V. was supported by the Ministry of Health of Brazil and by the grants from CNPq (process 303999/2016-0 and 440405/2016-5) and CAPES (Zika fast track project). P.-Y.S. was supported by a grant from Pan American Health Organization SCON2016-01353, NIH grants AI142759, AI127744, and AI136126, and awards from the Kleberg Foundation, John S. Dunn Foundation, Amon G. Carter Foundation, Gilson Longenbaugh Foundation, and Summerfield Robert Foundation.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Ethics Committee of INSTITUTO EVANDRO CHAGAS (protocol code 3.601.702).
Informed Consent Statement
Patient consent was waived by the Institutional Ethics Committee of INSTITUTO EVANDRO CHAGAS (IEC) due to the fact that all the samples used in this study belongs to the institutional sample collection end were sent from all across the country for the diagnostic of arbovirus, rendering impossible to track down all patients to request their consent. All samples are stored in the IEC bio-bank and were made available for this study with the authorization of the institution high administration.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Dick G.W., Kitchen S.F., Haddow A.J. ZIKA virus. I. Isolations and serological specificity. Trans. R. Soc. Trop. Med. Hyg. 1952;46:509–520. doi: 10.1016/0035-9203(52)90042-4. [DOI] [PubMed] [Google Scholar]
- 2.Marchette N.J., Garcia R., Rudnick A. Isolation of ZIKA virus from Aedes aegypti mosquitoes in Malaysia. Am. J. Trop. Med. Hyg. 1969;18:411–415. doi: 10.4269/ajtmh.1969.18.411. [DOI] [PubMed] [Google Scholar]
- 3.Marcondes C.B., Ximenes M.F. ZIKA virus in Brazil and the danger of infestation by Aedes (Stegomyia) mosquitoes. Rev. Soc. Bras. Med. Trop. 2016;49:4–10. doi: 10.1590/0037-8682-0220-2015. [DOI] [PubMed] [Google Scholar]
- 4.Basarab M., Bowman C., Aarons E.J., Cropley I. ZIKA virus. BMJ. 2016;352:i1049. doi: 10.1136/bmj.i1049. [DOI] [PubMed] [Google Scholar]
- 5.Haby M.M., Pinart M., Elias V., Reveiz L. Prevalence of asymptomatic ZIKA virus infection: A systematic review. Bull. World Health Organ. 2018;96:402–413. doi: 10.2471/BLT.17.201541. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Saiz J., Vasquez-Calvo A., Blazquez A., Merino-Ramos T., Escribano-Romero E., Martín-Acebes M.A. ZIKA virus: The latest newcomer. Front. Microbiol. 2016;7:496. doi: 10.3389/fmicb.2016.00496. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Calvez E., O’Connor O., Pol M., Rousset D., Faye O., Richard V., Tarantola A., Dupont-Rouzeyrol M. Differential transmission of Asian and African ZIKA virus lineages by Aedes aegypti from New Caledonia. Emerg. Microbes Infect. 2018;7:1–10. doi: 10.1038/s41426-018-0166-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hu T., Li J., Carr M.J., Duchene S., Shi W. The Asian Lineage of Zika Virus: Transmission and Evolution in Asia and the Americas. Virol. Sin. 2019;34:1–8. doi: 10.1007/s12250-018-0078-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Haddow A.D., Nasar F., Guzman H., Ponlawat A., Jarman R.G., Tesh R.B., Scott C.W. Genetic Characterization of Spondweni and ZIKA Viruses and Susceptibility of Geographically Distinct Strains of Aedes aegypti, Aedes albopictus and Culex quinquefasciatus (Diptera: Culicidae) to Spondweni Virus. PLoS Negl. Trop. Dis. 2016;10:e0005083. doi: 10.1371/journal.pntd.0005083. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Aliota M.T., Bassit L., Bradrick S.S., Cox B., Garcia-Blanco M.A., Gavegnano C., Friedrich T.C., Golos T.G., Griffin D.E., Haddow A.D., et al. ZIKA in the Americas, year 2: What have we learned? What gaps remain? A report from the Global Virus Network. Antivir. Res. 2017;144:223–246. doi: 10.1016/j.antiviral.2017.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Liu Z.Y., Shi W.F., Qin C.F. The evolution of ZIKA virus from Asia to the Americas. Nat. Rev. Microbiol. 2019;17:131–139. doi: 10.1038/s41579-018-0134-9. [DOI] [PubMed] [Google Scholar]
- 12.Musso D., Gubler D.J. ZIKA Virus. Clin. Microbiol. Rev. 2016;29:487–524. doi: 10.1128/CMR.00072-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.De Oliveira C.S., Vasconcelos P.F.C. Microcephaly and ZIKA virus. J. Pediatr. 2016;92:103–105. doi: 10.1016/j.jped.2016.02.003. [DOI] [PubMed] [Google Scholar]
- 14.Azevedo R.S.S., De Sousa J.R., Araujo M.T.F., Martins Filho A.J., De Alcantara B.N., Araujo F.M.C., Queiroz M.G.L., Cruz A.C.R., Vasconcelos B.H.B., Chiang J.O., et al. In situ immune response and mechanisms of cell damage in central nervous system of fatal cases microcephaly by ZIKA virus. Sci. Rep. 2018;8:1. doi: 10.1038/s41598-017-17765-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Alcantara B.N., Imbeloni A.A., Durans D.B.S., Araújo M.T.F., Cruz E.R.M., Carvalho C.A.M., Mendonça M.H.R., Sousa J.R., Moraes A.F., Martins Filho A.J., et al. Histopathological lesions of congenital ZIKA syndrome in newborn squirrel monkeys. Sci Rep. 2021;11:6099. doi: 10.1038/s41598-021-85571-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kobres P.Y., Chretien J.P., Johansson M.A., Morgan J.J., Whung P.Y., Mukundan H., Dell Valle S.Y., Forshey B.M., Quandelacy T.M., Biggerstaff M., et al. A systematic review and evaluation of ZIKA virus forecasting and prediction research during a public health emergency of international concern. PLoS Negl. Trop. Dis. 2019;13:e0007451. doi: 10.1371/journal.pntd.0007451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Sharp T.M., Fischer M., Muñoz-Jordán J.L., Paz-Bailey G., Staples J.E., Gregory C.J., Waterman S.H. Dengue and ZIKA Virus Diagnostic Testing for Patients with a Clinically Compatible Illness and Risk for Infection with Both Viruses. MMWR Recomm. Rep. 2019;68:1–10. doi: 10.15585/mmwr.rr6801a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Panning M. ZIKA Virus Serology: More Diagnostic Targets, more Reliable Answers? EBioMedicine. 2017;16:12–13. doi: 10.1016/j.ebiom.2017.01.045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lanciotti R.S., Kosoy O.L., Laven J.J., Velez J.O., Lambert A.J., Johnson A.J., Stanfield S.M., Duffy M.R. Genetic and serologic properties of ZIKA virus associated with an epidemic, Yap State, Micronesia, 2007. Emerg. Infect. Dis. 2008;14:1232–1239. doi: 10.3201/eid1408.080287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Shan C., Muruato A.E., Nunes B.T.D., Luo H., Xie X., Medeiros D.B.A., Wakamiya M., Tesh R.B., Barrett A.D., Wang T., et al. A live-attenuated ZIKA virus vaccine candidate induces sterilizing immunity in mouse models. Nat. Med. 2017;23:763–767. doi: 10.1038/nm.4322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Shan C., Muruato A.E., Jagger B.W., Richner J., Nunes B.T.D., Medeiros D.B.A., Xie X., Nunes J.G.C., Morabito K.M., Kong W.P., et al. A single-dose live-attenuated vaccine prevents ZIKA virus pregnancy transmission and testis damage. Nat. Commun. 2017;8:676. doi: 10.1038/s41467-017-00737-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Malafa S., Medits I., Aberle J.H., Aberle S.W., Haslwanter D., Tsouchnikas G., Wolfe S., Huber K.L., Percivalle E., Cherpillod P., et al. Impact of flavivirus vaccine-induced immunity on primary ZIKA virus antibody response in humans. PLoS Negl. Trop. Dis. 2020;14:e0008034. doi: 10.1371/journal.pntd.0008034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Shan C., Xie X., Ren P., Loeffelholz M.J., Yang Y., Furuya A., Dupuis A.P., Kramer L.D., Wong S.J., Shi P.Y. A Rapid ZIKA Diagnostic Assay to Measure Neutralizing Antibodies in Patients. EBioMedicine. 2017;17:157–162. doi: 10.1016/j.ebiom.2017.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ministério da Saúde Monitoramento dos casos de dengue, febre de chikungunya e febre pelo vírus ZIKA até a Semana Epidemiológica 10, 2017. Bol. Epidemiológico Ministério Saúde. 2017;48:1–10. [Google Scholar]
- 25.Salles T.S., Sá-Guimarães T.E., de Alvarenga E.S.L., Guimarães-Ribeiro V., de Meneses M.D.F., de Castro-Salles P.F., Dos Santos C.R., Melo A.C.A., Soares M.R., Ferreira D.F., et al. History, epidemiology and diagnostics of dengue in the American and Brazilian contexts: A review. Parasites Vectors. 2018;11:264. doi: 10.1186/s13071-018-2830-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Brazil Reemergência da Febre Amarela Silvestre no Brasil, 2014/2015: Situação epidemiológica e a importância da vacinação preventiva e da vigilância intensificada no período sazonal. Bol. Epidemiológico Ministério Saúde. 2015;46:1–10. [Google Scholar]
- 27.Couto-Lima D., Madec Y., Bersot M.I., Campos S.S., de Albuquerque Motta M., Dos Santos F.B., Vazeille M., da Costa Vasconcelos P.F., Lourenço-de-Oliveira R., Failloux A.B. Potencial risk of re-emergence of urban transmission of Yellow Fever virus in Brazil facilitated by competent Aedes populations. Sci. Rep. 2017;7:4848. doi: 10.1038/s41598-017-05186-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Faria N.R., Azevedo R.D.S.D.S., Kraemer M.U.G., Souza R., Cunha M.S., Hill S.C., Theze J., Bonsall M.B., Bowden T.A., Rissanen I., et al. ZIKA virus in the Americas: Early epidemiological and genetic findings. Science. 2016;352:345–349. doi: 10.1126/science.aaf5036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Vicente C.R., Cerutti C., Jr., Froschl G., Romano C.M., Cabidelle A.S.A., Herbinger K.-H. Influence of demographics on clinical outcome of dengue: A cross-sectional study of 6703 confirmed cases in Vitória, Espírito Santo State, Brazil. Epidemiol. Infect. 2016;145:46–53. doi: 10.1017/S0950268816002004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Azevedo R.S.S., Araujo M.T., Filho A.J.M., Oliveira C.S., Nunes B.T.D., Cruz A.C.R., Nascimento A.G.P.A.C., Medeiros R.C., Caldas C.A.M., Araujo F.C., et al. Zika virus epidemic in Brazil. I. Fatal disease in adults: Clinical and laboratorial aspects. J. Clin. Virol. 2016;85:56–64. doi: 10.1016/j.jcv.2016.10.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Nóbrega M.E.B., Araújo E.L.L., Wada M.Y., Leite P.L., Dimech G.S., Pércio J. Outbreak of Guillain-Barré syndrome possibly related to prior Zika virus infection, Metropolitan Region of Recife, Pernambuco, Brazil, 2015. Epidemiol. Serviços Saúde. 2017;27 doi: 10.5123/S1679-49742018000200016. [DOI] [PubMed] [Google Scholar]
- 32.Teixeira M.G., Alcina M.S., Andrade M.C.N., Costa J.S.M., Castro F.L.S., Oliveira C.S.B., Goes M.M., Santana E.B., Nunes B.T.D., Vasconcelos P.F.C. East/Central/South African Genotype Chikungunya Virus, Brazil, 2014. Emerg. Infect. Dis. 2015;21:906. doi: 10.3201/eid2105.141727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Aragão C.F., Cruz A.C.R., Neto J.P.N., Monteiro H.A.O., Silva E.V.P., Silva S.P., Andrade A.T.S., Tadei W.P., Pinheiro V.C.S. Circulation of Chikungunya virus in Aedes aegypti in Maranhão, Northeast Brazil. Acta Trop. 2018;186:1–4. doi: 10.1016/j.actatropica.2018.06.022. [DOI] [PubMed] [Google Scholar]
- 34.Zanluca C., Melo V.C.A., Mosimann A.L.P., Santos G.I.V., dos Santos C.N.D., Luz K. First report of autochthonous transmission of Zika virus in Brazil. Memórias Inst. Oswaldo Cruz. 2015;110:569–572. doi: 10.1590/0074-02760150192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Cordeiro M.T. Postnatal Imaging Findings of Congenital Zika Syndrome. Top. Magn. Reson. Imaging. 2019;28:15–17. doi: 10.1097/RMR.0000000000000190. [DOI] [PubMed] [Google Scholar]
- 36.Roehring J.T., Nash D., Maldin B., Labowitz A., Martin D.A., Lanciotti R.S., Campbell G.L. Persistence of virus-reactive serum immunoglobulin M antibody in confirmed West Nile encephalitis cases. Emerg. Infect. Dis. 2003;9:376–379. doi: 10.3201/eid0903.020531. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cordeiro M.T., Brito C.A., Pena L.J., Pena L.J., Castanha P.M.S., Gil L.H.V.G., Lopes K.G.S., Dhalia R., Meneses J.A., Ishigami A.C., et al. Results of a Zika virus (ZIKA) immunoglobulin M–specific diagnostic assay are highly correlated with detection of neutralizing anti-ZIKA antibodies in neonates with congenital disease. J. Infect. Dis. 2016;214:1897–1904. doi: 10.1093/infdis/jiw477. [DOI] [PubMed] [Google Scholar]
- 38.Araújo V.E.M., Bezerra J.M.T., Amâncio F.F., Passos V.M.d.A., Carneiro M. Increase in the burden of dengue in Brazil and federated units, 2000 and 2015: Analysis of the Global Burden of Disease Study 2015. Rev. Bras. Epidemiol. 2015;20:205–216. doi: 10.1590/1980-5497201700050017. [DOI] [PubMed] [Google Scholar]
- 39.Grifoni A., Pham J., Sidney J., O’Rourke P.H., Paul S., Peters B., Martini S.R., de Silva A.D., Ricciardi M.J., Magnani D.M., et al. Prior Dengue Virus Exposure Shapes T Cell Immunity to Zika Virus in Humans. J. Virol. 2017;91:e01469-17. doi: 10.1128/JVI.01469-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Shope R.E. The use of micro-hemagglutinaation-inhibition test to follow antibody response after arthropod-borne virus infection in a community of forest animais. An. Microbiol. 1963;11:167–171. [Google Scholar]
- 41.CDC Guidance for U.S. Laboratories Testing for Zika Virus Infection. [(accessed on 16 November 2016)]; Available online: https://www.cdc.gov/zika/pdfs/laboratory-guidance-zika.pdf.
- 42.Shan C., Xie X., Muruato A.E., Rossi S.L., Roundy C.M., Azar S.R., Yang Y., Tesh R.B., Bourne N., Barrett A.D., et al. An infectious cDNA clone of Zika virus to study viral virulence, mosquito transmission, and antiviral inhibitors. Cell Host Microbe. 2016;19:891–900. doi: 10.1016/j.chom.2016.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Zou G., Xu H.Y., Qing M., Wang Q.Y., Shi P.Y. Development and characterization of a stable luciferase dengue virus for high-throughput screening. Antivir. Res. 2011;91:11–19. doi: 10.1016/j.antiviral.2011.05.001. [DOI] [PubMed] [Google Scholar]
- 44.Castanha P.M., Cordeiro M.T., Martelli C.M., Souza W.V., Marques E.T., Jr., Braga C. Force of infection of dengue serotypes in a population-based study in the northeast of Brazil. Epidemiol. Infect. 2013;141:1080–1088. doi: 10.1017/S0950268812001367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Rabe I.B., Staples J.E., Villanueva J., Hummel K.B., Johnson J.A., Rose L., Hills S., Wasley A., Fischer M., Powers A.M. Interim Guidance for Interpretation of Zika Virus Antibody Test Results. Morb. Mortal. Wkly. Rep. 2016;65:543–546. doi: 10.15585/mmwr.mm6521e1. [DOI] [PubMed] [Google Scholar]
- 46.Ayres M., Jr., Ayres D.L., Santos A.A.S. BioEstat 5.3: Aplicações Estatísticas nas Áreas das Ciências Biológicas e Médicas. 5th ed. Publicações Avulsas do Mamirauá; Belém, Brazil: 2007. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.


