Evaluation of an Epitope-Blocking Enzyme-Linked Immunosorbent Assay for the Diagnosis of West Nile Virus Infections in Humans

Abstract

An epitope-blocking enzyme-linked immunosorbent assay (b-ELISA) was evaluated for the diagnosis of West Nile virus (WNV) infections in humans. Sera from patients diagnosed with WNV infections from an outbreak in 2003 in Colorado and from patients diagnosed with dengue virus infections from Mexico and Thailand were tested with the b-ELISA. The b-ELISAs were performed using the WNV-specific monoclonal antibody (MAb) 3.1112G and the flavivirus-specific MAb 6B6C-1. Although the WNV-specific b-ELISA was effective in diagnosing WNV infections in humans from Colorado, it was not efficacious for diagnosing WNV infections in serum specimens from Mexico and Thailand. In serum specimens from patients from Colorado, the WNV b-ELISA and the WNV plaque reduction neutralization test showed an overall agreement of 91%. The sensitivity and specificity of the WNV b-ELISA were 89% and 92%, respectively, with a false-positive rate of 5%, based on receiver operating characteristic analysis. In contrast, false-positive rate results in specimens from the countries of Mexico and Thailand, where flaviviruses are endemic, were 79% and 80%, presumably due to the presence of antibodies resulting from previous dengue virus infections in Mexico and/or Japanese encephalitis virus infections or vaccination in Thailand. Thus, in regions where people have experienced previous or multiple flavivirus infections, the use of the b-ELISA for WNV diagnosis is contraindicated.

The most medically important flaviviruses include dengue virus (DENV), Japanese encephalitis virus (JEV), West Nile virus (WNV), yellow fever virus (YFV), tick-borne encephalitis virus (TBEV), and Saint Louis encephalitis virus (SLEV) (16, 31, 38). Flaviviruses are positive-strand RNA viruses with genomes of approximately 11 kb that encode three structural and seven nonstructural (NS) proteins in the gene order C (capsid), M (membrane), E (envelope), NS1, NS2A, NS2B, NS3, NS4A, NS4B, and NS5. WNV is a member of the JEV serocomplex within the genus Flavivirus, family Flaviviridae. The virus has been isolated in Africa, Australia, Eastern Europe, the Middle East, North America, and South America (7, 20, 24). WNV was first detected in the United States in July 1999 and spread rapidly throughout the country, causing large numbers of infections in humans, horses, and birds (19, 31).

Prior to 1999, flavivirus infections in humans in the United States were infrequent, and most were attributed to sporadic cases of SLEV and travel-associated cases of DENV (41). In Thailand, all four DENV serotypes and JEV circulate (39), resulting in very high flavivirus transmission and seroprevalence rates. In the Yucatán Peninsula of Mexico, all four DENV serotypes circulate and seroprevalence rates are very high (8). Serological diagnosis of WNV infections is complicated by the high rates of both primary DENV infections and secondary DENV infections in inhabitants of Thailand and Yucatan, Mexico, with seroprevalence rates of >85% in Thailand (1) and 72% in the Yucatán in 1985 (12, 28). WNV introduction into the Yucatán in 2002 was revealed by detection of antibodies in horses (29) and then later in migratory and resident birds (10) and in zoo animals (11). However, no WNV infections of humans have been diagnosed in the Yucatán.

The immunoglobulin M (IgM) capture enzyme-linked immunosorbent assay (ELISA) is the preferred test used for diagnosis of WNV in humans in the United States (32). The test is used to detect antibodies to WNV in serum and/or cerebrospinal fluid. The plaque reduction neutralization test (PRNT) is the gold standard for serodiagnosis of flavivirus infections and for identifying the infecting agent (2). However, both of these tests can be confounded if patients have had previous flavivirus infections. Indeed, diagnosis of flavivirus infections in humans is very difficult in geographic areas where multiple flaviviruses are circulating and cause sequential infections. Because of “original antigenic sin” the highest antibody titer may be due to a previous flavivirus infection rather than to the etiologic agent (18, 26). Serological diagnosis of WNV, SLEV, and YFV infections is extremely difficult in patients from areas where DENV is hyperendemic.

Previously, we exploited an epitope-blocking ELISA (b-ELISA) to detect antibodies to WNV in diverse species of birds and domestic mammals (3, 4). The WNV b-ELISA measures the ability of antibodies present in sera to block the binding of a monoclonal antibody (MAb) to a WNV-specific epitope on the NS1 protein (17). The WNV b-ELISA had not been previously evaluated for use in humans. In this study, a WNV-specific and a flavivirus broadly reactive b-ELISAs were evaluated for their abilities to detect antibodies against WNV in human serum specimens from countries with differing levels of flavivirus endemicity: the United States, Thailand, and Mexico. The objectives of this study were (i) to determine the ability of the b-ELISA to detect antibodies to WNV in human serum samples and (ii) to determine the effects of previous flavivirus infections of patients (e.g., DENV and JEV) on the diagnostic efficacy of the WNV b-ELISA.

MATERIALS AND METHODS

Serum samples: description and preliminary characterization.

A total of 725 serum samples from patients diagnosed with WNV infections from an outbreak in 2003 in Colorado (6) and from dengue patients from Thailand and Mexico were obtained for this study. General information about the patients was obtained from collaborating laboratories in the United States, Thailand, and Mexico (Table 1). Most of the serum samples obtained from Thailand were from children of an average age of 8 years old. The Colorado samples were mainly from middle-aged adults, averaging 50 years old. A broader age range was found among the patients from Yucatán, with a mean age of 29 years. All serum samples were heat treated (56°C for 30 min) before testing.

TABLE 1.

History and information for 725 laboratory-confirmed serum samples from patients infected with WNV or DENV or without evidence of flavivirus infection

Area	Institutiona	Laboratory techniques	Yr of collection	Age range (yrs)	Mean age (yrs)	No. of samples tested	No. positive for WNV	No. positive for DENV
Colorado, United States	CDPHE	IgM-ELISA for WNV and PRNT for WNV	2003	6-95	50	366	292	0
Bangkok, Thailand	AFRIMS	IgM and IgG ELISA for DENV and JEV, RT-PCR for DENV and PRNT for DENV	2001	2-28	8	195	0c	157
Yucatán, México	UADY	IgM ELISA for DENV, PRNT for DENV, and RT-PCR for DENV and/or virus isolationb	1997-2001	2-67	29	19	0	16
Yucatán, México	UADY	IgM ELISA for DENV, PRNT for DENV, and RT-PCR for DENV and/or virus isolationb	2002-2005	0-85	29	145	0	34

^aCDPHE, Colorado Department of Public Health and Environment; AFRIMS, Armed Forces Research Institute of Medical Sciences; UADY, Universidad Autónoma de Yucatán.

^bSamples from 1997 to 2001 were from before the introduction of WNV into the Yucatan; samples from 2002 to 2005 were from after the introduction of WNV.

^cValues of 0 indicate that samples were not tested for all DENV serotypes.

A total of 366 serum specimens (292 positive and 74 negative) were obtained from patients who presented with West Nile fever symptoms during the 2003 outbreak in Colorado (Table 1). All samples had been previously analyzed by WNV IgM ELISA (6, 32) at the Colorado Department of Public Health and Environment and by the PRNT for WNV in biosafety level 3 (BSL-3) facilities at Colorado State University (CSU). The WNV positive controls were sera that were positive in both tests; the negative control samples were only sera that were negative in both tests.

A total of 195 serum samples were obtained from Thai children with suspected DENV infections (157 positive and 38 negative). Sera were previously tested by DENV IgM and IgG capture ELISA (23) and reverse transcription-PCR (27) at the Armed Forces Research Institute of Medical Sciences (AFRIMS) in Bangkok, Thailand, and by PRNT for DENV in the BSL-3 facilities at CSU (Table 1).

A total of 164 serum samples from patients from Mexico with suspected dengue virus infections were included in this study. Among these 164 Mexican serum samples, 50 cases were diagnosed as dengue positive by IgM capture ELISA, reverse transcription-PCR, and/or virus isolation, and the remaining 114 samples were classified as negative. For the b-ELISA analysis, 19 samples were included from serum collections between 1997 and 2001, before the introduction of WNV into Mexico. Of these, 16 samples were confirmed as dengue positive. An additional 145 serum samples were collected between 2002 and 2005, after the introduction of WNV in Mexico, and 34 of the samples were confirmed as dengue virus positive (Table 1). WNV, DENV, and SLEV IgM capture ELISAs were also conducted at CSU to confirm diagnostic results for the Mexican samples. It is important to highlight that the first evidence of WNV transmission in the Yucatán Peninsula was the detection of antibodies to WNV in horses in mid-2002 (29).

As described in the previous section, DENV infections were diagnosed by AFRIMS in 80.5% (157/195) of serum specimens from Thailand, and the Yucatán laboratory confirmed DENV infections in 30.5% (50/164) of serum specimens from Mexico. When tested by the laboratories in Thailand and Mexico, antibodies to flaviviruses were detected in 98.5% (193/195) and 86.6% (142/164) of the Thai and Mexican samples, respectively, by using IgG ELISA at AFRIMS and PRNT at CSU for the Thai samples and PRNT for DENV at CSU for the Mexican samples. These percentages confirm the high prevalence of antibodies to flaviviruses among people from Thailand and Mexico. Collection and testing of human sera were approved by the institutional ethical review boards of the participating institutions.

Preparation of b-ELISA antigen.

The antigen used for the WNV b-ELISA was prepared from Aedes albopictus C6/36 cells that had been infected with WNV (NY-99 strain) at a multiplicity of infection of 0.1 (3). At 120 h postinfection, the cells were scraped from the flask and pelleted by centrifugation at 4,000 rpm for 10 min at 4°C. Cell pellets were washed four times with borate saline (1.5 M NaCl, 0.5 M H₃BO₂, 1.0 M NaOH, pH 9.0) and the final pellet was resuspended in 0.1% sodium dodecyl sulfate and 1% Triton X-100. The cells were sonicated on ice at a 20% output setting for 30 s and centrifuged at 8,000 rpm for 10 min at 4°C. Supernatants were aliquoted and stored at −70°C until use.

b-ELISA.

The b-ELISA for detection of antibodies to WNV was performed using MAb 3.1112G (Chemicon International, Inc., Temecula, CA) and horseradish peroxidase-labeled rabbit anti-mouse IgG (catalog no. 61-6520; Zymed Laboratories). The b-ELISA for detection of antibodies to flaviviruses was performed using horseradish peroxidase-labeled MAb 6B6C-1 (CDC, Fort Collins, CO) using the methods and protocols of Hall et al. (17) as modified by Blitvich et al. (3). MAb 3.1112G is specific for the NS1 glycoprotein of WNV (17). MAb 6B6C-1 is specific for the flavivirus E protein (22, 36). Briefly, coating antigen, conjugated antibodies, and monoclonal antibodies were independently titrated against negative and positive control serum samples. The internal 60 wells of an ELISA plate (96-well plate) were coated with WNV antigen produced in C6/36 cells and diluted in coating buffer (carbonate-bicarbonate buffer, pH 9.6) and incubated overnight at 4°C. Plates were washed with washing buffer five times. Blocking buffer (phosphate-buffered saline, 0.1% Tween 20, and 5% nonfat dry milk) was added for 40 min at 37°C. Plates were washed with washing buffer. Test sera and positive and negative serum controls were diluted 1:10 in blocking buffer and incubated for 2 h at 37°C. Wells were then washed five times with washing buffer. MAb diluted in blocking buffer was added and incubated for 1 h at 37°C. After washing, for the WNV b-ELISA horseradish peroxidase-labeled rabbit anti-mouse IgG was added for 1 hour at 37°C, and for the flavivirus b-ELISA the MAb added was horseradish peroxidase-conjugated 6B6C-1. After washing, 2,2′-azinobis(3-ethylbenzthiazolinesulfonic acid) developing solution was added and incubated at 37°C. Optical densities (ODs) were measured at 415 nm at different intervals.

Optimal dilutions that yielded an A₄₁₅ of 0.30 with negative controls were chosen. Percentages of inhibition of MAb binding in the b-ELISA were calculated using the following formula: percent inhibition = 100 × [1 - (OD of sample − OD of background)/(OD of negative control - OD of background)], where the value for the OD of the negative control was the average optical density of the six negative controls (17).

Optical density data collected were entered into Microsoft Excel (Microsoft, Bellingham, WA), and the percent inhibition was calculated using the above formula. A selected pool of 50 human serum samples from Colorado was used as the negative control sera. These samples tested negative for the presence of antibodies to WNV, SLEV, and DENV by PRNT and by IgM ELISA.

IgM capture ELISA.

Serum samples from Mexican patients were assayed for the presence of IgM antibodies to WNV, SLEV, and DENV as described elsewhere (32). Briefly, plates were coated with antibodies to human IgM, incubated overnight at 4°C, and blocked with phosphate-buffered saline containing 0.5% Tween 20 and 5% nonfat dry milk. Serum samples were diluted 1:400 in washing buffer and added to the wells. Plates were incubated in a humidified chamber for 1 h at 37°C. Positive antigens (cell lysates infected with WNV, SLEV, or DENV) and negative antigens (noninfected cell lysates) were added to the samples and incubated overnight at 4°C. The following day, peroxidase-labeled MAb 6B6C-1 was added. After the addition of the MAb, wells were washed 10 times with washing buffer to reduce background. The conjugate was detected using enhanced K-blue substrate, which contains both 3,3′,5,5′-tetramethylbenzidine and hydrogen peroxide. Readings of the spectrophotometric absorbance values were conducted at A₄₅₀. The positive and negative antigens used for the test were the following: (i) recombinant WNV envelope protein expressed in COS-1 cells; (ii) normal COS-1 cells; (iii) suckling mouse brain infected with SLEV; (iv) normal control suckling mouse brain; (v) pooled tissue culture supernatants of C6/36 cells infected with each of the four dengue virus serotypes; (vi) tissue culture supernatant of noninfected C6/36 cells. A panel of negative and positive control serum samples used for proficiency testing by San Juan Laboratories (CDC) was included in the test.

Plaque reduction neutralization test.

Serum samples were tested by PRNT to detect the presence of neutralizing antibodies to WNV, SLEV, and DENV (2). PRNTs were conducted using Vero cells in the BSL-3 facilities at CSU. The following prototype virus strains were used in this test: WNV (NY-99 35261-11), DENV-1 (Hawaii), DENV-2 (Jamaica 1409), DENV-3 (H-87), DENV-4 (H-241), and SLEV (TBH-28). Neutralizing antibody titers were calculated as the reciprocal of the serum dilution with a 90% reduction of the number of plaques (PRNT₉₀). Twofold dilutions of serum samples were tested starting at a 1:10 dilution. Primary and secondary infections were assigned based on PRNT titers and criteria for IgM and IgG capture ELISA titers in Thai samples (9, 42).

Diagnostic cutoff values in ELISAs.

Different diagnostic cutoff values were established by (i) measuring values using sera from noninfected individuals and adding 2 or 3 standard deviations to the mean negative control value, (ii) using receiver operating characteristic (ROC) analysis to determine the cutoff value that best differentiated negative and positive groups based on the value obtained in the area under the curve (15, 21, 33), and (iii) dot plot analysis (25) (MedCalc for Windows, version 8.2.). Serum samples from 366 (292 WNV-positive and 74 WNV-negative) patients presenting with WNV-like symptoms in 2003 were used in the ROC and dot plot analyses to determine the optimized cutoff value (13, 14). The optimized cutoff value maximizes both the diagnostic sensitivity and the diagnostic specificity and minimizes both false-positive and false-negative rates of the b-ELISA. For the ROC analyses, a complete sensitivity/specificity report was prepared using the MedCalc program (data not shown), and the ROC curve was calculated using the percentage of blocking from the 292 WNV-positive and the 74 WNV-negative samples.

The ROC curve analysis (30) indicated that 8% blocking was the optimal cutoff value for the WNV b-ELISA using monoclonal antibody 3.1112G and provided maximum diagnostic sensitivity and specificity. With 8% blocking, the area under the curve was 0.962, meaning that 96% of the individuals from the WNV-positive group had test results that were greater than the individuals from the WNV-negative group. Since the ROC analysis is not necessarily the definitive method for final determination of cutoff values, dot plot results were also used to confirm the ROC analyses cutoff values (30). To construct the dot plots, the data sets of the negative and positive groups were plotted as separate groups of data points and are presented as vertical groups of dots in a dot diagram. A horizontal line was drawn to indicate the cutoff point that provided the best separation between the two groups. The sensitivity and specificity corresponding to the test were indicated at the right side of the dot plot. Applying the ROC-determined cutoff values to dot plots also gave excellent dot plot sensitivity and selectivity results, confirming the ROC values. The ROC and dot plot analyses indicated that a 13% blocking cutoff value was optimal for the flavivirus b-ELISA when using monoclonal antibody 6B6C-1. A 30% cutoff value, established from other animal studies, was also evaluated in this study.

Statistical software packages used for the analysis.

The ROC curve analysis and cutoff values, sensitivity, specificity, positive predictive value, negative predictive value, and other statistical analyses were determined using the following programs: MedCalc for Windows, version 8.2 (Mariakerke, Belgium) and Diagnostic Agreement Statistics (www.mhri.edu.au/biostats/DAG_Stat/). The SAS program, version 9.1 (Cary, NC) was used for logistic regression analysis on primary and secondary infections. Significance was defined as a P value of <0.05. Microsoft Office Excel 2003 was used to calculate the mean, the variance, and 2 and 3 SD values for the negative control samples.

RESULTS

Determining the diagnostic efficacy of the b-ELISA using serum specimens from Colorado.

The 366 human serum specimens from patients presenting with WNV-like symptoms (292 WNV positive and 74 WNV negative) were used to determine overall agreement and sensitivities and specificities of the b-ELISA, based on different diagnostic criteria and PRNT results. To determine the best cutoff values for the b-ELISAs using the WNV-specific MAb 3.1112G and the flavivirus-specific MAb 6B6C-1, three diagnostic criteria were compared. These were the following: (i) the mean result plus 2 SD for the negative control serum samples based on optical densities obtained with the b-ELISA; (ii) the mean results plus 3 SD for the negative control serum specimens based on the optical densities with the b-ELISA; (iii) the 8% and 13% blocking values determined by ROC analyses described above. The human sera from patients diagnosed as WNV positive or negative from the 2003 WNV outbreak in Colorado in 2003 were tested using all three diagnostic criteria (Table 2). The overall agreement of b-ELISA results with the gold standard test for flaviviruses and the sensitivities and specificities of each diagnostic criterion are summarized in Table 2.

TABLE 2.

Sensitivity and specificity of the b-ELISA using different diagnostic criteria^a

MAbb	Method of calculation of cutoff valuesc	Cutoff valued	% Overall agreement with PRNT90	Sensitivity (%)	Specificity (%)
3.1112G	2 SD	0.278	79	73	100
	3 SD	0.245	70	63	95
	ROC	8%	91	89	92
	ROC	30%	70	63	95
6B6C-1	2 SD	0.254	70	63	97
	3 SD	0.192	56	45	100
	ROC	13%	79	75	93
	ROC	30%	64	57	97

^aDiagnostic results using the b-ELISA for the detection of antibodies to WNV and other flaviviruses are shown for 366 human serum samples from Colorado. Results were compared with PRNT₉₀ values, the gold standard for WNV diagnosis.

^bThe MAb 3.1112G is WNV specific and the MAb 6B6C-1 is flavivirus specific.

^cCutoff based on 2 standard deviations for the noninfected reference population based on optical densities obtained with the b-ELISA; ROC, based on receiver operating characteristic curve analysis.

^dCutoff values were determined by the different methods described above and are shown as percentages for the ROC analyses.

The ROC analyses consistently determined cutoff values that gave the best sensitivities, the best specificities, and the best overall agreement between WNV b-ELISA and PRNT results (Table 2). The optimum ROC analyses cutoff value (8%) produced sensitivities, specificities, and overall agreement with PRNT results at levels above 89%. The 2 SD and 3 SD cutoff values gave unacceptably low sensitivities of just 73% and 63%, respectively (Table 2). The cutoff value of 30% gave unacceptably low overall agreement and sensitivity of just 70% and 62.7% (Table 2). These results demonstrate the superiority of using ROC analyses to determine the optimized cutoff value for the WNV b-ELISA.

ROC analysis also produced a better cutoff value than the other three cutoff value approaches for the flavivirus b-ELISA using MAb 6b6C-1 (Table 2). The ROC analysis-determined 13% cutoff values gave overall agreement with PRNT results that were superior to the other three approaches, but the absolute agreement with PRNT results was just 79%. The ROC analysis also provided the best combination of sensitivity and specificity, yet the sensitivity was quite low at just 75%, corresponding to a 25% false-negative rate (Table 2). Thus, in all subsequent characterizations of the b-ELISA, the ROC 8% cutoff value was used with the WNV-specific MAb 3.1112G and the ROC 13% cutoff value was used with the MAb 6B6C-1.

Cohen's kappa analyses were then used to determine the diagnostic efficacy of the WNV b-ELISA using the 8% cutoff value (Table 3). The agreement of the results obtained with the WNV PRNT₉₀ and IgM capture ELISA was determined. There was a substantial agreement between the WNV b-ELISA and the WNV PRNT₉₀ test; the test kappa value obtained was 0.7473 (95% confidence interval [CI], 0.6666 to 0.8280).

TABLE 3.

Comparisons of serologic tests for diagnosis of WNV infections using sera of patients from Colorado and Cohen's kappa analysis

Comparisona	No. positive in both tests	No. positive in PRNT90 and negative in ELISA	No. positive in ELISA and negative in PRNT90	No. negative in both tests	Cohen's kappa analysis (95% CI)
IgM ELISA vs WNV PRNT90	292	0	0	74	1.0000 (1.0000-1.000)
WNV b-ELISA vs WNV PRNT90	265	27	6	68	0.7473 (0.6666-0.8280)

^aCutoff values for tests: for PRNT₉₀, WNV titer of ≥1:10; for IgM ELISA, positive/negative ratio of ≥3.0; for b-ELISA WNV, 8%.

The IgM ELISA and WNV PRNT results for Colorado serum samples were completely concordant, since there were no false-positive or false-negative results. Using the WNV PRNT₉₀ as the gold standard, the WNV b-ELISA percentages of false-positive and false-negative results were 9.25% and 8.11% (data not shown), corresponding to a sensitivity and specificity of 90.75% and 91.89%, respectively.

Evaluation of WNV b-ELISA using serum samples from Mexico and Thailand.

The purpose of using serum samples from patients with previously confirmed DENV or JEV infections was to determine the ability of the WNV b-ELISA to discriminate new WNV infections from antibodies from prior flavivirus infections. If the antibodies from prior flavivirus infections yield positive results in WNV b-ELISA but there was no WNV infection as determined by PRNT, the gold standard test, then it was considered a false-positive WNV b-ELISA result. While it was a goal of the research to investigate potential interference of antibodies to JEV in the WNV b-ELISA, it was not possible since none of the Thai samples was found to contain JEV antibodies.

When the samples were assayed at CSU by WNV b-ELISA (using a cutoff value of 8%), 81% (156/195) of the specimens from Thailand and 100% (19/19) of the specimens from Mexico collected before 2001, prior to the introduction of WNV, and 76% (110/145) of the samples collected from Mexico between 2002 and 2005 yielded false-positive results for the diagnosis of WNV infections (Table 4). In contrast, the false-positive rate for serum specimens from the United States was only 5% of the known WNV-negative samples. The WNV b-ELISA false positives were identified by comparing the b-ELISA results to both PRNT and IgM ELISA-negative serum results for the Thai, Mexican, and U.S. samples.

TABLE 4.

False positive WNV b-ELISA diagnostic results obtained with WNV or DENV antibody-positive and -negative patients from the United States, Thailand, and Mexico^a

Country of origin of samples	Yr(s) of sample collection	No. of samples classified as WNV negative	No. (%) of false-positive samples for WNV infectionb	False-positive ratec
United States	2003	74	4 (5)	0.054
Thailand	2001	195	156 (80)	0.80
Mexico	1997-2001	19	199 (100)	1.0
Mexico	2002-2005	145	110 (76)	0.758

^aAll serum samples were tested by WNV b-ELISA at AIDL (Colorado State University).

^bThe WNV b-ELISA was conducted using the MAb 3.1112G.

^cThe false-positive rate was based on the proportion of WNV-negative samples that were positive by b-ELISA.

Surveillance of flavivirus infections by b-ELISA with the monoclonal antibodies 3.1112G and 6B6C-1.

In routine surveillance for WNV and other flavivirus infections in animals, b-ELISAs are conducted using two different monoclonal antibodies (3) to either detect antibodies specific for WNV (MAb 3.1112G) or detect antibodies that cross-react between flaviviruses (MAb 6B6C-1). To evaluate the ability of the b-ELISA to detect antibodies to flaviviruses and to WNV in humans in areas that differ in flavivirus endemicity, we conducted ROC analyses with data obtained from patients from the United States, Thailand, and Mexico (Table 5). The ability of the test to distinguish between the positive and negative groups was based on the value obtained in the area under the curve as indicated after the ROC analysis.

TABLE 5.

ROC analysis for serum samples from the United States, Thailand, and Mexico with the b-ELISA using two monoclonal antibodies, the WNV-specific MAb 3.1112G and the flavivirus broadly reactive MAb 6B6C-1

Geographic location and MAb used	Cutoff value (%)	Sensitivity (%)	Specificity (%)	Area under the curvea	Significanceb (P value)
Colorado, United States
3.1112G	8	89	92	0.962	0.0001
6B6C-1	13	75	93	0.903	0.0001
Bangkok, Thailand
3.1112G	30	63	81	0.767	0.0001
6B6C-1	74	89	47	0.685	0.0001
Yucatan, Mexico
3.1112G	28	65	74	0.752	0.0001
6B6C-1	93	76	40	0.549	0.1454

^aWhen the test cannot distinguish between positive and negative results, the area under the curve is equal to 0.5.

^bThe significance level between groups was determined by using P < 0.05 confidence limits.

For the serum samples from Colorado, the WNV-specific and the flavivirus-specific b-ELISAs distinguished between positive and negative groups. The highest area under the curve was 0.962 when using the MAb 3.1112G and was 0.903 when using MAb 6B6C-1. The sensitivity rate was 89% and the specificity rate was 92% with MAb 3.1112G, and for MAb 6B6C-1 the sensitivity and specificity were 75% and 93%, respectively (Table 5).

In contrast, the b-ELISA results were unsatisfactory for the samples from Thailand and Mexico. The areas under the ROC curves ranged from 0.767 to 0.549 (Table 5), compared to >0.9 for the samples from Colorado. Similarly, the sensitivity and specificity of the WNV b-ELISA were 63% and 81% for the Thailand samples and 65% and 74% for the Mexico samples.

The ability of the b-ELISA to distinguish between the positive and negative groups was reflected in the values obtained for the areas under the curves. A value of 1 indicates that the test perfectly distinguishes between positive and negative groups. This evaluation confirmed the previous results, that the b-ELISA methods tested did not work well in the likely presence of antibodies from prior flavivirus infections in humans.

DISCUSSION

The performance of the b-ELISA was evaluated for detection of WNV and flavivirus antibodies in 366 human serum samples from Colorado. The ROC analysis of WNV b-ELISA results determined an optimal cutoff value of 8%, which yielded a sensitivity of 89% and specificity of 92%, corresponding to an 11% false-negative rate and an 8% false-positive rate (Table 2). The ROC analysis provided superior diagnostic cutoff values for the tests when using human specimens, while application of the 30% cutoff value used previously in animal studies yielded an unacceptably high 37% false-negative rate. The ROC analyses' cutoff values yielded better results than both 2 SD and 3 SD statistical approaches to determine cutoff values (Table 2). The reason for the 8% false-positive rate remains to be determined. False-positive results generally arise from anomalies in either the samples or the methodology. The six false-positive results for the Colorado specimens are likely representative and valid, since the sample size was reasonably large (from a group of 68 negatives and 366 total samples). These false-positive WNV b-ELISA results could be explained by the inclusion of specimens from patients previously infected by dengue virus, patients originally from Latin America, or tourists who had been infected when visiting areas where dengue is endemic. There are significant Hispanic immigrant populations in the Denver, Greeley, and ski resort areas of Colorado which likely include dengue-positive individuals. Unfortunately, the sample identification numbers provided by the State of Colorado were coded, and we did not screen for all DENV infections, so we could not assess this possibility. The premise of prior flavivirus infections causing false-positive WNV b-ELISA results is supported by the correspondingly high rates of false-positive WNV b-ELISA results for samples from Mexico and Thailand (Table 5).

In contrast to the WNV b-ELISA results, the ROC analysis applied to flavivirus b-ELISA results for 366 human sera samples from Colorado determined an optimal cutoff value of 13%. Application of this optimal cutoff yielded specificity of 93% and unacceptably low sensitivity of 75%, which corresponds to a 7% false-positive rate and a 25% false-negative rate (Table 2). The reason for this unacceptably high flavivirus b-ELISA false-negative rate with the Colorado human sera samples remains to be determined. The areas under the ROC curves also confirmed this shortcoming; every flavivirus b-ELISA ROC area was less than its corresponding WNV b-ELISA ROC result. Since WNV infections are a subset of flavivirus infections, the flavivirus b-ELISA method should consistently give equivalent or higher numbers of positive cases than WNV b-ELISA testing. The results demonstrated that the flavivirus b-ELISA should not be the test of choice for screening for WNV infections, since it misses too many positive flavivirus cases.

Unfortunately, the WNV b-ELISA cannot be used to accurately diagnose infections in humans in areas where other flaviviruses are endemic due to unacceptably high rates of false-positive test results (Table 5). The seroprevalence rates for flavivirus infections are very high in many regions in the tropics, and secondary flavivirus infections, which induce very high titers of cross-reactive antibodies to flaviviruses, make differentiation of these antigenically closely related viruses even more difficult. In addition, secondary infections can induce antibodies to previous viral infections. Studies of antibody responses in hamsters sequentially infected with WNV, JEV, SLEV, and YFV clearly demonstrated that it is difficult to make serologic diagnosis of WNV infections in animals with preexisting flavivirus immunity (40).

The DENV seroprevalence rates are extremely high in the Yucatán (72%), and secondary infections are also very common (12). We have not detected WNV infection in humans in the Yucatán, despite serological evidence that the virus is circulating there in birds, horses, and other animals (10). The reason for the lack of human infections in the Yucatán is unknown. Human infections (seven cases) have been diagnosed in northern Mexico (www.salud.gob.mx). It is possible that the high DENV seroprevalence rates in the Yucatán protect against WNV infection. Studies conducted in the 1960s suggested that infections with DENV confers cross-protection against WNV and SLEV infections (5, 35, 37).

The WNV b-ELISA readily detects the infecting virus in primary flavivirus infections; however, viruses causing secondary infections are very difficult to identify because of cross-reactions and original antigenic sin. Even in primary infections, b-ELISA-positive samples should be tested using PRNT or complement fixation to confirm the etiologic agent (34, 37).

In summary, the WNV b-ELISA could be used for the diagnosis of WNV infections in humans in areas where only WNV circulates but should not be used in areas where other flaviviruses are circulating or there are significant immigrant populations from areas where flaiviruses are endemic. The PRNT and the IgM ELISA performed better than the WNV b-ELISA in all areas, so they are the serological methods of choice for diagnosing human WNV infections. The WNV b-ELISA continues to be an important tool for WNV surveillance in animals.