Evaluation of a Diagnostic Algorithm Using Immunoglobulin M Enzyme-Linked Immunosorbent Assay To Differentiate Human West Nile Virus and St. Louis Encephalitis Virus Infections during the 2002 West Nile Virus Epidemic in the United States

Abstract

A diagnostic algorithm was developed to differentiate between human infections of West Nile virus (WNV) and St. Louis encephalitis virus (SLEV) using positive-to-negative (P/N) ratios derived from the immunoglobulin M capture enzyme-linked immunosorbent assay (MAC-ELISA). To validate this algorithm, we tested 1,418 serum and cerebrospinal fluid (CSF) samples from confirmed WNV and SLEV infections collected during the WNV epidemic of 2002 in the United States. WNV P/N-to-SLEV P/N ratios (W/S ratios) were calculated and used to identify the infecting virus. These results were compared to results from the plaque reduction neutralization test (PRNT), which is currently the standard assay used to discriminate between closely related flavivirus infections. If the W/S ratio was ≥1, the predictive value positive (PNP) for WNV was 97.8%, where 95% of flavivirus cases were due to WNV infection and only 3.7% of specimens would require PRNT to differentiate WNV from SLEV infection. Use of the W/S ratio as part of the testing algorithm to interpret MAC-ELISA results generates reportable probable cases quickly, alleviating the need for PRNT in most instances.

Since the introduction of West Nile virus (WNV) into the United States in 1999, the virus has spread into areas where a closely related virus, St. Louis encephalitis virus (SLEV), is endemic (3). Both viruses are medically important members of the Japanese encephalitis virus serocomplex of the family Flaviviridae (6), causing similar clinical syndromes which range from inapparent infection to encephalitis. Simultaneous transmission of both viruses in the same area, while not common, is possible and occurred in 2001 in Louisiana. Simultaneous transmission events increase the necessity to distinguish between these viruses serologically. The primary diagnostic screening test to detect both WNV- and SLEV-specific antibody is the immunoglobulin M (IgM) capture enzyme-linked immunosorbent assay (MAC-ELISA) (9). Due to cross-reactivity of flaviviral antibodies in the MAC-ELISA, serological differentiation of these viruses requires using the plaque reduction neutralization test (PRNT) (7) or virus isolation, both of which are cumbersome and time-consuming. Furthermore, the PRNT involves manipulation of live WNV and SLEV, which requires biosafety level 3 containment. Since many public health laboratories do not have this biosafety level 3 containment, alternative rapid methods are needed to differentially diagnose WNV and SLEV infections without using PRNT for all samples.

To this end, an algorithm was established that exploited WNV and SLEV antibody cross-reactivity in the MAC-ELISA. Results of simultaneous testing with these two viruses in the MAC-ELISA expressed as positive/negative (P/N) optical density (OD) ratios were compared (8). A P/N ratio is defined as the result of dividing the average OD reading of an unknown specimen tested against the positive antigen (P) by the average OD reading of a normal human serum tested against positive antigen (N). This algorithm suggested that when the WNV P/N ratio in a MAC-ELISA was at least three times greater than the SLEV P/N ratio in the same test, WNV infection was correctly identified in 92% of the serum specimens and 100% of cerebrospinal fluid (CSF) specimens. However, if the infecting virus was SLEV, identification was correct in only 59% of specimens and the P/N ratio for SLEV was usually only twice or less the P/N ratio for WNV. This algorithm required that all reagents used in the MAC-ELISA be uniformly standardized and was established with a battery of well-defined PRNT-confirmed serum and CSF samples containing WNV- or SLEV-specific antibody.

Midway through the WNV epidemic of 2002, samples submitted to the Centers for Disease Control and Prevention (CDC) for confirmation of WNV infection in humans were triaged using this algorithm. If the MAC-ELISA P/N ratio for WNV was three times or greater than the MAC-ELISA SLEV P/N ratio, the results were interpreted as showing evidence of recent infection with WNV and PRNT was not performed. These cases were reported as probable WNV infections. We report a retrospective examination of the data from the 2002 transmission season to better evaluate the algorithm and determine its validity of use in an epidemic situation.

MATERIALS AND METHODS

Using the method previously described (9), MAC-ELISA was performed and P/N ratios were calculated for 1,336 serum and 82 CSF specimens submitted for diagnostic testing. WNV-to-SLEV P/N ratios (W/S ratios) were calculated. Corresponding PRNT data were generated by a technique adapted from Lindsey et al. (7) and were tabulated to determine the number of confirmed cases caused by each virus. All submitted specimens acquired less than 10 days after the reported onset of symptoms were eliminated from the analysis, since neutralizing antibody may not have fully formed by that time.

Specimens considered to have WNV-specific antibody were those that had a MAC-ELISA P/N ratio of ≥3 to either WNV, SLEV, or both and a positive WNV PRNT (≥1:10) that was at least fourfold greater than the SLEV PRNT titer (1). Specimens considered to have SLEV-specific antibody were those that had a MAC-ELISA P/N ratio of ≥3 to either WNV, SLEV, or both and a positive SLEV PRNT titer (≥1:10) that was at least fourfold greater than the WNV PRNT titer. Less than a fourfold difference between the WNV and SLEV titers was acceptable if the PRNT titer of the corresponding virus was negative (<1:10) in each case above. Specimens that were considered indeterminate were those that had a MAC-ELISA P/N ratio of ≥3 to either WNV or SLEV or both and positive PRNT titers (≥1:10) to WNV and SLEV, but with less than a fourfold difference between them. Specimens having positive MAC-ELISA P/N ratios (≥3) to either WNV or SLEV or both but negative PRNT (<1:10) to both WNV and SLEV were considered unconfirmed. These PRNT-unconfirmed specimens could contain antibody elicited by another flavivirus that cross-reacts with either WNV or SLEV, such as antibody formed in response to a vaccination with Japanese encephalitis or yellow fever virus, or legitimate false-positive results in the MAC-ELISA.

RESULTS

Of the 1,336 serum samples used in the study, 1,263 were considered to have WNV-specific antibody and 40 were considered to have SLEV-specific antibody; the 33 remaining samples were indeterminate or unconfirmed (Table 1). More than half of serum samples with WNV-specific antibody had a W/S ratio of ≥5.0, whereas, more than half of samples with SLEV-specific antibody had a W/S ratio of <1.0 (Table 1). A similar pattern may have existed for the CSF samples, although there were too few CSF samples, particularly those containing SLEV-specific antibody, to draw definite conclusions. Therefore, the remainder of the data analysis will involve serum samples only.

TABLE 1.

MAC-ELISA W/S ratios and the correlation to respective PRNT results in serum and CSF tested in the 2002 WNV outbreak in the United States

MAC-ELISA W/S ratio	Serum					CSF
	PRNT confirmed		Indeterminate by PRNT (% total)	Unconfirmed by PRNT (% total)	Total no. of serum specimens	PRNT confirmed		Indeterminate by PRNT (% total)	Unconfirmed by PRNT (% total)	Total no. of CSF specimens
	WNV-specific antibody positive (% total)	SLEV-specific antibody positive (% total)				WNV-specific antibody positive (% total)	SLEV-specific antibody positive (% total)
<1.00	11 (1)	23 (58)	0 (0)	0 (0)	34	1 (1)	0 (0)	0	0 (0)	1
1.00-1.99	103 (8)	8 (20)	4 (31)	5 (25)	120	7 (9)	2 (67)	0	0 (0)	9
2.00-2.99	130 (10)	5 (13)	1 (8)	3 (15)	139	15 (20)	1 (33)	0	1 (25)	17
3.00-3.99	148 (12)	3 (8)	4 (31)	4 (20)	159	9 (12)	0 (0)	0	0 (0)	9
4.0-4.99	135 (11)	1 (3)	0 (0)	3 (15)	139	6 (8)	0 (0)	0	0 (0)	6
≥5.00	736 (58)	0 (0)	4 (31)	5 (25)	745	37 (49)	0 (0)	0	3 (75)	40
Total	1,263 (100)	40 (100)	13 (100)	20 (100)	1336	75 (100)	3 (100)	0	4 (100)	82

To assess the ability of the W/S ratio to discriminate serum samples which contain WNV-specific antibody from those containing SLEV-specific antibody, we analyzed data from the 1,303 samples determined to have either WNV- or SLEV-specific antibody by PRNT (Table 1). The sensitivity of various W/S ratio cutoffs to identify samples that truly had WNV-specific antibody was determined by calculating the proportion of serum samples considered to have WNV-specific antibody by PRNT that had a W/S ratio at or above each cutoff. The specificity of various W/S ratio cutoffs to discriminate samples that did not have WNV-specific antibody was determined by considering the proportion of serum samples considered to have SLEV-specific antibody by PRNT that was less than the cutoff value. The sensitivities and specificities for various cutoffs of the W/S ratio are presented in Table 2.

TABLE 2.

Calculated sensitivity and specificity of the W/S ratio for determining the presence of WNV-specific antibody in serum samples

W/S cutoff	No. of WNV samples at or above cutoff	Sensitivity (%)a	No. of SLEV samples below cutoff	Specificity (%)b
>0.0	1,263	100	0	0
≥1.0	1,252	99	23	58
≥2.0	1,149	91	31	78
≥3.0	1,019	81	36	90
≥4.0	871	69	39	98
≥5.0	736	58	40	100

^aSensitivity was determined among samples determined to have WNV-specific antibody by PRNT.

^bSpecificity was determined among samples determined to have SLEV-specific antibody by PRNT.

Given these sensitivities and specificities, we then calculated the predictive value positive (PVP) and predictive value negative (PVN) of the W/S ratio at cutoffs ranging from 1.0 to 4.0 in hypothetical populations in which 80 to 95% of the infections were due to WNV and the remainder were due to SLEV (Table 3; Fig. 1). Of note, 97% of serum samples submitted for diagnostic testing at CDC during the 2002 WNV outbreak in which WNV- or SLEV-specific antibodies could be demonstrated had WNV-specific antibody (Table 1). If a PVP of 97% is desired (e.g., only 3% of the samples at or above the cutoff would be classified as having WNV antibody, but actually would have SLEV-specific antibody had PRNT testing been done) (Fig. 2), then a cutoff of 3.0 would be required if 80% of the actual infections were due to WNV (Table 3). This cutoff could be lowered to 1.0 to achieve a 97% PVP if at least 95% of the actual infections were due to WNV, which likely occurred in 2002 (Table 3).

TABLE 3.

Estimated PVP and PVN of various W/S ratio cutoffs in three hypothetical populations^a

W/S cutoff	Result for:
	80% WNV/20% SLEV			90% WNV/10% SLEV			95% WNV/5% SLEV
	PVP	PVN	% Requiring PRNT	PVP	PVN	% Requiring PRNT	PVP	PVN	% Requiring PRNT
≥1.0	90.3	94.3	12.2	95.5	87.8	6.6	97.8	77.2	3.7
≥2.0	94.2	68.3	22.7	97.4	49.0	15.9	98.7	31.2	12.4
≥3.0	97.0	53.9	33.4	98.6	34.1	26.4	99.4	19.7	22.8
≥4.0	99.1	44.0	44.3	99.7	26.0	37.7	99.8	14.2	34.3

^aThe populations represent MAC-ELISA (P/N ≥3.0) serum sample results positive for WNV, SLEV, or both where 80 to 95% of infections are due to WNV and the remainder are due to SLEV. The PVP represents the proportion of samples with a W/S ratio at or above the cutoff that actually have WNV-specific antibody determined by PRNT. The PVN represents the proportion of samples testing below the cutoff that actually have SLEV-specific antibody determined by PRNT. If a testing algorithm is used in which all samples with a W/S ratio less than the cutoff are tested by PRNT, the proportion of samples requiring PRNT is indicated.

FIG. 1.

Estimated PVP and PVN for hypothetical flavivirus antibody-positive populations where 95, 90, and 80% of flaviviral infections are caused by WNV, while the remainder are caused by SLEV. Data points are W/S ratios and increase from left to right: 4.0, 3.0, 2.0, and 1.0.

FIG. 2.

Percentage of serum specimens requiring PRNT to identify the infecting virus determined by using W/S ratios in populations with changing ratios of WNV to SLEV infections.

The PVNs were generally low, indicating that a significant proportion of specimens with W/S ratios below the cutoffs also contained WNV-specific antibody.

DISCUSSION

In the summer and fall of 2002, WNV caused the largest epidemic of arboviral meningoencephalitis in U.S. history (3). The virus was shown to be transmitted to humans by a number of methods in addition to mosquito bite: solid organ transplantation, blood transfusion, and potentially through breast milk (2, 4). Concurrent with the 2002 WNV epidemic, sporadic cases of SLEV infection were reported and patient samples with SLEV-specific antibodies were identified by PRNT at CDC (Table 1). While surveillance, mosquito control, and patient treatment with regard to these two flaviviruses differ very little, distinguishing between these viruses is essential to defining the clinical and epidemiological characteristics of WNV in North America, as well as tracking its spread. During the 2002 outbreak, state health department and commercial diagnostic laboratories quickly became overwhelmed with the number of submitted specimens for WNV antibody testing.

Current U.S. screening algorithms involve the initial screening of specimens by MAC-ELISA (and IgG ELISA at the facility's discretion) followed by confirmation of positive MAC-ELISA results by PRNT to distinguish WNV infections from SLEV infections (5). This testing regimen takes approximately 2 weeks to complete. Experiments have suggested that this regimen during outbreaks can be simplified by using W/S ratios to differentiate WNV from SLEV infections (8). These original experiments demonstrated that when the WNV P/N ratio was three times greater than the SLEV P/N ratio in the same test (W/S ratio of ≥3), WNV was identified as the infecting virus 92% of the time.

The results presented here from a much larger body of data from the 2002 human epidemic show that a PVP of at least 95% for WNV can be achieved with a W/S ratio as low as 1.0 during an epidemic in which at least 90% of the initially positive MAC-ELISA specimens truly have WNV-specific antibody (Table 3; Fig. 1). The PVNs are somewhat lower, indicating that PRNT testing is necessary to differentiate WNV from SLEV among specimens not meeting the W/S cutoff. Therefore, the optimal strategy would be to test with PRNT all MAC-ELISA-positive (P/N ratio of ≥3.0) samples that were below the chosen W/S cutoff to determine whether these samples contained WNV- or SLEV-specific antibodies. The percentage of all MAC-ELISA-positive samples requiring PRNT testing according to this algorithm would be as low as 4% if a cutoff of 1.0 is used during a WNV epidemic with little concurrent SLEV activity or 95% WNV/5% SLEV (Table 3; Fig. 2). Higher cutoffs are required to achieve a similar PVP if a lower proportion of the flavivirus infections are due to WNV; however, the number of required PRNTs is still substantially reduced (Fig. 2). All cases not confirmed by PRNT would be reported to CDC as probable cases under the current case definition (1).

There are several important limitations to these data. First, these results have not been validated by other commercial MAC-ELISA kits based on the CDC procedure or for P/N cutoffs different from those for the CDC MAC-ELISA. Second, we did not consider indeterminate or unconfirmed samples in this analysis. These accounted for 2.4% of the study samples during the 2002 outbreak. Including these would lower the PVP of a screening algorithm incorporating W/S ratios. Factors that change the pretest likelihood of flavivirus infection, such as time of year or the presence of clinically compatible symptoms, will determine the proportion of samples that test indeterminate or are unconfirmed. Previous flavivirus exposure will likely increase both indeterminate and unconfirmed specimen percentages. Third, the PVP of the W/S ratio is determined by the underlying proportion of infections due to WNV versus SLEV in the population as well as the chosen cutoff. A sufficient number of samples must be tested by PRNT to establish that most infections are due to WNV, in order to determine if the W/S cutoff can be used as part of the screening algorithm and what cutoff should be used. Fourth, algorithms involving W/S ratios should not be used in settings where substantial numbers of secondary infections may be occurring. Because a second exposure to a flavivirus elicits inconsistent IgM production, the W/S ratio may be unreliable (10). Only a limited number of WNV secondary infections have been evaluated, and more will need to be tested to determine with certainty how the algorithm would perform with regard to these specimens. Finally, insufficient CSF specimens, especially testing positive for SLEV, were available for accurate data analysis. Data were included to indicate rudimentary testing patterns.

In 2003, another WNV outbreak of similar magnitude to the 2002 outbreak occurred. The wider availability of MAC-ELISA testing caused an increase in the number of diagnostic tests performed, largely among persons with WN fever. Many of the states heavily impacted during the 2002 outbreak were also heavily impacted in 2003, suggesting that large outbreaks in North America may continue for the foreseeable future. Because the number of MAC-ELISA-positive samples requiring differentiation of WNV- from SLEV-specific antibodies will likely continue to outstrip the availability of PRNT testing, the judicious use of W/S ratios in screening algorithms will have a continued public health benefit.