State Health Department Perceived Utility of and Satisfaction with ArboNET, the U.S. National Arboviral Surveillance System

SYNOPSIS

Objectives.

We assessed the perceived utility of data collected through ArboNET, the national arboviral surveillance system, and evaluated state health department user satisfaction with system function.

Methods.

We used an online assessment tool to collect information about types of arboviral surveillance conducted, user satisfaction with ArboNET's performance, and use of data collected by the system. Representatives of all 53 reporting jurisdictions were asked to complete the assessment during spring 2009.

Results.

Representatives of 48 (91%) jurisdictions completed the assessment. Two-thirds of respondents were satisfied with ArboNET's overall performance. Most concerns were related to data transmission, particularly the lack of compatibility with the National Electronic Disease Surveillance System (NEDSS). Users found mosquito (85%), human disease (80%), viremic blood donor (79%), and veterinary disease (75%) surveillance data to be useful. While there was disagreement about the usefulness of avian mortality and sentinel animal surveillance, only 15% of users supported eliminating these categories. Respondents found weekly maps and tables posted on the U.S. Geological Survey (92%) and CDC (88%) websites to be the most useful reports generated from ArboNET data. Although many jurisdictions were willing to report additional clinical or laboratory data, time and resource constraints were considerations. Most respondents (71%) supported review and possible revision of the national case definition for human arboviral disease.

Conclusions.

As a result of this assessment, CDC and partner organizations have made ArboNET NEDSS-compatible and revised national case definitions for arboviral disease. Alternative data-sharing and reporting options are also being considered. Continued evaluation of ArboNET will help ensure that it continues to be a useful tool for national arboviral disease surveillance.

ArboNET, the national arboviral surveillance system, was developed in 2000 to monitor West Nile virus (WNV) infections in humans, mosquitoes, birds, and other animals. This comprehensive approach helped track the progression of WNV activity across the United States and improved our understanding of the epidemiology, ecology, and transmission of WNV. Since WNV was first detected in the Western Hemisphere in 1999, it has become the leading cause of arboviral encephalitis in the U.S.^1–3 The national incidence of WNV disease peaked in 2002 and 2003 and then declined;^3,4 however, focal seasonal outbreaks continue to occur, resulting in substantial morbidity and mortality and requiring significant public health response and resources. In addition to WNV, other domestic arboviruses continue to cause sporadic and seasonal outbreaks.² Furthermore, there is the potential for the emergence of other arboviruses, which could become established if introduced,⁵ a vulnerability demonstrated by recent episodes of local dengue transmission in Texas and Florida.^6,7

In response to changing national trends in arboviral epidemiology, ArboNET has been modified and expanded since its initial development in 2000. For example, following the identification of transfusion-associated WNV transmission in 2002,^8,9 routine screening of all blood donations for WNV was initiated in the U.S., and ArboNET was modified to include reports of WNV presumptive viremic blood donors (PVDs). Beginning in 2003, ArboNET was expanded to include all other nationally notifiable domestic arboviruses (i.e., California serogroup, eastern equine encephalitis, Powassan, St. Louis encephalitis, and Western equine encephalitis viruses) and selected travel-associated arboviruses (e.g., chikungunya, dengue, Japanese encephalitis, tick-borne encephalitis, and yellow fever viruses). Despite the expanding scope of the ArboNET system, national funding to state and local health departments to support arboviral surveillance and testing has been drastically reduced in recent years.

In 2008 and 2009, we conducted the first formal evaluation of ArboNET to determine if the current structure, function, and content provided useful information for public health decision-making, response, and research. As a component of the evaluation, we queried public health departments to assess the utility of ArboNET data and satisfaction with system function. We report the findings of this assessment and make recommendations for system improvement.

METHODS

ArboNET is a passive electronic system managed by the Centers for Disease Control and Prevention (CDC). Arboviral surveillance data are reported to ArboNET by 50 state and three local or territorial health departments (New York City, the District of Columbia, and Puerto Rico). Reporting jurisdictions receive data from public health and commercial reference laboratories, blood-collection agencies, and health-care providers. Jurisdictions transmit data to ArboNET using one of three methods: (1) uploading records from an existing electronic system using an Extensible Markup Language (XML) message, (2) uploading from a Microsoft® Access database using an XML message, or (3) entering records manually using a Web-based form. ArboNET data are routinely reviewed, summarized, and disseminated through Epi-X (the Epidemic Information Exchange, CDC's secure communications network for public health professionals), the Morbidity and Mortality Weekly Report (MMWR), as well as the websites of CDC, the U.S. Geological Survey (USGS), and the Public Health Agency of Canada. In addition, researchers, pharmaceutical companies, the media, and the general public can request limited-use datasets from ArboNET.

Domestic human arboviral diseases, which are nationally notifiable conditions, are reported to ArboNET using standardized case definitions that include clinical and laboratory criteria.¹⁰ WNV PVDs are identified through universal screening of the blood supply; case definitions and reporting practices for PVDs vary by jurisdiction and blood services agency. Data routinely collected in ArboNET for human disease cases and PVDs include patient demographics, county and state of residence, date of illness onset or blood donation, case status (i.e., confirmed, probable, suspect, or not a case), clinical syndrome (e.g., encephalitis, meningitis, or uncomplicated fever), and outcome. Cases reported as encephalitis, meningitis, or acute flaccid paralysis are collectively referred to as neuroinvasive disease; others are considered non-neuroinvasive disease. Nonhuman arboviral surveillance is voluntary and performed variably across different jurisdictions. Data typically reported to ArboNET for nonhuman arboviral infections include animal species, state and county, and date of symptom onset or specimen collection. Reporting of total numbers of mosquitoes or birds tested is encouraged, but reporting of these data is often incomplete.

The assessment tool was designed to collect information regarding types of surveillance conducted in each jurisdiction, user satisfaction with ArboNET's performance, and how the data were used. In addition, questions were asked about proposed changes to ArboNET and national arboviral surveillance and jurisdictions' ability and willingness to provide additional data. In the spring of 2009, the assessment was distributed by the Council of State and Territorial Epidemiologists (CSTE) to state and territorial epidemiologists in all 53 reporting jurisdictions via SurveyMonkey^TM, an online administration tool.¹¹ State epidemiologists could either complete the assessment themselves or forward it to another appropriate respondent within the jurisdiction. Participants were asked to provide responses representing the official opinions and interests of their jurisdictions, not merely personal opinions. We calculated the proportion of jurisdictions providing each response. For a subset of questions, these proportions were calculated after categorizing jurisdictions by WNV neuroinvasive disease incidence. Jurisdictions were categorized according to incidence of WNV neuroinvasive disease as high incidence (≥0.50 incidents per 100,000 population, n=18), medium incidence (0.10–0.49 incidents per 100,000 population, n=19), or low incidence (<0.10 incidents per 100,000 population, n=16) using the mean annual incidence of WNV neuroinvasive disease reported from 1999 through 2008.³

RESULTS

Of the 53 assessments distributed, 48 (91%) were completed. Of the 46 respondents who provided their titles, 28 (61%) were vector-borne disease surveillance coordinators, seven (15%) were state epidemiologists, six (13%) were state public health veterinarians, and five (11%) were staff epidemiologists. Most respondents (67%) were satisfied with the overall performance of the system as compared with other national disease surveillance systems (Figure 1). Respondents were generally satisfied with the system's reliability/availability (77%) and structure/ease of operation (69%). Approximately half of users were satisfied with the ability of the system to accommodate and adapt to changes in events under surveillance, case definitions, and technology.

Figure 1.

Percentage of U.S. state and territorial health departments satisfied with the performance of the ArboNET^a surveillance system, as reported in a 2009 assessment (n=48)

^aArboNET is a passive electronic surveillance system managed by the Centers for Disease Control and Prevention. Arboviral surveillance data are reported to ArboNET by 50 state and three local or territorial health departments (New York City, the District of Columbia, and Puerto Rico).

National arboviral surveillance data are currently used by state and local health departments in a variety of ways, particularly to provide information to state and local stakeholders and monitor national and regional epidemiology. Table 1 lists the reported current uses of ArboNET data by the 48 jurisdictions. Among the national reports routinely generated, users found the weekly USGS maps (92%) and the weekly CDC tables and maps (88%), both of which are available on the Internet, to be most useful. However, there were a number of suggestions for improvements to these websites. For example, many respondents (85%) reported that it would be useful to be able to view multiple data types simultaneously on the USGS maps. Some jurisdictions suggested including on maps an indication of timing of reports to better track the progression of seasonal activity and/or outbreaks. Additionally, an online interactive data-querying tool was suggested, particularly to allow for easier access to historical data (data not shown).

Table 1.

Current uses of ArboNET^a surveillance data by U.S. state and territorial health departments, as reported in a 2009 assessment

^aArboNET is a passive electronic surveillance system managed by the Centers for Disease Control and Prevention. Arboviral surveillance data are reported to ArboNET by 50 U.S. state and three local or territorial health departments (New York City, the District of Columbia, and Puerto Rico).

Most respondents reported that surveillance for human arboviral neuroinvasive disease cases (96%), non-neuroinvasive disease cases (90%), PVDs (83%), infections in mosquitoes (92%), and veterinary diseases (90%) was conducted in their jurisdictions in 2008 (Figure 2). All jurisdictions with a high or medium incidence of neuroinvasive disease also conducted surveillance for non-neuroinvasive disease compared with 77% of low-incidence jurisdictions. PVD surveillance was conducted in 100% of high-incidence, 81% of medium-incidence, and 77% of low-incidence jurisdictions. Mosquito surveillance was conducted in 100% of high-incidence, 88% of medium-incidence, and 85% of low-incidence jurisdictions. Surveillance for arboviral infections in dead birds and sentinel animals was conducted less frequently (data not shown).

Figure 2.

Arboviral disease surveillance activities performed by U.S. state and territorial health departments in 2008, as reported in a 2009 assessment (n=48)

Although 83% of the jurisdictions reported the number of mosquito pools that tested positive for arbovirus infections (i.e., numerator data), only 58% reported the total numbers of mosquitoes trapped and tested (i.e., denominator data). Similarly, more jurisdictions reported the number of dead birds found in the community that tested positive (63%) than the total numbers of birds tested (42%). Commonly stated obstacles to reporting the numbers of birds and mosquitoes tested were the time and resources required to collect and report these data. Nearly all (92%) respondents said that they were willing to provide an annual list of surveillance activities conducted within their jurisdictions, but only 18 (38%) said that they would be able to enumerate these activities at the county level (data not shown).

Most respondents stated that human, mosquito, and veterinary arboviral surveillance data are useful (Table 2). High-incidence jurisdictions were less likely to report that veterinary disease (67%), avian mortality (53%), and sentinel animal (27%) surveillance data were useful than medium- and low-incidence jurisdictions (90%, 79%, and 62%, for veterinary disease, avian mortality, and sentinel animal surveillance data, respectively) (data not shown). Although there was not strong agreement on the usefulness of avian mortality (65%) and sentinel animal (48%) surveillance (Table 2), only 15% of respondents were in favor of eliminating these data categories from ArboNET (data not shown). No low-incidence states supported the elimination of any nonhuman surveillance data categories. As shown in Table 3, few respondents reported that funding cuts had caused them to decrease or eliminate surveillance for human disease cases or PVDs. However, many respondents said that funding cuts had caused them to decrease or eliminate nonhuman surveillance efforts, particularly for mosquitoes (63%) and dead birds (60%).

Table 2.

U.S. state and territorial health departments that consider human and nonhuman ArboNET^a surveillance data to be useful, by surveillance category, as reported in a 2009 assessment

^aArboNET is a passive electronic surveillance system managed by the Centers for Disease Control and Prevention. Arboviral surveillance data are reported to ArboNET by 50 state and three local or territorial health departments (New York City, the District of Columbia, and Puerto Rico).

Table 3.

U.S. state and territorial health departments that have decreased or eliminated arboviral surveillance activities as a result of decreased federal funding, as reported in a 2009 assessment

Many comments expressed dissatisfaction with data transmission mechanisms, particularly the system's inability to accept National Electronic Disease Surveillance System (NEDSS)-compliant messages. This incompatibility has resulted in double data entry for many jurisdictions. Most respondents (60%) said that their jurisdictions had electronic surveillance systems that were capable of sending Health Level Seven International messages to NEDSS. Twenty jurisdictions (42%) reported that they would find it useful to receive additional training on ArboNET data-entry and transmission mechanisms. In addition, respondents reported that it would be useful to be able to access historical data (94%) and import tabular data to the system (77%) (data not shown).

When jurisdictions were queried about several proposed changes to ArboNET and national arboviral surveillance, respondents were generally supportive (71%) of a review and possible revision of the national arboviral surveillance case definition. There was also support (65%) for the identification and funding of arboviral surveillance sentinel sites. There was less support (54%) for expanding ArboNET to include other vector-borne infectious diseases; cited reasons for opposing the expansion included lack of necessary funding and excessive time required to enter reports because of NEDSS incompatibility (data not shown).

Jurisdictions were queried on their ability and willingness to provide additional clinical (e.g., signs and symptoms) and laboratory (e.g., specimen type, specimen collection date, and type of test performed) data for human arboviral disease cases. Most respondents (88%) currently collect clinical data and would be willing to report them to ArboNET (73%) (Table 4). Most respondents also currently collect laboratory data (90%), specifically arboviral diagnostic testing results, and would be willing to report them (63%). There was a lack of agreement on whether it would be useful to report additional clinical (58%) or laboratory (52%) data to ArboNET. Cited obstacles to reporting these additional data were primarily related to the time and resources required to collect and report this information. Sixty-seven percent of respondents reported that they currently report travel-associated or imported human arboviral disease cases; a larger proportion (79%) said that they were willing to report these data. Most respondents (65%) said that reporting imported cases would be useful (Table 4). Stated obstacles to reporting imported cases included difficulty in confirming diagnoses, lack of support for reporting diseases that are not designated as nationally notifiable, and lack of standardized surveillance case definitions (data not shown).

Table 4.

U.S. state and territorial health departments that already collect additional human arboviral disease surveillance data, would be willing to report additional data, and think it would be useful to have additional data reported to ArboNET^a as reported in a 2009 assessment (n=48)

^aArboNET is a passive electronic surveillance system managed by the Centers for Disease Control and Prevention. Arboviral surveillance data are reported to ArboNET by 50 state and three local or territorial health departments (New York City, the District of Columbia, and Puerto Rico).

^bJurisdictions that already collect or receive these data through routine surveillance

^cJurisdictions that are willing to report, or collect and report if necessary, additional data to ArboNET

^dJurisdictions that think it would be useful to have these additional data available in ArboNET

^eIncludes travel-associated cases of yellow fever, Japanese encephalitis, dengue, and tick-borne encephalitis

DISCUSSION

Given the changing trends in arboviral epidemiology and the increasingly limited availability of funding to support arboviral surveillance, continued evaluation and modification of ArboNET is essential to ensure that surveillance is as efficient and useful as possible. The assessment described in this article was conducted as part of a broad evaluation of ArboNET performed by a working group representing primary stakeholders in arboviral surveillance. The objectives of the overall evaluation were to evaluate the quality and utility of ArboNET data, identify and implement changes to better serve ArboNET users, and develop a five-year plan for national arboviral surveillance. The findings of the user assessment were considered in combination with the findings of other evaluation components to identify areas of greatest concern and develop recommendations for improvement.

In general, state and local health departments found ArboNET data to be useful and were satisfied with the performance of ArboNET relative to other national disease surveillance systems. Surveillance for human arboviral disease cases is conducted in most jurisdictions. Although slightly fewer jurisdictions conducted surveillance for viremic blood donors, PVD surveillance data were reported to be as useful as human disease surveillance data. Surveillance for nonhuman infections is more variable and appears to depend on funding and incidence of disease. If funding to support arboviral surveillance continues to decrease, it seems likely that fewer jurisdictions will be able to maintain robust nonhuman surveillance programs. This change will decrease our ability to detect virus activity with adequate time to implement public health control measures to prevent human disease epidemics or outbreaks. Mosquito surveillance data were reported to be the most useful when compared with other surveillance categories, perhaps because arboviral infections in mosquitoes are an indicator of immediate human risk and can be reported without the delays associated with investigating human infections. While there was disagreement on the utility of avian mortality and sentinel animal surveillance, there was little support for eliminating these or any other data categories from ArboNET, particularly among lower-incidence states, which were more likely to find nonhuman surveillance data useful. Jurisdictions seemed reluctant to support the elimination of data categories that might be useful to others, especially as reporting of nonhuman data is voluntary.

The variability in nonhuman surveillance practices is one of the major limitations of ArboNET data. Establishing a mechanism to record the relative completeness and intensity of surveillance activities of each jurisdiction at the beginning of each season would assist with data interpretation. Almost all jurisdictions reported that they would be willing to do so to some extent.

Many specific complaints were related to problems with data transmission and manipulation, particularly the historic lack of NEDSS compatibility. ArboNET has recently been updated to accept NEDSS-compliant messages to facilitate integration with other state and national electronic surveillance systems. This new mechanism of data transmission was piloted in one state in 2009 and was made available to all states prior to the start of the 2010 arboviral transmission season. Since this survey was completed, two ArboNET user-training sessions have been held. Site visits may be planned in the future in response to other identified problems and to ensure that users are making the most efficient use of ArboNET.

Arboviral disease summary data and maps continue to be available on CDC, USGS, and Public Health Agency of Canada websites, which are updated weekly during the arboviral transmission season. More detailed data are available by request; ArboNET data-release guidelines have been revised to be consistent with those recommended by CSTE. A newly revised weekly domestic arboviral disease report, which now includes data reported for La Crosse virus, eastern equine encephalitis virus, Powassan virus, St. Louis encephalitis virus, and WNV, is circulated to state and local health department partners by e-mail and is posted to Epi-X. Other options to improve ArboNET user ability to access, export, manipulate, and analyze data, including the ability to query data, retrieve data in tabular format, superimpose layers of geospatial data, and access historical data, are being investigated.

There was general support for a review and possible revision of the national case definition for human arboviral disease. CSTE and CDC reviewed and revised the case definition, and the revised definition was adopted at the 2010 CSTE Annual Conference. The addition of laboratory variables to ArboNET, including type of laboratory test, is under consideration. Reporting additional human clinical and laboratory data to ArboNET would allow for some confirmation of case classification and case status at the national level. Although most jurisdictions are willing and able to report additional human clinical and laboratory data to ArboNET, time and resource constraints must be carefully considered before additional data fields are incorporated into the system.

Limitations

The findings of this study were subject to at least two limitations. For one, the methods used by respondents to rate ArboNET's utility and performance were subjective and likely varied by jurisdiction. Participants were asked to provide responses representing the official opinions and interests of their jurisdictions, but it is possible that some respondents provided personal opinions. Secondly, although 91% of jurisdictions completed the assessment, the results might have differed if all jurisdictions had completed the assessment.

CONCLUSIONS

This article presents findings from the first assessment of the perceived utility of and user satisfaction with the ArboNET surveillance system. Most users were satisfied with the performance of ArboNET. Users disagreed about the usefulness of some surveillance data categories, but most did not support the elimination of any categories from ArboNET. Although many jurisdictions are willing to report additional clinical and laboratory data, time and resource constraints and data utility need to be considered before supplemental fields are added to the system. As a result of this assessment, CDC and partner organizations have made ArboNET NEDSS-compatible, revised national case definitions for arboviral disease, and revised routine ArboNET reports. Other proposed changes are being considered and may be implemented during the coming years. Continued evaluation of ArboNET will help ensure that it continues to be a useful tool for national arboviral disease surveillance.