A review of CDC's Web-based Injury Statistics Query and Reporting System (WISQARS™): Planning for the future of injury surveillance

Abstract

Introduction

The Centers for Disease Control and Prevention (CDC) developed the Web-based Injury Statistics Query and Reporting System (WISQARS^TM) to meet the data needs of injury practitioners. In 2015, CDC completed a Portfolio Review of this system to inform its future development.

Methods

Evaluation questions addressed utilization, technology and innovation, data sources, and tools and training. Data were collected through environmental scans, a review of peer-reviewed and grey literature, a web search, and stakeholder interviews.

Results

Review findings led to specific recommendations for each evaluation question.

Response

CDC reviewed each recommendation and initiated several enhancements that will improve the ability of injury prevention practitioners to leverage these data, better make sense of query results, and incorporate findings and key messages into prevention practices.

1. Introduction

In 1999, the Centers for Disease Control and Prevention's (CDC) National Center for Injury Prevention and Control (NCIPC) became early adopters of leveraging newly emerging internet technology to meet the data needs of injury and violence prevention practitioners (Centers for Disease Control and Prevention, 2016a). The Web-based Injury Statistics Query and Reporting System (WISQARS™), was developed as a user-friendly system that allowed the public 24/7 access to injury surveillance data and customizable reports. Initially WISQARS™ provided fatal injury reports and leading causes of death reports (Table 1), and over the next 10 years gradually expanded the scope of WISQARS™to include additional modules such as non-fatal injury reports, the National Violent Death Reporting System (NVDRS) (Blair et al., 2016), fatal injury mapping, and cost of injury reports (Table 2).

Table 1.

Ten leading causes of death by age group, United States, 2014.

	Age goups
Rank	<1	1–4	5–9	10–14	15–24	25–34	35–44	45–54	55–64	65+	Total
1	Congenital anomalies 4746	Unintentional injury 1216	Unintentional injury 730	Unintentional injury 750	Unintentional injury 11,836	Unintentional injury 17,357	Unintentional injury 16,048	Malignant neoplasms 44,834	Malignant neoplasms 115,282	Heart disease 489,722	Heart disease 614,348
2	Short gestation 4173	Congenital anomalies 399	Malignant neoplasms 436	Suicide 425	Suicide 5079	Suicide 6569	Malignant neoplasms 11,267	Heart disease 34,791	Heart disease 74,473	Malignant neoplasms 413,885	Malignant neoplasms 591,699
3	Maternal pregnancy comp. 1574	Homicide 364	Congenital anomalies 192	Malignant neoplasms 416	Homicide 4144	Homicide 4159	Heart disease 10,368	Unintentional injury 20,610	Unintentional injury 18,030	Chronic low. respiratory disease 124,693	Chronic low. respiratory disease 147,101
4	SIDS 1545	Malignant neoplasms 321	Homicide 123	Congenital anomalies 156	Malignant neoplasms 1569	Malignant neoplasms 3624	Suicide 6706	Suicide 8767	Chronic low. respiratory disease 16,492	Cerebro–vascular 113,308	Unintentional injury 136,053
5	Unintentional injury 1161	Heart disease 149	Heart disease 69	Homicide 156	Heart disease 953	Heart disease 3341	Homicide 2588	Liver disease 8627	Diabetes mellitus 13,342	Alzheimer's disease 92,604	Cerebro– vascular 133,103
6	Placenta cord. membranes 965	Influenza & pneumonia 109	Chronic low. respiratory disease 68	Heart disease 122	Congenital anomalies 377	Liver disease 725	Liver disease 2582	Diabetes mellitus 6062	Liver disease 12,792	Diabetes mellitus 54,161	Alzheimer's disease 93,541
7	Bacterial sepsis 544	Chronic low respiratory disease 53	Influenza & pneumonia 57	Chronic low respiratory disease 71	Influenza & pneumonia 199	Diabetes mellitus 709	Diabetes mellitus 1999	Cerebro– vascular 5349	Cerebro– vascular 11,727	Unintentional injury 48,295	Diabetes mellitus 76,488
8	Respiratory distress 460	Septicemia 53	Cerebro– vascular 45	Cerebro– vascular 43	Diabetes mellitus 181	HIV 583	Cerebro– vascular 1745	Chronic low. respiratory disease 4402	Suicide 7527	Influenza & pneumonia 44,836	Influenza & pneumonia 55,227
9	Circulatory system disease 444	Benign neoplasms 38	Benign neoplasms 36	Influenza & pneumonia 41	Chronic low respiratory disease 178	Cerebro– vascular 579	HIV 1174	Influenza & pneumonia 2731	Septicemia 5709	Nephritis 39,957	Nephritis 48,146
10	Neonatal hemorrhage 441	Perinatal period 38	Septicemia 33	Benign neoplasms 38	Cerebro– vascular 177	Influenza & pneumonia 549	Influenza & pneumonia 1125	Septicemia 2514	Influenza & pneumonia 5390	Septicemia 29,124	Suicide 42,773

Table 2.

Current WISQARS™ modules.

Module	Year launched
Fatal injury reports	2000
Leading causes of death	2000
Nonfatal injury reports	2001
Leading causes of nonfatal injury	2001
Fatal years of potential life lost (YPLL)	2002
Violent deaths	2008
Fatal injury maps	2010
Cost of injury reports	2011
Mobile applications	2014

CDC research and scientific programs periodically undergo external review to maintain the quality, relevance, and impact of the centers' activities. Since 2005, NCIPC has conducted a number of Portfolio Reviews on topic areas such as youth violence, falls among older adults (Sleet et al., 2008), traumatic brain injuries, Injury Control Research Centers, motor vehicle injuries, State Injury Control Core Programs, and sexual violence (DeGue et al., 2012). In 2015, NCIPC completed a WISQARS™ Portfolio Review. In this article we describe the essential elements of the Review, and how this process has informed future development and improvement of an important national resource.

2. Review process and methods

Multiple groups of individuals were involved in the review. The Review Work Group, which included key science, policy, and communication experts in NCIPC, developed evaluation questions; provided guidance on goals, scope, and process; and helped to identify the External Peer Review Panel. The Evaluation Team supported the day to day activities and oversaw the work of an external contractor, who supported the planning and implementation, and development of the final report. Lastly, the External Peer Review Panel, which included key non-CDC subject matter experts, reviewed the report and developed recommendations.

The Review addressed the following questions:

1. Are WISQARS™ data being utilized for scientific and programmatic purposes by key stakeholders? [Utilization]

2. How can modern technology and innovation enhance the use of WISQARS™? [Technology and innovation]

3. What are the opportunities to expand the data sources/datasets? [Data sources]

4. What training, tools, and resources would facilitate actionable data translation? [Tools and training]

• Information from a number of sources was obtained, critically analyzed, and synthesized to form the basis for recommendations.

• Environmental scan of other data systems: internal and external web-based data querying systems (WBDQS) were identified and assessed.

• Technical features of these interfaces were summarized and documented.

• Review of peer-reviewed literature: An electronic search of the Pub Med database identified information on use and usability of a sample of WBDQS that had a similar form and function to WISQARS™. This search identified 118 potential references between 2004 and 2014. Of these 48 were determined to be relevant and subsequently were reviewed and summarized.

• Review of grey literature and a web search: A search of the New York Academy of Medicine Grey Literature database and Google was used to identify science reports that cited WISQARS™ as a data source, and to better understand who is using WISQARS™, how and for what purpose WISQARS™ data are being used, and the types of WISQARS™ data being accessed by topic (e.g., homicide, suicide, poisoning, drowning) and module (e.g., fatal, nonfatal, NVDRS, cost).

• Stakeholder Interviews: Thirty-two individuals were interviewed to better understand users' data needs, system accessibility, and how data were being used. Stakeholders included NCIPC staff, representatives from other federal agencies, academia, and additional external stakeholders from policy groups, state and local health departments, non-government organizations, and traditional media outlets.

3. Results

Collected data yielded the following findings (organized by evaluation question):

3.1. Utilization

The grey literature review showed:

• Of the first 100 websites from Google searches, data came most frequently from the fatal injury or non-fatal injury modules.

• NGOs sponsored most websites, followed by academic institutions, agencies of the federal government or a tribal, local, or state health department; or social media.

• Websites most often used WISQARS™ to summarize data in citations, presentations, or teaching initiatives.

Stakeholder interviews revealed:

• Stakeholders also most frequently used the fatal and non-fatal injury modules.

• Data were most frequently accessed on suicides and poisonings, followed by homicides, motor vehicle crashes, and falls. Additional topics included falls in the elderly, firearm deaths, and child maltreatment injuries.

• Stakeholders most often used WISQARS™data to respond to data requests, educate decision makers, conduct further analysis, or for teaching and planning.

3.2. Technology and innovation

The environmental scan showed:

• Data in most WBDQS are primarily from federal data systems, and provide national- and state-level estimates.

• Most WBDQS allow the user to download results immediately, and documentation and user support are available.

• Ability of users to customize queries and results tables/graphs varies across WBDQS.

Stakeholders suggest:

• For easier navigation, change the existing URL to something short and easy to remember, and include the word “injury.”

• To improve webpage layout, use responsive design strategies.

• To improve data display, use ‘heat maps’ to show the areas of highest burden.

• Adapt some of the mobile app data visualization functions for use in the PC version.

• Enable users to cut and paste graphs.

3.3. Data sources

Stakeholders want more data on:

• Nature of the injury and area of the body harmed.

• Circumstances and geographic location of the event, such as the weapon used in firearm injuries, state laws where event takes place, and the type and class of drugs in drug overdoses.

• Social and economic contextual factors.

• More cost data, such as payer source, with breakdowns for emergency department visits, rehab, etc.

• Lesbian, gay, bisexual, and transgender (LGBT) and institutionalized populations.

Stakeholders want to link data, for example:

• Behavioral Risk Factor Surveillance System (BRFSS) data on seat belt use or inadequate sleep linked to WISQARS™ fatal/nonfatal data on motor vehicle crash injuries.

• Substance Abuse and Mental Health Administration (SAMHSA) alcohol/substance abuse data linked to WISQARS™ data on fatal/nonfatal drug overdose.

• Population prevalence of psychiatric diagnoses linked to WISQARS™ data on fatal/nonfatal injuries due to violence.

3.4. Tools and training

Stakeholders suggest:

• Allowing online training to be customized; webinars are preferred.

• Making online videos short and show step-by-step site navigation.

• Posting a table comparing WISQARS™ data, features, and capabilities to other WBDQS.

• Providing WISQARS™ fact sheets to explain data sources.

• Including explanatory pop-up boxes.

4. Response to the review and future directions for WISQARS™

NCIPC established internal workgroups for the four recommendation categories (Table 3.). Each workgroup included five to seven individuals and met multiple times to review and discuss each recommendation and the specific actions included in the report. The findings and recommendations often overlapped across categories.

Table 3.

Recommendations from the WISQARS^TM Portfolio Review Expert Panel.

Utilization Conduct appropriate research, testing, and evaluation at the conceptual, developmental, and implementation phases for the new WISQARS™ experience. Develop a matrix of functionality/function uses for each priority audience, and better define the requirements for use by different audiences. Develop a defined vision and strategy for WISQARS™ moving forward. Technology and innovation Develop more capacity for users to export both data and graphics. Explore the possibility of a query tool capable of accessing and aggregating across disparate datasets. Improve visualization functionality in the system. Shift the mobile strategy from the proliferation of mobile apps to mobile responsiveness. Data sources Establish an expert working group to identify the holes/gaps in the data and envision existing and future data sources. Explore ways to incorporate or bridge other injury-focused datasets to inform the technology required to make the WISQARS™ system scalable. Plan for expansion by incrementally including additional data sets. Tools and training Add communications capacity to the WISQARS™ team to look at enhancements and strategies to raise awareness of WISQARS™. Create system-wide capacity to provide better guidance on using WISQARS™ and identify ways to integrate this guidance into the users' experience. Provide more examples within WISQARS™ of how the data can be used.

The workgroups agreed that many recommendations were strong ideas, with NCIPC already having considered several. Several recommendations were not considered to be feasible at this time for practical reasons. For example, linking data sets is problematic without personal identifiers, different definitions for common variables, and restrictions on data use. Given their number and the resources required to adopt them, the team's planning process included prioritizing the remaining recommendations into short term versus longer term actions.

One of the difficulties with addressing many of the recommendations is aligning these recommendations with the NCIPC's overall vision of WISQARS™as a tool for injury and violence surveillance, rather than a research database. NCIPC's goal is to have the most up-to-date data available online, and to identify opportunities to use current technology, strategies, and best practices for disseminating these data through queried reports, charts, and maps. Users of WISQARS™range from individuals with very limited public health and epidemiology experience (e.g., reporters, members of the general public), to those highly trained and skilled in using injury and violence data (e.g., academic researchers). It is challenging to design and maintain WISQARS™such that it meets every possible need. WISQARS™queries can lead to important research questions, but is the system is not by design a research tool, so advanced researchers who wish to run more complicated analyses should download the full datasets, when available, and use statistical software outside of the WISQARS™ platform.

The Portfolio Review proved to be an important catalyst for a substantial reexamination by NCIPC of the function of WISQARS™ and the ways in which it could be improved to optimally meet its intended purpose. The response to the review provided an opportunity for NCIPC to think strategically about potential enhancements to the system and its future direction. NCIPC is moving forward with changes that directly address recommendations and in the next twelve months will complete the following development projects.

4.1. Visualization pilot

Interactive data visualization platforms for injury surveillance can improve data use, analytic capacity, and the communication of key messages (Martinez et al., 2016). NCIPC is starting a pilot in 2016 to develop an internal interactive data visualization prototype that utilizes injury death data. The goals of this pilot are to learn about the development process and the needed skills and resources to implement, and to have a working internal prototype to show the potential of data visualizations for modules in WISQARS™. Initial key activities include reviewing existing online data visualizations applications and developing visual wireframes for what the prototype looks like and how it functions. After a fully functional internal prototype is developed and tested, NCIPC will assess the feasibility for external release that complies with federal website requirements.

4.2. Additional data sets

The Data Source workgroup reviewed the Inventory of National Injury Data Systems (Centers for Disease Control and Prevention, 2016b) to discuss if any of these systems should be added to WISQARS™. The discussion led to one of the main limitations with WISQARS™-that state-level non-fatal (morbidity) data are not available to query; only national estimates based on National Electronic Injury Surveillance System-All Injury Program (NEISS-AIP) data are currently available. NCIPC is reviewing opportunities to add state-level morbidity data to WISQARS™. Before a new fully functional module is developed, NCIPC will assess the availability of data sets that could fill this gap. For potential data sets, this process includes understanding data quality (e.g., representativeness, timeliness, completeness, availability of external causes of injury) and data usage requirements and restrictions to adding these data to the WISQARS™ platform. Additionally, NCIPC is reviewing the time, resources, and skills needed to develop and maintain a new WISQARS™ module.

4.3. Mobile responsiveness

As smartphones and tablets become more common, it is important that WISQARS™is usable through these devices. NCIPC has been assessing the mobile responsiveness of all the WISQARS™webpages. All WISQARS™landing pages have already been converted to a mobile responsive design, but the querying and output pages presently are not. Mobile-friendly testing currently is being run on all 23 querying pages and 20 output pages. This includes identifying mobile responsiveness issues and solutions. As solutions are implemented, NCIPC will develop one strategy that converges the mobile responsiveness of the full WISQARS™ websites and modules to replace WISQARS™ mobile apps.

NCIPC also will look for opportunities to improve WISQARS™tools and support. Strategies may include reviewing, updating, and reorganizing the existing help files online; connecting users who query specific topics to other CDC materials that may be of interest; and compiling materials that effectively used WISQARS's modules to demonstrate potential uses of the system.

5. Conclusion

Effective injury surveillance is an essential component of a public health approach to injury prevention. By identifying the nature and extent of the problem, surveillance enables appropriate prioritization of action, whether this be in relation to prevention strategies or services, and facilitates the monitoring of intervention success. Just as critically, surveillance provides the foundations for new developmental work by researchers and practitioners. WISQARS™can be a powerful tool that allows for timely access to key existing injury data systems in the United States. Enhancements to the system will improve the ability of injury prevention practitioners to leverage these data, better make sense of query results, and incorporate findings and key messages into prevention practices (McClure & Mack, 2016). Ultimately, enhancements to WISQARS™will lead to improvements in the injury- and violence-related health of the population.

Biographies

Dr. Michael F. Ballesteros, PhD, MS, is the Chief for the Statistics, Programming, and Economics Branch within the Division of Analysis, Research, and Practice Integration at the Centers for Disease Control and Prevention. His branch provides statistical, economic, IT programming, and informatics expertise to CDC's National Center for Injury Prevention and Control. Dr. Ballesteros received his a Masters in Preventive Medicine and doctorate in epidemiology from the University of Maryland at Baltimore. He is a graduate of CDC's Epidemic Intelligence Service (EIS).

Mr. Kevin Webb is the Team Lead for the Programming Team within the Division of Analysis, Research, and Practice Integration at the Centers for Disease Control and Prevention. His team provides IT programming expertise and oversight of NCIPC's web application hosting platform for CDC's National Center for Injury Prevention and Control. Mr. Webb has 17 years of experience managing the design, development, administration, and maintenance of WISQARS and other web applications. He was one of the original creators of WISQARS in 2000.

Dr. Roderick McClure MBBS, PhD, FAFPHM, is the Director of the Division of Analysis, Research and Practice Integration at the National Center for Injury Prevention and Control. Dr. McClure is a public health physician and injury epidemiologist with 30 years of experience in injury control covering the full research-to-practice continuum.