Visualization and Analytics Tools for Infectious Disease Epidemiology: A Systematic Review

Abstract

Background

A myriad of new tools and algorithms have been developed to help public health professionals analyze and visualize the complex data used in infectious disease control. To better understand approaches to meet these users' information needs, we conducted a systematic literature review focused on the landscape of infectious disease visualization tools for public health professionals, with a special emphasis on geographic information systems (GIS), molecular epidemiology, and social network analysis. The objectives of this review are to: (1) Identify public health user needs and preferences for infectious disease information visualization tools; (2) Identify existing infectious disease information visualization tools and characterize their architecture and features; (3) Identify commonalities among approaches applied to different data types; and (4) Describe tool usability evaluation efforts and barriers to the adoption of such tools.

Methods

We identified articles published in English from January 1, 1980 to June 30, 2013 from five bibliographic databases. Articles with a primary focus on infectious disease visualization tools, needs of public health users, or usability of information visualizations were included in the review.

Results

A total of 88 articles met our inclusion criteria. Users were found to have diverse needs, preferences and uses for infectious disease visualization tools, and the existing tools are correspondingly diverse. The architecture of the tools was inconsistently described, and few tools in the review discussed the incorporation of usability studies or plans for dissemination. Many studies identified concerns regarding data sharing, confidentiality and quality. Existing tools offer a range of features and functions that allow users to explore, analyze, and visualize their data, but the tools are often for siloed applications. Commonly cited barriers to widespread adoption included lack of organizational support, access issues, and misconceptions about tool use.

Discussion and Conclusion

As the volume and complexity of infectious disease data increases, public health professionals must synthesize highly disparate data to facilitate communication with the public and inform decisions regarding measures to protect the public's health. Our review identified several themes: consideration of users' needs, preferences, and computer literacy; integration of tools into routine workflow; complications associated with understanding and use of visualizations; and the role of user trust and organizational support in the adoption of these tools. Interoperability also emerged as a prominent theme, highlighting challenges associated with the increasingly collaborative and interdisciplinary nature of infectious disease control and prevention. Future work should address methods for representing uncertainty and missing data to avoid misleading users as well as strategies to minimize cognitive overload.

1. Introduction

In the last 20 years, an increasing focus on the need for informatics and analytics in public health has resulted in a growing investment in information systems [1-7]. This investment has generated a myriad of new tools for different public health activities and jurisdictions, including tools and systems developed by federal, state and local governments, as well as research organizations [8-12]. Advances in electronic reporting and interoperability, computer technology, biotechnology (e.g. genetic sequencing), and other methods (e.g. social network analysis and geographic information systems) have put pressure on the informatics discipline and public health practitioners alike to translate these advances into common practice [1, 7, 13, 14]. This pressure has been particularly acute for the surveillance and management of infectious diseases with pandemic or bioterrorism potential [7, 15-17].

To characterize the variety of tools and analytical approaches developed for infectious disease control, we conducted a systematic literature review of informatics tools for infectious diseases, with a focus on platforms for information visualization. In this review, we assessed the current landscape of these tools in terms of information needs and user preferences, features and system architectures of existing tools, as well as usability and adoption considerations. Due to the challenges of integrating, analyzing, and displaying public health data, particularly new types of data encountered in public health, this review places a special emphasis on efforts to visualize geographic information systems (GIS)¹, molecular epidemiology, and social networks.

1.2. Background

Since John Snow first plotted cholera cases on a map of London, graphs and visualizations have played important roles in epidemiology, supporting communication, aggregation, analysis, and use of data for hypothesis testing and decision making [18, 19]. In the electronic age, computer-aided generation of charts, maps, and reports have enabled a further increase in the use of visualization tools to supplement individual-level clinical data and population-level statistics [7, 15]. Infectious disease burden in the population, whether measured for programmatic or outbreak management purposes, is now commonly analyzed in terms of geographic distribution, clinical risk factors, demographics, molecular and phylogenetic features, or sources of exposure such as social networks [20-23]. While routine features of public health reports include epidemic curves and choropleth maps, new visualization motifs such as social network graphs and phylogenetic trees have increasingly been used to characterize disease outbreaks [24, 25]. Indeed, a keyword search by year in PubMed highlights the increased reference to GIS, molecular epidemiology, and social network analysis in publications relative to all indexed PubMed publication (Figure 1).

Figure 1. Increased Reference to Common Complex Data Types.

Keyword search for GIS, molecular epidemiology, and social network analysis in PubMed highlights the increase in these terms relative to all PubMed index articles. The frequency of other biomedical informatics terms (usability, electronic health record) is shown for comparison. Although the growth of social network analysis has been more recent, the inset shows that this concept has also experienced rapid growth in the published literature.

Tools for these three types of complex data allow public health professionals and researchers to integrate, synthesize, and visualize information pertaining to disease surveillance, prevention, and control. The ability to track disease distribution with GIS tools has helped public health professionals and researchers alike to detect disease clustering, analyze spread of disease in communities and across territories, and to predict outbreaks [26-30]. Surveillance of different strains of tuberculosis, influenza, and other diseases via characterization of molecular markers is commonly used to identify potential risk factors, pathogenicity, potential outbreaks, and prepare adequate interventions [31-36]. With the growth of network theory and the availability of modern computing, social network analysis and network-based epidemic models have been increasingly used to depict outbreaks and disease dynamics [37-40], identify potential cases and focus control efforts by prioritizing contacts [24], and evaluate strategies to interrupt transmission [40-42]. Together these data types can tell a compelling story about disease risk factors, spread and transmission, and can lead to more effective control measures and interventions.

However, this surge in surveillance capacity has produced more complex and disparate data, leading to new discussions about data sharing and interoperability, data confidentiality, and strategies for managing redundancies as well as incomplete data [1, 17, 29, 43-46]. For example, public health practitioners and researchers are faced with integrating diverse data sources such as mortality data (e.g. autopsy reports), clinical data (e.g. laboratory reports, immunization records), geographical data (e.g. address of work, residence, preschool), relationships (e.g. names of family, friends, partners), patient and pathogen genetics, medical imaging, travel plans, and timelines. Each of these types of information can be recorded, stored, accessed, evaluated, and displayed in many different systems and formats. Organizations are therefore challenged to maximize the potential of this flood of data to impact public health practice. Visualization tools have the potential to improve comprehension of this data by increasing the memory and processing resources available to users, reducing the search for information, enhancing the detection of patterns, and providing mechanisms for inference [47]. However, visualization tools also risk misleading users due to misinterpretation or cognitive overload [48, 49].

As such, funders and developers of visualization tools encounter a range of challenges when designing new tools for public health data, generating a growing collection of tools as new ideas and approaches are explored. However, these tools are often developed in silos, limiting their use in practice [50]. And despite the advances in public health informatics, many public health professionals still use visualization tools and data management systems that may no longer suit their current needs [6, 7, 51]. The unique focus of this systematic review on visualizing GIS, molecular epidemiology, and social network data for infectious diseases highlights the progress to date in public health informatics for infectious disease by identifying information needs and user preferences, characterizing features and system architectures of existing visualization tools, as well as identifying usability and adoption considerations. Finally, we explore commonalities among complex data types and underscore some of the challenges that lie ahead for novel visualization tool development.

2. Methods

This review explored the lifecycle of development and adoption of infectious disease visualization tools from conception to evaluation in practice. Infectious disease surveillance and control efforts encompass a wide variety of fields and require integration, synthesis, and analysis of information [21, 52, 53]. Consequently, we employed a sensitive search strategy for this review in hopes of capturing relevant literature from diverse fields. This review focused on the following objectives:

1. Identify public health user needs and preferences for infectious disease information visualization tools.

2. Identify existing infectious disease information visualization tools and characterize their architecture and features.

3. Identify commonalities among complex data types 4. Describe tool utility and usability evaluation efforts, and characterize barriers to the adoption of such tools.

To gain a comprehensive understanding of the current landscape of these tools, we identified articles published in English from January 1, 1980 to June 30, 2013 from the following bibliographic databases: National Library of Medicine's MEDLINE through PubMed, Cochrane Library, New York Academy of Medicine's Grey Literature, Web of Science, and IEEE Digital Library. Articles identified through additional manual searches were subject to the same inclusion criteria.

2.1 Selection Criteria

The final search strings were (A and B), (A and C and D), and (C and E and F), where each chain is defined as the following: (A) infectious disease OR public health data; (B) information needs OR task analysis; (C) visualization OR visualisation OR mapping; (D) molecular epidemiology OR social network analysis OR geospatial OR geographic OR GIS OR adoption OR utility OR outbreak OR surveillance OR disease mapping OR contact investigation OR usability OR functional requirements OR interactive OR real time OR needs assessment; (E) software; (F) usability OR adoption OR functional requirements.

To ensure a focus on the current landscape of infectious disease visualization tools, articles were excluded if the primary focus of the article was: clinical trials, decision-making aids, learning behavior, cognitive behavioral theory, disease or outbreak case studies, (health) information networks, data mining, concept mapping, systems mapping, programming language, ontologies and taxonomies, software methodology or framework, and resource mapping. Moreover, studies were excluded if the primary emphasis was: laboratory methodology, epidemic modeling or statistics, risk mapping, public health interventions, non-human infectious disease, architecture or system visualization, software case studies, and healthcare or medical treatment. The final set of articles were assessed for quality, with a focus on methods and risk of bias (e.g. selection, detection, reporting).

3. Results

Of the 247 articles we screened, a total of 88 articles are included in this review (Appendix) and the process is described in Figure 2 per the PRISMA guidelines [54]. The articles primarily included descriptive reports, qualitative studies (e.g. interviews, focus groups), and usability studies. None were excluded due to methodological deficiencies. The literature included in this review is comprised of articles from both US and non-US journals. The content was abstracted and the articles were organized into the following categories based on the primary topics discussed in the articles: information needs and learning behavior (n=18); architecture of existing tools (n=22); user preferences (n=20); features of existing tools (n=54); usability and evaluation (n=15); and implementation and adoption (n=27). These categories highlight the logical progression of novel tool development. Note that these categories are not mutually exclusive. Summaries of findings in each category are described in the sections below.

Figure 2. Flowchart of Literature Review Process.

*Off-topic exclusions: clinical trials, decision-making aids, learning behavior, cognitive behavioral theory, disease or outbreak case studies, (health) information networks, data mining, concept mapping, systems mapping, programming language, ontologies and taxonomies, software methodology or framework, and resource mapping

†Full text Exclusions: laboratory methodology, epidemic modeling or statistics, risk mapping, public health interventions, genome mapping, not human infectious disease, architecture or system visualization, software case studies, and healthcare or medical treatment

3.1 Information Needs and Learning Behavior

The types of information required by public health professionals have been studied in many contexts. The studies meeting our inclusion criteria offered several insights about information seeking behavior among public health professionals. While the public health workforce is extremely diverse [55-60] and public health information sources are often disparate and unstandardized [46, 55, 56, 60], several themes held constant. Public health professionals need timely access to current data from reliable, high quality sources [9, 55, 58, 59, 61, 62]. Furthermore, public health professionals need synthesized and collated data on relevant information such as best practices, effective prevention strategies or interventions, and evidence-based research, to name a few [9, 55, 59, 61, 63]. Public health professionals gather information from colleagues, literature and health departments [55, 57, 58, 62, 64]. However, multiple studies suggested that public health professionals are still often unaware of available information resources, and emphasized collaboration to improve search outcomes [56, 60, 63, 65]. Additional challenges associated with meeting information needs include external barriers (e.g. lack of time, sufficient staff), technological barriers (e.g. inadequate equipment, lack of internet access), internal barriers (e.g. stress, lack of confidence in ability to complete task, lack of training), and lack of trust in the information source [9, 55-58, 60, 62-65]. These studies suggested centralized access to reliable resources, as well as improved access to and delivery of timely information, as key to overcoming these barriers.

However, information needs specifically pertaining to information visualization tools have not been as well explored. Two studies explored the context in which participants learned about, used, and synthesized information from visualization tools through interviews and questionnaires with public health professionals [62, 66]. The first highlighted the importance of prior knowledge and intuition to give context to the results, and demonstrated participants' frustration with tools that were not intuitive or were too awkward for regular use [66]. The second study indicated that public health professionals spend less than 10 hours per month learning about new tools or methods for work and primarily learn about them from internet, literature, conferences and colleagues [62]. Participants wanted to know how the tool was developed and by whom (e.g. author's name(s), fields of expertise, credentials, affiliations) [62] as well as how the tool provided its results [66]. Additional studies also highlighted the importance of the users' perception of, and trust in, the tool's reliability as a potential learning barrier to new visualization tools [9, 49, 67]. In a study of user needs and preferences for visualization tools, sixty percent of users indicated they typically use more than one visualization tool for their visualization and analysis needs in a recent questionnaire [67]. This finding was supported in multiple studies wherein users indicated that no one existing tool or system met all their data needs [49, 55, 58, 66]. Further, studies indicated that users most commonly created static graphics, and many users relied on Microsoft Office suite [49, 62, 66, 67]. Collectively, these findings indicate many users are interested in learning about new tools in a time-efficient manner, and support an important relationship between user trust, tool credibility, and transparency.

Multiple articles raised concerns regarding interpretation of graphics, specifically misinterpretation of results and cognitive overload. For example, some users voiced concerns that data can be manipulated or unintentionally misrepresented due to confusion about how the tool works, or what type of graphic to use if given options [9, 49, 68]. Cognitive overload, wherein a user is presented with more information than they are able to successfully process, was addressed in several studies. This highlights the unique challenge of displaying complex and large datasets without reducing usability reaching the technical limits of the platform or the cognitive limits of the user [69, 70]. Strategies to minimize cognitive overload were less defined, although Herman [69] suggested human-centered design as a means of improving data visualization interpretation.

The data sources available for a given target end-user influence the architecture of the visualization tool. The next section explores common architectures reported by the articles included in this review.

3.2 Architecture of Existing Tools

We considered architecture to address the means by which a system was constructed in the software design sense, referring to the way in which system components fit together. Components may be individual classes in a software program or larger components, like a database management system, a web service, and the connections in between these components. Other features, such as interface design, operation workflow, functionality, features, visualization layouts, and analysis algorithms are often independent of underlying system architecture. These are covered in later sections of this review.

Several articles in this review made only cursory reference to system architecture. For example, some papers referenced use of specific components such as a particular database, management system, GIS, or statistics package [71-76]. Others alluded to particular architectural choices through discussion of other technical issues, for example the computational complexity of a statistical routine [73, 77]. However, these references alone gave little insight into the structure of the system as a whole. Some of articles in this review contained more significant coverage of system architecture, including a discussion of the general architectural design in terms of the number and function of system tiers [28, 29, 78, 79]. This may reflect the purpose behind many such publications, which typically focused on the utility of design features for public health purposes or the challenges inherent in linking data to visualization tools. One publication, however, explicitly described the structured application framework for Epi Info (SAFE), a set of application development guidelines to improve the software design and modularity of public health information systems developed using components from the Epi Info tool provided by the Centers for Disease Control (CDC) [80].

Web-based systems, or systems having some web accessible components, were the delivery platform of choice in many cases [28, 29, 46, 70, 72, 74, 77-79, 81-86]. These were often intended to permit distributed access by public health staff, reduce software implementation costs, or expose public health information for public dissemination. As such security and privacy was a frequently noted concern. Although privacy of health data was mentioned as a concern in many studies, only one article specifically discussed implementation of security protocols [84]. Others discussed methods for aggregating or otherwise de-identifying data [74, 79].

Total data volume, size of data transfer packets, or processing complexity in time or space were cited in a few studies [28, 29, 82, 87]. These articles suggested the use of data warehousing and caching as possible approaches to address processing time related issues, noting that it takes time to calculate statistical values for use in infectious disease mapping. Several studies also mentioned cost as a major factor affecting architectural component choices [28, 29, 79, 81, 82]. Presented solutions included using open source or free proprietary software, using free web resources like the Google Maps API [88], and building modular reusable components such as web services [28, 29, 79, 80, 82, 89]. Overall, there appears to be a trend away from standalone visualization systems, and toward modular, service-oriented architectures and web-based user interfaces.

3.3 User Preferences

User preferences highlight how users prefer to interact with a tool or system, and can provide insights into possible sources of usability issues or adoption barriers. Studies of academic researchers and public health professionals indicated a preference for tools that help users evaluate disparate and complex high-quality data [44, 46, 49, 62, 90, 91], with the goal of improving comprehension and communication, as well as facilitating decision-making [19, 44, 46, 49, 62, 66, 67, 90-92]. Additionally, participants in qualitative and quantitative studies emphasized the importance of user-friendly, reliable tools, with high-quality online documentation, and easy access to the source code [9, 46, 62, 67, 68, 93]. Users in a variety of settings raised concerns regarding interoperability of new and existing tools, data sharing, and data confidentiality [46, 49, 66, 68, 93]. Additionally, analysis of a survey conducted by Bassil and Keller [67] indicated that users in academic settings are nearly twice as sensitive to the cost of a new tool as are users in industry. This finding is consistent with many studies exploring or advocating for open-source and web-based infectious disease visualization tools to overcoming cost and resource barriers [62, 70, 73, 77, 79, 84-86]. Moreover, these preferences mirror key themes from Section 3.1, namely user trust, tool credibility and transparency.

A host of studies highlighted user preferences for data abstraction, each with the underlying theme of making complex data digestible and useful for users. Users expressed a strong interest in dynamic, interactive graphics that allow them to review their data at different levels (e.g. population or individual level) [19, 44, 46, 66, 67, 69, 90-92]. With such a function, users felt they could incrementally explore the data to evaluate both the big picture and the finer details. In addition, users valued common interface features such as zoom, pan, search, filter, save, undo, and work history [9, 46, 67-69, 91, 93]. Users also showed interest in high-quality automated layouts and customizable features (e.g. color, size, shape) to facilitate understanding of the data [67-69, 90]. Furthermore, some users demonstrated high interest in tools with multiple views or panels, enabling them to review their data from different perspectives [44, 46, 67-69, 91-95]. In concert, users preferred easy navigation between views and synchronized browsing (e.g. monitor the same variable across panels) [67, 68]. The ability to layer data, particularly among GIS users, was a common request to facilitate understanding of interactions or risk factors that overlap with disease outcomes [9, 46, 49, 93]. Overall, these preferences emphasize the importance of information discovery and synthesis through iterative data exploration.

Such preferences guide the development of infectious disease visualization tools, and can inform strategies for incorporating the tools into routine practice. The corresponding features and functions have the potential to help users discover complex or hidden patterns [96].

3.4 Features of Existing Tools

Having identified common information needs, system architectures, and user preferences, the following subsections explore existing tools and applications in more depth as they pertain to GIS, molecular epidemiology, and social network analyses. Each section also provides examples of common representations of GIS, molecular epidemiology, and social network data, respectively.

3.4.1 GIS

The development of increasingly sophisticated geographic information systems (GIS) has provided a new set of tools for public health professionals to monitor and respond to health challenges. These systems can help pinpoint cases and exposures, identify spatial trends, identify disease clusters, correlate different sets of spatial data, and test statistical hypotheses. Often, these analyses are aided by visualization and mapping of data, provided via web services or a user interface. Our review identified many approaches to delivering GIS functions based on various sources of public health data. Common functions among these studies and systems were geocoding [8, 72, 73, 79, 97], integrating data sources [72, 73, 98, 99], and cluster detection [84, 97]. Mapping of data was commonly achieved through dot maps (Figure 3A) [8, 46, 72, 73, 75, 76, 81, 84-86, 96, 99-106], choropleth maps (Figure 3B) [8, 44, 71, 75, 78, 95, 96, 99-102, 104, 105], and isopleth or gradient maps (Figure 3C) [8, 76, 81, 87, 107]. Recurrent considerations cited within these papers included the privacy of public health data [29, 75, 79, 82, 101], the alignment of GIS analytics to users' needs [76, 96, 101, 103, 105, 108], the motivations to make analysis services accessible, and the interoperability of data or system [8, 29, 79, 101]. Since many GIS analytical services and geographic data are available through providers such as ESRI, Google, or the U.S. Census [88, 109] , GIS systems in our review often utilize an architecture based on these services and map data.

Figure 3. Common geographic (GIS) visualizations.

A dot map (left) uses dots to represent a certain measure or feature displayed over a geographical map. They are often used to present the geographical distribution of various disease cases in infectious disease surveillance. This figure represents hypothetical infectious disease cases in the state of California. Each dot represents a specific disease case. These maps may help identify clusters of disease. In interactive tools, users may click individual cases or select subsets of cases to obtain further information. Individual level data is often aggregated in a choropleth map (middle), which uses graded colors or shades to indicate the values of some aggregate measure in specified areas. This figure shows the incidence rate per 100,000 persons of cases from map (left). Differences in the incidence rates by county are indicated with different shades (green), with a darker color indicating a higher rate. Interactive choropleth maps allow selection of regions to obtain additional information. Individual or aggregate level data may be used to statistically derive a spatial risk gradient (right). Other visualization features may allow zooming/panning of maps, introduction of other map layers such as roads, or selection of color scales.

The systems reviewed were designed with various targeted users in mind. Two broad divisions of these were systems intended for public access using publicly available data, and restricted systems intended for users with access to private public health data. In many cases, these systems cited the use of publicly available maps and cartographic data as a basis for spatial integration of other information [72-74, 98, 101]. Many systems utilize administrative geographic units as a basis to merge data across different health and population databases, for example to calculate incidence rates based surveillance data and a population census. Other approaches may either map other sources into an internal data model [77] or to an ontology that supports data integration [29].

Visualization methods for GIS in public health in our review focused on functions geared toward simplifying, integrating, or analyzing data in a spatial context. The simplest visualizations plot or aggregate spatial data to deliver static point or choropleth maps of individual or aggregate data, respectively. Many systems incorporated a temporal component, enabling either animation of data through time or restriction of the data displayed to a time window of interest [77, 79, 84, 110]. A step beyond mere display of information, some GIS or spatial statistical methods seek to perform kernel-based smoothing to estimate risk maps [87, 97, 107], visualize disease risk according to a statistical model [29, 46, 76, 81, 85, 86, 107, 111, 112], or compare one feature to another [71, 84, 97, 100, 101]. While the ability to zoom and pan to navigate maps [79, 96, 105] is a common interactive feature enjoyed by users, more advanced systems contain interactive controls to enable users to retrieve information about selected items or regions, visualize the results of arbitrary queries [79], control visualization options, control temporal ranges of data returned [77, 79], or link displays of data with alternate or comparative visualizations [78, 79].

3.4.2 Molecular Epidemiology

Molecular epidemiology is concerned with understanding the distribution or clustering of genetic variants, strains, serotypes, or other molecular groupings of pathogens. In molecular epidemiology, relationships between isolates are often calculated and conveyed through phylogenetic trees or dendrograms (Figure 4). Visualization tools for molecular epidemiology often included phylogenetic analysis and visualization capabilities [113-115] and visualization of contextual data using connected graphs [70, 113]. The tools we reviewed were primarily designed to be accessed through the internet [70, 113, 114, 116]. Most studies in our review included the capability to integrate GIS or location-based data with genetic or serotype visualizations [11, 70, 84, 113-117]. Two of the tools were designed to produce visualization (KML) files for display in other GIS packages [11, 115], while other web based tools made use of external GIS services embeded within the website, primarily Google maps, ESRI/ArcGIS or HealthMap [70, 116, 117].

Figure 4. Dendrogram.

A dendrogram, or phylogenetic tree, is a branching diagram or “tree” showing the evolutionary history between biological species or other entities based on their genetic characteristics. Species or entities joined together by nodes represent descendants from a common ancestor and are more similar genetically. This figure shows a hypothetical example of a rooted dendrogram, wherein the horizontal position of individuals represents the genetic distance from a specific progenitor. With the advancement of DNA sequencing technologies, phylogenetic trees have been used widely in infectious disease control to depict the genetic similarities and differences between strains and variants of a certain disease pathogen. Knowing whether infectious diseases occurring in different areas are from the same strain provides key information on the source of infection and how the disease may been transmitted. Interactive features of these visualizations may include the ability to collapse or color/label branches.

Some tools were designed with specific organisms in mind, for example staphylococcal [117] or influenza [116] infections. Driscoll et. al. [70] developed Disease View, a set of tools to understand host-pathogen molecular epidemiology. They demonstrated the use of this tool to analyze aspects of the Vibrio cholera outbreak that occurred in the aftermath of the 2010 Haiti earthquake. These tools allow spatial views of molecular epidemiological properties associated with outbreaks, for example showing sequence variation of genes associated with disease virulence between outbreak locations. Other tools were designed to accommodate multiple organisms or user-specified organisms [113-115]. One such tool, designed specifically for geospatial surveillance of genomic characteristics of NIAID category A–C viral and bacterial pathogens, is GeMIna [11]. This tool collects curated meta data relating to the diseases. Other views of the distribution of genotypes across a large geographic scale help to understand the relationship between the population biology and geography of a pathogen species [118](He et al). This is sometimes known as phylogeography.

As with GIS systems, data integration was a key component of the web based tools, with all web based tools incorporating access to or prepopulated with existing sets of data or meta-data, including pathogen, isolate and sequence data. Several studies discussed approaches for integration of genetic and social network data [35, 36, 38, 89, 119]. In the absence of known exposures between cases, or in the case of ineffective contact investigations, molecular epidemiology or genomic approaches can identify potential members of an outbreak cluster. These studies showed social network data alongside genetic data using custom visualizations, but tools with the capacity to visualize the interplay of these data types systematically are still being developed.

3.4.3 Social Network Analysis

In addition to geographic and molecular epidemiologic data, networks of social contact or disease exposure are a third type of complex data that are increasingly being used to understand disease outbreaks. As shown in Figure 1, social network analysis as a field is growing relative to health literature as a whole; however, it is at an earlier stage than for the other two topics. In order to describe the use of social network visualizations for public health, we therefore considered a broader set of publications that often described visualizations of single outbreaks or analyses, in addition to those directly describing tools used to visualize outbreak networks. Nevertheless, these publications inform desiderata for visualizations of these networks, which in turn inform the features or design requirements such systems should consider. Applications of social network analysis in public health typically focus on routes of infection in communicable disease contact investigation; hence, most of the publications in our survey address this topic.

Although only eight articles were directly pertinent to social network analysis, these papers did address a variety of uses of and challenges for the application of network analysis for infectious disease control. Common purposes of network analysis included risk stratification of contacts, identifying common characteristics of those infected, visually communicating or mapping cases for improved understanding of outbreaks, and identifying potential pathways of transmission [24, 38, 120]. Among the considerations for data visualization addressed by these studies, several common features were observed: use of shape, color, and graph position to convey information [24, 38, 89, 120, 121]; display of individual case features or identity; and identification of important clusters or paths in the network [24, 89, 120]. In more advanced analyses, studies may seek to compare or estimate networks across other variables, such as including a temporal dimension in the study [38, 83, 89, 120, 121] (Figure 5); integrating geographic or location features [83, 89, 110, 120, 121], or identifying exposures via molecular epidemiology as discussed in the previous section.

Figure 5. Social network diagram.

A social network is a graphical representation of social relations or exposures consisting of nodes (individuals within the network) and ties (relationships between individuals). Nodes are usually represented as points or other shapes while ties are represented by lines between the nodes. Differences in the shapes or lines of the diagram may be used to represent different characteristics of the individuals or the relationships. This figure shows a hypothetical example of a force-directed social networks diagram. Social networks analyses in infectious disease control have been gaining importance in the past decade. Examining these social relationships between disease cases and their secondary contacts may be beneficial to tracking the spread of infectious diseases within interconnected social networks. It is especially useful in identifying the index/source case and predicting which individuals are more likely to become infected and further infect others.

Consistent with our findings in other sections of the study, other important considerations recognized within the network analysis studies focused on the importance of designing network visualizations that provide the right information to users without confusing them. These considerations took the form of discussions about information overload from complex graphs [120], the inclusion of diverse user preferences for visualization [120, 121], and the importance of training to help users understand and utilize these graphics [24, 89]. Viegas and Donath [121] and Hansen et. al [120] studied non-standard network layouts, and included user assessments to help evaluate how these could best be used. Although most publications discussed the use of networks in a disease control context, Andre et. al. [24], Cook et. al. [89], and McElroy et. al. [38] explicitly described how network visualizations could be used to aid decision-making via prioritization of resources or investigations. Other less common considerations for network analyses described in our review include the use of repeated contacts as a heuristic for risk, studies of population mixing [24], the use of touch-screen interfaces to navigate networks [120], the importance of aggregated data visualization options to prevent information overload [120], and the use of simulation to augment missing data [83].

Mostly absent from these studies were visualization methods to help users understand network structures at an aggregate or summarized level, comparable to the choropleth map in GIS. Although visualizations like collapsed nodes, flow diagrams, and network metric distributions (such as node degree distribution) [122] have been used in other domains, these techniques may not yet be familiar interfaces for lay users, and hence have not been widely employed in tools for public health. As network data become increasingly integrated with GIS, molecular epidemiology, and other health indicators, evaluation of more diverse methods of network visualization consistent with end-user preferences, training level, statistical literacy, and cognitive ability will be needed.

3.5 Usability and Evaluation

The usefulness of a system is often used to describe a system's overall effectiveness. The concept of usefulness can be measured as a combination of utility and usability. Traditional system evaluation has focused on utility, determining whether an information system is able to meet the functional requirements of a user who wants to accomplish a specific set of work tasks. This is demonstrated in studies which evaluate information systems based on a strict set of functional metrics, such as accuracy and efficiency [77].

In addition to evaluating system functionality, it is becoming increasingly important to evaluate system usability. Some researchers have conducted usability evaluations to provide justification for the time and effort spent developing and deploying these complex tools [123]. In addition, the intended benefit of many information systems is to facilitate interaction between users and data, and so usability itself is the primary measure of system usefulness we examined have revealed that [124]. However, the articles features which improve the usability of one system cannot always be generalized to other systems, since different users may have different task-specific system requirements [62, 93].

Even though specific design recommendations may not apply broadly across systems, studies cited common methods to reliably evaluate the usability of a system. These methods include the use of qualitative investigation techniques, such as participant observations, interviews, and workflow analysis [51, 125, 126]. Participant observations involve watching users as they perform their work, during which researchers have encouraged users to “talk aloud” during interactions with information systems. These observations have served as the basis for semi-structured interviews and focus groups used to obtain in-depth descriptions of user behavior [51, 125, 126]. Published studies also describe the use of interviews to highlight areas for further investigation, either by pinpointing particular aspects that a user does not like, or by uncovering new interactions that a user would like to see added [125]. In addition, observations and interviews have been combined with questionnaires containing Likert scale questions, asking users to rate their satisfaction with information systems [46].

However, researchers acknowledged that efforts to simplify interactions between users and data may have the unintended consequence of limiting functionality [62]. For this reason, some researchers found it important to engage users in the design and development processes. This was accomplished by employing usability evaluation techniques in conjunction with participatory design methods, allowing feedback to be incorporated into the system throughout the development process [8, 70, 125, 126]. Researchers also expressed interest in studying user work behaviors over longer time periods [93], an aspect which might be addressed by soliciting feedback during an ongoing participatory design process [91, 124].

3.6 Implementation and Adoption

Barriers to adoption vary widely and are not mutually exclusive within a given organization or individual. System-level barriers, such as access issues (e.g. lack of internet or finances) and lack of organizational support were significant barriers in organizations worldwide [8, 9, 47, 49, 57, 58, 79, 99, 101, 127, 128]. Jurisdictions often struggle to share data due to lack of data standardization (e.g. data heterogeneity, missing data, lack of interoperability) and face data confidentiality concerns which collectively compound the already-complex task of monitoring diseases [8, 9, 29, 46, 49, 73, 99, 127, 128]. Furthermore, user-level concerns may also result in adoption barriers. Confusion regarding how to create or use effective graphics, and a lack of familiarity with the concepts in the tool could be substantial learning barriers [47, 58, 99]. Fear of change and an interest in staying within one's comfort zone, in addition to a lack of trust and misconceptions about the use of the tool, may also prevent adoption of a valuable tool [47, 49, 68]. Indeed, studies indicated that many users relied on other tools (e.g. Mircosoft Office suite) because they felt that many existing tools were too complex and had a substantial learning curve [8, 9, 47, 68, 79, 85, 96, 99, 128].

Despite the potential for data visualization tools to monitor and aid control efforts for infectious diseases, such tools have had only limited adoption [49, 97] and only one system was assessed for distribution [129]. Usability studies and implementation projects are remarkably interdependent, as successful adoption often requires developers to re-design elements of the tool to further address the users' needs [8, 70, 125]. The resulting iterative design process often helps users identify previously unexpressed or unknown information needs [93], resulting in the need for subsequent usability studies. However, this process can be time consuming, and users may find alternative systems that meet their current needs before the tool is completed [8, 127]. Moreover, existing tools are largely isolated to the jurisdictions and organizations that developed them and may be based on proprietary systems [8, 46]. Such silos could prevent the widespread adoption of tools by other agencies or organizations.

While the specifics of adoption strategies may vary depending on the particular organization or agency and their needs, some common strategies emerged from the literature review. Several studies recommended ongoing user collaboration with the tool developers to ensure that the users' needs were heard early on in the project, and to create the opportunity for regular feedback [8, 9, 68, 79, 125, 127]. Further, studies advocated for open source tools to reduce access barriers, particularly in low-resource settings [71, 79, 99, 127]. Integrating the tool into existing workflow was also recommended as a strategy to encourage users to regularly utilize the tool [8, 47, 51, 68, 99, 124]. Additionally, providing adequate user training and education, as well as ongoing technical support, for staff was considered essential for successful adoption of a novel tool in many studies [8, 46, 47, 49, 62, 79, 80, 92, 96, 99, 101, 124, 127, 128]; effective user training may build the users' self-confidence in the use of the tool and encourage them to try the tool [125, 127]. In concert, these strategies may create an environment for sustained use.

4. Discussion and Conclusion

In this review, we assessed the current landscape of visualization tools developed for infectious disease epidemiology. We characterized these tools in terms of information needs and user preferences, features and system architectures of existing tools, as well as usability and adoption considerations. By focusing on visualizations of GIS, molecular epidemiological, and social network data, we also explored similarities among these three types of increasingly common data types. The richness of the information offered by these data for communication and decision making are counterbalanced by difficulties in displaying, interpreting, and trusting these data sources. In our review of tools throughout their lifecycle from conception to development to adoption, several themes and challenges emerged pertaining to both individual stages as well as broader topics. Despite the different scholarly approaches of the included articles, the following themes emerged: (1) importance of knowledge regarding user needs and preferences; (2) importance of user training; integration of the tool into routine work practices; (3) complications associated with understanding and use of visualizations; and (4) the role of user trust and organizational support in the ultimate usability and uptake of these tools. Another broader theme that became apparent is that individual tools and datasets are rarely sufficient, even for local decision making. Therefore, interoperability of tools and the importance of data sharing and integration were important goals that should factor into the design of visualization tools.

The utility of visualization tools is constrained by the extent to which they address the information needs of users. Information needs are as complex and varied as the tasks performed by public health professionals. Consequently, developing information visualization tools to meet these needs is correspondingly complex. Indeed, developers have addressed information needs in a multitude of ways, resulting in the current diversity of data visualization tools, each serving as a case study for one approach to resolve these needs. Regardless of the study population, users indicated that they needed timely access to reliable, high-quality information to perform their duties. Efforts to map users' queries of common data types (e.g. GIS, molecular epidemiology, and social networks) to meaningful visualizations have raised concerns regarding the potential for misinterpretation and cognitive overload due to the complexity of infectious disease data [130].

Despite results from studies with users emphasizing the value of dynamic, interactive graphics to facilitate data exploration and abstraction, existing tools are largely still static. And while static graphics are extremely useful, pairing them with interactive features may give users more freedom to explore and learn from their data. Sophisticated data analysis and visualization systems, such as R [131], SAS [132] and Matlab [133] have traditionally enabled expert users to create hard coded (but rapidly adjustable) graphics using code. The increasing use of these platforms to create user-friendly, interactive, web-based versions of these visualizations through technologies such as scalable vector graphics (SVG), dynamic HTML (DHTML), and Shiny [134] has the potential to greatly simplify users' access to interactive, web-based visualizations. The distinction between visualization tools requiring coding and online visualization tools is also somewhat blurred by the ability to embed fully functional data analysis and visualization within web applications, as has been done using RStudio [135] to allow the use of R within the Centers for Disease Control and Prevention's BioSense surveillance system [136].

In addition to BioSense, several well-known surveillance systems are not included in this review, including the Centers for Disease Control's EARS (Early Aberration Reporting System) and Johns Hopkins's ESSENCE (Electronic Surveillance System for the Early Notification of Community-based Epidemics). These systems have a limited representation in scholarly literature (as they are commonly developed and evaluated internally) and are not discussed in terms of visualization features. For example, a published evaluation of the EARS system focused chiefly on its aberration detection algorithms [137]. Consequently, they were not captured in the scope of this review. However, these systems face many of the same constraints as those discussed in this review: data standardization and quality in diverse jurisdictions, limitations of user knowledge and organizational capacity to implement the tool, as well as generation of accurate and easy-to-understand visualizations.

Visualizations with interactive features or sophisticated visual elements may require sufficient rendering capability and user experience to maximize their potential. For example, to access an area of interest in a 3D representation, users will typically need to adjust other visual cues (e.g. rotate the graphic, change transparency or depth queuing) [69]. Koenig et. al. [92] explored visual perceptions among public health users in GIS environments and demonstrated a preference for a blue and red color scheme to represent health and morbidity, respectively. However, studies emphasized that color schemes and visual elements should be sensitive to multi-cultural users, users with color-blindness, and rendering limitations of existing systems [46, 67, 92]. These visual elements also contribute to data (mis)interpretation. Consequently, guidelines have been proposed for color schemes and visual elements to minimize the risk of misinterpreting the data. For example, use of single-hue color progression (e.g. white to dark blue) to show sequential data is more intuitive than spectral schemes (e.g. rainbow) that force users to assign arbitrary magnitudes to rainbow colors [138].

Together with utility (functional effectiveness), usability (perceived ease of use) is sometimes considered to be a core component of determining the overall usefulness of a system. This makes usability one of the dimensions that can contribute to the adoption of a new information system [139]. Usability has been assessed by examining several dimensions including learnability, memorability, error prevention/recovery, efficiency, and user satisfaction [140]. However, usability also varies depending on the specific information needs of an individual user, particularly because efficiency depends on the task being performed. This presents an interesting problem when trying to highlight best practices with regards to usability. After a system has been developed, usability evaluation techniques can be used to assess its overall usability. The evaluation can contain quantitative assessment of accuracy and time efficiency as compared to a previous system or suitable alternative, such as a spreadsheet or database. With a sufficient pool of users and clearly defined metrics, a usability evaluation can yield statistically significant results, although this is not necessarily meaningful when assessing qualitative aspects, such as user satisfaction and perceived learnability.

Further, there was little discussion in the included literature about how to organize and sustain the implementation phase. The literature highlighted minimal success in widespread implementation and adoption of data visualization tools. While substantial barriers exist, there are strategies to address many of them, including obtaining management support, providing ongoing user training support, and starting a pilot program to integrate the tool into existing workflow. However, with extensive variability in data management systems, needs, and attitudes, widespread adoption of a given tool is difficult task. For example, integrating the novel tool into a given workflow requires collaboration between agencies and organizations, qualified staff for observation and interview studies, and time. Due to the variability of site structure, and thus workflow, the optimal implementation strategy may vary, limiting the desired widespread adoption. Consequently, implementation becomes a site-specific endeavor, rather than a one-size-fits all task. The participatory design approach can increase the amount of exposure that users have to the system, allowing for a better approximation of usage habits over time and understanding of the users' needs. Obtaining management support and creating a pilot implementation project may benefit from theory-driven communication campaigns to raise interest and support. For example, the literature supports a highly variable knowledge of and support for data visualization tools among management and staff. Behavior change theories, such as the Stages of Change Model [141] or the Diffusion of Innovations [142], may improve adoption rates by targeting messages to different populations based on their readiness and interest in adopting the novel tool.

Many studies highlighted the importance of adequate and ongoing training for users, providing a possible avenue to explore in more depth to minimize the risk of misinterpretation as well as improve adoption. In a recent study, more than half of the participating public health professionals indicated they were likely to seek training in a variety of tasks, including data visualization, epidemic modeling, GIS, cluster analysis and statistical modeling [62]. They also preferred a variety of training styles (e.g. task-oriented tutorials, user guides, and hands-on training). Such training opportunities may also improve the perceived transparency of the tool. Further, integrating user training time, cost of the tool, and support staff into site budgets may also encourage more consistent, trained use of the tool. An atmosphere supporting regular use of the tool can encourage users to spend more time learning about the tool's features and functions while helping them become more savvy, creative, and comfortable with the tool [96, 125]. However, few of these studies addressed the growing need for enhanced statistical education to enable users to better understand their data in more depth. For many non-expert users, a trade-off is often made in favor of easy-to-use, “black box” programs instead of in-depth understanding of the limitations of data analysis. The desire for a system that allows users to query the data and receive results in plain language may undermine the very nature of complex data. Future research should endeavor to help users strike a balance between in-depth understanding of data and system usability.

Lastly, pragmatic constraints of widespread tool adoption, including funding considerations, jurisdictional constraints, as well as data sharing and confidentiality concerns, may prove more difficult to overcome. Public health organizations worldwide face technological and financial access barriers preventing them maximizing the potential of visualization tools for epidemiology. Finite funding streams often force organizations to adapt existing systems that may not best serve their needs. Further, jurisdictional constraints and data sharing concerns create information silos, leading to reduced data potential. Infectious diseases do not follow jurisdictional boundaries, and new policies are needed to increase secure data sharing across organizations to facilitate decision making and improve distribution of resources. Public health organizations need more funding to explore customizable visualization tools for infectious disease that include public health users throughout development. Such work would further inform best practices for visualization tools of the complex data types public health professionals are expected to synthesize and act upon.

4.1 Future Directions

As data types and sources become increasingly large and complex, so too should the strategies to integrate disparate and often incomplete data into novel visualization tools. Concerns regarding data quality and accuracy are particular relevant for visualization tools as these tools can be limited by the inputted data. Discussions of current data limitations highlight issues of scale and uncertainty, accuracy of datasets for spatial and epidemic models used in tools, and the impact of residential address errors in geocoding, to name a few [143-147]. In order to draw meaningful and accurate conclusions from the data, visualization tools should represent missingness and uncertainty clearly. For instance, a recent study demonstrated that participants interpreting graphics with missing data tended to misinterpret results, but with equal confidence in their interpretations as those viewing more complete graphics [148]. Similarly, geographic analyses are known to be sensitive to overestimation of rates in small populations, which often correspond to large, sparsely populated regions, resulting in visual biases in interpreting choropleth maps [130]. These results suggest that users may not be aware of the need for better representation of missingness and uncertainty, and studies to evaluate the best means of doing so are still in their infancy. Continuing research on visualization algorithms that account for missing and uncertain data is needed to overcome these hurdles.

Another important challenge for future developers of information visualization tools for public health is to focus not only on individual user needs and comprehension of graphics, but also to plan and develop these tools in the broader contexts of available data, existing algorithms/services, team collaboration, and inter-organizational and interdisciplinary needs. Too many software projects are developed as new information silos, resulting in redundancy of effort, failure to integrate data and tools, and challenges to training and adoption. Further, many existing systems (e.g. BioSense) are access-restricted, limiting their use in infectious disease epidemiology, and may not have completed (or shared) evaluations of their visualization features. Visualization tools of the future should be developed to be compatible with existing data formats and standards, and interoperable with each other. Future tools should also adapt to the increasing pressure to be open-access, allowing users from low-resource settings, academia, and industry to capitalize on the advances in surveillance and visualization technology. This level of interoperability could support more advanced features such as phylogeography (the study of genetic variation across geographic space), inference of person-to-person contact from molecular epidemiology, statistical cluster detection based on joint spatiotemporal and genomic data, integration of remote sensing and environmental data, and other tasks as users become increasingly savvy in their use of advanced analytical and visualization tools for public health.

4.2 Limitations

Although this systematic review covers a wide range of visualization tools for infectious diseases, there are three main limitations. First, while the scope and search terms of the review were purposefully broad, we likely missed relevant articles. Second, many public health informatics tools, if described in manuscripts at all, may be published in non-indexed conference proceedings, and thus more recent or undersold tools may not have been retrieved. Further, systems or informatics needs assessments that have no associated publications on their visualization features (e.g. developed and used in practice only) were not readily available for our study, and systems with access-controlled content could not be assessed in context with the other tools identified here. For example, the Centers for Disease Control and Prevention's Public Health Information Network (PHIN) and BioSense as well as the International Society for Disease Surveillance have non-indexed content that the review did not capture. Lastly, we focused on English articles for practical reasons, but by doing so we may have excluded valuable contributions from teams around the world. However, our review included English articles in journals worldwide.