Information visualizations of symptom information for patients and providers: a systematic review

Abstract

Objective

To systematically synthesize the literature on information visualizations of symptoms included as National Institute of Nursing Research common data elements and designed for use by patients and/or healthcare providers.

Methods

We searched CINAHL, Engineering Village, PsycINFO, PubMed, ACM Digital Library, and IEEE Explore Digital Library to identify peer-reviewed studies published between 2007 and 2017. We evaluated the studies using the Mixed Methods Appraisal Tool (MMAT) and a visualization quality score, and organized evaluation findings according to the Health Information Technology Usability Evaluation Model.

Results

Eighteen studies met inclusion criteria. Ten of these addressed all MMAT items; 13 addressed all visualization quality items. Symptom visualizations focused on pain, fatigue, and sleep and were represented as graphs (n = 14), icons (n = 4), and virtual body maps (n = 2). Studies evaluated perceived ease of use (n = 13), perceived usefulness (n = 12), efficiency (n = 9), effectiveness (n = 5), preference (n = 6), and intent to use (n = 3). Few studies reported race/ethnicity or education level.

Conclusion

The small number of studies for each type of information visualization limit generalizable conclusions about optimal visualization approaches. User-centered participatory approaches for information visualization design and more sophisticated evaluation designs are needed to assess which visualization elements work best for which populations in which contexts.

INTRODUCTION

Information visualization is a key aspect of informatics and highly relevant to health care.¹ Information visualizations (ie, techniques that support the understanding of abstract data²) have been used to present rich, complex information in ways that facilitate patient understanding of information and patient-provider communication.³ Information visualizations also have been shown to reduce information overload and improve recall.^4,5

To date, information visualizations for patients and providers have been most widely studied in health risk communication.^6–8 Two recent literature reviews of studies of visualizations for health risk communication noted that icons, graphs (eg, line, bar), and other types of visualizations (eg, pie charts, maps, photographs) have been used for communication.^6,8 They reported that pictographs/icon arrays and bar graphs hold some promise for improving comprehension among users.^6,8 Similarly, Garicia-Retamero and Cokley conducted a systematic review to evaluate the benefits of visual aids in risk communication. They reported that 87% of the studies reviewed showed that static visual aids tend to be beneficial. In addition, 75% of the studies that investigated the effect of static icon arrays found that they tend to be particularly helpful in improving accuracy of risk understanding and recall.⁷

A less frequently studied area is using information visualizations to return individual research results to participants.⁹ However, the All of Us precision medicine initiative is a key driver for escalating research in this area, given the intent to return a broad variety of information about genes, environment, and lifestyle (including symptom status) to All of Us participants.⁹ Research by multiple authors has addressed this context. For example, in a series of studies, Arcia and colleagues have reported the application of participatory design and other informatics methods to create and return individual research results through information visualizations to Latinos who participated in a large community survey.^3,10–12 In the domain of returning laboratory results to patients, Zikmund-Fisher and colleagues have developed and tested a range of information visualizations, including icon arrays and color scales, to communicate risk to patients.^13–15

The use of information visualizations has increased as the technologies utilized to create them have become more widely available and easy to use. These include modules integrated into electronic health records such as reporting software used for clinical dashboards,^16,17 standalone specialized modules such as the Electronic Tailored Infographics for Community Education, Engagement, and Empowerment (EnTICE³) program,^10,11 visualization software (eg, Tableau, R), and interactive websites such as Visualizing Health (vizhealth.org), Icon Array Generator (iconarray.com), or Chart Chooser (labs.juiceanalytics.com/chartchooser/index.html). A recently released National Academies report on return of individual biomarker results to research participants highlighted the importance of strategies that facilitate understanding of the meaning and limitations of the results and recommends leveraging new and existing health information technology to enable tailored, layered, and large-scale communications.¹⁸

An understudied, but important area given the prevalence of symptoms, is the use of information visualizations for communication of symptom status for patients and/or providers. Symptoms are burdensome and difficult to manage.¹⁹ Management of symptoms relies heavily on effective communication between providers and patients. In the area of pain, evidence suggests that ineffective patient-provider communication influences poor pain symptom management and may contribute to the opioid crisis.^20–22 More generally, patient-rated physician communication quality has been positively associated with patient-rated symptom management quality; patients who rated their physician’s communication as high were more likely to report their symptom management needs as being met.²³

A substantial body of literature has documented that there are disparities in assessment and treatment of symptoms.^24–28 Research has shown that healthcare providers fail to recognize patient symptoms 50% to 80% of the time during visits.^29–33 In addition, studies have shown that assessment and treatment of symptoms vary based on patient race and ethnicity. Racial/ethnic minorities, including Hispanic and black patients, are more likely to report having unmet symptom management needs compared to non-Hispanic, white patients.^34,35 Overall, targeted interventions are needed to improve communication of symptoms between providers and patients in order to reduce disparities in symptom management.

Given the priority of symptom science in its strategic plan, the National Institute of Nursing Research (NINR) has designated a set of symptoms as common data elements (CDEs) and recommended measures for those symptoms: pain, fatigue, sleep, affective mood, affective anxiety, and cognitive function.³⁶ As interest in symptom and information visualization research has increased,¹⁹ a better understanding of how the use of information visualizations can improve symptom assessment and reporting for self-management and communication between patients and providers is an essential foundation for improved symptom management. However, gaps in the literature remain. For example: Which symptoms have been studied using information visualizations? What populations (both patient and provider) have been the target of information visualizations and what are their characteristics? What types of information visualizations have been studied? How have the information visualizations been evaluated? To our knowledge, there has been no systematic review of the research in this area.

Therefore, the purpose of this systematic review is to examine and synthesize the research literature on information visualizations of symptoms included as NINR CDE symptoms for use by patients and/or providers.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was used to guide our reporting.³⁷

Search strategy

We searched for publications within CINAHL, Engineering Village, PsycINFO, PubMed, ACM Digital Library, and IEEE Explore Digital Library between 2007 and 2017. We used a set of search terms that were relevant to the query options for each database and included NINR CDE symptom terms (eg, “pain,” “fatigue,” “sleep disturbance,” “cognitive function,” “anxiety,” “depression,” “depression symptom,” “affective symptom,” or “mood”) and information visualization terms (eg, “visualization*,” “visual*,” “graph,” or “infographic”) (Supplementary Table S1). As a means to focus and narrow the symptom content of this review, we selected symptoms designated as NINR CDEs. Search yields were uploaded to Covidence, a web-based software platform that facilitates systematic reviews (covidence.org). Additional articles were identified using the reference lists of articles selected for inclusion based on database queries.

Inclusion criteria

Studies were eligible if they: (1) were full text, (2) peer reviewed, (3) written in English; (4) included information visualizations; (5) focused on 1 or more NINR CDE symptoms for use by patients and/or providers, and (6) used qualitative or quantitative methods to assess the information visualizations. Studies were excluded if the visualizations focused on radiologic imaging, presentation of research findings, and data collection or assessment tools. Studies were independently reviewed for inclusion by 2 of the authors (ML, TAK). Conflicts were resolved through discussion.

Quality assessment of study methodology

We used the Mixed Methods Appraisal Tool (MMAT) to evaluate the methodological quality of the studies reviewed.³⁸ MMAT is designed to allow researchers to concurrently appraise quantitative, qualitative, and mixed-methods research and produce comparable scores across study designs. The study quality appraisal score is determined by dividing the number of criteria met by the total criteria in each applicable domain. Application of the MMAT has demonstrated high reliable inter-class correlation ranging from 0.84 to 0.94.^38–41 Two reviewers (ML, TAK) independently reviewed and calculated scores for each study. Discrepancies between the reviewers were resolved by discussion and reviewing the studies again.

Quality assessment of visualizations

We characterized the information visualizations by assessing whether or not the study included: (1) an image(s) of the visualization, (2) a description of how the visualization was designed, (3) a description of how the visualization was presented to patients and/or providers, and (4) a description of how the visualization was evaluated. A point was awarded for each criterion that was met, and the total score was converted to a percentage.

Data extraction

We extracted the following information from each study: (1) methods (eg, design, sample); (2) visualization type (eg, graph, icon) and delivery medium (eg, web-based, paper); (3) visualization evaluation methods; and (4) study findings. The findings were organized according to the subjective (ie, perceived usefulness, perceived ease of use) and objective categories (ie, efficiency, effectiveness) of the Health Information Technology Usability Evaluation Model (Health-ITUEM).⁴² Findings that could not be classified into Health-ITUEM categories were initially classified as “other” and subsequently categorized as subjective preference or intent to use.

RESULTS

Of 18 studies that met our inclusion criteria (Figure 1), 9 used mixed methods,^43–51 7 used quantitative descriptive methods,^52–58 1 used qualitative methods,⁵⁹ and 1 used a quantitative randomized controlled design⁶⁰ (Table 1). Ten of 18 studies met all MMAT criteria (Supplementary Table S2).

Figure 1.

PRISMA flow diagram.

Table 1.

Characteristics of studies reviewed

	Demographic characteristics of studies				CDE Symptoms Studied
Study	Design	Population	Sample size (n)	Age (years)	Fatigue	Pain	Sleep/Sleep disturbances
Ameringer et al. 201543	Mixed methods	Patients (oncology)	76 Patients	xŵ=18.5±4.2		✓	✓
Brundage et al. 201744	Mixed methods	Providers (physicians, nurse practitioners, physician assistants)	233 Providers	Not reported	✓	✓
Davis et al. 200852	Quantitative descriptive	Patients (oncology)	64 Patients	Patient: x̄=64±12.4	✓
		Providers (physicians, nurses, pharmacists)	22 Providers	Provider: Not reported
Duke et al. 201060	Quantitative randomized control trial	Providers (physicians)	24 Providers	Not reported	✓
Hochstenbach et al. 201653	Quantitative descriptive	Patients (oncology)	11 Patients	Patient: x̄=53±15		✓
		Providers (nurses)	3 Providers	Providers: x̄=52±2
Ismail et al. 201654	Quantitative descriptive	Patients (women experiencing menopausal symptoms)	30 Patients	Range 40-60	✓	✓	✓
Jamison et al. 201755	Quantitative descriptive	Physicians (pain)	90 Providers	x̄=47.1±13.5		✓
Kourtis et al. 201156	Quantitative descriptive	Patients (asthma)	84 Patients	Not reported			✓
Kuijpers et al. 201657	Quantitative descriptive	Patients (oncology)	548 Patients	Patient: x̄=60.6±12.3	✓	✓	✓
		Providers (nurses, physicians, paramedical professionals)	227 Providers	Providers: x̄=45.2±10.8
Lalloo and Henry 201145	Mixed methods	Patients (pain)	23 Patients	Not reported		✓
Lalloo et al. 201446	Mixed methods	Patients (pain)	50 Patients	x̄=50±14		✓
Lalloo et al. 201447	Mixed methods	Patients (pain)	17 Patients	Patient: x̄=15.4±SD 1.7		✓
		Providers (clinic administrative coordinator, anesthesiologist, advanced practice nurse, physiotherapist, psychologist, psychiatrist)	8 Providers	Provider: x̄=46.7±9.6
Lin et al. 201748	Mixed methods	Patients (oncology)	106 Patients	Range 3-9		✓
Loth et al. 201649	Mixed methods	Patients (brain tumor)	40 Patients	x̄=52.7±13.7	✓	✓
Macpherson et al. 201458	Quantitative descriptive	Patients (oncology)	72 Patients	Range 13-29	✓	✓	✓
Schroeder et al. 201759	Qualitative	Patients (irritable bowel syndrome)	10 Patients	Patient: x̄=33		✓
		Providers	10 Providers	Provider: Not reported
Wallace et al. 201750	Mixed methods	Providers (clinician/SleepCoacher mobile application users)	3 Providers	Not reported			✓
Woods et al. 201551	Mixed methods	Patients (women experiencing menopausal symptoms)	30 Patients	Range 40-60	✓	✓	✓

Study and symptom characteristics

Study participants included patients (n = 9),^{43,45,46,48,49,51,54,56,58} providers (n = 4),^44,50,55,60 or both (n = 5)^{47,52,53,57,59} (Table 1). Sample sizes were 10 to 548 patients and 3 to 233 providers. The age range was 3 to 76 for patients and 54 to 61 for providers. In the 12 studies that reported gender, the majority of the participants were females.^{43,44,47–49,52–59} Of the 6 studies that reported patient education levels, the majority of the participants had a college education.^{47,49,51,52,54,57} In the 3 studies that reported race or ethnicity, the majority of the participants were white.^44,52,58

While 2 studies related to depression, anxiety, and cognitive function were reviewed in the full-text stage, they did not meet inclusion criteria; thus, no studies focused on the symptoms of affective mood, affective anxiety, or cognitive function were included in our review. The 3 remaining NINR CDE symptoms (pain, fatigue, and sleep) were identified in the 18 studies (Table 1) with 7 studies focused on more than 1 symptom.^{43,44,49,51,54,57,58} Pain was the most studied NINR CDE symptom and was featured in all but 4 studies.^50,52,56,60

Visualizations

The types of visualizations in the 18 studies included (Table 2): graphs (n = 14),^{43,44,49–60} icons (n = 4),^45–48 and virtual body maps (n = 2).^46,47 Studies featured both simple (eg, line and bar graphs) and complex graphs. Complex graphs included radial heat maps, radial bar charts, and symptom relationship graphs. The radial heat map was used by Wallace et al.⁵⁰ as a visualization of a 24-hour clock; it has 5 distinct rings that display an average data value at 15-minute intervals for noise anomalies, movement anomalies, sleep start time, actual wake time, and scheduled wake time. The radial bar chart has rings that reflect a 2-day segment with 15-minute intervals for current sleep state including normal sleep, oversleep, nap, noise anomaly, movement anomaly, noise and movement anomaly, aware, and early awakening.⁵⁰ To create symptom relationship graphs, participants selected and clustered symptoms, drew causal relationships among the symptoms, and characterized the bothersomeness of the symptom cluster.^43,51,54,58 Other studies used icons to symbolize symptoms. Lalloo et al.^45–47 used icons as visual metaphors of 5 pain qualities including a flame on a matchstick (burning pain), an ice cube (freezing pain), a vice (squeezing pain), a knife (lacerating pain), and an anvil (aching pain). Virtual body maps are interactive diagrams of the anterior and posterior aspects of a body and were used in multiple studies of pain.^46,47 The information visualizations were predominantly presented in an electronic format (n = 16);^{43–47,49–55,57–60} 9 were interactive.^{43,45–47,50,51,54,58,60}

Table 2.

Characteristics of visualizations

	Types of visualizations used in studies				Visualization format of delivery
Study	Graph	Icons	Virtual Body Map	Description	Electronic	Paper	Unclear	Interactive	Description
Ameringer et al. 201543	✓			Symptom relationship graphs	✓			✓	A native iPad application
Brundage et al. 201744	✓			Line graph, Pie chart, Bar graph	✓				Web-based
Davis et al. 200852	✓			Bar graph,Line graph	✓				Laptop with touch screen
Duke et al. 201060	✓			Bar graph	✓			✓	Web-based
Hochstenbach et al. 201653	✓			Line graph	✓				Web-based application accessed via iPad
Ismail et al. 201654	✓			Symptom relationship graphs	✓			✓	iPad application
Jamison et al. 201755	✓			Line graph	✓				Smartphone application
Kourtis et al. 201156	✓			Modular line graph			✓		Not reported
Kuijpers et al. 201657	✓			Bar chart,Heat map	✓				Web-based
Lalloo and Henry 201145		✓			✓			✓	Web-based
Lalloo et al. 201446		✓	✓		✓			✓	Laptop computer
Lalloo et al. 201447		✓	✓		✓			✓	Laptop computer
Lin et al. 201748		✓		Frequency images		✓			Image card
Loth et al. 201649	✓			Bar graph	✓				Tablet personal computer
Macpherson et al. 201458	✓			Symptom relationship graphs	✓			✓	A native iPad application
Schroeder et al. 201759	✓			Bubble graph, Bar graph, Parallel coordinates plot	✓				Web-based
Wallace et al. 201750	✓			Time series area chart, Radial heat map, Stacked radial bar chart with/without radial spiral design	✓			✓	Native application
Woods et al. 201551	✓			Symptom relationship graphs	✓			✓	iPad application

Many studies had more than 1 goal for the visualization, including reporting, monitoring, understanding, making decisions, and/or communicating (Table 3). However, the most frequent goal (n = 13) was understanding symptoms,^{49,50,52,53,55,57} including understanding relationships of symptoms to disease processes and other symptoms^{43,51,54,58,59} or treatments.^44,60

Table 3.

Goals of visualization

	Reporting symptom(s)a	Monitoring symptom(s)b	Understanding symptom(s) and/or its relationships/ trends	Decision making about treatment	Communicating with healthcare providers or patients
Ameringer et al. 201543		✓	✓
Brundage et al. 201744			✓		✓
Davis et al. 200852			✓
Duke et al. 201060			✓	✓
Hochstenbach et al. 201653		✓	✓	✓	✓
Ismail et al. 201654		✓	✓
Jamison et al. 201755		✓	✓	✓	✓
Kourtis et al. 201156				✓
Kuijpers et al. 201657			✓
Lalloo & Henry 201145	✓				✓
Lalloo et al. 201446	✓				✓
Lalloo et al. 201447	✓				✓
Lin et al. 201748	✓
Loth et al. 201649			✓
Macpherson et al. 201458		✓	✓
Schroeder et al. 201759			✓	✓
Wallace et al. 201750			✓		✓
Woods et al. 201551		✓	✓

^aReporting symptoms refers to visualizations that are used to help patients report their symptoms (eg, pain) to their providers.

^bMonitoring symptoms refers to visualizations of patient data used to help either patients or providers monitor patient symptoms.

Visualization assessment and findings

Thirteen studies met all visualization quality assessment criteria^{43–47,49–51,53,54,58–60} (Supplementary Table S3). Five studies did not descibe how the visualization was designed (n = 1), presented to participants (n = 3), or evaluated (n = 1), and/or did not include an image of the visualization (n = 2).

Health-ITUEM classification and other findings

All studies reported at least 1 assessment that could be classified into the Health-ITUEM subjective and objective categories (Figure 2). Ten studies included subjective assessments (ie, preference and intent to use) not included in the Health-ITUEM categories. Study findings are summarized in Supplementary Table S4.

Figure 2.

Study Findings Based on the Health Information Technology Usability Evaluation Model (HEALTH-ITUEM)*

*The numbers in the figure represent the number of articles.

Subjective findings related to visualizations

Thirteen studies reported findings related to perceived ease of use^{44–47,49,52–55,57–60} and 12 reported perceived usefulness.^{43,45–47,50,51,53–56,58,59} Patients and/or providers perceived bar^49,52 and/or line graphs^44,57 as “(very) easy” to understand. Likewise, icons (ie, Iconic Pain Assessment tool) and/or virtual body maps (ie, Pain-QuILT) used to facilitate self-report of pain were rated as “(very) easy” to use^45,47 by patients and providers or to have “greater ease of use” by patients.⁴⁶ In some instances, the ease of use ratings were more generally applied to an application that included a graph. For example, in the study by Jamison et al.⁵⁵, the majority of patients who used the pain application, which featured a line graph, rated the application as “easy to use.” Similarly, patients who used the Computerized Symptom Capture Assessment Tool (C-SCAT) to visualize symptom relationship graphs described the tool as “easy to use”⁵⁴ and rated the instructions as “(very) easy.”⁵⁸ Providers reported having a good or excellent experience using the RXplore tool, which displays horizontal bar graphs of a drug’s most common side effects.⁶⁰ In contrast, half of providers reported being worried that bar and line graphs of food symptom triggers (eg, pain) were too complicated for patients to interpret.⁵⁹ Loth et al.⁴⁹ found that patients had difficulty detecting differences in pain bar graphs displaying changes over time.

Patients perceived line graphs,^53,56 icons,⁴⁵ and the combination of icons and virtual body maps^46,47 for pain as useful. This included use of the Pain-QuILT for initiating and promoting clear communication with the health team.⁴⁶ Providers rated sleep visualizations using radial heat maps as useful.⁵⁰ Schroeder et al.⁵⁹ reported that line graphs and bar charts of patient nutrients and symptoms elicited patient and provider questions about the data collection and requests for comparative population data. Patients reported symptom relationship graphs as accurate depictions of symptoms,^51,58 and were able to use an interactive digital interface (ie C-SCAT) to report, cluster, name, and articulate relationships among symptoms.^43,51 One study highlighted ways that visualizations could be changed to improve usefulness.⁵³ Nurses in the study by Hochstenbach et al.⁵³ stated that the graphs needed to display more information to see pain trends.

Objective findings related to visualizations

Nine studies reported efficiency^{43,46,47,53–56,58,60} and 5 reported effectiveness.^{44,48,49,57,60} In regard to efficiency, studies reported completion time^43,54,56,58 and compared completion rates to other clinical assessment tools.^46,47,60 Patient completion rates were reported for line graphs as part of a pain application,⁵⁵ a pain diary,⁵³ and symptom relationship graphs.^42,53,57 One study focused on clinicians.⁵⁶ Kourtis et al.⁵⁶ reported time required for the clinican to enter 1 month of patient diary-card information into the computer system to generate a line graph for patients. For comparison to other tools, patients who used a method that incorporated icons and virtual body maps (ie, Pain-QuILT) for self-reporting pain had similar completion times to traditional pain assessment methods, including verbal pain interview,⁴⁶ McGill Pain Questionnaire, and Brief Pain Inventory.⁴⁷ In addition, clinicians using a tool for visualizing drug side effects (ie, RXplore) answered questions faster compared to using an online evidence-based resource with no visualizations (ie, UpToDate).⁶⁰

Five studies evaluated effectiveness of graphs^44,49,57,60 and icons,⁴⁸ 3 via percentage of correct answers to objective understanding questions.^49,57,60 The percentage of correct answers ranged from 59%⁵⁷ to 80%⁴⁹ in patients and 74%⁵⁷ to 78%^57,60 in providers. Brundage et al. found that clinician interpretation accuracy was dependent on both the type of graph and how the information was displayed within the graph. Clinicians had more accurate responses for line graphs in which higher scores correspond to better symptom outcomes.⁴⁴ In addition, Lin et al. evaluated the effectiveness of pain frequency images by assessing comprehension. They found that children were unable to understand meanings conveyed by the symbols represented in the pain frequency images.⁴⁸

Other findings: preference and intent to use

Nine studies^{44,46–48,50,52,57,59} reported findings that we classified as preference (n = 6) or intent to use (n = 3). Providers preferred bar charts^44,52 and heat maps⁵⁷ for symptom visualization over other graphical formats (eg, pie charts, line graphs). Three studies evaluated patient preferences for pain visualizations.^46–48 Lalloo et al.⁴⁷ found that patients preferred the method that incorporated icons and virtual body maps (ie, Pain-QuILT) for self-reporting pain compared to traditional clinical pain assessment methods (ie, McGill Pain Questionnaire and Brief Pain Inventory). In addition, Lin et al.⁴⁸ reported that children preferred “spikey” pain frequency images compared to other visual representations such as icons.

Three studies reported on intent to use the visualizations in practice.^50,59,60 Providers reported that they were likely to use graphical tools for patient communication about sleep,⁵⁰ identifying food symptom triggers in patients with irritable bowel syndrome,⁵⁹ and visualizing drug side effects (ie, RXplore).⁶⁰

DISCUSSION

To the best of our knowledge, this study represents the first systematic review related to information visualizations of NINR CDE symptoms for patients and/or providers. We found that information visualizations have been used to represent symptom information about pain, fatigue, and sleep with pain being the most studied symptom. Although we did review the full text of 2 articles that focused on the remaining NINR CDE symptoms, ie, cognitive function, depression, and anxiety, these studies were ultimately not included in the review, as they only proposed and did not evaluate the visualizations.^61,62

Furthermore, the majority of studies focused on pain in the context of cancer. This finding is not surprising, as cancer and cancer treatment related symptoms represent the most widely studied area of symptomatology research.¹⁹ The limited use of information visualizations for other clinical contexts suggests the need for additional symptom and symptom information visualization research for other symptoms and in common chronic and rare diseases.^63,64

Only a few studies reported race, ethnicity, and education, and no studies explicitly addressed health literacy, numeracy, graph literacy, or limited English proficiency. Thus, it is not possible to assess the generalizability of findings to other populations. More research on symptom information visualizations is needed in populations at high risk for health disparities. This work is especially prudent, as research has consistently shown that there are disparities in symptom management among racial/ethnic minorities.^34,35,65 Another health disparate population that has not been well studied and is increasing in the United States is individuals with limited English proficiency⁶⁶ for whom communication is challenging.^67,68 The use of symptom information visualizations should be explored as a potential solution.^3,12

Consistent with the health risk communication literature,^6,7 the majority of studies in our review used simple graphs (eg, line graph, pie chart, and bar graph). This is likely due to the wider availability of tools to support the creation of such graphs (eg, commonly used spreadsheet and statistical software) or developers’ familiarity with the creation of graphs. Such graphs are appropriate and may be preferred for visualization of some measures. In other instances, information-rich designs that provide additional context are preferred.³ Five studies in this review used information-rich designs—radial sleep graphs⁵⁰ and symptom relationship graphs^43,51,54,58—to visualize multiple dimensions (eg, time of day or relationships among symptoms). The goals of the information visualizations for symptoms were multi-faceted and included reporting, monitoring, understanding, treatment decision making, as well as patient-provider communication. However, the review did not reveal a pattern of matching between goal and type of visualization. Although guidance exists about the match between data type/attributes and information visualization in general (eg, Ware, 2012⁶⁹) and, in a more limited manner, for patient-reported outcomes in specific populations,^10–12,67 this review suggests that there is a substantial knowledge gap regarding information visualizations related to patient symptoms.

The majority of studies evaluated visualizations using both subjective and objective measures. Subjective measures are an important foundational step in evaluation of information visualizations, and all studies included a subjective assessment. Only 7 studies compared preferences for 2 or more types of information visualizations, so no conclusions can be made. In terms of objective measures, 8 studies measured only efficiency and 4 measured only effectiveness; 1 study measured both. Such outcomes should be studied in combination in order for the visualizations to be relevant for clinical purposes, as there are multiple factors that can influence implementation including the provider time^70,71 and patient health literacy.⁷² Additionally, no studies in our review evaluated the actual impact of use of symptom information visualizations on patient outcomes and symptom management, indicating a significant knowledge gap. A randomized clinical trial published after the time period covered in our review begins to address this gap. Kroenke et al.⁷³ assessed the effectiveness of providing patient reported outcome measure information system (PROMIS) symptom scores to clinicians using graphs and found that use of graphs did not significantly contribute to symptom improvement at 3-month follow-up.

The majority of studies included in this review used mixed methods and descriptive quantitative designs. Only 1 study involved users in the development of the information visualization.⁵⁰ According to Chen, 3 out of the 10 unsolved problems that hinder the growth of information visualization relate to the lack of user-centered perspectives.⁷⁴ User-centered participatory design in the development of information visualizations can address this gap,³ which may be particularly important for symptoms, which are inherently patient perceptions.

Limitations

Several limitations influence the interpretation and generalizability of the review findings. First, our review was limited to articles focused on visualizations of NINR CDE symptoms. Second, our search terms did not include nonspecific symptom concepts such as “energy” in place of “fatigue” that have been used in some studies (eg,³) Third, we restricted our searches to English language published literature and did not include “gray literature.” Consequently, our review may have missed some relevant studies. Moreover, the small number of studies for each type of information visualization limits generalizable conclusions about optimal visualization approaches.

CONCLUSION

A variety of visualizations have been developed to represent symptom information for patients and/or providers. The increasing availability of tools for the design and dissemination of information visualizations provides the opportunity for visualizations beyond those (eg, bar graphs, line graphs) that can be created in statistical programs. User-centered participatory approaches for information visualization development and more sophisticated evaluation designs are needed to assess which visualization elements work best for which populations in which contexts. While studies in this review focused on subjective perceptions (ease of use, usefulness, preference, and intent to use) and objective findings (efficiency and effectiveness), rigorous studies are also needed to test the impact of visualizations on symptom management and patient outcomes.

FUNDING

This work is supported by the Reducing Health Disparities Through Informatics training grant (T32NR007969) and the Precision in Symptom Self-Management (PriSSM) Center (P30NR016587). Dr Koleck is also supported by K99NR017651.

CONTRIBUTORS

Study conceptualization (SB, ML, TAK), search and retrieval of relevant articles (ML, TAK), analysis (ML, TAK), manuscript drafting and substantive edits (SB, ML, TAK).

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Conflict of interest statement. None declared.