Abstract
Current electronic health records (EHRs) are often ineffective in identifying patient priorities and care needs requiring nurses to search a large volume of text to find clinically meaningful information. Our study, part of a larger randomized controlled trial testing nursing care planning clinical decision support coded in standardized nursing languages, focuses on identifying format preferences after random assignment and interaction to 1 of 3 formats (text only, text+table, text+graph). Being assigned to the text+graph significantly increased the preference for graph (P = .02) relative to other groups. Being assigned to the text only (P = .06) and text+table (P = .35) was not significantly associated with preference for their assigned formats. Additionally, the preference for graphs was not significantly associated with understanding graph content (P = .19). Further studies are needed to enhance our understanding of how format preferences influence the use and processing of displayed information.
Keywords: electronic health records, standardized nursing terminology, clinical decision support systems, nursing, data visualization
INTRODUCTION
Millions of new pieces of clinical data are generated daily in electronic health records (EHRs), requiring clinicians to review, process, prioritize, and remember important data elements used during the care process.1 Identifying when to prioritize patients and their care requires clinicians to search and filter a large volume of text information to locate clinically meaningful information to inform decisions.2 In nursing, Florence Nightingale is recognized as a pioneer in using data visualization to effectively communicate public health information.3,4 In 1858, Nightingale used a polar-area diagram to demonstrate the relationship between sanitary conditions and soldiers’ deaths on battlefield.3,4 Since then, data visualizations such as graphs have been incorporated into nursing documentation, but the narrative text remains the most common EHR information format for nurses and other clinicians. Evidence suggests that graphs and other forms of data visualization combined with text may enhance the quality of nurse decision-making in clinical practice,5–11 but knowledge in this area is limited. There is a need for further research to validate existing findings and foster a more comprehensive understanding of the factors that facilitate the enhancement of text suggestions in clinical decision support (CDS) systems by incorporating data visualizations, such as tables and graphs. Presenting information in a preferred format may enhance a nurse’s satisfaction with the CDS and understanding of the patient’s data leading to better-informed decisions that improve the efficiency and quality of care. A well-designed CDS can support the quick identification and prioritization of appropriate care. This study explores nurses’ CDS preferences and understanding of format after interaction with 1 of 3 CDS formats (text only, text+table, text+graph).
OBJECTIVES
To determine the nurse’s CDS format preference among 3 CDS formats after random assignment and interaction with 1 of the 3 formats.
To assess nurses’ understanding of the CDS content and the extent of influence on adopting CDS recommendations.
MATERIALS AND METHODS
Design
This study was part of a larger national internet-based randomized controlled trial that evaluated the relationship between CDS formats, nursing care planning time, and adoption of evidence-based practices under simulated conditions.12 In the current study, we identified nurses' CDS preferences (no preference, text only [TO], text+table [TT], text+graph [TG]) and assessed nurses’ understanding of content in the CDS after interaction with 1 of 3 formats (TO, TT, TG). Nurses created nursing care plans coded with standardized nursing languages (SNLs) from NANDA-International, the international nursing diagnoses classification (NANDA-I),13 Nursing Interventions Classification (NIC),14 and Nursing Outcomes Classification (NOC).15 The study was funded by the National Institute of Nursing Research (1R01NR018416) and approved by the Universities of Florida (IRB201902611) and Iowa (201910213) Institutional Review Boards.
Participants
A sample of 38 700 nurses was drawn from the 10 State Board of Nursing registry lists, representing Northeast (New Jersey, Vermont), Southeast (Florida, Arkansas), Midwest (Nebraska, Ohio), Southwest (New Mexico, Texas), and West (California, Oregon). The regions were purposely selected to promote geographic and work-setting diversity.12 Our method was adapted from the Nursing Work Life Study.16 Nurses were eligible if they were 18 years or older, possessed a registered nursing license, and had worked on adult units in a hospital within the last 2 years. The exclusion criteria were: limited or no access to the internet and a desktop, laptop, or tablet (screen smaller than 9 inches), unable to complete the study due to physical or cognitive impairment, unable to speak, read, or write English. To enhance representation, quotas ensured ample numbers of males (goal 20%), minority nurses (goal 35%), and nurses with ADN as their highest degree (goal 30%).12 Nurses were recruited using mail, email, or mixed approaches. The response rates were low. Of the 979 nurses who responded, 323 (33%) were eligible to participate, and 203 were assigned to 1 of 4 CDS formats (TO, TT, TG, tailored to graph literacy) and completed the study.12 Computer software exceldemy was employed to carry out the random assignments. This study excluded participants in the tailored group to avoid potential selection bias. The tailored group was based on graph literacy (eg, with higher graph literacy scores, participants were allocated to graph format), which is a potential confounder for the graph format.
Nursing care planning and CDS
Nurses were first oriented to the nursing care plan system they would interact with via the web. The orientation did not purposely include descriptions of the CDS formats to evaluate participants’ responses when exposed to the intervention. Nurses were directed to adjust the nursing care plans of 2 hypothetical palliative patients across 3 simulated shifts based on hand-off reports and alerts (eg, 9 clusters of CDS suggestions). The nursing care plans and CDS recommendations included nursing diagnoses, interventions, and outcomes coded with NANDA-I,13 NIC,14 and NOC.15 CDS recommendations were provided as text (TO), text and table (TT), or text and graph (TG) showing projected outcomes for accepting and not accepting recommendations (Figures 1 and 2).
Figure 1.
Intervention care planning CDS formats assigned to nurses.
Figure 2.
Graphs showing the trend of patient outcomes for Pain Level (A) and Comfortable Death (B); evidence-based information; and pop-ups that include checkable recommendations for changes to the nursing care plan based on the patient’s needs. (A) In the current nursing shift, the patient’s Pain Level is 2, and the expected is 5 (no pain). Thus, to achieve the patient goal, the CDS graph format alerts nurses with evidence-based recommendations to prioritize the nursing diagnosis of Acute Pain (NANDA-I) and add the nursing intervention Positioning (NIC) to improve the patient’s pain. (B) In the current nursing shift, the patient’s Comfortable Death is 3, and the expected is 5 (highest comfort). CDS alert shows an evidence-based recommendation to add intervention Consultation: Palliative Care (NIC) to improve physical and psychological needs.
Outcome measurement
The primary outcome was the nurse’s reported CDS preference after the intervention. This outcome was assessed with a single question asking nurses to indicate their CDS format preference (TO, TT, TG, or no preference) from pictures. Our secondary outcome was understanding the content in the graph CDS format, assessed with 3 questions asking nurses to interpret the evidence in a hypothetical graph CDS and indicate if patient care is projected to improve, deteriorate, or cannot be projected. Additionally, we evaluated, with a single question, the extent to which those in the TG utilized the graph content for their decision to adopt or not adopt the CDS recommendations.
Statistical analysis
Descriptive statistics of participants were summarized using mean (SD), median, and IQR for continuous variables and frequencies for categorical variables. Frequencies of format preference were tallied and a multinomial logistic regression model was estimated to study the effect of a nurse’s assignment to the TO, TT, or TG on preferences (TO, TT, TG, or no preference). The multinomial logistic regression is usually used for categorical outcome variables with levels. Average marginal effects and 95% CIs were estimated, and a 2-tailed P < .05 was used to indicate statistical significance. Multinomial logistic regression was performed with R (version 4.1.2) with the package multinom. T-tests were used to test graph understanding for differences between groups.
RESULTS
Participants characteristics
Our sample comprised 157 nurses randomly assigned to CDS format groups: TO (n = 55), TT (n = 54), and TG (n = 48). Participants were female (79%), White (77%), with a mean age of 40.6 (SD = 12.2) years, and median bedside nursing experience of 9 years, IQR=13.5.
Nurse’s preferences for CDS formats
We examined the nurse’s post-intervention preference for the assigned CDS format. Figure 3 illustrates the format preferences of participants by the group. Nurses exposed to the TG format reported a 69% preference for graphs after the intervention. Only 33% of the TO and 20% of TT preferred their assigned formats. Furthermore, both TO (49%) and TT (50%) groups indicated a remarkably strong preference for the graph in spite of not interacting with it during the study. Overall, the majority, 55% (n = 87) of participants indicated a preference for the graph format.
Figure 3.
Nurses’ preferences for care planning CDS formats. TG: Nurses were assigned to the text+graph group. TT: Nurses were assigned to the text+table group. TO: Nurses were assigned to the text only group.
Format assigned: TG
Table 1 reveals that the random assignment of the nurse to the TG format had a positive and statistically significant marginal effect on the probability of nurses preferring TG. The TG assignment increased the probability of preferring TG by 0.195 (P = .02). We found no evidence that TG assignment impacted preferences of other formats, TO (P = .07), TT (P = .42), or no preference (P = .85). These results suggest that the TG assignment was an important determinant of the nurse's preference for graphs when creating nursing care plans coded with SNLs.
Table 1.
Average marginal effects of assigned CDS formats on nurses’ preferences from multinomial logistic regression
| Nurses’ preferences | Average marginal effect (95% CI) | P |
|---|---|---|
| Panel A. Format assigned text+graph (TG) | ||
| Graph | 0.195 (0.033 to 0.357) | 0.02 |
| Table | –0.054 (–0.187 to 0.077) | 0.42 |
| Text | –0.147 (–0.304 to 0.010) | 0.07 |
| No preference | 0.006 (–0.055 to 0.067) | 0.85 |
| Panel B. Format assigned text+table (TT) | ||
| Graph | –0.080 (–0.240 to 0.080) | 0.33 |
| Table | 0.055 (–0.061 to 0.173) | 0.35 |
| Text | –0.000 (–0.140 to 0.139) | 0.99 |
| No preference | 0.024 (–0.037 to 0.086) | 0.43 |
| Panel C. Format assigned text only (TO) | ||
| Graph | –0.088 (–0.249 to 0.072) | 0.28 |
| Table | –0.000 (–0.119 to 0.119) | 0.99 |
| Text | 0.124 (–0.004 to 0.254) | 0.06 |
| No preference | –0.036 (–0.118 to 0.045) | 0.39 |
Three binary variables indicate the format group each nurse participant was assigned to during the study. Panel A: text+graph has a value of 1 if the participant was assigned to the text+graph group and 0 otherwise. Panel B: text+table, has a value of 1 if the participant was assigned to the text+table group and 0 otherwise. Panel C: text only has a value of 1 if the participant was assigned to the text only group and 0 otherwise.
Study statistical significance considered P < 0.05.
Format assigned: TO and TT
The results of the multinomial regression analyses indicated that assignment to the TO and TT formats did not significantly impact nurses post study preferences for these formats (TO P = 0.06, TT P = 0.35) (Table 1).
Nurse’s understanding of CDS formats
We assessed nurses understanding of the content appearing in the graph format. We found that nurses who preferred graphs showed similar levels of understanding of graphs (mean = 2.7, SD = 0.6) compared to those with other preferences (mean = 2.5, SD = 0.6) (P = 0.19).
For the TG group, we also compared the extent to which evidence in the graph influenced decisions to adopt recommendations for those who preferred graphs versus those with other preferences. The difference was not statistically significant in this sample (mean = 2.2, SD = 1.1 vs mean = 1.6, SD = 1.0) (P = 0.09).
DISCUSSION
Our results revealed that nurses’ exposure to the TG in nursing care planning coded with SNLs was associated with a significantly high preference for this format post-intervention. Those who experienced the TO and TT did not mimic this preference for their assigned formats. The majority indicated a preference for graphs, suggesting the need to consider regularly CDS recommendations coded with SNLs to nurses utilizing graphs. This is mainly due to the graph offering additional easy to interpret visual evidence supporting the adoption of CDS recommendations. The TO and TT formats may have been less effective in delivering this evidence. By providing visualizations that support easy tracking of patient outcomes over time, nurses can quickly identify trends and patterns that indicate a need for changes leading to a more efficient and better quality of patient care.17
Another possible reason for the higher graph preference was the use of color to indicate deterioration (red) and improved outcomes (green). TT did not utilize color due to the difficulty in meaningfully adding it to tables of cells and numbers. While the use of color does not work for nurses who are color blind, it can be effective in helping other nurses identify the most pertinent content of visualizations and speed decision-making. Previous studies showed that clinicians find green shading for positive content and red shading for alarming content helpful in quickly interpreting patient outcomes.9,18,19 Thus, data visualizations and color usage can be useful to accentuate context in ways not possible in text formats.
While the graph was the most preferred format, it is important to note that 45% of our sample did not select this format (24% selected TO, 17% TT, and 4% had no preference) for delivering nursing care planning. These findings indicate the importance of considering other factors and tailoring designs for preference when designing effective CDS.
Our findings add to a small body of research on nurses’ decision-making using different visual formats for information. Our team’s earlier research found that over half of our sample had low graph literacy.10 This raises concern that when nurses with low graph literacy interact with graph format, they may misinterpret the graph and make a decision that does not lead to optimal patient outcomes. Similarly, another study found that nurses with either low numeracy or graph literacy had poorer comprehension of CDS formats.8 In addition, a recent systematic review of visual formats found no universal visualization preference among clinicians.9 Other studies are needed to identify when graphs should be used to achieve the greatest benefits for clinicians and patients.
IMPLICATIONS AND CONCLUSIONS
The actionable finding from this study is to encourage CDS designers to consider incorporating graph into EHRs to help nurses visualize and track patient outcomes. By providing more opportunities for nurses to use and interact with graphs, patient outcomes may be improved. This study also highlights the need to consider individual differences in analytical skills when designing CDS interfaces with visual information. Format preferences may be related to one’s ability to draw inferences from numerical data, such as interpreting and predicting numerical trends and understanding risk information. Thus, it is important to deepen our understanding of the relationships between nurses’ numeracy and graph literacy effective formats.
A study limitation was that participants’ preferred format was assessed with a single question after engaging with 1 of the 3 formats. Thus, when the user selected a format preference different from the trialed, it cannot be assumed that the preference was based on an equivalent assessment of all 3 CDS formats.
Contributor Information
Fabiana Cristina Dos Santos, Department of Family, Community, and Health Systems Science, College of Nursing, University of Florida, Gainesville, Florida, USA.
Yingwei Yao, Department of Biobehavioral Nursing Science, College of Nursing, University of Florida, Gainesville, Florida, USA.
Tamara G R Macieira, Department of Family, Community, and Health Systems Science, College of Nursing, University of Florida, Gainesville, Florida, USA.
Karen Dunn Lopez, College of Nursing, University of Iowa, Iowa City, Iowa, USA.
Gail M Keenan, Department of Family, Community, and Health Systems Science, College of Nursing, University of Florida, Gainesville, Florida, USA.
FUNDING
This work was funded by the National Institutes of Health (NIH), National Institute of Nursing Research (NINR) (1R01NR018416-01). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH or NINR.
AUTHOR CONTRIBUTIONS
FCDS designed the study, contributed to data collection, conducted analysis, and revised the manuscript. YY, KDL, TGRM, and GMK helped design the study, contributed to analysis, and revised the manuscript. All authors had full access to the data, including statistical reports and tables in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors approved the final version of the manuscript.
CONFLICT OF INTEREST STATEMENT
None declared.
DATA AVAILABILITY
The data underlying this article will be shared on reasonable request to the corresponding author.
REFERENCES
- 1. Manor-Shulman O, Beyene J, Frndova H, et al. Quantifying the volume of documented clinical information in critical illness. J Crit Care 2008; 23 (2): 245–50. [DOI] [PubMed] [Google Scholar]
- 2. Pollack AH, Pratt W.. Association of health record visualizations with physicians' cognitive load when prioritizing hospitalized patients. JAMA Netw Open 2020; 3 (1): e1919301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Nightingale F. Notes on Matters Affecting the Health, Efficiency, and Hospital Administration of the British Army Founded Chiefly on the Experience of the Late War. London: Harrison and Sons; 1858. [Google Scholar]
- 4. Lienharg J. The Engines of Our Ingenuity, Episode 1712: Nightingale’s Graph. Podcasts Between 1988–2002. http://www.uh.edu/engines/epi1712.htm. Accessed March 1, 2023.
- 5. Bantug ET, Coles T, Smith KC, et al. ; PRO Data Presentation Stakeholder Advisory Board. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: what makes a pro picture worth a thousand words? Patient Educ Couns 2016; 99 (4): 483–90. [DOI] [PubMed] [Google Scholar]
- 6. West VL, Borland D, Hammond WE.. Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2015; 22 (2): 330–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Brundage MD, Smith KC, Little EA, et al. ; PRO Data Presentation Stakeholder Advisory Board. Communicating patient-reported outcome scores using graphic formats: results from a mixed-methods evaluation. Qual Life Res 2015; 24 (10): 2457–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Dowding D, Merrill JA, Onorato N, et al. The impact of home care nurses' numeracy and graph literacy on comprehension of visual display information: implications for dashboard design. J Am Med Inform Assoc 2018; 25 (2): 175–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Albers EAC, Fraterman I, Walraven I, et al. Visualization formats of patient-reported outcome measures in clinical practice: a systematic review about preferences and interpretation accuracy. J Patient Rep Outcomes 2022; 6 (1): 18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Lopez KD, Wilkie DJ, Yao Y, et al. Nurses’ numeracy and graphical literacy: informing studies of clinical decision support interfaces. J Nurs Care Qual 2016; 31 (2): 124–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Bouyer-Ferullo S, O'Connor C, Kinnealey E, et al. Adding a visual communication tool to the electronic health record to prevent pressure injuries. AORN J 2021; 113 (3): 253–62. [DOI] [PubMed] [Google Scholar]
- 12. Lopez KD, Yao Y, Cho H, et al. Conducting a representative national randomized control trial of tailored clinical decision support for nurses remotely: methods and implications. Contemp Clin Trials 2022; 118: 106712. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. NANDA International. Nursing Diagnoses Definitions & Classification 2021–2023. 11th ed. New York, NY: Thieme; 2018. [Google Scholar]
- 14. Butcher HK, Bulechek GM, Dochterman JM, Wagner CM. (Eds). Nursing Interventions Classification (NIC). 6th ed. Amsterdam, Netherlands: Elsevier; 2018. [Google Scholar]
- 15. Moorhead S, Swanson E, Johnson M, Maas ML. (Eds). Nursing Outcomes Classification (NOC): Measurement of Health Outcomes. 7th ed. Amsterdam, Netherlands: Elsevier; 2018. [Google Scholar]
- 16. Smiley RA, Lauer P, Bienemy C, et al. The 2017 national nursing workforce survey. J. Nurs. Regul 2018; 9 (3): S1–88. [Google Scholar]
- 17. Dowding D, Randell R, Gardner P, et al. Dashboards for improving patient care: review of the literature. Int J Med Inform 2015; 84 (2): 87–100. [DOI] [PubMed] [Google Scholar]
- 18. Kuijpers W, Giesinger JM, Zabernigg A, et al. Patients’ and health professionals’ understanding of and preferences for graphical presentation styles for individual-level EORTC QLQ-C30 scores. Qual Life Res 2016; 25 (3): 595–604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Smith KC, Brundage MD, Tolbert E, et al. ; PRO Data Presentation Stakeholder Advisory Board. Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care Cancer 2016; 24 (10): 4149–57. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data underlying this article will be shared on reasonable request to the corresponding author.