Nurses’ Numeracy and Graphical Literacy: Informing Studies of Clinical Decision Support Interfaces

Abstract

We present findings of a comparative study of numeracy and graph literacy in a representative group of 60 practicing nurses. The paper focuses on a fundamental concern related to the effectiveness of numeric information displayed in various features in the electronic health record during clinical workflow. Our findings suggest the need to consider numeracy and graph literacy when presenting numerical information as well as the potential for tailoring numeric display types to individual’s cognitive strengths.

With our current Information Age comes the availability of large amounts of both simple to complex data now ubiquitous in developed countries. The data mass is particularly an issue for the electronic health records (EHRs) with hundreds if not thousands of variables collected longitudinally on a large proportion of the population. The enormous amount of data makes it difficult, if not impossible, for clinicians to effectively process and integrate it into patient care decisions. Compounding this problem is the vast array of evidence disseminated in health journals that are underused by busy practicing clinicians. Computational models using both data mining and statistical sciences can yield important information that predicts risk of poor outcomes and likelihood of success for nearly every minor or major health care decision.^1–3 To have its biggest impact on health care, however, the findings from complex data analysis must be presented in ways that are easy and efficient for clinicians with high temporal demands to interpret.

The invention of clinical decision support (CDS), now required in the United States to comply with federal meaningful use standards,⁴ has the potential to improve care quality and may address barriers to evidence based decision making.⁵ The CDS can provide point of care support for decisions through a variety of modalities. These modalities include providing patient specific information and advice to clinicians through “alerts” to signal negative patient trends, “reminders” to advise on established best practices, and “tooltips” to offer suggestions for possible treatments.⁶ Developing a system that computes and delivers meaningful clinical decision support based on real time patient information presents both opportunities and hazards. For example, there is an opportunity to provide visual aids such as graphs in CDS to simplify complex data into graphical representations. A key advantage of graphs is the ability to display simple visual representations of complex data that can potentially speed information processing time over traditional text displays.⁷ A potential hazard of graphical data presentation, however, is the risk that the information presented in it will be misinterpreted, applied inappropriately, or not used at all by subsets of users.

Numbers, tables and graphs are used frequently to present longitudinal patient information. This presentation, however, assumes that clinicians have the requisite numeracy and graphical literacy needed to accurately interpret the meaning of the information in these formats. Unfortunately, little is known about clinicians’ numeracy and graph literacy and the relationship of these to correct interpretation of numerical data presented in tables and graphs. Having such knowledge can help build clinical decision support systems that are usable and useful to clinicians in the practice setting. The purpose of our study was to determine: (1) numeracy and graph literacy scores and score variability among a diverse sample of practicing nurses, and (2) relationships between demographic characteristics and numeracy or graph literacy.

NUMERACY AND GRAPH LITERACY AND DECISION MAKING IN HEALTH CARE

Numeracy is the “ability to comprehend, use and attach meaning to numbers.”⁸ High numeracy, scores in the top half of samples, promotes the ability to draw stronger and more precise inferences from numbers.⁹ Low numeracy is often defined based on specific scores on objective numeracy tests or scores below the mean.¹⁰ In practical terms, individuals are considered to have low numeracy if their skills are limited to simple operations (mostly addition), and they cannot solve math problems that require more than 1 step.¹⁰ Low numeracy in patients has been shown to negatively impact perceptions of risk when making health care decisions that range from preventive screening to medication compliance¹¹ to avoidance of decision making.¹²

A focus on the numeracy of clinical providers in health care is rare and mostly limited to the examination of numeracy skills related to drug dose calculating for student and novice nurses.¹³ A recent study that included 44 practicing registered nurses (RNs) found that nearly half of the sample failed the numeracy test.¹³ Exceedingly low numeracy test scores and skills among practicing nurses had been noted in earlier research as well.¹⁴ Overlooked in these studies that focus on drug calculations are the wide range of decisions confronting nurses that require the ability to interpret, predict numeric trends, and understand risk for uncertain outcomes. With the advent of increased data available through EHRs, the need for high numeracy in nursing may increase as nurses endeavor to interpret complex information to both plan nursing care and accurately communicate with patients about health care decisions.

Graph literacy is relatively newer term defined as “the ability to understand graphically presented information.”¹⁵ Well-designed graphs can depict numeric information in ways that are easy and efficient to understand¹⁶ and play a specific a role in healthcare in the interpretation and communication of risk.¹⁷ Graphs are frequently used in health care journals, but appear to be less so in EHRs. However, like other forms of literacy, there are individual differences in graph literacy that impact accuracy and confidence in interpretation.¹⁸ High and low graph literacy is commonly defined as scores above or below a sample’s mean or average score.^18,19 Unlike studies of graph literacy in patient populations and despite the frequent use of graphs in nursing and health care journals, a focus on the graph literacy of clinical providers in health care is exceedingly rare.

METHOD

Design

We conducted a descriptive, comparative study of the responses to the Subjective Numeracy Scale (SNS) and Long Graph Literacy Scale (LGLS) gathered from all RN subjects who participated in a simulated decision support clinical trial in the summer of 2014. The Institutional Review Board at the authors’ University granted approval for this research.

Setting, sample, and procedures

The data were collected in the Electronic Visualization Laboratory at the University. A purposive sampling method was used to recruit a representative sample of 60 RNs who also participated in a stratified randomized control trial of simulated clinical decision support. Our goal was to have a diverse sample with representation from White, African American, Hispanic and Asian nurses; males and female; Associate Degree in Nursing (ADN) vs. Bachelor of Science Degree (BSN) or above; and less than versus more than 1 year of clinical nursing experience.

After receiving instructions about informed consent and providing written consent, nurses completed procedures that are reported elsewhere,²⁰ and then were asked to complete pencil and paper surveys specific to this study.

Instruments

In addition to a demographic and nursing experience survey (and 3 other surveys not relevant to this analysis), we administered 2 surveys to address nurses’ numeracy and graph literacy and skills.

The SNS is an 8-item tool that uses a 1–6 Likert type response scale.²¹ The SNS consists of items that measure a person’s preference for the presentation of numeric information and items that measure 2 areas: perceptions about the subject’s mathematical problem solving ability and how helpful numbers are for interpreting information. Items are scored from either 1 (having the least perceived skill) to 6 (having the highest perceived skill). The reliability (Cronbach’s alpha =.82 to .84) and predictive validity of this tool have been demonstrated in earlier studies.²²

The LGLS is a 13-item tool that measures ability to understand information presented in graphical forms. The LGLS is a paper and pencil, open-ended fill-in-the-answer test that requires the subjects to read the data and make inferences from the graphs that takes approximately 10 minutes to complete. Graphs presented include bar, pie, icon and line with a score derived by counting the number of correct answers. No partial points are given. All of the questions are health-related focusing on diseases and treatments making the instrument particularly relevant for our study sample. Reliability (Cronbach alpha=.85), and convergent validity (r=.44), values suggest satisfactory to high psychometric properties.

Data analysis

We described the sample using standard descriptive statistics for the entire sample for demographic (age, gender, race and ethnicity), experiential (years of nursing experience and highest education level), numeracy, and graphical literacy variables. The nurses were segmented into low and high numeracy and graph literacy groups using methods reported in other research based on the sample’s means.¹⁹ We applied t-tests between the dichotomous variables and ANOVA for categorical variables to determine if there were differences in numeracy and graphical literacy for any of the demographic and experiential variables. We also conducted Kendall rank correlations between graph literacy and the continuous demographic variables (age and years of nursing experience).

RESULTS

Sample characteristics

Our sample of 60 RNs was demographically diverse. There were 12 males, 25 White, 13 African American, 16 Asian and 6 RNs self-identifying as other races. Only 5 RNs also identified as Hispanic ethnicity. The mean age of the sample was 33.7 ±10.8 years (ranged from 21–71), and the mean years of nursing experience was 8.1 ± 9.7 (ranged from 0–44). Our sample also had a range of highest education levels; 4 ADNs, 42 BSNs, and 14 with an Master’s Degree in Nursing (MSN) or above.

Numeracy and Graph Literacy Descriptive Results

Nurses had a mean total numeracy score of 4.8 (±0.8, ranged from 3–6). Nurses scored lowest in their perceived skill in interpreting fractions (mean 4 ± 1.3, ranged from 1–6). In addition, our sample of RNs had a strong preference for words over numbers (mean 5 ± 1.5, 1–6). Cronbach’s alpha for the SNS scale with our sample was .76. The mean graph literacy scores for the subjects were 11.1 ±1.4 (ranged from 8–13). In our sample the Cronbach alpha for the LGLS scale was .40.

The percentage of nurses with high numeracy (42%) and high graph literacy (45%) was less than half of our sample (Table). From the total sample, only 25% of the nurses had high numeracy and graph literacy, with 38% having both low numeracy and low graph literacy. The remainder of the sample represents a mix, with 17% having high numeracy but low graph literacy and 20% having low numeracy but high graph literacy (Table).

Table.

Numeracy and Graph Literacy by Mean Scores

Numeracy	High (> 5) 42%	Low (≤5) 58%
Graph Literacy
High (> 11) 45%	15 (25.0%)	12 (20.0%)
Low (≤ 11) 55%	10 (16.7%)	23 (38.3%)

Relationships between Numeracy and Graph Literacy and Nurse Characteristics

We examined relationships between age and years of nursing experience as continuous variables. We found that age (p=.05), but not years of nursing experience, was negatively associated with graph literacy. There were no significant relationships between subjective numeracy and age or years of nursing experience. There also were no statistically significant relationships between either numeracy or graph literacy with gender, race, novice nurse status and highest nursing education level (Supplemental Digital Content, Figure).

Figure 1.

Numeracy and Graph Literacy Means and Standard Deviations by Nurse Characteristics

DISCUSSION

Our research adds to a small body of work that examines numeracy and graph literacy in RNs. In this sample of 60 RNs, age had a negative relationship with graph literacy but not numeracy. This suggests that older nurses are at highest risk for misinterpreting graphs in clinical practice. There were no statistically significant relationships between numeracy or graph literacy and gender, race/ethnicity, years of nursing experience, novice nurse status, and highest education level.

Over half of our sample scored low for either numeracy of graph literacy. Importantly we found a low correlation (r=.34) between numeracy and graph literacy in our sample, with about a third of the sample being high in either numeracy or graph literacy but low in the other. These findings are consistent with earlier general population research that found these conceptually related variables are only moderately correlated (r=.4)^10,15,23

Earlier work has raised concerns of low numeracy in nurses with one study finding 45% of RNs failing a numeracy test and 89% failing drug calculation tests. ¹² Although low numeric abilities can potentially decrease quality and safety of health care, for our sample, we were encouraged to see RNs mean score almost 1 point higher in their mean perception of their numeracy and had less variability (3–6) when compared to general population samples (1–6).²¹ This may be related to nurses’ high use of numbers in their daily work that includes medications and treatment dose and timing often with multiple patients. Alternatively, this may be a function of the subjective test we used and the large portion of the sample with graduate degrees. Additional research is needed to confirm in a larger sample that is more representative of practicing RNs.

Graph literacy is a newer term and therefore its measurement in RN samples, or any clinical profession, are rare. One study of RN clinical decision support usability (N=45) found that that many RNs subjects were unable to accurately interpret the graphs in the interface.²⁴ A second smaller study of graph literacy included 16 RNs who reported increased 1-week recall of information when health risk information was presented using graphs.²⁵ While these findings are important, they may not be particularly relevant to acute care RNs who often need to make important patient care decisions in close proximity to the time that a graph is presented.

When compared to previous general population samples of graph literacy, our results show some important differences. RNs in our sample had a higher mean graph literacy score (by almost 2 points) and approximately half the variability then general population samples.¹⁵ This finding is not surprising given the high level of education of RNs. However, a nurse in our sample who earned a mean score of 11 (out of 13) would have incorrectly interpreted about 15% of the graphs on the survey. Importantly these misinterpretations occurred in a laboratory setting with no time constraints and are likely to occur more frequently when time is constrained in clinical practice. Furthermore, for the half of the sample who scored lower than the mean, their rate of graph misinterpretation would likely be higher. In acute care situations when nurses are making critical decisions, this amount of potential graph misinterpretation is concerning.

One can respond to this potential for graph misinterpretation with a one-size fits all practice to avoid using graphs in CDS. However, among people with high graph literacy rate, graphical information is easier and more efficient to interpret when compared to numeric information alone, with the opposite being true for low graph literacy subjects.²³ This means that potentially 45% of our subjects with high graph literacy may benefit from graphs to aid decision making and that 38% of the sample with both low numeracy and graph literacy are most at risk for incorrect interpretation of both graphs and other numeric information.

Over a third of our sample had mixed abilities, high on one variable and low on the other. Graphical visual aids have been found to be most effective for subjects with low numeracy but high graphical literacy,¹⁹ a condition present in 20% of our sample. Identifying numeracy and graph literacy abilities may help identify which nurses benefit from graph presentation in decision support and could hold great potential for improving overall accuracy and efficiency of CDS interpretation.

Our findings of nurses’ numeracy and graph literacy levels also have implications for setting sampling schemas for research of clinical decision support. Although the goal of some clinical research is to have racially and ethnically diverse samples, for research that focuses on decision making in health care, it is equally important to consider cognitive variables such as numeracy and graph literacy to promote samples with cognitive diversity.

In summary, our findings underscore the importance of evaluating the numeracy and graphic literacy and using this information to build decision support systems genuinely useful to nurses in clinical practice. Tailoring the CDS to accommodate the literacy levels of the individual is one possible evidence-based approach. Another is training tailored based on the nurse’s numeracy and graphic literacy scores to enhance accurate interpretation of a specific CDS feature. Future research is needed to establish the relationships between the numeracy and graph literacy levels and nurses’ interpretation accuracy on a variety of CDS prototypes that incorporate different ways to present numeric information such as graphs, tables, and narratives.

Limitations

Our sample had a low representation from ADN nurses, Hispanic nurses, and nurses with less than 1 year of nursing experience so we only had the power to detect very large effects. In addition, a measure of the time spent on the surveys was not collected but would have been a useful indication of efficiency for the objectively measured graph literacy test.

CONCLUSIONS

Nurses’ ability to correctly interpret numeric and graphical trends in EHR data and CDS alerts related to patient trends is critical to facilitating high quality decisions. In this sample with racial, ethnic, education, and experiential diversity, we noted wide distributions of scores for numeracy and graph literacy that are different than ranges reported in other samples. These findings suggests that these 2 variables, which are rarely assessed in studies of practicing nurses, should be incorporated when developing CDS features that include numbers or risk presentation in EHRs.