Abstract
Introduction
Because current evidence suggests that numeracy affects how people make decisions, it is an important factor to account for in studies assessing the effectiveness of medical decision support interventions. Subjective and objective numeracy assessment methods are available that vary in theoretical background, skills assessed, known relationship with decision making skills, and ease of implementation. The best way to use these tools to assess numeracy when conducting medical decision-making research is currently unknown.
Methods
We conducted Internet surveys comparing numeracy assessments obtained using the subjective numeracy scale (SNS) and five objective numeracy scales. Each study participant completed the SNS and one objective numeracy measure. Following each assessment, participants indicated willingness to repeat the assessment and rated its user acceptability.
Results
The overall response rate was 78% resulting in a total sample size of 673. Spearman correlations between the SNS and the objective numeracy measures ranged from 0.19 to 0.44. Acceptability assessments for the short form of the Numeracy Understanding in Medicine Instrument (NUMI) and the SNS did not differ significantly. The other objective scales all had lower acceptability ratings than the SNS.
Conclusions
These findings are consistent with prior research suggesting that objective and subjective numeracy scales measure related but distinct constructs. Due to current uncertainty regarding which construct is more likely to influence the effectiveness of decision support interventions, these findings warrant further investigation to determine the proper use of objective vs subjective numeracy assessments in medical decision-making research. Pending additional information, a reasonable approach is to measure both objective and subjective numeracy so that the full range of actual and perceived numeracy skills can be taken into account.
Introduction
Numeracy, “the ability to comprehend, use, and attach meaning to numbers” [1] is emerging as a potentially important factor affecting how people utilize information when making decisions. Low numeracy is associated with increased susceptibility to cognitive biases, greater use of irrelevant information, and reduced willingness to consider long-term versus short-term outcomes. Not surprisingly, people with low numeracy skills experience a variety of poorer health outcomes, report lower health status, and are less interested in becoming actively involved in managing their health care than more numerate people. [1,2]
These results suggest that numeracy may influence the effectiveness of decision support interventions designed to actively engage patients in their care and possibly health services in general. Although correlated, numeracy is not strongly predicted by education, intelligence, health literacy, or other commonly assessed personal attributes. Therefore it needs to be assessed directly. [1,3] Unfortunately, the best way to assess numeracy, in both research and clinical settings, is unclear.
Recent interest in numeracy has generated multiple assessment instruments. Numeracy is a multi-dimensional concept ranging from basic understanding of quantitative information through interpretation of probabilistic and statistical information. [4] Because theoretical frameworks to guide numeracy assessment are just now being developed, the approaches taken to assessing numeracy have varied and the relationships among them are not fully delineated. [5] This situation makes it difficult to interpret the expanding literature regarding the effects of numeracy on medical decision making that is based on the results of different numeracy assessment measures.
The most common approach to numeracy assessment has been to objectively assess a respondent’s ability to manipulate and interpret numerical information. More recently, instruments to specifically assess healthcare-related numeracy skills (defined as “… the degree to which individuals have the capacity to access, process, interpret, communicate, and act on numerical, quantitative, graphical, biostatistical, and probabilistic health information” [4]) have also been developed. [6,7] Objective numeracy measures have been used in multiple studies and several, but not all, have been associated with differences in decision-making behavior. [6–10]
To allay concerns about the response burden and user acceptability of the test-like questions included in objective assessment instruments, subjective numeracy assessment methods have also been developed. The subjective numeracy scale (SNS) - the most commonly used subjective numeracy measure – was initially found to be well correlated with the objective Lipkus extended numeracy scale (Pearson correlations 0.63–0.68), faster, and better accepted by patients suggesting that it could be an ideal numeracy assessment instrument for medical decision making research. [11] However, subsequent studies found correlations between the SNS and the Lipkus objective numeracy scale as low as 0.40. [12–16] The SNS has been correlated with the Berlin Numeracy Scale, a newly introduced objective scale, but correlations with other recently introduced objective measures are unknown. [17] Moreover there are few data available regarding their relative usability and user acceptance, two factors that are important in selecting a measure for use in both decision making research and clinical practice.
Additional research is needed to learn how to efficiently and effectively assess the impact of numeracy on medical decision making. One place to start is to examine differences between subjective and objective numeracy measures, two fundamentally different approaches to numeracy assessment. Subjective and objective numeracy scores appear to measure similar but not identical constructs, relative differences in respondent burden and user acceptance between the SNS and objective numeracy assessment instruments are not fully defined, and the relationship between numeracy assessed using the SNS versus newly introduced objective measures of health numeracy is unknown. [2] The two objectives of this study were: 1) to compare numeracy assessments obtained using the subjective numeracy scale versus five objective numeracy scales and 2) to compare the user acceptability of the SNS versus the same five objective numeracy scales.
Methods
Study population
The study population consisted of US residents at least 18 years old who are members Internet survey panels maintained by Cint, a private research company, and Research Match, an online panel maintained by the NIH Clinical Translational Science Award consortium. [18,19] We surveyed the Cint samples using Fluid Surveys (an online survey platform) targeted to obtain a demographically representative sample of the United States. [20] The Research Match sample was surveyed using an online survey created with Zoomerang (another online survey platform). [21]
This study was approved by the University of Rochester Research Subjects Review Board (IRB) who deemed a formal informed consent process unnecessary. Therefore, before participating, all study subjects were instructed to review an IRB-approved study informational document.
Study intervention
Study participants completed the Subjective Numeracy Scale (SNS) and one of five currently available objective numeracy measures: the Lipkus extended numeracy scale, the Berlin Numeracy Scale, the Weller abbreviated numeracy scale, the short version of the Numeracy Understanding in Medicine Scale (NUMI), and the 6-item version of the General Health Numeracy Test (GHNT).
The SNS is a general numeracy measure consisting of 8 items and contains two subscales: 4 questions regarding perceived numeric ability and 4 questions regarding preference for quantitative information. Responses are made on a 6-point Likert scale and averaged to derive an overall score. [22] The Lipkus expanded numeracy scale is a general numeracy scale that primarily tests ability to work with probabilistic information. It contains 11 items and asks respondents to give the correct answer. [8] The Berlin Numeracy test is a general numeracy measure that also focuses on ability to work with probabilistic information. It comes in several versions; we used the 4 item multiple choice format which asks respondents to pick the correct answer out of four choices. [23] The Weller Abbreviated Numeracy Scale is a general numeracy measure that tests ability to work with both probabilistic information and perform algebraic calculations. It contains 8 items that ask respondents to enter the correct answer. [9] Both the 8 item Numeracy Understanding in Medicine (NUMI) Short Form and the 6 item General Health Numeracy test (GHNT) were specifically designed to measure health numeracy. Both include tests of several skills including ability to use numeric scales, interpret probabilities, and use information provided on a food label and ask respondents to provide a correct answer for each survey item. [7,24]
We conducted 10 separate paired surveys, two for each objective measure. Participants in the SNS versus Weller Abbreviated Numeracy scale surveys were obtained from the Research Match panel; the others were obtained from the Cint panel. For each pair, one survey asked participants to complete the SNS followed by the objective measure and in the other the order of the numeracy assessments was reversed. In each case there was a “washout” period between the two numeracy assessments consisting of questions regarding demographic characteristics and assessment of health literacy assessment of health literacy using the one question Chew subjective literacy test. [25] We considered respondents who indicated they were either extremely or quite a bit confident when filling out forms by themselves as having adequate health literacy.
Following each numeracy assessment, participants were asked if they would be willing to do another survey with similar questions and to assess the acceptability of the scale by indicating the extent to which they found the numeracy assessment questions enjoyable, annoying, stressful, and frustrating using a 6 point scale ranging from strongly disagree to strongly agree. An example of one of the study surveys is included in a supplemental file.
Data analysis
We summarized each numeracy assessment using the standard procedures described for each scale. Because both the SNS and objective numeracy results were not normally distributed, we used Kruskal-Wallis Analysis one-way analysis of variance to determine if the order of scale presentation affected the results within each paired set of surveys and measured the relationship between the paired SNS and objective numeracy assessments using Spearman’s rank correlation. We assessed the statistical significance of differences between the SNS and the objective numeracy instruments by comparing their standardized correlations. [26]
We dichotomized the acceptability responses into two categories, agree and disagree. We then compared participants’ assessments regarding willingness to do again and user evaluations using Fisher’s exact test. All analyses were performed using Excel, MedCalc, or JMP. [27–29] In all cases we defined statistical significance as p < 0.05.
Result
The characteristics of the study participants are summarized in Table 1. The overall response rate was 75% (647/868). Responses to the individual surveys ranged from 67% for the Weller comparisons to 86% for the NUMI comparisons. The overall average age of the study participants was 47 years and ranged from 18 to 80 years. Sixty-seven percent of survey participants were female, 82% were white, and 23% Hispanic. Eighty percent had some education beyond high school and 77% had adequate health literacy skills. Statistically significant variations exist among the survey samples with regard to age, gender, race, health literacy, and education: compared to the four Cint panel surveys, participants in the Weller survey, drawn from the Research Match database, were older, more literate and highly educated and more likely to be female. Because the order of presentation did not affect the numeracy assessment results, all paired surveys were combined for analysis. [See supplemental file for details.]
Table 1.
Study Population
| NUMI N=124 |
GHNT n = 111 |
Lipkus N = 146 |
Berlin N = 182 |
Weller N = 108 |
Overall N = 675 |
P value | |
|---|---|---|---|---|---|---|---|
| Mean age, years | 44.9 | 51.8 | 44.2 | 45.0 | 53.1 | 47.0 | < 0.001 |
| Number (percent)
|
|||||||
| Gender | < 0.001 | ||||||
| male | 40 (32%) | 61 (55%) | 49 (33%) | 54 (30%) | 22 (20%) | 228 (34%) | |
| female | 84 (68%) | 50 (45%) | 99 (67%) | 128 (70%) | 85 (79%) | 449 (67%) | |
| Race | 0.15 * | ||||||
| White | 96 (77%) | 97 (87%) | 118 (80%) | 151 (83%) | na | 465 (82%) | |
| African-American | 13 (10%) | 5 (5%) | 15 (10%) | 16 (9%) | 52 (9%) | ||
| American Indian | 3 (2%) | 0 | 3 (2%) | 0 | 4 (1%) | ||
| Hawaiian/Pacific Islander | 0 | 0 | 0 | 1 (1%) | 1 (0%) | ||
| Asian | 5 (4%) | 5 (5%) | 6 (4%) | 7 (4%) | 24 (4%) | ||
| mixed | 3 (2%) | 2 (2%) | 3 (2%) | 3 (2%) | 11 (2%) | ||
| other | 4 (3%) | 2 (2%) | 3 (2%) | 4 (2%) | 13 (2%) | ||
| Ethnicity | 0.22 † | ||||||
| hispanic | 12 (10%) | 5 (4%) | 10 (7%) | 19 (10%) | na | 46 (8%) | |
| non-hispanic | 112 (90%) | 106 (96%) | 135 (93%) | 163 (90) | 521 (92%) | ||
| Education level | < 0.001 | ||||||
| less than high school | 7 (6%) | 3 (3%) | 4 (3%) | 7 (4%) | 0 | 23 (3%) | |
| high school | 23 (19%) | 17 (15%) | 31 (21%) | 38 (21%) | 2 (2%) | 114 (17%) | |
| some college † | 44 (35%) | 43 (39%) | 54 (37%) | 66 (36%) | 22 (20%) | 229 (34%) | |
| college | 35 (28%) | 25 (23%) | 41 (28%) | 48 (26%) | 37 (34%) | 186 (28%) | |
| post-graduate | 15 (12%) | 23 (21%) | 18 (12%) | 23 (13%) | 47 (44%) | 126 (19%) | |
| NUMI N=124 |
GHNT n = 111 |
Lipkus N = 148 |
Berlin N = 182 |
Weller N = 108 |
Overall N = 675 |
||
|---|---|---|---|---|---|---|---|
| Number (percent)
|
|||||||
| Subjective literacy | < 0.001 | ||||||
| extremely | 56 (45%) | 48 (43%) | 56 (38%) | 80 (44%) | 86 (80%) | 326 (48%) | |
| quite a bit | 38 (31%) | 32 (29%) | 55 (37%) | 53 (29%) | 17 (16%) | 196 (29%) | |
| somewhat | 20 (16%) | 23 (21%) | 26 (18%) | 30 (16%) | 4 (4%) | 104 (15%) | |
| a little bit | 9 (7%) | 8 (7%) | 8 (5%) | 14 (8%) | 0 | 41 (6%) | |
| not at all | 1 (1%) | 0 | 3 (2%) | 5 (3%) | 0 | 9 (1%) | |
Weller survey not included due to missing data.
Category includes associate’s degree.
Response to question: How comfortable do you feel filling out forms by yourself? Adequate literacy was defined as a response of either extremely or quite a bit.
Abbreviations: SNS refers to the subjective numeracy scale; NUMI refers to the Numeracy Understanding in Medicine short form; GHNT refers to the 6 item version of the General Health Numeracy Test; Lipkus refers to the Lipkus expanded numeracy scale; Berlin refers to the 4 item, multiple choice version of the Berlin Numeracy Test; Weller refers to the Weller Abbreviated Numeracy Scale; na - data not available due to survey form error.
The results of the paired numeracy assessments and their correlations are summarized in Table 2 and illustrated in Figure 1. The correlations between the SNS and the objective numeracy measures ranged from 0.19 (95%CI: 0.05 to 0.33) for the Berlin numeracy test, to 0.44 (95% CI: 0.28 to 0.58) for the GHNT. The correlation between the SNS and the Berlin numeracy test was significantly lower than the correlation between the SNS and the GHNT (p = 0.02).
Table 2.
Numeracy Assessment Results
| Paired Comparison | n | SNS mean (sd) | SNS range * | ONS mean (sd) | ONS range * | Spearman r | 95% ci | p value |
|---|---|---|---|---|---|---|---|---|
| SNS vs Numi | 124 | 4.2 (0.95) | 1.8 to 6 | 6 (1.5) | 2 to 8 | 0.35 | 0.18 to 0.49 | < 0.001 |
| SNS vs GHNT | 113 | 4.3 (1.08) | 1 to 6 | 3.5 (1.7) | 0 to 6 | 0.44 | 0.28 to 0.58 | < 0.001 |
| SNS vs Lipkus | 151 | 4.0 (1.0) | 1.1 to 6 | 7.1 (2.7) | 1 to 11 | 0.39 | 0.24 to 0.52 | < 0.001 |
| SNS vs Berlin | 182 | 4.1 (1.0) | 1.1 to 6 | 1.1 (0.93) | 0 to 4 | 0.19 † | 0.05 to 0.33 | 0.009 |
| SNS vs Weller | 100 | 4.6 (0.79) | 2.5 to 5.8 | 4.4 (2.0) | 0 to 8 | 0.34 | 0.15 to 0.50 | < 0.001 |
Maximum score ranges: SNS 1 to 6; Numi 0 to 8; GHNT 0 to 6; Lipkus 0 to 11; Berlin 0 to 4; Weller 0 to 8.
Correlation significantly different from GHNT vs SNS correlation (p = 0.02) and marginally different from Lipkus vs SNS correlation (p = 0.06).
Abbreviations: SNS refers to the subjective numeracy scale; ONS refers to one of the objective numeracy instruments, NUMI refers to the Numeracy Understanding in Medicine short form; GHNT refers to the 6 item version of the General Health Numeracy Test; Lipkus refers to the Lipkus expanded numeracy scale; Berlin refers to the 4 item, multiple choice version of the Berlin Numeracy Test; Weller refers to the Weller Abbreviated Numeracy Scale.
Figure 1.
Scatter plots showing the relationship between health numeracy assessments using the subjective numeracy scale and five objective health numeracy scales.
Table 3 summarizes the respondents’ assessments of the numeracy scales. Participants who used the NUMI and the SNS were essentially equal in their willingness to do a repeat assessment: 90% and 92% respectively. Participants who used the other objective scales were less willing to repeat it than the SNS; in two cases, the Lipkus (81% vs 94%) and Berlin (77% vs 91%), the differences were statistically significant, p ≤ 0.001. Acceptability ratings for all objective scales were significantly lower than the SNS with the exception of the NUMI, where there were no statistically significant differences.
Table 3.
User Evaluations of Time Required and Usability of Subjective versus Objective Numeracy Assessments
| Paired subjective vs objective numeracy assessment comparisons
|
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| SNS vs Numi
|
SNS vs GHNT
|
SNS vs Lipkus
|
SNS vs Berlin
|
SNS vs Weller
|
||||||
| SNS | Numi | SNS | GHNT | SNS | Lipkus | SNS | Berlin | SNS | Weller | |
| n = 124 | n = 111 | n = 148 | n =182 | n = 109 | ||||||
| Willing to do again | ||||||||||
| yes | 114 (92%) | 112 (90%) * | 103 (93%) | 95 (86%) * | 138 (94%) | 119 (81%) * | 165 (91%) | 140 (77%) * | 104 (96%) | 96 (90%) * |
| p = 0.82 | p = 0.13 | p = 0.001 | p = 0.0006 | p = 0.07 | ||||||
|
| ||||||||||
| Perceived Time required | ||||||||||
| < 2 minutes | 85 (69%) | 30 (24%) | 84 (76%) | 17 (15%) | 107 (72%) | 32 (22%) | 127 (70%) | 50 (28%) | 93 (86%) | 10 (9%) |
| 2 to 5 minutes | 36 (29%) | 77 (62%) | 23 (21%) | 67 (60%) | 38 (26%) | 86 (58%) | 44 (24%) | 97 (53%) | 15 (14%) | 66 (61%) |
| 5–10 minutes | 3 (2%) | 15 (12%) | 3 (3%) | 24 (22%) | 2 (1%) | 22 (15%) | 6 (3%) | 26 (14%) | 0 | 28 (26%) |
| > 10 minutes | 0 | 2 (2%) | 1 (1%) | 3 (3%) | 1 (1%) | 8 (5%) | 5 (3%) | 9 (5%) | 0 | 4 (4%) |
| p < 0.001 | p < 0.001 | p < 0.001 | p < 0.001 | p < 0.001 | ||||||
|
| ||||||||||
| I enjoyed answering the questions | ||||||||||
| disagree | 10 (8%) | 16 (13%) | 11 (10%) | 22 (20%) | 16 (11%) | 32 (22%) | 22 (12%) | 61 (34%) | 3 (3%) | 26 (24%) |
| agree | 114 (92%) | 108 (87%) | 100 (90%) | 89 (80%) | 132 (89%) | 116 (78%) | 160 (88%) | 121 (66%) | 106 (97%) | 81 (76%) |
| p =0.29 | p=0.06 | p=0.02 | p < 0.001 | p < 0.001 | ||||||
|
| ||||||||||
| The questions were annoying | ||||||||||
| disagree | 112 (90%) | 110 (89%) | 95 (86%) | 84 (76%) | 126 (85%) | 116 (78%) | 161 (87%) | 127 (70%) | 99 (92%) | 86 (80%) |
| agree | 12 (10%) | 14 (11%) | 16 (14%) | 27 (24%) | 22 (15%) | 32 (22%) | 24 (13%) | 55 (30%) | 9 (8%) | 22 (20%) |
| p = 0.83 | p = 0.09 | p = 0.17 | p < 0.001 | p = 0.02 | ||||||
|
| ||||||||||
| The questions made me feel stressed | ||||||||||
| disagree | 112 (90%) | 102 (82%) | 100 (90%) | 78 (70%) | 133 (90%) | 105 (76%) | 154 (85%) | 111 (61%) | 106 (98%) | 74 (69%) |
| agree | 12 (10%) | 22 (18%) | 11 (10%) | 33 (30%) | 15 (10%) | 33 (24%) | 28 (15%) | 71 (39%) | 2 (2%) | 33 (31%) |
| p = 0.10 | p = 0.0003 | p = 0.0002 | p < 0.001 | p < 0.001 | ||||||
|
| ||||||||||
| The questions were frustrating | ||||||||||
| disagree | 116 (94%) | 113 (91%) | 97 (87%) | 77 (69%) | 133 (90%) | 108 (73%) | 159 (87%) | 113 (62%) | 104 (97%) | 84 (78%) |
| agree | 8 (6%) | 11 (9%) | 14 (13%) | 34 (31%) | 15 (10%) | 40 (27%) | 23 (13%) | 69 (38%) | 3 (3%) | 24 (22%) |
| p = 0.63 | p < 0.001 | p < 0.001 | p < 0.001 | p < 0.001 | ||||||
Difference statistically significant by Fisher’s exact test, p < 0.01.
Abbreviations: NUMI = Numeracy Understanding in Medicine short form; GHNT = General Health Numeracy Test short form; Lipkus = Lipkus Expanded Numeracy Test; Berlin = Berlin Numeracy test, 4 item, multple choice form; Weller = Weller Abbreviated Numeracy Scale.
Discussion
Our study outcomes were developed to address two key considerations in choosing an appropriate numeracy assessment scale for use in medical decision making research: a) how comparable are the data obtained with alternative scales and b) what is their relative ease of use and user acceptability. Like the original SNS development studies, we found participants generally viewed the SNS more favorably than the Lipkus scale and were more willing to answer similar questions in the future. In contrast to the original studies, our participants were much more willing to do a repeat assessment with both scales (91% vs 50% for the SNS and 81% vs 8% for the Lipkus scale) and the correlation between the two numeracy measures was significantly lower: 0.38 (95% CI: 0.23 to 0.51) vs 0.68. [11] Other studies comparing the SNS and the Lipkus scale have also reported lower correlations ranging from 0.40 and 0.48. [12–16]
Our study extends these findings to include comparisons between the SNS and four recently introduced objective numeracy scales. The correlations are modest, ranging from 0.34 to 0.44 except for the much lower correlation between the Berlin numeracy test and the SNS: 0.19. User assessments for the Berlin, GHNT, and Weller scales indicate that the SNS is somewhat easier to complete and more acceptable. In contrast, the short form NUMI did not differ significantly from the SNS in terms of user acceptance.
Our study was not designed to compare how well the SNS and objective numeracy measures predict decision-making behavior or influence the effectiveness of decision support interventions such as decision aids. Current evidence about these questions is inconclusive. SNS scores have been shown to predict several skills important in medical decision making nearly as well as the Lipkus scale including recall of risk information, interpretation of survival curves, and ability to perform time-trade off utility assessments. [30] They were also more predictive of willingness to pay for breast cancer testing than the 3 item Schwartz objective numeracy scale (a subset of the Lipkus scale) among women who were members of a high risk BRCA registry and outperformed the Wide Range Achievement Test - 4 (an objective numeracy assessment procedure) at predicting 30 day emergency room recidivism rates in patient with congestive heart failure [31–33]
Objectively assessed numeracy on the other hand has been shown to influence decision making skills including resistance to framing effects, consistent use of risk information regardless of how it is presented, and affectively responding to data appropriately. [34] A recent study of college students found that objective numeracy was a more reliable and powerful predictor of performance on decision making related tasks than subjective numeracy. [3,34] A study in older individuals found that they substantially overestimated their numeric abilities, one of the recognized weaknesses of subjective measures. [16] There is also evidence that health numeracy may not be directly comparable to more general numeracy: in one study people were found to have more difficulty comprehending quantitative health-related information than comparable non-health related information. [35]
It is currently uncertain how well the available numeracy assessment measures identify skills that influence medical decision-making. Consequently it is unclear how numeracy should be assessed in medical decision-making research studies and how to combine findings obtained with different numeracy assessment methods. The modest correlations we found between the SNS and all objective measures indicate that either type of measure, used alone, may be insufficient to assess either the impact of numeracy on medical decisions or the effectiveness of decision support interventions. Pending additional information, our findings suggest that a reasonable approach to assessing the role of numeracy in medical decision making research is to measure both objective and subjective numeracy so that the full range of actual and perceived numeracy skills can be taken into account. They also indicate that this combined approach is feasible given the relatively high levels of user acceptance ratings of both the SNS and several of the objective scales most notably the short form NUMI. Further investigation to determine the proper role of objective vs subjective numeracy assessments in medical decision-making research is a high priority for further investigation.
Supplementary Material
Acknowledgments
This study was supported by grant 1 K24 RR024198-02 by the National Heart Lung and Blood Institute.
References
- 1.Nelson W, Reyna VF, Fagerlin A, Lipkus I, Peters E. Clinical Implications of Numeracy: Theory and Practice. Ann Behav Med. 2008;35:261–74. doi: 10.1007/s12160-008-9037-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Reyna VF, Nelson WL, Han PK, Dieckmann NF. How numeracy influences risk comprehension and medical decision making. Psychological Bulletin. 2009;135:943–73. doi: 10.1037/a0017327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lag T, Bauger L, Lindberg M, Friborg O. The Role of Numeracy and Intelligence in Health-Risk Estimation and Medical Data Interpretation. Journal of Behavioral Decision Making. 2014;27:95–108. [Google Scholar]
- 4.Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE. A Definition and Operational Framework for Health Numeracy. Am J Prev Med. 2005;29:375–76. doi: 10.1016/j.amepre.2005.06.012. [DOI] [PubMed] [Google Scholar]
- 5.Lipkus IM. Numeric, Verbal, and Visual Formats of Conveying Health Risks: Suggested Best Practices and Future Recommendations. Medical Decision Making. 2007;27:696–713. doi: 10.1177/0272989X07307271. [DOI] [PubMed] [Google Scholar]
- 6.Schapira MM, Walker CM, Cappaert KJ, Ganschow PS, Fletcher KE, McGinley EL, et al. The Numeracy Understanding in Medicine Instrument: A Measure of Health Numeracy Developed Using Item Response Theory. Med Decis Making. 2012;32:851–865. doi: 10.1177/0272989X12447239. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Osborn CY, Wallston KA, Shpigel A, Cavanaugh K, Kripalani S, Rothman RL. Development and validation of the General Health Numeracy Test (GHNT) Patient Educ Couns. 2013;91:350–56. doi: 10.1016/j.pec.2013.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making. 2001;21:37–44. doi: 10.1177/0272989X0102100105. [DOI] [PubMed] [Google Scholar]
- 9.Weller JA, Dieckmann NF, Tusler M, Mertz CK, Burns WJ, Peters E. Development and testing of an abbreviated numeracy scale: A rasch analysis approach. Journal of Behavioral Decision Making. 2012 doi: 10.1002/bdm.1751. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ghazal S. Predicting biases in very highly educated samples: Numeracy and metacognition. Judgment and Decision Making. 2014;9:15–34. [Google Scholar]
- 11.Fagerlin A, Zikmund-Fisher BJ, Ubel PA, Jankovic A, Derry HA, Smith DM. Measuring Numeracy without a Math Test: Development of the Subjective Numeracy Scale. Med Decis Making. 2007;27:672–80. doi: 10.1177/0272989X07304449. [DOI] [PubMed] [Google Scholar]
- 12.Hess R, Visschers VHM, Siegrist M, Keller C. How do people perceive graphical risk communication? The role of subjective numeracy. Journal of Risk Research. 2011;14:47–61. [Google Scholar]
- 13.Liberali JM, Reyna VF, Furlan S, Stein LM, Pardo ST. Individual Differences in Numeracy and Cognitive Reflection, with Implications for Biases and Fallacies in Probability Judgment. Journal of Behavioral Decision Making. 2012;25:361–381. doi: 10.1002/bdm.752. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rakow T, Wright RJ, Bull C, Spiegelhalter DJ. Simple and multistate survival curves: can people learn to use them? Med Decis Making. 2012;32:792–804. doi: 10.1177/0272989X12451057. [DOI] [PubMed] [Google Scholar]
- 15.Rolison JJ, Wood S, Hanoch Y, Liu P. Subjective Numeracy Scale as a Tool for Assessing Statistical Numeracy in Older Adult Populations. Gerontology. 2013;59:283–88. doi: 10.1159/000345797. [DOI] [PubMed] [Google Scholar]
- 16.Taha J, Sharit J, Czaja SJ. The impact of numeracy ability and technology skills on older adults’ performance of health management tasks using a patient portal. Journal of applied gerontology. 2014;33:416–36. doi: 10.1177/0733464812447283. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Winman A, Juslin P, Lindskog M, Nilsson H, Kerimi N. The role of ANS acuity and numeracy for the calibration and the coherence of subjective probability judgments. Frontiers in psychology. 2014;5:851. doi: 10.3389/fpsyg.2014.00851. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.CINT AB. Stockholm: http://www.cint.com. [Google Scholar]
- 19.ResearchMatch. https://www.researchmatch.org.
- 20.FluidSurveys. http://fluidsurveys.com.
- 21.Zoomerang. http://www.zoomerang.com.
- 22.Subjective Numeracy Scale. http://cbssm.med.umich.edu/how-we-can-help/tools-and-resources/subjective-numeracy-scale.
- 23.Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia-Retamero R. Measuring Risk Literacy: The Berlin Numeracy Test. Judgment and Decision Making. 2012;7:25–47. [Google Scholar]
- 24.Schapira MM, Walker CM, Miller T, Fletcher KE, Ganschow PS, Jacobs EA, et al. Development and validation of the numeracy understanding in medicine instrument short form. Journal of health communication. 2014;19 (Suppl 2):240–53. doi: 10.1080/10810730.2014.933916. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, et al. Validation of Screening Questions for Limited Health Literacy in a Large VA Outpatient Population. J Gen Intern Med. 2008;23:561–66. doi: 10.1007/s11606-008-0520-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Field A, Miles J, Field Z. Discoving statistics using R. London: Sage Publications, Ltd; 2012. [Google Scholar]
- 27.MedCalc. Mariakerke. 12. Belgium: MedCalc Software; [Google Scholar]
- 28.JMP 10.0.2. SAS Institute, Inc; [Google Scholar]
- 29.Microsoft Corporation. Excel [Google Scholar]
- 30.Zikmund-Fisher BJ, Smith DM, Ubel PA, Fagerlin A. Validation of the Subjective Numeracy Scale: Effects of Low Numeracy on Comprehension of Risk Communications and Utility Elicitations. Med Decis Making. 2007;27:663–671. doi: 10.1177/0272989X07303824. [DOI] [PubMed] [Google Scholar]
- 31.Miron-Shatz T, Hanoch Y, Doniger GM, Omer ZB, Ozanne EM. Subjective but not objective numeracy influences willingness to pay for BRCA1/2 genetic testing. Judgment and Decision Making. 2014;9:152–158. [Google Scholar]
- 32.McNaughton C, Wallston KA, Rothman RL, Marcovitz DE, Storrow AB. Short, subjective measures of numeracy and general health literacy in an adult emergency department. Acad Emerg Med. 2011;18:1148–1155. doi: 10.1111/j.1553-2712.2011.01210.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.McNaughton CD, Collins SP, Kripalani S, Rothman R, Self WH, Jenkins C, et al. Low Numeracy Is Associated With Increased Odds of 30-Day Emergency Department or Hospital Recidivism for Patients With Acute Heart Failure. Circulation: Heart Failure. 2013;6:40–46. doi: 10.1161/CIRCHEARTFAILURE.112.969477. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Peters E, Vastfjall D, Slovic P, Mertz C, Mazzocco K, Dickert S. Numeracy and Decision Making. Psychological Science. 2006;17:407–13. doi: 10.1111/j.1467-9280.2006.01720.x. [DOI] [PubMed] [Google Scholar]
- 35.Levy H, Ubel PA, Dillard AJ, Weir DR, Fagerlin A. Health numeracy: the importance of domain in assessing numeracy. Med Decis Making. 2014;34:107–15. doi: 10.1177/0272989X13493144. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.