Abstract
Background and Objective
Efficiency in scale design reduces respondent burden. A brief but reliable measure of numeracy may provide a useful research tool eligible for integration into large epidemiological studies or clinical trials. Our goal was to validate a 3-item version of the Subjective Numeracy Scale (SNS-3).
Design and Setting
We examined seven separate cross-sectional datasets (patients in the emergency department (n=208), clinic (n=205), and hospital (n=460; n=2,053); patients with chronic kidney disease (n=147), diabetes (n=318), and on hemodialysis (n=143)).
Measurements
Internal reliability of the SNS-3 was assessed with Cronbach’s alpha. Criterion validity was determined by nonparametric correlations of the SNS-3 with SNS-8 and other measures of numeracy; construct validity was determined by correlations with measures of health literacy and education.
Results
The SNS-3 had good internal reliability (median Cronbach’s alpha = 0.78) and correlated highly with the full SNS (median rho = 0.91). The SNS-3 was significantly correlated with other measures of numeracy (e.g., median rho = 0.57 with the WRAT), health literacy (e.g., median rho = 0.35 with the S-TOFHLA), and education (median rho = 0.41), providing good evidence of criterion and construct validity.
Conclusion
The SNS-3 is sufficiently reliable and valid to be used as a measure of subjective numeracy.
Introduction
Although important research questions still remain, numeracy plays an fundamental role in decision making, including cognitive reasoning, understanding treatment options and risks, and the ability to convey preferences about health communication to healthcare providers.(1–6) Fagerlin et al. (7, 8) developed and validated the 8-item Subjective Numeracy Scale (SNS-8), composed of self-reported numeracy abilities and preferences, as an alternative to objective measures of numeracy. Efficiency in scale design is important to limit potential burden on respondents while achieving the overall research objective and to provide a useful research tool eligible for integration into large epidemiological studies or clinical trials. The initial report of the acceptability and validity of the SNS-8 suggested that it could be shortened to 5 items (items 1, “fractions”; 2, “percentages;” 3, “15% tip;” 5, “newspaper;” and 6, “chance of something happening”), although to our knowledge implementation and use of the 5-item version is not widespread.
Our research team aimed to identify and use the fewest SNS items that contributed to valid differentiation of subjective numeracy skills among respondents while maintaining a balance between items that assess abilities versus those that measure preferences. In 2011, we reported in a post-hoc data analysis of 208 adult emergency department (ED) patients that a 3-item version of the SNS (SNS-3) appeared to perform as well as the full SNS-8 (9), although we did not report the correlations between the SNS-3 and SNS-8 or other measures of numeracy or health literacy. The shortened SNS-3 consisted of the following items each on a 6-point Likerttype scale, with interior responses labeled by numbers 2 through 5: (1) How good are you at working with fractions? (“Not good at all, 1” to “Extremely Good, 6”); (2) How good are you at figuring out how much a shirt will cost if it is 25% off? (“Not good at all, 1” to “Extremely Good, 6”); and (3) How often do you find numerical information to be useful? (“Never, 1” to “Very Often, 6”). The first two questions focus on self-reported numeracy skills (“fractions” and “shirt”), while the third focuses on subject preference (“useful”).
The purpose of this report is to describe the criterion and construct validity of the SNS-3 in seven different study samples of patients, including the sample of ED patients, between 2010 and 2014. The full SNS-8 was administered in six studies among more than 1,400 subjects, while in the seventh study (n=2,053) the SNS-3 was the sole measure of numeracy. In each study, other more expansive measures of numeracy or health literacy were also administered. We also report associations with education, which has consistently been shown to correlate with measures of numeracy and health literacy.
Methods
The SNS-3 items were selected to maximize correlation coefficients with the SNS-8 and objective measures of numeracy while preserving content validity of the SNS-8 and demonstrating comparability of the SNS-3’s and SNS-8’s correlation with measures of health literacy. Six independent adult patient samples from a single academic medical center where the SNS-8 was administered were examined:
ED patients (n=208) completed the Wide Range Achievement Test-4 (WRAT-4 (10)), Rapid Estimate of Adult Literacy in Medicine (REALM (11)), the reading portion of the Shortened Test of Functional Health Literacy in Adults (S-TOFHLA (12)), and the Brief Health Literacy Screen (BHLS (13, 14)), a subjective measure of health literacy. The SNS-3 items were identified using this study sample (15).
Patients with kidney disease, half of whom were on dialysis (n=75) and the other half with chronic kidney disease not yet requiring dialysis (n=75) completed the Lipkus numeracy assessment (16), REALM, and BHLS.
Primary care (PC) outpatients (n=205) completed the WRAT-3 and REALM(17).
Patients at four outpatient hemodialysis units (n=146) completed the WRAT-3, the math and reading portions of the TOFHLA (12, 18), the REALM, and the BHLS.
Patients with Type 2 diabetes enrolled from 10 State Health Department clinics (n=318) completed the reading portion of the S-TOFHLA and the 5-item version of the Diabetes Numeracy Test (DNT-5 (19)) as part of a baseline battery for the Partnering to Improve Diabetes Education (PRIDE) study(20).
Patients with hypertension admitted to a university hospital in the Health Literacy Screening (HEALS) study (21) (n=460) completed the BHLS and the reading portion of the S-TOFHLA at enrollment.
In a seventh study (Vanderbilt Inpatient Cohort Study, VICS (22)), 2,053 subjects hospitalized for acute coronary syndrome and/or heart failure completed the SNS-3, the BHLS, and STOFHLA.
Surveys were administered by research assistants and completed using pencil/pen and paper in the two kidney disease study cohorts and in the PRIDE study; PRIDE subjects with low health literacy were administered surveys orally by research assistants. In the ED, primary care, HEALS, VICS study cohorts research assistants read the surveys to subjects. Convenience sampling was used for all studies. The SNS-8 (range: 8 – 48) was scored by summing the responses to the eight SNS items after reverse-coding item 7 (“weather”). The SNS-3 (range: 3 – 18) was scored by summing items 1, 4, and 8 from the SNS-8. Internal consistency reliability for the SNS-3 and SNS-8 was computed using Cronbach’s alpha. Spearman’s correlation coefficients were used to calculate all associations between the SNS measures and the other variables. In these secondary analyses, only complete surveys were used. All seven studies received institutional review board approvals, and all subjects gave their informed consent to participate.
Results
Table 1 presents the means, standard deviations and Cronbach’s alphas for the two versions of the SNS among the six study samples. Table 2 presents Spearman’s rank order correlations between the SNS-3 and SNS-8 with other measures of numeracy, measures of health literacy, and education.
Table 1.
Means, Standard Deviations, and Cronbach’s Alphas for the SNS-8 and SNS-3 Among Seven Study Samples
| SNS-8 Mean (SD) [Alpha] |
SNS-3 Mean (SD) [Alpha] |
|
|---|---|---|
| Emergency Department Patients (n=208) | 34.58 (8.83) [0.83] | 13.50 (3.87) [0.78] |
| Kidney Disease Patients (n=150) | 27.55 (9.19) [0.83] | 10.79 (4.04) [0.78] |
| Primary Care Clinic Patients (n=204) | 29.74 (9.37) [0.87] | 11.81 (4.20) [0.86] |
| Hemodialysis Patients (n=143) | 27.77 (9.14) [0.84] | 10.99 (3.85) [0.78] |
| Diabetes Clinic Patients (n=318) | 25.51 (9.76) [0.83] | 9.83 (4.21) [0.76] |
| Hospitalized Hypertension Patients (n=460) | 33.33 (9.23) [0.80] | 13.52 (3.76) [0.67] |
| Hospitalized Cardiovascular Patients (n=2,053) | -- | 13.21 (4.00) [0.80] |
Abbreviation: SNS-8, 8-item Subjective Numeracy Scale; SNS-3, 3-item version of the Subjective Numeracy Scale; SD, standard deviation
Table 2.
SNS-3 Cross Validation: Spearman’s correlations of the SNS-3 and SNS-8 other measures of numeracy, health literacy, and education
| Emergency Department Patients (n=208) |
Kidney Disease Patients (n=150) |
Primary Care Clinic Patients (n=205) |
Hemodialysis Patients (n=143) |
Diabetes Clinic Patients (n=318) |
Hospitalized Hypertension Patients (n=460) |
Hospitalized Cardiovascular Patients (n=2,053) |
|||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SNS-8 | SNS-3 | SNS-8 | SNS-3 | SNS-8 | SNS-3 | SNS-8 | SNS-3 | SNS-8 | SNS-3 | SNS-8 | SNS-3 | SNS-3 | |
| Reference Standard Numeracy Measures | |||||||||||||
| WRAT | 0.59*** | 0.59*** | -- | -- | 0.59*** | 0.57*** | 0.41*** | 0.39*** | -- | -- | -- | -- | -- |
| TOFHLA math portion | -- | -- | -- | -- | -- | -- | 0.30** | 0.25** | -- | -- | -- | -- | -- |
| Diabetes Numeracy Test | -- | -- | -- | -- | -- | -- | -- | -- | 0.44*** | 0.41*** | -- | -- | -- |
| Lipkus | -- | -- | 0.54*** | 0.56*** | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| SNS-8 | 1 | 0.89*** | 1 | 0.94*** | 1 | 0.95*** | 1 | 0.89*** | 1 | 0.92*** | 1 | 0.89*** | -- |
| Health Literacy Measures | |||||||||||||
| REALM | 0.37*** | 0.38*** | 0.42*** | 0.46*** | 0.36** | 0.33** | 0.33*** | 0.26** | -- | -- | -- | -- | |
| S-TOFHLA | 0.40*** | 0.38*** | -- | -- | -- | -- | -- | -- | 0.26*** | 0.24*** | 0.41*** | 0.35*** | 0.27*** |
| TOFHLA reading portion | -- | -- | -- | -- | -- | -- | 0.27** | 0.21** | -- | -- | -- | -- | -- |
| BHLS | 0.40*** | 0.41*** | 0.44*** | 0.48*** | -- | -- | 0.49*** | 0.40*** | 0.34*** | 0.30*** | 0.45*** | 0.43*** | 0.35*** |
| Education+ | 0.49*** | 0.50*** | 0.40*** | 0.40*** | 0.51*** | 0.48*** | 0.41*** | 0.35*** | 0.45*** | 0.40*** | 0.34*** | 0.31*** | 0.45*** |
Abbreviations: SNS-3, 3-item version of the Subjective Numeracy Scale ; SNS-8, full Subjective Numeracy Scale; PRIDE, Partnering to Improve Diabetes Education study; HEALS, Health Literacy Screening Study; WRAT, Math portion of the Wide Range Achievement Test; REALM, Rapid Estimate of Adult Literacy Measure; TOFHLA, Test of Functional Health Literacy in Adults; S-TOFHLA, short form of the TOFHLA; BHLS, Brief Health Literacy Screen
p < 0.001;
p < 0.01;
p < 0.05
Education: Categorized as elementary, middle, high school, college, graduate school in the ED study; otherwise, years of education
Internal Reliability
Cronbach’s alphas for the SNS-3 ranged from 0.67 to 0.86 (median alpha = 0.78) for the seven study samples, including the VICS subjects who completed the SNS-3 as a standalone measure of numeracy, compared to a range of 0.80 to 0.87 (median alpha = 0.83) for the full SNS-8.
Criterion Validity: Correlations with other Measures of Numeracy
The SNS-3 correlated very highly (range: 0.89 to 0.95; median rho = 0.91) with the SNS- 8 (Table 2). The SNS-3 also demonstrated statistically significant correlations with the other measures of numeracy, as did the SNS-8, thus providing evidence of criterion-related validity for both versions of the SNS. In the hemodialysis sample, the correlations of the SNS-3 with the WRAT-3 and numeracy portion of the TOFHLA were comparable to correlations for the SNS-8. The same was true for the PRIDE study where the correlations of the SNS-8 and SNS-3 with the DNT were comparable with one another.
Additional Construct Validity: Correlations with Measures of Health Literacy and Education
The SNS-3 demonstrated significant correlations with measures of health literacy and education, with correlations between 0.21 and 0.51, varying by measure of health literacy and study sample (Table 2).
Discussion
Among more than 3,500 patients in seven studies across a variety of healthcare settings, we found the SNS-3 to be an internally reliable and valid measure of subjective numeracy compared to reference measures of numeracy or health literacy. The SNS-3 demonstrated adequate criterion validity, as evidenced by its correlations with four objective measures of numeracy, and construct validity as shown by its consistent pattern of correlations with measures of health literacy and years of education.
Our findings are important because they describe SNS-3 function among a broad range of subjects and settings and include a large study in which the SNS-3 items were administered as a stand alone measure of numeracy, not embedded within the full 8-item version. The SNS-3 may serve a role in large cohorts, advancing our general understanding of self-reported numeracy skills and preference as mediators or moderaters in decision making. Future use of the SNS-3 may enable a more comprehensive characterization of the relationships between numeracy and patient, healthcare provider, and clinical practice variables. The SNS-3 could also be integrated into trials examining the efficacy of interventions or decision aids, or it could be used to better characterize patient numeracy preferences and self-reported skills, in large healthcare populations in order to identify and meet the numeracy-related needs of patients.
Given the high internal consistency of the SNS-8, it could be argued that there might be other combinations of three SNS items that would serve as well as those we selected for the SNS-3. In fact, in our initial item analyses among ED patients we saw that two of the items, “shirt” and “tip,” both focused on numeracy skills, behaved equivalently and we could have chosen either one. The SNS-3 includes assessment of self-reported numeracy skills, which may be more closely related to objective measures of numeracy or measures of self-efficacy, as well as numeracy preferences, which may be more closely related to health literacy or communication preferences. Within the wide range of alternative measures of both numeracy and health literacy across the seven included studies, distinguishing whether the SNS-3 may function better as a measure of numeracy skills verses preference is outside the scope of our analysis.
Internal reliability and validity coefficients for the SNS-3 when administered as a standalone assessment were comparable to that when derived from administration of the SNS-8 in the other six studies, demonstrating that the psychometric characteristics of the shortened scale are not dependent on the other five SNS items. We did, however, note an overall trend to lower internal reliability and slightly lower correlations for the SNS-3 and reference measures of literacy and numeracy, compared to the SNS-8. Researchers should bear these potential tradeoffs in mind when deciding how to best measure numeracy. In addition, future research is needed in order to examine the ability of the SNS-3 to predict health behaviors or outcomes.
Conclusions
The 3-item Subjective Numeracy Scale (SNS-3) correlates very highly with and is nearly as internally consistent as the original 8-item version (SNS-8). In seven distinct patient samples, validity coefficients for the shortened version of the SNS are comparable to those of the longer version.
Acknowledgments
Funding
Financial support for this study was provided entirely by grants from the NIH (R01 HL109388, R21 HL096581, R03 DK093852, R18 DK083264, K12 HL109019, K23 DK080952, K23 AG040157, 2 UL1 TR000445-06, and UL1 TR000445) and from the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health or the Department of Veterans Affairs. The authors’ funding sources did not participate in the planning, collection, analysis or interpretation of data or in the decision to submit for publication.
References
- 1.Anderson BL, Schulkin J. Physicians' Perceptions of Patients' Knowledge and Opinions Regarding Breast Cancer: Associations with Patient Education and Physician Numeracy. Breast Care (Basel) 2011;6(4):285–288. doi: 10.1159/000331284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schapira MM, Fletcher KE, Gilligan MA, King TK, Laud PW, Matthews BA, et al. A framework for health numeracy: how patients use quantitative skills in health care. J Health Commun. 2008 Jul-Aug;13(5):501–517. doi: 10.1080/10810730802202169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Davids SL, Schapira MM, McAuliffe TL, Nattinger AB. Predictors of pessimistic breast cancer risk perceptions in a primary care population. J Gen Intern Med. 2004 Apr;19(4):310–315. doi: 10.1111/j.1525-1497.2004.20801.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Schapira MM, Davids SL, McAuliffe TL, Nattinger AB. Agreement between scales in the measurement of breast cancer risk perceptions. Risk analysis : an official publication of the Society for Risk Analysis. 2004 Jun;24(3):665–673. doi: 10.1111/j.0272-4332.2004.00466.x. [DOI] [PubMed] [Google Scholar]
- 5.Schwartz LM, Woloshin S, Black WC, Welch HG. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med. 1997 Dec 1;127(11):966–972. doi: 10.7326/0003-4819-127-11-199712010-00003. [DOI] [PubMed] [Google Scholar]
- 6.Rothman RL, Montori VM, Cherrington A, Pignone MP. Perspective: the role of numeracy in health care. J Health Commun. 2008 Sep;13(6):583–595. doi: 10.1080/10810730802281791. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Fagerlin A, Zikmund-Fisher BJ, Ubel PA, Jankovic A, Derry HA, Smith DM. Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Med Decis Making. 2007 Sep-Oct;27(5):672–680. doi: 10.1177/0272989X07304449. [DOI] [PubMed] [Google Scholar]
- 8.Zikmund-Fisher BJ, Smith DM, Ubel PA, Fagerlin A. Validation of the Subjective Numeracy Scale: effects of low numeracy on comprehension of risk communications and utility elicitations. Med Decis Making. 2007 Sep-Oct;27(5):663–671. doi: 10.1177/0272989X07303824. [DOI] [PubMed] [Google Scholar]
- 9.McNaughton C, Wallston KA, Rothman RL, Marcovitz DE, Storrow AB. Short, subjective measures of numeracy and general health literacy in an adult emergency department. Acad Emerg Med. 2011 Nov;18(11):1148–1155. doi: 10.1111/j.1553-2712.2011.01210.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wilkerson GS, Robertson GJ. WRAT4 Wide Range Achievement Test Professional Manual. Lutz, FL: Psychological Assessment Resources, Inc; 2006. [Google Scholar]
- 11.Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, et al. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993 Jun;25(6):391–395. [PubMed] [Google Scholar]
- 12.Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999 Sep;38(1):33–42. doi: 10.1016/s0738-3991(98)00116-5. [DOI] [PubMed] [Google Scholar]
- 13.Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med. 2004 Sep;36(8):588–594. [PubMed] [Google Scholar]
- 14.Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, et al. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008 May;23(5):561–566. doi: 10.1007/s11606-008-0520-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.McNaughton C, Rothman R, Storrow A, Wallston K. Measuring numeracy and health literacy in the emergency department. Acad Emerg Med. 2014 Aug;21(8):944–945. doi: 10.1111/acem.12426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making. 2001 Jan-Feb;21(1):37–44. doi: 10.1177/0272989X0102100105. [DOI] [PubMed] [Google Scholar]
- 17.Osborn CY, Wallston KA, Shpigel A, Cavanaugh K, Kripalani S, Rothman RL. Development and validation of the General Health Numeracy Test (GHNT) Patient Educ Couns. 2013 Jun;91(3):350–356. doi: 10.1016/j.pec.2013.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med. 1995 Oct;10(10):537–541. doi: 10.1007/BF02640361. [DOI] [PubMed] [Google Scholar]
- 19.Huizinga MM, Elasy TA, Wallston KA, Cavanaugh K, Davis D, Gregory RP, et al. Development and validation of the Diabetes Numeracy Test (DNT) BMC Health Serv Res. 2008;8:96. doi: 10.1186/1472-6963-8-96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.White RO, Eden S, Wallston KA, Kripalani S, Barto S, Shintani A, et al. Health communication, self-care, and treatment satisfaction among low-income diabetes patients in a public health setting. Patient Educ Couns. 2015 Feb;98(2):144–149. doi: 10.1016/j.pec.2014.10.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Wallston KA, Cawthon C, McNaughton CD, Rothman RL, Osborn CY, Kripalani S. Psychometric Properties of the Brief Health Literacy Screen in Clinical Practice. J Gen Intern Med. 2013 Aug 6;1:119–126. doi: 10.1007/s11606-013-2568-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Meyers AG, Salanitro A, Wallston KA, Cawthon C, Vasilevskis EE, Goggins KM, et al. Determinants of health after hospital discharge: rationale and design of the Vanderbilt Inpatient Cohort Study (VICS) BMC Health Serv Res. 2014;14:10. doi: 10.1186/1472-6963-14-10. [DOI] [PMC free article] [PubMed] [Google Scholar]