Validity of Single-Item Screening for Limited Health Literacy in English and Spanish Speakers

Abstract

Objectives. To evaluate 3 single-item screening measures for limited health literacy in a community-based population of English and Spanish speakers.

Methods. We recruited 324 English and 314 Spanish speakers from a community research registry in Dallas, Texas, enrolled between 2009 and 2012. We used 3 screening measures: (1) How would you rate your ability to read?; (2) How confident are you filling out medical forms by yourself?; and (3) How often do you have someone help you read hospital materials? In analyses stratified by language, we used area under the receiver operating characteristic (AUROC) curves to compare each item with the validated 40-item Short Test of Functional Health Literacy in Adults.

Results. For English speakers, no difference was seen among the items. For Spanish speakers, “ability to read” identified inadequate literacy better than “help reading hospital materials” (AUROC curve = 0.76 vs 0.65; P = .019).

Conclusions. The “ability to read” item performed the best, supporting use as a screening tool in safety-net systems caring for diverse populations. Future studies should investigate how to implement brief measures in safety-net settings and whether highlighting health literacy level influences providers’ communication practices and patient outcomes.

Health literacy—ability to obtain, process, and understand basic health information and services needed to make health decisions—is a key health determinant, particularly for Hispanic immigrants.^1,2 Recent reforms following the Affordable Care Act ask health care systems to identify low health literacy patients, provide special assistance, and incorporate health literacy into quality metrics.³ Administration of validated measures such as the Short Test of Functional Health Literacy in Adults (STOFHLA) takes 3 to 8 minutes.⁴ Brief single-item measures that indirectly assess literacy and are documented in electronic health records are needed. However, these measures have been studied mostly among English speakers^5–7 who are chronically ill.^8–10 Safety-net systems are willing to conduct literacy screening¹ but need valid measures for their diverse population, particularly Spanish-speaking patients (40% of US Hispanic individuals are foreign-born, and fewer than 25% of them report speaking English very well¹¹).

We evaluated 3 single items against the STOFHLA, a well-accepted, commonly used measure, in a community-based population of English and Spanish speakers.

METHODS

We randomly selected study participants (n = 638) in 2011 to 2012 from our community research registry of individuals who joined between 2009 and 2012.^12,13 Our registry enrolls Dallas County, Texas, community members by (1) inviting adults attending local health events or waiting in ambulatory clinics of the Dallas County’s safety-net system¹⁴ or (2) enabling current registry members to refer their friends or family. Registry members aged 18 to 70 years received an invitation letter (with a toll-free number to opt out) requesting help to identify strategies that improve communication between patients and providers. Bilingual research assistants called potential participants 1 week later to ascertain interest, assess eligibility, and schedule an in-person appointment. To be eligible, individuals had to report ability to read English or Spanish; if bilingual, they were asked to complete study procedures in the language they preferred to use with their provider. At the appointment, research assistants used a script to obtain consent, administer a paper version of the STOFHLA, and verbally ask the single-item measures; 49.2% of the participants completed study procedures in Spanish and 50.8% in English.

STOFHLA, a 40-item scale with validity established in Spanish and English,¹⁵ assesses reading comprehension and numerical ability. We used recommended cutpoints to dichotomize scores (inadequate vs marginal or adequate).¹⁶ The 3 single items described in Table 1 used 5-point responses: (1) How would you rate your ability to read?; (2) How confident are you filling out medical forms by yourself?; and (3) How often do you have someone help you read hospital materials?

TABLE 1—

Sociodemographic Characteristics and Health Literacy Items Among English- and Spanish-Speaking Participants Enrolled in a Community Registry: Dallas County, Texas, 2011–2012

	English (n = 324), No. (%)	Spanish (n = 314), No. (%)
Sociodemographic Characteristicsa
Age, ya
18–34	110 (34.0)	78 (24.8)
35–49	110 (34.0)	170 (54.1)
50–70	104 (32.1)	66 (21.0)
Sexa
Male	114 (35.2)	91 (29.0)
Female	210 (64.8)	223 (71.0)
Racea
Black/American Indian/Alaska Native/Asian	171 (52.8)	4 (1.3)
Whiteb	153 (47.2)	310 (98.7)
Marital statusa
Married or living with partner	128 (39.5)	214 (68.2)
Single/divorced/widowed/separated/other	196 (60.5)	100 (31.9)
Educationa,c
Grade school	4 (1.2)	72 (23.0)
Some high school	31 (9.6)	95 (30.4)
High school diploma/GED/technical school	102 (31.5)	101 (32.3)
Some college/graduated college	187 (57.7)	45 (14.4)
Health Literacy Items
STOFHLAd
Inadequate (score = 0–16)	9 (2.8)	58 (18.5)
Marginal or adequate (score = 17–36)	315 (97.2)	256 (81.5)
How would you rate your ability to read?
Very poor (1)	1 (0.3)	2 (0.6)
Poor (2)	8 (2.5)	3 (1.0)
OK (3)	42 (13.0)	76 (24.2)
Good (4)	81 (25.0)	151 (48.1)
Very good (5)	192 (59.3)	82 (26.1)
How often do you have someone help you read hospital materials?
Always (1)	8 (2.5)	6 (1.9)
Often (2)	16 (4.9)	2 (0.6)
Sometimes (3)	39 (12.0)	64 (20.4)
Occasionally (4)	66 (20.4)	63 (20.1)
Never (5)	195 (60.2)	179 (57.0)
How confident are you filling out medical forms by yourself?
Not at all (1)	0 (0.0)	1 (0.3)
A little bit (2)	12 (3.7)	13 (4.1)
Somewhat (3)	44 (13.6)	31 (9.9)
Quite a bit (4)	57 (17.6)	164 (52.2)
Very (5)	211 (65.1)	105 (33.4)

Note. GED = general equivalency diploma; STOFHLA = Short Test of Functional Health Literacy in America. The sample size was n = 638.

^aChi-square analysis for each sociodemographic characteristic revealed significant differences between English and Spanish speakers (P < .001).

^bIncludes “don’t know” or “did not want to reply.”

^cOne Spanish-speaking participant was excluded because of missing data.

^dSTOFHLA: inadequate (score = 0–16) vs marginal or adequate (score = 17–36).

In analyses stratified by language, we calculated area under the receiver operating characteristic (AUROC) curves comparing each item with dichotomized STOFHLA scores. This comparison used a nonparametric approach^17,18 based on generalized U-statistics theory following a χ² distribution. Past literature examining single-item measures indicated that AUROC curve values greater than 0.7 are justified for use in health care settings^5,9,19,20; therefore, we used this value to compare performance separately for English- and Spanish-speaking samples. We calculated sensitivity and specificity with cutpoints suggested by the AUROC curves. Analyses were conducted in SAS version 9.3 (SAS Institute, Cary, NC).

RESULTS

Compared with English speakers, Spanish speakers were more likely to be White, middle-aged, married or living with a partner, and less educated (Table 1). More Spanish than English speakers received an inadequate STOFHLA score (18.5% vs 2.8%). More Spanish than English speakers rated “ability to read” as no better than “OK” (25.8% and 15.8%). “Help reading hospital materials” followed a similar pattern—20.4% of Spanish and 12.0% of English speakers reported “sometimes” having help. For “confidence filling out medical forms,” 85.6% and 82.7% of Spanish and English speakers, respectively, were “quite a bit or very confident.”

AUROC curve estimates for English and Spanish speakers had similar ranges (Table 2). For English speakers, there were no significant differences among the 3 questions (P = .39). For Spanish speakers, “ability to read” identified inadequate literacy better than did “help reading hospital materials” (AUROC curve = 0.76 vs 0.65; P = .019). Overall, 95% confidence intervals were wider (indicating less precision) for English speakers. Table 2 reports sensitivity and specificity based on optimal cutpoints suggested by the AUROC curves.

TABLE 2—

Area Under the Receiver Operating Characteristic (AUROC) Curves, Sensitivity, Specificity, and 95% Confidence Intervals (CIs) Comparing Each Health Literacy Screening Item With the STOFHLA, Stratified by Language: Dallas County, TX, 2011–2012

	English (n = 324)			Spanish (n = 314)
	AUROC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	AUROC (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)
How would you rate your ability to read? (very poor/poor/OK/good vs very good)	0.73 (0.56, 0.91)	0.78 (0.40, 0.97)	0.60 (0.55, 0.66)	0.76 (0.70, 0.83)	0.93 (0.83, 0.98)	0.30 (0.25, 0.37)
How confident are you filling out medical forms by yourself? (not at all/a little bit/somewhat/quite a bit vs very)	0.77 (0.60, 0.94)	0.78 (0.40, 0.97)	0.66 (0.61, 0.72)	0.69 (0.62, 0.76)	0.84 (0.73, 0.93)	0.38 (0.32, 0.44)
How often do you have someone help you read hospital materials? (always/often/sometimes/occasionally vs never)	0.66 (0.47, 0.84)	0.67 (0.30, 0.93)	0.61 (0.55, 0.66)	0.65 (0.57, 0.73)	0.59 (0.45, 0.71)	0.61 (0.54, 0.67)

Note. The sample size was n = 638. AUROC curves compare each item with STOFHLA scores dichotomized as inadequate vs marginal or adequate. STOFHLA (Short Test of Functional Health Literacy in America): inadequate (score = 0–16) vs marginal or adequate (score = 17–36).

DISCUSSION

Single-item health literacy screening tools have advantages over longer, time-consuming standard measures. Administration is quick and can be performed by any health care team member. Results are easy to interpret and document in the electronic health records and can guide providers’ communication practices.

Our findings from a community sample indicated support for the “ability to read” item. It performed the best at distinguishing inadequate from marginal or adequate health literacy (AUROC curve = 0.73 and 0.76 for English and Spanish speakers, respectively). Although our findings for English speakers were consistent with those of past studies,^5,9,10,19 the same was not true for our Spanish-speaking sample. Specifically, our AUROC curve estimates for “confidence filling out medical forms” and “help reading hospital materials” were lower than those previously reported^9,20 and slightly lower than the 0.7 threshold recommended by the literature.^5,8,9,19,20 Differences in AUROC curve estimates between our study and previous studies may stem from sample characteristics and STOFHLA score distributions, highlighting the importance of replication studies that build evidence for validity in a variety of populations. Previous studies examined item performance with chronically ill populations who have more frequent contact with health care systems and more opportunity to practice their health literacy skills. In addition, the Sarkar et al.⁹ sample had a wider distribution of STOFHLA scores.

Our study had some limitations. One limitation was that few participants gave responses at the lower threshold of the 3 questions. Eligibility criteria requiring ability to read in English or Spanish (to complete STOFHLA) may have excluded individuals with the lowest health literacy levels.⁴ Second, the STOFHLA is well accepted but is not a gold standard; thus, the degree to which it is an imperfect reference standard potentially introduced inaccuracy in the AUROC analysis.²¹ We did not measure social desirability, so we cannot determine the extent to which that type of bias affected participants’ survey responses.²² Finally, researchers acknowledge that health literacy is a multidimensional construct, but there is little agreement about what dimensions must be assessed. The single items we examined focus on reading comprehension.⁴ These measures do not assess verbal communication, ability to navigate the health care system, or health care decision-making. It is unclear whether awareness of these skills would improve the delivery of health care.

This study highlights the importance of replication studies in building evidence for validity in various populations. Our study contributes data on the health literacy of Dallas County community members, of whom 39% are Hispanic, 35% speak Spanish at home, and 23% are uninsured.^23,24 Given our low sensitivity and specificity estimates, alternative screening items should be evaluated. Future studies also should investigate how to implement brief measures in safety-net settings²² and whether highlighting health literacy level influences providers’ communication practices and, in turn, patient outcomes.