Comparison of Newest Vital Sign and Brief Health Literacy Screen Scores in a Large, Urban Hispanic Cohort

Abstract

Objective:

Prior studies comparing subjective and objective health literacy measures have yielded inconsistent results. Our aim was to examine the concordance between Newest Vital Sign (NVS) and Brief Health Literacy Screen (BHLS) scores in a large cohort of English- and Spanish-speaking urban Hispanic adults.

Methods:

Item means, standard deviations, corrected-item total correlations, Cronbach’s alpha, and Spearman correlations and area under receiver operating characteristic (AUROC) curve analysis were used to compare NVS and BHLS items and total scores.

Results:

N = 2,988 (n = 1,259 English; n = 1,729 Spanish). Scores on both measures demonstrated good internal consistency (NVS: α = .843 English, .846 Spanish; BHLS: α = .797 English, .846 Spanish) but NVS items had high difficulty; more than half of respondents scored 0. Measures were only weakly correlated (r_s = .21, p < .001, English; r_s = .19, p < .001, Spanish). The AUROC curves were .606 (English) and .605 (Spanish) for discriminating the lowest NVS scoring category.

Conclusion:

Subjective health literacy scores were poor predictors of objective scores. Objective scores demonstrated floor effects, precluding discrimination at low levels of the trait continuum.

Practice Implications:

Subjective health literacy scores may fail to identify individuals with limited health literacy.

1. Introduction

The Newest Vital Sign (NVS) is an objective measure of health literacy, frequently used in research because of its brevity and purported criterion-related validity in relation to longer “gold standard” measures like the Test of Functional Health Literacy in Adults (TOFHLA, and its short form S-TOFHLA) and the Rapid Estimate of Adult Literacy in Medicine (REALM)[1]. The Brief Health Literacy Screen (BHLS) [2] is a subjective measure of health literacy that is attractive for use in clinical practice and research because it can be self-administered, is even shorter the NVS, and there is some evidence that it yields scores that are good predictors of NVS scores [3]. However, the psychometric performance of these measures—individually and in relation to each other—has varied by population and language of administration [4,5] and thus may not be equally appropriate for use with all populations and sub-populations. Hispanic/Latinx¹ people are the largest ethnic minority group in the US representing 18.5% of the total population but are not a monolith; there is cultural heterogeneity related to countries of origin [6]. Therefore, it is important to understand how these commonly-used health literacy measures perform within individual subgroups to ensure the validity and rigor of any attendant health literacy research.

1.1. Study Purpose

The purpose of this study is to describe the psychometric characteristics of NVS and BHLS scores in English and Spanish in a large cohort of English- and Spanish-speaking urban Hispanic respondents, and to examine the concordance between them.

1.2. Newest Vital Sign (NVS)

The NVS is an objective, interviewer-administered measure of health literacy based on six question items about a nutrition label on a container of ice cream [1]. Administration takes 3-5 minutes. The NVS has an emphasis on numeracy, as the first four items require simple numerical calculations (e.g., multiplying the calories per serving by the total number of servings). Item 5 asks if it would be safe for a person with a list of four allergies to eat the ice cream. Only if the respondent answers correctly (i.e., “no”) are they asked item 6 (“Why not?”) to which they should respond that it is because the ice cream contains peanut oil. If item 5 is answered incorrectly, item 6 is also marked as incorrect. The total score is the number of items correct (0 to 6). This value can then be converted into one of three scoring categories: high likelihood (50% or more) of limited literacy (0 to 1), possibility of limited literacy (2 to 3), and almost always indicates adequate literacy (4-6).

The NVS was developed simultaneously in English and Spanish and validated with 250 English- and 250 Spanish-speaking respondents in Tucson, Arizona [1]. Cronbach’s alpha was reported as .76 for English and .69 for Spanish. To assess criterion-related validity, NVS scores were compared to scores on the TOFHLA. Weiss et al. (2005) reported significant Pearson r correlations (.59 English; .49, Spanish) and area under the ROC curve as .88 (95% CI, .84 - .93; p < .001) for English and .72 (95% CI, .66 - .79; p < .001) for Spanish.

1.3. Brief Health Literacy Screen (BHLS)

The BHLS² is a subjective measure of health literacy consisting of three self-report items about the degree to which respondents: 1) are confident filling out medical forms independently, 2) have problems learning about a medical condition because of difficulty with written information, 3) need help interpreting health-related materials [11]. Exact item wording and response options has varied slightly between studies. For instance, some studies specify the language of the medical forms (item 1) [12] and others do not. Typically, there are five response options that indicate frequency or intensity (e.g., ‘None of the time’ to ‘All of the time’; ‘Never’ to ‘Always’). BHLS items were originally proposed as single-item screeners because “combining questions resulted in no meaningful improvement in their screening performance of detecting inadequate or marginal health literacy” when benchmarked against the S-TOFHLA (N = 332) (Chew et al., 2004, p. 591). A second, larger validation study by Chew et al. [2] (N = 1,796) reported that Confident with Forms had the largest AUROC curve of the three items for detecting inadequate health literacy on the S-TOFHLA (.74, 95% CI, .69 - .79) and on the REALM (.84, 95% CI, .79 - .89). Stagliano & Wallace [3] found that Confident with Forms (English) was a useful predictor of NVS scores and reported AUROC curves ranging from .79 (95% CI, .73–.85) to .83 (95% CI, .76–.89).

Despite Chew et al.’s (2004) recommendation to use just one item, some studies have used all three items and treated them as a scale (e.g., [13]). When all three items are used, Confident with Forms must be reverse coded in relation to the other two items.

1.4. Research Context

This study of health literacy measures is drawn from a larger study, Washington Heights/Inwood Informatics Infrastructure for Community-Centered Comparative Effectiveness Research (WICER), the aim of which was to characterize the health of these two northern Manhattan neighborhoods. The large majority of neighborhood residents have roots in the Dominican Republic. In addition to measures of health literacy, the WICER survey included varied measures such as overall health, blood pressure, height, weight, waist circumference, fruit and vegetable consumption, physical activity levels, anxiety, depression, and chronic stress [14,15]. More than 6,700 respondents participated in the baseline WICER survey (2010-2013) across three types of data collection sites (Columbia Community Partnership for Health, community settings including participant households, ambulatory care clinics); of those, nearly half completed a follow-up survey approximately one year later. Data in this study were drawn from the follow-up surveys and were collected in community members’ households and other community settings and ambulatory care clinics.

2. Methods

2.1. Setting and Sample

This study was approved by the Columbia University Irving Medical Center Institutional Review Board. The follow-up WICER survey was administered verbally by bilingual research staff in community settings or in private rooms in two ambulatory care clinic sites within the WICER catchment area (clinical sample). Eligibility for the WICER survey was based on age (18 or over), home zip code in Washington Heights or Inwood, and ability to speak English or Spanish. For this analysis, only respondents self-reporting Hispanic ethnicity were included. Additional criteria for the analysis were complete data on the 1-year Follow-Up survey for the health literacy measures, defined as responses to all three BHLS items and at least five of the NVS items³.

2.2. Instruments

Both the NVS and BHLS were collected. The NVS was scored as described in section 1.2. Response options from ‘Never’ to ‘Always’ were used for all three BHLS items. BHLS items were coded such that total scores ranged from 3 to 15 with high scores indicating high subjective health literacy. We also collected demographic characteristics (age, education, health insurance, place of birth, marital status, overall self-reported health, and for those born outside the 50 US states, age at arrival in the US and years living in the US).

2.3. Procedures

Informed consent was obtained from all respondents. Respondents were recruited via probability and snowball sampling for the community sample and convenience sampling for the clinical sample. Survey questions were administered, interview style, by bilingual study staff in either English or Spanish, per respondent preference. Administration of the complete [study name] survey lasted about 45-75 minutes. Respondents received $25 in grocery coupons, transit tickets, or movie vouchers for their time.

2.4. Analysis

Data were cleaned through automated and manual measures using REDCap [16] and SAS 9.4. We used SPSS Statistics 27 for analysis. We calculated descriptive statistics. Cronbach’s alpha was calculated to evaluate internal consistency of NVS and BHLS scores. Mean, standard deviation, corrected-item total correlation, and alpha if item deleted were calculated for all NVS and BHLS items. We examined bivariate relationships with Spearman correlations. Statistical significance was set at p < .001. Demographic variables that are known correlates of health literacy [17] and had bivariate correlations > |0.2| were entered into multiple linear regression models to explore predictors of NVS and BHLS scores. We conducted an area under receiver operating characteristic (AUROC) curve analysis to examine the ability of the BHLS to discriminate between NVS scoring categories.

3. Results

3.1. Participants

Participant characteristics are summarized in Table 1. Of the total respondent pool (N = 2,988), 58% responded in Spanish (n = 1,729) and 42% in English (n = 1,259). The mean age was 53 years (SD = 16). Thirty-four percent described their educational attainment as 8^th grade or less, 16% had some high school, 24% had a high school diploma, and 12% had some college or technical/vocational training. For health insurance, 53% had Medicaid, 5% had Medicare, and 19% had both. Most participants were born in the Dominican Republic (78%) and 13% were born in the United States. For all those born outside the US, mean age of arrival was 29 (SD = 14) and mean number of years living in the US was 28 (SD = 14).

Table 1.

Participant Characteristics by Language of Survey Administration

	English (n = 1,259)		Spanish (n = 1,729)		Total (N = 2,988)
	n	(%)	n	(%)	n	(%)	p-value
Recruitment Setting							< .001
Household	1,178	(93.6)	1,386	(80.2)	2,564	(86)
Ambulatory Care Clinic	81	(6.4)	343	(19.8)	424	(14)
Gender							.55
Female	929	(74)	1,290	(75)	2,219	(74)
Male	328	(26)	436	(25)	764	(26)
Transgender (male to female)	1	(< 0.1)	0	(0)	1	(< 0.1)
Not reported	1	(< 0.1)	3	(0.2)	4	(< 0.1)
Education							.42a
Never went to school	19	(2)	35	(2)	54	(2)
Eighth grade or less	430	(34)	590	(34)	1,020	(34)
Some high school	177	(14)	305	(18)	482	(16)
High school diploma	315	(25)	398	(23)	713	(24)
Some college or technical/vocational	149	(12)	198	(12)	347	(12)
Associate’s degree	60	(5)	72	(4)	132	(4)
Bachelor’s degree	83	(7)	103	(6)	186	(6)
Master’s degree	13	(1)	11	(0.6)	24	(1)
Doctoral degree	1	(0.1)	2	(0.1)	3	(0.1)
Not reported	12	(1)	15	(1)	27	(1)
Health Insurance							.19b
Medicare & Medicaid	256	(20)	303	(18)	559	(19)
Medicaid	671	(53)	926	(54)	1,597	(53)
Medicare	53	(4)	82	(5)	135	(5)
Private	72	(6)	73	(4)	145	(5)
Other	38	(3)	58	(3)	96	(3)
None	89	(7)	120	(7)	209	(7)
Not reported	80	(6)	167	(10)	247	(8)
Place of Birth							.02c
Dominican Republic	966	(77)	1,377	(80)	2,343	(78)
United States	194	(15)	206	(12)	400	(13)
Argentina	1	(0.1)	1	(0.1)	2	(< 0.1)
Colombia	12	(1)	10	(0.6)	22	(0.7)
Cuba	12	(1)	14	(0.8)	26	(0.9)
Ecuador	13	(1)	29	(2)	42	(1)
El Salvador	3	(0.2)	9	(0.5)	12	(0.4)
Guatemala	1	(0.1)	3	(0.2)	4	(0.1)
Haiti	0	(0)	1	(.01)	1	(< 0.1)
Honduras	2	(0.2)	2	(0.1)	4	(0.1)
Mexico	10	(0.8)	25	(1)	35	(1)
Nicaragua	3	(0.2)	4	(0.2)	7	(0.2)
Peru	2	(0.2)	2	(0.1)	4	(0.1)
Puerto Rico	21	(2)	34	(2)	55	(2)
Venezuela	2	(0.2)	2	(0.1)	4	(0.1)
Not reported	17	(1)	10	(0.6)	27	(1)
Marital Status							.21
Single, never married	373	(30)	555	(32)	928	(31)
Married	391	(31)	515	(30)	906	(30)
Divorced	297	(24)	351	(20)	648	(22)
Widowed	74	(6)	113	(7)	187	(6)
Living with partner	47	(4)	60	(4)	107	(4)
Not reported	77	(6)	135	(8)	212	(7)
Would you say that in general your health is…?							.82d
Poor	31	(3)	26	(2)	57	(2)
Fair	267	(21)	361	(21)	628	(21)
Good	291	(23)	379	(22)	670	(22)
Very Good	250	(20)	410	(24)	660	(22)
Excellent	343	(27)	382	(22)	725	(24)
Not reported	77	(6)	171	(10)	248	(8)
	M	SD	M	SD	M	SD
Mean age	53	16	53	16	53	16
Age at arrival in the United States	29e	13e	29f	14f	29g	14g
Years living in the United States	28e	14e	27f	14f	28g	14g

Note.

^a8^th grade or less, some high school or diploma, more than high school;

^bMedicaid, Medicare, dually eligible, all others;

^cDominican Republic, United States, all others;

^dPoor or Fair vs. Good or Very Good or Excellent;

^en = 960;

^fn = 1,410;

^gn = 2,370

3.2. NVS Scale and Item Analyses

Scale and item statistics for the NVS are shown in Table 2. For English, Cronbach’s alpha was .843, scale mean was 1.48 (SD = 1.93) and item means ranged from .21 to .30. For Spanish, Cronbach’s alpha was .846, scale mean was 1.28 (SD = 1.83) and item means ranged from .13 to .29. All items had strong corrected item-total correlations (.523 - .691, English; .493 - .741, Spanish).

Table 2.

Scale and Item Statistics for Newest Vital Sign and Brief Health Literacy Screener by Language

Item	English						Spanish
	α	M	SD	C-ITC	α if item deleted	α	M	SD	C-ITC	α if item deleted
NVS	.843a					.846b
1		.24	.43	.523	.836		.18	.38	.536	.837
2		.30	.46	.691	.803		.29	.45	.741	.797
3		.22	.41	.585	.825		.13	.34	.493	.844
4		.24	.43	.631	.816		.17	.38	.554	.834
5		.27	.44	.658	.810		.28	.45	.735	.798
6		.21	.41	.647	.813		.23	.42	.705	.805
Scale		1.48	1.93				1.28	1.83
BHLS	.797c					.846d
1		3.52	1.12	.548	.814		3.38	1.13	.655	.843
2		3.34	1.38	.669	.697		3.09	1.34	.710	.793
3		3.71	1.23	.723	.634		3.37	1.31	.792	.706
Scale		10.57	3.16				9.84	3.31

Note. C-ITC = corrected item-total correlation. Cases with missing data (81 for NVS English; 112 for NVS Spanish) were excluded from analysis.

^an = 1,178.

^bn = 1,617.

^cn = 1,259.

^dn = 1,729.

3.3. BHLS Scale and Item Analyses

Scale and item statistics for the BHLS are shown in Table 2. For English, Cronbach’s alpha was .797, scale mean was 10.57 (SD = 3.16) and item means ranged from 3.34 (item 2) to 3.71 (item 3). For Spanish, Cronbach’s alpha was .846, scale mean was 9.84 (SD = 3.31) and item means ranged from 3.09 (item 2) to 3.38 (item 1). All items had strong corrected item-total correlations (.548 - .723, English; .655 - .791, Spanish). In Spanish, no item would have yielded a higher alpha if deleted. However, in English, Cronbach’s alpha would increase to .814 if item 1 were deleted.

3.4. Score Distributions

Score distributions for the NVS and BHLS are shown in Figure 1 and Table 3. A bimodal distribution is visible for the NVS with 61% of respondents scoring 0 and the rest distributed normally from 1 to 6. Total BHLS scores are distributed normally from scores of 4 to 13 but peak again at the highest score level.

Figure 1.

Score Distributions for Total NVS Scores, Total BHLS scores, and Individual BHLS Items

Note. The weak correlation between NVS total scores (top left) and BHLS total scores (top right) is explained by the dissimilarity in their distributions. With few exceptions, scores for all measures were fairly consistent across languages. Results shown include NVS scores with up to one item missing (n = 81, English; n = 112, Spanish). Of those with a total NVS score of zero, one item was missing for n = 11 for English and n = 34 for Spanish. There were no missing data for BHLS scores.

Table 3.

Score distributions for NVS and BHLS

	English (n = 1,259)	Spanish (n = 1,729)	Total (N = 2,988)
	n (%)	n (%)	n (%)
NVS category and total score
High likelihood of limited literacy	741 (59)	1,079 (62)	1,820 (61)
0	692 (55)	1,015 (59)	1,707 (57)
1	49 (4)	64 (4)	113 (4)
Possibility of limited literacy	230 (18)	353 (20)	583 (20)
2	116 (9)	155 (9)	271 (9)
3	114 (9)	198 (12)	312 (10)
Almost always indicates adequate literacy	288 (23)	297 (17)	585 (20)
4	177 (14)	171 (10)	348 (12)
5	58 (5)	66 (4)	124 (4)
6	53 (4)	60 (4)	113 (4)
1. How confident are you filling out medical forms by yourself / ¿Qué tan seguro(a) se siente al llenar formas usted solo(a)?
(1) Never / Nada	85 (7)	158 (9)	243 (8)
(2) Occasionally / Un poco	104 (8)	142 (8)	246 (8)
(3) Sometimes / Algo	418 (33)	594 (34)	1,012 (34)
(4) Often / Bastante	375 (30)	559 (32)	934 (31)
(5) Always / Extremadamente	277 (22)	276 (16)	553 (19)
2. How often do you have problems learning about your medical condition because of difficulty understanding written information? / ¿Qué tan seguido tiene problemas aprendiendo sobre su condición miéica porque es dificil de entender información escrita?
(1) Always / Siempre	113 (9)	204 (12)	317 (11)
(2) Often / A menudo	295 (23)	459 (27)	754 (25)
(3) Sometimes / Algunas veces	326 (26)	467 (27)	793 (27)
(4) Occasionally / Rara vez	105 (8)	184 (11)	289 (10)
(5) Never / Nunca	420 (33)	415 (24)	835 (28)
3. How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy? / ¿Qué tan seguido necesita usted tener a alguien que le ayude cuando usted lee instrucciones, folletos u otros materiales escritos por su doctor o su farmacia?
(1) Always / Siempre	76 (6)	189 (11)	265 (9)
(2) Often / A menudo	134 (11)	257 (15)	391 (13)
(3) Sometimes / Algunas veces	334 (26)	456 (26)	790 (26)
(4) Occasionally / Rara vez	251 (20)	375 (22)	626 (21)
(5) Never / Nunca	464 (37)	452 (26)	916 (31)
BHLS Total Score
3 (lowest literacy)	42 (3)	115 (7)	157 (5)
4	16 (1)	29 (2)	45 (2)
5	16 (1)	37 (2)	53 (2)
6	24 (2)	40 (2)	64 (2)
7	59 (5)	122 (7)	181 (6)
8	108 (9)	175 (10)	283 (9)
9	288 (23)	323 (19)	611 (20)
10	159 (13)	273 (16)	432 (14)
11	96 (8)	143 (8)	239 (8)
12	60 (5)	69 (4)	129 (4)
13	38 (3)	46 (3)	84 (3)
14	135 (11)	126 (7)	261 (9)
15 (highest literacy)	218 (17)	231 (13)	449 (15)

Note. Percentages may not sum to 100 because of rounding.

3.5. Correlation Between NVS and BHLS

As shown in Table 3, the Spearman correlations between total BHLS scores and total NVS scores were statistically significant but weak for both languages (r_s = .21, p < .001, English; r_s = .19, p < .001, Spanish). The cross-tabulation of the BHLS items with NVS categories in Figure 2 demonstrates that although the majority of respondents were in the lowest NVS category (i.e., “high likelihood of limited literacy”) their ratings on BHLS items spanned all five levels of subjective literacy.

Figure 2.

Cross-Tabulation Heat Map of NVS Categories and BHLS Item Score Levels by Language of Administration

Note. English n = 1,259 (top row); Spanish n = 1,729 (bottom row). Values for each 3x5 matrix sum to 100 and represent the proportion of respondents who fell within that cell. Darker shades represent a larger proportion of participants. This heat map demonstrates that although the majority of respondents were in the lowest NVS category (i.e., lowest level of objective health literacy) their ratings on BHLS items nevertheless spanned all five levels of subjective literacy.

3.6. Relationships with Demographic Variables

We examined age, age at arrival in the US, years since arrival in the US, education, and self-reported health for bivariate correlations > |0.2| with total NVS and BHLS scores. Based on these, we selected age and education for multiple regression models, shown in Table 4. Together, age and education accounted for 13% of the variance in total NVS scores in English but only 6% in Spanish (R² = .13, p < .001; R² = .06, p < .001). Age and education accounted for 14% (English) and 15% (Spanish) of the variance in total BHLS scores (R² = .14, p < .001; R² = .15, p < .001). In all four models, controlling for education decreased the impact of age.

Table 4.

Multiple Regressions of Associations Between Demographic Variables and Health Literacy Scores

Variable	Total NVS Score, English			Total NVS Score, Spanish			Total BHLS Score, English			Total BHLS Score, Spanish
	B	SE	β	B	SE	β	B	SE	β	B	SE	β
Constant	1.19*	.28		1.99*	.24		9.75*	.45		8.91*	.40
Age	−.02*	.00	−.13	−.02*	.00	−.18	−.02*	.00	−.12	−.03*	.01	−.13
Education	.33*	.04	.28	.12*	.03	.10	.58*	.06	.30	.66*	.05	.31
R	.36			.24			.37			.39
R2	.13			.06			.14			.15

^*p < .001

3.7. Area Under the Receiver Operating Characteristic Curve (AUROC)

For BHLS total scores, the AUROC curves were .606 (English) and .605 (Spanish) for discriminating the lowest NVS lowest scoring category, as shown in Figure 3 and Table 5.

Figure 3.

Area Under the Receiver Operating Characteristic (AUROC) Curves

Table 5.

Area Under the Receiver Operating Characteristic (AUROC) Curves

Test Result Variables	NVS High Likelihood of Limited Literacy
	English	Spanish
BHLS Total	.606	.605
BHLS Item 1	.611	.586
BHLS Item 2	.558	.523
BHLS Item 3	.566	.617

4. Discussion and Conclusion

4.1. Discussion

In this study, scores for both the NVS and BHLS demonstrated good internal consistency but were weakly correlated with each other. Although the correlations were statistically significant, they were likely driven by the sample size (N = 2,988). These results are in contrast to Stagliano and Wallace [3] who found that BHLS scores were good predictors of NVS scores in English among non-Hispanic participants with higher educational attainment than our sample had. Of note in our sample, although the majority of respondents were in the lowest NVS category, their ratings on BHLS items spanned all five levels of subjective literacy. This suggests that individuals may overestimate their literacy or perhaps that the numeracy requirements to score high on the NVS are different than what is needed to complete typical tasks related to health care.

Our results are consistent with Waters et al. [18], who examined the English NVS and BHLS scores of 1,005 respondents (9.7% Hispanic) as indicators of latent factors (objective and subjective literacy, respectively) and found a correlation of only .18 (p < .001) between the factors. They are also similar to those of Gutierrez and colleagues [19] who surveyed 265 patients in a public clinic in Dallas, TX (62% Hispanic, 54% in Spanish) and found that 53% rated Confidence with Forms highly but only 18.9% had NVS scores indicating adequate literacy.

Other studies with Spanish-speaking cohorts reported somewhat better performance by Confident with Forms against criterion measures than we found. Hadden et al. [12] surveyed 200 Spanish-speaking participants using the NVS as a benchmark and two versions of Confident with Forms that specified the language of the forms. They found that Confident with English Forms had an AUROC curve of .70 whereas it was .63 (confidence intervals were not reported) for Confident with Spanish Forms. Cordasco et al. [20] studied a geriatric, monolingual Spanish-speaking cohort (N = 134) and found that although Confident with Forms reasonably discriminated S-TOFHLA scores (AUROC .82, 95% CI .75–.89) educational attainment was slightly better (AUROC .88, 95% CI .78–.97); the other two BHLS items had AUROC curves less than .50.

Given the limitations of our study design, we can only speculate as to what accounts for the weak relationship between the BHLS and NVS in both English and Spanish for Hispanic participants. We propose several potential explanations. First, NVS items were of very high difficulty for this population and demonstrated floor effects, with 61% of all respondents scoring 0. Perhaps a lower-difficulty measure might have produced more score variability and potentially a stronger correlation with BHLS scores. However, the TOFHLA has lower difficulty than the NVS [21,22] and studies in the US that compared the English BHLS or its items to TOFHLA or S-TOFHLA scores have found widely varying correlations (r_s = .21 to .49) and AUROC curves (.62 to .87), so no clear pattern emerges [7,23,24].

A second potential explanation is that BHLS scores could have been inflated due to social desirability bias or overconfidence on the part of the respondents. Health care providers tend to overestimate patients’ literacy [25,26], so it is reasonable to question the accuracy of individuals’ assessment of their own abilities [27] and their subsequent confidence in perceived abilities. These potential pitfalls are inherent limitations of subjective measures and may not be easily remedied.

Lastly, it is possible that the problem is conceptual. The premise underlying comparison of the BHLS to the NVS is that subjective and objective health literacy (with an emphasis on numeracy) are closely related concepts. If the NVS and BHLS both have good construct validity, our study calls that premise into question and raises concerns about the utility of these concepts. In light of the many and varied definitions of health literacy and in the varying ways those definitions have been operationalized into measurement, there has been a call for an improved definition of health literacy that better acknowledges its contextual nature [28] and its importance as a social determinant of health [29]. The Department of Health and Human Services (DHHS) recently answered this call with updated definitions for personal health literacy and organizational health literacy [30]. This change warrants a re-evaluation of health literacy measurement methods to ensure they are keeping pace with evolving definitions.

4.1.1. Strengths and Limitations

A notable strength of this study is the very large sample size of our cohort (N = 2,988) and the fact that the English and Spanish groups were very similar along demographic characteristics. Limitations include that it was a convenience sample with a disproportionate number of women. Also, the wording of the BHLS items did not specify whether or not the forms, written materials, etc. were in the respondent’s preferred language. Our participants were overwhelmingly from or had roots in the Dominican Republic; results may not be applicable to Hispanic people from other regions.

4.2. Conclusion

As a scale, the BHLS had good internal consistency in both English and Spanish but had poor ability to discriminate between NVS score categories in a large, urban Hispanic cohort. The NVS also functioned well in both languages and had the advantage of sidestepping the potential pitfalls associated with subjective self-report measures. However, the NVS was a very high difficulty measure in this population which may make it clinically useful for identifying people at risk of inadequate health literacy, but yields score distributions with strong floor effects that preclude discrimination at low levels of the trait continuum, and thus limiting its utility for some research studies.

4.3. Practice Implications

Measures of subjective health literacy may fail to identify individuals with limited literacy and so objective measures may be preferable. Researchers desiring score variability within lower levels of health literacy may want to use objective measures that are of lower difficulty than the NVS.

Highlights.

• Comparison of subjective and objective measures of health literacy

• Subjective scores were poor predictors of objective scores among Hispanic adults

• Results consistent across English and Spanish versions of both measures

• High difficulty of objective measure led to floor effects

• Subjective measure may lack utility for research and practice