Performance of a brief survey to assess health literacy in patients receiving hemodialysis

Abstract

Background

Health literacy is associated with important outcomes among patients with kidney disease, but widely used measures of health literacy can be burdensome. In an effort to make a practical assessment available, we compared the performance of the three-item brief health literacy screen (BHLS) to other widely used measures of health literacy among patients with end-stage renal disease (ESRD).

Methods

Adult hemodialysis patients (n = 150) from four urban dialysis facilities participated in a cross-sectional study from 2009 to 2012. Three health literacy measures were administered including (i) the rapid estimate of adult literacy in medicine (REALM), (ii) the short test of functional health literacy in adults (S-TOFHLA) and (iii) the three-item BHLS. The mini-mental state exam assessed cognitive status, and the chronic hemodialysis knowledge survey (CHeKS) and perceived kidney disease knowledge survey (PiKS) assessed kidney knowledge. Spearman's ρs and area under the receiver-operating curves examined relationships between the aforementioned variables.

Results

Participants had received dialysis for a mean of 4.6 years. They were 49% female, 73% African American and averaged 52 years of age. Less education and less cognitive capacity were each associated (P < 0.05) with lower health literacy for all three health literacy measures. Performance on the BHLS was significantly associated with the REALM [0.35 (95% confidence interval (95% CI): 0.20–0.49); P < 0.001] and S-TOFHLA [0.49 (95% CI: 0.35–0.69); P < 0.001], the CHeKS [0.43 (95% CI: 0.28–0.55); P < 0.001] and PiKS [0.41 (95% CI: 0.27–0.54); P < 0.001].

Conclusions

The BHLS demonstrates evidence of construct validity among ESRD patients. Furthermore, health literacy was associated with kidney knowledge, supporting it as a potential intervention target to improve outcomes among patients with lower health literacy.

Introduction

Health literacy represents the communication among individuals, caregivers, providers and other constituents of a healthcare organization to promote the successful transfer, understanding, and application of information to make informed health decisions [1, 2]. Estimates suggest that more than 90 million people in the USA have low health literacy, and the few available studies report a prevalence of low health literacy as high as 50% among patients receiving dialysis care [3–8]. It is suggested that there are similar observations worldwide [9]. In patients with kidney disease and other conditions, low health literacy is associated with less knowledge of one's health condition, less participation in self-care and a higher risk of hospitalizations and mortality [3, 5, 10, 11]. Patients need to routinely process recommendations related to medications, diet, dialysis prescription time and visits to other healthcare providers necessary to optimally execute dialysis-specific self-care. Additionally, cognitive impairment is common in dialysis patients, associated with less successful self-care and likely contributes to an individual's health literacy skills [12]. Health literacy has been recognized as a potential factor in patient-centered kidney disease care and reports elsewhere provide a comprehensive discussion [13, 14]. Despite the recognition of health literacy as a key component of kidney disease care [14], current reports are limited by small sample sizes and there have been no trials, to our knowledge, testing health literacy intervention strategies to improve outcomes in this population.

The personnel and time resources required to assess health literacy using common measures may be an important explanation for its narrow application in kidney disease. Two frequently reported assessments, the rapid estimate of adult literacy in medicine (REALM) [15, 16] and the short test of functional health literacy in adults (S-TOFHLA) [17], require administration by trained personnel and may take up to 12 min to complete. In contrast, the three-item brief health literacy screen (BHLS), requiring 1–2 min or less, has demonstrated validity across diverse patient populations and has been associated with important clinical outcomes including hospitalizations and death [18–21]. However, the validity of the BHLS in patients with end-stage renal disease (ESRD) has not been evaluated. Understanding the characteristics of health literacy assessment among patients receiving dialysis care is needed to both advance its appraisal within existing large cohort research studies and to inform clinical care as advocated by the Institute of Medicine [22].

The aim of this study was to determine the validity of the BHLS among patients receiving chronic hemodialysis therapy for ESRD. First, we examined the BHLS' concurrent validity by relating it to the two most widely used measures health literacy. Second, to generate further evidence of construct validity, we examined the BHLS' relationship with cognitive status and two measures of dialysis knowledge.

Materials and methods

Study population

From June 2009 to February 2012, we conducted a cross-sectional study at four outpatient dialysis facilities in the Nashville, Tennessee area, including two affiliated with Vanderbilt University. Eligible participants were adults age >18 and <80, receiving chronic hemodialysis for at least 1 month who spoke self-reported fluent English. Patients, who had a known diagnosis of dementia, psychosis or cognitive impairment, as well as those with poor visual acuity, were excluded from participation. Potential participants were identified by review of clinic rosters and consultation with clinical providers. Vanderbilt University Institutional Review Board approved all study procedures prior to participant enrollment.

Data collection

A trained research assistant administered the measures at a time pre-selected by the participant. The majority of participants selected to perform the interview just prior to or just after initiating the dialysis procedure. Participants provided written informed consent and received a small monetary compensation.

In addition to providing demographic information, each participant completed the mini-mental state exam (MMSE) [23] as a measure of general cognitive performance and three assessments of health literacy including the REALM [15, 16] and the TOFHLA [24]. The TOFHLA is a timed 50-item reading comprehension and 17-item numeracy item test. In this study, the 36 reading comprehension items that contribute to the S-TOFHLA [17] were used to calculate the final score from this assessment. This was chosen as the assessment because all prior reported research compares the BHLS to the S-TOFHLA, and in this study the S-TOFHLA demonstrated excellent correlation with the TOFHLA reading score (r = 0.90; P < 0.0001). In specific analyses, these scores were categorized as limited (0–22) or adequate (23–36) [17, 24]. The REALM score was similarly categorized (limited = 0–61; adequate = 61–66) [16]. In addition, the BHLS consisted of three questions derived from those previously validated in large inpatient [19–21] and outpatient samples [25]: (i) How confident are you filling out forms by yourself? (ii) How often do you have someone help you read hospital materials? and (iii) How often do you have problems learning about your medical condition because of difficulty reading hospital materials? Each question is scored on a 5-point response scale and is summed to produce a total score ranging from 3 to 15. This score was divided into lower (3–9) and higher (10–15) categories, based on the previous studies [19, 26]. Higher scores indicate higher subjective health literacy.

Health literacy is often strongly associated with health condition-specific knowledge [27]. For example, we previously reported positive associations between health literacy assessed using the REALM and kidney-specific knowledge among patients with chronic kidney disease [11, 28]. In this study, we administered the validated chronic hemodialysis knowledge survey (CHeKS) [29] to evaluate objective kidney knowledge, and an adaptation of the perceived kidney disease knowledge survey (PiKS) [28] oriented to issues specific to hemodialysis care (see Supplementary Material) to assess perceived kidney knowledge.

Statistical analysis

Descriptive statistics were performed for the overall sample and results presented as means and standard deviations (SDs), or percentages. The median and interquartile ranges (IQRs) were also presented for the health literacy assessments. Chi-squared test or Kruskal–Wallis test examined the relationship between each health literacy measure and participant characteristics. Cronbach's α estimated the internal consistency reliability of the BHLS. Associations between the BHLS and the REALM, the S-TOFHLA, cognitive status and the knowledge scores were performed by calculating the Spearman's ρ with bootstrapped 95% confidence intervals (95% CIs). Next, separate linear regression models examined the association between the BHLS and the other health literacy measures (REALM and S-TOFHLA) adjusting for age, gender, race, income and education. Finally, using the categories of adequate health literacy for the REALM and the S-TOFHLA, receiver-operating characteristic (ROC) analyses were used to generate area under the receiver operator characteristic (AUROC) curves for the BHLS. These curves summarize the predictive strength of the brief assessment, and values ≥0.7 are considered acceptable [30]. Statistical tests were performed using SPSS Statistics version 21 and STATA version 11.1. Findings with a P ≤ 0.05 were considered statistically significant.

Results

Among 233 eligible chronic hemodialysis patients, 150 (64%) enrolled and completed study activities (Figure 1). Participants were a mean (SD) of 52.2 (13.9) years old, 49% female, 73% African American, 52% with an annual income of <USD$20 000, 24% with less than a high school education and a mean (SD) time receiving dialysis of 4.6 (5.5) years (Table 1).

Fig. 1.

Study enrollment flow diagram.

Table 1.

Patient characteristics and association with health literacy, by health literacy assessment

Characteristic	All Subjects (n = 150)	BHLS (n = 145)		REALM (n = 146)		S-TOFHLA (n = 143)
		≤9 points (n = 33)	>9 points (n = 112)	Limited (0–60) (n = 39)	Adequate (61–66) (n = 107)	Limited (0–22) (n = 12)	Adequate (23–36) (n = 131)
Female	49%	43%	50%	41%	52%	35%	63%
African American	73%	79%	72%	87%	68%*	67%	74%
Income <$20,000/year	52%	55%	50%	61%	48%	70%	49%
Private insurance	32%	26%	41%	24%	36%	17%	33%
Education
Less than high school	24%	42%	17%***	38%	18%***	50%	20%***
High school equivalent	34%	55%	30%	46%	30%	42%	34%
Greater than high school	42%	3%	53%	15%	52%	8%	46%
Age (years) mean (SD)	52.2 (13.9)	55.5 (13.8)	51.2 (14.0)	52.6 (14.6)	51.7 (13.4)	54.6 (16.3)	51.7 (13.6)
MMSE score mean (SD)	26.8 (3.2)	25.6 (3.7)	27.2 (3.0)*	24.9 (3.3)	27.6 (2.9)***	25.7 (3.9)	27.0 (3.2)
Dialysis vintage (years) mean (SD)	4.3 (5.0)	4.1 (4.4)	4.4 (5.1)	5.7 (5.9)	3.9 (4.5)	7.7 (11.2)	4.4 (4.8)

SD, standard deviation; MMSE, mini-mental state exam; BHLS, brief health literacy screen; REALM, rapid estimate of adult literacy in medicine; S-TOFHLA, short test of functional health literacy in adults.

*P ≤ 0.05.

***P < 0.001.

Health literacy assessments

The median (IQR) scores for the REALM and the S-TOFHLA were 64 (60–65) and 34 (32–35), respectively. The BHLS median (IQR) score was 12.0 (10.0–14.0) and the mean (SD) was 11.6 (3.0) (Figure 2). Each of the individual item scores was similar with a mean (SD) of 3.85 (1.26) for the confidence with forms, 3.77 (1.34) for help reading hospital materials and 4.03 (1.07) for problems learning about their health condition. Cronbach's α for the three items was 0.72. Having fewer years of education was significantly associated with having lower health literacy according to all three measures (Table 1). Having a lower income was more common among participants with lower health literacy compared with higher health literacy measured with the BHLS (55 vs. 50%). Although this pattern of income differences was similar for the REALM (61 vs. 48%) and the S-TOFHLA (70 vs. 49%), the differences were not statistically significant for any of the health literacy assessments. Less cognitive capacity was associated with lower health literacy regardless of the method of assessment [BHLS: 25.6 (3.7) vs. 27.2 (3.0), P = 0.01; REALM: 24.9 (3.3) vs. 27.6 (2.9), P < 0.01; S-TOFHLA: 25.7 (3.9) vs. 27.0 (3.2), P = 0.18]. Age, gender and race were not significantly associated with health literacy in this sample.

Fig. 2.

Distribution of brief health literacy screen scores.

Performance on the BHLS was compared with both the REALM and the S-TOFHLA. Spearman's ρ was 0.35 (95% CI: 0.20–0.49; P < 0.001) for the REALM and 0.49 (95% CI: 0.35–0.69; P < 0.001) for the S-TOFHLA. Similar results were found when comparing with the BHLS and the TOFHLA reading score. In this study, the relationship between the REALM and the STOFHLA was also moderate (ρ = 0.41; P < 0.001). In adjusted linear regression models, the BHLS remained significantly (P < 0.001) associated with both the REALM and the S-TOFHLA, adding 17.4% unique variance to the REALM and 8.8% unique variance to the S-TOFHLA above and beyond the covariates. The receiver operating curves comparing the BHLS with identifying limited health literacy are shown in Figure 3. The areas under the curves were calculated to be 0.71 (95% CI: 0.61–0.80) for the REALM, and 0.73 (95% CI: 0.59–0.88) for the S-TOFHLA.

Fig. 3.

Area under the ROC curve for the BHLS compared with the REALM (A) and S-TOFHLA (B). REALM, rapid estimate of adult literacy in medicine; S-TOFHLA, short test of functional health literacy in adults.

Kidney knowledge

The BHLS was associated with both objective kidney knowledge [CHeKS: 0.43 (95% CI: 0.28–0.55), P < 0.001] and perceived kidney knowledge [PiKS: 0.41 (95% CI: 0.27–0.54), P < 0.001]. Participants with lower BHLS scores had lower CHeKS scores than participants with higher BHLS scores [mean (SD): 61.7% (19.7%) vs. 74.5% (14.5%), P < 0.001). Similar results were observed between the BHLS low/high scores and the dialysis PiKS scores [1.44 (0.66) vs. 1.89 (0.71), P < 0.001].

Discussion

Administration of the BHLS among patients receiving hemodialysis demonstrates concurrent validity by its consistent correlations with two other traditionally used assessments of health literacy. The associations, as hypothesized, between the BHLS and patient characteristics, including education, cognitive status and kidney knowledge further support its construct validity. The BHLS is an internally consistent and valid assessment of health literacy among patients with ESRD receiving hemodialysis, and will contribute to advancing our understanding of the role of health literacy in kidney disease through its application in both research and clinical settings.

The BHLS was significantly associated with both the REALM and the S-TOFHLA. The magnitude of the relationships was moderate, comparable with the significant, the moderate relationship between the REALM and the S-TOFHLA. Furthermore, in two previously reported diverse samples of either hospitalized or outpatients, the correlation between the BHLS and the S-TOFHLA was 0.48 (0.40–0.55) and 0.42 (0.29–0.55), respectively [25]. Finally, among a sample of 300 patients receiving emergency room care, comparisons of the BHLS and the REALM [0.26 (95% CI: 0.13–0.38)] and S-TOFHLA [0.33 (95% CI: 0.20–0.45)] were similar to that observed in this study [31]. Although our findings are consistent with prior research, the moderate correlations are likely in part due to differences in the assessment itself, including that the BHLS is a subjective evaluation of health literacy while the other measures are objective assessments. However, all of the findings do provide construct validity for the BHLS to identify patients with lower health literacy.

The AUROC curves for the brief measures to identify limited health literacy in this study were also of similar strength as prior research [18, 25, 31]. One of the initial reports of the BHLS' validation reported the best AUROC for the overall score as 0.72 (0.69–0.76) (REALM) or 0.71 (0.68–0.74) (S-TOFHLA) [20]. Again, these are similar to our study results. A recent study examined the single-item literacy screen (SILS) [32], a one question assessment of health literacy among hemodialysis patients. When compared with the S-TOFHLA to detect inadequate health literacy, the AUROC was 0.67 (0.60–0.74) [33]. The three-item BHLS may be somewhat better than the SILS in identifying lower literacy among hemodialysis patients because with more items there is potential for greater variability in the overall score, although additional studies are needed to add to the evidence of this comparison. Overall, the performance of the BHLS is similar to prior research with other patient populations, supporting its use as a measure of health literacy in the context of hemodialysis.

Understanding the determinants and outcomes related to patient health literacy is necessary to develop, test and ultimately disseminate interventions that aim to reduce disparities in health literacy-related outcomes. Cognitive abilities, fundamental to the processing of health information, are recognized as important factors closely related to health literacy and its association with outcomes [34]. Here, we found a significant association between cognitive ability according to the MMSE and health literacy assessed with the BHLS. Furthermore, a patient's knowledge related to their chronic condition is often a determinant of self-care [35], including in models of frameworks that illustrate the connection among knowledge, health literacy and outcomes in kidney disease [14, 36]. In pre-ESRD chronic kidney disease, both objective and perceived kidney knowledge has been previously reported to be associated with health literacy [11, 28]. In this study of hemodialysis patients, we again found that the BHLS score was associated with objective and perceived kidney knowledge. In addition to this evidence, further supporting construct validity of the BHLS in the context of dialysis, it also suggests the potential application of screening for health literacy to identify patients at risk for difficulty learning about their health condition as a component of clinical care.

A critical barrier to the assessment of health literacy in both research and clinical practice is the resources required and survey burden related to traditional measures. While the S-TOFHLA may be self-administered, it is usually administered by trained personnel in part because of its requirement to be timed. The REALM also requires administration by trained personnel. Previous studies of health literacy in dialysis used these assessments. We recently reported that the three-item BHLS is valid when administered as either a verbal interview or as a written self-administration [37]. Although trained research personnel supervised administration in this study of hemodialysis patients, we have also recently reported that assessment by a clinical nurse in routine care is comparable with that of formally trained personnel [25]. The Institute of Medicine recently includes in its attributes of a health literate organization a mandate to seek input from and determine the effect of programs among people with lower literacy, which requires in some way an assessment of individual health literacy [22]. Our research supports the resource efficiency of using the BHLS to characterize health literacy, and may be considered in both the clinical care and research related to the delivery of dialysis.

There are several study limitations to consider. First, participants were recruited from only four dialysis facilities and the findings may not be broadly generalizable. The sample size, and smaller proportion with lower health literacy, may have limited our ability to detect a relationship between health literacy and other participant characteristics. The assessments were administered only in English, although the BHLS has been demonstrated to be valid in Spanish among a sample of adults with diabetes receiving primary care [18]. The reported use of the BHLS is growing, although the score used to distinguish lower and adequate health literacy remains variable depending upon the study [18, 21, 38]. We applied a cutoff of 9 based upon experience in larger patient groups in our local hospital [38]; however, the distribution of BHLS scores in this study of hemodialysis patients was different with a median (IQR) of 12 (10–14) and mean (SD) 11.6 (3.0) compared with that of the general population with higher medians and means ranging from 13–15 and 12.1–13.9, respectively [25]. Similar to other studies [14, 15, 29], moderate ceiling effects were observed with all of the health literacy assessments which limit the ability to distinguish among higher degrees of health literacy. This was a cross-sectional study with health literacy evaluated only at a single point in time. The reproducibility of the BHLS or other health literacy measures was not assessed in this study limiting the ability to evaluate test–retest reliability. Longitudinal hemodialysis patient cohorts are needed to determine if health literacy varies over time, and if categories of health literacy assessed by the BHLS are meaningfully associated with clinical patient outcomes. If it can be shown that the BHLS scores do change over time this would support its use to identify patients at high risk and candidates for health literacy informed interventions.

In conclusion, the BHLS is an internally consistent and valid measure of health literacy among hemodialysis patients. Its simple, brief format, and flexible mode of administration requires less resources enhancing the feasibility for its use in both research and clinical practice. With continued critical evaluation of the BHLS' performance and utility, this tool has the potential to efficiently advance our understanding of the implications of health literacy in dialysis care.

Supplementary data

Supplementary data is available online at http://ndt.oxfordjournals.org.

Funding

C.Y.O. is supported by a Career Development Award from the National Institute of Diabetes and Digestive and Kidney Diseases (K01 DK087894). K.L.C. is supported by a Career Development Award from the National Institute of Diabetes and Digestive and Kidney Diseases (K23 DK80952). T.A.I. is supported by an award from the National Institute of Diabetes and Digestive and Kidney Diseases (K24 DK62849). The work was also supported in part by an American Society of Nephrology Carl W. Gottschalk Research Scholar Grant (K.L.C. and K.A.W.).

Conflict of interest statement

None declared.