Health literacy and digital media use: Assessing the Health Literacy Skills Instrument – Short Form and its correlates among African American college students

Abstract

Improving health literacy is increasingly seen as a solution to health problems and inequalities. This study assesses how one of the more recent measures of health literacy, the Health Literacy Skills Instrument – Short Form, performs among African American college students, and ascertains if and how media use relates to health literacy. Results indicate that both the use of health-related websites and apps as well as overall time spent with the media were positively, but conditionally, linked to health literacy. However, findings also pointed to the need for further test development.

Introduction

Discussions surrounding health increasingly center on individuals and their ability to make healthy decisions and handle the health-related demands from their environment, i.e., health literacy.¹ Studies have shown that adequate levels of health literacy can lead to improved health outcomes.^2,3

One of the initial steps in understanding and improving health literacy is to measure it. How to measure health literacy has been the subject of much debate in recent years. The best-known and most widely used measures of health literacy today are the Rapid Estimate of Adult Literacy in Medicine (REALM)⁴ and the Test of Functional Health Literacy in Adults (TOFHLA).⁵ The REALM focuses on people’s ability to read medical terms. The TOFHLA was developed in response to critiques of REALM’s sole focus on print literacy, and thus also assesses numeracy skills. Although both tests are used extensively in measuring health literacy, and deemed strong from a psychometric perspective,⁶ they are not without criticism.

A main critique of extant measures is their inability to measure skills relevant to health. Critics have recommended more comprehensive measures that include health-related questions.^2,6 One recommended measure is the Health Literacy Skills Instrument (HLSI).^7,8 The HLSI is particularly interesting as it assesses print, document, and quantitative literacy, as well as internet-based information seeking skills, and includes a short-form (HLSI-SF).^7,9 The measure assesses people’s health literacy skills by testing their ability to read charts and graphs, nutrition labels, and understand and interpret written health-related information presented through multiple platforms. What sets this measure apart from others is its focus on information disseminated through various media. Considering the fact that more people increasingly rely on the media—especially the internet—for their health information,^10,11 and that there are some concerns about people’s ability to find and evaluate it,¹² the HLSI looks especially promising. The HLSI is one of the first measures to view one’s ability to use new media not just as an administration tool, but as part of the set of skills that health-literate people should possess. Although the HLSI adhered to a five-factor structure,⁷ the HLSI-SF was reported as fitting a one-factor solution in confirmatory factor analysis.⁹

First, this study aims to assess how the HLSI-SF performs among African American college students. In health literacy research, African American college students, although more likely than their white counterparts to be affected by health disparities, are generally an under-investigated group.^13–15 Moreover, college students mostly do not possess the necessary quantitative skills to deal with medical information¹⁶ and thus possibly lack a desired level of health literacy. At the same time, college-aged individuals, more than any other group, have relatively greater access to resources necessary to increase health literacy.¹³

Second, this study aims to examine how variables that have been linked to health literacy and/or health outcomes relate to African American college students’ scores on the HLSI-SF. Some of these predictors, i.e., education and ethnicity, are less relevant given the homogeneity of our present study population. However, this does allow us to test for other variables. Health outcomes have been shown to correlate with health literacy, hence we consider variables that have been linked with positive health outcomes, including physical activity¹⁷ and health-related behaviors such as eating and drinking habits and getting enough sleep.^2,3 Physical activity is especially relevant, as it has also been linked to health literacy directly.¹⁷ Another variable this study will look at is where or how one obtains one’s health-related information. As shown by Paek and colleagues,¹⁸ interpersonal communication has as much of an influence on health literacy as receiving health-related information from mediated sources. Health outcomes are also affected by overall media use, both “old” media in the form of television and “new” media in the form of computers and handheld devices.^18–21 Therefore, in this study, we will also consider where people obtain their health information, whether this is from friends, family, doctors, and counselors, or mediated sources such as medical talk shows and health-related websites and apps. Considering that previous research has shown that negative health outcomes are tied to lower levels of health literacy, whereas positive health behaviors indicate higher levels of health literacy, it is possible that behaviors that lead to both poorer or better health (such as extensive media use) are also associated with health literacy. The HLSI-SF, which includes questions testing one’s ability to search for and comprehend mediated information, could serve as a useful indicator of health literacy among a demographic known for its engagement with new media.²² This study will thus also consider if the source of one’s health-related information (interpersonal or mediated) impacts health literacy as measured by the HLSI-SF.

Method

Participants

Students (N = 220) enrolled at a medium-sized historically black university in the southeastern United States of America participated for course credit. The majority of the respondents (94.5%) identified as Black/African American, with 1.8% identifying themselves as Multiracial, 0.9% as White/Caucasian, and 0.9% as Hispanic/Latino. The mean age was 23.60 (SD = 6.79); three-quarters (74.1%) were women; 2.3% were freshmen, 13.2% sophomores, 39.5% juniors, 43.2% seniors, and 1.8% were graduate students. Further, 75.5% had a relationship status of “single.” The only inclusion criteria were that respondents had to be 18 or older and enrolled in college.

Procedure

The study was carried out online. After providing consent, participants completed measures assessing their exercise, eating and drinking habits, unhealthy behaviors, overall health, information-seeking behavior, and media usage. The respondents then completed the HLSI-SF and demographic questions.

Measures

Health literacy

The HLSI-SF⁹ was used to measure health literacy. Each scale item is a multiple-choice question with just one correct answer and three or four incorrect options; the total number of correct responses is summed. Questions center on people’s ability to understand and interpret health-related information. Among other things, items ask people to make sense of nutrition and drug labels, descriptions of specific diseases, healthy eating tips, medicine records and hospital maps. Furthermore, the measure assesses people’s ability to use an online interactive calorie-tracking tool, understand an audio recording with medical information, and gain information from watching a video on exercise. The 10 items had lower reliability than expected, α = .415 (cf. α = .70 in Bann et al.).⁹ The mean score was M = 5.64, SD = 1.73. The mean average reported by Bann et al.⁹ was M = 6.7, SD = 2.3.

Correlates

Variables were selected based on extant research about factors related to health literacy. As health outcomes have been consistently related to health literacy levels, we also considered factors related to the respondents’ overall health. Several of the health-related items were taken from the 2013 College Student Health Survey,²³ and are marked accordingly, using double-dagger symbols (‡). The health-related variables that were assessed included: body mass index (BMI), weight perception‡, body image satisfaction‡, participation in sports activities, and weekly exercise frequency‡. We also investigated healthy eating‡, the consumption of sugary drinks‡, and the engagement in unhealthy behaviors‡ including eating fast food, binge eating, and using laxatives to control weight. Respondents indicated the number of days their physical and mental health was not good, how many days they felt stress‡, and how effectively they felt they handled stress‡. Finally, respondents rated for how many of the past seven days they slept enough to feel rested‡, how many hours they had slept in the past 24 hours‡, as well as reporting how often they typically ate breakfast‡, lunch, and dinner.

The communication-related variables included in the study were whom respondents would consult regarding a non-life-threatening medical concern (“talk to my family physician,” “visit the (Name University) health clinic,” “look for information online,” “ask my friends for advice,” “ask a family member for advice,” “talk to someone at (Name University)’s counseling center,” “go to ER/Urgent Care Clinic,” and “do nothing” (reverse-coded)) as well as how frequently they consulted each source; which medical talk shows they watched as well as how often they watched them (Dr. Oz, The Doctors, Dr. Sanjay Gupta, Dr. Phil, Dr. Nandi, Dr. Drew); which health-related websites or apps they used (WebMD, Healthline, Mayo Clinic Medical, Medline/PubMed4Hh; tracking eating: MyFitnessPal, Weight Watchers; and tracking exercise: Runkeeper, and use of health-monitoring technology (pedometer, blood pressure reader, heart rate monitor, GPS watch)) and how much overall time they spent using all types of media each day‡.

Data preparation

Missing data due to attrition ranged from 2.3% for questions in the middle of the instrument to 12.7% for questions on the final page (demographics). To address this issue, the “hot deck” technique²⁴ was used for multiple imputation of scores from similar “donor” cases with equivalent scores on non-missing variables.

Results

A one-factor structure for the HLSI-SF yielded satisfactory results in confirmatory factor analysis conducted in linear structural relations (LISREL) software, version 9.10: χ²/df = 1.51, p = .026; root mean square error of approximation (RMSEA) = .048, 90% confidence interval (CI) [.017, .074]; comparative fit index (CFI) = .861. The five-factor solution of the full HLSI fit less well: χ²/df = 1.82, p = .007; RMSEA = .061, 90% CI [.031, .089]; CFI = .842, and was rejected. Although the CFI for either solution fell short (>.90), these analyses support the idea that the short form of the HLSI works best as a single-dimensional construct.⁹

Those factor analyses contrast with the HLSI-SF’s low reliability, α = .415. Corrected item-total correlations were examined to assess if the removal of items might improve performance. Two items (7 and 9) performed especially poorly, producing correlations > .10. One of these items (9) was also problematic in original scale development,^7,9 but retained for content validity. Removal of these items only increased reliability to α = .520, short of conventional levels. However, scales with fewer, especially dichotomous, responses options, such as the HLSI-SF, tend to have artificially low reliability.²⁵ Therefore, reliability levels are still at useful levels, capable of providing valid inference.²⁶ Biases in Cronbach’s alpha can hinder its utility compared to factor analysis.²⁷ Indeed, our factor analysis showed adequate model fit. The analyses reported below were conducted with both the full 10-item measure and the revised 8-item measure; each yielded substantively similar results, hence we proceed by using the full 10-item version.

Next, the raw correlations between study variables were examined. Few associations with health literacy were found. Negative relationships with HLSI-SF were seen for sugary drinks and sleep over the previous 7 days. A number of significant relationships emerged among other key study variables (Table 1).

Table 1.

Raw correlations between study variables.

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
1. HLSI-SF
2. Info Seeking	−.05
3. Health TV	−.02	.06
4. Health Web	.06	.22b	.34d
5. Health  Technology	−.07	.13a	.32d	.20c
6. Media Time	.07	.15b	−.08	.03	−.07
7. Sports	−.02	−.17b	.19c	.01	.24d	−.09
8. Exercise	.06	−.10	−.03	−.07	.14b	−.01	.35d
9. Healthy Food	−.09	.01	.26d	.16b	.21c	.14b	.17c	.11
1. Sugary Drinks	−.15b	−.03	.11	.03	.004	.15b	.11	.07	.38d
11. Unhealthy  Behavior	−.11	−.15b	−.11	.06	−.06	.10a	.05	−.01	.03	.28d
12. Breakfast	−.08	−.01	.24d	.11	.13a	−.07	.01	.05	.26d	.03	.02
13. Dinner	−.05	.21c	−.06	.05	−.003	.06	−.06	−.02	.08	.04	.16b	.11a
14. Sleep Week	−.13a	.06	.01	.02	−.08	.12a	−.09	−.06	.04	−.06	.07	.05	.11
15. Sleep Day	.02	.14b	−.11	−.17b	.10	.06	−.01	.07	−.05	−.12a	−.08	.04	.11	.21c
16. Weight  Perception	.13a	.02	.08	−.06	.16b	−.08	−.04	−.05	.03	−.08	−.04	−.02	−.003	−.05	−.07
17. BMI	.06	−.04	.03	.02	.12	−.15b	−.06	−.06	.08	−.03	−.06	.01	−.05	−.01	−.12a	.67d
18. Age	−.09	.04	.27d	.07	.16b	−.08	−.19c	−.03	.10	−.08	−.13b	.25d	−.05	.10	.06	.12a	.19c
19. Sex	.01	.09	.14a	.09	−.04	−.01	−.001	−.13a	−.03	−.003	−.01	−.01	.0002	.03	−.04	.17b	.003	.09

BMI: body mass index; HLSI-SF: Health Literacy Skills Instrument - Short Form.

^ap < .1.

^bp < .05.

^cp < .01.

^dp < .001.

To test relationships between health literacy and correlates, controlling for study variables, we used multiple linear regression. Variables were grouped into four blocks for hierarchical entry into regression models (Table 2). The first block was demographics and the second included health characteristics: BMI, healthy and unhealthy behaviors, self-perceptions of health and stress, sleep, body image perceptions, and meal frequency. The third block of variables addressed communication: Information seeking, use of health-related television and websites, health-monitoring technologies, and time with media. The fourth block consisted of interaction terms of interest, described below.

Table 2.

Correlates of health literacy (HLSI-SF).

Variables	Model 1	Model 2	Model 3	Model 4
Block 1: ΔR2 = .027
Age	−.085	−.116	−.088	−.096
Sex	.013	.018	.004	.003
Ethnicity	−.044	−.043	−.089	−.073
Class Rank	−.108	−.117	−.156b	−.158b
Relationship Status	.062	.083	.039	.058
Sexuality	.018	.070	−.011	.023
Block 2: ΔR2 = .091
BMI		.020	.035	.064
Sports		−.085	−.053	−.067
Exercise		.063	.075	.105
Healthy Eating		−.009	−.043	−.090
Sugary Drinks		−.130	−.126	−.856c
Unhealthy Behavior		−.017	−.048	.533a
Unhealthy Days		−.009	−.039	−.073
Mentally Unhealthy  Days		.005	.039	.027
Stress Days		−.023	−.021	−.009
Handle Stress		.096	.118	.104
Sleep 7d		−.155b	−.118b	−.164b
Sleep 24 h		.072	.121	.113
Weight Perception		.126	.154	.100
Body Image  Satisfaction		.069	.074	.047
Breakfast		−.062	−.065	−.056
Lunch		.001	−.024	−.031
Dinner		−.016	.016	.014
Block 3: ΔR2 = .054b
Information  Seeking			−.114	−.090
Health TV			.048	.056
Health Web			.173b	.164b
Health Tech			−.133a	−.126
Media Time			.151b	.215
Block 4: ΔR2 = .044c
Media Time ×  Sugary Drinks				1.013c
Media Time ×  Unhealthy  Behavior				−.899b

Note. Standardized coefficients. BMI: body mass index; HLSI-SF: Health Literacy Skills Instrument - Short Form.

^ap < .1.

^bp < .05.

^cp < .001.

There were minimal demographic differences in health literacy. After controlling for all variables, only a negative relationship with class rank emerged (see Table 2, Model 3). Similarly, there was little association of health literacy with various health variables. Getting sufficient sleep over the past seven days was associated with lower health literacy.

Several communication variables yielded relationships with health literacy. The use of health-related websites was a significant positive correlate—and overall time spent with media was a marginally positive correlate—of health literacy while controlling for study variables. Additionally, use of health-related monitoring technologies had a marginally negative association with health literacy.

To examine whether health literacy’s links with health web and media time might be conditioned on health characteristics, interaction terms were tested. Specifically, each health variable from Block 2 was examined as a possible moderator of health web and media time. Statistically significant interactions emerged, and were isolated in a final version of Block 4, reported in Table 2, Model 4. Media time’s relationship with health literacy appeared to be moderated by sugary drink consumption and unhealthy behavior. PROCESS²⁸ was used for Johnson-Neyman interaction probing. This revealed that time spent with media had a positive association with health literacy when sugary drink consumption was high (>2.26, 40.91% of the sample) or unhealthy behavior was low (<2.65, 67.27% of the sample).

Discussion

This study assessed the effectiveness of the HLSI-SF among an African American college student sample. Besides measuring participants’ health literacy using the HLSI-SF, we also measured variables related to health behaviors and media usage. Results showed an average health literacy score of 5.64, SD = 1.73, lower than reported in the original study. Although the HLSI-SF showed weak reliability (α = .415; α = .520 if the two problematic items were removed), factor analysis showed satisfactory fit, meeting RMSEA criteria and nearly meeting CFI criteria. In short, even though the instrument provides helpful insight into the levels of health literacy among African American college students, the usefulness of this instrument among this particular population needs to be explored further, a notion underlined by the findings from the initial testing of the measure.^7,9

Furthermore, usage of health-related websites and apps was positively related to health literacy, overall time spent with media was marginally related, and the use of health-monitoring technology was negatively linked to health literacy. Interestingly, the association of these communication variables with health literacy was moderated by health variables (drinking sugary drinks and avoiding unhealthy eating behaviors). On the other hand, the HLSI-SF was—surprisingly—not strongly associated with health characteristics and behaviors generally indicative of health literacy. This may be attributable to weak reliability, or perhaps weak validity. Although the HLSI-SF exhibits more content validity than earlier measures of health literacy, it may still inadequately capture health literacy, especially among populations facing health disparities. The findings are interesting considering that previous research on the relationship between overall media use and health was divided about the nature of media influence.^20,21 Our results suggest overall media use and the use of health-related websites and apps may be beneficial for health-related knowledge, whereas people using health-related technology may merely view the technology as a fun gimmick or tool.

The presence of both healthy and unhealthy behaviors as moderators of media usage and health literacy, especially in a cross-sectional survey, raises questions about the causal chain of these variables. Looking at the relationships among overall media usage, consumption of sugary drinks, and unhealthy behaviors and health literacy, for instance, raises the question of what came first. A possible explanation is that the media increase the knowledge one has about health behaviors (and thus one’s health literacy), but that this increased knowledge does not necessarily translate into a healthier lifestyle. Likewise, causal relationships between health-related websites and apps, time spent sleeping, and health literacy require further hypothesizing and testing. Longitudinal data would be especially valuable in teasing apart these influences, as the interaction between media and health behaviors may reflect a reinforcing spiral.²⁹ Self-reinforcement of health behaviors through media use may impact levels of health literacy, or perhaps health literacy and media use interact to impact health behaviors.

The findings indicate the HLSI-SF did not perform as well as expected with regard to reliability and predictive validity in this particular sample and study. The instrument has only been tested in one setting,^7,9 and although that setting did include participants of various ethnic minorities, the findings that Non-Hispanic White respondents who were married and employed scored highest on the test, and that poorly performing items were particularly unsuccessful at capturing health literacy among non-White participants, suggest the instrument (long- and short-form) requires redevelopment and further testing with a variety of populations.

Our findings also point to a larger problem regarding the conceptualization and measurement of health literacy. The REALM³⁰ and the TOFHLA³¹ appear to work well among ethnic minorities, but this could be explained by the key critique of these instruments: They mainly measure the ability to read and perform some basic math, which may not reflect one’s ability to do well in a health-related environment.² The HLSI-SF, developed in response to this critique, operationalized health literacy in a more pragmatic manner, yet as shown by this study, may not work as expected either.

The study has several limitations. First of all, respondents took a single-session survey in an online, non-supervised setting, and self-reported their behaviors. It is possible that this attenuated findings and contributed to the high number of missing values. A second, broader limitation is that focusing on individual-level health literacy can overlook structural factors outside individual control, especially among relatively deprived social groups.³²

Future studies should continue to investigate relationships between health and media-related factors and health literacy, especially with regard to causal relationships. Measures of information-seeking behaviors should be expanded to include the ability to assess the quality of the information one finds, as well as the ability to use this information to alter health behaviors. Given the threats to reliability posed by dichotomous items,²⁵ we recommend health literacy scales avoid binary response options, as they may hinder accurate respondent reporting. Moreover, further investigations into health literacy should consider focusing specifically on racial and ethnic minorities. This focus will facilitate the design of an instrument that captures health literacy among a variety of populations and elaborate on factors that impact health literacy across ethnic and racial groups.

In conclusion, this study shows that although the HLSI-SF needs work if it is to adequately measure health literacy among this population, it can be used to draw tentative conclusions about correlates of health literacy among African American college students. Whereas previous studies have pointed at health behaviors, education, and race as correlates of health literacy, the present study was able to add overall media usage and use of health-related websites and apps to that list, firmly adding media use to the map for further health literacy research.

Contributorship

Rosenbaum and Deane conceived the study, all authors designed the methodology, Rosenbaum and Deane collected the data, Rosenbaum and Johnson wrote the literature review, Johnson carried out the statistical analysis, and Rosenbaum and Johnson wrote up the Results section and the Discussion. All authors reviewed and edited the final manuscript.

Conflict of interest

None declared.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported in part by Albany State University’s Title III program.

Ethical approval

The study received approval from the Institutional Review Board at Albany State University (#517552-2/3).

Guarantor

Judith E. Rosenbaum.

Peer review

This manuscript was reviewed by 2 people who have chosen to remain anonymous.