Abstract
Objective:
We examined the impact of language preference and health literacy on health information-seeking experiences in a multilingual, low-income cohort.
Methods:
We administered a modified Health Information National Trends Survey in English, Spanish, and Chinese to a sample of San Francisco city/county residents. Using multivariable logistic regression analyses, we assessed how language and health literacy impact health information-seeking experiences (confidence, effort, frustration, quality concerns, and difficulty understanding information), adjusting for age, gender, race/ethnicity, education, usual place of care, health status, information-seeking behaviors, and smartphone ownership.
Results:
Of 1,000 participants (487 English-speaking, 256 Spanish-speaking, 257 Chinese-speaking), 820 (82%) reported at least one negative health information-seeking experience. Chinese-language was associated with frustration (aOR=2.56; 1.12–5.86). Difficulty understanding information was more likely in Spanish-language respondents (aOR=3.58; 1.25–10.24). Participants with limited health literacy reported more effort (aOR=1.97; 1.22–3.17), frustration (aOR=2.09; 1.28–3.43), concern about quality (aOR=2.72; 1.60–4.61), and difficulty understanding information (aOR=2.53; 1.58–4.05). Language and literacy impacted confidence only in the interaction term between Chinese-speakers and health literacy.
Conclusion:
We found that negative health information-seeking experiences were common in non-English speaking populations those with limited health literacy.
Practice Implications:
Health communication efforts should consider both language preference and health literacy to ensure accessibility for all patients.
Keywords: health literacy, language preference, health information-seeking
1. Introduction
Challenges in information delivery and gaps in knowledge contribute to significant health disparities among communities of color.[1–4] Seeking health information can help individuals understand and manage their health problems, and engage in preventative care.[5,6] Health information-seeking is a key mediator through which individual characteristics (such as sociodemographics, language, or health literacy) impact health behaviors and decisions.[7,8]
Racial/ethnic minority and immigrant populations experience gaps in information.[9,10] Prior studies demonstrate that immigrants, lower socioeconomic status (SES) populations, or those who are uninsured, are are less likely to seek health information.[8,10–16] Moreover, individuals with less education, lower income, or who come from racial/ethnic minority or immigrant populations encounter negative information-seeking experiences. Negative experiences when seeking information (such as difficulty understanding information or concerns about quality) can adversely impact information-seeking as well as self-efficacy, health behaviors, and outcomes.[17–19]
Experts suggest that language barriers, health literacy issues, and cultural differences are potential reasons for disparities in health information-seeking behaviors and experiences.[2,20] Prior research among Spanish-speakers demonstrates challenges with information seeking[2,21–23] and suggests that limited availability of health information in non-English languages contributes to these difficulties.[24,25]
Health literacy, or an individual’s ability to find, understand, and use information to make health-related decisions[26], is know to be critical for health information seeking.[11] Information-seeking requires that individuals have the skills to find and determine the trustworthiness of the sources of information. Disparities in health literacy skills may also impact how much benefit individuals receive from seeking health information: those with high health literacy gained more positive outcomes from their information search compared to those with limited health literacy.[27]
While some studies have shown that English literacy skills and non-English language preference are independently associated with health information seeking,[28,29] most prior studies have explored the role of either health literacy or language preference individually on health information seeking.
Previous studies on information-seeking experiences have focused mostly on cancer patients.[30–32] Among cancer patients, negative information-seeking experiences are common, lead to fewer future attempts to obtain health information online,[3] and are associated with worse health-related quality of life and lower perceptions of health competence.[33] The Health Information National Trends Survey (HINTS),[34] a population-based survey administered by the National Cancer Institute (NCI), has contributed significantly to our understanding of how people access and use cancer information. However, the national HINTS survey is available only in English and Spanish, does not include health literacy questions, and focuses on cancer communication. It is unclear from the existing literature whether our current understanding of factors that impact health information-seeking experiences translates across all health conditions or only for cancer, particularly for low-income, minority populations.
Elucidating the association between language preference, literacy skills, and health information seeking can improve communication efforts, particularly among historically marginalized populations who are at high risk for poor health outcomes. In order to address these gaps in knowledge, we developed a local survey named SFHINTS (San Francisco Health Information National Trends Survey), which included general health communication questions. This survey was designed to inform future delivery of health information to reduce health disparities. SFHINTS was available in English, Spanish, and Chinese and we focused survey administration in low-income, non-White populations.[35] The multilingual and low-income population in the SFHINTS cohort, along with the addition of validated questions about health literacy, creates a unique opportunity to explore the impact of health literacy and language preference (English, Spanish, or Chinese), independently and together, on general health information-seeking experiences in low-income, non-White populations. We hypothesized that language preference and health literacy both independently contributed to health information-seeking experiences.
2. Methods
2.1. Survey Development and Administration
We used questions from HINTS[36] and added validated questions about health care access[37] and health literacy[38,39] to create a local survey (SFHINTS Survey, see Appendix A). Development and translation of the full survey as well as recruitment and survey administration are described elsewhere in greater detail.[35] Briefly, the survey asked about demographic characteristics, self-reported health status, health information-seeking sources and experiences, and healthcare access and behaviors. For the Spanish version, a staff member used existing questions from the Spanish version of HINTS 4; additional questions were translated from the original source. For the Chinese version of the survey, a native Cantonese and Mandarin-speaking research staff member translated questions from the English version of the SFHINTS survey. Questions were translated and back-translated. Both Spanish and Chinese versions of the survey were reviewed by local researchers and community-based organizations for language translation and cultural sensitivity.
The survey was administered in the City and County of San Francisco, where Chinese and Spanish are the most common non-English languages spoken.[40] Participants included adults who were 18 to 75 years old, lived in San Francisco, and able to complete the survey in English, Spanish, or Chinese. Using community-based sampling, local community-based organizations recommended locations and neighborhoods for participant recruitment. We had prespecified targets for language (50% English, 25% Spanish, 25% Chinese) and race/ethnicity (25% African-American). The survey was administered verbally from May to September 2017 in participants’ preferred language by professionally trained, bilingual research staff. The UCSF Institutional Review Board approved this study (16–20707).
2.2. Outcomes of Interest
We explored participants’ health information-seeking experiences in five domains using the Information Seeking Experiences Scale (ISEE), which is used in HINTS[17]: 1) confidence in getting health information; 2) feeling that a lot of effort was required to find information; 3) frustration levels during information search; 4) concern about quality of the information; and 5) difficulty in understanding information provided (SFHINTS Survey, Appendix A, Section B). For the confidence domain, participants were asked, “Overall, how confident are you that you could get health-related advice or information if you needed it?” Low confidence was defined as participants answering “somewhat confident”, “a little confident”, and “not confident at all”. For the remaining four domains, participants were asked, “Based on the results of your most recent search for information about health or medical topics, how much do you agree or disagree with the following statements? a) It took a lot of effort to get the information you needed; b) You felt frustrated during your search for information; c) You were concerned about the quality of information; d) The information you found was hard to understand. These domains were scored on a four-point scale ranging from “strongly agree” to “strongly disagree.” To facilitate interpretation, we dichotomized these domains into agree vs. disagree. We created a separate outcome variable (as a binary variable) to indicate if a participant reported a negative experience in any of the five domains, which we defined as “yes” if participants reported low confidence (as defined above); or if they strongly/somewhat agree with the aforementioned statements regarding effort, frustration, concern about quality, or difficulty understanding information.
2.3. Key Predictor Variables and Co-variates
To identify a list of potential predictors, we adopted the Longo framework (Appendix B)[41], which explores personal and contextual factors that influence patients’ information-seeking behavior and use. Our key predictor variables were language preference and health literacy. Co-variates included personal variables, which included age, gender, race/ethnicity, education, limited English proficiency, and self-reported health status; contextual variables included having at least one usual place of care, first asking a health care provider for health information, first going on the internet to find health information, and smartphone ownership.
Language preference was asked at study recruitment and the survey was conducted in patients’ preferred language. Chinese-language respondents included both Cantonese and Mandarin speakers, as the same written instrument was used for both dialects. Health literacy was asked in reference to materials in participants’ preferred language and was reported as limited if participants reported “sometimes”, “often”, or “always” to the question: “How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy?”[38] We elected to use this single-item self-report health literacy item because it has been validated against sentence-completion and vocabulary-based direct health literacy measures in English and Spanish[38,39] and has been used in multiple studies in lieu of burdensome health literacy assessments.[42–44]
The co-variates age (i.e., 18–34 as the reference category, 35–49, 50–64, and ≥65 years), race/ethnicity (i.e., White as the reference category, Black, Asian, Latinx, and Other); and educational attainment (less than high school, high school or equivalent, some college/vocational training, and college graduate or higher as the reference category) were categorical variables. English proficiency was reported as limited if participants reported that they spoke English “not at all”, “poor”, or “not well”.[45] Variables about where patients first sought health information (internet vs. healthcare provider) were based on participants’ response to the question: “the most recent time you looked for health information about health or medical topics, where did you go first?” (SFHINTS Survey, Appendix A, Section A).
2.4. Statistical Analyses
We report descriptive statistics for predictor and outcome variables. Among language groups, we assessed differences in key predictor variables and co-variates using chi-square tests. Among respondents without missing data, we assessed the relationships between key predictor variables and co-variates and each of the five health information-seeking experiences using bivariate logistic regressions. We excluded patients who completed the English survey but reported limited English proficiency in our analyses. In addition, we used multivariable logistic regression to assess the impact of language and health literacy on each of the five outcomes, adjusting for co-variates.
Survey weights were computed using iterative proportional fitting (raking), a technique used for nonprobability samples that involves raking over a set of variables (age and gender) iteratively to reweight the cohort population to match the distribution of the reference population (San Francisco). We determined no significant collinearity between variables. Given the potential interaction between language preference and limited English proficiency, we conducted sensitivity analyses that evaluated models with both limited English proficiency and language, with each variable alone, and with both variables and their interaction. In these analyses, only language preference was significant, and therefore, our final model only included language preference.
Finally, we evaluated models that included an interaction term between language preference and health literacy. In these analyses, the interaction term was only significant for confidence, which we presented. The interaction between language preference and health literacy was not significant for effort, frustration, concern about quality, or difficulty understanding information so our final models for these outcomes did not include the interaction term.
We analyzed the data using STATA 16 (College Station, TX)[46] and assessed statistical significance at the 0.05 level for all tests.
3. Results
3.1. Participant characteristics
Of the 1,000 completed surveys, half were administered in English, a quarter in Spanish, and a quarter in Chinese (Table 1). The mean age was 47.3 (SD 16.7) years old, slightly over half of the participants were female, and half of participants had an annual household income of less than $20,000. There were statistically significant differences between language groups for age, race/ethnicity, education, income, first seeking information from a healthcare provider, first seeking information from the internet, and smartphone ownership. Chinese-survey respondents and Spanish-survey respondents reported higher rates of limited health literacy compared to English-survey respondents.
Table 1.
Characteristics of SFHINTS participants (n=1000) by survey language
| Total | English | Spanish | Chinese | |
|---|---|---|---|---|
| N=1,000 n (%) |
N=487 n (%) |
N=256 n (%) |
N=257 n (%) |
|
| Age (mean (SD))† | 47.3 (16.7) | 44.5 (15.4) | 47.0 (15.3) | 52.8 (18.9) |
| Female | 515 (52%) | 265 (54%) | 122 (48%) | 128 (50%) |
| Race/Ethnicity† | ||||
| Non-Hispanic White | 40 (4%) | 39 (8%) | 1 (0.4%) | 0 (0%) |
| Non-Hispanic Black | 238 (24%) | 237 (49%) | 1 (0.4%) | 0 (0%) |
| Latinx | 361 (36%) | 111 (23%) | 250 (98%) | 0 (0.0%) |
| Non-Hispanic Asian/ Pacific-Islander | 306 (31%) | 47 (10%) | 2 (0.8%) | 257 (100%) |
| Other | 55 (6%) | 53 (11%) | 2 (0.8%) | 0 (0%) |
| Highest grade or level of schooling completed† | ||||
| Less than high school education | 231 (23%) | 56 (12%) | 88 (34%) | 87 (34%) |
| High school graduate or equivalent | 296 (30%) | 151 (31%) | 75 (29%) | 70 (27%) |
| Some college or vocational training | 278 (28%) | 161 (33%) | 59 (23%) | 58 (23%) |
| College graduate or higher | 181 (18%) | 110 (23%) | 31 (12%) | 40 (16%) |
| Income† | ||||
| < $10,000 | 232 (23%) | 124 (25%) | 58 (23%) | 50 (19%) |
| 10,000 to under $20,000 | 261 (26%) | 104 (21%) | 62 (2924 | 95 (37%) |
| $20,000 to under $35,000 | 107 (11%) | 46 (9%) | 33 (13%) | 28 (11%) |
| $35,000 to under $50,000 | 109 (11%) | 57 (12%) | 28 (11%) | 24 (9%) |
| > $50,000 | 156 (16%) | 92 (19%) | 30 (12%) | 34 (13%) |
| Limited English proficiency† | 317 (32%) | 0 (0%) | 147 (57%) | 170 (66%) |
| Limited health literacy† | 403 (40%) | 145 (30%) | 114 (45%) | 144 (56%) |
| Fair or poor self-reported general health† | 341 (34%) | 134 (28%) | 89 (35%) | 118 (46%) |
| Has health insurance† | 818 (82%) | 384 (79%) | 210 (82%) | 224 (87%) |
| At least one usual place of care | 830 (83%) | 413 (85%) | 222 (87%) | 195 (76%) |
| First sought information from healthcare provider† | 358 (36%) | 173 (36%) | 114 (45%) | 71 (28%) |
| First sought information from the internet† | 395 (40%) | 233 (48%) | 97 (38%) | 65 (25%) |
| Smartphone ownership† | 771 (78%) | 382 (78%) | 211 (82%) | 182 (71%) |
Data were missing for <10 participants for sex; 14 participants for education; 135 for income; 12 participants for health literacy; 46 participants for presence of insurance; and 60 participants for presence of usual source of care.
Significant differences between groups (p<0.05)
3.2. Health information-seeking experiences
Most participants reported at least one negative health information-seeking experience: 389 (80%) English-language respondents, 204 (80%) Spanish-language respondents, and 227 (89%) Chinese-language respondents. While concern about quality of information was uniform across the three languages, in unadjusted analyses, there were significant differences in negative health information-seeking experiences based on survey language for confidence, effort, frustration, and difficulty understanding information (Figure 1A). In unadjusted analyses, participants with limited health literacy were more likely to report all five negative health information-seeking experiences compared to those who were health literate (Figure 1B) (for details, see Appendix C).
Figure 1. Frequency of negative health information-seeking experiences among SFHINTS participants by a) survey language and b) health literacy level.
†Significant differences between groups (p<0.05)
In multivariable analyses, age, gender, education, survey language, limited health literacy, self-reported general health, having a usual place of care, and first seeking information from the internet were associated with at least one negative health information-seeking experiences (Table 2). Race/ethnicity, first seeking information from a healthcare provider, and smartphone ownership were not significant predictors of any negative health information-seeking experiences.
Table 2.
Predictors of negative health information-seeking experiences among SFHINTS participants in weighted multivariable analyses, San Francisco, CA, 2017
| Predictor | Low confidence aOR [95% CI] |
Effort aOR [95% CI] |
Frustrated aOR [95% CI] |
Concern about quality aOR [95% CI] |
Hard to understand aOR [95% CI] |
|---|---|---|---|---|---|
| Age | |||||
| 18–35 years | Ref | Ref | Ref | Ref | Ref |
| 36–49 years | 1.16 [0.62, 2.18] | 1.35 [0.72, 2.53] | 1.28 [0.68, 2.40] | 0.74 [0.41, 1.33] | 1.63 [0.83, 3.19] |
| 50–64 years | 0.83 [0.43, 1.59] | 1.74 [0.87, 3.47] | 1.27 [0.65, 2.50] | 0.77 [0.41, 1.48] | 1.59 [0.78, 3.21] |
| 65+ years | 0.36 [0.16, 0.91] | 1.63 [0.69, 3.84] | 1.35 [0.61, 3.00] | 0.74 [0.31, 1.74] | 1.37 [0.62, 3.03] |
| Gender | |||||
| Male | 1.13 [0.71, 1.80] | 0.96 [0.60, 1.54] | 0.95 [0.59, 1.53] | 1.67 [1.05, 2.66] | 0.94 [0.56, 1.57] |
| Race/ethnicity | |||||
| White | Ref | Ref | Ref | Ref | Ref |
| Black | 0.85 [0.38, 1.94] | 1.22 [0.56, 2.64] | 0.85 [0.38, 1.88] | 1.42 [0.67, 3.01] | 1.28 [0.56, 2.95] |
| Latinx | 1.10 [0.42, 2.93] | 1.02 [0.40, 2.63] | 0.83 [0.34, 2.05] | 1.09 [0.43, 2.77] | 0.54 [0.17, 1.66] |
| Asian | 1.61 [0.61, 4.25] | 1.49 [0.57, 3.93] | 0.56 [0.19, 1.60] | 1.06 [0.40, 2.82] | 1.48 [0.54, 4.09] |
| Other | 1.59 [0.64, 3.90] | 0.83 [0.33, 2.09] | 0.68 [0.27, 1.75] | 0.95 [0.40, 2.28] | 0.65 [0.24] |
| Education | |||||
| Less than high school education | 0.97 [0.43, 2.19] | 0.50 [0.25, 1.00] | 0.34 [0.16, 0.73] | 0.20 [0.09, 0.46] | 0.50 [0.23, 1.09] |
| High school graduate /equivalent | 1.52 [0.72, 3.19] | 1.03 [0.53, 2.00] | 0.38 [0.20, 0.72] | 0.38 [0.19, 0.77] | 0.66 [0.33, 1.31] |
| Some college/ vocational training | 1.17 [0.61, 2.24] | 1.20 [0.62, 2.35] | 0.65 [0.33, 1.29] | 0.38 [0.20, 0.75] | 1.12 [0.54, 2.34] |
| College graduate or higher | Ref | Ref | Ref | Ref | Ref |
| Survey language | |||||
| English | Ref | Ref | Ref | Ref | Ref |
| Spanish | 0.71 [0.22, 2.30] | 0.90 [0.36, 2.22] | 1.11 [0.52, 2.33] | 0.91 [0.40, 2.1] | 3.58 [1.25, 10.24] |
| Chinese | 1.17 [0.50, 2.76] | 1.19 [0.56, 2.50] | 2.56 [1.12, 5.86] | 1.37 [0.60, 3.09] | 2.01 [0.92, 4.43] |
| Limited health literacy | 0.65 [0.31, 1.39] | 1.97 [1.22, 3.17] | 2.09 [1.28, 3.43] | 2.72 [1.60, 4.61] | 2.53 [1.58, 4.05] |
| Spanish x limited health literacy | 1.82 [0.58, 5.72] | -- | -- | -- | -- |
| Chinese x limited health literacy | 5.07 [1.89, 13.62] | -- | -- | -- | -- |
| Fair/poor general health | 2.67 [1.55, 4.58] | 1.42 [0.83, 2.41] | 1.52 [0.91, 2.54] | 1.21 [0.70, 2.09] | 1.35 [0.79, 2.31] |
| At least one usual place of care | 1.22 [0.64, 2.35] | 0.94 [0.47, 1.88] | 0.96 [0.48, 1.94] | 1.20 [0.62, 2.29] | 1.95 [1.03, 3.72] |
| First sought information from healthcare provider | 0.70 [0.43, 1.15] | 1.11 [0.70, 1.78] | 0.84 [0.52, 1.36] | 0.93 [0.58, 1.49] | 0.81 [0.50, 1.33] |
| First sought information from internet | 0.92 [0.56, 1.51] | 1.03 [0.62, 1.70] | 0.73 [0.44, 1.22] | 1.81 [1.08, 3.06] | 0.95 [0.58, 1.55] |
| Smartphone ownership | 1.00 [0.52, 1.93] | 1.62 [0.84, 3.13] | 1.36 [0.76, 2.45] | 1.19 [0.64, 2.23] | 1.15 [0.2, 2.14] |
*Bolded values are statistically significant.
Some co-variates were predictive of at least one negative health information-seeking experience, including older participants for low confidence (adjusted odds ratio [aOR]=0.36, 95% CI 0.16–0.91), men for concern about quality (aOR=1.67, 95% CI 1.05–2.66) (Table 2), and lower educational attainment for feeling frustrated during the search. Relative to college graduates, all other individuals reported lower concern about the quality of information. Those who reported poor or fair general health had higher odds of reporting low confidence (aOR=2.67, 95% CI 1.55–4.58), first seeking information from the internet was associated with increased odds about concern about quality (aOR=1.81, 95% CI 1.08–3.06), and having at least one usual place of care was associated with higher odds of reporting that the information was hard to understand (aOR=1.95, 95% CI 1.03–3.72).
Compared to English-survey respondents, Spanish-survey respondents had higher odds of reporting that the information found was hard to understand (aOR= 3.58, 95% CI 1.25–10.24). Chinese-survey respondents had higher odds of feeling frustrated during search (aOR=2.56, 95% CI 1.12–5.86). Participants with limited health literacy had higher odds of reporting all negative information-seeking experiences except for low confidence in searching for health information. For low confidence, while language and health literacy individually were not significant, the interaction term was significant: language and health literacy were only significant in Chinese-survey respondents who also had limited health literacy, who had increased odds of low confidence during the information search (aOR=5.07, 95% CI 1.89–13.62).
4. Discussion and Conclusion
4.1. Discussion
In this study of health information-seeking experiences in a low-income, multilingual cohort, 82% of respondents reported at least one negative health information-seeking experience. After adjustment, Chinese-speakers reported more frustration, and Spanish-speakers had more difficulty understanding information. Those with limited health literacy reported more negative health information-seeking experiences compared to those who were health literate.
Prevalence of negative general health information-seeking experiences among each of our five domains ranged from 40–54%, which is comparable to prior studies where half of patients reported negative experiences (including requiring a lot of effort, expressing frustration, and having concerns about quality of information found.[17,33] Similar to our findings, the NCI reported from HINTS data that Spanish-speaking Hispanics were more likely to report negative health information-seeking experiences than non-Hispanics.[22] The persistence of negative health information-seeking experiences over the last 10 years suggests that current methods of health information dissemination do not serve non-English-speaking populations and those with limited health literacy optimally, regardless of whether participants are searching for general or cancer-related health information.
We found that Spanish- and Chinese-language preference was associated with higher odds of negative health information-seeking experiences even after accounting for known predictors of health information-seeking experiences, including age, race/ethnicity, and education. These results are similar to prior studies that found that non-English speakers reported lower confidence and faced more challenges in obtaining cancer information.[21] This finding may reflect that healthcare organizations frequently do not provide health information in multiple languages, despite serving many patients with non-English language preferences.[47] The availability of information online in multiple language may not be sufficient for those seeking health information. Even though machine translation tools[48] have the potential to lower barriers to language-concordant health information, non-English speaking populations’ experiences with seeking health information continues to be suboptimal. Pragmatic use of available information technology tools may facilitate information access in multiple languages,[49,50] though this may not improve health information-seeking experiences as using the internet to find information was associated with increased concern about the quality of information found.
In sensitivity analyses, adding limited English proficiency to the model did not improve prediction for any outcomes; neither did analysis of the interaction between limited English proficiency and language preference. Our finding of an association between language preference, but not language proficiency, with health information-seeking experiences is contrary to prior studies that suggest that limited English proficiency is more closely associated with health outcomes.[51] Given the ease of attaining language preference data, health systems generally collect language preference information in an individual’s electronic health record. This study suggests that language preference may be a reasonable proxy for limited English proficiency in certain situations, and health systems can consider systematically collecting language preference information for their patients to inform their communication efforts.
However, while English proficiency and language preference are often used interchangeably, our findings suggest these constructs may impact health information-seeking behaviors and experiences in different ways.[52] English proficiency is related to an individual’s skill with language as a tool, thereby influencing the ability to navigate the health care system and effective communication between provider and patient.[53] On the other hand, while language preference alone may not explain all the differences in health information-seeking behaviors and experiences, language preference can suggest an individual’s ability to use English as well as reflect underlying cultural values, social networks, and acculturation, which can impact health behaviors in ways that are not addressed by intervening only on English proficiency.[52,54] It is possible that English proficiency and language preference have a similar impact on some health behaviors, but meaningful differences in other behaviors.[55] For example, patients may speak English well, but strong ties to their immigrant communities may impact their access to and experience with seeking health information in addition to their English proficiency.
We found consistent associations between limited health literacy and information-seeking experiences, even after adjusting for language, similar to prior studies.[56] Language effects were only significant in Chinese-survey respondents who also had limited health literacy, which conferred increased odds of low confidence. The measure of health literacy among Chinese-survey respondents was not validated, which may impact our results. These findings highlight that in order to better anticipate barriers to health communication, health-promoting organizations should assess patients not only for health literacy but do so in their preferred language[57] using validated tools.[39,58] Our findings on the additional vulnerabilities of patients who have both non-English language preference and limited health literacy in their most proficient language suggest that both must be addressed to improve care for non-English speaking and populations with limited health literacy.
Contrary to other studies that found that lower education was associated with negative health information-seeking experiences,[10,20] our study found that those with lower educational attainment reported fewer negative health information-seeking experiences. We did not specify in what country participants received their education, which may influence our findings. Additionally, it is possible that those with lower educational attainment are less able to assess the quality of information, which is consistent with findings that individuals’ evaluation of health information can be subjective and misinformed.[59,60] This is particularly concerning because the quality of much of the information online remains unreliable.[60]
Individuals who rate their health as poor search the internet for health information more frequently.[5] Patients who are sicker may be motivated to learn more about their medical condition. In our study, those who rated their health as poor or fair had lower confidence in their ability to seek information. These results suggest a need to provide additional support to patients in worse health to ensure they can obtain high quality, accurate medical information.
We found that those seeking information from the internet were more likely to be concerned about the quality of information found. Additionally, despite the high rates of smartphone ownership and studies showing that smartphones contribute to how people seek information[23,61], smartphone ownership in our study did not influence participants’ experience seeking information. This may suggest that individuals are uncertain how to access high quality online health information or that online health information replicates the same accessibility errors as other information (i.e., written at a high literacy level or lack of non-English materials). Furthermore, while healthcare providers are cited as the most trusted and one of the most frequently chosen sources of health information,[23,62] first seeking information from a healthcare provider was not associated with any health information-seeking experiences. However, having a usual place of care was associated with increased odds of finding information hard to understand. It may be that the health information provided by healthcare providers are not provided in a way that optimally serves non-English speaking populations and those with limited health literacy. Although health information may be more accessible than ever before, it has not meaningfully impacted individuals’ experiences seeking health information.
Our study has several limitations. First, the survey was administered in a single location. We prioritized this diverse, low-income population to inform local communication efforts. Additionally, we are unable to assess the type of health information that patients were referring to when answering questions and we are unable to assess trends over time. Furthermore, the health literacy measure we used has not been validated in Chinese, because no gold-standard health literacy assessment was available in Chinese at the time of survey design.[63] Moreover, participants could only select a language preference among three languages (English, Spanish, and Chinese); there may have been participants with non-English language preferences among the English-language respondents. Our study is also limited by potential unmeasured factors that influence health information-seeking experiences that were not captured in our survey or analysis. Finally, health information-seeking experiences are based on participants’ self-report, which may not accurately represent the communication difficulties in our population. However, our findings are consistent with prior literature on health information-seeking experiences, and we used a widely cited, validated survey to formulate these questions.
4.2. Conclusions
The results presented in this paper highlight the relationship between language preference and health literacy on health information-seeking experiences. Negative health information-seeking experiences are very common in non-English speaking populations and those with limited health literacy. While prior studies emphasize the importance of addressing health literacy or the need for information in non-English languages, this study is unique in that we assessed participants for both language preference and health literacy concurrently. Our results highlight the importance of healthcare systems and public health departments considering both language preference and health literacy throughout the process of developing and disseminating health information, rather than merely as afterthoughts.
4.3. Practice Implications
Our study findings have implications for clinicians, professional and disease-specific associations, health systems, and public health departments. Providers of healthcare information should adhere to recommendations from the American Medical Association and others that health information be provided at or below 6th grade reading level. It is unclear if this recommendation applies across all languages, and specific recommendations for non-English languages should be identified. The stable high rates of negative health information-seeking experiences in individuals with limited health literacy support prior studies that health information continues to be written at a reading level inaccessible to many.[64] Implementing the Centers for Medicare & Medicaid Services toolkit for developing clear and effective written material[65] or the Centers for Disease Control and Prevention’s clear communication index[66] can help organizations develop materials understandable to populations with limited health literacy. Beyond improving the understandability of health information, healthcare organizations should also take organizational efforts to support patients with limited health literacy as recommended by Healthy People 2030[67] and the National Academy of Medicine, including educating healthcare team members about how to support patients with limited health literacy or explicitly integrating health literacy considerations into quality improvement efforts.[68]
At the healthcare system level, systems should provide providers with readily available access to high-quality interpreter services and can consider implementing universal health literacy precautions, particularly as medical care becomes increasingly complex.[68] This will help all patients, not just those at greatest risk for communication gaps. Patients with both limited health literacy and non-English language preference likely encounter additional barriers to accessing care. Since poor communication negatively impacts patient care and outcomes, healthcare organizations should support non-English speaking patients and those with limited health literacy throughout their interactions with the healthcare systems, for example by assigning patient advocates or ensuring patient feedback instruments are available in other languages.
Public health departments should consider both language and health literacy when disseminating health information. Timely and equitable access to health information is critical to achieving desired public health outcomes. To improve information-seeking experiences in diverse and low-income populations, future health communication interventions should explore strategies that improve the availability and access to both language-concordant and health literate resources, including collecting disaggregated public health data to examine if these populations are being served effectively.
Supplementary Material
Acknowledgements
The authors would like to acknowledge Cynthia Cheung, Corina Liew, Alejo Perez-Stable, Francisco Quintanilla, Pamela Williams, Ching Wong, as well as the San Francisco Cancer (SF CAN) Initiative Steering Committee and the SFHINTS steering committee (Anna Napoles, Tung Nguyen, and Rena Pasick). They also want to acknowledge community partners including Angela Sun at the Chinese Community Health Resource Center, Monique LaSarre at the Rafiki Health Center, and Brenda Storey at Mission Neighborhood Health Center.
This work was supported by the NCI through an Administrative Supplement to the University of California, San Francisco Helen Diller Family Comprehensive Cancer Center Support Grant under grant number P30CA082103–18S7, the National Institutes of Health under grant number K24CA212294, the National Heart Lung and Blood Institute of the NIH under grant number K12HL138046, the National Center for Advancing Translational Sciences of the NIH under grant number KL2TR001870, the National Institutes of Health’s (NIH’s) National Research Service Award under grant number T32HP19025.
Footnotes
Conflict of interest statement
The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The authors do not have any disclosures, financial or otherwise, to report.
References
- [1].Angeles J, Somers SA, From Policy to Action: Addressing Racial and Ethnic Disparities at the Ground-Level, Cent. Health Care Strateg (2007). https://www.chcs.org/resource/from-policy-to-action-addressing-racial-and-ethnic-disparities-at-the-ground-level/ (accessed July 19, 2021).
- [2].Viswanath K, Ackerson LK, Race, Ethnicity, Language, Social Class, and Health Communication Inequalities: A Nationally-Representative Cross-Sectional Study, PLoS ONE 6 (2011) e14550. 10.1371/journal.pone.0014550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].McCloud RF, Okechukwu CA, Sorensen G, Viswanath K, Beyond access: barriers to internet health information seeking among the urban poor, J. Am. Med. Inform. Assoc 23 (2016) 1053–1059. 10.1093/jamia/ocv204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Viswanath K, Kreuter MW, Health Disparities, Communication Inequalities, and eHealth, Am. J. Prev. Med 32 (2007) S131–S133. 10.1016/j.amepre.2007.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5].Weaver JB, Mays D, Weaver SS, Hopkins GL, Eroğlu D, Bernhardt JM, Health Information–Seeking Behaviors, Health Indicators, and Health Risks, Am. J. Public Health 100 (2010) 1520–1525. 10.2105/AJPH.2009.180521. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Ayers SL, Kronenfeld JJ, Chronic illness and health-seeking information on the Internet, Health Interdiscip. J. Soc. Study Health Illn. Med 11 (2007) 327–347. 10.1177/1363459307077547. [DOI] [PubMed] [Google Scholar]
- [7].Glanz K, Rimer KB, Viswanath K, Health behavior and health education: Theory, research, and practice, Jossey-Bass, San Francisco, CA, 2008.
- [8].Anker AE, Reinhart AM, Feeley TH, Health information seeking: A review of measures and methods, Patient Educ. Couns 82 (2011) 346–354. 10.1016/j.pec.2010.12.008. [DOI] [PubMed] [Google Scholar]
- [9].Viswanath K, Public Communications and Its Role in Reducing and Eliminating Health Disparities, National Academies Press (US), 2006. https://www.ncbi.nlm.nih.gov/books/NBK57046/ (accessed July 19, 2021). [Google Scholar]
- [10].Ramanadhan S, Viswanath K, Health and the Information Nonseeker: A Profile, Health Commun 20 (2006) 131–139. 10.1207/s15327027hc2002_4. [DOI] [PubMed] [Google Scholar]
- [11].Jacobs W, Amuta AO, Jeon KC, Health information seeking in the digital age: An analysis of health information seeking behavior among US adults, Cogent Soc. Sci 3 (2017) 1302785. 10.1080/23311886.2017.1302785. [DOI] [Google Scholar]
- [12].Oh KM, Zhou Q, Kreps G, Kim W, The Influences of Immigration on Health Information Seeking Behaviors Among Korean Americans and Native Koreans, Health Educ. Behav 41 (2014) 173–185. 10.1177/1090198113496789. [DOI] [PubMed] [Google Scholar]
- [13].Kontos E, Blake KD, Chou W-YS, Prestin A, Predictors of eHealth Usage: Insights on The Digital Divide From the Health Information National Trends Survey 2012, J. Med. Internet Res 16 (2014) e172. 10.2196/jmir.3117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Adjei Boakye E, Mohammed KA, Geneus CJ, Tobo BB, Wirth LS, Yang L, Osazuwa-Peters N, Correlates of health information seeking between adults diagnosed with and without cancer, PLOS ONE 13 (2018) e0196446. 10.1371/journal.pone.0196446. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].Cotten SR, Gupta SS, Characteristics of online and offline health information seekers and factors that discriminate between them, Soc. Sci. Med 59 (2004) 1795–1806. 10.1016/j.socscimed.2004.02.020. [DOI] [PubMed] [Google Scholar]
- [16].Shieh C, Mays R, McDaniel A, Yu J, Health Literacy and Its Association With the Use of Information Sources and With Barriers to Information Seeking in Clinic-Based Pregnant Women, Health Care Women Int 30 (2009) 971–988. 10.1080/07399330903052152. [DOI] [PubMed] [Google Scholar]
- [17].Arora NK, Hesse BW, Rimer BK, Viswanath K, Clayman ML, Croyle RT, Frustrated and Confused: The American Public Rates its Cancer-Related Information-Seeking Experiences, J. Gen. Intern. Med 23 (2008) 223–228. 10.1007/s11606-007-0406-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Chen Y, Feeley TH, Numeracy, Information Seeking, and Self-Efficacy in Managing Health: An Analysis Using the 2007 Health Information National Trends Survey (HINTS), Health Commun 29 (2014) 843–853. 10.1080/10410236.2013.807904. [DOI] [PubMed] [Google Scholar]
- [19].Bernat JK, Skolarus TA, Hawley ST, Haggstrom DA, Darwish-Yassine M, Wittmann DA, Negative information-seeking experiences of long-term prostate cancer survivors, J. Cancer Surviv 10 (2016) 1089–1095. 10.1007/s11764-016-0552-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [20].Richardson A, Allen JA, Xiao H, Vallone D, Effects of Race/Ethnicity and Socioeconomic Status on Health Information-Seeking, Confidence, and Trust, J. Health Care Poor Underserved 23 (2012) 1477–1493. 10.1353/hpu.2012.0181. [DOI] [PubMed] [Google Scholar]
- [21].Vanderpool R, Kornfeld J, Finney Rutten L, Squiers L, Cancer Information-Seeking Experiences: The Implications of Hispanic Ethnicity and Spanish Language, J. Cancer Educ 24 (2009) 141–147. 10.1080/08858190902854772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [22].HINTS Briefs: Hispanics Less Likely to Seek Cancer Information than Non-Hispanics, National Cancer Institute, 2009. https://hints.cancer.gov/docs/Briefs/HINTS_Brief_12_accessible_021809.pdf. [Google Scholar]
- [23].Khoong EC, Le GM, Hoskote M, Rivadeneira NA, Hiatt RA, Sarkar U, Health information-seeking behaviors and preferences of a diverse, multilingual urban cohort, Med. Care 57 Suppl 6 Suppl 2 (2019) S176–S183. 10.1097/MLR.0000000000001050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Capurro D, Chaudhuri S, Turner AM, The Online Availability of Multilingual Health Promotion Materials Produced by Local Health Departments: an Information Assessment, Stud. Health Technol. Inform 216 (2015) 380–385. [PubMed] [Google Scholar]
- [25].Singh PM, Wight CA, Sercinoglu O, Wilson DC, Boytsov A, Raizada MN, Language preferences on websites and in Google searches for human health and food information, J. Med. Internet Res 9 (2007) e18. 10.2196/jmir.9.2.e18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].What is health literacy?, Cent. Dis. Control Prev (2021). https://www.cdc.gov/healthliteracy/learn/index.html (accessed July 19, 2021).
- [27].Neter E, Brainin E, eHealth literacy: extending the digital divide to the realm of health information, J Med Internet Res 14 (2012) e19. 10.2196/jmir.1619. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [28].Millar RJ, Sahoo S, Yamashita T, Cummins PA, Literacy skills, language use, and online health information seeking among Hispanic adults in the United States, Patient Educ. Couns 103 (2020) 1595–1600. 10.1016/j.pec.2020.02.030. [DOI] [PubMed] [Google Scholar]
- [29].Bjarnadottir RI, Millery M, Fleck E, Bakken S, Correlates of online health information-seeking behaviors in a low-income Hispanic community, Inform. Health Soc. Care 41 (2016) 341–349. 10.3109/17538157.2015.1064429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [30].Jo HS, Park K, Jung SM, A scoping review of consumer needs for cancer information, Patient Educ. Couns 102 (2019) 1237–1250. 10.1016/j.pec.2019.02.004. [DOI] [PubMed] [Google Scholar]
- [31].Yılmaz NG, Schouten BC, Schinkel S, van Weert JCM, Information and participation preferences and needs of non-Western ethnic minority cancer patients and survivors: A systematic review of the literature, Patient Educ. Couns 102 (2019) 631–650. 10.1016/j.pec.2018.11.018. [DOI] [PubMed] [Google Scholar]
- [32].Gunn CM, Paasche-Orlow MK, Bak S, Wang N, Pamphile J, Nelson K, Morton S, Battaglia TA, Health Literacy, Language, and Cancer-Related Needs in the First 6 Months After a Breast Cancer Diagnosis, JCO Oncol. Pract 16 (2020) e741–e750. 10.1200/JOP.19.00526. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [33].Arora NK, Johnson P, Gustafson DH, McTavish F, Hawkins RP, Pingree S, Barriers to information access, perceived health competence, and psychosocial health outcomes: test of a mediation model in a breast cancer sample, Patient Educ. Couns 47 (2002) 37–46. 10.1016/S0738-3991(01)00170-7. [DOI] [PubMed] [Google Scholar]
- [34].Nelson DE, Kreps GL, Hesse BW, Croyle RT, Willis G, Arora NK, Rimer BK, Viswanath KV, Weinstein N, Alden S, The Health Information National Trends Survey (HINTS): development, design, and dissemination, J. Health Commun 9 (2004) 443–460; discussion 81–84. 10.1080/10810730490504233. [DOI] [PubMed] [Google Scholar]
- [35].Rivadeneira NA, Hoskote M, Le GM, Nguyen TT, Nápoles AM, Pasick RJ, Sarkar U, Hiatt RA, Advancing Cancer Control in San Francisco: Cancer Screening in Under-represented Populations, Am. J. Prev. Med 58 (2020) e1–e9. 10.1016/j.amepre.2019.08.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [36].National Cancer Institute: Division of Cancer Control & Population Sciences, Health Information National Trends Survey, (2018). https://hints.cancer.gov/docs/Instruments/HINTS5_Cycle2_Annotated_Instrument_English.pdf. [Google Scholar]
- [37].NHIS - About the National Health Interview Survey, Cent. Dis. Control Prev (2019). https://www.cdc.gov/nchs/nhis/about_nhis.htm (accessed June 22, 2021).
- [38].Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, Bradley KA, Nugent SM, Baines AD, VanRyn M, Validation of Screening Questions for Limited Health Literacy in a Large VA Outpatient Population, J. Gen. Intern. Med 23 (2008) 561–566. 10.1007/s11606-008-0520-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [39].Sarkar U, Schillinger D, Lopez A, Sudore R, Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations, J Gen Intern Med 26 (2011) 265–71. 10.1007/s11606-010-1552-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [40].San Francisco, CA, Data USA (n.d.). https://datausa.io/profile/geo/san-francisco-ca/ (accessed July 20, 2021).
- [41].Longo DR, Understanding health information, communication, and information seeking of patients and consumers: a comprehensive and integrated model, Health Expect 8 (2005) 189–194. 10.1111/j.1369-7625.2005.00339.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [42].Jo A, Ji Seo E, Son Y, The roles of health literacy and social support in improving adherence to self‐care behaviours among older adults with heart failure, Nurs. Open 7 (2020) 2039–2046. 10.1002/nop2.599. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [43].Vollbrecht H, Arora V, Otero S, Carey K, Meltzer D, Press VG, Evaluating the Need to Address Digital Literacy Among Hospitalized Patients: Cross-Sectional Observational Study, J. Med. Internet Res 22 (2020) e17519. 10.2196/17519. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [44].Hawkins M, Elsworth GR, Hoban E, Osborne RH, Questionnaire validation practice within a theoretical framework: a systematic descriptive literature review of health literacy assessments, BMJ Open 10 (2020) e035974. 10.1136/bmjopen-2019-035974. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [45].Detailed Languages Spoken at Home and Ability to Speak English for the Population 5 Years and Over: 2009–2013, U. S. Census Bur. (2015). https://www.census.gov/data/tables/2013/demo/2009-2013-lang-tables.html (accessed June 22, 2021). [Google Scholar]
- [46].Stata Statistical Software, StataCorp LLC, College Station, TX, 2020. https://www.stata.com/ (accessed June 22, 2021).
- [47].Rodriguez JA, Davis RB, Percac-Lima S, Non–English Language Availability of Community Health Center Websites, Med. Care 57 (2019) S121–S126. 10.1097/MLR.0000000000001027. [DOI] [PubMed] [Google Scholar]
- [48].Dew KN, Turner AM, Choi YK, Bosold A, Kirchhoff K, Development of machine translation technology for assisting health communication: A systematic review, J. Biomed. Inform 85 (2018) 56–67. 10.1016/j.jbi.2018.07.018. [DOI] [PubMed] [Google Scholar]
- [49].Khoong EC, Steinbrook E, Brown C, Fernandez A, Assessing the Use of Google Translate for Spanish and Chinese Translations of Emergency Department Discharge Instructions, JAMA Intern. Med 179 (2019) 580. 10.1001/jamainternmed.2018.7653. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [50].Rodriguez JA, Fossa A, Mishuris R, Herrick B, Bridging the Language Gap in Patient Portals: An Evaluation of Google Translate, J. Gen. Intern. Med 36 (2021) 567–569. 10.1007/s11606-020-05719-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [51].Gee GC, Walsemann KM, Takeuchi DT, English Proficiency and Language Preference: Testing the Equivalence of Two Measures, Am. J. Public Health 100 (2010) 563–569. 10.2105/AJPH.2008.156976. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [52].Jia G, Aaronson D, A longitudinal study of Chinese children and adolescents learning English in the United States, Appl. Psycholinguist 24 (2003) 131–161. 10.1017/S0142716403000079. [DOI] [Google Scholar]
- [53].Berdahl TA, Kirby JB, Patient-Provider Communication Disparities by Limited English Proficiency (LEP): Trends from the US Medical Expenditure Panel Survey, 2006–2015, J. Gen. Intern. Med 34 (2019) 1434–1440. 10.1007/s11606-018-4757-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [54].Wang W, Yu N, Coping with a New Health Culture: Acculturation and Online Health Information Seeking Among Chinese Immigrants in the United States, J. Immigr. Minor. Health 17 (2015) 1427–1435. 10.1007/s10903-014-0106-8. [DOI] [PubMed] [Google Scholar]
- [55].Shin Y, Maupome G, Segmentation of Mexican-Heritage Immigrants: Acculturation Typology and Language Preference in Health Information Seeking, J. Immigr. Minor. Health 19 (2017) 1163–1173. 10.1007/s10903-016-0401-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [56].Manganello J, Gerstner G, Pergolino K, Graham Y, Falisi A, Strogatz D, The Relationship of Health Literacy With Use of Digital Technology for Health Information: Implications for Public Health Practice, J. Public Health Manag. Pract 23 (2017) 380–387. 10.1097/PHH.0000000000000366. [DOI] [PubMed] [Google Scholar]
- [57].Housten AJ, Hoover DS, Correa-Fernández V, Strong LL, Heppner WL, Vinci C, Wetter DW, Spears CA, Castro Y, Associations of Acculturation with English- and Spanish-Language Health Literacy Among Bilingual Latino Adults, HLRP Health Lit. Res. Pract 3 (2019). 10.3928/24748307-20190219-01. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [58].Lee S-YD, Stucky BD, Lee JY, Rozier RG, Bender DE, Short Assessment of Health Literacy-Spanish and English: A Comparable Test of Health Literacy for Spanish and English Speakers: Short Assessment of Health Literacy, Health Serv. Res 45 (2010) 1105–1120. 10.1111/j.1475-6773.2010.01119.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [59].Sun Y, Zhang Y, Gwizdka J, Trace CB, Consumer Evaluation of the Quality of Online Health Information: Systematic Literature Review of Relevant Criteria and Indicators, J. Med. Internet Res 21 (2019) e12522. 10.2196/12522. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [60].Zhang Y, Sun Y, Xie B, Quality of health information for consumers on the web: A systematic review of indicators, criteria, tools, and evaluation results: Quality of Health Information for Consumers on the Web, J. Assoc. Inf. Sci. Technol 66 (2015) 2071–2084. 10.1002/asi.23311. [DOI] [Google Scholar]
- [61].Khoong EC, Rivadeneira NA, Hiatt RA, Sarkar U, The use of technology for communicating with clinicians or seeking health information in a multilingual urban cohort: cross-sectional survey, J. Med. Internet Res 22 (2020) e16951. 10.2196/16951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [62].Swoboda CM, Van Hulle JM, McAlearney AS, Huerta TR, Odds of talking to healthcare providers as the initial source of healthcare information: updated cross-sectional results from the Health Information National Trends Survey (HINTS), BMC Fam. Pract 19 (2018) 146. 10.1186/s12875-018-0805-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [63].Mantwill S, Schulz PJ, Health literacy in Mainland China: Validation of a functional health literacy test in simplified Chinese, Health Promot. Int (2015) dav043. 10.1093/heapro/dav043. [DOI] [PubMed]
- [64].Hutchinson N, Baird GL, Garg M, Examining the Reading Level of Internet Medical Information for Common Internal Medicine Diagnoses, Am. J. Med 129 (2016) 637–639. 10.1016/j.amjmed.2016.01.008. [DOI] [PubMed] [Google Scholar]
- [65].Toolkit for Making Written Material Clear and Effective, Cent. Medicare Medicaid Serv (2020). https://www.cms.gov/Outreach-and-Education/Outreach/WrittenMaterialsToolkit (accessed July 20, 2021).
- [66].The CDC Clear Communication Index, Cent. Dis. Control Prev (2021). https://www.cdc.gov/ccindex/index.html (accessed July 20, 2021).
- [67].Healthy People 2030, US Dep. Health Hum. Serv (n.d.). https://health.gov/healthypeople (accessed July 20, 2021).
- [68].Brach C, Keller D, Hernandez LM, Baur C, Parker R, Dreyer B, Schyve P, Lemerise AJ, Schillinger D, Ten Attributes of Health Literate Health Care Organizations, NAM Perspect (2012). 10.31478/201206a. [DOI]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.