Accuracy in self-reported health literacy screening: a difference between men and women in Taiwan

Abstract

Objective

Self-reported screening questions are considered as an effective way to identify patients with limited health literacy. Yet research has shown that individuals tend to over-report their reading level. Moreover, the likelihood of over-reporting may differ between gender groups. This study examined if systematic differences exist between men and women in their response to self-reported screening questions.

Design

A national survey in Taiwan with participants selected using a multistage stratified, probability-proportional-to-size sampling strategy.

Participants

A total of 5682 Taiwanese adults aged 18 and older were sampled and recruited. Of those adults, 3491 participated in the survey, resulting in a 62.1% response rate. Both gender groups were equally represented in the final study sample.

Main measures

Self-reported health literacy was assessed using two sets of questions that asked how difficult it was for the respondent to understand written health materials and how often the participant needed assistance from others to understand written health materials. The objective level of health literacy was measured using the Mandarin Health Literacy Scale (MHLS).

Results

A significant gender difference was observed among participants who had inadequate health literacy: while women's self-report was in line with the MHLS test result, men had a significant tendency to over-report their comprehension of health information.

Conclusions

In Taiwan, screening questions are prone to socially desirable response and may underidentify male patients with inadequate health literacy. Development of a brief and easy-to-use health literacy test may be a more effective approach to health literacy screening in clinical settings. Alternatively, clinicians can verify patient comprehension of health information via the ‘teach back’ or ‘show me’ technique in order to improve communication and patient care. Research is needed to examine if gender differences in self-report of health literacy exist in other countries.

Strengths and limitations of this study.

• To our knowledge, this is the first study that compares self-reported comprehension of health information with an objective measure of health literacy.

• The study uses data collected from a nationally representative sample.

• It is not clear if gender differences in self-reporting of health information comprehension exist in clinical settings.

Introduction

Accumulated evidence suggests that health literacy—defined in the Institute of Medicine report Health Literacy: A Prescription to End Confusion¹ as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions”—is a key determinant for health knowledge, health behaviour, healthcare utilisation and disease outcomes.^2–16 Being able to assess patients' health literacy level so as to effectively communicate with patients and properly involve them in the care process is now considered a trademark of patient-centred care and a major part of the overall effort to improve healthcare quality and reduce costs.^{17 18}

As it is difficult to estimate patients’ health literacy level and most health literacy assessments are too cumbersome to use as a screening tool in clinical settings, several studies have proposed the use of simple screening questions—such as ‘How would you rate your ability to read?’ ‘How confident are you filling out medical forms by yourself?’ and ‘How often do you have someone help you read hospital materials?’—as an alternative way to identify patients who have problems understanding written health information and following medical instructions.^19–23 Although the exact wording of screening questions may differ, all the questions are designed to assess, based on patients’ self-report, their ability to comprehend health information and navigate the health delivery system. Validation of those self-reported screening questions showed that they had satisfactory correlations with such health literacy instruments as the Test of Functional Health Literacy in Adults (TOFHLA) and the Rapid Estimate of Adult Literacy in Medicine (REALM) and that they were good predictors of limited health literacy.^19–26 Two recent systematic reviews of health literacy screening instruments suggested that self-reported screening questions were a reliable, valid and feasible means for establishing health literacy.^{27 28}

Endorsement for the wide use of self-reported questions to screen for patients with inadequate health literacy may be premature. Research has shown that individuals tend to over-report their reading level.^{29 30} As such, self-reports of health literacy may also be prone to over-reporting. More importantly, the likelihood of over-reporting may differ systematically between gender groups because of a differential desire for social approval—that is, a social desirability bias or a tendency to shape one's response to create a favourable impression of oneself.³¹ Studies have found that women, more than men, are likely to under-report weight³² and dietary intake,³³ whereas men are more likely to downplay their fear of crime.³⁴ Extant research, however, has not examined gender differences in self-reported health literacy. If such differential reporting exists, the use of self-reported questions to establish health literacy level may lead to biased treatment, or overlook the need, of certain patient groups.

In this paper, we examine if systematic differences exist between men and women in their response to self-reported screening questions using data collected from a nationally representative sample of Taiwanese adults. An objective measure of health literacy is included in the analysis to adjust for participants’ baseline level of comprehension in the comparison of self-reported health literacy between men and women.

Methods

Data for this study came from a national survey conducted by the National Health Research Institutes (NHRI) in Taiwan in 2008–2009. The survey was to assess the level of health literacy in Taiwanese adults using a validated instrument, the Mandarin Health Literacy Scale (MHLS).^{35 36}

Research context—Taiwanese Health Care System

Taiwan has a national health insurance (NHI) programme.³⁷ The programme, enacted in March 1995, is a government-run, single-payer system. It incorporates three key features: compulsory universal coverage, uniform comprehensive benefits and financing through premiums and government subsidy. As a result of the NHI, the percentage of the population with health insurance has exceeded 99% since 2004.

The NHI benefit package is comprehensive, including all medical and laboratory services, dental care (except for orthodontics and prosthodontics), prescription and certain over-the-counter drugs, traditional Chinese medicine and home nurse visits. Most preventive services (annual check-ups, maternal and child healthcare, and regular well-child care up to age 6) are free. Expensive treatment for HIV/AIDS and organ transplants are covered. Regular office visits have a modest copayment (approximately US$5 for each visit at a private clinic and US$8 for a visit to a hospital ambulatory care clinic). Hospitalisation has a 10% coinsurance, which is capped at 6% of the average national income per person for each admission and 10% of the average national income per person for each calendar-year. Poor households are exempted from cost sharing. The comprehensive design of benefits has greatly reduced the financial barrier to healthcare and there is no evidence of a gender difference in healthcare access.

NHRI survey of health literacy in Taiwan

The health literacy survey was conducted on Taiwanese adults, aged 18 and older, selected based on the Taiwanese household registration system and using a multistage stratified, probability-proportional-to-size sampling strategy. A total of 5682 adults who met the initial inclusion criteria was sampled. Willing participants were interviewed in person by trained interviewers. At the scheduled interview, interviewers first explained the consent form and obtained the signature of participants if they agreed to the questionnaire interview. Interviewers then asked participants to read aloud a brief text to identify those who were illiterate or unable to read. To prevent participants who were unable to read (N=365) from feeling embarrassed or uncomfortable,^{38 39} they were not asked to complete the self-administered MHLS test and received a zero score. The other participants took the test, which lasted on average 25 min, and were scored. All the survey participants responded to the questionnaire, which was administered by interviewers and took about 25 min to complete. The questionnaire assessed, among other variables, the sociodemographic attributes of participants. It also included two sets of questions, described in detail below, that asked participants to report their ability to comprehend health information.

Excluding non-respondents (individuals who refused or could not be reached after five attempts of phone or mail contact) and 35 individuals who had uncorrectable vision and hearing problems, who were cognitively impaired and who refused the MHLS test, the effective sample size was 3491 adults and the response rate was 62.1%. Survey participants were not statistically different from those who were excluded in terms of age, gender and formal education.

The protocol of the survey was reviewed and approved by the human participant review board at the Taiwanese NHRI.

Measurement

Two sets of questions were included in the NHRI survey to assess respondents’ self-reported comprehension of health information. The first set contained the following four questions and the response was recorded on a five-point Likert scale (1=very difficult, 2=difficult, 3=neutral, 4=easy and 5=very easy).

• UND1: ‘How difficult is it for you to understand the information provided on a medication label?’

• UND2: ‘How difficult is it for you to understand the information provided on a medical appointment slip?’

• UND3: ‘How difficult is it for you to understand the information provided on a consent form?’

• UND4: ‘How difficult is it for you to understand the information provided in a self-care guide?’

The second set included four more questions with five response options (1=none of the time, 2=a little of the time, 3=some of the time, 4=most of the time and 5=all of the time):

• ASS1: ‘How often do you need assistance from others to understand the information provided on a medication label?’

• ASS2: ‘How often do you need assistance from others to understand the information provided on a medical appointment slip?’

• ASS3: ‘How often do you need assistance from others to understand the information provided on a consent form?

• ASS4: ‘How often do you need assistance from others to understand the information provided in a self-care guide?’

We reverse-coded the responses to the second set of questions (ie, 1=all of the time, 2=most of the time, 3=some of the time, 4=a little of the time and 5=none of the time) so that a higher score indicated better ability to comprehend health information.

The NHRI survey used the MHLS to objectively assess the respondent's health literacy level. The MHLS is a reading test designed for the population of Mandarin Chinese speakers that use Traditional Chinese characters—the standard writing in Taiwan.³⁶ The scale contains 50 items, of which 33 test comprehension of health-related texts and 17 assess numeracy skills. In a random sample of 323 Taiwanese adults, the scale was found to have a high correlation with years of formal education, suggesting high convergent validity. The scale was significantly associated with reading habit, health knowledge and receipt of assistance with reading written health materials, indicating good predictive validity. Furthermore, it had high internal reliability (Cronbach's α=0.95) and split-half reliability (Spearman-Brown correction=0.95). Based on the test score, respondents could be classified into three health literacy levels: inadequate (0–30), marginal (31–42) and adequate (43–50).⁴⁰

The main demographic variable of interest in this study was gender (male and female). We also included age, educational attainment (years of formal education), personal income (average monthly income) and residential location (metropolitan city, mid-sized city, small city, rural/remote area) as control variables in the comparison of men's versus women's response to self-reported questions regarding comprehension of health information. Educational attainment, personal income and residential location were proxies for a respondent's socioeconomic status.

Statistical analysis

We performed a descriptive analysis and t test to examine if men and women who had the same level of health literacy responded differently to self-reported questions regarding health information comprehension. As the observed differences may be confounded by age and socioeconomic status, we further modelled self-reported comprehension of health information as a function of gender (female as the reference) using multivariate regression analysis, holding constant health literacy level (adequate health literacy as the reference) as well as age, educational attainment, personal income and residential location.

Results

As indicated above, a total of 3491 adults completed the MHRI health literacy survey. The mean age of participants was 45.6 years. Both gender groups were about equally represented. Sixty-two per cent of participants had no more than 12 years of formal education (equivalent to a high school diploma or less).

Table 1 reports the mean level of self-reported comprehension of health information among men vis-à-vis women, classified by the health literacy level. The results are also displayed in the two bar charts in figure 1. Men consistently reported better comprehension—that is, better understanding of health information and needing less assistance with written health information—than women in their response to all eight survey questions. The differences were small in the groups that had an adequate or marginal level of health literacy and only few of the differences reached statistical significance at p<0.05. Greater differences were observed between men and women who had an inadequate level of health literacy. All the differences in that group were statistically significant at p<0.001.

Table 1.

Gender differences in self-reported comprehension of health information by health literacy level (N=3491)

		Self-report questions
Health literacy level	Gender	UND1	UND2	UND3	UND4	ASS1	ASS2	ASS3	ASS4
Adequate	Male	4.27	4.32	4.04	4.08	4.85	4.89	4.54	4.61
	Female	4.30	4.36	4.03	4.07	4.80	4.87	4.44	4.51
	p Value of t test	0.45	0.12	0.61	0.73	0.04	0.22	0.005	0.003
Marginal	Male	4.00	4.06	3.74	3.75	4.69	4.74	4.26	4.35
	Female	3.90	3.99	3.63	3.61	4.58	4.65	4.08	4.10
	p Value of t test	0.20	0.33	0.20	0.11	0.12	0.15	0.08	0.012
Inadequate	Male	2.99	3.08	2.58	2.54	3.42	3.46	2.71	2.82
	Female	2.11	2.16	1.76	1.73	2.26	2.31	1.64	1.66
	p Value of t test	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001

Figure 1.

Distribution of self-reported comprehension of health information by gender and health literacy level.

Table 2 presents the results of the multivariate regression analysis. Model 1, which included only the gender variable, showed that men were significantly more likely than women to report good comprehension of health information. Including control variables (model 2) reduced the coefficients associated with the male dummy variable, and only the four coefficients related to reading assistance remained statistically significant. In light of the observed variation by health literacy level in table 1 and figure 1, we added two interaction terms to the analysis (model 3). All, except two, coefficients of the male dummy variable became non-significant with the addition of the interaction terms. Moreover, only the interaction term between male and inadequate health literacy was statistically significant, and consistently so across all eight survey questions, indicating that gender differences existed only in adults with inadequate health literacy.

Table 2.

Multivariate analysis of gender differences in self-reported comprehension of health information (N=3491)

	Self-report questions
	UND1	UND2	UND3	UND4	ASS1	ASS2	ASS3	ASS4
OLS models	β-Coefficient	β-Coefficient	β-Coefficient	β-Coefficient	β-Coefficient	β-Coefficient	β-Coefficient	β-Coefficient
Model 1: gender only
Male	0.082**	0.076**	0.086**	0.084**	0.132**	0.127**	0.125**	0.133*
Model 2: gender+controls†
Male	0.016	0.009	0.021	0.016	0.066**	0.057**	0.065**	0.068**
Model 3: gender+controls+gender*HL
Male	−0.033*	−0.044**	−0.015	−0.021	0.009	−0.002	0.023	0.022
Male×marginal HL	0.023	0.021	0.016	0.022	0.007	0.008	0.009	0.019
Male×inadequate HL	0.161**	0.178**	0.121**	0.123**	0.208**	0.217**	0.152**	0.159**

*Statistically significant at p<0.05.

**Statistically significant at p<0.01.

†Controls include: age, health literacy level (adequate HL being the reference), educational attainment, personal income and residential location.

The analysis so far did not answer whether men (or women) with inadequate health literacy tended to over-report (or under-report) their comprehension. To answer this question, we converted the self-reported response and MHLS test score to z-scores—that is, we standardised them as z-score_self-report and z-score_MHLS. We then subtracted z-score_MHLS from z-score_self-report. The z-score differences in the group of inadequate health literacy adults were all positive. Moreover, the differences were small and non-significant in women and were significantly larger in men, suggesting that, of those tested to have inadequate health literacy, women's self-report was in line with the MHLS test result and men, on the other hand, had a significant tendency to over-report their comprehension of health information. The z-score differences in the adequate and marginal health literacy groups were all statistically non-significant (results available from the corresponding author).

Discussion

Knowing patients’ health literacy level may help physicians and other health professionals tailor their treatment and care management plans to the specific needs of patients.⁴¹ The development of screening questions based on patient self-report was intended to provide physicians and other healthcare providers an efficient and inexpensive tool to identify patients whose health literacy level is inadequate and who therefore need special assistance, education or additional support. Although the validity of screening questions was supported in prior research,^19–26 we found in our analysis of data from a nationally representative sample of Taiwanese adults that screening questions were an inaccurate means for identifying men with inadequate health literacy. Specifically, we found that men with inadequate health literacy had a tendency to disguise their low-health literacy ability by over-responding to screening questions. Therefore, using those questions in clinical practice or research is likely to underestimate the true degree of inadequate health literacy in men and overlook the needs of a risky group of the patient population, considering that men, compared with women, also exhibit more risky behaviours, are less aware of symptoms and problems needing medical attention, have poorer health and are less likely to seek healthcare.^{42 43}

The over-reporting observed in this study is most likely due to socially desirable response, stemming from a masculinity culture that is common in Eastern and Western countries.^{44 45} To the extent the explanation is valid, an implication of our study is that clinicians need to be aware of, and sensitive to, aspects of ‘maleness,’ including men's help-seeking behaviours and communication patterns, so as to present information in a way men feel comfortable with. Creating a supportive and accepting environment and couching practical health information in ‘male’ terms (eg, communicating to men that the masculine trait of being in control and independent extends to acknowledging, accepting and overcoming individual challenges and owning the responsibility to improve personal health), for example, may be effective strategies to promote positive health and help-seeking behaviours among male patients who have problems understanding health information and are reluctant to follow medical instructions.⁴⁴ A masculinity culture may also be related to men's desire for self-reliance and their tendency to avoid encounters with the formal health delivery system.⁴⁶ Understanding the tension between masculinity and health behaviours would contribute to improving health promotion and social marketing efforts towards men.⁴⁵

Our findings are similar to the results reported in Parikh et al³⁸ They found in a clinic sample recruited at a large, public hospital in Atlanta, Georgia a significant proportion of low-health literacy patients who were ashamed and hid their reading problems and who never told their spouses or children of their inability to read health information. Although the study did not examine the gender difference, it did indicate that most of the patients in the sample who had low-health literacy were men. Given these findings, it is curious to note that a majority of patients surveyed in two recent studies were receptive to health literacy screening or felt that the screening would be useful.^{41 47} Two likely explanations are that the positive response was primarily among patients who had adequate health literacy skills and that men constituted a very small portion of patients who had problems reading and comprehending written health information in the study samples. The extent to which these explanations are valid would require further investigation, and the results would inform clinicians and healthcare organisations of specific types of patients who are sensitive to health literacy screening and alternative ways of communicating with those patients. Another possibility is that health literacy screening, at least in one study, was conducted using a task-based assessment. This may have spared patients from having to demonstrate poor reading skills to the examiner, thus increasing the acceptance for screening. To the extent this is true, task-based assessments may avoid over-reporting and produce more accurate evaluation of patients’ health literacy level.

Three caveats of our study should be noted. First, data for this study came from Taiwan. Although the few cross-cultural and international studies have found universal gender differences in socially desirable response,^{48 49} further investigation is needed to verify our findings in other cultural environments. Second, our study sample was drawn from the general population. The response we observed in the survey may be different from that of patients receiving care in a clinical setting. Patients who understand the importance of providing physicians correct information to reach an accurate diagnosis and proper treatment may be more likely to admit their inability to read if they indeed have the problem. Third, most of the interviewers employed in the NHRI survey were women. The interviewer–interviewee gender difference may increase male respondents’ propensity to disguise their reading problems and to over-report their comprehension abilities.

While instituting health literacy screening to help physicians and other healthcare providers to assess their patients’ limited health literacy may improve healthcare quality and clinical outcomes, screening questions that are prone to socially desirable response may not meet the intended purpose. Perhaps a more effective strategy is to develop brief instruments that objectively test patients’ health literacy level and that are inexpensive and easy to use in clinical settings.⁵⁰ Alternatively, health professionals may verify patient comprehension via the ‘teach back’ or ‘show me’ technique.^{26 51} Finally, since there is yet no agreement that screening patients’ health literacy in the clinical setting is valuable, health professionals must develop new communication strategies—strategies that are sensitive to gender differences—in order to improve communication and patient care outcomes.