Abstract
Background
Health literacy has been increasingly recognized as one of the most important social determinants for health. However, an appropriate and comprehensive assessment tool is not available in many Asian countries. This study validates a comprehensive health literacy survey tool European health literacy questionnaire (HLS-EU-Q47) for the general public in several Asian countries.
Methods
A cross-sectional survey based on multistage random sampling in the target countries. A total of 10,024 participants aged ≥15 years were recruited during 2013–2014 in Indonesia, Kazakhstan, Malaysia, Myanmar, Taiwan, and Vietnam. The questionnaire was translated into local languages to measure general health literacy and its three domains. To evaluate the validity of the tool in these countries, data were analyzed by confirmatory factor analysis, internal consistency analysis, and regression analysis.
Results
The questionnaire was shown to have good construct validity, satisfactory goodness-of-fit of the data to the hypothetical model in three health literacy domains, high internal consistency (Cronbach's alpha >0.90), satisfactory item-scale convergent validity (item-scale correlation ≥0.40), and no floor/ceiling effects in these countries. General health literacy index score was significantly associated with level of education (P from <0.001 to 0.011) and perceived social status (P from <0.001 to 0.016), with evidence of known-group validity.
Conclusions
The HLS-EU-Q47 was a satisfactory and comprehensive health literacy survey tool for use in Asia.
Keywords: Validation, Health literacy, HLS-EU-Q47, Asian health literacy surveys, Confirmatory factor analysis
1. Introduction
Health literacy has been recognized as a key factor to improve health and well-being and reduce health inequities.1,2 The comprehensive definition of health literacy has been developed by Sorensen and colleagues to entail the knowledge, motivation, and competences to access, understand, appraise, and apply information in everyday life to make judgments and decisions in terms of health care, disease prevention, and healthy behaviors that maintain and promote quality of life throughout the life course.3 Adequate health literacy enables health-friendly environments, efficient health policy implementation, effective health promotion efforts, better self-care with fewer health risks, better health care outcomes, and lower health care costs.4
It is very important for health professionals to understand patients' health literacy before delivering interventions or education.5 Several instruments have been developed to measure health literacy in community and clinical settings, such as the Test of Functional Health Literacy in Adults (TOFHLA), which assesses the ability to understand health information6; the Rapid Estimate of Adult Literacy in Medicine (REALM), which assesses the ability to read health terms7; the Newest Vital Sign (NVS)8; Ishikawa's Japanese measure of Functional Communicative and Critical Health Literacy (FCCHL)9; the “2009 Chinese Health Literacy questionnaire”10; and the Mandarin Health Literacy Scale in Taiwan.11 However, comprehensive tools to measure health literacy in different settings and contexts are uncommon.
A comprehensive questionnaire to measure health literacy in populations has been designed for the European Health Literacy Survey (HLS-EU-Q47). This questionnaire is grounded in a conceptual framework and operationalized with a matrix with 12 dimensions, including four information processing domains (finding, understanding, judging, and applying) and three health domains (health care, disease prevention, and health promotion), which enabled comparisons within and between countries.3,12 The terms and notions of HLS-EU-Q47 were synchronized from 17 explicit health literacy definitions found in previous survey tools.3 The questionnaire is focused on measuring health literacy not merely in clinical settings, but also in populations and communities.13 On the other hand, a review by Nguyen et al. revealed that 64% of available measures did not include Asians.14 The authors recommended that the tool should be properly validated before wider use. This population study aims to validate the comprehensive health literacy questionnaire HLS-EU-Q47 in several Asian countries.
2. Methods
2.1. Study design
A population-based cross-sectional design was used to survey six Asian countries (Indonesia, Kazakhstan, Malaysia, Myanmar, Taiwan, and Vietnam) in 2013–2014 using the HLS-EU-Q47 comprehensive health literacy questionnaire. The questionnaire was translated and administered in seven languages: Indonesian, Kazakh, Russian, Malay, Myanmar/Burmese, Mandarin, and Vietnamese. They were pilot tested in each country in respective languages and evaluated by country experts. The survey was then administered in each country by well-trained interviewers, following a standardized protocol provided by the Asian Health Literacy Survey Consortium (AHLS), which was established by research partners in these invited countries as academic representatives of the Asian Health Literacy Association. This consortium was responsible for coordinating the survey and standardizing the procedure to manage and ensure the quality of interviews.
2.2. Sampling methods
Data in all participating countries were collected and coordinated by each country's respective member of the Asian Health Literacy Consortium. Samples were restricted to citizens aged 15 years and above, using similar sample selection criteria as in the HLS-EU-Study, which was based on EUROBAROMETER criteria.12,15
The multi-stage random sampling methods were used in different countries with different population structures. A community-based nationwide survey was conducted in Taiwan, as has been previously described in details,16 while community-based city or regional surveys were conducted in five main cities in Kazakhstan (Almaty, Aktobe, Atyrau, Ust-Kamenogorsk, and Kostanay); the region of North Java in Indonesia (Semarang); three main cities of Northern Vietnam, including urban, suburban, and island areas in Hanoi, Hai Duong, and Haiphong; the central Kuala Lumpur and Selangor regions and the Northern Perak regions of Peninsular in Malaysia17; and one state and five regions in Myanmar.18
2.3. Questionnaires and measurements
2.3.1. Health literacy survey questionnaire (HLS-EU-Q47)
The HLS-EU-Q47 contained 47 items measuring health literacy. The perceived difficulty of each item was rated on a 4-point Likert scale (1 = very difficult, 2 = difficult, 3 = easy, and 4 = very easy), with a possible lowest mean score of 1 and a possible highest mean score of 4. Therefore, the mean score varied from 1 to 4, and the range of the mean score was (4−1 = ) 3.13 The HLS-EU-Q47 was based on a conceptual model of health literacy and measures four competences to deal with health relevant information (access/obtain, understand, appraise/judge/evaluate, and apply/use health information) in three domains: health care, disease prevention, and health promotion.3
The indices for health literacy were standardized to unified metrics from 0 to 50 using the formula;
Index = (mean – 1)*(50/3).
Where Index was the specific index calculated, mean was the mean of all participating items for each individual, 1 is the minimal possible value of the mean (leading to a minimum value of the index of 0), 3 was the range of the mean, and 50 was the chosen maximum value of the new metric. Thus, an index value was obtained where 0 represented the lowest health literacy and 50 the highest health literacy.12,15
With the agreement from the HLS-EU consortium, the HLS-EU-Q47 was translated into Indonesian, Kazakh, Russian, Malay, Myanmar/Burmese, Traditional Mandarin, and Vietnamese (eAppendix 1), using the translation-back-translation method.19 The content of the questionnaire was verified by public health experts in each country to reflect cultural perspectives. The questionnaire was pre-tested for readability and understandability by experienced survey researchers in each country.
2.3.2. Personal characteristics and socio-demographics
Questions on the following were requested from the respondents during the survey: age (years), gender (male or female), the highest education attainment (elementary school, junior high school, senior high school, or college/university and above), ability to pay for medication (very difficult, fairly difficult, fairly easy, or very easy), and self-assessed social status (low, middle, or high).
2.4. Participant and data collection procedure
The interviewers contacted the selected participants and provided the self-reported questionnaire, and a total of 10,210 people in six countries participated in the study anonymously. In each country, participants were invited to take part in face-to-face interviews with well-trained interviewers following a standardized protocol. A consent form was obtained from each participant, and adequate time was allowed for all participants to answer the questionnaire.
After excluding unsatisfactory responses that included significant missing data in their questionnaire, the overall sample of 10,024 participants was analyzed, including 1029 from Indonesia, 1845 from Kazakhstan, 1600 from Myanmar, 462 from Malaysia, 3015 from Taiwan, and 2073 from Vietnam.
To ensure standardization and quality assurance in data collection, a standard work package was provided by the Consortium to each country coordinator. The country-specific surveys were conducted from February 2013 to December 2014. Each country provided technical reports and sent the data to the Consortium.
2.5. Ethical approval
The study was approved by the Institutional Review Board (IRB) in all partner countries: the Joint IRB of the Taipei Medical University in Taiwan (TMU-JIRB No. 201305007); the Ethics Committee of the Kazakhstan School of Public Health (No IRB - A043); the Institutional Ethical Review Committee of Hanoi School of Public Health, Vietnam (IRB of HSPH No. 014–254/DD-YTCC); the Institutional Ethical Review Committee of University of Medicine 1, Yangon, Myanmar; the Institutional Ethical Review Committee of Dian Nuswantoro University, Indonesia (No. 33/EC/FKM/2014); and the Medical Ethics Committee, University Malaya Medical Centre, Malaysia (MEC Ref. No: 896.34).
2.6. Data analysis
The survey questionnaires were translated into target languages using a forward-backward translation process, which followed the updated guideline for translation, adaptation, validation of instruments,19 and cultural perspectives were taken into account. The questionnaires were also pre-tested by research partners in selected countries. In this article, we analyzed the psychometric properties of the HLS-EU-Q47 questionnaire in different countries as follows:
2.6.1. Validity analyses
To establish construct validity, confirmatory factor analysis (CFA) was conducted separately for the three health literacy domains of health care, disease prevention, and health promotion, in which items were loaded onto four hypothetical factors related to finding, understanding, judging, and applying health information. The fit of the data to the model was examined using goodness-of-fit indices, including (i) absolute model fit: root mean square error of approximation (RMSEA) and goodness-of-fit index (GFI); (ii) incremental fit: adjusted goodness-of-fit index (AGFI), comparative fit index (CFI), incremental fit index (IFI), and normal fit index (NFI); and (iii) parsimonious fit, or the chi-square goodness-of-fit test (i.e., the chi-square/degrees of freedom ratio [χ2/df ratio]). More satisfied indices indicate better construct validity of the questionnaire.20
Item-scale convergent validity was examined using correlation between the item and its own theoretical scale,21 which was determined by the Pearson correlation coefficient. When the r-value was between 0.36 and 0.67, it was considered moderately correlated; r values between 0.68 and 1.0 were considered highly correlated.22
Known-group validity was also assessed through comparison of the means of general health literacy index between groups with different education attainment (junior high school and below, senior high school, or university and above), and social status (low, middle, or high).23
2.6.2. Reliability analyses
Internal consistency was tested with Cronbach's alpha, and values greater than or equal to 0.7 indicate satisfactory reliability.24 The split-half reliability was also examined.11,23
2.6.3. Floor and ceiling effects analyses
Due to the limited responsiveness of such a large-scale survey, floor or ceiling effects, which refer to a high percentage of participants scoring possibly the lowest score or achieving possibly the highest score, respectively, were concerned. Therefore, minimal cut-offs for significant floor and ceiling effects were recommended, and for the HLS-EU-Q47 scale, a percentage of 15% or more at floor or at ceiling was considered a significant effect.25
All statistical analyses were performed using the IBM SPSS Version 20.0, AMOS version 22.0 (IBM Corp, Armonk, NY, USA).26 The significance level was set at P < 0.05.
3. Results
The participants' characteristics were shown in Table 1.
Table 1. Characteristics of participants and their general health literacy index score in six Asian countries.
| Indonesia | Kazakhstan | Malaysia | Myanmar | Taiwan | Vietnam | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (n = 1029) | (n = 1845) | (n = 462) | (n = 1600) | (n = 3015) | (n = 2073) | |||||||
| n | % | n | % | n | % | n | % | n | % | n | % | |
| Age, mean (SD) years | 30.6 (12.5) | 35.1 (15.8) | 47.1 (6.6) | 39.6 (14.3) | 34.2 (16.8) | 41.1 (17.1) | ||||||
| Gender | ||||||||||||
| Women | 570 | 55.4 | 1052 | 57.5 | 258 | 55.8 | 1015 | 63.5 | 1654 | 54.9 | 1182 | 57.3 |
| Men | 459 | 44.6 | 776 | 42.5 | 204 | 44.2 | 584 | 36.5 | 1361 | 45.1 | 880 | 42.7 |
| Education | ||||||||||||
| Elementary school | 90 | 8.7 | 66 | 3.9 | 40 | 9.5 | 219 | 14.2 | 132 | 4.4 | 219 | 10.7 |
| Junior high school | 190 | 18.5 | 373 | 21.8 | 142 | 33.5 | 529 | 34.2 | 316 | 10.5 | 745 | 36.3 |
| Senior high school | 510 | 49.6 | 261 | 15.3 | 183 | 43.3 | 411 | 26.6 | 1263 | 41.9 | 656 | 31.9 |
| College/University and above | 239 | 23.2 | 1009 | 59.0 | 58 | 13.7 | 387 | 25.0 | 1301 | 43.2 | 434 | 21.1 |
| Ability to pay for medication | ||||||||||||
| Very difficult | 13 | 1.3 | 124 | 7.3 | 31 | 6.9 | 19 | 1.2 | 160 | 5.3 | 221 | 10.8 |
| Fairly difficult | 150 | 14.6 | 255 | 15.1 | 195 | 43.4 | 245 | 15.4 | 784 | 26.2 | 715 | 34.9 |
| Fairly easy | 696 | 67.6 | 913 | 53.8 | 180 | 40.1 | 755 | 47.3 | 1603 | 53.5 | 950 | 46.3 |
| Very easy | 169 | 16.4 | 404 | 23.8 | 43 | 9.6 | 576 | 36.1 | 450 | 15.0 | 163 | 8.0 |
| Self-assessed social status | ||||||||||||
| Low | 168 | 16.6 | 609 | 41.5 | 117 | 29.8 | 455 | 30.0 | 1181 | 40.1 | 877 | 43.2 |
| Middle | 744 | 73.7 | 603 | 41.1 | 208 | 52.9 | 866 | 57.1 | 1651 | 56.0 | 1056 | 52.0 |
| High | 98 | 9.7 | 255 | 17.4 | 68 | 17.3 | 195 | 12.9 | 114 | 3.9 | 98 | 4.8 |
| Mean (SD) Gen-HL index | 31.4 (5.8) | 31.6 (9.3) | 32.9 (7.2) | 31.3 (8.7) | 34.4 (6.6) | 29.6 (9.1) | ||||||
Gen-HL, General Health Literacy; SD, standard deviation.
3.1. Construct validity
The CFA was employed to test construct validity. The results showed a good fit of the data to the hypothetical model for three domains of health literacy in all the six countries. The RMSEA index was less than 0.10, and other goodness-of-fit indices (GFI, AGFI, CFI, IFI, and NFI) were 0.90 for most domains in different countries, which is adequate to be considered a good model-data fit.20 In particular, the AGFI was 0.85 for disease prevention health literacy (DP-HL) and 0.86 for health care health literacy (HC-HL) in Vietnam, and 0.86 for DP-HL in Malaysia, while the NFI index was 0.89 for HC-HL in Indonesia, 0.87 for HC-HL and 0.88 for DP-HL in Malaysia, representing a tolerable fit.27 The overall results supported the fitness of the four-factor structure within each of the three domains of the HLS-EU-Q47 (Table 2).
Table 2. Construct validity of the HLS-EU-Q47 in six Asian countries with goodness-of-fit indices.
| Modela | Absolute model fit | Incremental fit | Parsimonious fit | ||||
|---|---|---|---|---|---|---|---|
| RMSEA | GFI | AGFI | CFI | IFI | NFI | χ2/df | |
| Indonesia | |||||||
| HC-HL | 0.07 | 0.94 | 0.90 | 0.90 | 0.90 | 0.89 | 6.06 |
| DP-HL | 0.07 | 0.94 | 0.90 | 0.91 | 0.91 | 0.90 | 6.56 |
| HP-HL | 0.06 | 0.95 | 0.93 | 0.94 | 0.94 | 0.92 | 4.16 |
| Kazakhstan | |||||||
| HC-HL | 0.05 | 0.96 | 0.95 | 0.97 | 0.97 | 0.96 | 6.10 |
| DP-HL | 0.06 | 0.95 | 0.93 | 0.96 | 0.96 | 0.96 | 8.53 |
| HP-HL | 0.06 | 0.96 | 0.94 | 0.97 | 0.97 | 0.96 | 7.32 |
| Malaysia | |||||||
| HC-HL | 0.08 | 0.90 | 0.86 | 0.90 | 0.90 | 0.87 | 3.95 |
| DP-HL | 0.09 | 0.90 | 0.86 | 0.91 | 0.91 | 0.88 | 4.53 |
| HP-HL | 0.06 | 0.94 | 0.91 | 0.96 | 0.96 | 0.93 | 2.55 |
| Myanmar | |||||||
| HC-HL | 0.08 | 0.93 | 0.90 | 0.92 | 0.92 | 0.91 | 9.94 |
| DP-HL | 0.07 | 0.94 | 0.91 | 0.94 | 0.94 | 0.93 | 9.11 |
| HP-HL | 0.06 | 0.95 | 0.93 | 0.94 | 0.94 | 0.94 | 6.92 |
| Taiwan | |||||||
| HC-HL | 0.07 | 0.94 | 0.91 | 0.93 | 0.93 | 0.92 | 17.37 |
| DP-HL | 0.08 | 0.94 | 0.91 | 0.94 | 0.94 | 0.93 | 17.74 |
| HP-HL | 0.07 | 0.95 | 0.92 | 0.95 | 0.95 | 0.95 | 14.26 |
| Vietnam | |||||||
| HC-HL | 0.07 | 0.94 | 0.91 | 0.94 | 0.94 | 0.94 | 11.60 |
| DP-HL | 0.10 | 0.91 | 0.85 | 0.91 | 0.91 | 0.91 | 21.85 |
| HP-HL | 0.07 | 0.94 | 0.92 | 0.95 | 0.95 | 0.94 | 10.69 |
AGFI, adjusted goodness-of-fit index; CFI, comparative fit index; DP-HL, disease prevention health literacy; GFI, goodness-of-fit index; HC-HL, health care health literacy; HLS-EU-Q47, European Health Literacy Survey Questionnaire with 47 items; HP-HL, health promotion health literacy; IFI, incremental fit index; NFI, normal fit index; RMSEA, root mean square error of approximation; χ2/df, relative chi-square.
Four-factor model of each domain included finding, understanding, judging, and applying health information. The model-fit-indices were reported after dropping out certain items from whole HLS-EU-Q scale; e.g. item 4 from Taiwan and Myanmar surveys, item 17, 24 from Vietnam survey, item 4, 15, 21 from Indonesia survey.
3.2. Item-scale convergent validity
Most of these items were shown to have satisfactory item-scale convergent validity (item-scale correlation ≥0.40; Table 3). Item 45, “the ability to join a sports club or exercise class if you wished”, was shown to have a weak correlation (rho = 0.31) with its own scale in Myanmar.
Table 3. Item-scale convergent validity, internal consistency reliability, and floor and ceiling effects of HLS-EU-Q47 in six Asian countries with various sampling populations.
| Indonesia | Kazakhstan | Malaysia | Myanmar | Taiwan | Vietnam | ||
|---|---|---|---|---|---|---|---|
| (n = 1029) | (n = 1845) | (n = 462) | (n = 1600) | (n = 3015) | (n = 2073) | ||
| Item-scale convergent validity,range of correlations (rho) | |||||||
| Gen-HL | 0.42–0.58 | 0.62–0.73 | 0.49–0.68 | 0.44–0.66 | 0.46–0.68 | 0.56–0.67 | |
| HC-HL | 0.49–0.60 | 0.65–0.74 | 0.51–0.69 | 0.51–0.71 | 0.53–0.70 | 0.61–0.73 | |
| DP-HL | 0.52–0.68 | 0.67–0.75 | 0.60–0.72 | 0.57–0.72 | 0.59–0.74 | 0.64–0.75 | |
| HP-HL | 0.52–0.65 | 0.70–0.76 | 0.55–0.74 | 0.41–0.69 | 0.61–0.74 | 0.62–0.73 | |
| Reliability | |||||||
| Cronbach's alpha | Gen-HL | 0.94 | 0.97 | 0.96 | 0.96 | 0.96 | 0.97 |
| HC-HL | 0.85 | 0.93 | 0.88 | 0.88 | 0.89 | 0.92 | |
| DP-HL | 0.88 | 0.94 | 0.91 | 0.91 | 0.91 | 0.92 | |
| HP-HL | 0.88 | 0.94 | 0.91 | 0.90 | 0.92 | 0.92 | |
| Split-half Spearman-Brown coefficient | Gen-HL | 0.87 | 0.92 | 0.90 | 0.88 | 0.87 | 0.89 |
| HC-HL | 0.78 | 0.90 | 0.84 | 0.80 | 0.77 | 0.87 | |
| DP-HL | 0.79 | 0.87 | 0.81 | 0.85 | 0.80 | 0.87 | |
| HP-HL | 0.80 | 0.90 | 0.87 | 0.83 | 0.87 | 0.85 | |
| Floor effects, % | |||||||
| Gen-HL | 0.00 | 0.40 | 0.00 | 0.00 | 0.10 | 0.60 | |
| HC-HL | 0.00 | 0.50 | 0.00 | 0.00 | 0.10 | 1.40 | |
| DP-HL | 0.00 | 0.50 | 0.20 | 0.10 | 0.20 | 1.20 | |
| HP-HL | 0.00 | 0.80 | 0.40 | 0.10 | 0.30 | 1.00 | |
| Ceiling effect, % | |||||||
| Gen-HL | 0.30 | 5.20 | 0.60 | 0.30 | 1.10 | 2.10 | |
| HC-HL | 0.50 | 6.70 | 1.70 | 1.60 | 1.90 | 2.80 | |
| DP-HL | 1.60 | 8.20 | 2.60 | 3.30 | 3.40 | 3.90 | |
| HP-HL | 1.00 | 8.20 | 1.50 | 2.40 | 3.60 | 4.60 | |
DP-HL, disease prevention health literacy; Gen-HL, General Health Literacy; HC-HL, health care health literacy; HLS-EU-Q47, European Health Literacy Survey Questionnaire with 47 items; HP-HL, health promotion health literacy.
3.3. Floor and ceiling effects
There were no significant floor or ceiling effects, as the percentages of people with the lowest scores or the highest scores of health literacy were less than 15%. The percentage of scores at the floor ranged from 0.1% to 0.3% in Taiwan, 0.6%–1.4% in Vietnam, 0.4%–0.8% in Kazakhstan, 0–0.1% in Myanmar, 0 in Indonesia, and 0–0.4% in Malaysia. In addition, the percentage of scores at the ceiling ranged from 1.1% to 3.6% in Taiwan, 2.1%–4.6% in Vietnam, 5.2%–8.2% in Kazakhstan, 0.3%–3.3% in Myanmar, 0.3%–1.6% in Indonesia, and 0.6%–2.6% in Malaysia (Table 3). These results indicated that the responsiveness of HLS-EU-Q47 scale was satisfactory for surveying the general public in these countries.
3.4. Known-group validity
The results from simple linear regression analysis showed that the health literacy scores were significantly different between those with different levels of education (P from <0.001 to 0.011), and with various levels of self-perceived social status (P from <0.001 to 0.016) in these countries (Table 4).
Table 4. Simple linear regression analysis for known-group validity of the HLS-EU-Q47 in six Asian countries.
| b (95% CI) | ||||||
|---|---|---|---|---|---|---|
| Indonesia | Kazakhstan | Malaysia | Myanmar | Taiwan | Vietnam | |
| (n = 1029) | (n = 1845) | (n = 462) | (n = 1600) | (n = 3015) | (n = 2073) | |
| Education attainment | ||||||
| Junior high and below | Reference | Reference | Reference | Reference | Reference | Reference |
| Senior high | 0.41 (−0.43 to 1.24) | 0.97 (−0.46 to 2.39) | 2.33 (0.87–3.80)∗∗ | 2.22 (1.19–3.25)∗∗∗ | 1.12 (0.42–1.83)∗∗ | 1.86 (0.97–2.75)∗∗∗ |
| University and above | 2.95 (1.96–3.93)∗∗∗ | 3.08 (2.04–4.12)∗∗∗ | 2.75 (0.64–4.86)∗ | 2.72 (1.66–3.77)∗∗∗ | 1.36 (0.66–2.06)∗∗∗ | 3.70 (2.69–4.71)∗∗∗ |
| Self-assessed social status | ||||||
| Low | Reference | Reference | Reference | Reference | Reference | Reference |
| Middle | 0.78 (−0.19 to 1.76) | 1.76 (0.73–2.80)∗∗ | 1.99 (0.37–3.62)∗ | 1.79 (0.81–2.77)∗∗∗ | 1.59 (1.10–2.07)∗∗∗ | 1.09 (0.30–1.89)∗∗ |
| High | 2.45 (1.00–3.90)∗∗ | 5.18 (3.83–6.52)∗∗∗ | 2.80 (0.66–4.95)∗ | 2.61 (1.16–4.06)∗∗∗ | 3.59 (2.35–4.83)∗∗∗ | 8.76 (6.90–10.61)∗∗∗ |
b, non-standardized coefficient; CI, confident interval; HLS-EU-Q47, European Health Literacy Survey Questionnaire with 47 items.
P values ∗ 0.01 < P < 0.05, ∗∗ 0.001 < P < 0.01, ∗∗∗P < 0.001.
3.5. Reliability
The reliability of the HLS-EU-Q47 was very high: the internal consistency (Cronbach's alpha) for the 47 items was larger than 0.90 in all countries. Most of the sub-scales had high internal consistency, except HC-HL in Myanmar and Malaysia and three sub-scales in Indonesia, which were at acceptable levels (Cronbach's alpha 0.85 to 0.88). In addition, the split-half Spearman-Brown coefficients ranged from 0.77 to 0.92 and were satisfactory for Gen-HL and three domains (HC-HL, DP-HL, and HP-HL) (Table 3).
4. Discussion
The results showed that the HLS-EU-Q47 was a valid and reliable tool to measure health literacy in selected Asian countries, with satisfactory model-fit indices, evidence of known-group validity, adequate item-scale convergent validity, no apparent floor/ceiling effects, and high levels of internal consistency reliability.
4.1. Construct validity
The HLS-EU-Q47 was validated in six Asian countries, with satisfactory goodness-of-fit indices according to confirmatory factor analyses after omitting certain low-loading items from the whole scale.27 Specifically, some items with very low loading (loading <0.4) on certain domains could be considered less important,30 including item 4 for the survey in Taiwan and Myanmar, items 17 and 24 for the survey in Vietnam, and items 4, 25, and 21 for the survey in Indonesia. All 47 items were useful for surveys in Kazakhstan and Malaysia.
4.2. Item-scale convergent validity
Most items showed satisfactory item-scale convergent validity (item-scale correlation ≥0.40).21 Item 45 in Myanmar showed a weak correlation with its own scale and was suggested to be dropped from the scale to improve the item-scale correlation in Myanmar and the internal consistency reliability of HLS-EU-Q47.
4.3. Known-group validity
The HLS-EU-Q47 was able to distinguish health literacy between levels of education attainment and social status, as it had been shown in previous studies that those with higher education or social status were with better health literacy.12,15 This confirmed the known-group validity of the tool.23 Additionally, the HLS-EU-Q47 can predict the association between health literacy and associated factors, such as education attainment and social status, in these Asian countries, similar to previous studies using subjective measures in European countries1,12,15 and Japan31 and using objective measures like REALM32,33 or Newest Vital Sign (NVS)33 in the United States. This suggested that the HLS-EU-Q47 could be used to compare health literacy and its associated factors among countries with significant cultural variations.
4.4. Floor and ceiling effects
Since the percentages of people who had the lowest and the highest health literacy scores at floor and ceiling were both less than 15%, no significant floor/ceiling effects in the general public in these Asian countries was identified. The results indicated that the HLS-EU-Q47 was able to differentiate individuals with low or high health literacy in different languages, making it a valid tool to measure health literacy in several Asian countries.25,28
4.5. Reliability
The instrument was reliable, with high internal consistencies of HC-HL, DP-HL, and HP-HL similar to those identified in the original HLS-EU survey (Cronbach's alphas 0.87 to 0.97).12 This results represented equivalence and consistency among the responses to items of HLS-EU-Q47 in both Europe and Asia,29 suggesting that these items were homogenous in measuring people's health literacy. The internal consistency reliability was robust, with no floor/ceiling effects.25 The satisfactory split-half Spearman-Brown reliability suggested additional equivalence of the scale, assuring equivalence reliability of the HLS-EU-Q47.11 The HLS-EU-Q47 seemed to be a reliable survey tool in different Asian countries.
4.6. Limitations
The external validity of the tool may be limited by the sampling in this study, which may not have ensured adequate representativeness for the whole countries in Indonesia, Kazakhstan, Malaysia, Myanmar, and Vietnam. In general, proportions of women participants were higher than those of men. These proportions were similar to representative data in each country; in Myanmar the male:female ratio is about 98:100, so there are more females than males.34 Traditionally, males were responsible for earning income and females were responsible for housekeeping. Therefore, when surveys were conducted in households at working times (day time) of working days (Monday to Friday), the proportion of women would be expected to be higher than men. In addition, with limited resources, use of a proportionate sampling method (weighting) was not practical in the current study. We plan to conduct future researches using weighting methods and nationwide sampling in these countries in Asia. Furthermore, the sample size was relatively small in Malaysia. This might affect the extrapolation of results from the survey. Therefore, the results of this study should not be interpreted as results from national representative sampling. However, the multistage sampling method to select the participants had been applied across all countries to support the rigor of the study. Although the study provided evidences for potential use of the comprehensive health literacy survey tool (HLS-EU-Q47) in Asia, we could not examine the test-retest reliability due to the cross-sectional design of the present study, and we were not able to assess convergent validity. Further psychometric testing is needed to explore the wider use of the HLS-EU-Q47 tool in different populations and with different study designs.
4.7. Implications
Health literacy is a dynamic product of the interactions between individuals, patients, employees, organizations, and systems.35 On one hand, health literacy is influenced by aspects of the health care system at the population level3; which reflects the enormous complexity in the delivery of effective healthcare and quality health outcomes.3,36 Better health literacy enables health-friendly environments, efficient health policies, effective health promotional efforts, better self-care with fewer health risks, and is associated with better health care outcomes and lower health costs.4,37 The HLS-EU-Q47 is therefore shown to be a valid tool to provide comprehensive measurement.
It is suggested that researchers follow guidelines while translating, adapting, and validating the HLS-EU-Q47 for use in other countries in Asia, with cultural competence considered.19 Moreover, guidelines suggest using the same tool and sampling method to provide comparable results and to identify associated factors with health literacy between countries19; however, developed as a research tool rather than a screening tool, the HLS-EU-Q47 was relatively long and demanded substantial resources for data collection.
5. Conclusions
This study was the first to investigate the psychometric properties of HLS-EU-Q47 in Asia. Based on the results, the HLS-EU-Q47, which can serve as a comprehensive health literacy survey tool, was found to be a reliable, valid tool in several countries in Asia. We suggest using the same tool to assess health literacy in different countries and provide practical international comparison in the future.
Contributors
TVD conducted data management and statistical analysis, TTS, KS, JMP, SVDB provided study design and statistical analysis advices substantially, AA, GB, NN, TVP, KMP, TQT, KTN, WMO, EM, and HLH provided data collection support, PWC conducted the research design and the overall supervision of the manuscript. All authors approve the manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately clarified.
Funding
This study was partly supported in part by Taiwan's Ministry of Science and Technology and the Health Promotion Administration. The MJ Health Research Foundation had supported significantly on the survey. The funding agencies had no role in data collection, analysis, interpretation, or the decision to submit the results.
Conflicts of interest
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: the study was supported by Taiwan's Ministry of Science, Technology and Health Promotion Administration, and the MJ Health Research Foundation, as well as the Asian Health Literacy Association; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Transparency declaration
The lead authors (study guarantors) affirm that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
Data sharing
The data in this manuscript is a part of a nationwide dataset. We are planning on producing publications using this dataset. Afterwards, the technical appendix, statistical code, and dataset will be available from the corresponding author.
Acknowledgements
The authors would thank to all Asian Health Literacy Association partner institutions for collaborating and collecting data collection. Especially, the authors thank Prof. Chin-Fen Chen, Prof. Albert Li, Prof. Jung-De Wang, Prof. Kuo-Ying Wang, Dr. Ellen Hsu, Dr. Cheng-Fung Lu, Dr. Thung-Chiao Tseng, Yun-lu Chen, Betty Hsien-Ping Tung, Prof. Nguyen Thu Ha, Prof. Nguyen Thu Huong, Dr. Hoang Thi Giang, Dr. Kay Thi Lwin, Dr. Pa Pa Soe, Associate Prof. Dr. Hazreen Majid, Ms. Reena A/P Balan, and Associate Prof. Karuthan Chinna for help with data collection.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.je.2016.09.005.
Footnotes
Peer review under responsibility of the Japan Epidemiological Association.
References
- 1.Watson R. Europeans with poor “health literacy” are heavy users of health services. BMJ. 2011;343:d7741. [DOI] [PubMed] [Google Scholar]
- 2.Greenhalgh T. Health literacy: towards system level solutions. BMJ. 2015;350:h1026. [DOI] [PubMed] [Google Scholar]
- 3.Sørensen K, Van den Broucke S, Brand H, et al. . Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. 2012;12:80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ishikawa H, Yano E. Patient health literacy and participation in the healthcare process. Health Expect. 2008;11:113–122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Raynor DK. Health literacy. BMJ. 2012;344:e2188. [DOI] [PubMed] [Google Scholar]
- 6.Parker RM, Baker DW, Williams MV, Nurss JR. The Test of Functional Health Literacy in Adults (TOFELA): a new instrument for measuring patient's literacy skills. J Gen Intern Med. 1995;10:537–542. [DOI] [PubMed] [Google Scholar]
- 7.Davis T, Long S, Jackson R. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993;25:391–395. [PubMed] [Google Scholar]
- 8.Weiss BD, Mays MZ, Martz W, et al. . Quick assessment of literacy in primary care: the newest vital Sign. Ann Fam Med. 2005;3:514–522. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ishikawa H, Takeuchi T, Yano E. Measuring functional, communicative, and critical health literacy among diabetic patients. Diabetes care. 2008;31:874–879. [DOI] [PubMed] [Google Scholar]
- 10.Wang X, Guo H, Wang L, et al. . Investigation of residents' health literacy status and its risk factors in jiangsu province of China. Asia Pac J Public Health. 2013;27 NP2764–NP72. [DOI] [PubMed] [Google Scholar]
- 11.Tsai T-I, Lee S-YD, Tsai Y-W, Kuo KN. Methodology and validation of health literacy scale development in Taiwan. J Health Commun. 2010;16:50–61. [DOI] [PubMed] [Google Scholar]
- 12.HLS-EU Consortium Comparative Report of Health Literacy in Eight EU Member States. The European Health Literacy Project 2009–2012 Maastricht University; 2012. Retrieved from http://www.maastrichtuniversity.nl/web/file?uuid=d101b63c-dbbe-472d-971f-7a4eae14ba47&owner=d5b3681e-fc4a-476e-b9ff-a807c26760b9.
- 13.Sørensen K, Van den Broucke S, Pelikan J, et al. . Measuring health literacy in populations: illuminating the design and development process of the European Health Literacy Survey Questionnaire (HLS-EU-Q). BMC Public Health. 2013;13:948. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Nguyen TH, Park H, Han H-R, et al. . State of the science of health literacy measures: validity implications for minority populations. Patient Educ Couns. 2015;98:1492–1512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Sørensen K, Pelikan JM, Röthlin F, Ganahl K, Slonska Z, Doyle G, et al. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health:ckv043 (Published online first, 5 April 2015). http://dx.doi.org/10.1093/eurpub/ckv043. [DOI] [PMC free article] [PubMed]
- 16.Duong VT, Lin I-F, Sorensen K, et al. . Health literacy in Taiwan: a population-based study. Asia Pac J Public Health. 2015;27:871–880. [DOI] [PubMed] [Google Scholar]
- 17.Hazreen MA, Su TT, Jalaludin MY, et al. . An exploratory study on risk factors for chronic non-communicable diseases among adolescents in Malaysia: overview of the Malaysian Health and Adolescents Longitudinal Research Team study (The MyHeART study). BMC Public Health. 2014;14:S6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Oo WM, Soe PP, Lwin KT. Status and determinants of health literacy: a study among adult population in selected areas of Myanmar. Int J Res Med Sci. 2015;2:318–322. [Google Scholar]
- 19.Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract. 2011;17:268–274. [DOI] [PubMed] [Google Scholar]
- 20.Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess. 1995;7:286–299. [Google Scholar]
- 21.Hays RD, Hayashi T. Beyond internal consistency reliability: rationale and user's guide for multitrait analysis program on the microcomputer. Behav Res Meth Ins C. 1990;22:167–175. [Google Scholar]
- 22.Taylor R. Interpretation of the correlation coefficient: a basic review. J Diagn Med Sonogr. 1990;6:35–39. [Google Scholar]
- 23.Hays RD, Anderson RT, Revicki D. Assessing Reliability and Validity of Measurement in Clinical Trials. Quality of Life Assessment in Clinical Trials: Methods and Practice New York: Oxford University Press; 1998:169–182. [Google Scholar]
- 24.Cronbach LJ, Shavelson RJ. My current thoughts on coefficient alpha and successor procedures. Educ Psychol Meas. 2004;64:391–418. [Google Scholar]
- 25.Terwee CB, Bot SDM, de Boer MR, et al. . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42. [DOI] [PubMed] [Google Scholar]
- 26.IBM SPSS IBM SPSS Statistics for Windows, Version 20.0. New York: IBM Corp; 2011. [Google Scholar]
- 27.Kline RB. Principles and Practices of Structural Equation Modeling. third ed. New York, NY: The Guilford Press; 2011. [Google Scholar]
- 28.Lim CR, Harris K, Dawson J, Beard DJ, Fitzpatrick R, Price AJ. Floor and ceiling effects in the OHS: an analysis of the NHS PROMs data set. BMJ Open. 2015;5:e007765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Hooper D, Coughlan J, Mullen M. Structural equation modelling: guidelines for determining model fit. Electron J Bus Res Method. 2008;6:53–60. [Google Scholar]
- 31.Nakayama K, Osaka W, Togari T, et al. . Comprehensive health literacy in Japan is lower than in Europe: a validated Japanese-language assessment of health literacy. BMC Public Health. 2015;15:505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Haun J, Luther S, Dodd V, Donaldson P. Measurement variation across health literacy assessments: implications for assessment selection in research and practice. J Health Commun. 2012;17:141–159. [DOI] [PubMed] [Google Scholar]
- 33.Wolf MS, Curtis LM, Wilson EAH, et al. . Literacy, cognitive function, and health: results of the LitCog study. J Gen Intern Med. 2012;27:1300–1307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Department of Population The 2014 Myanmar Population and Housing Census. Nay-pyi-taw: Ministry of Immigration and Population, the Republic of the Union of Myanmar; 2015. Retrieved from http://unstats.un.org/unsd/demographic/sources/census/2010_phc/Myanmar/MMR-2015-05.pdf.
- 35.Naccarella L, Wraight B, Gorman D. Is health workforce planning recognising the dynamic interplay between health literacy at an individual, organisation and system level? Aust Health Rev. 2016 Feb;40(1):33–35 (Published online first, 30 June 2015). [DOI] [PubMed] [Google Scholar]
- 36.Rasu RS, Bawa WA, Suminski R, Snella K, Warady B. Health literacy impact on national healthcare utilization and expenditure. Int J Health Policy Manag. 2015;4:747–755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.İlgün G, Turaç İS, Orak S. Health literacy. Procedia Soc Behav Sci. 2015;174:2629–2633. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.je.2016.09.005.