Skip to main content
BMJ Open logoLink to BMJ Open
. 2020 Oct 5;10(10):e035987. doi: 10.1136/bmjopen-2019-035987

Measuring health literacy combining performance-based and self-assessed measures: the roles of age, educational level and financial resources in predicting health literacy skills. A cross-sectional study conducted in Florence (Italy)

Chiara Lorini 1, Vieri Lastrucci 1,, Diana Paolini 1, Guglielmo Bonaccorsi 1; Florence Health Literacy Research Group
PMCID: PMC7537461  PMID: 33020080

Abstract

Objective

The objective was to compare the results of performance-based and self-assessed measures of health literacy (HL) and to evaluate the contribution of their joint use in assessing some HL antecedents.

Design

This was a cross-sectional study.

Setting

The study was conducted on the general population in Florence (Italy).

Participants

This study is part of a larger one, where participants were randomly selected from the registries of 11 general practitioners working in the municipality of Florence. Inclusion criteria were the following: 18–69 years of age and Italian speaking. Exclusion criteria included cognitive impairment, severe psychiatric disease or end-stage disease. In this paper, 212 adults were included.

Outcome measures

HL was measured using the European Health Literacy Survey Questionnaire (HLS-EU-Q16) and the Newest Vital Sign (NVS). The HL levels obtained by means of the two measurement tools were combined into a new variable that described three different levels of HL skills: low HL skills, partial HL skills and high HL skills. Multivariate ordinal logistic regression analysis was performed to assess the predictive roles of age class, educational level and financial resources with respect to HL skills.

Results

Twenty-two per cent of the sample had high HL skills, 28.3% had low HL skills and 49.5% had partial HL skills. Educational level, age class and financial resources were significantly associated with HL skills, with OR values being higher than those obtained using the NVS or the HLS-EU-Q16 individually.

Conclusion

The combination of the results obtained using the NVS and the HLS-EU-Q16 improves the understanding of HL. The new variable generated by this combination could be considered as a different way to assess HL and its multidimensional contents.

Keywords: public health, preventive medicine, statistics & research methods


Strengths and limitations of this study.

  • In this study, for the first time, two different measures of health literacy (HL) were combined into a new measure, called ‘HL skills’.

  • The study design (sampling procedure and criteria for the combination of the HL measures) led to limitations in the generalisability of the results.

  • A different approach in combining the two measures could have led to different results.

Introduction

Health literacy (HL) is a multidimensional concept1 that deals with broader competences that are needed to communicate, navigate and actively participate within modern healthcare systems and, more generally, with an individual’s capacity to assess, understand and use health information in different settings.2 3 The skills that compose HL can be classified into three different typologies: the practical application of literacy skills ranging from those needed to be able to function effectively in everyday situations (functional); the cognitive and literacy skills that can be used to actively participate in everyday activities and to apply new information to changing circumstances (interactive); and cognitive skills that can be applied to critically analyse information and exert greater control over life events and situations (critical literacy).4 All of these competences enable a person to navigate within three domains: healthcare, disease prevention and health promotion.2 For these reasons, HL affects people’s health, and it is now considered as one of the main determinants of health inequality; it is significantly related to age, educational level and economic status5–8 and is suggested to partially mediate the effect of socioeconomic status on health-related outcomes.9–11 Moreover, HL can also be considered as the balance between individual skills and the demands and complexities of societal systems12; it is the combination of cognitive capacities, life experiences, knowledge and opportunities.13 14

To date, several different definitions of HL have been proposed in the literature; as a result, a considerable number of HL measurement tools have been developed by now. Although this variety of measures permits the use of specific tools for specific aims and target groups, it raises debate and poses some challenges. Indeed, more than 150 measures exist, but no ‘gold standard’ measure has emerged until now. Furthermore, only a small number of instruments examine multiple types of HL (functional, interactive and critical), while the majority deal solely with the functional component, with the risk of fragmentation. Apart from that, measurement tools may be classified as either performance-based (objective) or self-assessed (subjective), as they capture different aspects, for example, the objective ability to understand medical information versus the effect of emotional or motivational aspects on the decision-making process.15–18 As a consequence of the lack of a comprehensive approach to HL measurement, the use of different or fragmented HL measures leads to difficulties in comparing and/or to incomplete results in terms of the HL level and related outcomes, as well as to an increasing risk of misinterpreting the effectiveness of interventions aimed at improving HL.19–24 Besides, while performance-based tools can be assumed to objectively measure HL regardless of a person’s environment, self-assessed ones can be considered to be more situation specific; for instance, emotional or motivational aspects of the decision-making process are also the consequence of family, community and system support.13

For these reasons, many authors suggest measuring HL using different instruments at the same time, so as to assess different skills, abilities and competences that constitute such a multidimensional construct.15–25 Nevertheless, research using performance-based (ie, direct testing of competences) and self-assessed (perception based, ie, self-reported abilities) measures of different dimensions and types of HL simultaneously remains scarce, and the results of such studies are usually focused on highlighting the inconsistencies between the two types of tests, without assessing their potential joint contribution to measuring HL as a unique concept.14 19 26 27

In a study conducted on patients affected by diabetes or colon cancer, Waters et al 28 found that performance-based and self-assessed HL measures represent related but independent constructs; they are able to predict objective disease knowledge but not perceived disease knowledge in the same way. Due to these results, the author concluded that performance-based and self-assessed measures of HL are not interchangeable, although they tend to be consistent in categorising patients into different levels of HL.28 To the best of our knowledge, no studies adopting a similar approach to the analysis of the HL determinants have been published as yet.

The aim of this study is to compare the results of performance-based and self-assessed measures of HL and to evaluate the potential contribution of their joint use in assessing some HL antecedents (age and socioeconomical determinants) in a population-based sample. We believe that this is the first attempt to use the information obtained by different HL measurement tools to get further insight into the knowledge about the antecedents of HL.

Methods

This study is part of a larger one, conducted in a population-based sample in Florence, Italy, with the aim of measuring the HL level and validating some HL measurement tools. The study design is described elsewhere,29 as well as some of its results.30

Data collection

The study adopted a cross-sectional design that was carried out in a population-based sample. Participants were randomly selected from the registries of 11 general practitioners (GPs) working in the municipality of Florence. The municipality of Florence is about 102 km2 in size, with a population density of about 3500 inhabitants/km2; socioeconomic and health deprivation data are described elsewhere.31

The sample size of the study was calculated considering the first aim of the larger study (ie, to assess the level of functional HL using the Newest Vital Sign (NVS) in a population sample in Florence, Italy), as described elsewhere,7 and it was equal to 480 participants.

The GPs were recruited using convenience criteria: all of the GPs from the municipality of Florence were invited to join the study by both the Provincial Medical Council and the University Hospital of Florence. According to the study protocol, the first eight who voluntarily joined the study were included and were asked to select 80 subjects among those registered as patients using a simple random sampling method. Since oversampling was not enough to reach the sample size of 480, three more GPs were included, with a second random sample for the first eight. In Italy, every resident over the age of 18 years has to be registered in a general practice, and people are enrolled in the general practices according to their place of residence. This sampling method was chosen with the aim of increasing the population participation rate, as the invitation letter was jointly signed by the GPs and the researcher in charge of the study.

The sample was selected within each neighbourhood of the municipality of Florence, since the recruited general practices were based in all the areas of Florence.

The inclusion criteria were the following: 18–69 years of age and Italian speaking (since the survey was conducted in Italian). The inclusion criteria were defined according to those of the Italian behavioural risk factor surveillance system Progressi delle Aziende Sanitarie per la Salute in Italia (PASSI).32 The exclusion criteria included cognitive impairment, severe psychiatric disease or end-stage diseases. Each GP verified the inclusion and exclusion criteria when selecting the sample.

The larger study included two different arms (A and B) with different aims and questionnaires. Each subject was randomly allocated to one of the two arms. To meet the specific aims of the present study, only the B arm of the research was considered, since the short form (16 items) of the European Health Literacy Survey Questionnaire (HLS-EU-Q16) was only administered in this arm, together with the Italian version of the Newest Vital Sign (NVS-IT, hereinafter, NVS).

Data collection started in February 2017 and finished on 31 December 2017. Each selected subject was contacted via postal mail. Subjects received an information sheet signed by the GP and the person in-charge of the study, which included a short description of the study, an invitation to participate and a consent form. Participants were asked to sign the consent form and return it via mail to the researchers in charge. The mail also contained the nutritional label of the NVS. After receipt of the signed consent forms, the subjects were contacted for a computer-assisted telephone interview. If the consent form was not received within 2 weeks, a follow-up phone call was made by the research group. The phone call served to clarify any questions and to identify and support people having difficulty completing the consent form (ie, due to reading difficulty). Nine interviewers made the phone calls. Written instructions on how to conduct the interviews were drawn up and shared to standardise the procedure and limit interviewer bias. Each subject was randomly assigned to one of the nine interviewers and contacted a maximum of six times before being considered unreachable.

The questionnaire had a general section that included questions on sociodemographic, familial data (antecedents) and health-related outcomes (consequences), as described in the previous papers.29 In addition, the questionnaire included the NVS and the HLS-EU-Q16.

Age was collected as a continuous variable and then grouped into four classes (18–45; 46–55; 56–65; and >65 years old). Education was classified into three levels (less then high school diploma, high school degree and bachelor’s degree and higher), while the financial status was investigated by the item ‘Is your income adequate to meet monthly living expenses?’ with four possible response options (not enough, barely enough; enough and more than enough). This item was chosen since it is routinely used in the standardised questionnaire of the Italian behavioural risk factor surveillance system PASSI.33

HL measures

HL was measured using the NVS and the HLS-EU-Q16. The Italian version of the NVS was validated by Capecchi et al 34 from the UK version and then it was applied in many different contexts.35 36 It consists of an ice cream nutrition label with seven associated questions that measure functional HL (prose and numeracy) using a performance-based approach. It produces a final score ranging from 0 to 6, allowing participants to be classified into three categories—high likelihood of limited HL (score: 0–1), possibility of limited HL (score: 2–3) and adequate HL (score: 4–6). These cut-off values were identified by Weiss et al 37 in a validation study of the NVS, conducted in English-speaking and Spanish-speaking primary care patients, in which the HL measured using the Test of Functional Health Literacy in Adults was considered as a reference. The HL categories defined using the two cut-off values (1 and 3) are widely used in many countries.

NVS data related to the entire sample of the study (A and B arms) have been described elsewhere.7

The European Health Literacy Survey was the first large population study aimed at generating first-time data on HL across diverse populations in the European Union.38 To achieve this purpose, the HLS-EU-Q for measuring HL was developed39 on the basis of the recommendations of Pleasant et al 24 regarding the characteristics that a comprehensive measure of HL should have. In particular, starting from the HLS-EU Consortium conceptual framework of HL,2 the HLS-EU-Q assesses the processes of accessing, understanding, appraising and applying health-related information within the three domains of health: healthcare, disease prevention and health promotion. It measures self-assessed functional, critical and interactive HL (ie, general HL). The original full version of the HLS-EU-Q comprises 47 items (HLS-EU-Q47), and the HLS-EU-Q16 is its short version that was developed by selecting 16 items.27 The HLS-EU-Q16 has Likert-type responses (‘very easy’, ‘fairly easy’, ‘fairly difficult’ and ‘very difficult’) and an associated final score that measures interaction, comprehension, information seeking, application/function, decision making/critical thinking, evaluation, responsibility, confidence and navigation skills. To generate the score of the HLS-EU-Q16, the items are dichotomised into two categories with two scores: ‘easy’ (‘fairly’ or ‘very’ easy=1) and ‘difficult’ (‘fairly’ and ‘very’ difficult=0). ‘Don’t know/refusal’ was recoded for missing answers. The scale score was calculated as the sum of the scores of each item and varied between 0 and 16. As suggested by other studies,26 40 only respondents who gave an answer to at least 14 items were considered. Three levels of HL were defined considering the HLS-EU-Q16 score: inadequate HL (0–8), problematic HL (9–12) and sufficient HL (13–16). The cut-off values for defining the three levels were described by Pelikan and Ganahl41 using the results of the European Health Literacy Survey, with respect to the results obtained using the HLS-EU-Q47 and then have been widely used.

As previously described, the Italian version of the HLS-EU-Q16 was validated in this study.30

Statistical analysis

The Fisher’s exact test was used to evaluate associations between categorical variables.

A new HL measure, named ‘HL skills’, was defined by combining the results obtained using the two tests (HLS-EU-Q16 and NVS). The criterion used for combining the two measures was a simple approach that allowed to different levels of skills to be identified. In particular, the variable was created as follows:

  1. ‘Low HL skills’ level that comprises a high likelihood or possibility of limited HL measured by NVS and inadequate or problematic HL measured by HLS-EU-Q16.

  2. ‘Partial HL skills’ level that comprises a high likelihood or possibility of limited HL measured by NVS and sufficient HL measured by HLS-EU-Q16 or, conversely, adequate HL measured by NVS and inadequate or problematic HL according to HLS-EU-Q16.

  3. ‘High HL skills’ level that comprises adequate HL measured by NVS and sufficient HL measured by HLS-EU-Q16.

The subjects classified among those with ‘low HL skills’ presented some limitations in both functional and general HL; those with ‘partial HL skills’ presented some limitation either in functional or in general HL, while those with ‘high HL skills’ presented the highest level of HL skills in both functional and general HL.

A multivariate ordinal logistic regression analysis42 was performed to assess the predictive roles of age class, educational level and financial resources with respect to ‘HL skills’. Specifically, ‘HL skills’ was the dependent ordinal variable while age class, educational level and financial resources were the independent ordinal variables (covariates). In ordinal logistic regression model, the predictive role is expressed as the proportional OR, and it can be interpreted in the same way as ORs are interpreted for the conventional logistic regression for binary outcomes. The OR obtained from this model was a measure of the change in the odds from lower to higher levels, that is, from lower to higher HL skills. As a comparison, the same multivariate ordinal logistic regression analysis was applied considering the level of HL measured by each single HL test (ie, NVS and HLS-EU-Q16) as a dependent ordinal variable. Specifically, two models were developed: in the first one, the NVS level was the dependent variable (1: high likelihood of limited HL; 2: possibility of limited HL; 3: adequate HL), while in the second one, the HLS-EU-Q16 level was the dependent variable (1: inadequate HL; 2: problematic HL; 3: sufficient HL).

Statistical analyses were conducted using Stata V.15. All tests were two sided, and p values were considered to be statistically significant when below 0.05.

Patient and public involvement

The study population was not directly involved in the design, recruitment and conduct of this study. However, the Florence Health Literacy Research Group involved representatives from the Provincial Medical Council, Local Health Unit and University Hospital of Florence. All of these representatives were involved in the study design and questionnaire development and will disseminate the results from this work.

Results

The refusal rate was 15.6%, while 38.2% of the invited people did not respond to any contact attempts and were considered unreachable. Finally, 212 subjects (58% females; mean age: 53.6±11.9 years) were interviewed for the purpose of this study. The majority of them (96.7%) were Italian, with a high school (36.3%) or university (45.3%) degree and had enough or more than enough financial resources at their disposal from their own or family income to get to the end of the month (68.3%) (table 1).

Table 1.

Descriptive analysis of the collected data (n=212)

Variables N %
Age class
 18–45 49 23.1
 46–55 53 25.0
 56–65 73 34.4
 >65 37 17.5
Educational level
 Less than high school diploma 39 18.4
 High school degree 77 36.3
 Bachelor’s degree and higher 96 45.3
Financial resources at disposal from own or family income enough to get to the end of the month*
 Not enough 10 4.7
 Barely enough 56 26.4
 Enough 108 50.9
 More than enough 37 17.4
NVS level
 High likelihood of limited HL 22 10.4
 Possibility of limited HL 61 28.8
 Adequate HL 129 60.8
HLS-EU-Q16 levels
 Inadequate HL 25 11.8
 Problematic HL 117 55.2
 Sufficient HL 70 33.0

*1 missing value.

HL, health literacy; HLS-EU-Q16, European Health Literacy Survey Questionnaire; NVS, Newest Vital Sign.

According to the HLS-EU-Q16, 11.8% had inadequate, 55.2% had problematic and 33% had sufficient HL; considering the NVS, 10.4% had a high likelihood of having limited HL, 28.8% had a possibility of having limited HL and 60.8% had adequate HL (table 1).

As for NVS, the HL levels were significantly (p<0.05) associated with age class, educational level and financial resources, while when measured by HLS-EU-Q16, the HL levels were significantly (p<0.05) associated only with education. The percentage of people with low HL was higher when it was measured by HLS-EU-Q16 than for NVS in each category of age class, educational level and financial resources (figure 1). For both measures, the percentage of people with low HL increased with age and became more similar in older people: for HLS-EU-Q16, it ranged from 59.2% for those aged 18–45 years to 78.4% for those >65 years old; for NVS, it ranged from 20% for those aged 18–45 years to 67.5% for those aged >65 years old. Similar results were observed for educational level and financial resources: for both tests, the percentage of people with low HL increased with a decrease in educational level or financial resources; in the lowest subcategories (ie, less than high school diploma or not having enough financial resources), the percentage of people with low HL became similar between the two tests.

Figure 1.

Figure 1

Percentage of people with low HL by age class (A), educational level (B) and financial resources (C). HL, health literacy; HLS-EU-Q16, European Health Literacy Survey Questionnaire.

Combining the classifications of both tests (table 2), 22% of the sample had adequate levels of functional (measured by NVS) and sufficient general HL (measured by HLS-EU-Q16). However, 28.3% presented both low functional HL (high likelihood or possibility of limited HL measured by NVS) and low general HL (inadequate or problematic HL according to HLS-EU-Q16). However, a greater part of the sample (49.5%) presented inconsistent HL measurements with low functional HL and sufficient general HL or vice versa. In particular, the percentage of participants with adequate functional HL and low general HL (38.7%) was higher than the percentage of participants with low functional HL and sufficient general HL (10.8%).

Table 2.

Level of health literacy considering both measures (NVS and HLS-EU-Q16)

NVS (n (%)) Total (n (%))
High likelihood or possibility of limited HL Adequate HL
HLS-EU-Q16
 Inadequate or problematic HL 60 (28.3) 82 (38.7) 142 (67)
 Sufficient HL 23 (10.8) 47 (22.2) 70 (33)
Total 83 (39.1) 129 (60.9) 212 (100)

HL, health literacy; HLS-EU-Q16, European Health Literacy Survey Questionnaire; NVS, Newest Vital Sign.

The classification of the subjects into four HL groups (combining the two HL measures) was significantly associated with age class, educational level and financial resources (figure 2; online supplementary table S1). With an increase in age, the percentage of people with adequate HL for NVS and sufficient HL for HLS-EU-Q16 decreased; the percentage was similar between the 18–45 and 46–55 year-old age groups (about 30%), it halved in the 56–65 year-old age group and it halved again in the over 65-year-old age group. A similar tendency, although less markedly evident, was observed for those with sufficient HL for HLS-EU-Q16 and low HL for NVS. At the same time, the percentage of subjects with low HL for both tests increased with increasing age, ranging from 12.2% in the youngest age group to 54.1% in the oldest age group. Regarding education, with an increase in the education level, there was a decrease in the percentage of people with low HL in both tests. The highest percentage of subjects with adequate HL at NVS and sufficient HL at HLS-EU-Q16 was in the bachelor’s degree and higher group (35.4%), while the lowest percentage was registered in the high school degree group (10.4%); the latter education group also presented the lowest percentage of people with sufficient HL at HLS-EU-Q16 and low HL at NVS (9.1%) and the highest percentage of those with low HL at HLS-EU-Q16 and adequate HL at NVS (46.8%). Moreover, with the increase in the availability of financial resources, the percentage of people with adequate HL at NVS and sufficient HL at HLS-EU-Q16 increased and, at the same time, the percentage of people with low HL in both tests decreased; in particular, the percentage of people with low HL in both tests halved, moving from the category ‘not enough’ to ‘barely enough’ (from 70% to 32.1%). Finally, in the more ‘disadvantaged’ groups (elderly people, low educational level, not enough availability of financial resources), the percentage of people with discordant results regarding the HL level (ie, low functional HL and sufficient general HL or vice versa) was lower than that obtained in the other groups.

Figure 2.

Figure 2

Percentage of people with regards to HLS (HLS-EU-Q16) and NVS by age class (A), educational level (B) and financial resources (C). For each graph, p<0.05 (Fisher’s exact test). HLS-EU-Q16, European Health Literacy Survey Questionnaire; NVS, Newest Vital Sign.

Supplementary data

bmjopen-2019-035987supp001.pdf (181.9KB, pdf)

Considering the results of the multivariate ordinal regression model, all categories of the covariates showed significant associations with the outcome, with the exception of ‘high school degree’, with an evident trend. Moreover, OR values were greater than three in most cases (table 3). In particular, the OR of having high HL skills were higher as age decreased (OR value from 2.36 for 56–65 years old to 5.14 for 18–45 years old), financial resources increased (OR value from five for ‘barely enough’ resources to 8.65 for ‘more than enough’ resources) and for those with a bachelor’s degree or higher (OR=3.72). Table 4 reported the results of the same analyses conducted considering the level of HL as a dependent variable in accordance with the NVS (I model) and the HLS-EU-Q16 (II model). Considering the first model, functional HL significantly increased as age decreased (for those 46–55 years old: OR=5.84; for those 18–45 years old: OR=7.17) and for people with a bachelor’s degree or higher (OR=3.78), while financial resources did not show a predictive role. Regarding the second model, age class, educational level and financial resources were not significantly associated with general HL.

Table 3.

Multivariate ordinal logistic regression model (n=211)

Variables OR (95% CI) P>z
Age class
 >65 1
 56–65 2.36 (1.05 to 5.33) 0.038
 46–55 4.85 (2.01 to 11.71) <0.001
 18–45 5.14 (2.10 to 12.54) <0.001
Educational level
 Less than high school diploma 1
 High school degree 1.33 (0.59 to 3.02) 0.486
 Bachelor’s degree and higher 3.72 (1.64 to 8.44) 0.002
Financial resources at disposal from own or family income enough to get to the end of the month
 Not enough 1
 Barely enough 5.50 (1.19 to 25.36) 0.029
 Enough 5.57 (1.26 to 24.54) 0.023
 More than enough 8.65 (1.79 to 41.73) 0.007

LR χ2(10)=51.38; prob>χ2<0.001; log likelihood=–193.35519; pseudo R2=0.1173.

Dependent variable: HL skills (‘low HL skills’, ‘partial HL skills’ and ‘high HL skills’) obtained combing the results of the two HL measures (HLS-EU-Q16 and NVS)

Table 4.

Multivariate ordinal logistic regression models (n=211)

Variables I model: NVS as dependent variable* II model: HLS-EU-Q16 as dependent variable†
OR (95% CI) P>z OR (95% CI) P>z
Age class
 >65 1 1
 56–65 2.13 (0.96 to 4.70) 0.060 1.45 (0.66 to 3.18) 0.357
 46–55 5.84 (2.33 to 14.65) <0.001 1.60 (0.69 to 3.73) 0.271
 18–45 7.17 (2.70 to 19.04) <0.001 1.95 (0.83 to 4.61) 0.126
Educational level
 Less than high school diploma 1 1
 High school degree 1.90 (0.86 to 4.17) 0.110 0.65 (0.30 to 1.42) 0.285
 Bachelor’s degree and higher 3.78 (1.70 to 8.42) 0.001 1.31 (0.62.83) 0.493
Financial resources at disposal from own or family income enough to get to the end of the month
 Not enough 1 1
 Barely enough 1.76 (0.49 to 6.37) 0.386 2.31 (0.61 to 8.69) 0.215
 Enough 3.40 (0.97 to 11.94) 0.057 1.87 (0.52 to 6.71) 0.336
 More than enough 2.91 (0.73 to 11.64) 0.131 3.91 (0.98 to 15.60) 0.054

In the first model, dependent variable: NVS (three levels); in the second model, dependent variable: HLS-EU-Q16 (three levels)

*LR χ2(10)=50.6; prob>χ2<0.001; log likelihood=–163.36457; pseudo R2=0.1341.

†LR χ2(10)=15.64; rob>χ2=0.0479; log likelihood=–192.14072; pseudo R2=0.0391.

HLS-EU-Q16, European Health Literacy Survey Questionnaire.

Discussion

The aim of the study was to compare two different measures of HL and to evaluate the potential contribution of their joint use in assessing HL antecedents in a population-based sample. Our results showed that NVS and HLS-EU-Q16 led to results that did not completely overlap, as a relevant proportion of the population presented different HL levels when measured with different tools. Furthermore, the antecedents of HL investigated in this study have different weights in predicting NVS or HLS-EU-Q16 results. These results indicate that they measure different aspects of HL; these findings are in line with other studies conducted in other countries.14 43

A possible explanation for these findings may lay in the nature of the two HL measurement tools, as the HLS-EU-Q16 is a self-assessed measure for general HL, while NVS is a performance-based measure of reading, understanding and numeracy skills. In fact, what people think they know does not always correspond to what they actually know: people tend to be overconfident (they think they know more than they actually do) or underconfident (they think they know less than they actually do). Overconfidence and underconfidence are a consequence of the matching between knowledge, confidence, self-efficacy and emotional distress,43–47 and they may differ from country to country, as they are also influenced by cultural factors.48 49

However, high skills in reading and understanding health-related information (functional HL) do not necessarily imply high critical and interactive competencies (included in general HL), as these are also related to problem-solving skills, life experiences and empowerment.50 51 In fact, HL could also act as a balance between individual skills and the demands and complexities of societal systems.12 Since it represents the combination of cognitive capacities, life experiences, knowledge and opportunities,13 14 it can be influenced by the social environment in which it is assessed; this feature should and could be considered to tailor interventions aimed at increasing its levels.

The simultaneous use of the two HL measures highlights the presence of three distinct HL groups in the population. A first group is represented by the participants with an adequate level of functional and sufficient general HL; this group has a broader range of HL skills (high HL skills) that can be used to participate actively in everyday situations, extract health information and derive meaning from different forms of health communication. This can be applied to changing circumstances, to exert control over their care and so on.52 53 A second group is represented by the participants that presented both low functional HL and low general HL. These subjects lack a wide range of HL skills (low HL skills). Lastly, between these two opposing conditions, a third group (partial HL skills) is represented by about half of the sample and includes all the participants that presented inconsistent HL measurements with low functional HL and sufficient general HL or vice versa; these people have some HL skills but lack others.

As far as the demographic and socioeconomic characteristics of the HL groups defined by the two HL measures are concerned, it is interesting to note that the more vulnerable population groups (the older, less educated and poorer) presented lower levels of discrepancy in the results obtained with the two HL tools, and in most of cases, these groups presented a low HL level for both measures. On the contrary, the youngest participants (18–45 years), those with a high school degree and those with enough financial resources presented the highest percentage of people with partial HL skills, low functional HL, and sufficient general HL in most cases. There seems to be a social gradient in accessing, understanding, appraising and applying information that is useful for adopting appropriate behaviours in everyday life, and in this sense, HL reflects the disadvantage suffered by the most deprived people regarding education and wealth.

Regarding the combination of the results obtained by applying the two HL measures into a new variable, that is, HL skills, findings showed that the new variable strengthens the association between HL and the investigated antecedents. Indeed, the comparison of the three models of multivariate ordinal logistic regression showed that age, educational level and financial resources significantly and independently predict HL skills, with OR values generally being higher than those observed in the models that consider each single HL measure. These results suggest that a broader evaluation of HL dimensions—obtained by integrating the NVS and the HLS-EU-Q16 data—could better represent the real meaning of the complex and hard-to-measure concept of HL. However, further research is needed to confirm these results and to evaluate whether this approach will also better predict the association between HL and health-related outcomes.

Moreover, as widely described for diagnostic and screening tests,54 the use of parallel tests (ie, two tests administered at the same time followed by subsequent combination of the results) results in an increase in sensitivity—in this case, the identification of people with low HL skills. For these reasons, the integration of different HL measures using an approach similar to the one used in this study may help to widen the narrow view resulting from the use of a single measure and may serve as the basis for the design of a more comprehensive measurement tool for HL. In this regard, it should be underlined that the approach of integrating different HL measures is in line with what has been suggested by Pleasant et al 24 for the definition of a comprehensive measure of HL: multidimensional in content and methodology.

This study has several limitations. Some of them are related to the sampling procedure. In particular, one of the main limitations is that the data cannot be considered representative of the overall Italian or Florentine adult population since the population-based sample was obtained with a combination of convenience and probability sampling procedures. For this reason, the generalisability of the results to the entire Florentine population is limited. In fact, although participants were randomly selected from the registers of the GPs, the GPs were selected using convenience criteria, which may have introduced a selection bias. Additionally, results may have been influenced by a non-response bias. Indeed, a relevant proportion of participants resulted to have a high socioeconomic level, a bachelor’s degree or higher, or had more than enough financial resources. These limits could influence an external comparison of the study results, since age, educational level and financial resources are determinants of HL. Sex was not included in the analysis since, at the univariate analysis, it was not significantly associated with HL.

Other limitations are related to the cut-off values of both the NVS and the HLS-EU-Q16 that were used to categorise the levels of HL. Although widely used thresholds were applied, these cut-offs have not been previously validated for the Italian population, since large population-based studies using the NVS and HLS-EU-Q16 have not been performed yet. Moreover, some alternatives could have been considered for the combination of the two variables. In particular, one of them could be the combination of the items of the two measures into a single scale and assessing the reliability using the classic approach. The chosen methodology is related to the aim of giving an initial, simple approach for assessing the possibility of integrating different measures of HL, and this will be refined with future studies.

Conclusion

In conclusion, our findings suggest that the combination of the results obtained using a performance-based measure of functional HL (the NVS) and a self-assessed measure of general HL (HLS-EU-Q16) may improve the understanding of the HL skills of individuals and populations as well as the relationship between HL and its antecedents. In addition, the new variable generated by this combination of different HL measures (HL skills) may help to better identify people with low HL skills and could be considered as a new measure of HL or, at least, a different way of assessing HL and its multidimensional contents. Although further studies are needed to confirm our findings and to better define the potential of the combined use of different HL measures, we think that this paper can be considered to be a starting point for a novel approach to the investigation of HL, regardless of the limits of this research,.

Moreover, the results of our study seem to be in line with the evolution of HL proposed by The Secretary’s Advisory Committee on US National Health Promotion and Disease Prevention Objectives for Healthy People 2030: ‘Health literacy occurs when a society provides accurate health information and services that people can easily find, understand, and use to inform their decisions and actions’. Nowadays, we should consider HL to be a type of social competence and responsibility, and we should measure all its facets to make it a discipline that can contribute to a higher level of clarity, accessibility and actionability, so as to reduce inequalities in health.55

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

The authors want to thank the MDPI English editing service for the copyediting.

Footnotes

Collaborators: Florence Health Literacy Research Group: Elisabetta Alti (general practitioner, Local Health Unit – Toscana Centro, Italy), Sergio Baglioni (general practitioner, Local Health Unit – Toscana Centro, Italy), Leonardo Bellino (general practitioner, Local Health Unit – Toscana Centro, Italy), Niccolò Berzi (general practitioner, Local Health Unit – Toscana Centro, Italy), Jacopo Bianchi (School of Specialization in Hygiene and Preventive Medicine, University of Florence, Florence, Italy), Giuseppe Burgio (general practitioner, Local Health Unit – Toscana Centro, Italy), Alessandro Bussotti (Careggi University Hospital, Florence, Italy), Marco Del Riccio (Department of Health Science, University of Florence, Florence, Italy), Martina Donzellini (School of Specialization in Hygiene and Preventive Medicine, University of Florence, Florence, Italy), Angela Galdiero (Local Health Unit – Toscana Centro, Italy), Alessandro Grassi (general practitioner, Local Health Unit – Toscana Centro, Italy), Tommaso Grassi (School of Specialization in Hygiene and Preventive Medicine, University of Florence, Florence, Italy), Arrigo Lombardi (general practitioner, Local Health Unit – Toscana Centro, Italy), Sarah Mantwill (Department of Health Sciences & Health Policy, University of Lucerne, Lucerne, Switzerland), Federico Manzi (School of Specialization in Hygiene and Preventive Medicine, University of Florence, Florence, Italy), Alessandro Mereu (general practitioner, Local Health Unit – Toscana Centro, Italy), Donatella Messina (general practitioner, Local Health Unit – Toscana Centro, Italy), Chiara Milani (School of Specialization in Hygiene and Preventive Medicine, University of Florence, Florence, Italy), Marco Targonato (general practitioner, Local Health Unit – Toscana Centro, Italy), Marco Toccafondi (general practitioner, Local Health Unit – Toscana Centro, Italy), Gino Sartor (School of Specialization in Hygiene and Preventive Medicine, University of Florence, Florence, Italy) and Virginia Vettori (Department of Health Science, University of Florence, Florence, Italy).

Contributors: CL: conception and design of the study; analysis and interpretation of data; and drafting and revision of the manuscript. VL and DP: conception and design of the study; generation, collection, assembly and interpretation of data; and drafting and revision of the manuscript. GB: conception and design of the study; interpretation of data; and drafting and revision of the manuscript. Other component of the Florence Health Literacy Research Group: conception and design of the study; generation, collection and assembly of data; and drafting and revision of the manuscript.

Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Patient consent for publication: Not required.

Ethics approval: The study was approved by the Ethics Committee of the ‘Area Vasta Centro’ (Local Health Unit of Tuscany-Center, Careggi University Hospital and Meyer University Hospital; Ref. CEAVC: 10113, 1 December 2016).

Provenance and peer review: Not commissioned; externally peer reviewed.

Data availability statement: Data are available on reasonable request. The dataset generated and analysed during the current study is available from the corresponding author on reasonable request.

Contributor Information

Florence Health Literacy Research Group:

Elisabetta Alti, Sergio Baglioni, Leonardo Bellino, Niccolò Berzi, Jacopo Bianchi, Guglielmo Bonaccorsi, Giuseppe Burgio, Alessandro Bussotti, Marco Del Riccio, Angela Galdiero, Alessandro Grassi, Tommaso Grassi, Vieri Lastrucci, Arrigo Lombardi, Sarah Mantwill ChiaraLorini, Federico Manzi, Alessandro Mereu, Donatella Messina, Chiara Milani, Marco Targonato DianaPaolini, Marco Toccafondi, Gino Sartor, Virginia Vettori, and Martina Donzellini

References

  • 1. Kickbusch IS. Health literacy: addressing the health and education divide. Health Promot Int 2001;16:289–97. 10.1093/heapro/16.3.289 [DOI] [PubMed] [Google Scholar]
  • 2. Sørensen K, Van den Broucke S, Fullam J, et al. . Consortium health literacy project European. health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012;12:80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Van den Broucke S. Health literacy: a critical concept for public health. Arch Public Health 2014;72:10. 10.1186/2049-3258-72-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Freebody P, Luke A. ‘Literacies’ programs: debates and demands in cultural context. Prospect:an Australian journal of TESOL 1990;5:7–16. [Google Scholar]
  • 5. Kickbusch I, Pelikan LM, Apfel F, et al. . Health literacy. The solid facts. Copenhagen: World Health Organisation Regional Office for Europe, 2013. [Google Scholar]
  • 6. Kobayashi LC, Wardle J, Wolf MS, et al. . Aging and functional health literacy: a systematic review and meta-analysis. J Gerontol B Psychol Sci Soc Sci 2016;71:445–57. 10.1093/geronb/gbu161 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Bonaccorsi G, Lastrucci V, Vettori V, et al. . Functional health literacy in a population-based sample in Florence: a cross-sectional study using the newest vital sign. BMJ Open 2019;9:e026356. 10.1136/bmjopen-2018-026356 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Adams RJ, Appleton SL, Hill CL, et al. . Risks associated with low functional health literacy in an Australian population. Med J Aust 2009;191:530–4. 10.5694/j.1326-5377.2009.tb03304.x [DOI] [PubMed] [Google Scholar]
  • 9. Stormacq C, Van den Broucke S, Wosinski J. Does health literacy mediate the relationship between socioeconomic status and health disparities? integrative review. Health Promot Int 2018. [DOI] [PubMed] [Google Scholar]
  • 10. Pelikan JM, Ganahl K, Roethlin F. Health literacy as a determinant, mediator and/or moderator of health: empirical models using the European health literacy survey dataset. Glob Health Promot 2018;25:57–66. 10.1177/1757975918788300 [DOI] [PubMed] [Google Scholar]
  • 11. Lastrucci V, Lorini C, Caini S, et al. . Health literacy as a mediator of the relationship between socioeconomic status and health: a cross-sectional study in a population-based sample in Florence. PLoS One 2019;14:e0227007. 10.1371/journal.pone.0227007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Parker R. Measuring health literacy: What? So what? Now what : Measures of health literacy: workshop summary. Washington, DC: National Academies Press, 2009. [Google Scholar]
  • 13. Levin-Zamir D, Leung AYM, Dodson S, et al. . Health literacy in selected populations: individuals, families, and communities from the International and cultural perspective. Stud Health Technol Inform 2017;240:131–51. 10.3233/ISU-170834 [DOI] [PubMed] [Google Scholar]
  • 14. HLS-EU Consortium Comparative report on health literacy in eight EU member states. The European health literacy survey HLS-EU., 2012. Available: http://ec.europa.eu/chafea/documents/news/Comparative_report_on_health_literacy_in_eight_EU_member_states.pdf
  • 15. Nguyen TH, Paasche-Orlow MK, McCormack LA. The state of the science of health literacy measurement. Stud Health Technol Inform 2017;240:189–203. 10.3233/ISU-170827 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Altin SV, Finke I, Kautz-Freimuth S, et al. . The evolution of health literacy assessment tools: a systematic review. BMC Public Health 2014;14:1207. 10.1186/1471-2458-14-1207 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Haun JN, Valerio MA, McCormack LA, et al. . Health literacy measurement: an inventory and descriptive summary of 51 instruments. J Health Commun 2014;19 Suppl 2:302–33. 10.1080/10810730.2014.936571 [DOI] [PubMed] [Google Scholar]
  • 18. Jordan JE, Osborne RH, Buchbinder R. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol 2011;64:366–79. 10.1016/j.jclinepi.2010.04.005 [DOI] [PubMed] [Google Scholar]
  • 19. Kiechle ES, Bailey SC, Hedlund LA, et al. . Different measures, different outcomes? A systematic review of Performance-based versus self-reported measures of health literacy and numeracy. J Gen Intern Med 2015;30:1538–46. 10.1007/s11606-015-3288-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Marciano L, Camerini A-L, Schulz PJ. The role of health literacy in diabetes knowledge, self-care, and glycemic control: a meta-analysis. J Gen Intern Med 2019;34:1007–17. 10.1007/s11606-019-04832-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Malloy-Weir L, Cooper M, literacy H. Health literacy, literacy, numeracy and nutrition label understanding and use: a scoping review of the literature. J Hum Nutr Diet 2017;30:309–25. 10.1111/jhn.12428 [DOI] [PubMed] [Google Scholar]
  • 22. Lorini C, Santomauro F, Donzellini M, et al. . Health literacy and vaccination: a systematic review. Hum Vaccin Immunother 2018;14:478–88. 10.1080/21645515.2017.1392423 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Griffin JM, Partin MR, Noorbaloochi S, et al. . Variation in estimates of limited health literacy by assessment instruments and non-response bias. J Gen Intern Med 2010;25:675–81. 10.1007/s11606-010-1304-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Pleasant A, McKinney J, Rikard RV. Health literacy measurement: a proposed research agenda. J Health Commun 2011;16 Suppl 3:11–21. 10.1080/10810730.2011.604392 [DOI] [PubMed] [Google Scholar]
  • 25. McCormack L, Haun J, Sørensen K, et al. . Recommendations for advancing health literacy measurement. J Health Commun 2013;18 Suppl 1:9–14. 10.1080/10810730.2013.829892 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Almaleh R, Helmy Y, Farhat E, et al. . Assessment of health literacy among outpatient clinics attendees at Ain Shams university hospitals, Egypt: a cross-sectional study. Public Health 2017;151:137–45. 10.1016/j.puhe.2017.06.024 [DOI] [PubMed] [Google Scholar]
  • 27. Pelikan JM, Ganahl K. Measuring health literacy in general populations: primary findings from the HLS-EU Consortium’s Health Literacy Assessment Effort. Stud Health Technol Inf 2017;240:34–59. [PubMed] [Google Scholar]
  • 28. Waters EA, Biddle C, Kaphingst KA, et al. . Examining the interrelations among objective and subjective health literacy and numeracy and their associations with health knowledge. J Gen Intern Med 2018;33:1945–53. 10.1007/s11606-018-4624-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Lorini C, Santomauro F, Grazzini M, et al. . Health literacy in Italy: a cross-sectional study protocol to assess the health literacy level in a population-based sample, and to validate health literacy measures in the Italian language. BMJ Open 2017;7:017812 10.1136/bmjopen-2017-017812 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Lorini C, Lastrucci V, Mantwill S, et al. . Measuring health literacy in Italy: a validation study of the HLS-EU-Q16 and of the HLS-EU-Q6 in Italian language, conducted in Florence and its surroundings. Ann Ist Super Sanita 2019;55:10–18. 10.4415/ANN_19_01_04 [DOI] [PubMed] [Google Scholar]
  • 31. Bechini A, Pieralli F, Chellini E, et al. . Application of socio-economic-health deprivation index, analysis of mortality and influenza vaccination coverage in the elderly population of Tuscany. J Prev Med Hyg 2018;59:E18–25. 10.15167/2421-4248/jpmh2018.59.4s2.1116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Unim B, De Vito C, Massimi A, et al. . The need to improve implementation and use of lifestyle surveillance systems for planning prevention activities: an analysis of the Italian regions. Public Health 2016;130:51–8. 10.1016/j.puhe.2015.08.009 [DOI] [PubMed] [Google Scholar]
  • 33. Minardi V, Ferrante G, D'Argenio P, et al. . Roll-your-own cigarette use in Italy: sales and consumer profile-data from PASSI surveillance, 2015-2016. Int J Public Health 2019;64:423–30. 10.1007/s00038-019-01204-5 [DOI] [PubMed] [Google Scholar]
  • 34. Capecchi L, Guazzini A, Lorini C, et al. . The first Italian validation of the most widespread health literacy assessment tool: the newest vital sign. Epidemiol Prev 2015;39:124–8. [PubMed] [Google Scholar]
  • 35. Bonaccorsi G, Pieralli F, Innocenti M, et al. . Non-Familial paid caregivers as potential flu carriers and cause of spread: the primary prevention of flu measured through their adhesion to flu vaccination campaigns—A Florentine experience. Hum Vaccin Immunother 2019;15:2416–22. 10.1080/21645515.2019.1593726 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Bonaccorsi G, Grazzini M, Pieri L, et al. . Assessment of health literacy and validation of single-item literacy screener (SILS) in a sample of Italian people. Ann Ist Super Sanita 2017;53:205–12. 10.4415/ANN_17_03_05 [DOI] [PubMed] [Google Scholar]
  • 37. Weiss BD, Mays MZ, Martz W, et al. . Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med 2005;3:514–22. 10.1370/afm.405 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Sørensen K, Pelikan JM, Röthlin F, et al. . Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health 2015;25:1053–8. 10.1093/eurpub/ckv043 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Sørensen K, Van den Broucke S, Pelikan JM, et al. . Measuring health literacy in populations: illuminating the design and development process of the European health literacy survey questionnaire (HLS-EU-Q). BMC Public Health 2013;13:948. 10.1186/1471-2458-13-948 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Halbach SM, Ernstmann N, Kowalski C, et al. . Unmet information needs and limited health literacy in newly diagnosed breast cancer patients over the course of cancer treatment. Patient Educ Couns 2016;99:1511–8. 10.1016/j.pec.2016.06.028 [DOI] [PubMed] [Google Scholar]
  • 41. Pelikan JM, Ganahl K. Measuring health literacy in general populations: primary findings from the HLS-EU Consortium's health literacy assessment effort. Stud Health Technol Inform 2017;240:34–59. [PubMed] [Google Scholar]
  • 42. Koletsi D, Pandis N. Ordinal logistic regression. Am J Orthod Dentofacial Orthop 2018;153:157–8. 10.1016/j.ajodo.2017.11.011 [DOI] [PubMed] [Google Scholar]
  • 43. Fransen MP, Leenaars KEF, Rowlands G, et al. . International application of health literacy measures: adaptation and validation of the newest vital sign in the Netherlands. Patient Educ Couns 2014;97:403–9. 10.1016/j.pec.2014.08.017 [DOI] [PubMed] [Google Scholar]
  • 44. Alba JW, Hutchinson JW. Knowledge calibration: what consumers know and what they think they know. J Consum Res 2000;27:123–56. 10.1086/314317 [DOI] [Google Scholar]
  • 45. Carlson JP, Vincent LH, Hardesty DM, et al. . Objective and subjective knowledge relationships: a quantitative analysis of consumer research findings. J Consum Res 2009;35:864–76. 10.1086/593688 [DOI] [Google Scholar]
  • 46. Pieniak Z, Aertsens J, Verbeke W. Subjective and objective knowledge as determinants of organic vegetables consumption. Food Qual Prefer 2010;21:581–8. 10.1016/j.foodqual.2010.03.004 [DOI] [Google Scholar]
  • 47. Schinckus L, Dangoisse F, Van den Broucke S, et al. . When knowing is not enough: emotional distress and depression reduce the positive effects of health literacy on diabetes self-management. Patient Educ Couns 2018;101:324–30. 10.1016/j.pec.2017.08.006 [DOI] [PubMed] [Google Scholar]
  • 48. Stolp S, Zabrucky KM. Contributions of metacognitive and self-regulated learning theories to investigations of calibration of comprehension. Int. Electron J Elementary Educ 2017;2:7–31. [Google Scholar]
  • 49. Belmi P, Neale MA, Reiff D, et al. . The social advantage of miscalibrated individuals: the relationship between social class and overconfidence and its implications for class-based inequality. J Pers Soc Psychol 2020;118:254–82. 10.1037/pspi0000187 [DOI] [PubMed] [Google Scholar]
  • 50. Crondahl K, Karlsson LE. The nexus between health literacy and empowerment: a scoping review. Sage Open 2016;6:1–7. [Google Scholar]
  • 51. Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promot Int 2000;15:259–67. 10.1093/heapro/15.3.259 [DOI] [Google Scholar]
  • 52. Chinn D. Critical health literacy: a review and critical analysis. Soc Sci Med 2011;73:60–7. 10.1016/j.socscimed.2011.04.004 [DOI] [PubMed] [Google Scholar]
  • 53. van der Heide I, Heijmans M, Schuit AJ, et al. . Functional, interactive and critical health literacy: varying relationships with control over care and number of GP visits. Patient Educ Couns 2015;98:998–1004. 10.1016/j.pec.2015.04.006 [DOI] [PubMed] [Google Scholar]
  • 54. Fletcher RH, Fletcher SH, Fletcher Grant S. Clinical epidemiology: the essentials. fifth ed Philadelphia: Lippincott Williams & Wilkins, 2012. [Google Scholar]
  • 55. United States Government, Office of disease prevention and health promotion, office of the assistant Secretary for health, office of the Secretary, department of health and human services. solicitation for written comments on an updated health literacy definition for healthy people, 2030. Available: https://www.federalregister.gov/documents/2019/06/04/2019-11571/solicitation-for-written-comments-on-an-updated-health-literacy-definition-for-healthy-people-2030

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2019-035987supp001.pdf (181.9KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES