Psychometric validation of Swedish and Arabic versions of two health literacy questionnaires, eHEALS and HLS-EU-Q16, for use in a Swedish context: a study protocol

Abstract

Introduction

Equity in health and access to healthcare regardless of gender, ethnicity or social position is a major political issue worldwide. Regardless of an individual’s knowledge, motivation and competence, individuals are expected to be engaged and take responsibility of their own care. Migrants have been identified as a vulnerable population in healthcare, and an explanation for the inequity in health and in healthcare is limited health literacy. Furthermore, with increasing digitalisation in healthcare, it also puts demand on the individual to have digital or electronic health (eHealth) literacy.

The overall aim of this study is to conduct a psychometric evaluation of the Swedish and Arabic versions of HLS-EU-Q16 and eHEALS and to compare Arabic and Swedish speakers’ Health literacy and eHealth literacy levels in Sweden.

Methods and analysis

This is a prospective, psychometric evaluation study with the intent of including 300 Arabic-speaking and 300 Swedish-speaking participants. Questionnaires: The Health Literacy Survey European Questionnaire (HLS-EU-Q16) includes 16 items measuring perceived personal skills of finding, understanding, judging and applying health information to maintain and improve their health. The eHealth literacy scale (eHEALS) is an 8-item scale measuring health literacy skills in relation to online information and applications.

This study will be conducted in four phases. Phase 1: Translation of HLS-EU-Q16 and eHEALS from English to Swedish and Arabic versions following the principles of translation of questionnaires. Phase 2: Content validity testing of eHEALS, including face validity and interpretability, conducted with five Arabic and five Swedish-speaking participants. Phase 3: Psychometric testing including construct validity, reliability, feasibility and floor ceiling effects. Phase 4: Distribution and comparison of eHealth and HLS-EU-Q16 analysed with χ² and Fisher’s exact test as appropriate. To assess associations between HLS-EU-Q16, eHEALS and demographic variables, binary logistic regression analyses will be performed.

Ethics and dissemination

The project has been approved by the regional ethical review board in Stockholm, Sweden (2019/5:1) and will follow the principles outlined in the 1964 Helsinki Declaration and its later amendments. Results from this study will be disseminated in peer-reviewed journals, scientific conferences and social media.

Strengths and limitations of this study.

• A Swedish and Arabic version of eHEALS and HLS-EU-Q16 have not yet been psychometrically evaluated.

• eHealth literacy has not been investigated previously in a Swedish population.

• This is the first study to compare eHealth literacy between Swedish-speaking and Arabic-speaking citizens in Sweden.

• Although this study includes both Swedish-speaking and Arabic-speaking participants from different contexts in the society, the study is conducted in Sweden and may not be generalisable to other contexts.

Introduction

The importance of health literacy

Patients in today’s healthcare system are expected to take part and be engaged in their own care. Consequently, they have to be able to read and understand health instructions on how to manage their own recovery at home.¹ As such, a patient’s health literacy also influences their ability to take part in informed decision-making. In this article we use a comprehensive definition of health literacy:

Health literacy is linked to literacy and entails people’s knowledge, motivation, and competence to access, understand, appraise, and apply health information in order to make judgments and take decisions in everyday life concerning healthcare, disease prevention and health promotion to maintain or improve quality of life during the life course.²

Health literacy is regarded as a social determinant of health³ and has a strong social gradient.⁴ It means that health literacy can be an additional barrier to health for already disadvantaged and marginalised groups within societies. Limited health literacy is likely to affect patients’ quality of care, resulting in lower satisfaction with care and lower understanding of their medical situation⁵ and their safety, by decreasing the probability of an adverse medication due to misunderstanding instructions.^{5 6} Health literacy is also associated with the extent to which people benefit from health examinations⁷ and the quality of their postoperative recovery.⁸

With increasing digitalisation of information and services, modern healthcare and health promotion have become increasingly challenging for both patients and healthcare staff.⁹ This demands a range of digital competencies among users, requires new ways to describe and evaluate users’ digital capabilities and experiences in this rapidly changing health context.⁹ Consequently, the concept of digital or electronic health (eHealth) literacy has emerged^10–12 and been described as ‘the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem’.¹²

Equity in health and access to healthcare regardless of social position, gender, race or ethnicity is a major political issue in the European Region and worldwide.¹³ The Swedish Health and Medical Services Act¹⁴ states that the goal of healthcare is to promote ‘good health and care on equal terms for the entire population’.^{6 14} Migrants, whether they are labour migrants or refugees, have been identified as a vulnerable population, but there is heterogeneity in the degree to which they are vulnerable to inadequate healthcare.5 6 15 16 One explanatory factor for the inequity in health and in healthcare is limited health literacy on individual and organisational level.¹⁷ A Swedish study shows, for example, that newly arrived refugees with limited health literacy experienced poor quality of communication and benefited less from the health examination for asylum seekers to a higher degree than those with higher health literacy.⁷ Another study shows that newly arrived refugees with limited comprehensive health literacy have poorer general health and do not seek needed care as often as those with higher comprehensive health literacy.¹⁸

HLS-EU-Q16

In 2011–2012, Sørensen et al developed the Health Literacy Survey European Questionnaire, HLS-EUQ47, a self-reporting instrument consisting of 47 items.¹⁹ The instrument is based on a systematic literature review that derived an all-inclusive conceptual model and the definition of comprehensive health literacy used in this study. In 2013, the instrument was used in a large study including populations in eight European countries.⁴ As a result of that study’s analysis, the shorter 16-item version, HLS-EU-Q16, was developed.² Both the HLS-EUQ47 and HLS-EU-Q16 have been used frequently in many different countries and are available in a range of languages.2 18 20–22

The HLS-EU-Q16 aims to measure respondents’ perceived personal skills of finding, understanding, judging and applying health information to maintain and improve their health.¹⁹ Each item in the instrument is answered on a 4-point Likert scale with response options ranging from ‘very difficult’ to ‘very easy’. An overall HLS-EU-Q16 index will be calculated in three steps according to the developer.²³ First, the response categories for the 16 items will be dichotomised into easy (fairly easy and easy) giving the value 1, and difficult (difficult and very difficult) giving the value 0. Second, an overall sum score will be calculated. Third, sum scores will be divided into three categories: inadequate (0–8 score points), problematic (9–12 score points) and sufficient (13–16 score points) health literacy. HLS-EUQ16 has been psychometrically tested and showed to be in some migrant populations in which it was found to be reliable and valid.²⁴

The Arabic and Swedish versions of HLS-EU-Q16 have been translated in line with guidelines for the translation of instruments and tested for face validity among migrants in Sweden.²⁵ The Swedish and Arabic versions of the HLS-EU-Q16 have since been used in several studies in Sweden^{7 18 26} and Egypt.²⁰ However, the translated Swedish and Arabic versions have not yet been tested for other aspects of validity and comprehensive health literacy has not been compared between Swedish-speaking and non-Swedish-speaking people.

eHEALS

In 2006, Norman and Skinner¹² developed the eHealth literacy scale, eHEALS. It aims to measure a broad range of literacy skills, which might make it useful in assessing the effects of strategies to deliver online information and applications. eHEALS is an 8-item instrument with each item scored on a 5-point Likert scale with response options ranging from ‘strongly agree’ to ‘strongly disagree’. Total scores on the eHEALS range from 8 to 40, with higher scores representing higher self-perceived eHealth literacy.

The eHEALS is available in a range of languages11 12 27–34 and the English version has been successfully administered digitally via telephone.³⁵ Tests of the validity of eHEALS indicates that it is a reliable and valid instrument12 28 36–38 and also that the validity of it requires further investigation.¹¹ However, the eHEALS has not been tested for validity among the general Swedish-speaking or Arabic-speaking population in Sweden. eHealth literacy has not been examined in Sweden nor been compared between Swedish-speaking and non-Swedish-speaking people.

Aim

The overall aim of this study is to psychometrically evaluate the Swedish and Arabic versions of HLS-EU-Q16 and eHEALS and to compare Arabic and Swedish speakers’ Health literacy and eHealth literacy levels in Sweden.

Methods and analysis

Study design

This is a prospective, psychometric and comparative evaluation which is divided into four different phases (figure 1). Study recruitment will be started in February 2019 and planned to end in August 2019.

Figure 1.

Overview of the research process.

Phase 1: translation process

The original English version of eHEALS will be translated into Swedish and Arabic versions. Arabic was selected as it is the most common native language spoken among refugees in Sweden.³⁹ Independent translators with either Swedish or Arabic as their native language will translate the English version of eHEALS to Swedish and Arabic versions. These translations should use plain language and be comprehensible to a 12-year-old speaker of the target language.⁴⁰ Although it is important that the content of the items remain the same as in the original version, the wording or word order in the translated versions should be appropriate to the target language and understandable by speakers with various levels of education.

A translator group will consist of four professional translators: one for the forward translation into each of Swedish and Arabic version and one for backward translations from each language. The translators will be recruited from translator associations found and through personal contacts. Previous experience in translating survey questions within the health domain will be a criterion for recruitment as a translator.

A committee to examine the quality of the translations will be recruited on the basis of criteria recommended for committees used in cross-cultural adaption projects.⁴¹ The committee will consist of 12 members: 8 bilingual in English and Swedish and 4 multilingual in English, Arabic and Swedish. The bilingual members should have experience in plain language and health literacy and/or health communication. Multilingual members should have experiences from data collections with Arabic-speaking participants within the public health domain and previous experience of reviewing translated instruments.

The translation into Swedish will be completed first and reviewed by the researchers who will compare it with the original English version, examine how well it fits the Swedish context and check it for plain language. If necessary, adjustments will be made and the new versions will be discussed with the bilingual members in the committee to ensure that the content of the English and Swedish versions is the same and that plain language is used. The new versions will also be discussed with four Swedish-speaking laypeople of different ages, genders and education levels to ensure that the items are understandable. When the researchers are satisfied with the Swedish version, based on the feedback from the bilingual and laypeople, it will be back-translated into English and the researchers and the translator will compare it with the English original version. If the back-translation does not match the original eHEALS, the Swedish version will be adjusted and back-translated again as many times as necessary to obtain a back-translation consistent with the original English version.

Once the Swedish version is finalised, the translated Arabic version will be reviewed by the four multilingual members in the committee who together with the researchers will discuss how well the translated version fits the Swedish context, to what extent plain language has been used and how well the content of the translated version matches both the English original and the final Swedish versions. If necessary, adjustments will be made and the new versions will be discussed again with the multilingual members. When all are satisfied with the translated version, it will be back-translated into English by one independent translator and compared with the English original version by one of the multilingual members. If the back-translation does not match the English original version, it will be adjusted and back-translated again as many times as necessary to obtain a back-translation consistent with the original English version.

Phase 2: content validity testing of the Arabic and Swedish versions of eHEALS

Content validity

Content validity is the degree to which the content of an instrument is an adequate reflection of the construct it is meant to measure.⁴² As the plan is to use the instruments in this study in the general population, that is, in study populations they have been used previously in, no greater adaptions of the instruments are needed. Examination of the content of the instrument by an expert committee will therefore not be conducted. However, the face validity – the degree to which the items or the instrument as a whole appears to adequately reflect the construct to be measured⁴² – and the interpretability of the items⁴⁰ will be examined.

The face validity of the final Swedish and Arabic versions will be tested through qualitative interviews with five people in each language group to check whether these people understand the items as intended. Participants will be recruited purposively and through snowball sampling⁴³ either directly by the researchers or through the researchers’ personal contacts with key people in groups speaking Arabic. A mix of different ages, genders and educational levels will be sought. Information about the project and the meaning of informed consent will be given orally and in writing to the participants. If the person agrees to attend, the time for the interview will be booked.

In the interview, participants will be asked ‘What were you thinking of while you were answering that question?’ and if necessary, ‘Why did you select that response?’ Interviews will be audio recorded and notes written by the interviewer into templates containing the two predetermined questions. Participants will also be asked about their age, gender, highest level of education and years living in Sweden.

The five Swedish-speaking participants will be interviewed by two of the researchers and the five Arabic-speaking participants by a research assistant with Arabic as native language who will take notes in Swedish. Each interviewer will listen to the recorded audio files, complemented by the notes (if any) written immediately after each interview. The researchers, will then read all notes separately.

If the analysis shows that any items are difficult to understand and major changes are needed to either translation, the items concerned will be revised and tested again on five new participants. This procedure will be repeated until all items are easy to understand.

Phase 3: psychometric testing of the Arabic and Swedish versions of eHEALS and HLS-EU-Q16

Participants

Two different language groups will be recruited from different regions in Sweden to test the translated instruments. The intention is to include one group of participants representative of the general Swedish-born population (n=300) and one group representative of Swedish residents born in an Arabic-speaking country (n=300). The chosen sample size was first guided by the 10:1 ratio⁴⁰: 16 items on the HLS-EU-Q16×10 = 160 participants. However, a general rule of thumb for factor analysis is 300 cases or the more lenient 50 participants per factor.⁴⁴ HLS-EU-Q16 consists of only one factor,¹⁹ therefore a sample size of 300 participants is considered most appropriate. Inclusion criteria for participation will be being an adult (≥18 years), having sufficient language skills to read, understand and fill in a form in their native language (Swedish or Arabic) and being available on the days of data collection. Arabic speakers born outside of Sweden will also have a Swedish resident’s permit.

Participants will be recruited from university courses, municipal adult education courses (Komvux), larger workplaces with both academic and non-academic staff, non-governmental organisations serving elderly people, migrant associations, courses in civic orientation and supplementary academic courses for nurses with degrees from countries outside of Europe. These arenas are chosen because they attract many Swedish residents of different ages, genders and levels of education who speak Swedish or Arabic as their first language. A mix of ages, genders and educational levels will be sought in both groups. On recruitment of the participants, organisations manager or responsible person conducting group activities with potential study participants will be contacted and informed about the project.

On the day of the data collection, a researcher will visit the various arenas to inform people both orally and in writing about the project and the meaning of informed consent. In groups with Arabic-speaking people, the Arabic-speaking research assistant will provide the oral and written information about the project and the meaning of informed consent in the potential participants’ native language. People who agree to participate will be given a questionnaire and asked to fill it in onsite directly. By filling in the questionnaire, they also consent to take part in the study.

The following questionnaires and study-specific questions will be distributed in Swedish and Arabic versions, respectively, depending on native language.

• The eHEALS instrument (eight items) assessing eHealth literacy.

• The HLS-EU-Q16 instrument (16 items) assessing comprehensive health literacy.

• One question about general self-perceived health.

• One question about use of internet.

• Descriptive background questions (age, gender, highest level of education, country of birth).

• Number of years lived in Sweden (Arabic-speaking group only).

The questionnaire used in the Arabic-speaking group includes the same components plus one question about the number of years they lived in Sweden.

General self-perceived health will be measured by the questions ‘How do you assess your overall health status?’ Response options are very poor, poor, fair, good and very good.^{7 45} Internet use will be measured by the question ‘How often do you use the internet?’ Response options are almost every day, several days a week, about 1 day a week, less than I day a week and almost never.¹¹

Psychometric testing

The psychometric testing will be guided by COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN).^{40 42 46}

Construct validity

The construct validity⁴² focuses on evaluating tests of the hypotheses. This aspect of construct validity can be described as the degree to which the scores of an instrument are consistent with a hypothesis.⁴² The participants and questionnaire for collecting data will be the same as those described above.

Based on previous studies on health literacy showing positive associations between limited health literacy and high age,^{4 47–49} low education level,^{20 49 50} poor health7 8 22 48 51 and between eHEALS and low use of the internet,¹¹ several hypotheses will be used regarding correlations between HLS-EU-Q16/eHEALS and age, level of education, self-perceived general health and quantity of internet use. Moreover, hypothesis regarding correlations between eHEALS and HLS-EU-Q16 and certain HLs-EU-Q16 items will be used. All hypothesis are presented in figure 2.

Figure 2.

Hypothesis of correlations between eHEALS/HLS-EU-Q16 and various variables. Item a: How easy/difficult is it for you to find information on treatments of illnesses that concern you? Item k: How easy/difficult is it for you to judge whether information on health risks in the media (eg, on television or the internet) is reliable? Item l: How easy/difficult is it for you to decide how you can protect yourself from illness based on information in media (eg, in newspapers or leaflets or on the internet)? Item o: How easy/difficult is it for you to understand information in the media (eg, from the internet or daily or weekly magazines) on how improve your health?

Spearman’s rank order correlation between total mean scores on eHEALS and HLS-EU-Q16, self-perceived health, level of education and age will be used. A coefficient magnitude of >0.4 will be considered evidence of construct validity (ie, moderate to strong correlations).⁴⁶

Reliability

Reliability can be used as a term for a domain and as a term for a measurement property.⁴² In this study, we analysed two aspects of reliability: internal consistency and test–retest reliability.

Internal consistency

Internal consistency describes the inter-relatedness among items⁴² and will be analysed for both eHEALS and HLS-EU-Q16.

• Exploratory factor analysis will be used to identify the underlying relationships between items on eHEALS and HLS-EU-Q16.⁴⁰

• Cronbach’s alpha will be calculated for each instrument to assess the average correlation of items within each scale. Cronbach’s alpha in the range of 0.7–0.95 will be considered acceptable.^{40 52}

• Split-half reliability will be used to measure the correlation between random split segments and to determine how much error in a test score is due to poor test construction.⁵² A Spearman–Brown coefficient of 0.70–0.90 will be considered acceptable.^{53 54}

Test–retest reliability

Test–retest reliability can be described as the extent to which scores for the same patients are the same in measurements repeated over time.⁴² Intraclass correlation coefficient (ICC) will be used and an ICC value of ≥0.7 will be considered acceptable.⁴⁰ Because the sample size needed for test–retest is much smaller than for testing many other forms of validity, the sample size in the retest of 25 people per language category (total 50) is considered appropriate.⁵⁵

Participants in the test–retest groups will be invited to take part in the retest when they are recruited to participate in the main test. To minimise dropouts, participants in the test–retest groups will be recruited in the pre-existing groups having regular, at least weekly, meetings. A mix of different ages, genders and education levels will be sought. To compare answers from the test and the retest, the participants will mark their questionnaires with their birth date or any other self-contained code if they do not wish to give their date of birth. If they use their own code, they will be asked to write it down in a list of codes and the researcher will bring to the second measurement in case if they forget their code.

Feasibility

The feasibility of the clinical user-friendliness of the instruments will be assessed by their successful response rate.⁴⁶

Floor and ceiling effects

Floor and ceiling effects (the number of respondents who achieve the lowest or highest possible scores⁴⁰) will be examined. Floor and ceiling effects are considered a problem if more than 15% of a study population achieve the lowest or highest possible score.⁴⁰

Phase 4: distribution of eHEALS and HLS-EU-Q16 levels in the study population and comparisons of levels in the two language groups

The distribution of eHEALS and HLS-EU-Q16 levels in the study population as a whole and within each language group will be examined. χ² tests—or Fisher’s exact tests as appropriate—will be used to test for differences in eHEALS and HLS-EU-Q16 between language groups. Binary logistic regression analyses will be performed to assess associations. We hypothesise that there will be positive correlations between HLS-EU-Q16/eHEALS and level of education, self-perceived general health and quantity of internet use; and negative correlations between HLS-EU-Q16/eHEALS and age. Moreover, we hypothesise positive correlations between eHEALS, HLS-EU-Q16 and certain HLs-EU-Q16 items (figure 2).

All data will be analysed using SPSS V.24.0 for Windows (IBM Corporation). Two-tailed p values under 0.05 will be considered significant.

Patient and public involvement

No public involvement.

Discussion

A considerable proportion of both the European population and newly arrived refugees in Sweden have limited comprehensive health literacy.⁴ How this is distributed in the general Swedish population and the eHealth literacy levels of Swedish-speaking and Arabic-speaking people living in Sweden is rather unknown. Knowledge about comprehensive health literacy (and by extension, eHealth literacy) is important, as it is associated with people’s health status,¹⁷ use of health information and healthcare services.5 6 33 47 56 Being regarded as an important social determinant of health³ and having a social gradient health literacy⁴ in patients is important to consider in work to promote health, disease prevention and healthcare in order to secure that people with various levels of health literacy benefit equally from health efforts to reduce the risk to promote inequity in health and healthcare instead of vice versa.¹⁷ However, in Sweden validated instruments to measure comprehensive health literacy and eHealth literacy of Swedish-speaking and Arabic-speaking people are lacking, that is, needs to be developed.

Our research team has developed and tested a digital monitoring system, Recovery Assessment by Phone Points (RAPP), that enables day surgery patients to contact healthcare easily and to report from home about how they feel.⁵⁷ It has been tested in studies with Swedish-speaking patients and proved to be cost-effective⁵⁸ to help patients feel safer and less bothered by symptoms, such as pain, nausea, anxiety, in their postoperative recovery.^{57 59} Strong relationships between postoperative recovery, physical, health and mental health were also found.⁶⁰ However, so far RAPP is only available to Swedish-speaking patients. Next step is to develop RAPP for non-Swedish-speaking patients, starting with Arabic, and to compare postoperative recovery and unplanned medical contacts between non-Swedish-speaking Arabic patients and Swedish-speaking patients. We will also study differences in eHealth literacy, mental health and postoperative recovery between the groups and describe their experiences of postoperative recovery and of using a digital tracking system such as RAPP. However, before we can investigate these questions, we need to develop valid Arabic and Swedish versions of eHEALS to be able to assess eHealth literacy.

Ethics and dissemination

Participants will receive written and verbal information about the study, including the purpose and procedures, the voluntary nature of participation and their option to withdraw at any time. They will also be guaranteed confidentiality and secure data storage.