Cultural and linguistic appropriateness of the HLS₁₉-Q12 health literacy scale items among people in Norway with a Somali background

Abstract

Although health literacy (HL) is essential for achieving health equity, limited knowledge exists about HL among migrant populations. The cultural and linguistic appropriateness of existing HL measurement scales is scarce. Hence, the aim of this study was to explore the cultural and linguistic appropriateness of HLS₁₉-Q12 scale items among individuals of Somali background living in Norway. The HLS₁₉-Q12 measure general HL and is a short version of the European Health Literacy Survey Questionnaire (HLS-EU-Q47). By using a sequential mixed methods approach, we present results from quantitative, qualitative and mixed methods meta-inferences perspective. We used data from the Somali diaspora in the Norwegian part of the Health Literacy Survey 2019 (HLS₁₉) and tested the data against the partial credit Rasch model. Cross-cultural cognitive interviews (CCCI) were then conducted, in which data were analysed using reflexive thematic analysis. The mixed methods approach gave us a comprehensive understanding. The CCCIs provided detailed insights into deviating items identified by the Rasch modelling, and the CCCIs revealed that more items were problematic than those flagged by Rasch modelling. A large body of literature examine psychometric properties of different scales. While this is important, this article also shows the challenges and implications in HL surveys regarding developing health equity policies for all. Hence, exploring the cultural and linguistically validation of scale items should be a standard. The Somali version of the HLS₁₉-Q12 scale items could not be considered culturally or linguistically appropriate for people with a Somali background living in Norway.

Contribution to Health Promotion.

• To our understanding, this is the first study aiming to explore the cultural and linguistic appropriateness of health literacy (HL) scale items among people with a Somali background.

• A large body of literature examines validation of scales from a psychometric perspective. While this is important, this article shows in addition the implications regarding cultural and linguistically appropriateness of HL survey items.

• Validated HL scale items for different target groups are needed for better health promotion and accurate development of HL and health equity policies.

• The sequential mixed methods approach in this study gave a comprehensive understanding of the results.

Background

Health literacy (HL) is crucial for better health outcomes and considered important for equity in health (WHO 2017, Sørensen et al. 2021). Using a life course perspective, Sørensen et al. (2012) describe HL as an individual’s competence to access, understand, appraise, and apply health information across the health domains healthcare, disease prevention, and health promotion. HL and health research indicate that culture, social perspective, and language (language proficiency) are associated with HL outcome and how people perceive their health (Andrulis and Brach 2007, Parvanta et al. 2017, Bauer 2019, Mantwill and Diviani 2019, Ward et al. 2019, Sørensen et al. 2021, Stanzel et al. 2021). However, knowledge about HL among people with migrant backgrounds is scarce both internationally and nationally (Ward et al. 2019, Le et al. 2021, Pelikan et al. 2021, The HLS19 Consortium of the WHO Action Network M-POHL 2021, Diaz and Benavente 2025). There is a growing interest in gaining insights into HL among migrant groups (Ward et al. 2019) and an increased interest in supporting migrants’ access to health care services sensitive to their needs (WHO 2022, 2023).

Migrants from Somalia represent ∼13% of the migrant populations in the EU region (EU 2019). However, there are few HL studies on these groups and particularly in Norway (Ward et al. 2019). Meanwhile, the results from the limited studies are contradicting; A Swedish study indicated an increased risk for low functional HL among Somali-speaking refugees with low education level and/or being born in Somalia (Wångdahl et al. 2014). This was in line with a Norwegian study by Gele et al. (2016) who indicated that Somali refugee women in Norway have very limited HL. However, another Norwegian study (Le et al. 2021) found that people with Somali background have high HL. The latter study is a large-scale survey on HL among five migrants’ groups in Norway, among them migrants with Somali background which our study is based on. Here, HL was measured using, among others, the HLS₁₉-Q12 scale items, which is based on the conceptual model of Sørensen et al. (2012) (The HLS19 Consortium of the WHO Action Network M-POHL 2021). However, Le et al. (2021, pp. 25) reported having linguistic challenges during the data collection. This raises questions about how the items in HLS₁₉-Q12 were understood and culturally received by the migrant groups, such as individuals of Somali descent.

Health and HL surveys often largely reflect the Western culture, and survey items are often designed from this perspective (Edwards 2014, Mantwill and Diviani 2019). As HL is culturally context-dependent (Mantwill and Diviani 2019), results from HL studies might not be comparable across cultures. Cross-cultural health studies encounter three primary challenges: (1) the degree to which cultures have shared interpretations of the items, (2) the origin of the questions, such as whether they are newly developed items, and (3) the comparability and validity of the items across different cultures (Edwards 2014, pp. 643–645). Different culture groups have different languages, and multicultural societies therefore may have various languages within their society (Mantwill and Diviani 2019, Strathern and Stewart 2020). Health surveys, such as HL surveys, must be culturally and linguistically adapted, and items must be understood similarly across culture and languages. However, the cross-cultural aspects are often first considered in the translation stage (Edwards 2014, Mantwill and Diviani 2019), and translation is a decisive factor in whether the data we get can be trusted (Hagen-Zanker et al. 2023). To translate is a challenge since the best translation is said to never to be exact (Edwards 2014). One consequence of not having culturally appropriate scale items and validated survey instruments is that we do not know if the scale items work within specific groups of the population. Hence, to provide valid and reliable information in multicultural studies, it is important to assess the psychometric properties when scale items are translated into a new language and/or administrated in a new population (Christensen et al. 2013, pp. 309–368). To achieve valid and reliable results from cross-cultural HL surveys, it is also crucial to assess whether the items are interpreted as intended.

Information about the psychometric properties of different HL scale items when applied to people with Somali background has to our knowledge not been reported in any scientific journal. Neither have the cultural or linguistic appropriateness of the HL scale items. Knowledge about validated HL scale items and survey instruments regarding migrant groups are, at least from a Norwegian perspective, scarce (Diaz and Benavente 2025). One recommended method for evaluating psychometric properties of measurement scales constructed from ordinal scale items is the Rasch modelling (Christensen et al. 2013). Rasch modelling provides detailed information about item performance and can help us improve or understand the precision of the scale items (Hagquist et al. 2009, Boone 2016). Another method is Cross-Cultural Cognitive Interviews (CCCI) which just as traditional cognitive interviews can be used to establish content validity. It is also used to uncover language and cultural barriers (Drennan 2003, Knafl et al. 2007, Willis 2015). Hence, the aim of this study was to use Rasch modelling and CCCI to explore the cultural and linguistic appropriateness of the HL HLS₁₉-Q12 scale items among people in Norway with Somali background. This article refers to ‘people with a Somali background’ based on input from the Somali community itself at an early reference group meeting.

Methods

Study design

This study has an explanatory sequential mixed methods design (Creswell and Clark 2017). First, secondary analyses of the dataset from the Norwegian part of the International Health Literacy Population Survey (HLS₁₉-NO) data (Le et al. 2021, The HLS19 Consortium of the WHO Action Network M-POHL 2021) was conducted to assess the psychometric properties of the Somali HLS₁₉-Q12 scale items. For these purposes, Rasch modelling (Rasch 1960, 1980) was applied. Then, new qualitative data were collected through CCCIs (Willis et al. 2011, Willis 2015) and analysed using reflexive thematic analysis to explore the cultural and linguistic appropriateness of the Somali translation of the HLS₁₉-Q12 scale items. Results from the Rasch modelling impacted the interview guide for CCCIs. Furthermore, results from analyses of quantitative and qualitative data were combined (meta-inferenced), and qualitative results were used to explain results from the analysis of quantitative data (Ivankova 2014, Creswell and Clark 2017).

Setting

The Health Literacy Survey 2019–2021 (HLS₁₉) was an international large-scale cross-sectional study initiated by the WHO’s Action Network Measuring Population and Organizational Health Literacy (M-POHL) (The HLS19 Consortium of the WHO Action Network M-POHL 2021). The survey was conducted in 17 countries, including Norway. Data were collected in Norway from Norwegian-speaking residents and from five migrant groups, including individuals with Somali background. Questionnaires were administered in Norwegian as well as in translated versions (Le et al. 2021).

The HLS₁₉-Q12 scale

The HLS₁₉-Q12 scale is a short version of the European Health Literacy Survey Questionnaire, HLS-EU-Q47 (Sørensen et al. 2013, 2015). It was developed for the purpose of the HLS₁₉ study (The HLS19 Consortium of the WHO Action Network M-POHL 2021, Pelikan et al. 2022). The scale aims to measure general HL and is developed based on the definition and conceptual framework of Sørensen et al. (2012). The HLS₁₉-Q12 consists of 12 self-reported items and uses a four-point rating scale (1 = ‘very difficult’, 2 = ‘difficult’, 3 = ‘easy’, 4 = ‘very easy’), where a higher score indicates higher HL (The HLS19 Consortium of the WHO Action Network M-POHL 2021).

The HLS₁₉-Q12 scale items were translated from English to Norwegian, and a few of the items (Items 16, 42, and 44) were culturally adapted to a Norwegian context (Le et al. 2021, pp. 19–20). The national data agency responsible for data collection translated the scale items back into English and translated that version into the five languages spoken by the respective migrant groups, including Somali (Le et al. 2021, pp. 25). This translation procedure was conducted prior to this study. However, language-related difficulties were reported when collecting HLS₁₉ data (Le et al. 2021, pp. 25–26). Hence, for the purpose of this study, the Somali translations were therefore back translated into Norwegian and subjected to linguistic review to provide deeper understanding of the translation process, identifying potential language challenges and verifying the quality of the Somali version. The back translation was performed by the study’s bilingual and bicultural team member (author 3) together with another bilingual individual. Following the back translation, more substantial translation and linguistic issues were identified than anticipated, making conventional interview methods unsuitable. Consequently, CCCIs were conducted instead (Willis et al. 2011). Like traditional cognitive interviews, CCCIs are used to establish content validity and to confirm that the informants comprehend the item content (Drennan 2003). In addition, CCCIs explicitly aim to uncover linguistic, cultural, and conceptual barriers to item comprehension (Willis 2015). Given that this study examines the cultural and linguistic appropriateness of the HLS₁₉-Q12 scale items, it was essential to determine whether the translated items conveyed meanings equivalent to the originals and whether they were culturally and linguistically appropriate to the target population (Willis 2015, Ivankova 2014, Vujcich et al. 2021, Hagen-Zanker et al. 2023). Collaboration with the Somali community throughout the study was central to perceiving their perspectives, ensuring cultural and linguistic appropriateness, and promoting user involvement.

Data collection

The HLS₁₉-NO survey data were collected using computer-assisted telephone interviewing (CATI) from October to December 2020 (Le et al. 2021, pp. 27–30). The CATI interviews were conducted through an official data collection agency in Norway (Le et al. 2021, pp. 27). The migrant samples were selected from the National Population Register and the data agency used official phone lists to recruit participants. Stratification was not used as a very large proportion of the population was contacted to achieve the desired number of participants (Le et al. 2021, pp. 27). The respondents with Somali background could choose to respond either in Norwegian or Somali (n = 368 out of 379 Somali respondents chose Somali). The analysis in this study is based on the 368 respondents that responded in Somali.

In total, 11 participants from the eastern and the southern part of Norway were included in the CCCIs. CCCIs were conducted as semi-structured interviews guided by a loosely structured interview guide with follow-up probes. The results from the Rasch modelling influenced the CCCI interview guide, and items that functioned invariantly between people with a Somali background and the general Norwegian population were emphasized in the CCCIs. The CCCIs were conducted between June 2023 and May 2024 and had an average length of 60 minutes. The informants were recruited via contact/networks in the Somali community by using the snowball method (Crouse and Lowe 2018). Nine interviews were conducted in Norwegian and two in Somali. We wanted to include informants who were fluent in both Somali and Norwegian so they could assess the scale in both languages. However, based on early CCCIs findings and recommendations, we also added two informants who spoke only Somali, to see whether monolingual Somali speakers understood the scale items differently from Somali Norwegian bilingual informants. We also ensured that the informants varied in age, gender, length of residency in Norway and educational level as such factors could influence how the translated version of the scale was perceived. Author 1 moderated the interviews in Norwegian, and author 3 was the moderator in the interviews that was conducted in Somali. Three interviews were conducted in person, while eight were conducted online using Zoom. The CCCIs were recorded using an audio recording app authorized for scientific research, with the recordings being automatically transmitted to a secure server.

Ethical considerations

This study was approved by the Norwegian Social Science Data Service (SIKT), ref. 168323 and by the committee for research ethics (KoFE) at the University of Inland Norway. Subject participation was voluntary and based on informed consent. The HLS₁₉ survey was completed anonymously.

Analyses

The Rasch modelling

The psychometric properties of the HLS₁₉-Q12 items were assessed by testing the data against the unidimensional partial credit Rasch model (PCM) (Masters 1982, pp. 149–174) by using the RUMM2030 statistical software package (Andrich and Sheridan 2019). Rasch modelling is preferable, as the model meets specific requirements for measurement (Tennant and Conaghan 2007, Andrich and Marais 2019). The Rasch model application provides detailed information on items’ performance and could be used to support item revision (Andrich and Marais 2019). In line with the aim of this study, we emphasize the item-level fit of HLS₁₉-Q12 scale items and report on data-model fit, item discrimination, ordering of response categories, and differential item functioning (DIF) (Boone 2016, Andrich and Marais 2019). Results concerning the psychometric properties of the HLS₁₉-Q12 scale items at an overall level are provided in Supplementary File S1.

A non-significant chi-square value and a z-fit residual within the range ± 2.5 indicate sufficient item fit (Andrich and Marais 2019). Response categories were considered ordered if the thresholds were significantly different and, in the right ‘order’ (Hagquist et al. 2009). DIF is present if an item does not work similarly across levels of a person factor (Andrich and Marais 2019), and two-way analysis of variance of standardized residuals was used to detect DIF (Andrich and Marais 2019). Uniform DIF refers to different item thresholds across groups and non-uniform DIF refers to different discrimination across groups (Hagquist et al. 2009, Andrich and Hagquist 2012, 2015). Statistically significance was set at Bonferroni-adjusted 5% level.

Based on findings from the HLS₁₉-NO study (Le et al. 2021), extended conversation with the data collection agency concerning the data collection, extended conversations with the translators and a technical report on Rasch analyses of international HLS₁₉ data (Guttersrud et al. 2023), DIF analyses were performed for 11 person factors. The 11 person factors were gender, age, education, the ability of paying bills, employment, length of residence, understanding written Norwegian, how well the respondents speak Norwegian, self-reported health, self-reported long-term illnesses, and general Norwegian (reference group) vs Somali population (focal group). All person factors were dichotomized, except for self-reported health, which was trichotomized (good, fair, bad). Residence time in Norway (‘10 and less’ and ‘11 and more’) was dichotomized in accordance with statistics from The Directorate of integration and diversity (Meland Leksen and Lundl 2024). Since the target of this study is the cross-cultural aspect, we specifically focused on cross-cultural differential item functioning (ccDIF) between the reference group (population sample HLS₁₉ in Norway n = 5998) and the focal group (additional data in HLS₁₉ Norway, Somali sample n = 368). Missing data were handled by using full information maximum likelihood estimation.

CCCI

The data analysis of the CCCIs, mostly an inductive approach, was inspired by Braun & Clarke’s guidelines process of reflexive thematic analysis (RTA) (Braun and Clarke 2006, 2019, 2022a, 2025). The RTA guidelines include getting to know the data, transcribing the data and re-reading the data and making notes. For this study the data were transcribed digitally verbatim, pseudonymized and then reviewed manually by author 1. The transcriptions were re-read several times and reflection notes were made. In addition, author 1 reviewed the audio recordings to identify variations in tone and emotional expression. Hence, not only transcribed materials were used, but also audio files playback notes of the informants’ feelings towards some of the items and the translations. Thereafter author 1 manually coded the data. Codes with similar content were grouped into potential themes. Four of the authors (KKØ, ØG, JW, HSF) having interdisciplinarity academic background (e.g. education, public health, nursing, and mathematics), reviewed the codes and generated themes from the codes. The process to create and generate the themes went through different phases and an initial thematic map was made. The themes and their labelling were discussed by the authors in several loops. In the end three themes were created. The four authors who created the themes are of Western origins. The labelling of the themes was therefore discussed with the bilingual co-worker and moderator in the CCCI interviews (author 3) to make sure the language in the themes was non-judgemental, not stereotyped in any way and ensuring the bicultural reflective perspective (Braun and Clarke 2023). According to the last part of the procedure of Braun and Clarke (2022b, 2024, 2025) the findings were thereafter summarized.

Results

The results are divided into three sections. First, we present quantitative results from the Rasch modelling followed by qualitative results from the CCCI analyses, and finally the mixed method results.

The Rasch modelling

The sample included 368 participants with a predominance of females (61%) (Table 1). The respondents were aged 16–79 (mean 36.1), and 41% of the informants reported lower secondary school or below as their highest level of completed education. Most participants (86%) reported sufficient financial resources to easily pay their bills, 93% reported having good health, and only 15% of the participants reported one or more long-term illnesses. The majority reported a strong ability to understand written Norwegian (89%) and speak Norwegian (85%), and half (50%) indicated that they had lived in Norway for 10 years or less.

Table 1.

Sample characteristics of the HLS₁₉ survey focal group (n = 368), people with Somali background living in Norway.

Characteristics	n (%)
Gender
Female	224 (61)
Male	144 (39)
Age (years)
16 to 36	195 (53)
37 or older	173 (47)
Min/max	16/79
Education level
ISCED 0–2 (lower secondary education or below)	152 (41)
ISCED 3–8 (upper secondary education or above)	199 (54)
Status of employment
Employed	245 (67)
Unemployed	97 (26)
Ability to pay bills
Easy/very easy	317 (86)
Difficult/very difficult	32(9)
Self-reported health
Good	343 (93)
Fair	8 (2)
Bad	14 (4)
Self-reported long-term illness(es)
Yes, 1 or more	57 (15)
No	301 (82)
Self-reported ability to speak Norwegian
Good/very good	314 (85)
Poor/very poor	39 (11)
Self-reported ability to understand written Norwegian
Good	326 (89)
Poor	41 (11)
Years lived in Norway
10 or less	185 (50)
11 or more	151 (41)
Mean	12.43
Standard deviation	7.24
Min/max	2/55

The use of response categories

Most respondents selected the response categories 3 (easy) or 4 (very easy) when responding to the HLS₁₉-Q12 scale items. Very few respondents used category 2 (difficult) and even fewer used category 1 (very difficult) (Table 2).

Table 2.

Distribution of responses on the HLS₁₉-Q12 items among people with a Somali background living in Norway (n = 368).

Item no	HLS19 item no.	Item wording	Very difficult (1)	Difficult (2)	Easy (3)	Very easy (4)
1	4	Find out where to get professional help when you are ill? (Instruction: doctor, nurse, pharmacy staff, psychologist)	1	13	187	160
2	7	Understand information about what to do in a medical emergency?	5	17	182	157
3	10	Judge the advantages and disadvantages of different treatment options?	6	21	175	123
4	16	Act on advice from your doctor or pharmacist? (Instruction: do as your doctor suggests)	4	16	178	146
5	18	Find information on how to handle mental health problems? (Instruction: stress, depression, or anxiety)	8	39	143	91
6	23	Understand information about recommended health screenings or examinations? (Instruction: measure blood sugar, measure blood pressure)	4	17	167	137
7	24	Judge if information on unhealthy habits, such as smoking, low physical activity, or drinking too much alcohol, are reliable?	2	24	173	89
8	31	Decide how you can protect yourself from illness using information from the mass media? (Instruction: newspapers, TV, or Internet)	9	31	171	104
9	32	Find information on healthy lifestyles such as physical exercise, healthy food, or nutrition?	3	18	176	116
10	37	Understand advice concerning your health from family or friends?	7	21	167	112
11	42	Judge how your housing conditions may affect your health and well-being?	3	15	182	112
12	44	Make decisions to improve your health and well-being?	0	26	162	126

Psychometric properties of HLS₁₉-Q12 items

Most HLS₁₉-Q12 items displayed acceptable fit to the Rasch model (Table 3), but item 31 (‘decide how you can protect yourself from illness using information from the mass media’) displayed misfit and a tendency to under-discriminate (fit residual of 2.67, Chi sq. P = .003). Item 10 displayed slightly disordered thresholds. Items 10 and 42 displayed non-uniform DIF for self-reported health and employment status, respectively (not reported in the table). Moreover, several items (Items 10, 16, 18, 24, 31, and 32) displayed uniform ccDIF, while other items (10, 16, and 31) displayed both uniform and non-uniform ccDIF for Somali (focal group) versus general population (reference group).

Table 3.

Psychometric properties of HLS₁₉-Q12 items among people with a Somali background living in Norway (n = 368).

Item no.	HLS19 item no.	Item wording	Location	Fit resid	Chi sq	Chi sq prob	ccDIF	Thresholds
1	4	Find out where to get professional help when you are ill? (Instruction: such as doctor, nurse, pharmacy staff, psychologist)	−0.592	−2.700	6.318	0.097	–	–
2	7	Understand information about what to do in a medical emergency?	0.169	−1.702	7.810	0.050	–	–
3	10	Judge the advantages and disadvantages of different treatment options?	0.455	1.953	4.418	0.220	Nor < Soma	Disordered
4	16	Act on advice from your doctor or pharmacist? (Instruction: do as your doctor suggests)	0.067	−2.496	9.808	0.020	Nor > Soma	–
5	18	Find information on how to handle mental health problems? (Instruction: stress, depression or anxiety)	0.683	2.130	8.846	0.031	Nor < Som	–
6	23	Understand information about recommended health screenings or examinations? (Instruction: measure blood sugar, measure blood pressure)	0.066	−1.256	7.157	0.067	–	–
7	24	Judge if information on unhealthy habits, such as smoking, low physical activity or drinking too much alcohol, are reliable?	−0.043	−1.410	0.827	0.843	Nor > Som	–
8	31	Decide how you can protect yourself from illness using information from the mass media? (Instruction: newspapers, TV or Internet)	0.680	2.666	13.813	0.003	Nor < Soma	–
9	32	Find information on healthy lifestyles such as physical exercise, healthy food or nutrition?	0.046	−0.833	1.858	0.602	Nor > Som	–
10	37	Understand advice concerning your health from family or friends?	0.457	0.882	4.514	0.211	–	–
11	42	Judge how your housing conditions may affect your health and well-being?	0.048	−1.459	5.642	0.130	–	–
12	44	Make decisions to improve your health and well-being?	−2.035	−1.026	5.090	0.165	–	–

Chi sq prob, Chi-square probability; ccDIF, cross-cultural differential item functioning in favour of Somali sample (<Som) or general population in Norway (Nor>; n = 5889), DIF, differential item functioning; Fit resid, z-fit residual; Nor, general population in Norway (n = 5889); Som, Somali population in Norway (n = 368). HLS₁₉ item number refer to the original 47 item version (HLS₁₉-Q47).

^aNon-uniform DIF.

CCCI findings

In total, four men and seven women aged 23–58 with Somali background and Norwegian residency participated in the CCCIs. Their length of residency varied between 10 and 26 years. Nine of the participants spoke both Norwegian and Somali, while two needed a Somali-speaking moderator (author 3). Four of the informants had primary school as their highest educational level, three had completed upper secondary school and four informants were bachelor students and/or had completed a bachelor’s degree.

Three themes were generated: (i) language barriers make it challenging to comprehend the items, (ii) cultural differences raise issues in the survey, and (iii) target group and survey mode considerations are critical elements in cross-cultural surveys (Fig. 1).

Figure 1.

Themes and sub-themes.

Language barriers make it challenging to comprehend the items

This theme addresses challenges and issues related to translation problems, linguistic concerns, and the overall barriers associated with using different languages in a survey. It consists of two sub-themes: (i) translation-related issues that resulted in frustration and misunderstandings and (ii) linguistic challenges arising from language-specific words and concepts.

Translation-related issues resulted in frustration and misunderstandings

The informants encountered difficulties in comprehending the translated version of the HLS₁₉-Q12 items. This led to some frustration and for some informants an item resignation. The informants did not, understand the Somali translation of ‘to judge/to consider’ (in Somali tixgelinta). The word was rather understood as ‘to respect’ something. The items that began with this word therefore created confusion and uncertainty for the informants. Furthermore, items containing this word were perceived as statements rather than questions. As one expressed: ‘This is not a question, it is a statement’ (informant 7). The Somali wording of item 42 about how housing conditions affect the respondent’s health and well-being had a different interpretation compared with the Norwegian and English versions. This came as a surprise in the interviews. The Somali version of the item was understood by the informants as a question on one’s health and quality of life, while the original item asked, ‘judge how your housing conditions may affect your health and well-being’.

Some of the informants wondered if the translations of the HLS₁₉-Q12 items were done by Google translate or a machine rather than a translator. Hence, the informants stated that Somali and Norwegian are very different languages, and they clearly pointed out the challenges concerning translation from Norwegian/English to Somali and the importance of non-verbatim awareness. Some informants expressed that the survey should be retranslated before subsequently use.

Linguistic challenges arising from language-specific words and concepts

The informants indicated that certain words, such as the concepts of mental health, stress, depression, anxiety, and mass media, do not exist in the Somali language or its vocabulary. One example is item 18 (‘find information on how to handle mental health problems? Instruction: stress, depression or anxiety’). As one informant expressed that they must rely on the written help instruction in the item to understand the question. One of the informants exemplified this; ‘This is directly translated; the question itself says health in the brain as physical not mental health’ (Informant 3).

Cultural differences raise issues in the survey

This theme addressed the informants’ understanding of the items from a cross-cultural point of view and includes two sub-themes: (i) the interviewer’s impact on honesty and (ii) the items do not align with cultural and religious perspectives.

The interviewer’s impact on honesty

Several informants noted that it was easier to be more honest with an interviewer who was ethnically Norwegian than to someone with a Somali background. This is largely due to the small size of the Somali community in Norway, making it possibly socially and culturally challenging to answer all the questions honestly. As one informant noted:

‘I will not be honest with a fellow Somali. I have a reputation to take care of’ (Informant 2).

Items do not align with cultural and religious perspectives

Some items such as everyday habits influencing health were viewed as culturally and religiously sensitive questions (e.g. alcohol consumption, smoking, and mental health). Several informants indicated challenges concerning honest responses to questions about mental health or alcohol consumption. They anticipated answers that are culturally accepted. The informants distinguished between religious and culturally sensitivity and saw these items to be more about Somali culture than religion.

Target group and survey mode considerations are critical elements in cross-cultural surveys

This theme conveys to the survey mode, the intended audience for the survey, and how the layout influences respondents’ answers. Several informants asked who the survey was intended for. The informants believed that the survey needed to be much more targeted to reach out to a more specific aimed group (in the HLS₁₉ survey everyone between the ages of 16 and 100 could respond). The fact that most informants responded in Somali regardless of their length of residence or age was surprising to them. Informants believed that a young person with long residency would prefer to answer in Norwegian. Some of the informants questioned whether it was a real option to answer in Norwegian. Several informants expressed uncertainty about whether responding in Norwegian was a feasible option, and whether the interviewer in the original HLS₁₉-CATI interviews was sufficiently proficient in Norwegian to conduct the interviews in that language. All the bilingual informants expressed a wish to respond to the survey in Norwegian as they found this version easier to understand. Hence, Norwegian was the preferred language. Most of the informants found the data collection method, CATI, unsuitable as it felt invasive that a fellow Somali conducted the telephone interview. The informants therefore preferred written/online surveys. The informants suggested that the items could be written in Norwegian with a Somali translation below each item. Further, they suggested that information about the survey could be addressed to the Somali community, i.e. through Mosques, Somali organizations, and TV stations, prior to data collection. Thus, increasing the community’s awareness of the survey and promoting more honest answers from the survey’s target group. Some informants informed that they would not select the response category ‘very difficult’. They explained that if something is culturally perceived as very difficult, they can seek assistance from family and friends. Finally, the informants noted that the response categories did not make sense for items that were interpreted as statements as opposed to questions. As one of the informants noted: ‘If it is difficult to understand the item as a question, it is impossible to answer in the categories’ (Informant 8).

Mixed methods result

The sequential mixed methods approach made it possible to explore the results separately and interwoven. It also made it possible to check for undetected/unexpected results and findings both in the quantitative and the qualitative data material. Table 4 summarizes the results from the quantitative and qualitative approaches, and the meta-inference. The results show an overall imbalance and overall lack of harmony in the data set. This is based on the undetected results from the quantitative data when using CCCIs. The CCCIs explored items that had performed unexpectedly when tested against the Rasch model. In addition, the CCCIs gave more in-depth answer to the findings from the Rasch modelling and revealed that more items than those identified by the Rasch modelling are problematic.

Table 4.

Mixed method results.

1. Quantitative method	2. Qualitative method	3. Mixed methods
Rasch modelling	CCCI interviews	Combining process 1 and 2 into meta inferences
Main results: Cross-cultural DIF Under-discriminating item Unordered response categories	Main results: Three themes: Language barriers make it challenging to comprehend the items Cultural differences raise issues in the survey Target group and survey mode considerations are critical elements in cross-cultural surveys	Results: A. CCCI gave in-depth information about item weaknesses revealed from Rasch modelling. B. CCCI revealed that more items than those identified by Rasch modelling were problematic.

Discussion

This study explored the cultural and linguistic appropriateness of the HLS₁₉-Q12 scale items among people with Somali background living in Norway. To our knowledge, this is the first study that directly aims to explore cultural and linguistic appropriateness of the HLS₁₉-Q12 scale items among migrants.

The main findings of the mixed methods results are that the CCCI gave in-depth information about item weaknesses revealed from the Rasch modelling, such as ccDIF. Further, that more items than those identified by the Rasch modelling were problematic. The Rasch modelling indicated clearly that several items displayed ccDIF between the reference group (population sample) and the focal group (Somali sample). This finding indicates that the Somali sample and the population sample perceived these items differently. According to the CCCIs, the ccDIF results came from language barriers (Theme 1), cultural differences in understanding an item (Theme 2), and target group and survey mode issues (Theme 3). Though, the fact that the population sample and Somali focal sample perceived the items differently is in alignment with other research that shows a common challenge when using multiple languages in a survey (Willis 2015). This is also highlighted by Hagen-Zanker et al. (2023) that points out that one word in one language can mean something completely different in another language or not mean anything at all. For our study, the CCCIs displayed that the wording for mental health, anxiety, depression, and stress do not exist in the Somali language according to our informants. Hence, translating concepts that does not exist in all survey languages implies that the items in the original version (Norwegian) and the Somali version were understood differently. Thus, according to Hagen-Zanker et al. (2023), this changes the meaning of an item or the whole experience of an entire survey for the respondents. Highlighted in the research literature, and shown in our mixed methods results, designing scale items and conducting surveys in cross-cultural studies are challenging (Edwards 2014, Pan and Fond 2014, Hagen-Zanker et al. 2023). The ccDIF between the Norwegian population and the Somali focal group also means that we cannot compare HL between Somalis and the general population. Such a comparison would have been biased.

The Rasch modelling indicated a slightly disordered threshold for item 10 (‘judge the advantages and disadvantages of different treatment options’). However, very few Somali respondents in the HLS₁₉-NO survey selected response Category 1 or 2. The informants in the CCCIs informed that they did not understand some of the items as questions. The informants also pointed out that some of the directly translated wording shifted the item from a question to a statement (Theme 1). Therefore, the informants did not use the response categories as intended. Furthermore, the response Category 1 (very difficult) did not apply to the Somali respondents according to the CCCIs informants since they could just ask a friend/family member (Theme 3). The different understanding of response categories is in line with other research that indicates that response categories may work differently in cross-cultural studies (Benítez et al. 2022).

For item 18 (‘find information on how to handle mental health problems’), the Somali sample scored higher compared with the overall population despite the same HL proficiency. From the CCCIs, it was evident that the informants struggled to understand this item due to translation and lack of similar terms in the Somali language (Theme 1). According to Pan and Fond (2014), this is a common challenge. Hence, there can be problems with the informants understanding of the translated questions. Item 18 was also one of the more sensitive questions (Theme 2). Pan and Fond (2014) have pointed out that cultural preferences can influence how an item is understood due to different understandings of different concepts across cultures. Furthermore, an important finding is that the informants are concerned about being honest with ‘a fellow Somali’, especially when asking culturally sensitive questions. Consequently, the chosen survey mode for the HLS₁₉-NO survey was problematic for the informants (Theme 3). We have not seen this been addressed in any other HL survey.

Item 31 (‘decide how you can protect yourself from illness using information from the mass media’) displayed statistically significant misfit to the Rasch model at Bonferroni-adjusted 5% level. It is worth noting that Item 31 also displayed misfit in several European countries (Guttersrud et al. 2023). Hence, this item could be considered a challenge regardless of cultures and languages exploration. However, through the CCCI, this study can provide more insight, at least for the current target group. The informants in the CCCIs expressed that they did not understand the word mass media (Theme 1) and therefore did not understand the translation or the question. Item 31 also tended to under-discriminate, which means the item tapping into one or other constructs (Masters 1988), such as whether mass media could be considered a reliable source for health information.

Even though most of the HLS₁₉-Q12 items in Somali performed statistically well in the Rasch modelling, the CCCI results indicated that the items were not quite understood or understood differently than the Norwegian HLS₁₉-Q12 version. All the informants interpreted Item 42 (‘judge how your housing conditions may affect your health and well-being’) to be about health and well-being. The Somali version of the item was reported as not asking about housing conditions at all. This might lead the Somali respondents in the survey to answer a question that was unmatched in the original Norwegian scale items. This is commonly understood as a serious translation shortage (Pan and Fond 2014, Willis 2015). On the other hand, it made it possible for us to flag the item as problematic.

One of the results from the CCCIs (Theme 3) was that the CATI as data collection mode for this target group was unsuitable. This was due to difficulties concerning the informants’ honest response to an interviewer from the Somali community. Other research suggests that mode suitable for the general population may not be optimal for a specific population group (Edwards 2014). Our overall findings underline the complexity and variability of cross-cultural surveys. Designing and conducting cross-cultural studies is challenging. This is also highlighted in other studies (Edwards 2014, Willis 2015, Hagen-Zanker et al. 2023). In line with Pan and Fond (2014), our study demonstrates the importance to explore the scale items culturally and linguistically for better quality and better-quality outcome. This is by Edwards (2014) referred to as harmonized data set outcome. To our knowledge, there is not a consensus of cross-cultural survey procedure, which is also asked for in another study (Gustafsdottir et al. 2020). However, as indicated in other research studies, the importance of including the target audience early in the translations is essential in cross-cultural HL surveys (Edwards 2014, Gustafsdottir et al. 2020). Pretesting, and not only post-testing, of such perceptions-based measures is therefore of great importance in cross-cultural studies. HLS₁₉-Q12 is a self-reporting and perceptions-based scale, and the CCCIs results highlighted shortcomings in these types of measuring instruments. Responses to an item depend on how the item content is interpreted. Hence, it is crucial to ensure high-quality translations to avoid misunderstandings and to ensure that items are interpreted consistently across languages (Zhao et al. 2024).

Based on these findings, the construct and content validity of the Somali version of the HLS₁₉-Q12 may be compromised. Validity breaching can be discussed through both a quantitative and qualitative validity lens. In this study, construct validity was assessed by using Rasch modelling. The Rasch modelling indicated that several items displayed ccDIF; Item 10 indicated slightly disordered threshold and 31 demonstrated misfit. In the qualitative data, and through the CCCIs, we sought to establish content validity (Knafl et al. 2007, Willis 2015). In cross-cultural surveys, the validity of the translations is particularly crucial (Nkwake 2023, Zhao et al. 2024), as this concerns the equivalence between the original and translated items—that is, whether the original and translated versions are expressed and understood in a similar way (Cruchinho et al. 2024, Zhao et al. 2024). The CCCIs analysis suggested that several HLS₁₉-Q12 items were not interpreted as intended. This indicates uncertainty between the translated scale items version and the original scale item. Consequently, translational validity is not fulfilled (Cruchinho et al. 2024, Zhoe et al 2024). The Somali version of the HLS₁₉-Q12 scale items was not interpreted as intended. As a result, the Somali version does not fully operationalize the construct of general HL.

Strengths and limitations of the study

Our study has several strengths and limitations. One of the strengths of this study is the use of mixed methods design. One purpose for our use of sequential mixed methods design was to explain quantitative results by means of qualitative methods. The use of mixed methods and secondary data is also a strength (Cohen et al. 2018, pp. 587; Fauser 2018), since it gave us an opportunity to have a larger sample size and conduct further alternatives studies such as the CCCIs. We argue that this study is strengthened by using mixed methods. The interdisciplinarity background of the involved authors in this study is another strength, as it gave different perspectives, and a richer and more nuanced RTA analysis. An additional strength of this study is the use of bilingual and bicultural co-workers and a continuous contact with the Somali community. The use of bilingual and bicultural co-workers is in line with WHO recommendation (WHO 2019). Diaz and Benavente (2025) and Laue et al. (2023) emphasize the importance of participatory approaches to better understand migrant health issues as well as a source to reach health equity. A further strength is that the CCCIs inclusion criteria are similar to the participants in the quantitative data set, regarding age, residence time, educational level, etc.

One limitation in our study is the use of secondary data. Using secondary data, we did not have control over the data collection process, including the translation procedure or item wording (Cohen et al. 2018). Another limitation is that some subgroups consisted of relatively few individuals; for example, only 14 individuals reported poor health. As a result, additional DIF may exist that we were unable to identify. The CCCIs sample was recruited through a snowball sampling, which can be an additional limitation. This may lead to the inclusion of informants who are not geographically diverse and who may not adequately represent the broader population in terms of characteristics such as gender and educational background (Lindstrom 2016). This in turn influences the transferability of our study. Moreover, most CCCIs were conducted in Norwegian. The two informants who spoke only Somali were therefore unable to understand or express themselves in Norwegian, which may have introduced inaccuracies in translation. Furthermore, the use of a Somali-speaking moderator may have influenced the data, as informants might have withheld certain views or meanings due to concerns about full disclosure when interacting with a fellow Somali.

Implications

In line with Pan and Fond (2014) and Vujcich et al. (2021), we argue that one of the most important steps for the research community to improve the quality of migrant HL survey data, as well as other cross-cultural health surveys, are to conduct and publish more testing studies (both pretesting and field testing and evaluating studies). Valid surveys are important for public health policies and as a basis for health promoted interventions. Policy making based on invalid data can be a source of prejudice and discrimination (Auer and Ruedin 2019, Sturge 2022). For future cross-cultural HL surveys, we argue for greater involvement of target groups and the establishment of participatory advisory boards to discuss e.g. data collection modes and procedures. This is important, as the choice of data collection mode has significant implications for survey quality (Edwards 2014). Pretesting of survey translations and their cultural appropriateness should be incorporated at the survey design stage in the original language, with an explicit focus on translatability as well as translation quality (Edwards 2014, Hagen-Zanker et al. 2023). Future research should examine whether cultural and linguistic challenges are confined to specific HLS₁₉ scale items and particular target groups, or whether such issues affect a broader range of items within the HLS₁₉ survey or extend to other migrant groups.

Conclusion

This study reveals that there were major challenges with the Somali version of the HLS₁₉-Q12 scale items. To address our aim, the scale items cannot be seen as culturally or linguistically appropriate for the informants from a psychometric quantitative, CCCI qualitative or a mixed methods analyses perspective. This emphasizes the importance of evaluating and examining the results of cross-cultural HL studies in journals and research milieus and to a much greater extend involve the target groups in the pretesting phase. Good quality data are difficult to access for decision makers (Sturge 2022); therefore, the importance of having reliable and valid scale items for HL and health equity policy making cannot be stressed enough.

Author Contributions

Kathrine Krüger Østbøll (Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Validation, Writing—original draft), Øystein Guttersrud (Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Validation, Writing—review & editing), Falastin Alasoow (Data curation, Writing—review & editing), Christopher Le (Data curation, Writing—review & editing), Josefin Wångdahl (Conceptualization, Formal analysis, Methodology, Supervision, Writing—review & editing), and Hanne Søberg Finbraten (Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Validation, Writing—review & editing)

Supplementary material

Supplementary material is available at Health Promotion International online.

Conflicts of interest

The authors declare that they have no competing interest.

Funding

The quantitative data collection was supported by the Norwegian Directorate of Health, while qualitative data collection was supported by University of Inland Norway.

Data availability

The dataset used and analysed during the current study is available from the corresponding author upon request.