Evaluation of the psychometric properties of the Brazilian version of the Digital Health Literacy Instrument for individuals with heart failure

Abstract

Background

Digital health literacy is a critical skill for heart failure self-care, which should be assessed through valid and reliable instruments.

Objective

To evaluate the psychometric properties of the Brazilian version of the Digital Health Literacy Instrument in individuals with heart failure.

Design

A psychometric study analyzing internal structure validity and reliability.

Settings

The participants were recruited at the cardiomyopathy outpatient clinic of a public hospital in São Paulo, Brazil.

Participants

Outpatients with heart failure who used the internet.

Methods

The instrument was administered to 127 individuals, with bootstrapping to 500. Internal structure validity and reliability were assessed through Exploratory and Confirmatory Factor Analyses. Dimensionality was determined via Parallel Analysis, and data extraction employed Robust MORGANA factor analysis. Measure of Sampling Adequacy and Kaiser-Meyer-Olkin were expected to be close to 1 and Bartlett’s test should have p < 0.05. Target explained variance was approximately 60 %, with factor loadings >0.3 and communalities >0.4. Fit indices (Comparative Fit Index, Tucker-Lewis Index and Goodness of Fit Index) should be approximately 1 and Root Mean Square Error of Approximation should be 0.05–0.08. Reliability was measured using ordinal alpha and McDonald’s omega (>0.9).

Results

A unidimensional model emerged. Five items were removed due to factor loadings <0.2, Heywood case, excessive residuals, and double saturation. The final 16-item model had Kaiser-Meyer-Olkin = 0.86, Measure of Sampling Adequacy >0.75, explained variance = 59 %, factor loadings 0.53–0.86, and communalities 0.28–0.74. Reliability was high (alpha= 0.94, omega= 0.95). The adjusted model showed Comparative Fit Index = 0.99, Tucker-Lewis Index = 0.99 and Goodness of Fit Index = 0.98 and Root Mean Square Error of Approximation = 0.07.

Conclusions

The Brazilian version of the Digital Health Literacy Instrument demonstrated strong internal structure validity and reliability for individuals with heart failure, supporting its use in research and clinical practice.

What is already known about the topic.

• ••Digital health literacy is essential for navigating online health information and making informed health decisions.
• ••The Digital Health Literacy Instrument is used to measure digital health literacy 1.0 and 2.0 but it is not validated for Brazilians with heart failure.

Alt-text: Unlabelled box

What this paper adds.

• ••A robust psychometric evaluation of the Brazilian version of The Digital Health Literacy Instrument showed that it is a valid and reliable tool for patients with heart failure.
• ••The exploratory factor analysis supported a unidimensional structure of the Digital Health Literacy Instrument in Brazil.
• ••The Br-Digital Health Literacy Instrument can support assessment of digital health literacy and guide interventions to improve confidence, self-care, and self-management.

Alt-text: Unlabelled box

1. Introduction

The early 21st century saw the emergence of Web 2.0, a transformative shift that empowered users to actively engage with health information by not only consuming but also creating and disseminating it - Health 2.0. (Belk, 2014; van De Belt et al., 2010; van der Vaart and Drossaert, 2017; van der Vaart et al., 2013). Consequently, the internet has become the primary source of health information (Sun et al., 2022), yet its widespread availability, without adequate comprehension and evaluation, impacts individual and community healthcare (van der Vaart and Drossaert, 2017; Battineni et al., 2020; Freire et al., 2021; Yeung et al., 2022). While information dissemination enhances communication and empowers individuals, it also heightens the risk of medical misinformation, as seen during the COVID-19 pandemic (Yeung et al., 2022).

Digital health literacy refers to the ability to effectively access, understand, appraise, and utilize digital health information for decision-making (van der Vaart and Drossaert, 2017; Norman and Skinner, 2006). Enhanced digital health literacy not only supports individual self-management but also contributes to broader public health, innovation, and economic progress (Civil Society Working Group, 2024). To support digital integration in healthcare, the Brazilian Ministry of Health launched the Digital Health Strategy for Brazil 2020–2028, aiming to expand telemedicine and digital health access (Brasil, 2020). Assessing and improving digital health literacy is critical for its success, requiring validated instruments.

The Digital Health Literacy Instrument (DHLI), developed in the Netherlands, evaluates both digital health literacy 1.0 and 2.0, encompassing competencies related to searching, evaluating, interacting, and safely sharing information in digital environments (van De Belt et al., 2010; van der Vaart and Drossaert, 2017; Xie et al., 2024). Developed from empirical studies on digital engagement among patients with rheumatic conditions, the DHLI assesses seven skill areas essential for interacting with online health environments. Each domain is evaluated through three self-assessment items, totaling 21 questions. Responses are recorded on a four-point scale, reflecting the perceived difficulty of each digital health task. These skills include technical use, web navigation, search strategies, critical evaluation of information (both general reliability and personal relevance), content generation, and privacy management (van der Vaart et al., 2013). Each of these seven competencies is assessed through three self-report items, resulting in a 21-item scale that captures users’ perceived difficulty and challenges encountered in digital health environments. The instrument is rated on a four-point Likert scale, with higher scores indicating higher levels of digital health literacy (van der Vaart and Drossaert, 2017).

The DHLI has been adapted and validated in multiple countries, including Germany (Dadaczynski et al., 2021), South Korea (Chun et al., 2022; Kim et al., 2021), Italy (Lorini et al., 2022), Portugal (Martins et al., 2022; Rosário et al., 2020), Slovenia (Vrdelja et al., 2021), Spain (Rivadeneira et al., 2022), Turkey (Çetin and Gümüş, 2023), China (Xie et al., 2024), and Japan (Mitsutake et al., 2024; Miyawaki et al., 2024), with variations for university students, adults, and older populations. In Brazil, it has been validated for adolescents (Barbosa et al., 2024) and adults with hypertension and diabetes (unpublished data) and demonstrated content validity for individuals with heart failure (Aprile et al., 2024). Higher digital health literacy levels in patients with heart failure correlate with better disease knowledge, confidence in self-care, and improved management, addressing challenges such as low adherence to self-care guidelines, hospitalizations, and mortality (Chuang et al., 2019; Riegel et al., 2022; Fabbri et al., 2020; Nesbitt et al., 2021).

The prevalence of heart failure in Brazil rose from 0.67 million in 1990 to 1.7 million in 2017, driven by population aging (Oliveira et al., 2022), which can contribute to higher hospital mortality and economic burdens (Alghamdi et al., 2021; Bundgaard et al., 2022; Steen Carlsson et al., 2024). Considering the growing burden of heart failure in Brazil, particularly among aging populations, strategies are needed to enhance heart failure management, reduce hospitalizations, and alleviate financial strain (Arruda et al., 2022; Alghamdi et al., 2021). The validation of a contextually appropriate digital health literacy assessment tool is critical in the country. The DHLI offers a pathway to identify DHL gaps and inform targeted interventions for individuals with heart failure.

2. Methods

A psychometric study (Brown, 2015; Hair et al., 2019; Lorenzo-Seva and Ferrando, 2021; Timmerman and Lorenzo-Seva, 2011) was conducted to assess the internal structure validity and reliability of the DHLI for individuals with heart failure.

2.1. Ethical aspects

Permission to adapt and assess the psychometric properties of the instrument was obtained from the authors of the original instrument. The project was submitted to and approved by the Research Ethics Committee at Universidade Federal de Sao Paulo (Protocol 3.346.051). All participants provided informed consent and were assured of their anonymity. They were also informed of their right to withdraw from the study at any time without any consequences.

2.2. Inclusion criteria

Participants were required to have a diagnosis of heart failure, the ability to speak and read Brazilian Portuguese, and internet usage skills. Individuals with visual impairments that hindered reading the instrument, severe cognitive impairment, or psychiatric conditions (as indicated in their medical records) were excluded from the study.

2.3. Data collection

A convenience sample was recruited in person at the cardiomyopathy outpatient clinic of a public hospital in São Paulo, Brazil, between June and December 2023. The Brazilian version of the DHLI, adapted for individuals with heart failure and with its content validity tested (Aprile et al., 2024), was administered to 127 individuals with heart failure. Participants also completed a sociodemographic and health questionnaire for sample characterization.

The DHLI - Brazilian version, whose content validity was previously assessed by content experts, comprises 21 self-reported items distributed across seven domains: Operational Skills, Navigation Skills, Information Searching, Evaluation of Reliability, Determination of Relevance, Addition of Self-Generated Content, and Privacy Protection. Respondents rate the perceived difficulty or frequency of completing various digital tasks using a 4-point Likert scale, where higher scores indicate greater difficulty. Experts considered this version to be clear, with items demonstrating both theoretical relevance and practical pertinence (Aprile et al., 2024).

Regarding conventions on ideal sample size, minimum sample requirements, or observation-to-parameter ratios, some authors recommend minimum sample sizes ranging from 50 to 1000 participants or participant-to-item ratios of 5:1, 10:1, or even 20:1 (Costello and Osborne, 2005; Gaskin and Happell, 2014; Howard, 2016; Rogers, 2022). In the present study, we adopted a ratio of 5:1. Additionally, the bootstrapping method was employed, as recommended in contemporary psychometric analyses (Goretzko and Bühner, 2022), to enhance the reliability of the results in both small and large samples.

2.4. Statistical analysis

The demographic and clinical characteristics of the participants, along with the DHLI scores, were entered into a Microsoft Office Excel 16.3 spreadsheet for analysis. Descriptive statistics were used to analyze the data. Categorical variables were presented as absolute and relative frequencies (n,%), while continuous variables were expressed as mean ± standard deviation.

The psychometric analyses were conducted using FACTOR^Ⓡ software (version 12.04.05) and JASP^Ⓡ, with bootstrapping with 500 resamples (Goretzko and Bühner, 2022). The Measure of Sampling Adequacy (MSA) was assessed using the Robust MSA correlation test, with values exceeding 0.5 considered acceptable (Lorenzo-Seva and Ferrando, 2021), alongside the Kaiser-Meyer-Olkin (KMO) test, where values closer to 1 indicate adequacy (Kaiser, 1974). Bartlett’s test of sphericity was applied, with p-values < 0.05 considered significant (Bartlett, 1950).

Both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed (Brown, 2015). Although the original instrument had adequate validity and reliability, contemporary recommendations for establishing internal structure validity suggest using a combination of analytical techniques to ensure a robust assessment of the evidence from multiple perspectives (Bandalos, 2018). Due to cultural differences and population-specific factors that may influence how individuals interpret and respond to items, there is no guarantee that an instrument will retain the same psychometric properties as the original version. Therefore, the exploratory factor analysis was used to assess whether the same constructs would emerge in the Brazilian sample or if a different structure would better fit the data without imposing a predefined model, thereby ensuring the structure would be empirically grounded in the new context. The confirmatory factor analysis then tested how well our proposed factor structure fit the data.

In the exploratory factor analysis, dimensionality was examined using several methods: the Optimal Implementation of Parallel Analysis (PA) (Timmerman and Lorenzo-Seva, 2011), the Minimum Average Partial Test (MAP) (Velicer, 1976), the HULL Method for determining the number of common factors (Lorenzo-Seva et al., 2011), and the Closeness of Dimensionality test (Ferrando and Lorenzo-Seva, 2018). Data extraction was conducted via Robust MORGANA factor analysis (Ferrando et al., 2023), followed by oblique rotation using the Promin method to enhance factor simplicity (Lorenzo-Seva, 1999). Residuals, including doublets, were addressed through the Expected REsidual correlation direct Change (EREC) method (Ferrando et al., 2022).

Item retention or removal in exploratory factor analysis was based on criteria including factor loadings (>0.3), communalities (>0.4), explained variance (∼60 %), and the absence of doublets or cross-loadings, ensuring an optimal model fit (Costello and Osborne, 2005; Hair et al., 2019).

For the confirmatory factor analysis, all analyses adhered to the structural equation modeling (SEM) framework, evaluating model fit with several indices: chi-square (χ²) with degrees of freedom (df), Comparative Fit Index (CFI) , Tucker-Lewis Index (TLI) , Goodness of Fit Index (GFI) , Root Mean Square Residual (RMSR) , and Root Mean Square Error of Approximation (RMSEA). A model was considered to fit the data well if Comparative Fit Index , Goodness of Fit Index, and Tucker-Lewis Index values were ≥ 0.90, and if the Root Mean Square Error of Approximation and Root Mean Square Residual values were < 0.08 (Brown, 2015; Hu and Bentler, 1999).

Reliability was assessed by calculating internal consistency using ordinal alpha coefficients (Dominguez, 2012; Elosua and Zumbo, 2008) and McDonald’s Omega (McDonald, 1999), with values > 0.9 considered excellent for both metrics (Dominguez, 2012; Elosua and Zumbo, 2008; McDonald, 1999).

3. Results

3.1. Descriptive analysis

The characteristics of the 127 participants are summarized in Table 1. The majority were male (54.3 %), with a mean age of 55.6 ± 13.4 years. Most participants accessed the internet primarily via mobile phones (95.6 %) and did so daily (92.1 %). Additionally, a substantial proportion self-reported their digital skills as moderate (40.2 %).

Table 1.

Sociodemographic and clinical characteristics of participants with heart failure (n = 127).

Variable	Participants (n = 127)
	Mean	SD
Age	55.6	13.4
Left ventricular ejection fraction	47	13.6
	N	%
Sex
Female	58	45.7
Male	69	54.3
Highest educational level
Incomplete primary education	35	27.6
Complete primary education	23	18.1
Incomplete secondary education	9	7.1
Complete secondary education	28	22.0
Incomplete higher education	11	8.7
Complete higher education	21	16.5
Marital status
Married	59	46.5
Single	36	28.3
Divorced	13	10.2
Widowed	11	8.7
Civil union	8	6.3
Heart failure etiology
Ischemic	27	21.3
Chagas’ disease	15	11.8
Hypertensive	13	10.2
Idiopathic	19	15.0
Others	32	25.2
Functional class of the New York Heart Association
I	49	38.6
II	49	38.6
III	7	5.5
Means of internet accessa
Mobile phone	124	97.6
Personal computer	24	18.9
Laptop	23	18.1
Work computer	17	13.4
Frequency of internet use
Every day	117	92.1
4 days per week	4	3.1
3 days or less per week	4	3.1
1 day or less per week	2	1.6
Self-Assessed internet skills
Excellent	15	11.8
Good	41	32.3
Average	51	40.2
Fair	13	10.2
Poor	7	5.5
Have you ever used the internet toa
Read a health-related text	104	81.9
Search for information about health or illness	91	71.7
Read a forum post or social media content related to health	69	54.3
Schedule appointments with a healthcare provider	61	48.0

SD: standard deviation; more than one response was possible^a.

3.2. Internal structure validity

Unlike the original model with seven factors and 21 items, the exploratory factor analysis yielded a unidimensional model with 16 items. Bartlett’s test was significant (χ² = 1187.8, df = 210, p < 0.00001), and the KMO measure was 0.88, indicating an adequate sample size. However, item 20 did not meet factorability criteria (MSA < 0.5) (Lorenzo-Seva and Ferrando, 2021). Dimensionality analysis using PA (Timmerman and Lorenzo-Seva, 2011) confirmed a unifactorial model, further supported by the MAP test, HULL method, and Closeness of Dimensionality test (Velicer, 1976; Lorenzo-Seva et al., 2011; Ferrando and Lorenzo-Seva, 2018), except for items 20 and 21.

The seven-factor model exhibited inconsistencies, including a Heywood case for item 2 (Wang et al., 2023) and cross-loadings, particularly involving item 20. Communalities were acceptable (>0.40), but the explained variance (47.5 %) was below the ideal threshold (Hair et al., 2019). Residual analysis (EREC) (Ferrando et al., 2022) identified eight doublets, three involving item 20 and two involving item 18.

In the unidimensional model, item 20 was removed due to lack of factorability (Lorenzo-Seva and Ferrando, 2021), and item 21 for failing to support unidimensionality (Ferrando and Lorenzo-Seva, 2018). Subsequent analyses led to the exclusion of item 2 (Heywood case), item 18 (doublet involvement), and item 19 (communality < 0.2). These removals increased explained variance from 47.5 % to 59.1 %, with factor loadings from 0.53 to 0.86 and communalities from 0.28 to 0.74 (Hair et al., 2019). Table 2 presents the factor loadings and communalities for both the initial and the adjusted unidimensional models. Residual correlation analysis confirmed the absence of doublets in the final model (Ferrando et al., 2022).

Table 2.

Factor loadings and communalities of the initial 21-item unidimensional model compared to the final 16-item unidimensional model (excluding items 2, 18, 19, 20, and 21).

	Initial model		Adjusted model
Item	Factor loadings	h2	Factor loadings	h2
01) usar o teclado de um computador (use the keyboard of a computer)	0.76	0.58	0.77	0.59
02) usar o mouse (use the mouse)	0.65	0.42	–	–
03) usar os botões ou links nas páginas da internet (use the buttons or links and hyperlinks on websites)	0.80	0.64	0.76	0.58
04) escolher uma informação entre todas as encontradas (make a choice from all the information you find)	0.83	0.69	0.84	0.70
05) usar as palavras ou frases de busca adequadas para encontrar a informação que você procura (use the proper words or search query to find the information you are looking for)	0.85	0.73	0.86	0.74
06) encontrar a informação exata que você procura (find the exact information you are looking for)	0.83	0.69	0.85	0.73
07) decidir se a informação encontrada é confiável ou não (decide whether the information is reliable or not)	0.66	0.44	0.69	0.48
08) decidir se a informação foi escrita com interesses comerciais (decide whether the information is written with commercial interests)	0.77	0.60	0.77	0.60
09) verificar diferentes páginas da internet para ver se fornecem a mesma informação (check different websites to see whether they provide the same information)	0.76	0.58	0.75	0.57
10) decidir se a informação que você encontrou é aplicável a você (to decide if the information you found is applicable to you)	0.58	0.34	0.60	0.36
11) aplicar a informação que você encontrou no seu dia a dia (to apply the information you found in your daily life)	0.47	0.22	0.67	0.44
12) usar a informação que você encontrou para tomar decisões sobre sua saúde (to use the information you found to make decisions about your health)	0.66	0.43	0.66	0.44
13) você perder a noção de onde você está na página ou na internet (you lose track of where you are on a website or the internet)	0.63	0.40	0.61	0.37
14) você não saber como retornar a uma página anterior (you do not know how to return to a previous page)	0.57	0.33	0.57	0.32
15) você clicar em alguma coisa e ver algo diferente do que você esperava (you click on something and get to see something different than you expected)	0.55	0.31	0.53	0.28
16) formular claramente sua pergunta ou preocupação relacionada à saúde (clearly formulate your question or health-related worry)	0.79	0.63	0.77	0.60
17) expressar sua opinião, pensamentos ou sentimentos por escrito (express your opinion, thoughts or feelings in writing)	0.71	0.51	0.67	0.45
18) escrever sua mensagem para que as pessoas entendam exatamente o que você quer dizer (write your message as such, for people to understand exactly what you mean)	0.73	0.53	–	–
19) você acha difícil saber quem poderá acessar (do you find it difficult to judge who can read along)	0.37	0.13	–	–
20) você compartilha suas informações privadas, por exemplo, seu nome ou endereço, com ou sem intenção (do you - intentionally or unintentionally - share your own private information)	0.10	0.01	–	–
21) você compartilha informações pessoais de outras pessoas, com ou sem intenção (do you - intentionally or unintentionally - share some else’s private information)	0.34	0.12	–	–

h²⁼ communalities.

In the confirmatory factor analysis, three models were compared: the original seven-domain model (21 items), the initial unidimensional model (21 items), and the adjusted unidimensional model (16 items). Both 21-item models exhibited inconsistent sample adequacy measures, with KMO and Bartlett’s test yielding unrepresentable values. In contrast, the 16-item model showed appropriate sampling and modeling adequacy (KMO = 0.86, χ² = 1683.84, df = 120, p < 0.0001). Fit indices confirmed the adjusted unidimensional model’s superior fit: GFI = 0.98, CFI = 0.99, TLI = 0.99, RMSEA = 0.07, RMSR = 0.07 (Table 3).

Table 3.

Comparisons between the measure of sampling adequacy and fit indices for the original multidimensional model, initial unidimensional model (21 items), and final unidimensional model (16 items).

MSA/adjust index	Models
	Multidimensional (21 items)	Unidimensional (21 items)	Unidimensional (16 items)
KMO	NaN	NaN	0.86
x2	NaN	NaN	1683.84
df	210	210	120
p	NaN	NaN	<0.0001
CFI	1	0.98	0.99
TLI	1	0.98	0.99
GFI	0.99	0.97	0.98
RMSEA	0	0.09	0.07
RMSR	0.07	0.1	0.07

df: Degrees of freedom / MSA: Measure Sampling Adequation / NaN: not a number /.

CFI: Comparative Fit Index / GFI: Goodness of Fit Index / TLI: Tucker-Lewis Index / RMSEA: Root Mean Square Error of Approximation/ RMSR: Root Mean Square Residual.

Standard errors for items 19, 20, and 21 in the original model exceeded 0.07 but improved in the unidimensional model. However, these items still showed factor loadings below 0.50, particularly item 20 (0.12). After removing items 2, 18, 19, 20, and 21, all factor loadings were adequate (0.57–0.90) (Fig. 1), and measurement errors were reduced (0.02–0.03) (Table 4).

Fig. 1.

Factor loadings and structure of the confirmatory factor analysis for the unidimensional model with 16 items.

Table 4.

Comparisons between factor loadings and standard errors of the original multidimensional model, initial unidimensional model, and final unidimensional model.

	Models
	Original multidimensional (21 items)		Initial unidimensional (21 items)		Final unidimensional (16 items)
Item	loading	Standard error	loading	standard error	loading	standard error
1	0.94	0.03	0.84	0.02	0.82	0.02
2	0.76	0.03	0.68	0.02	–	–
3	0.93	0.03	0.83	0.02	0.83	0.02
4	0.86	0.02	0.82	0.02	0.82	0.02
5	0.93	0.02	0.89	0.02	0.9	0.02
6	0.91	0.02	0.88	0.02	0.89	0.02
7	0.7	0.03	0.66	0.02	0.68	0.02
8	0.83	0.03	0.78	0.02	0.79	0.02
9	0.81	0.03	0.76	0.02	0.77	0.02
10	0.74	0.03	0.65	0.02	0.67	0.02
11	0.83	0.03	0.73	0.02	0.73	0.02
12	0.78	0.03	0.69	0.02	0.7	0.03
13	0.72	0.04	0.61	0.02	0.62	0.03
14	0.65	0.04	0.56	0.03	0.57	0.03
15	0.68	0.04	0.58	0.02	0.57	0.03
16	0.9	0.02	0.81	0.02	0.82	0.02
17	0.93	0.02	0.89	0.02	0.79	0.02
18	0.94	0.02	0.88	0.02	–	–
19	0.7	0.09	0.39	0.03	–	–
20	0.37	0.07	0.12	0.04	–	–
21	0.63	0.08	0.33	0.04	–	–

3.3. Reliability

The ordinal alpha and McDonald's omega of the instrument were 0.944 and 0.945, respectively, indicating excellent internal consistency (Dominguez, 2012; Elosua and Zumbo, 2008; McDonald, 1999). The final Brazilian version of the DHLI was named Br-DHLI (Supplementary material 1).

4. Discussion

This study is the first to assess the psychometric properties of the Br-DHLI in Brazilian individuals with heat failure. Through rigorous statistical analysis, a unidimensional 16-item model was identified, demonstrating high reliability and strong internal structure validity, supported by appropriate model fit indices.

Enhancing digital health literacy fosters informed societies and healthier individuals, promoting sustainable economic growth and inclusive innovation (Civil Society Working Group, 2024). Health literacy, a key factor in heart failure management, should be assessed using reliable instruments with robust validity evidence, given its direct positive impact on self-care (Chuang et al., 2019). The DHLI, developed within the Health 2.0 context (van der Vaart and Drossaert, 2017), minimizes self-report biases by evaluating individuals' perceptions and behaviors in seeking health information (Xie et al., 2024).

Several studies have demonstrated the validity of the DHLI in other countries. Most studies (Dadaczynski et al., 2021; Chun et al., 2022; Lorini et al., 2022; Martins et al., 2022; Rosário et al., 2020; Rivadeneira et al., 2022) used a shortened, online version targeting students, adapting five domains for the Covid-19 context (Dadaczynski et al., 2020), assuming that "digital natives" did not need assessment of "operational skills" and "navigation skills", which were presumed to be highly developed in this population.

Conversely, also using a digital version of the questionnaire, studies on adults over 60 years old emphasized the need to assess all domains, particularly operational skills, which are crucial for using the internet to search for health information, especially for individuals less accustomed to the virtual environment (Kim et al., 2021; Mitsutake et al., 2024; Miyawaki et al., 2024). A Chinese study revised the instrument by removing the "privacy protection" domain, resulting in 18 items and six factors (Xie et al., 2024). In Brazil, the full instrument was used for individuals with hypertension, diabetes (unpublished), and adolescents (Barbosa et al., 2024).

This study used the full seven-domain instrument, applied in-person to adults with heart failure. However, a unidimensional model was confirmed using robust dimensionality tests, including Parallel Analysis (Timmerman and Lorenzo-Seva, 2011) MAP (Velicer, 1976), the HULL method (Lorenzo-Seva et al., 2011) and the Closeness of Dimensionality test (Ferrando and Lorenzo-Seva, 2018). Several studies have yielded versions of the DHLI with fewer dimensions than the original seven. For example, adaptations have resulted in six factors (Çetin et al., 2023; Barbosa, 2023), five (Kim et al., 2021; Rivadeneira et al., 2022; Amador, 2021), four (e.g., Chun et al., 2022; Dadaczynski et al., 2021; Lorini et al., 2022; Martins et al., 2022; Rosario et al., 2020), and even three (Vrdelja et al., 2021; Xie et al., 2024). Notably, Kim et al. (2021) observed that items originally mapped to “Information searching”, “Determining relevance”, and “Evaluating reliability” were better represented under a unified dimension they labeled “Information searching”. These differences are expected, as the internal structure of an instrument may manifest differently in diverse populations.

A detailed statistical analysis revealed that the variation in eigenvalues across the factors in the original model (7.58 to 0.82) was excessive, and the eigenvalues for the dimensions "Determining relevance" and "Protecting privacy" (<1) suggested that these items could not be grouped into seven dimensions (Kim et al., 2021). Similarly, the Spanish study suggested that the "Protecting privacy" dimension, with an eigenvalue <1, should be discarded and recommended revisiting its semantics for future comparisons (Rivadeneira et al., 2022).

Items 19, 20, and 21 from the "privacy protection" domain exhibited psychometric challenges, including semantic issues highlighted in the literature (Dadaczynski et al., 2021; Lorini et al., 2022; Martins et al., 2022; Rosário et al., 2020; Rivadeneira et al., 2022). For example, item 19, originally worded as “When you post a message on a public webpage, how often do you find it difficult to know if you could access it later?” could be reworded to: “When you post a message on a public webpage, how often do you find it difficult to assess how your private information is protected by the media provider?” This revision would shift the focus to protective measures by internet providers (Dadaczynski et al., 2021). Items 20 - "When you post a message on a public webpage, how often do you share private information, such as your name or address (with or without intention)?” - and 21 - "When you post a message on a public webpage, how often do you share personal information about others (with or without intention)?" - also received criticism for their semantics, which could lead to confusion or misinterpretation (Dadaczynski et al., 2021; Lorini et al., 2022; Martins et al., 2022; Rosário et al., 2020; Rivadeneira et al., 2022).

Likewise, in this study, items 19, 20, and 21 were removed also due to both the semantic issues highlighted in the literature and psychometric inconsistencies. Specifically, in the initial analysis of the Br-DHLI sample adequacy measures in the multidimensional model, item 20 failed to factor (MSA <0.5). Item 19 also failed to factor in the initial unidimensional model (MSA <0.5) and exhibited inadequate communality values (<0.2) (Lorenzo-Seva and Ferrando, 2021; Costello and Osborne, 2005). Item 21 did not support unidimensionality according to Closeness of Dimensionality (Ferrando and Lorenzo-Seva, 2018) and was involved in residual doublets, observed in the exploratory factor analysis. For instance, items 17 (“…express your opinion, thoughts, or feelings in writing”) and 18 (“…write your message in a way that people understand exactly what you mean”) were complementary within a domain, but their similar wording made them sound redundant to respondents, suggesting they measured the same latent variable, leading to overestimation of the results (Ferrando et al., 2022). Items 19, 20, and 21 in the confirmatory factor analysis also exhibited higher standard errors and remained inconsistent, even within the unidimensional model, with inadequate factor loadings and model fit indices.

Items 2 and 18 were also removed in this study. Item 2, which asked about the difficulty of using a mouse, presented inconsistencies, as 97.6 % of participants with heart failure accessed the internet primarily through mobile phones. This reflects broader trends in Brazil, where the proportion of households with computers or tablets decreased from 45.9 % in 2016 to 39 % in 2023, while households with mobile phones increased from 93.1 % to 96.7 % during the same period (Brasil, 2023). Additionally, showed inconsistencies in the exploratory factor analysis and confirmatory factor analysis, including its status as a Heywood case (Wang et al., 2023), suggesting invalid interpretation of the results, it was involved in multiple residual pairs and did not support unidimensionality (Ferrando and Lorenzo-Seva, 2018). Item 18, related to message clarity, was also removed for failing to support unidimensionality (Ferrando and Lorenzo-Seva, 2018) and was involved in multiple residual pairs in the exploratory factor analysis.

After removing items 20, 21, 2, 19, and 18, the explained variance increased from 47,5 % to 59.1 %, which is closer to the values typically found in the literature, ranging from 65 % to 75 % (Xie et al., 2024; Kim et al., 2021; Lorini et al., 2022; Rivadeneira et al., 2022; Çetin and Gümüş, 2023). The KMO and Bartlett’s test also improved, indicating the factorability of the model (Kaiser, 1974; Bartlett, 1950).

These items were removed in an effort to balance quantitative (statistical) and qualitative (item meaning) principles. The decision-making process followed a rationale based on the adequacy of factor loadings and communalities, explained variance, and the absence of doublets, aiming to achieve a more precise model. The parsimony applied in item removal did not compromise the theoretical integrity of the instrument, especially due to the sample size (Gaskin & Happell, 2014).

In the CFA, the seven-factor model failed to meet sample adequacy criteria, with unrealistic fit indices (CFI and TLI both equal to 1, and RMSEA of 0). In contrast, the unidimensional model with 21 items showed improvements but still had high RMSR (0.10) and RMSEA (0.09), above ideal thresholds. After removing problematic items, the 16-item unidimensional model showed a factorable solution with improved fit indices (KMO = 0.86, CFI = 0.99, TLI = 0.99, RMSEA = 0.07, RMSR = 0.07). No further items were removed to maintain the instrument's theoretical integrity (parsimonious approach), considering the small sample size (Bandalos, 2018).

Regarding the instrument's reliability, in this study, both the ordinal alpha and McDonald’s omega values showed adequate internal consistency. In this study, the Likert scale contained four categories, and five items were removed during the statistical analysis. While this removal might have reduced the overall reliability of the instrument, the use of McDonald’s omega and ordinal alpha was considered more appropriate for psychometric analyses in this context, as they provide more reliable estimates, especially in cases where fewer response categories are present (Elosua and Zumbo, 2008; Bandalos, 2018).

The availability of an adapted version of the DHLI for the Brazilian context, with solid evidence of validity and reliability, is crucial for capturing the cultural and linguistic nuances unique to Brazil. This ensures a more accurate assessment of digital health literacy in Brazilian individuals with heart failure. The instrument will facilitate the evaluation of digital health literacy in this population, potentially guiding interventions that enhance confidence, self-care maintenance, and self-management (Chuang et al., 2019). Furthermore, it aligns with Brazil's Digital Health Strategy for 2020–2028 (Brasil, 2020). Future studies will seek to further validate the instrument across diverse contexts and compare the results with findings from other populations. In particular, subsequent validation efforts will focus on assessing validity based on relationships with other variables, including discriminant validity across relevant sociodemographic and clinical subgroups.

4.1. Limitations

This study was conducted exclusively in an outpatient clinic in the southeastern region of Brazil, limiting the generalizability of the findings. Future research on the psychometric properties of the Br-DHLI should ideally involve participants from all regions of the country to enhance external validity. Additionally, the second part of the questionnaire, which focused on individual performance, was excluded from the validity evidence analysis based on the authors' recommendation to treat this section as separate subscales. Future studies with larger samples could assess the validity of these items within the Brazilian context, as objectively evaluating digital health literacy is crucial. Also, the voluntary nature of participant recruitment may have resulted in participants potentially being more interested in digital health information.

5. Conclusion

The Br-DHLI provides satisfactory evidence of internal structure validity and reliability for individuals with heart failure. Future studies will aim to further validate the instrument by exploring its relationship with additional variables, such as age, education level, health literacy, and self-care, to better understand their implications for digital health literacy among individuals with heart failure.

Glossary

CFI: Comparative fit index

DHLI: Digital Health Literacy Instrument

EREC: Expected residual correlation direct change

GFI: Goodness of fit index

KMO: Kaiser-Meyer-Olkin

MSA: Measure of sampling adequacy

PA: Parallel analysis

RMSEA: Root mean square error of approximation

RMSR: Root mean square residual

TLI: Tucker-Lewis index

Funding

This work was supported by the National Council for Scientific and Technological Development (CNPq), Process No. 420.156/2018–6 and Process No. 308337/2023-9.

Data availability

Data will be made available on request.

CRediT authorship contribution statement

Daniele Cristina Bosco Aprile: Writing – review & editing, Writing – original draft, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Karina Aparecida Lopes da Costa: Writing – review & editing, Investigation. Renata Eloah de Lucena Ferretti-Rebustini: Writing – review & editing, Methodology, Formal analysis. Vinicius Batista Santos: Writing – review & editing, Methodology. Mirian Ueda Yamaguchi: Writing – review & editing, Supervision, Methodology, Conceptualization. Camila Takáo Lopes: Methodology, Funding acquisition, Conceptualization, Writing – review & editing, Supervision, Resources, Project administration.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.