Validation of the Ethiopian Version of eHealth Literacy Scale (ET-eHEALS) in a Population with Chronic Disease

Abstract

Background

Although the measurement scale developed by Norman and Skinner is the widely used scale to assess consumers’ eHealth literacy, translating and validating the scale for the language of the target population under consideration is necessary. Amharic is the official national language of Ethiopia, with 29.3% of native speakers.

Methods

The total sample size calculated was 187 with 6% non-response rate. The internal consistency of the ET-eHEALS was measured using Cronbach’s alpha coefficient. Test–retest reliability was assessed by re-administering the ET-eHEALS questionnaire to 74 patients which is 40% of the total sample size. Construct validity was evaluated using exploratory factor analysis. The Kaiser–Meyer–Olkin (KMO) statistic and Bartlett’s test of sphericity were used to check the suitability of performing the factor analysis.

Results

Of the respondents, 63.1% (n=118) were males and 55.1% (n=103) were aged between 18 and 35 years, with 57.2% (n=107) of the participants being educated to high school diploma level or higher. Cronbach’s alpha coefficient for the translated ET-eHEALS total score was 0.94. Test–retest reliability of ET-eHEALS total score was acceptable for both agreements and consistent with ICC (interclass correlation coefficient) of 0.92. The KMO ratio of sampling appropriateness was acceptable (0.91), and Bartlett’s test of sphericity was significant with p < 0.001. The EFA (exploratory factor analysis) extracted two factors based on an extraction principle of a minimum eigenvalue of one. The extracted factor explained 80.2% of the common variance which is 51.8% for factor 1 and 28.4% for factor 2. Except for item, item fit for both infit and outfit mean squares were within the adequate range (0.5–1.5).

Conclusion

This study depicted that ET-eHEALS is a consistent and valid instrument to evaluate Amharic-speaking chronic patients’ eHealth literacy level. Since there is no prior validation of eHEALS in low-income country, this finding may indicate important directions for further improvement in eHEALS item performance in resource-limited settings.

Introduction

The widespread of internet use and technological advancement in every sector creating a simplified and accessible environment for the world as whole.^1–4 Despite the reality of digital divide among nations,^5,6 introducing new systems in the working environment has its own drawbacks in complicating a system which is already complex enough systems like healthcare systems.^7–10 To overcome the challenge in advancing the healthcare system, there is a need to assess the current level of exposure to the internet and digital artifacts among the potential users. Different scholars developed and modified the assessment tool for eHealth literacy skill in different population in different time.

Although the measurement scale developed by Norman and Skinner is the widely used scale to assess consumers eHealth literacy level,¹¹ translating and testing the validity of the scale for the language of the study population under consideration is compulsory.^12,13 Amharic is the official national language of Ethiopia, with 29.3% native Amharic language speakers.¹⁴ It is the first and the widely spoken language in the Amhara region where this study was conducted.

eHealth Literacy Scale (eHEALS)

The eHEALS is an eight-item scale used to asses self-reported skill of eHealth consumers to find, appraise, and use health-related information from the internet to answer their health-related queries and fetch for solutions to their health-related issues.¹¹ According to Norman and Skinner, the eHealth Literacy Scale (eHEALS) is a promising tool to evaluate users’ ease and skills to use the internet in order to obtain health-related information.^11,15 The scale has been used in various studies in different populations exhibiting considerable reliability and validity of items.^16–23 eHealth literacy scale consists of six basic skills (traditional literacy, health literacy, information literacy, scientific literacy, media literacy and computer literacy).¹⁵ As the eHEALS aims to asses a wide-ranging summary of literacy skills, it is a potential instrument to evaluate the comprehensive literacy skill of eHealth consumers.

Versions of eHealth Literacy Scale

The eHEALS have demonstrated significant reliability and validity in its original English version.¹¹ The reliability and validity of eHEALS have also been supported by different studies in different settings.^11,24-26 eHEALS has been translated and validated in different language versions, for instance Korean,²⁷ Iranian,²⁸ Dutch,²⁵ Hebrew,²⁹ German,³⁰ Japanese,²⁶ Spanish,³¹ Chinese³² and Italian.³³ Apparently, all these diverse linguistic versions of the eHEALS demonstrated greater internal consistency with Cronbach’s alpha coefficient greater than 0.7 which is perceived as a key indicator for the reliability of the eHEALS in these dissimilar demographics. Prior to this study, there was no literature regarding the validation of eHEALS in low resource settings and also there was no Ethiopian (Amharic) version of eHEALS to be used with reasonable level of confidence which can indicate the performance of eHEALS in resource constrained environment. The aim of the present study is to translate the eHEALS to Amharic and assess the reliability and validity of the translated Ethiopian version of eHEALS (ET-eHEALS) among patients with chronic disease in University of Gondar comprehensive and specialized hospital.

Methods

Study Design and Setting

A cross-sectional study was conducted to assess the reliability and validity of ET-eHEALS among chronic patients in University of Gondar comprehensive and specialized hospital. The hospital provides a health service for more than five million peoples living in north west Ethiopia. Chronic cases like hypertension, diabetes and asthma are the most commonly diagnosed morbidities recorded for chronic follow-up. The hospital works in a way that patients with specific chronic cases are scheduled to be seen on some days in a week. For instance, diabetes cases are scheduled for every Tuesday and Friday while cardiovascular cases are scheduled for every Monday and Wednesday.

Sampling and Procedure

The sample size for this study was calculated using the recommended range of sample size by sample size by Costello and Osborne, ratio of item to person 1:22 which is considered satisfactory.³⁴ The total sample size calculated was 187 with 6% non-response rate. All individuals with chronic disease who were attending their follow up during the study period were included in this study whereas those who could not participate in the study because they were under critical conditions or intensive care and those who had no previous internet exposure were excluded.

The participants for this study were recruited using convenient sampling method. Data collection was done by diploma holder health informatics professionals who were trained for collecting data for this study.

Translation

The translation process was handled according to Kaiser and Steinmetz study protocol.³⁵ The translation of eHEALS was performed separately by three native Amharic language speakers who were eloquent in English language. The independent translations were matched for coherency and a single unified version was endorsed. Then, it was translated back to English by two English language experts who were native Amharic speakers and naive to the original version of eHEALS. A group of experts with different field of studies like public-health, pharmacy and health informatics extensively reviewed the translated version of eHEALS in contrast with the original English version of eHEALS. Questions and feedbacks considered important were addressed after the discussion. The later version was pretested among fifteen chronic patients for its readability and understandability. After the necessary minor adjustment, the final translated version of ET-eHEALS was endorsed for further validity and reliability assessment.

Analysis

The statistical analyses were performed using statistical software for social science (SPSS) version 23, Winsteps and Rstudio (ltm package).

After describing item characteristics, ceiling and floor effect was tested by calculating the percentage frequency scores of the highest and lowest values attained where more than fifteen percentage frequency of those values were considered a signal for the presence of ceiling or floor effect.³⁶

The internal consistency of the ET-eHEALS was measured using Cronbach’s alpha coefficient and values greater than 0.7 were considered adequate.^37,38 Test–retest reliability was assessed by re-administering the ET-eHEALS questionnaire to 74 patients which is 40% of the total sample size as suggested by Walter and Eliasziw³⁹ after two weeks of the first interview. Interclass correlation coefficients (ICC) >0.7 were considered satisfactory.³⁶

Content validity of the ET-eHEALS was assessed for its understandability for the target populations that in this case were chronic patients. Content validity index was used to measure the agreement of experts on the performance of items and more than eighty percent agreement was considered satisfactory.⁴⁰

Originally, the eHEALS scale was proposed as a single factor model by Norman and Skinner.¹¹ Later, Soellner and Huber explained that eHEALS exhibits two factors with five items loading under “competency in seeking online health information” and the next three items loading under “appraisal of health information on the internet”.³⁰ In this study, the two models were compared for better fitting the data using Confirmatory Factor Analysis (CFA) and model fit indices like Chi-square (χ²),⁴¹ Comparative Fit Index (CFI), Root Mean Squared Error of Approximation (RMSEA), Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI).⁴²

Construct validity was assessed using exploratory factor analysis. The Kaiser–Meyer–Olkin statistic and Bartlett’s test of sphericity were used to check the adequacy of performing the factor analysis. Kaiser–Meyer–Olkin values >0.5 and significant Bartlett’s test p-value <0.05 were considered adequate.⁴³ Eigenvalues >1 were considered to be expressive and were retained for rotation. Convergent validity was assessed by using hypothesis testing method. The hypothesis was tested for a significant correlation between computer literacy and eHealth literacy skill.

Ethical Consideration

In conducting the study ethical clearance was secured from Debre Markos University ethical review board. Additional permissions to access participants were obtained from the offices of hospital directors and written informed consent from the respondents and parent/legal guardian for participants under the age of 18 years was also attained. This study was conducted in accordance with the Declaration of Helsinki-ethical principles for medical research.

Results

In total, 187 chronic patients took the survey. Of the respondents, 63.1% (n=118) were males, and 55.1% (n=103) were between the age of 18 and 35 years, with 57.2% (n=107) of the participants being educated to high school diploma level or higher. Almost half of the participants (n=94) reported that they use internet daily, whereas only 6% (n=11) of the participants reported their frequency of internet use as less than day in a week. Thirty-nine point six percent (n=74) were employees of governmental institutions and 32.1% (n=60) were cardiovascular patients. Table 1 shows the demographics of participants.

Table 1.

Sociodemographic Characteristics of Participants

Sociodemographic Characteristics	n	%
Age
<18	13	6.90%
18–35	103	55.10%
>35	71	38%
Sex
Male	118	63.10%
Female	69	36.90%
Frequency of internet use
Daily	94	50.30%
Several days a week	67	35.80%
One day a week	15	8%
Less than one day a week	11	5.90%
Chronic disease type
Cardiovascular diseases	60	32.10%
Chronic respiratory disease	32	17.10%
Diabetes	55	29.40%
Epilepsy	16	8.50%
Others	24	12.90%
Occupation type
Government	74	39.60%
Private	48	25.70%
None	65	34.70%

Floor effects showed “worst eHealth literacy scores” which is “strongly disagree” and ceiling effects showed “best eHealth literacy scores” which is “strongly agree”. The result indicated that floor and ceiling effects were acceptable, with no participants scoring the worst possible score (8), and 4 participants scoring the best possible score (40).

Cronbach’s alpha coefficient for the translated ET-eHEALS total score was 0.94. The intercorrelations between the eight items of ET-HEALS and the total scores ranged from 0.35 to 0.56, and all correlations were significant with p-values <0.01. The retest reliability of ET-eHEALS was achieved with an ICC coefficient of 0.92 between the two scores measured two weeks apart.

The content validity index measure indicated that 92% of participants approved the relevance and understandability of items and the mean CVI was 0.93. No item was added or removed from the original questionnaire of eHEALS after an extensive review by group of experts. All questions were well understood by the chronic patients who appraised the clarity, cultural appropriateness and readability of the translated ET-eHEALS.

The Kaiser–Meyer–Olkin test of sampling appropriateness was satisfactory (0.91), and Bartlett’s test of sphericity was statistically acceptable with p-value < 0.001. The exploratory factor analysis extracted two separately loaded factors. The extracted factors explained 80.2% of the common variance with 51.8% and 28.4% for factor 1 and factor 2, respectively). The two factors were orthogonally rotated using varimax orthogonal approach. Accordingly, they had high loading and were considered strong (factor one included five items with loading value between 0.85 and 0.95) designated as “competency in seeking online health information”. The second factor included three items with item loadings ranging from 0.84 to 0.92, which was designated as “appraisal of health information on the internet”. Communalities were acceptably good ranging from 0.68 to 0.85. Table 2 details the Principal Component Analysis (PCA) output.

Table 2.

Exploratory Factor Analysis with Factor Loadings of ET-HEALS

ET-HEALS Items	Factor Loadings	Eigenvalue	Cronbach’s Alpha	Variance Explained (%)
Factor 1		5.59	0.95	51.8
Item 1	0.90
Item 2	0.94
Item 3	0.95
Item 4	0.85
Item 5	0.89
Factor 2		2.34	0.83	28.4
Item 6	0.87
Item 7	0.84
Item 8	0.92
Total score			0.94	80.2

The single factor CFA explained 70% of the variance and communalities were also fairly acceptable ranging from 0.62 to 0.82. The comparison of model fit was done based on model fit indices in Table 3.

Table 3.

Model Fit Indices

Model Fit Indices	Single Factor Model	Two Factor Model
Chi square	135.68	685.15
CFI	0.71	0.98
GFI	0.84	0.93
AGFI	0.71	0.89
RMSEA	0.08	0.01
Variance Explained	70%	80.2%

Abbreviations: CFI, comparative fit index; GFI, goodness of fit index; AGFI, adjusted goodness of fit index; RMSEA, root mean squared error of approximation.

The Pearson’s correlation between computer literacy and eHealth literacy scale confirmed that eHealth literacy was significantly correlated with computer literacy with correlation coefficient of (0.68, p<0.001).

The Item response theory model dimensionality assumption was assessed using Kendall’s rank order correlation coefficient and chi-square test for pairwise connotation which showed high correlation. Subsequently, effort was made to identify the better fitting IRT model to the data and the result is summarized in Table 4.

Table 4.

Models Comparison

IRT Model	AIC	BIC
Rasch	2929.78	3033.17
1PL	2782.99	2889.62
GPCM	2662.32	2791.56

Note: Bold indicates the lowest AIC and BIC values.

Abbreviations: IRT, item response theory; AIC, Akaike Information criterion; BIC, Bayesian Information criterion; 1PL, one parameter logistic; GPCM, generalized partial credit model.

When compared to other models, the generalized partial credit model had relatively low Akaike Information Criterion and Bayesian Information Criterion indices, which specifies its appropriateness in best fitting the data.

The item measure, discrimination, infit and outfit indices and model standard error are indicated in Table 5. Except for item 8, item fit for both infit and outfit mean squares were within the adequate range (0.5–1.5). The measure indicated that items number 1, 2, 3, 4 and 5 were relatively easy whereas items number 6, 7 and 8 were difficult. Item number two was the best item in discriminating latent trait whereas item number 8 was the worst.

Table 5.

ET-eHEALS Infit and Outfit Indices

Item	Measure	Model Standard Error	Discrimination	Infit		Outfit
				MNSQ	ZSTD	MNSQ	ZSTD
ET-eHEALS Item 1	−0.97	0.20	2.992	1.04	0.29	0.93	−0.23
ET-eHEALS Item 2	−0.49	0.19	4.052	0.76	−1.29	0.92	−0.27
ET-eHEALS Item 3	−0.78	0.19	3.701	0.58	−2.54	0.51	−2.48
ET-eHEALS Item 4	−0.45	0.13	3.800	0.79	−1.10	0.78	−0.91
ET-eHEALS Item 5	−0.20	0.19	3.254	0.78	−1.17	0.64	−1.41
ET-eHEALS Item 6	1.08	0.18	1.730	0.80	−1.13	0.72	−1.41
ET-eHEALS Item 7	1.24	0.16	1.591	0.88	−0.62	1.05	0.29
ET-eHEALS Item 8	0.58	0.12	0.657	1.79	3.48	2.65	5.23

Abbreviations: MNSQ, mean-square; ZSTD, z-standardized.

Discussion

To the best of my knowledge, this research work is the first validation of the Ethiopian version of the eHealth literacy scale items as an assessment tool for eHealth literacy skill among chronic patients in Ethiopia. The core finding of this study was that ET-eHEALS has an acceptable level of reliability and validity as a measurement tool for eHealth literacy in Amharic-speaking chronic patients with some modification.

Before applying an assessment tool to a new language, translating and validating its applicability to the target population is compulsory. The translation enables comparison with similar research works.^12,13 The translation process was handled according to Kaiser and Steinmetz study protocol.³⁵ While translating, more focus was given to upholding the original meaning of the questions and it was found to be content valid after in-depth discussion and agreement by the groups of experts. The pretest also confirmed the understandability and readability of the translated items. The arrangement of ET-eHEALS was equivalent to the original eHEALS, that is 8 items with 5-point Likert scale.

The Cronbach’s alpha coefficient indicated that the translated version of eHEALS had very good internal consistency and the test–retest analysis demonstrated high retest reliability with intraclass correlation 0.92. The overall reliability of ET-eHEALS was similar to several validation studies done in different languages.^27,28,30,31

Regarding construct validity, the exploratory factor analysis suggested a two-factor structure of the ET-eHEALS with a strong factor loading for each construct. This indicates its construct validity and the finding is similar to the two-factor model suggested by Soellner and Huber.³⁰ However, it is not confirming the common results of a single factor model^25,31,33,44 or of the three-factor model as suggested by Hyde LL and Boyes AW.⁴⁵ The common variance explained by the two-factor model of this study is also relatively higher compared to other similar validation studies in different languages.^27,31,46 Based on the comparison made to distinguish the best fitting model structure for the data, the CFA model fit indices comparison between single- and two-factor model ascertains that the two-factor model was the best fitting with better model fit indices and 80.2% of common variance explained. Prior studies supported the strong relationship between eHealth literacy and computer literacy.^15,20 Strong correlation between those variables was observed confirming criterion validity of ET-eHEALS.

Previous studies using item response theory have supported the unidimensional nature of eHEALS.^25,31 This study acknowledged two constructs of the eHealth literacy scale. The probable reason for this inconsistency may be due to the digital divide among nations, since this study is the only validation study in low-income countries.²⁴ It could be that the ability to use the Internet to find relevant information and the ability to appraise online health information are two different sets of skills for peoples in low-income country but not for developed nations as they are more connected with the digital world. Further investigation is therefore compulsory in diverse populations to examine whether the eHealth literacy scale is composed of two separate constructs.

Concurrently, the item analysis revealed that the last three items were relatively difficult for the chronic patients. The three items exhibited relatively lower performance in discriminating participants ability and items difficulty. With this regard, further research should be conducted to maximize the performance of eHEALS in low resource settings.

Conclusion

This study depicted that ET-eHEALS is a consistent and valid instrument to evaluate Amharic-speaking chronic patients’ eHealth literacy level. Since there is no prior validation of eHEALS in low-income country, this finding may indicate an insight for further improvement in eHEALS item performance in resource-limited settings. Moreover, further investigation is required to determine the applicability of ET-eHEALS in different populations.

Funding Statement

The author received no funding for this study.

Data Sharing Statement

All data generated or analyzed during this study are included in this article.

Disclosure

The author declares that he has no competing interests.