Testing Measurement Properties of the Norwegian Version of Electronic Health Literacy Scale (eHEALS) in a Group of Day Surgery Patients

Abstract

Background and Aim

In order to assess patients’ ability to search, understand, and benefit from Internet-based information, several screening tools have been developed. One of these tools, which has been widely used, is the eHealth Literacy Scale (eHEALS). The aim of this study was to examine the measurement properties of the Norwegian version of the eHEALS, as it was used in a group of patients undergoing day surgery.

Methods

A cross-sectional survey study was conducted among 119 patients scheduled for day surgical treatment in a Norwegian hospital. The questionnaire included the screening tool eHEALS, which contains 8 items for assessing a person’s information awareness skills, information seeking skills, and skills to evaluate and act based on the information. Cronbach’s alpha coefficients and item-total correlations were assessed for estimating reliability of the eHEALS. Exploratory factor analysis with Oblimin rotation was used for assessing the validity of the scale. Eigenvalue was set to 1.0.

Results

A Cronbach’s alpha coefficient of 0.89 for the total scale, values >0.82 for Alpha if Item Deleted, and moderate to high item-total correlations supported the homogeneity and internal consistency of the scale. A two-component solution explained a total of 74.8% of the variance, with the first component explaining 59.53% of the variance in the scale and included the items reflecting information awareness and seeking. The second component explained 15.23% of the variance, including items reflecting the ability to evaluate and act.

Conclusion

The reliability of the Norwegian version of eHEALS, used in a group of patients undergoing day surgery, was good. The internal structure, with two distinct factors, is in line with several previous studies. The eHEALS appears to be an appropriate tool for assessing eHealth literacy among this patient group.

Introduction and Background

Day surgery (also referred to as elective, or ambulatory, surgery) is a very common form of treatment for patients with various types of diagnoses in Western countries, and there are many advantages to this treatment. Besides the incentives to increase efficiency and reduce costs associated with expensive hospital stays, day surgery has been shown to prevent postoperative complications as a result of hospitalization, and this form of treatment is often the patients’ own preference.^1,2 Day surgery is found to enhance patient satisfaction and have less impacts on their families’ daily life and routines.³

Patients undergoing day surgery are normally discharged from the hospital on the same day after the operation. Due to the rapid process and lack of time when undergoing day surgery, one of the challenges is that the patients do not receive enough information about the surgery.² Insufficient information may affect the patients in negative ways, such as anxiety and stress before the surgery, and pain and other complications in the recovery phase.² Patients who experience less stress and anxiety preoperatively may have less pain and better recovery and healing after the surgery.⁴

One problem with day surgery is that patients must absorb large amounts of information in a short period of time. There is a risk for mismatch between the patients’ expectations and needs for information about the surgery and the pre- and postoperative phase, and the information they received.⁵ The consequence may be that the patients, who are already in vulnerable situation, experience that information is not adapted to their individual needs. Other aspects were reported by Grønnestad and Blystad,⁶ who pointed out that having previous experience as a patient and having personal resources such as motivation and self-involvement, are particularly important for understanding information about surgical procedures. According to the authors, previous experience as a patient increased their sense of safety and diminished the need for information because it makes them aware of the importance of self-involvement in the information process. It is, therefore, easier for them to obtain the information they need.⁶ Thus, those who lack such experience might be particularly at risk for poor information acquisition. Mordiffi, Tan and Wong⁷ examined the correspondence between information given to patients prior to a surgical procedure and the patients’ expressed needs. They found that health professionals did not tailor preoperative information to the patients’ needs, and that lack of understanding and documentation of what was important to the patient contributed to this information gap. In addition, it was stated that health professionals assumed that the patient would ask questions during the conversation, while the patient expected that health professionals should provide them with the necessary information without having to ask about it.⁷

The concept of “health literacy” is defined by the World Health Organization⁸ as “the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health”. Further, the concept is commonly described to include functional, communicative, and critical skills.^9–13 Thus, a more rudimentary definition of “literacy”, as the ability to read and write, has been expanded to include other skills like social functioning and ability to understand and solve problems.¹³ Functional skills are necessary for reading and writing in order to operate in everyday situations, communicative skills are more advanced skills to extract and apply (new) information in different situations, and critical skills are needed for analyzing and reflecting on information or advice.⁹ Health literacy includes the ability to comprehend information for managing one’s own health.¹⁴

Internet and communication technologies, including web-based education and information programs, have provided us with a wide range of possibilities that can be particularly useful for day surgery patients. Due to the limited time for information exchange and absorption before, during and after the surgery, patients can obtain the information they need at their own convenience through these web-based programs.¹⁵ However, when healthcare, and information, is provided in the form of technology or digital services, it is essential for the clinicians to know the levels of electronic health literacy among their patients to be able to provide services that suit their actual needs and abilities.^16,17 In 2006, Norman and Skinner¹⁶ introduced this concept – “electronic health literacy” (later referred to as eHealth literacy), and they described the concept as the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem.¹⁶ eHealth literacy is found to have positive and protecting effects on persons’ health. For instance, Lin et al¹⁸ reported that eHealth worked as a protector for patients with heart failure.

In order to assess patients’ ability to search, understand, and benefit from Internet-based health information, useful screening tools for assessing patients’ eHealth literacy are necessary. While several scales are available for measuring health literacy in general, scales developed to measuring eHealth literacy are less prevalent.^19,20 However, one instrument developed for this purpose, ad which has been widely used, is the eHealth Literacy Scale (eHEALS) developed by Norman and Skinner.¹⁶ The eHEALS is a measurement containing 8 items reflecting eHealth literacy, and developed to measure persons’ combined knowledge, comfort, and perceived skills at finding, evaluating, and applying electronic health information to health problems.¹⁶ It is a self-report measurement, which, according to Norman and Skinner,¹⁶ is based on the individual’s perception of own skills and knowledge within specific literacy areas. The eHEALS is based on the Lily-model, comprising six core skills (or literacies): traditional literacy, health literacy, information literacy, scientific literacy, media literacy, and computer literacy.¹⁷ Further, the six literacies are organized into two central types: analytic (traditional, media, information) and context-specific (computer, scientific, health). The analytic element involves skills that are relevant to a wide range of information sources regardless of the focus or context, while the context-specific element relies on skills that are more situational.¹⁷

In a systematic review of selected tools aimed measure eHealth literacy, Karnoe and Kayser²⁰ found that eHEALS was the only scale that had been used in more than one study. The same conclusion was stated by Sudbury-Riley et al¹² in a later publication. The original English version¹⁶ has been translated into several languages, such as Portuguese,²¹ Dutch,²² Hebrew,²³ German,²⁴ Japanese,²⁵ Spanish,²⁶ and Turkish.²⁷

In general, previous studies have shown solid values for internal consistency and reliability of the eHEALS. In their early work, Norman and Skinner¹⁶ found a Cronbach’s alpha coefficient of 0.88 on the total scale. Later studies have reported Cronbach’s alpha coefficients between 0.80 and 0.97.^{19,22,23,26–28} The structural components of the eHEALS have been widely discussed,¹² and many studies have found that it appears as a unidimensional scale.^12,22,28 This was also concluded in the initial work by Norman and Skinner.¹⁶ On the other hand, several studies have also shown an inherent structure with two^12,19,23,24 or three^12,30 factors reflecting the underlying content of electronic health literacy, as previously described.

However, to our knowledge, no studies have published testing results, ie, psychometric properties, of a Norwegian version of eHEALS. Nor have we been able to identify studies that have used the eHEALS among day surgery patients in any country.

Therefore, the aim of this study was to examine the measurement properties of the Norwegian version of the Electronic Health Literacy Scale (eHEALS) used in a group of patients undergoing day surgery.

Materials and Methods

This study used a cross-sectional survey design.

Sample and Recruitment

Participants included patients scheduled for day surgery at a hospital in South-eastern Norway, during a one-month period in March 2019. The only inclusion criteria were that they should be 18 years or older and were scheduled to undergo day surgical treatment. Among 254 available patients, 116 participants were randomly selected for inclusion. They received written and oral information about the study before being asked to consent to participate. Those who consented received an anonymous paper-based questionnaire upon arrival and filled out the form while waiting for their operation. A total of 112 patients filled out the form. Three questionnaires were incompletely answered, and the final sample consisted of 109 patients.

Data Collection

The only background variable in the questionnaire was the age group of the participants registered as age groups (options: <20 years, 20–34 years, 35–49 years, 50–65 years, and >65 years).

Instrument

The 8-item instrument electronic Health Literacy Scale (eHEALS) was used to measure participants’ perceived skills in finding, evaluating, and applying electronic health information to health problems.^16,31 The eight items in eHEALS are scored on a 5-point Likert scale ranging from 1 = “strongly disagree” (the most disadvantageous option) to 5 = “strongly agree” (the most advantageous option). Thus, the minimum and maximum achievable total scores were between 8 and 40, with higher scores representing higher self-perceived eHealth literacy. In addition, two items regarding perceived usefulness and importance of health-related information on the Internet were included in the questionnaire together with the eight-items scale.¹⁶

The original version of the eHEALS was translated into Norwegian in accordance with the recommended translating and back-translating procedure described by Streiner and Norman.³² First, the questionnaire was translated into Norwegian by a group of health professionals conducting this study. Thereafter, the Norwegian version was reviewed by a professional expert panel at the University of Agder, and minor revisions were made for clarity. A bilingual professional (ie, medical doctor) made the back-translation of the questionnaire from Norwegian to English. The translated English version was then compared to the original version by the expert panel, and only minor insignificant differences were found. The Norwegian questionnaire was pilot tested by five health professionals who had experiences with day surgery patients. All approved the questionnaire without further comments.

Analyses

Descriptive statistics were used to get an overview of scores on each item, ie, median (Md), mean (M), standard deviation (St.d.), skewness, and kurtosis. The reliability of the Norwegian version of the eHEALS was assessed by estimating the internal consistency (homogeneity) of the scale using Cronbach’s alpha reliability coefficients of the total scale and Alpha if Item Deleted. Internal consistency was also estimated by corrected item-total correlations, ie, correlations between each item and the total scale after omitting the individual item from the total scale.³² The construct validity was explored using the principal component method of factor analysis with Oblimin rotation and eigenvalue≥1. Possible correlations between the participants’ age and the total eHEALS score and scores on the individual eHEALS items were explored using Spearman’s rank correlations (r_s).

IBM SPSS statistical software version 25.0 (Armonk, NY, USA) was used for performing statistical analyses, and a P-value of <0.05 was considered significant.

Ethics

Ethical principles for clinical research were followed.³³ Permission to conduct the study was obtained from the hospital’s authority. Further, the study was discussed with the Norwegian Centre for Research Data (NSD), and they replied that it was unnecessary to report the study to them if it was completely anonymous and no sociodemographic information beyond the participants’ age group was collected, as was the case in the current study. The participants were provided oral and written information about the study on beforehand. The voluntariness and anonymity of the participants were emphasized, and the participants were informed about the purpose of the study and how the data would be used. Answering the questionnaire was considered consent to participate.

Results

The Participants

The sample consisted of 109 patients referred to day surgery at a hospital in South-eastern Norway. Only one of the participants was younger than 20 years. The age distribution in the rest of the study group was as follows: 26.9% was between 20 and 34 years, 25.9% was between 35 and 49 years, 31.5% was between 50 and 65 years, and 14.8% was more than 65 years.

The total mean eHEALS score for the sample was 29.0 (St.d. 5, 11, Max–min=13–40). We found a significant, though weak, negative correlation between the participants’ age and the item “I can tell high-quality from low-quality health resources on the Internet” (r_s=−0.24, p=0.014). There were no significant correlations between the participants’ age and the other eHEALS items or the total eHEALS. Descriptive statistics for each item in the scale are displayed in Table 1.

Table 1.

Overview of the Descriptive Statistics (ie, Md, M, St.d. Skewness and Kurtosis) on the eHEALS Items

eHEALS Items	Md	M	St.d.	Skewness	Kurtosis
I know what health resources are available on the Internet	4	3.62	0.75	−0.341	−0.096
I know where to find helpful health resources on the Internet	4	3.67	0.79	−0.414	−0.103
I know how to find helpful health resources on the Internet	4	3.77	0.77	−0.861	1.386
I know how to use the Internet to answer my health questions	4	3.84	0.82	−0.642	0.681
I know how to use the health information I find on the Internet to help me	4	3.78	0.75	−0.594	0.400
I have the skills I need to evaluate the health resources I find on the Internet	4	3.59	0.91	−0.429	0.087
I can tell high-quality from low-quality health resources on the Internet	4	3.53	0.93	−0.550	0.572
I feel confident in using information from the Internet to make health decisions	3	3.24	0.96	−0.379	−0.116

The Reliability and Validity of the Norwegian Version of eHEALS

The internal consistency of the scale was shown in a Cronbach’s alpha coefficient = 0.89. Internal consistency was also displayed in Item-Total correlations, and Alpha values if Item Deleted, as shown in Table 2.

Table 2.

Corrected Item-Total Correlations and Alpha if Item Deleted for the Total eHEALS and the Individual Items

Items	Corrected Item-Total Correlationsa	Alpha if Item Deleted
I know what health resources are available on the Internet	0.70	0.82
I know where to find helpful health resources on the Internet	0.72	0.88
I know how to find helpful health resources on the Internet	0.71	0.88
I know how to use the Internet to answer my health questions	0.71	0.88
I know how to use the health information I find on the Internet to help me	0.77	0.88
I have the skills I need to evaluate the health resources I find on the Internet	0.71	0.88
I can tell high-quality from low-quality health resources on the Internet	0.56	0.90
I feel confident in using information from the Internet to make health decisions	0.61	0.89

Note: ^aAll correlations were significant at p<0.001.

Construct validity of the eHEALS was reflected in the explorative factor analysis, and two extracted factors with an eigenvalue ≥1 explained 74.8% of the variance in the scale. Factor 1 and Factor 2 contributed to 59.53% and 15.23% of the variance, respectively. The factor loadings and the distribution of items on the factors are presented in Table 3.

Table 3.

Standardized Factor Loadings for the eHEALS

eHEALS Items	Pattern Coefficients		Structure Coefficients
	Factor 1	Factor 2	Factor 1	Factor 2
I know what health resources are available on the Internet	0.934		0.902	0.382
I know where to find helpful health resources on the Internet	0.932		0.904	0.403
I know how to find helpful health resources on the Internet	0.923		0.907	0.396
I know how to use the health information I find on the Internet to help me	0.634	0.330	0.792	0.634
I know how to use the Internet to answer my health questions	0.495	0.422	0.698	0.660
I can tell high-quality from low-quality health resources on the Internet		0.929	0.333	0.875
I feel confident in using information from the Internet to make health decisions		0.862	0.408	0.859
I have the skills I need to evaluate the health resources I find on the Internet	0.313	0.623	0.612	0.773

Note: The highest loadings are marked in bold.

Discussion

The present study aimed to examine the measurement properties of the Norwegian version of the Electronic Health Literacy Scale (eHEALS) used in a group of patients undergoing day surgery. Patients referred to day surgery might be in a particularly vulnerable situation due to the shortage of time for pre-surgery preparation and information, and limited time for communication with the hospital staff before being discharged.² Therefore, relevant information available on the Internet may be particularly advantageous for this patient group. An important condition is, however, that the patient can acquire, understand, and benefit from the information. This ability is what eHEALS is designed to measure.

The total eHEALS scores in the study group, ie, M=29, showed that the patients’ eHealth literacy was fairly good, by means of obtaining and understanding adequate and available health information on the Internet. This result is in line with total eHEALS scores reported in other studies. For instance, Richtering et al¹⁹ reported a total eHEALS score of M=27.1 in a study including patients at risk of developing cardiovascular disease.

The participants in our study were relatively evenly distributed regarding age groups, and the results might indicate that those in the younger age groups were more able to assess the quality of the information available on the Internet. This tendency is also reported in other studies.^23,30 For instance, Paige et al³⁰ found that older adults experienced greater challenges regarding eHealth resource awareness, information seeking skills, and ability to evaluate and act upon online health information, compared with younger adults. However, the Internet use among older people is steadily increasing, as they search and visit websites for on-line health information and discussion.³⁴ There has been a tremendous development on the Internet front since the original eHEALS was designed 15 years ago, and, as Stellefson et al³⁴ point out, it is important to consider modifying the scale to better fit the relevance for use among an older population.

The high Cronbach’s alpha for homogeneity of the scale and the high correlations between each item and the total scale shows that the internal consistency of the eHEALS was very high in our study among day surgery patients. These results might indicate that the reliability of the scale is excellent and that included items largely reflect what the scale is intended to measure. However, one should be aware that too high alpha and correlation values may indicate that the items overlap.³² On the other hand, previous studies in which eHEALS has been tested and used have shown similarly high values for reliability.^{19,21,22,24,26–29}

The analyses of factor structure of the eHEALS have shown variable results regarding dimensionality, also depending on the analyses used to examine the internal structure of the scale. Several studies have found the scale to be unidimensional with high loading on one single factor.^16,28,30 The one-factor structure of the scale is described as advantageous because it allows health-care professionals to easily understand and use it as a whole in clinical settings.²⁸ However, other studies have found limited evidence to support unidimensionality,^19,30 mostly including either a two- or three-factor model and reflecting the underlying theoretical assumptions and aspects of the scale. The first factor included five items reflecting information awareness and information seeking, and the second factor included three items reflecting information engagement. This result is in line with other studies that have found that the eHEALS consists of two dimensions, defined to measure information seeking and information appraisal.^19,24,30,32

As Karnoe and Kayser²⁰ claim, the eHEALS is an easy tool to administer, but it provides a concept-based measure for eHealth literacy without thoroughly investigate whether inadequate eHealth literacy is a result of insufficient health literacy, digital literacy, or a combination hereof. In another study, van der Vaart et al²² found a weak correlation between the eHEALS and people’s internet use, as the scale was unable to differentiate between those with high and low internet skills. In order to do so, it is important to have a broader perspective on health literacy than is the case in the original eHEALS. To meet the requirements for such a measurement tool today, it should include operational, formal, informational skills as well as strategic internet skills.²² Questions have also been raised that the developers of the scale have not explained satisfactorily enough how they have anchored it theoretically in the Lilly-model, which has its roots in social cognitive theory and self-efficacy theory.¹²

Self-reported questionnaires were used in the data collection. One challenge when using this method is that one can never be entirely sure whether the respondents have fully understood the content of questions being asked. Therefore, there is always uncertainty about the respondent’s intended meaning. Another limitation is that the study was cross-sectional, and data were collected only once. Evaluation of the questionnaire’s stability, using a test–retest design, was therefore unavailable. Nor was it possible to test criterion-related validity because external criteria were not available in this study. These aspects should be further investigated in new studies. However, to our knowledge, this was the first testing study of the eHEALS measurement among this patient group, and thus providing information about its usefulness also for this group.

Conclusion

The test of measurement properties of the Norwegian version of the eHEALS in this study showed promising results regarding internal consistency, and the internal scale structure including the two distinct dimensions. The results also indicate that the scale might be an appropriate tool for measuring eHealth literacy in the actual population. Criticism has been raised, and weaknesses and limitations have been pointed out, when using eHEALS as a measurement tool for people’s eHealth literacy. Despite this, it is probably the most widely used measurement for this purpose. The EHEALS is still the most popular and widely used tool, probably because it is concise and easy to use.²⁸

Disclosure

The authors report no conflicts of interest in this work.