Cut-off value of the eHEALS score as a measure of eHealth skills among rural residents in Gansu, China

Abstract

Background

In this study, our aim was to quantify eHealth literacy and determine a cut-off value for eHEALS scores that signifies adequate eHealth literacy among rural residents in Gansu, China.

Methods

This cross-sectional investigation encompassed 18 rural areas across three cities—Lanzhou, Wuwei, and Dingxi—in Gansu. A total of 451 residents were recruited and underwent evaluations for both eHealth literacy and eHealth skills, utilizing the eHEALS and self-constructed tasks, respectively. A receiving operator characteristic curve was plotted with eHealth skills as the dependent variable and eHealth literacy levels as the independent variable, aiming to determine a cut-off value for eHEALS indicating adequate eHealth literacy and evaluate its predictive capacity.

Results

Among the 451 respondents, 10.9% did not possess a personal electronic device with Internet access, while 6.4% owned but had never used them. Within the remaining 373 residents, the mean eHealth literacy score was 25.85 (SD:10.93), item scores ranged from 3.12 (SD:1.45) to 3.42 (SD:1.60). Completion rates for three eHealth skills varied from 39.1% to 59.8%. The cut-off value was 29.5 determined by the Youden index. The area under the receiver operating curve was 0.829, with a sensitivity of 86.7%, and a specificity of 66.8%.

Conclusions

Our findings emphasized that eHealth literacy among rural residents remains at a low level. Moreover, we identified a cut-off value of 29.5 for eHEALS scores that signifies adequate eHealth literacy within this demographic.

Introduction

The World Health Organization (WHO) emphasized that health literacy plays a critical role in determining health outcomes and has the potential to empower individuals, ensuring health equity. ¹ eHealth services have the capacity to alleviate the strain on healthcare systems by offering prompt, high-quality healthcare information. ² Furthermore, they enhance healthcare quality by fostering collaborative decision-making and empowering citizens, patients, and healthcare professionals.^3,4 In recent decades, the proliferation of the Internet had increased the utilization of eHealth services in disease prevention and health promotion. Individuals were increasingly adopting appropriate self-care practices and managing their health through eHealth.^5,6 Individuals across different regions can readily access healthcare information and utilize eHealth services, contributing to a more equitable distribution of healthcare resources. Amidst the COVID-19 pandemic, when stringent control measures were enforced and healthcare resources were strained, eHealth emerged as a solution for delivering remote individual health services, ⁷ facilitated the implementation of outbreak-related protective measures,^8,9 consequently mitigating the risk of infection and transmission. eHealth gained widespread interest, catalyzing a broader public recognition of its significance.^10,11 Overall, the advocacy and adoption of eHealth yield substantial benefits. Nevertheless, in certain contexts, the dissemination of disease-related misinformation, individual misinterpretation, and subsequent adverse behaviors could lead to unfavorable health outcomes.^12–14 The concept of eHealth literacy was created to assess the skills of individuals using eHealth, which is defined as the ability to seek, find, understand, and appraise health information from electronic sources and using the knowledge gained to address or solve a health problem. ¹⁵ Proficiency in adeptly employing eHealth services––reflecting a high level of eHealth literacy––is a pivotal factor in both the execution of eHealth services and the enhancement of healthcare.

Since its introduction, the concept of eHealth literacy has attracted increasing attention worldwide. Research has spanned various domains, including surveys on eHealth literacy levels,^16,17 evaluation of influencing factors,^18,19 analysis of the correlations between eHealth literacy and health behaviors, mental health, lifestyle, and quality of life,^20–22 as well as the development and evaluation of interventions to enhance eHealth literacy,^23,24 etc. However, studies have been restricted in terms of the population, primarily concentrating on adolescents, university students, and urban populations, with less focus on rural residents. This is concerning because people in rural areas tend to have poorer health and lower health literacy due to geographical isolation, socioeconomic disadvantages, inadequate healthcare infrastructure, subpar sanitation, and limited access to medical services, etc.^25–27 There is a significant urban–rural disparity exists in the distribution of healthcare resources and individual health literacy. ²⁸ Presently, 500 million Chinese people (36.11%) still reside in rural regions. ²⁹ As of June 2021, the 48th Statistical Report on China's Internet Development indicated that over 99 percent of administrative villages had access to optical fiber and 4G connections, attaining a state of “same network and same speed” across rural and urban areas. With the improvement of communications infrastructure, the cost of internet use in rural areas has gradually declined, and the urban–rural Internet penetration rate has narrowed to 19.1 percent (78.3% vs. 59.2%), which provides a guarantee for the use of eHealth by rural residents. ³⁰ However, not everyone possesses substantial internet access or eHealth skills to utilize the benefits of eHealth, the eHealth services require a certain level of eHealth literacy of the user to be equitably adopted. ³¹ Factors such as advanced age, lower income, and limited education contribute to a lower level of eHealth literacy among China's rural inhabitants compared to their urban counterparts. ³² A review suggested that eHealth literacy frequently receives inadequate attention in interventions aimed at socially vulnerable groups, thereby exacerbating healthcare disparities. ³¹ Amidst China's COVID-19 prevention and control efforts, the urban–rural gap could potentially increase. As a valuable means for rural residents to access healthcare services and enhance their well-being, eHealth literacy warrants more attention.

Norman et al. developed the eHealth literacy scale (eHEALS) to assess individual's eHealth literacy. ³³ The scale has been translated into various languages, and its validity has been demonstrated across diverse populations.^34–39 In China, Ma et al. evaluated the psychometric properties of the eHEALS within a rural population, using the classical test theory and item response theory methods. Their findings endorsed the applicability of eHEALS for measuring eHealth literacy among rural inhabitants. ⁴⁰ As a subjective measure of perceived confidence and skills in finding, assessing, and using electronic health information to solve health problems, ³³ it is worth considering how perceived skills (as measured by eHEALS) can be used to assess respondents’ objective eHealth skills. Quantitatively, a Dutch study found that eHealth literacy is not always a good predictor of actual eHealth skills. ³⁶ In rural areas, the majority who cannot migrate to urban areas is less educated, in poorer health, and less likely to use the internet to access eHealth services.^41,42 There might be a greater disparity between eHealth literacy and actual eHealth skills. Qualitatively, a study in Taiwan formulated a scale for digital health literacy assessment (DHLA) grounded in eHEALS and found the different cut-off values for DHLA could distinguish those at low, medium, and high risk of misjudging health information on the internet. ⁴³ These findings suggest that a qualitative cut-off value of eHEALS might be more suitable for assessing actual eHealth skills. Furthermore, adopting a cut-off value could yield additional benefits and greater practical significance. With a cut-off value, we can efficiently distinguish between high and low eHealth literacy levels among residents in a specific area, thus determining their possession of eHealth skills and identifying the key target demographic for intervention. Moreover, the cut-off value can be directly employed to appraise intervention efficacy or serve as a target for interventions. By assessing the discrepancy between current and desired levels, specific interventions can be tailored in terms of type, intensity, and scope to amplify intervention effectiveness. To the best of our knowledge, there is no relevant study to explore an appropriate cut-value for eHEALS. Some studies have employed values such as 20, 26 (without explicit criteria)^,44,45 or 32 (80% of the total score) to distinguish between high and low levels of eHealth literacy,^18,46,47 which still needs to be validated and examined in future research. Therefore, it's necessary to determine a cut-off value for eHEALS to assess the acquisition of eHealth skills.

For obtaining a cut-off value, we need to adopt an appropriate methodology for evaluating participants’ eHealth skills. In a Dutch study, eHealth skills of 88 urban and rural residents were assessed through nine assignments encompassing four facets of eHealth skills—operational, formal, informational, and strategic. These tasks spanned 17 scenarios and took around 1.5 h to complete. The study then evaluated the correlation between eHealth literacy and eHealth skills. The tasks were designed to be accessible and relevant to the general user population (e.g. find the Web address of a health clinic, or search for information on vitamins). ³⁶ A UK study evaluated practical health skills by surveying 54 participants (comprising university students and staff) on their completion levels for six health-related questions including analgesic medication, obesity, diabetes, nutrition, and influenza retrieval tasks, and assessed the relationship with eHealth literacy (e.g. you have a headache and buy a packet of Co-Codamol tablets from the chemist, which are the two largest ingredients in Co-Codamol tablets, or diabetic retinopathy is an eye complication associated with diabetes, can you name three common symptoms of this condition). ⁴⁸ A study in Taiwan measured the risk of misunderstanding health information by constructing an online health information database containing 310 items across categories like food safety, drugs, medical devices, cosmetics, medical treatment, preventative health, disease screening, health promotion, and general medicine. Then selected five items at random from this database and had online participants determine the correctness of these items (e.g. it is helpful to drink bone broth, which can supplement calcium, or smoking e-cigarettes is not addictive). ⁴³ These studies employed objective methods to evaluate eHealth skills. Nevertheless, findings from study designs with smaller sample sizes might not be representative of real-world scenarios and may not be applicable to large-scale investigations. Additionally, specific health issues may not align with the realities of rural China, rendering the surveys less practically significant. Lastly, taking longer to complete surveys could lead to diminished participant engagement and completion rates. Given these circumstances, it might be impractical to replicate similar studies in rural China. Consequently, we need to develop assessments that are adapted to specific health topics and simplify the assessment approach as much as possible, in order to assess the eHealth skills of rural residents in China.

Therefore, the objectives of this study were (a) to investigate the eHealth literacy levels among rural Chinese residents (b) the first to determine a cut-off value for adequate eHealth literacy by assessing the eHealth skills in real-world settings.

Methods

Study design

This cross-sectional study was conducted in Gansu, a major agricultural province situated in northwest China. Gansu encompasses 12 prefecture-level cities (Lanzhou, Jiayuguan, Jinchang, Baiyin, Tianshui, Wuwei, Zhangye, Pingliang, Jiuquan, Qingyang, Dingxi, Longnan) and two autonomous prefectures (Linxia Hui Autonomous Prefecture and Gannan Tibetan Autonomous Prefecture). The study employed a multistage stratified sampling approach to recruit participants. In the first phase, based on the GDP levels from the Gansu Development Yearbook 2020, 14 prefecture-level cities were divided into three classes—high (Jiayuguan, Lanzhou, Jinchang, Jiuquan, Zhangye), medium (Qingyang, Gannan, Baiyin, Wuwei, Pingliang) and low (Tianshui, Longnan, Linxia, Dingxi). One city was then randomly selected at each class, three cities, Lanzhou, Wuwei, and Dingxi were selected. In the subsequent phase, two districts were randomly picked from each of these three cities. Following this, 18 villages and towns were conveniently selected, considering the degree of on-site cooperation. This process led to the recruitment of participants from rural areas across six districts in Gansu Province: Yuzhong and Gaolan from Lanzhou, Gulang and Tianzhu from Wuwei, and Anding and Tongwei from Dingxi. Specific townships including Dinyuan, Lianta, Sanjiaocheng, Xiaguanying, Chengguan, Qingshui, Shenchuan, Shidong, Dingning, Sishui, Dachaigou, Huazangsi, Saishensi, Songshan, Lijiapu, Chankou, Beichengpu, and Biyu.

Participants and procedure

The recruitment process was conducted through two methods: (a) household surveys led by village doctors and (b) Face-to-face surveys conducted among clinic attendees at village health centers. During the survey, enumerators worked in pairs to address any encountered problems, ensuring the quality and efficiency of the survey. Participants were provided with a comprehensive briefing regarding the survey's objectives, content, anonymity, and voluntary nature before granting their consent to participate. Those who agreed to participate in the study were requested to complete a questionnaire on their own or with the assistance of an enumerator, the entire process typically required approximately 30 min. Upon finishing the questionnaire, the participants would receive a gift (worth approximately RMB 25) as a reward for their participation, as well as compensation for their time. During the 1-month survey period (29 March–28 April 2021), 451 subjects consented to participate in survey and completed the questionnaire. After checking the questionnaire's completeness and reasonableness, the subjects were all included in the study. The inclusion criteria were as follows: age ≥18 years, local permanent residents, and have the ability to complete the questionnaire on one's own or under the assistance of an enumerator.

Measures

Demographics

The following data regarding general sociodemographic characteristics were collected: age, gender, ethnicity, place of domicile, education level, marital status, occupation, living with children, and personal monthly income.

eHEALS

The eHEALS, as a classic eHealth literacy measurement tool, has been widely used across various cultures and populations. Its simplicity, relatively ease of comprehension, and quick completion time make it suitable for rural residents. Furthermore, its measurement validity has been confirmed in rural China. ⁴⁰ Consequently, eHEALS was used to measure eHealth literacy. ³³ The assessment encompassed tests to evaluate the ability to access internet health information and usability of services (1–5), judgment ability (6–7), and decision-making ability (8). Sample items included: “I know how to find helpful health resources on the Internet,” “I feel confident in using information from the Internet to make health decisions.” Each item was rated on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The total score was calculated by summing all 8 items, ranging from 8 to 40, with higher scores representing a higher level of eHealth literacy. The Chinese ⁴⁹ version has been widely used to evaluate eHealth literacy levels in various populations. The Cronbach's α was 0.962.

Ehealth skills

Three tasks were constructed to assess participants’ practical eHealth skills (Cronbach's α = 0.831). For the tasks, various types of diseases were considered, including acute and chronic infectious diseases, as well as acute non-infectious diseases. Additionally, factors including general disease information, access to specific health services, and emergency disease response scenarios were incorporated. The participants conducted searches using their electronic devices, and subsequently, investigators judged task completion. A participant who completed all three tasks was considered to have good eHealth skills. The three self-constructed tasks are described below.

Firstly, regarding the choice of acute infectious diseases, the study was conducted during a period when China was implementing dynamic zero-control measure for COVID-19, it was beneficial to understand the dynamic development of the epidemic in terms of the degree of implementation of one's own protective measures. Additionally, COVID-19 was a hot topic within Chinese society during that period, information regarding the disease was rapidly spreading across various channels. Therefore, we wanted to investigate whether rural residents were able to access correct and effective information amidst the diversified available sources:

• Task 1: Search for official notification data on yesterday's (the day before the respondent was surveyed) nationwide COVID-19 epidemic situation.

The judgment criteria were as follows: the respondents were considered to have completed the task if they used one of the following sources for their search, official government web pages (e.g. the National Health Commission or Chinese Centre for Disease Control and Prevention, epidemic bulletin by Dr Ding Xiang (a professional healthy lifestyle coach in China's pan-health field)), the official websites of Tencent, Sina, or Baidu (Internet news platforms with a wide audience in China for real-time updates on the latest developments in COVID-19), or official WeChat public account. If the participants retrieved data consistent with the above channels, the task was also considered complete, otherwise, it was considered incomplete.

Secondly, regarding the choice of chronic infectious diseases, we initially considered HIV (human immunodeficiency virus) and TB (tuberculosis). However, the focus of HIV epidemic in China was not on rural areas, we ultimately included TB as part of task construct. Despite the great progress in TB control, China had approximately 780,000 new cases of TB, with the third highest (after India and Indonesia) worldwide in 2021. ⁵⁰ And according to a national epidemiological sample report conducted in 2010, the incidence of active TB cases in rural areas (569 per 100,000) was significantly higher than that in urban areas (307 per 100,000). ⁵¹ Considering the distinctive pattern of TB treatment in China, specific medical institutions are officially set up to diagnose and treat TB. Therefore, we focused our questions on search for specific TB medical institution sites:

• Task 2: Search for designated TB clinics within your area.

This task was judged based on the following criteria. Participants conducted searches on official websites of the Health Commission of Gansu Province or Gansu Provincial Centre for Disease Control and Prevention using the keyword “tuberculosis sentinel.” If the participants found a list of clinics through this search method or other means, the task was considered complete, otherwise, the task was considered incomplete.

Thirdly, as China experiences ongoing socioeconomic development and an accelerated aging process, the incidence of cardiovascular disease cases within the nation continues to rise. According to the China Cardiovascular Health and Disease Report 2021, cardiovascular disease stood as the primary contributor to total deaths among urban and rural residents in 2019, with two out of every five deaths were attributed to cardiovascular disease. Particularly in rural areas, this proportion reached 44.26%. ⁵² Confronted with this substantial disease burden, our objective was to assess whether rural residents could acquire the right information and undertake effective countermeasures when confronted with sudden cardiovascular diseases:

• Task 3: If someone around you is experiencing “severe heart pain,” please search online for the right way to deal with it.

The criteria for evaluating this task were as follows. Given that there is no single standardized answer for this task. Participants who derived solutions from healthcare professionals, encompassing actions such as promptly discontinuing ongoing activity, administering oral rapid-acting heart relievers or nitrates (if accessible), seeking urgent medical attention, or getting content similar to the answer, were considered to have accomplished the task. If no search was conducted or the search results showed advertisements without effective information extraction, the task was considered incomplete.

Following the survey, the investigators would demonstrate the correct search process to the participants or provide suggestions for improvement based on their search processes, with the hope that they could implement correct and effective measures in similar situations that may occur in the future.

Statistical analysis

The receiver operating characteristic (ROC) curve is a two-dimensional curve with sensitivity as the vertical axis and specificity as the horizontal axis. Each point on the curve reflects the corresponding sensitivity and specificity corresponding to the observed sample. The area under the curve (AUC) refers to the size of the area under the ROC curve, which is a standard used to evaluate the quality of a classification model and reflects the models’ accuracy. Typically, AUC values ranged from 0.5 to 1.0, with a larger AUC representing better model performance.

A ROC curve was plotted, with eHealth skills as the dependent variable and eHealth literacy level as the independent variable. The AUC was employed to assess the predictive power of eHealth literacy on actual eHealth skills. The Youden index was used to determine the optimal cut-off value yielding the greatest combined sensitivity and specificity for eHEALS that indicates sufficient literacy (larger Youden index represented better discriminatory results).

To evaluate the validity of the results, the cut-off value was used to divide the respondents into two groups, those with and without eHealth skills. Descriptive statistics were used to demonstrate the proportion of correct responses for each task, with means and standard deviations of the eHEALS scores and the number of tasks completed in two groups. Chi-square tests were used detect the relationship between eHealth skills task completion and groups. Analysis of variance tests were used to compare the differences in eHEALS scores and number of tasks completed across two groups. Discriminant analysis on the two groups was used to explore the probability of correct classification based on the eHEALS scores.

All the analyses were conducted using IBM SPSS (Version 25).

Results

Descriptive statistics

Slightly over half of the participants were aged below 50 years (53.7%) and female (54.8%). The majority of the participants were Han Chinese (91.4%), domiciled in the local area (99.1%), married (92.7%), and possessed a mobile phone with Internet access (89.1%). More than half had studied only until middle school or lower (64.8%) and living with their children (68.3%). Approximately half of the participants engaged in agriculture, forestry, animal husbandry, fishery, and water production tasks (55.7%) and had a monthly income less than CNY 2000 (56.5%). Detailed demographic characteristics are presented in Table 1.

Table 1.

General sociodemographic characteristics.

Sociodemographic characteristics	n	Percentage (%)
Age
≤50 years	242	53.7
>50 years	209	46.3
Sex
Male	204	45.2
Female	247	54.8
Ethnicity
Han	412	91.4
Others	39	8.6
Domicile location
Local	447	99.1
Field	4	0.9
Education level
Primary and lower	163	36.2
Middle school	129	28.6
High school/secondary	80	17.7
Specialist	46	10.2
Undergraduate	33	7.3
Marital status
Married	418	92.7
Others	33	7.3
Occupation
Agricultural, forestry, animal husbandry, fisheries, and water production tasks	251	55.7
Other	200	44.3
Living with children
Yes	308	68.3
No	143	31.7
Personal monthly income (CNY)
≤2000	255	56.5
2001–4000	121	26.8
4001–6000	49	10.9
>6000	26	5.8
Own a mobile phone with internet access
Yes	402	89.1
No	49	10.9

Ehealth literacy

Among the 451 survey respondents, 49 did not possess a personal electronic device with Internet access and 29 owned but had never used it. Consequently, 373 participants were included in the analysis. The average eHealth literacy score of the participants was 25.85 (SD:10.93). Item scores ranged from 3.12 (SD:1.45) to 3.42 (SD:1.60). The highest scoring item was “I know how to find helpful health resources on the Internet,” and the lowest scoring item was “I feel confident in using information from the Internet to make health decisions.” Specific scores are shown in Table 2.

Table 2.

Ehealth literacy score.

Item	Mean	SD
I know how to find helpful health resources on the Internet	3.42	1.60
I know how to use the Internet to answer my health questions	3.29	1.60
I know what health resources are available on the Internet	3.14	1.55
I know where to find helpful health resources on the Internet	3.20	1.56
I know how to use the health information I find on the Internet to help me	3.24	1.56
I have the skills I need to evaluate the health resources I find on the Internet	3.21	1.51
I can tell high-quality from low-quality health resources on the Internet	3.22	1.47
I feel confident in using information from the Internet to make health decisions	3.12	1.45
Total	25.85	10.93

Ehealth skills

The completion rates for the three tasks varied from 39.1% to 59.8%. The highest was “Search the official data of yesterday's national COVID-19 outbreak” (59.8%), while the lowest was “Please search online for the correct treatment for someone who is experiencing severe heartburn” (39.1%). The distribution of respondents got 0, 1, 2, and 3 items correct were 138 (37.0%), 46 (12.3%), 69 (18.5%), and 120 (32.2%), respectively. Details are provided in Table 3.

Table 3.

Completion profile of eHealth skills.

Item	n	%
Search the official notification data of yesterday's national COVID-19 outbreak	223	59.8
Search for a designated tuberculosis provider in your area	175	46.9
If someone around you is experiencing “severe heart pain,” please search online for the right way to deal with it	146	39.1
The number of respondents got 0 task correct	138	37.0
The number of respondents got 1 task correct	46	12.3
The number of respondents got 2 tasks correct	69	18.5
The number of respondents got all 3 tasks correct	120	32.2

The correlation between eHealth literacy and eHealth skills

The eHealth literacy and eHealth skills were significantly correlated, with correlation coefficients ranged from 0.518 to 0.694. There were also significant correlations between eHealth skills, the correlation coefficients ranged from 0.568 to 0.683. Specific correlations are shown in Table 4.

Table 4.

The correlation between eHealth literacy and eHealth skills.

Item	eHealth literacy	eHealth skills task 1	eHealth skills task 2	eHealth skills task 3	eHealth skills total
eHealth literacy	1	-	-	-	-
eHealth skills task 1	0.694*	1	-	-	-
eHealth skills task 2	0.607*	0.683*	1	-	-
eHealth skills task 3	0.556*	0.568*	0.611*	1	-
eHealth skills total	0.518*	0.565*	0.733*	0.859*	1

The numbers represent the correlation coefficients of the row and column elements in which they are located, *p < 0.001, -: repetition correlation coefficient.

Cut-off points

The AUC for the three eHealth skills predicted by the eHealth literacy ranged from 0.823 to 0.886, with corresponding cut-off values ranged from 22.5 to 28.5. Taken together, the AUC predicted by eHEALS scores was 0.829, while the cut-off value was determined as 29.5 by the Youden index. The sensitivity was 86.7%, and the specificity was 66.8%. Details are shown in Table 5. The receiver ROC curve is presented in Figure 1.

Table 5.

Cut-off values for eHealth literacy levels to predict eHealth skills.

eHealth Skills	AUC	95% CI	Cut-off value	Youden index	Sensitivity (%)	Specificity (%)
Item 1	0.886	0.850–0.923	22.5	0.628	92.8	70.0
Item 2	0.846	0.807–0.885	28.5	0.549	81.1	73.7
Item 3	0.823	0.782–0.864	26.5	0.534	89.0	64.3
Total	0.829	0.788–0.869	29.5	0.535	86.7	66.8

Figure 1.

Receiver operator characteristic curves for eHealth literacy levels to predict eHealth skills. ((a) Task 1, (b) Task 2, (c) Task 3, (d) Total; the red line on the ROC curve corresponds to the diagonal line, which has an AUC of 0.5, indicating the performance of a random classifier that would achieve accuracy equivalent to random guessing. The blue line represents the performance of the classifier in our study, whose ROC curve lies above the diagonal line, indicating better performance than random guessing).

Validation of the cut-off value

The differences in eHealth skills task completion, eHEALS scores, and the number of tasks completed across two groups were all statistically significant. To be specific, in the group with eHealth skills, the proportions of each eHealth skills task completed were all higher than in the group without eHealth skills, ranged from 49.7% to 58.5%, the proportions of all eHealth skills tasks completed was 46.7% higher than in the group without eHealth skills, and the eHEALS scores (34.83 vs 16.68) and number of tasks completed (2.27 vs 0.64) were both higher than in the group without eHealth skills. And according to the discriminant analysis result, 92.2% of original participants were correctly classified. Details are shown in Table 6.

Table 6.

Comparing tasks completion and eHEALS scores in groups with or without eHealth skills.

Item	Possess eHealth skills		χ2/F	p
	Yes (n = 188)	No (n = 185)
eHEALS scores (Mean/SD)	34.84 (3.63)	16.68 (7.78)	843.147	<0.001
Search the official notification data of yesterday's national COVID-19 outbreak (n/%)	167 (88.8%)	56 (30.3%)	132.996	<0.001
Search for a designated tuberculosis provider in your area (n/%)	139 (73.9%)	36 (19.5%)	111.111	<0.001
If someone around you is experiencing “severe heart pain,” please search online for the right way to deal with it (n/%)	120 (63.8%)	26 (14.1%)	96.983	<0.001
The number of respondents got all three tasks correct (n/%)	104 (55.3%)	16 (8.6%)	93.073	<0.001
The number of tasks completed (Mean/SD)	2.27 (0.96)	0.64 (1.01)	253.708	<0.001
Discriminant analysis results	-	-	-	-
Possess eHealth skills /Yes (n/%)	188 (100%)	0 (0%)	-	-
Possess eHealth skills /No (n/%)	29 (17.8%)	156 (88.2%)	-	-

92.2% of participants were correctly classified; -: not involved.

Discussion

The study findings emphasize the low levels of eHealth literacy among rural residents in Gansu. Moreover, this study is the first to determine a cut-off value of 29.5 for adequate eHealth literacy

Compared with previous studies conducted in Zhengzhou(13.76) ⁵³ and Taian (15.80), ³² the eHealth literacy level has improved in Gansu. This improvement could potentially be ascribed to variances in study design; the mean age of our study's respondents was comparatively younger. In rural settings, the health literacy of older residents tends to decrease as a result of a multitude of factors, including personal, interpersonal, and social/community elements.^18,53,54 Consequently, in rural areas, younger populations were more likely to have higher levels of eHealth literacy. Moreover, there were disparities in timing of the studies, with the development of communication technology and the economy, the popularity of smartphones has facilitated access to eHealth services. The COVID-19 pandemic has contributed to the development of eHealth services, likely contributing to the elevation of the population's eHealth literacy levels.^11,55,56 In the early stages of the COVID 19 pandemic, medical consultations in hospitals under the control of the National Health and Wellness Commission grew 17 times compared to the previous year. ⁵⁷ At Alibaba's Ali health, each doctor received an average of 100 patient consultations online per day. ⁵⁸ And Ping An Good Doctor, the most popular online e-health APP in China, had 67.3 million users monthly. ⁵⁹ Despite the progress made, approximately 17.3% (78/451) of the respondents in the study did not own an electronic device with Internet access or had one but did not use it at all. And eHealth literacy remained at low levels among the respondents who have their own electronic devices. A nationwide web-based survey conducted in February 2020 showed that the eHealth literacy of whole population was 30.32, ¹⁰ which was similar to the results of two surveys among university students (30.16 and 30.68, respectively) in China,^60,61 while in our study, eHealth literacy was only 25.85. Low levels of eHealth literacy continue to be a health concern in rural China. In the future, more interventions need to be developed to improve eHealth literacy among rural residents.

Our findings suggested that the actual eHealth skills of respondents are probably not sufficient to fulfill their daily eHealth service requirements. Among the three tasks examined, the best performance was in the COVID-19 epidemic search results. A possible reason was the relative familiarity with the COVID-19 or higher levels of interest, underscoring the crucial role of eHealth in facilitating the control and management of the COVID-19 epidemic.^27,62–64 However, the respondents exhibited relatively weaker eHealth skills in tasks involving other diseases. Thus, there is a general and broad-based requirement to promote general eHealth skills training among residents. The normalization of COVID-19 epidemic prevention and control provided an opportunity for eHealth development including training the public in general eHealth skills.

Additionally, our study also identified a cut-off value for eHEALS scores that indicates adequate eHealth literacy among rural residents in Gansu. The scale is a good predictor of actual eHealth skills (AUC = 0.829) with a cut-off value of 29.5. In validation section, it was ascertained that the value was effective in differentiating between individuals with high and low levels of eHealth literacy. That is, in rural areas, a respondent with eHealth literacy scores greater than 29.5 is likely to possess good eHealth skills. In future eHealth intervention practices, based on this cut-off value, it is possible to quickly categorize respondents and identify priority intervention groups. Furthermore, the cut-off value can be used as a desired goal to guide the development of appropriate intervention strategies and the evaluation of intervention resource investment. Lastly, it can be used to evaluate the effectiveness of the interventions. Following the above steps, the efficiency of interventions may improve, which is helpful in improving the eHealth literacy of rural residents and making eHealth more useful in rural areas. Applying this cut-off value to assess the eHealth literacy of participants indicated that the eHealth literacy levels of rural residents are generally low, probably making it difficult for them to meet daily healthcare service needs. Detailed and comprehensive eHealth literacy interventions alongside the enriching eHealth literacy assessment criteria need to be explored and developed to guide residents toward theoretical to practical behavioral changes. This method and criteria should be validated and examined across other populations and regions in future research.

Conclusion

Our findings indicated the eHealth literacy among rural residents remains at a low level; interventions should be developed to improve the eHealth literacy level within this population. Additionally, we determined a cut-off value of 29.5 for adequate eHealth literacy among rural residents, this method and criteria should be validated and examined across other populations in future research.

Limitations

This study had several limitations. Firstly, due to the absence of a gold standard, we used three self-developed tasks related eHealth skills as a reference standard for assessing eHealth literacy. While we incorporated a wide range of disease elements, the skill assessment tasks might not comprehensively represent the eHealth skill diversity of the respondents. Secondly, due to lack of experience with household surveys, we neglected to record those individuals who declined to participate. Consequently, we were unable to report survey response rates and compare the characteristics of non-participants with those of responders potentially introducing incompleteness or bias into the sample. Thirdly, our sample was obtained by convenience sampling, which might compromise its representativeness and lead to sampling errors. Finally, our sample was from Gansu Province. Consequently, the outcomes may not accurately reflect the eHealth literacy levels of rural residents in other parts of China. The respondents’ eHealth literacy levels might be below the national average due to various factors, including economic level or geographic location.