Evaluation of online learning readiness in the new distance learning normality

Abstract

The COVID-19 pandemic influenced teaching and learning in higher education. The transformation towards digital education challenged Faculty and students. This research examines the online learning readiness of students in a Higher Education Institution in Mexico. Specifically, we investigated how much prior digital skills, as well as having used the digital resources available by the university, influenced their academic achievement in distance learning settings. Seven dimensions of online learning readiness were selected to evaluate the student's preparation for the online learning process. Questionnaires were applied before the start and at the end of digital courses. Follow-up tools were offered to support the student, and two groups were observed, users and non-users of the digital devices. It was observed that students who used the support developed significantly better critical thinking, problem-solving, and time organization skills than non-users. On the other hand, although the evaluations were not significantly different, the lowest averages were found in the non-user group. Our results indicate that prior training in the use of digital tools is essential for the success of online education; in the same way, a timely follow-up with technical and pedagogical assistance is necessary for developing competencies. Training more autonomous and independent students capable of distance learning in a global world demands experts in digital education urgently. Educational institutions must embrace new technologies and teaching methods to meet the ever-changing needs of students. This research is expected to play a crucial role in promoting constructive discussions and facilitating informed decisions concerning the creation of future educational models.

1. Introduction

Distance learning (often-called remote learning) encompasses all forms of education where the learner is physically separated from the instructor or institution [1]. It includes traditional correspondence, video conferencing, satellite broadcasts, and other forms of technology-enabled communication [2]. Distance learning can be conducted synchronously or asynchronously and may or may not involve online technologies [3]. It is also related to online learning (or e-learning), a mode of education delivered through online platforms such as learning management systems, web conferencing tools, and educational apps [4,5]. In this sense, online learning can be perceived as an evolution of distance learning [6]. It allows students to access course materials, participate in discussions, and complete assignments from anywhere on the internet. Online learning also can be synchronous (students and instructors engage in real-time interactions) or asynchronous (students access materials and complete assignments on their own time), or a mix of both [7,8].

Online learning in higher education (HE) levels has grown significantly in the last two decades. In the United States, in 2019, 17.5 % of all students enrolled in degree-granting postsecondary institutions were in exclusively distance education [9]. This statistic differs for other countries. For example, while for some countries such as Spain, Colombia, Sweden, and Germany, the percentage of students exclusively enrolled in distance HE was higher than 15 % (15.4–25 %), in other countries, such as Japan, Slovenia, Poland, Estonia, and Belgium, this figure was close to 0 % (0%–1%); at the same time, in Turkey, Chile, Italy, and Norway only 1.1–8.7 % of students were exclusively in distance-education programs [10]. In Mexico, according to the National Association of Universities and Institutions of HE [11], during the 2019–2020 academic year, 21.5 % of all university students were enrolled in a distance or non-schooled career. This data does not include students whose studies are carried out mainly in person but with one or more courses online.

The COVID-19 pandemic profoundly impacted online learning [12]. With the closure of universities and the need for social distancing, distance learning became the norm for many students and Faculty, and digital technologies played a crucial role in enabling this transition [13,14]. The pandemic highlighted both the benefits and challenges of online learning [15]. On the one hand, it demonstrated the potential for technology to enable flexible and accessible learning opportunities for students who may not have been able to attend classes in person [16,17]. On the other hand, it also highlighted the importance of face-to-face interactions and the social and emotional aspects of learning that may be more difficult to replicate in online environments [18,19]. Overall, it accelerated the adoption of online learning and highlighted the importance of digital technologies in education and the need for HE institutions to be prepared to respond quickly to changing circumstances and to ensure that all students have equitable access to technology and learning resources [[20], [21], [22]].

On the other hand, online learning demands greater self-regulation and other support from Faculty than face-to-face courses [23]. Recent evaluations of online learning courses suggest that online learning methods are less effective when students need more preparation or knowledge of digital tools [24]. Similarly, students' learning outcomes in online courses have been related to several aspects. Readiness for online work among them [25] is fundamental to their preparation. Finding ways to prepare students to be successful in an online learning environment has greatly interested those who design and manage online courses [23,26], particularly with the situation experienced in 2020 with the COVID-19 pandemic.

Online learning requires more independent and autonomous students, capable of organizing, communicating, and managing technology to carry out individual and collaborative work. According to Pearson's 2020 Global Learning Survey, 88 % of people believe that distance-learning experiences are here to stay [27]. Currently, students from all over the world study or have studied at least partially in some distance modality, so understanding the needs of students in these study methodologies will be essential.

2. Aim of this work

Aware of the need to understand the needs of students at a distance or online modalities, this work aims to report on the results of the application of a test for the students of a HE Institution in Mexico to evaluate online learning readiness in seven dimensions: organization of time, self-regulation of learning, interest and motivation, willingness to work at a distance, study habits, literacy skills, and technological skills. At the end of the test, the students, according to the results obtained, receive a series of didactic resources (videos, readings, infographics) to reinforce those skills required to improve some aspects. Our research is based on two specific research objectives.

• 1.To know how students evaluate themselves concerning their preparation, knowledge, and abilities to take subjects in a distance modality, and
• 2.To assess if there is a change in students' self-evaluation after reviewing support resources focused on developing and strengthening their abilities to take subjects in a distance modality.

Thus, the objective of this work is to analyze the impact of previous and systematic digital skills gained through didactic resources on the student's academic achievement. Academics can use the results of this work in many areas of knowledge as a reference for evaluating distance learning readiness as a prerequisite for distance learning to face a new reality such as the one lived in 2020 because of COVID-19.

This manuscript comprises a literature review on online learning readiness, including a Theoretical framework and a Methodology section, where this work's population and sampling, research approach, and research design are presented. Results and Discussion of the results section follow separately, ending with a conclusions section including recommendations and future work.

3. Literature review

Online education provides access to learning experiences without geographical limits [28]. It improves educational opportunities for more people by incorporating technology with flexibility and the ability to promote varied interactions in an academic environment [29]. Designing online learning experiences in HE implies students and faculty have specific skills and abilities to interact in a virtual environment.

3.1. Competencies to interact in virtual environments

Some essential digital skills for HE includes: A) the ability to navigate and use online learning management systems such as Blackboard, Canvas, and Moodle to visualize and understand content; B) The knowledge and the use of software tools to create content, like Adobe Creative Suites or Canva; C) The use of academic databases and scholarly journals, to stay updated; D) The knowledge and use of R or SPSS to analyze data and draw conclusions; and finally, E) The use of digital tools to collaborate with peers and instructor such as Google Docs or Microsoft Teams [[30], [31], [32]]. Incorporating digital skills into HE can enhance student engagement, provide new opportunities for learning and collaboration, and better prepare students for the modern workforce [33].

The proficiency in applying those abilities confidently, critically, and responsibly in a defined context (i.e., education) is a digital competence [34]. Since 2006, digital competence has been one of the eight necessary competencies in the EU for lifelong learning [35]. HE institutions can support the development of digital competencies by providing training and resources for students and faculty, incorporating technology into the curriculum, and promoting it as a core competence for all graduates [36,37].

Digital competencies are essential components of online learning readiness since online learning often requires learners to use technology and digital tools to complete coursework and interact with instructors and peers [38]. Studies indicate that low levels of online learning readiness among students have several essential deficiencies in the degree of participation and the depth of learning, leading them to major academic failures, such as low grades, failed subjects, or even dropping out [[39], [40],[41], [42]]. Therefore, in any online HE program, the student's levels of preparation for online learning must be evaluated to provide them with guidance and timely help and complete their online program [43].

3.2. Online learning readiness

Online learning readiness refers to an individual's ability and willingness to participate and succeed in an online learning environment. Warner et al. [44] first described the concept, focusing on three aspects: 1) students' preferences for the delivery format as opposed to the face-to-face modality, 2) students' confidence in the use of technology for learning, and 3) the ability to participate in self-directed learning activities. Since then, many other researchers have also studied this concept and defined the dimensions of online learning [[45], [46], [47]]. Some factors that may affect online learning readiness are [48].

• •To have equipment and software affordance (computer, reliable internet connection, and access to relevant software and applications).
• •Self-discipline, time management, and working independently (setting goals, managing time effectively, and staying motivated).
• •Basic digital literacy skills (navigating the internet, using email, and online learning platforms).
• •To feel comfortable communicating and collaborating with instructors and peers only online.
• •Support networks (including family and friends) and access to technical support and academic advising may better prepare for online learning.

To assess online learning readiness, institutions may use online readiness assessments or self-assessment tools [5]. These tools can help individuals identify their strengths and weaknesses and provide guidance on improving their online learning readiness. Additionally, institutions can provide resources such as technical support, academic advising, and online learning orientation programs to support students in developing the skills and support network necessary for success in online learning [45].

3.3. Theoretical framework

With the sudden shift towards online learning, assessing the student's ability to engage effectively in this mode of instruction has become essential. To do this, one must understand the theories and concepts underpinning studies in online learning readiness. This evaluation is necessary to adapt to the new normal in education.

3.4. Assessment of online learning environment

In the ever-changing world of online education, assessing prospective students' readiness is crucial. Ensuring they have the necessary competencies is paramount for their success [49]. Tools such as the Online Learning Readiness Questionnaire (OLRQ) and the Online Learning Readiness Scale (OLRS) can be used to assess one's preparedness for online learning. Penn State University has developed the OLRQ, licensed under Creative Commons for public use [49]. This assessment evaluates crucial factors for successful online learning, such as technological proficiency, self-regulation, motivation, and learning aptitude. These factors are distributed in five topics (in the order each section appears on the OLRQ): goal-setting, self-determination for learning, self-discipline for learning, internet self-efficacy, and technology self-efficacy [49]. It is an essential and reliable tool for assessing students' readiness and competence in effectively participating in digital learning environments. The Online Learning Readiness Scale (OLRS) by Ref. [45] focuses on five domains of online learning readiness: self-directed learning, motivation for learning, computer/Internet self-efficacy, learner control, and online communication self-efficacy. Scales such as the Tertiary students' readiness for online learning (TSROL) [50], Online Learning Strategies Scale (OLSS) [51], and others [52,53] have also been developed. Using scales can provide valuable insights into students' readiness levels for online learning, capturing various aspects of their readiness and identifying areas where additional support may be needed.

3.5. Digital skills and study habits

Digital literacy refers to the internet's ability to locate, assess, exchange, and produce information and content [54]. Digital literacy skills, such as information literacy and media literacy (such as reading and writing), are essential for succeeding in online learning [55]. Considering the Covid-19 pandemic and the resulting lockdowns, many higher education institutions are now prioritizing digital literacy readiness [54]. Moreover, a framework of 21st-century digital skills has been recently studied [56]. One elaborated on seven core skills supported by Information and Communication Technologies (ICT): technical, information management, communication, collaboration, creativity, critical thinking, and problem-solving [57]. On the other hand, the conceptual foundation that guides research on online learning readiness also involves different dimensions that may include social presence (examining students' ability to engage in collaborative learning, form meaningful connections, and actively participate in online discussions and group activities) [58], study habits (considering individual differences in learning styles and preferences) [59] and prior experience and support systems. Studies have found that student access to technology, along with their technology skills, lifestyle factors, teaching presence, cognitive presence, social presence, and study habits, determine online learning readiness [60].

3.5.1. Technology acceptance model

The Technology Acceptance Model (TAM) evaluates how people adopt technology based on usefulness and ease of use [61]. It can be used to understand student attitudes toward online learning and factors involved in adopting teleworking, such as training, trust, personal flexibility, and electronic communication [62]. The model has also been extended to consumer behavior patterns like online banking, e-shopping, and telemedicine [[63], [64], [65], [66], [67]]. Studies have shown that faculty members have concerns about student success, reputation, technical support, workload, and class enrollment in online courses [68]. E-learning self-efficacy and subjective norms are critical constructs in explaining the causal process of the TAM model [69]. Performance expectancy, effort expectancy, and social influence positively affect online learning readiness [70]. Research shows that online acceptance is crucial for improving people's behavioral intention toward technology [71]. Studies have shown that factors like perceived usefulness, ease of use, and emotional engagement positively affect students' willingness to learn online [45]. However, technological barriers and lack of institutional support can also affect this willingness [72].

3.5.2. Self-regulated learning

SRL refers to learners' feelings and actions to set goals, manage time, motivate perception, and use strategies to control their behaviors to achieve goals in a certain environment [73,74]. Self-Regulated Learning Theories are conceptual frameworks to understand the cognitive, motivational, and emotional aspects of learning. Several models focus on learners' ability to set goals, manage their time effectively, monitor their progress, and regulate their learning [75]. These theories provide valuable insights into the self-regulatory skills necessary for successful engagement in online learning [76]. During the Covid-19 outbreak, learners must study online and are required to adapt to new learning environments inevitably. SRL and motivation are considered key determinants influencing academic achievement [76]. Understanding the roles of the two factors is essential for stakeholders in higher education since we have been exposed to fully online teaching and learning for almost two years [77]. Lin & Hsieh [78] found that successful online learners make their own decisions to meet their needs at their own pace and by their existing knowledge and learning goals.

3.6. Supporting students in their online learning readiness

Taking courses specifically designed to enhance online learning readiness has been found to positively impact students' preparedness for online learning [79]. These courses should familiarize students with the technical tools, communication platforms, and learning strategies essential for successful online learning experiences. Research demonstrated that students who participated in an online learning orientation course reported lower anxiety levels [80]. Preparation in digital knowledge before taking online classes is also a critical point in determining the success of any online course, including the danger of deviations (tricks) in the digital expertise of “digital natives” due to digital growth based on electronic games or the increased use of the internet as a tool for services and information source [81].

These findings underscore the significance of targeted interventions in enhancing students' readiness and promoting a smoother transition to online learning environments. Besides, some researchers to have a positive correlation with students' learning achievement [82] have investigated students’ online learning readiness. Under these frameworks, we examined the effectiveness of preparatory courses in enhancing students' online learning readiness. We explored the relationships between the intervention, students' attitudes, and motivation and how this impacts the resulting readiness outcomes. We employed some of the constructs already mentioned in this theoretical framework and developed a questionnaire for understanding the potential benefits of preparatory courses in supporting students' transition and success in online learning environments.

4. Methodology

This project had two main objectives; the first one was to know how students self-evaluate regarding their preparation to take courses online, and the second one was to evaluate whether, over some time, this self-evaluation changed explicitly due to the impact of a strengthening strategy based on several educational resources.

4.1. Population and sampling

Convenience sampling was employed to randomly select 196 students, 98 men and 98 women, to whom self-assessment instruments were applied. All students were from the first three semesters located nationwide at the different campuses of the university enrolled in online classes. The students who participated in the study were between 18 and 21 years old.

4.2. Ethical considerations

The complete self-evaluation and follow-up project considered ethical aspects, the confidentiality of the student's data, their responses, and the non-execution of academic judgments and categorization of the results; the center of the strategy is to support students to strengthen their skills and have a better college career. A group of pedagogical and learning psychology experts from the Digital Education area of Tecnologico de Monterrey was summoned for the survey's design. After integrating the descriptions with their indicators, the validity and reliability of the survey were evaluated through a qualitative review with the judgment of experts [83]. For this, a general review script was designed using google forms to record comments (data not shown). In the second evaluation, a differentiating question was integrated to find out the students who did consult the digital resources provided by the university.

The approval of the Institutional Research Ethics Committee of Tecnologico de Monterey was sought. However, there was no number attached to the approval since ethical review and approval were waived for this study for the following reason: the data collected from the workshop was originally for the sole purpose of meeting the ViceRectory for Academic Affairs and Educational Innovation's objectives, i.e., providing effective, educative digital resources for the students of Tecnologico de Monterrey. The data were not collected for human subject research following the local legislation and institutional requirements. Additionally, the data was collected anonymously without identifiers. Written informed consent to participate in this study was not required from the participants, as the study involves only the analysis of non-identifiable private information in accordance with national legislation and institutional requirements.

4.3. Research approach

The study was conducted in 2020, at the beginning and end of the fall semester, with a quantitative approach and a descriptive scope incorporating a pre-experimental pre-post-test design. The students received an email invitation to carry out the self-assessment to determine their level of preparation for online learning. After answering the instrument, each student received a series of resources to reinforce those dimensions in which they self-assessed at a low-performance level or to strengthen: “I need to improve” and “moderately prepared”. The second application, made at the end of the semester, was like the first survey but with two additional questions regarding the resources’ usefulness and the final grade.

4.4. Instrument design

In the design of the self-assessment instrument, seven dimensions to evaluate the preparation of the student in the online learning process were selected.

• (1)Technological ability: The ability to search for information, process, and communicate efficiently through any digital device.
• (2)Reading and writing skills: Reading and writing comprehension skills to process information through digital devices.
• (3)Study habits: Methods and strategies for studying learning contents.
• (4)Willingness to work remotely: Intention to work remotely using different electronic means.
• (5)Interest and motivation: Emotional state that drives the student to undertake and maintain a behavior with a specific objective. In this case, it is a predisposition to act in a particular direction.
• (6)Self-regulation of learning: Ability to control the learning pace (self-management).
• (7)Organization of time: Planning and organization of the time devoted to the study of the learning contents and development of activities (individual and collaborative)

These dimensions were considered after reviewing multiple self-assessment instruments and selecting those of interest for Tecnologico de Monterrey. The final version of the instrument entitled "Do you feel prepared?" integrated the seven dimensions mentioned in the experimental design. A Likert scale was employed using four levels of achievement: 1) I need to improve, 2) moderately prepared, 3) prepared, and 4) highly prepared.

4.5. Research design

The process and the experience the student would have to participate in this project were designed. The strategy focused on curating and designing educational resources that could support the student in improving their preparation for online study. It began with the search for open educational resources that would help develop and strengthen the dimensions of online learning readiness. Subsequently, some were chosen and shared with the students due to their quality, duration, and relevance for the HE student. In total, there were three resources for each dimension.

When the resources were ready, the students received an email explaining the project and the invitation to join in answering an instrument to detect deficiencies. This was done before the start of the Fall term. Afterward, students could study the provided resources independently and attend their digital classes regularly. At the end of the term, those students who choose to participate in the study answer the instrument again, including the two new questions regarding the usefulness of the resources and final grade.

4.6. Data analysis and evaluation of the results

The instrument's internal consistency was evaluated using Cronbach's alpha coefficient, which consists of calculating the variance of the results of the responses [84]. Statistical analysis was performed using the MINITAB® software (Pennsylvania, United States) for data processing.

5. Results

To find out how the students self-assessed in terms of their level of preparation to face a digital course, a descriptive statistical analysis was carried out using bar graphs that allowed us to analyze the self-assessment (Fig. 1). Of the 196 students who participated in the study, 165 indicated that they did consult the institution's digital resources during the course, and 31 did not use them. The instrument's internal consistency was evaluated using Cronbach's alpha coefficient, which consists of calculating the variance of the response results. In the first self-evaluation (before the beginning of the course), a Cronbach's alpha coefficient of 0.746 was obtained, and in the second (after the term ended), a coefficient of 0.769 was observed. According to the literature, the minimum acceptable value for the Cronbach coefficient is 0.70 [84], so we determined that the current version of the instrument has internal consistency.

Fig. 1.

Four Level Dimension Graph for the student's online learning readiness self-evaluation. A. Distribution of answers from students who consulted the digital resources provided by the University (n = 165). B. Distribution of answers from students who did not consult the resources (n = 31). The first column (1) in each pair reflects the answers of the first survey (at the beginning of the fall term) and the second (2) of the final survey (at the end of the fall term).

Fig. 1A shows the results of the 165 students that consulted the resources. The seven dimensions and the level of the student's initial (first) and final (second column) self-evaluation results are shown. As it can be seen, the dimensions in which the students were self-considered well prepared before starting the course were technological skills (38.79 % answered "highly prepared" and 47.88 % "prepared"), reading and writing habits (29.09 % "highly prepared" and 52.73 % "prepared") and study habits (27.88 % "highly prepared" and 50.30 % "prepared"). The dimensions in which these students self-evaluated the lowest before the course was: interest and motivation (7.88 % answered "I need to reinforce" and 33.33 % "moderately prepared") and willingness to work at a distance (8.48 % "I need to reinforce" and 24.85 % "moderately prepared") and finally the organization of time (2.42 % "I need to reinforce" and 27.88 % "moderately prepared").

As can be seen, the dimensions in which the students were self-considered well or highly prepared after taking this course (second column in each pair) were technological skills (49.70 % answered "highly prepared" and 41.82 % "prepared"), reading, and writing habits (36.97.14 % "highly prepared" and 53.33 % "prepared") and study habits (36.97 % "highly prepared" and 53.33 % "prepared"). The dimensions in which these students self-evaluated the lowest after the course were: interest and motivation (8.48 % answered "I need to reinforce" and 35.15 % "moderately prepared") and willingness to work at a distance (9.09 % "I need to reinforce" and 20.61 % "moderately prepared") and finally the organization of time (2.42 % "I need to reinforce" and 27.88 % "moderately prepared").

By contrast, Fig. 1B shows the same survey results but considers the answers of 31 students who did not consult the resources in this case. As can be seen, the dimensions in which the students were self-considered well prepared before starting the course were very similar to the previous group of study: technological skills (41.94 % answered "highly prepared" and 45.16 % "prepared"), reading, and writing habits (25.81 % "highly prepared" and 54.84 % "prepared") and study habits (25.81 % "highly prepared" and 51.61 % "prepared"). The dimensions in which this second group of students self-evaluated the lowest before the course were: interest and motivation (29.03 % answered "I need to reinforce" and 25.81 % "moderately prepared") and willingness to work at a distance (16.13 % "I need to reinforce" and 32.26 % "moderately prepared") and finally self-regulation of learning (9.68 % "I need to reinforce" and 38.71 % "moderately prepared").

The dimensions in which the students were self-considered well prepared after the course were: technological skills (41.94 % answered "highly prepared" and 45.16 % "prepared"), reading and writing habits (25.81 % "highly prepared" and 54.84 % "prepared") and study habits (25.81 % "highly prepared" and 51.61 % "prepared"). The dimensions in which this second group of students self-evaluated the lowest after the course were: interest and motivation (25.81 % answered "I need to reinforce" and 41.94 % "moderately prepared") and willingness to work at a distance (25.81 % "I need to reinforce" and 35.48 % "moderately prepared") and finally the organization of time (32.26 % "moderately prepared").

To determine the effect of the course on the development of the seven dimensions studied and, on the other hand, the impact of digital resources, we conducted a fine study of the data from the self-assessment survey. First, we added the percentages of "highly prepared" and "prepared" as a positive evaluation and, on the other hand, the rates of "I need to reinforce" and "moderately prepared" as a negative evaluation. Thus, we can compare each opinion and its variation before and after the course. Our analysis allows us to evaluate the impact of the use of digital resources available by the institution. Table 1 shows the percentages of change in each of the opinions before and after taking the course by users of the resources and non-users. A negative percentage implies that the readiness level decreased after the course, and a positive variation indicates that the readiness level increased.

Table 1.

Variation in the percentage of positive (highly prepared" and "prepared") or negative ("I need to reinforce" and "moderately prepared") answers of each online learning readiness dimension before and after the fall term from students who used digital resources and those who did not use them.

Dimension	Users of Digital Resources		Non-Users of Digital Resources
	Negative Opinion	Positive Opinion	Negative Opinion	Positive Opinion
Technological Skills	−4.84	+4.84	+0.01	−12.91
Reading and Writing Skills	−8.49	+8.49	0.00	0.00
Study Habits	−8.48	+8.48	+3.23	−3.23
Willingness to Work Remotely	−3.63	+3.63	+12.90	−12.91
Interest and Motivation	+2.42	−2.42	+12.91	−12.90
Self-Regulation of Learning	−12.01	+12.11	−19.36	+19.36
Time Organization	−1.83	+5.50	−16-14	+12.91

We can see in Table 1 that negative opinions on technological skills decreased −4.84 %, a percentage that was added to positive opinions; for the case of students who did not use digital resources, there was no significant variation between before or after the course in both opinions. Opinions on reading and writing skills had very similar behaviors where negative thoughts decreased −8.49 % in users of the resources, and there was no variation in non-users. Regarding the dimension of study habits, there was a significant variation in its improvement in the users of resources (+8.49 %). However, non-users felt that the deficiency in study habits increased (+3.23 %). The availability of working remotely increased the favorable opinion of resource users; in contrast, those who did not use digital resources increased negative thoughts by +12.90 %. Regarding interest and motivation, in both cases, negative opinion increased; for users, it increased by only +2.42 %, but for non-users of digital resources, it increased by +12.41 %. Interestingly, in both cases, the dimension of self-regulation of learning increased the positive opinion significantly in resource users by +12.11 %. However, there was an increase of +19.36 % in non-users of resources, the most significant increase in a favorable opinion. Similarly, the positive opinion on the time organization increased by +5.50 and + 12–91 % among users and non-users, respectively.

5.1. Statistical analyses

We were interested in analyzing whether there was a statistically significant difference between the results of group 1 (users of digital resources) versus the results of group 2 (non-users of digital resources) in the second self-assessment (after the course). First, a Kolmogorov-Smirnov regular distribution test was performed [85], and the analysis showed that the data did not have a normal distribution (data not shown). Subsequently, the Mann-Whitney test was chosen, which is used to calculate the difference between medians of a population. This test evaluates two independent samples, which compare two groups of cases with a variable, in this case, the use or not of educational resources [86].

Therefore, a comparison was made between groups 1 (n1) and 2 (n2), and the significance was analyzed to determine in which dimensions there was a significant difference. This was done considering 95 % confidence, so the values that showed to be below 0.05 of significance show a statistical difference (Table 2).

Table 2.

Statistical analyses of the answers of each online learning readiness dimension before and after the fall semester from students who used digital resources and those who did not use them.

	Mann-Whitney
Median of n1 (users) minus median of n2 (non-users)
Difference: n1 - n2
Confidence achieved	95 %
Dimensions	p-value
Time organization	0.080
Self-regulation of learning	0.007
Interest and motivation	0.003
Willingness to remote work	0.001
Study habits	0.110
Reading and writing skills	0.103
Technological skills	0.037

After the analysis, it was identified that there is a significant difference in 4 of the dimensions between the two groups analyzed and considering the consultation of resources as the variable: Self-regulation for learning, Interest and motivation, Willingness to work at a distance, and Technological skills.

5.2. Performance indicators

After analyzing the instrument results, we investigated how much prior digital skills, as well as having used the digital resources available by the university, influenced academic achievement. Fig. 2 shows the student's final grade during the fall semester 2020. Fig. 2A shows the total scores (n = 196), showing a mean of 95.15. When analyzing the grades of the 165 students who did use digital resources, the average was 95.2 (Fig. 2B), and when the 31 students who did not use the resources, the average was 94.87 (Fig. 2C). However, the frequency curve clearly showed a distribution with lower grades.

Fig. 2.

Grades of students after a semester of digital courses. A) Total sample of students, B) Students that took the digital resources available from the institution, and C) Students that did not take the digital resources.

6. Discussion

To succeed in various fields, including higher education, possessing digital literacy has become increasingly necessary in today's digital age [87,88]. The ability to effectively use technology and digital tools has become crucial to success [[80], [81], [82], [83], [84], [85], [86], [87], [88], [89], [90], [91]] and is applicable across diverse fields and disciplines. Being born between the mid-1990s and the mid-2000s (Z generation, often referred to as “digital natives”) does not automatically mean that one has the necessary digital skills to navigate and use technology effectively [92]. While the Z generation may be familiar with digital technology, they still need to develop these skills to use technology effectively and responsibly [93]. Moreover, not all members of the Z generation have equal access to digital technology and digital literacy training. Differences in socio-economic status, geographic location, and cultural background can affect access to digital technology and digital literacy resources. Therefore, it is essential to provide digital literacy training and resources to all learners, regardless of their background or level of digital experience [94].

During 2020 and 2021, the challenge of transforming most face-to-face courses to online courses due to the COVID-19 pandemic brought concern about how much Faculty and students were prepared to face a sudden distance education. What tools do we have to face this challenge? The present manuscript focused on answering this question by analyzing online learning readiness and following up with digital tools to support students who require them. Even today, all the factors studied in this work are necessary for successful online learning. Learners need to have the technological ability, to be proficient in using digital tools and be prone to access online resources, participate in online discussions, and complete assignments [95,96]. Online learning often involves reading and writing assignments and participating in online discussions. Learners need strong reading and writing skills to understand the course material and communicate their ideas effectively [97,98]. They must develop effective study habits, such as setting goals, managing time, staying organized, and being self-directed and motivated [99,100].

When comparing the results of the student users between the first and second evaluations, it was observed that there was a substantial improvement in six of the seven dimensions. The group obtained positive increases in the dimensions of time organization and self-regulation of learning. In contrast, Zinchenko Yury et al. [101] found that students had more difficulty organizing their time during pandemics than older adults. Older adults can independently adapt and find self-regulation strategies [102]. Thus, the improvement observed in this work may result from using resources and having other subjects online. However, this could result from maturing as an independent student by taking the entire semester online; this positively impacted their self-assessment and, more than that, their growth and development as separate individuals.

On the contrary, the only category with a slight decrease was the willingness to work remotely. This result may be a consequence of the fatigue of students studying online learning throughout the semester. Other works have also encountered this [103]. They may be attributed to all the barriers faced during pandemics, which also impacted social connections and healthy habits [104]. The willingness to study at a distance is essential to students’ success and satisfaction with online learning [71].

The students who did not consult the resources decreased their self-evaluation in the “highly prepared” category in five dimensions (organization of time, self-regulation of learning, interest, motivation, willingness to work remotely, and technological ability). They remained the same in study habits and reading and writing skills. Additionally, it was observed that the students who did not consult the resources obtained negative results in motivation and disposition of the students to work remotely. When we compared the effects of both groups, it was clear that while the group that did consult the tools improved their abilities to take online courses, the other group had a decrease. Thus, the available tools may have had a positive effect. Support strategies for online learning readiness dimensions, such as self-regulated learning, have also positively affected other contexts [105].

When evaluating the impact that the use of resources could have on the student's academic performance, the final grade average was compared; there was no significant difference between the students who used the resources and those who did not use them (data not shown). However, the lowest scores belonged to the second group. Studies have found a positive correlation between the use of digital resources and academic achievement [106,107] and the use of digital resources on student engagement and motivation, which can, in turn, lead to improved academic performance [108]. For example, the study by Chen & Jang [109] found that digital storytelling increased student motivation and engagement, improving academic performance. However, it is essential to note that using digital resources alone is insufficient to improve academic performance. The effective use of digital resources requires careful planning, appropriate instructional design, and ongoing support and feedback [110]. The context and an effective and meaningful use must be considered as well.

Digital resources allow students to access learning materials more conveniently and flexibly, collaborate with their peers, and interact with course content. Offering digital resources to students is an effective way to enhance their skills in studying remotely and increase their achievements in this mode of learning. The aim is to support students in developing their competencies in distance learning and guarantee their success in this mode. Therefore, a punctual and timely intervention by the HE institutions to ensure the online preparation of its students can mean better academic results and be reflected in greater student satisfaction with their courses and better self-perception [111].

To tackle the issue of dropouts in higher education, digital resources can be a helpful tool [112]. By offering students flexible learning options, such as accessing course materials and participating in discussions remotely, universities can enhance learning accessibility and improve student retention rates. In addition, digital resources can cater to personalized learning, making education more tailored to the needs and preferences of individual students [113]. However, it's important to note that more than digital resources are needed to solve the complex issue of HE dropouts. Other factors, such as academic and social support, student motivation, and financial barriers, must also be addressed by universities to achieve long-lasting results [114].

7. Conclusion

The teaching-learning process is not static but dynamic and constantly evolving in response to changes in technology, society, and the environment. As technology advances and society changes, educators must adapt their teaching methods and learning materials to ensure students receive a relevant and practical education. For example, the widespread availability of digital tools and online resources has transformed how educators deliver instruction and students access and engage with learning materials. Similarly, societal and environmental changes such as the COVID-19 pandemic have necessitated a shift towards remote and online learning, presenting new challenges and opportunities for educators and students.

Universities can measure and improve digital skills among students by conducting digital skills assessments to evaluate students' digital skills and identify areas where they need improvement. Digital skills assessments can take many forms, such as online quizzes, self-assessments, or project-based assessments. They offer digital skills training to help students develop their skills to succeed in the digital age (workshops, online courses, or peer-to-peer learning opportunities). Embedding digital skills into the curriculum ensures HE students develop the skills they need to succeed in their fields. This can be done by providing access to digital tools and resources, such as online libraries, digital media production facilities, and software programs, to enable students to develop their digital skills. However, while digital resources can contribute to developing students’ abilities to study in a distance environment and improve their academic success, universities must also consider other factors to reduce dropouts in distance settings.

This research has some limitations, as the fact that all the courses that students took during the semester were online due to the pandemic may have influenced the results in the student's self-assessment of their abilities to study online and in a sudden saturation due to the enforced online modality. However, this type of study will be beneficial for the future discussion of future educational designs.

Recommendations and future research avenues

To ensure that the teaching-learning process remains effective and relevant, HE institutions must stay up-to-date with the latest technological and societal trends and assess and adapt their instructional strategies and materials to meet the evolving needs of students. It requires a commitment to ongoing professional development and a willingness to experiment with new teaching methods and technologies to create an engaging and effective student learning experience.

An aspect of this work to consider for future research is to define an instrument or mechanism to evaluate the student's skills to take their subjects online and compare these results with the self-evaluation that the student does. Another aspect to be assessed in future research is investigating why students score lower in interest, motivation, and willingness to work remotely. A third aspect to consider is whether these results are consistent with students from other educational levels, such as baccalaureate and postgraduate.

Funding

No funding was received to produce this work.

Availability of data and materials

The datasets generated and analyzed during the current study are available from the corresponding author upon reasonable request.

Data availability

The data associated with the study has not been deposited into a publicly available repository. Data will be made available on request.

CRediT authorship contribution statement

Maribell Reyes-Millán: Conceptualization. Myriam Villareal-Rodríguez: Conceptualization. M. Estela Murrieta-Flores: Formal analysis, Investigation, Methodology, Writing – original draft. Ligia Bedolla-Cornejo: Conceptualization. Patricia Vázquez-Villegas: Investigation, Writing – original draft, Writing – review & editing. Jorge Membrillo-Hernández: Investigation, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.