Design principles for internet skills education: results from a design-based research study in higher education

Abstract

The current generation of students lives in a globally connected world where Internet technologies are ubiquitous. As a result, they learn how to use various digital tools from a very young age, although, for the most part, the skills they develop are not adequate to use internet technologies in academic settings effectively. Additionally, while digital skills are essential in Higher Education (HE), developing programs for the effective use of internet technologies is still an issue in question. Research lacks empirical investigations regarding the design of such programs. To address this gap, the authors applied a design-based research (DBR) methodology to empirically explore an instructional intervention that aimed to enhance undergraduate students’ digital skills for the effective use of the Internet during their studies. Specifically, the authors drew from multiple sources and utilized a triangulation approach to interpreting the findings emphasizing the aspects of digital skills development and assessment and learning design. The results clearly show that the project-based learning intervention and the proposed design principles can positively impact digital skills development and support learning in academic settings. The authors conclude with implications for further research in the field focusing on digital skills frameworks, assessment instruments, instructional approaches, and learning content.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43545-022-00428-2.

Introduction

Several organizations worldwide have acknowledged that the preparation of future generations for the digital world is necessary as digital skills are becoming increasingly important in a wide range of professions (Adams Becker et al. 2018; Jørgensen, 2019; Sursock, 2015). In this context, universities ensure that graduates possess the necessary skills to utilize digital technologies and be prepared for the labor market (Jørgensen, 2019). Digital skills are also crucial during university studies (Brooks, 2016; Dahlstrom and Bichsel, 2014). Students who possess such skills can access services, support, and information provided by HEIs (Beaunoyer et al. 2020). Such skills in academic settings were particularly prominent during the COVID-19 pandemic, which raised the demand for HEIs to use technologies to deliver courses due to the immediate call for online teaching and virtual education (Beaunoyer et al. 2020; Daniel, 2020). Specifically, studies have reported that the transition from F2F teaching to online instruction was challenging due to the lack of students' digital skills and suggested developing relevant programs in HE (Aristovnik et al. 2020; Turnbull et al. 2021). However, the need to address students' lack of digital skills is not new. Scholars have previously debated whether HE students have the capabilities to take full advantage of technology for learning purposes (Bullen et al. 2011; Helsper and Eynon, 2010; Kennedy et al. 2008).

Previous research has mainly focused on understanding the purposes and practices of technology use among young students (Corrin et al. 2010; Gallardo-Echenique et al. 2015; Kennedy and Fox, 2013). Researchers reported several other factors influencing the effective use of technology for learning in HE. Such factors are closely related to the instructional practice and the learning experience (Bond et al. 2018; Kirkwood and Price, 2014; Littlejohn et al. 2012), including the perceived affordances of digital technologies to support learning (Margaryan et al. 2011; Ng, 2012), and the application of instructional practices that foster technology integration (Kivunja, 2014). Previous research reported a lack of Design-Based Research (DBR) approaches that could inform the formulation of policies regarding the methodology of teaching and learning for digital skills education (Spante et al. 2018). Accordingly, to address this gap, the present study applied a DBR approach aiming to answer the following questions:

• What are the characteristics of an effective program aimed at developing students’ digital skills in the context of HE?

• Which design principles could support the development of digital skills programs in HE?

In the following sections, the authors present the literature review regarding digital skills in the context of HE, the instructional practices that can support the development of such skills, and the existing frameworks for digital literacy. Consequently, the authors describe the DBR approach focusing on the processes and the results, followed by a discussion on the why, what, and how of learning with internet technology in HE and implications for further research.

Students’ digital skills in the context of HE

Today’s university students belong to Generation Z and have experienced a globally connected world where the Internet is readily available (Seemiller and Grace 2017). They have also been called “digital natives” due to their constant immersion in technology (Evans and Robertson, 2020). As a result, they have already experienced various online tools and services before entering HE (Guzmán- Simón et al. 2017; Gurung and Rutledge, 2014; Brooks, 2016; Bond et al. 2018). However, research has shown that such experiences are limited, not homogeneous, and might not adequately cover all the skills individuals should possess to take full advantage of digital technologies, especially for academic purposes (Corrin et al. 2010; Kennedy and Fox 2013; Ng, 2012; Šorgo et al. 2017).

More specifically, students seem familiar with online communication tools (Bullen et al. 2011; Kennedy and Fox, 2013; Margaryan et al. 2011) and social networking services (Gabriel et al. 2012; Gosper et al. 2013; Shopova, 2014). Also, they are familiar with Learning Management Systems (Bond et al. 2018; Ng, 2012), search engines, and Wikipedia (Biddix et al. 2011; List et al. 2016; Margaryan et al. 2011). However, they experience difficulties applying more advanced information search strategies and selecting credible resources to support their academic work, such as libraries and institutional repositories (Hargittai et al. 2010; Head 2013; Lee et al. 2012). They also find it challenging to critically evaluate and use online information sources to complete a task (List et al. 2016). Furthermore, they do not feel competent in applying copyright-related regulations when using online content (Gudmundsdottir et al. 2020; Shopova, 2014). Regarding the learning experience, research shows that students use the Internet more for “consuming” rather than creating content (Kennedy and Fox, 2013; López-Meneses et al. 2020; Ng, 2012, 2015). More recently, studies regarding the use of digital tools for online instruction during the Covid-19 pandemic reported that students felt confident about their skills in using communication platforms, browsing online for information, and sharing digital content (Aristovnik et al. 2020). However, they reported a lack of skills in critically evaluating online information (Sales et al. 2020), adjusting advanced settings of software and programs, and using online platforms, such as Learning Management Systems (Aristovnik et al. 2020). Students also faced challenges in delivering online teachings, such as increased workloads, minor interactions, poorer communication, and confusion (Lemay et al. 2021).

Instructional practices for utilizing digital technologies in HE

Research on digital technologies for teaching purposes in HE has reported a technology-led rather than a technology-enhanced learning approach (Kirkwood and Price, 2014). Specifically, studies have shown that students are not aware of the affordances of digital technologies to support learning (Brooks 2016; Ng, 2012), even though understanding the characteristics and benefits of each tool is essential for academic practice (Kennedy and Fox, 2013; Ng, 2012). Additionally, while universities play a vital role in the digital transformation of today's societies, they rarely make the necessary distinction between the individual needs of students (Jørgensen, 2019). Specifically, a lack of customized support highlights the limitations and benefits of using such digital tools to meet students' individual learning needs (Beetham et al. 2009). The above fact contradicts studies showing that students' digital skills and familiarity with digital devices and tools vary (Corrin et al.; Hargittai, 2010; Helsper and Eynon, 2010).

Regarding students' preferences about learning with digital technologies, research reported approaches that promote learning through discovery and experimentation, using audiovisual sources, working on different tasks simultaneously (multitasking), and getting immediate satisfaction (Teo, 2016). However, researchers have pointed out that the continuous exposure of this generation to digital content seems to be associated with the possibility of distraction from the subject matter, lack of concentration, and the difficulty in communicating work correctly (e.g., grammar, spelling, writing style) (Brooks, 2016; Issa and Isaias, 2016). Consequently, instructional practices in academic settings should incorporate alternative teaching approaches to enhance students' participation in the learning process and address the challenges above, considering students' different characteristics and learning expectations (Kennedy and Fox, 2013; Ng, 2012). Previous studies reported that teaching with digital technologies in the context of HE should capitalize on constructionist approaches that promote hands-on learning experiences and active participation, such as inquiry-based learning, problem-based learning, and project-based learning (Kivunja, 2014; Wekerle et al. 2022; Guo et al. 2020; Ng, 2015).

Frameworks for the development of skills for using digital technologies

During the past decades, research in HE regarding the use of digital technologies has been associated with the terms digital literacy and digital competence (Spante et al. 2018), as the literature converged on the use of the term “digital” (Goodfellow, 2011). According to Spante et al. (2018), a general categorization relates to the context when using the above terms. At a policy-making level, the term digital competence appears more often. For example, in the European Commission's “DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe,” digital competence refers to the self-confident, critical, and creative use of digital technology for accomplishing goals in various contexts, such as work, academic, or leisure (Ferrari, 2013). At a research level, digital literacy denotes an approach based on acquiring skills and know-how (Spante et al. 2018). Digital literacy refers to “the awareness, attitude and ability of individuals to use digital tools and facilities to identify, access, manage, integrate, evaluate, analyze and synthesize digital resources, construct new knowledge, create media expressions, and communicate with others, in the context of specific life situations, to enable constructive social action; and to reflect upon this process” (Martin and Grudziecki, 2006, p. 255). Other researchers have used the term “digital literacies” to refer to the use of digital technologies in situated knowledge practices in the academic environment (Goodfellow 2011; Lankshear and Knoebel, 2008; Littlejohn et al. 2012). Van Dijk and Van Deursen (2014) have adopted the term “digital skills” to emphasize the interactive performance of digital media and refer to “interactions with programs and other people, transactions in goods and services and continually making decisions” (p. 140).

Since the Internet is the medium that university students use to access services and resources for study purposes (Ng, 2012), the present study adopts a skills-oriented approach focusing on the digital skills required to use the Internet. Such an approach was necessary to gather measurable results and draw conclusions (Iordache et al. 2017). Several researchers attempted to identify the skills for using the Internet effectively. Hargittai (2005) proposed “web skills,” focusing on effective information retrieval. Potosky (2007) referred to “internet knowledge,” which relates to the familiarity with terms connected to the Internet, such as the browser, and the knowledge of the processes for carrying out tasks using the Internet, such as practical information retrieval. In the same context, Livingstone and Helsper (2010) defined “internet literacy” as a multidimensional concept that includes accessing, analyzing, evaluating, and creating online content. Bunz (2004) introduced “computer-email web (CEW) fluency,” which refers to the use of the Internet for information and communication purposes. Finally, Van Dijk and Van Deursen (2014) proposed a theoretical framework that classified Internet skills into six areas: (a) Operational, (b) Formal, (c) Information, (d) Communication, (e) Content Creation, and (f) Strategic. They also suggested a skills distinction between the technical aspects of using the Internet as a medium (medium-related skills) and the fundamental aspects related to online content (content-related skills).

Research purpose

To remedy the lack of research for investigating the teaching of digital skills education in the context of HE, the authors of this paper adopted a DBR approach to study the design and development of programs for the effective use of the Internet in higher education learning environments.

Method

DBR aims to develop new theories, artifacts, and practices that potentially impact learning and teaching in natural environments (Anderson and Shattuck, 2012; Barad andSquire, 2004). The present study’s character was developmental and had two objectives: (a) the development of an intervention proposed as a solution to a problem and (b) the construction of design principles (Plomp, 2007). The research design included three phases which run for three consecutive semesters during a university course.

On a conceptual level, the development of the intervention was based on Van Dijk and Van Deursen’s conceptualization of digital skills. This approach is appropriate because (a) it covers the skills identified as essential during the literature review, and (b) it includes several skills that refer to processes rather than the use of specific Internet tools. Specifically, Operational skills refer to the most basic technical skills required to use the Internet, such as browsers to access web applications. Formal skills refer to navigating through various websites with different layouts. Information skills indicate the skills for searching, selecting, and evaluating online information. Social skills refer to using online communication services, interacting with others, and exchanging meaning. Creative skills are the skills someone needs to create different types of acceptable quality content (e.g., text, audio, video) and publish it or share it with others on the Internet. Lastly, Strategic skills refer to the fulfillment of personal goals using the Internet (e.g., making the right decisions toward achieving a goal and securing the benefits of using the Internet).

Participants

The intervention took place during three consecutive semesters, comprising three research cycles. The total number of participants was 58 university students who attended an elective course about the use of internet technologies. Approximately, 26% of the students were males, and 74% were females.

Research instruments

For the present study, the authors have utilized four different instruments for data collection:

1. Students' self-assessment questionnaire for internet skills

2. Design materials for the learning environments

3. Open-ended questionnaire to gather students' views about the learning experience

4. Observation data from each session

The students’ self-assessment questionnaire was structured on a five-point Likert scale ranging from “Not at all true of me” to “Very true of me” and consisted of skills items from the Internet Skills Scale (ISS) proposed by Van Deursen, Helsper, and Eynon (2014, 2016). The ISS followed the theoretical framework proposed by Van Dijk and Van Deursen (2014) and the methodology used by Helsper and Eynon (2010) to distinguish the different types of Internet use. The researchers validated the questionnaire for the context of HE (Miliou and Angeli, 2021). The final version consisted of 27 items referring to five skills areas: Operational, Information-Navigation, Social, Creative, and Critical (Online Appendix 1). Due to small sample sizes per research cycle, the authors performed a within-subjects comparison during pre-and post-intervention measures with the Wilcoxon Signed Rank Test (Cohen et al. 2018). The results were used to draw conclusions about students’ skills levels before and after the intervention and refine the learning materials in the upcoming research cycles based on their needs. The students' views about the learning experience emerged from an open-ended questionnaire issued during the first research cycle (Online Appendix 2). The answers were analyzed using content analysis, suitable for analyzing qualitative data (Popping, 2015). The design data referred to the tools and activities designed during the second and third research cycles. They were classified chronologically and thematically. Finally, the researchers recorded field notes during all research cycles for all groups taught and observed by the researchers. Thematic coding analysis followed, which corresponded to the ISS skills areas. Such type of analysis was necessary due to the comparative nature of the study (Cohen et al. 2018).

Research procedures

The authors followed an iterative design process by implementing three research cycles. In Cycle 1, the intervention followed the established course content and teaching methods (Online Appendix 3). The research purpose was exploratory, aiming to examine students' familiarity with internet technologies, their skills level, and their learning preferences. At first, researchers distributed an open-ended questionnaire that referred to students’ preferences regarding their learning experience. Secondly, the self-assessment questionnaire was administered before and after the intervention to determine the effectiveness of the teaching process in enhancing students' skills. Finally, the field notes focused on usability and affordances and informed the future design and development work. The authors reflected on the outcomes and adjusted the learning materials to improve the intervention during Cycle 2. Self-assessment questionnaires (before and after the intervention) and field notes were used to collect data for comparison purposes. Finally, Cycle 3 followed more consistently structured activities and tools. Specifically, the researchers reviewed the intervention procedures to address the challenges faced in the research Cycle 2 and applied the same data collection processes (e.g., self-assessment questionnaires, field notes).

Results

Research cycle 1

During Cycle 1, the authors studied and analyzed the pre-existing course design, the tools, and the internet-based activities to clarify the teaching context of this research. The course followed a linear path, starting with presenting the different types of internet tools. Then, time was devoted to teaching each tool separately. Some tools took up to three meetings (Weblogs), while some others took up to two sessions (Wikis) and the rest of them one meeting. For this purpose, the researchers emphasized Weblogs and Wikis during the lessons, and students' assignments for the course semester involved using both tools. Also, part of the course material was related to the internet skills indicators identified during the preliminary research phase (e.g., references to social networking tools and content creation tools about items from the Social and Creative skills sets). However, there was a lack of content related to Operational, Information-Navigation, and Critical skills.

Findings of research cycle 1

Concerning Operational skills, statistically significant results were reported for the skills item “I know how to complete online forms” (z = − 3.162, p = 0.002). Specifically, students answered confidently after the intervention that they possessed the skill above. Although the instructional material did not refer to web browsers, students were familiar with their use. Completing an online form to access several online tools seemed to be an easy and typical process for them. A few students used bookmarks to save the course's material, while some reported that they had not used them in the past. Additionally, students showed a lack of readiness in using shortcuts for more advanced operations rather than copy-paste (e.g., screen splitting) and limited knowledge in adjusting the privacy settings (e.g., how to manage to browse history data.). As none of the above skills appeared in the instructional material, students did not use them during the lessons. As a result, the analysis of students' responses before and after the intervention did not show statistically significant results for the corresponding items, namely “I know how to use shortcut keys (e.g., CTRL-C for copy, CTRL-S for save)” (z = − 763, p = 0.445), “I know how to bookmark a website" (z = − 1.311, p = 0.190), and "I know how to adjust privacy settings” (z = − 302, p = 0.763). Additional issues were found regarding Internet safety, specifically the creation of passwords. In general, an incomplete assessment of the potential risk for personal data theft was observed, which highlighted the need to raise students' awareness of issues related to securing their online identity and presence. While the course included relevant teaching material on Internet safety, such as references to general risks when browsing the Internet (virus downloads and attacks by hackers), students could not transfer the acquired knowledge to a new context because there were no consolidation activities. To conclude, it was considered essential for the intervention to include a dedicated meeting for the operational skills, presenting the most advanced use of internet browsers, such as adjusting privacy settings and browsing safely, using keyboard shortcut functions, and bookmarking.

Students seemed to be relatively familiar with the web's different interfaces regarding information-navigation skills. However, relevant educational material was not foreseen during the course. There were no statistically significant differences in student self-assessment before and after the intervention for the relevant items: “All the different website layouts make working with the Internet difficult for me.” (z = − 1.706, p = 0.088), “I find the way in which many websites are designed confusing.” (z = − 1.931, p = 0.053), “Sometimes I end up on websites without knowing how I got there.” (z = − 1.734, p = 0.083), and “I find it hard to find a website I visited before.” (z = − 1.633, p = 0.102). Similarly, no statistically significant differences were reported for the online information search processes and specifically for the items: “I find it hard to decide what the best keywords are to use for online searches.” (z = − 1.228, p = 0.219), and “I get tired when looking for information online.” (z = − 333, p = 0.739). Students' work revealed a lack of knowledge in using advanced search strategies, such as keyword search or searching in specific databases to conduct their assignments. Most students used a particular search engine, while very few used the online library catalog or other databases. Some students reported that managing a large amount of information made it difficult to look for quality.

In contrast, others said they did not know where to look for information and were not satisfied with their results. The self-assessment results were statistically significant for the skill item “I should take a course on finding information online.” (z = − 2.028, p = 0.043). Specifically, before taking the course, students were confident that they did not need it. After the intervention, most of them answered neutrally. To sum up, there was a need to enrich the curriculum with content about information search strategies, search engines, meta-search engines, and databases with diverse digital material, such as documents, images, or videos.

Regarding Social skills, students' assignments and, in particular, their blogs showed that they already possessed relevant skills. Most students could integrate and use the "Follow" function, and they had no difficulty deciding which blogs to follow. For this reason, the analysis showed no statistically significant difference for the item: “I feel comfortable deciding whom to follow online (e.g., on services like Twitter or Tumblr)” (z = − 1.134, p = . 257). As students mentioned, they preferred to use the Follow function over the RSS because it did not require registration in another service and corresponded better to how they used other internet services. Therefore, the researchers thought replacing it with the "Follow" function was more appropriate.

Additionally, no statistically significant differences were found for the items that refer to communicating and sharing information: “I know when I should and shouldn't share information online” (z = − 1.342, p = 0.180), “I am careful to make my comments and behaviors appropriate to the situation I find myself in online” (z = − 1.732, p = 0.083), “I am confident about writing a comment on a blog, website or forum” (z = − 1.508, p = 0.132), “I know which information I should and shouldn't share online” (z = -0.586, p = 0.558), and “I would feel confident writing and commenting online” (z = − 0.749, p = 0.454). Such results may be attributed to the lack of instructional content associated with netiquette rules, personal information sharing, and digital footprint. For example, in some cases, the incorrect use of language was observed. Concerning the management of their profiles, statistically significant results were found for the item “I know how to change whom I share content with (e.g., friends, friends of friends or public)” (z = − 2.121, p = 0.034). Specifically, students answered with higher confidence after the intervention that they possessed the above skill, probably because they had to change their blog’s settings from private to public to complete their assignments. However, there was a lack of readiness to change the default settings of the tools used. The above results indicated the need to enhance the instructional materials with netiquette rules, online information-sharing practices, and their implications.

Regarding Creative skills, blogs and wikis enabled students to modify, adapt, and reorganize their content. For example, two students came up with creative suggestions about editing content and were prompted by their fellow students to share their ideas with the whole group. Such experience resulted in positive attitudes toward learning and self-improvement. Students also enjoyed the fact that they could choose their blog's theme, mainly based on their topics of interest. By no surprise, students' self-assessment showed positive, statistically significant results for almost all items: “I know how to create something new from existing online images, music or video” (z = − 3.477, p = 0.001), “I know how to make basic changes to the content that others have produced” (z = − 3.583, p < 0.001), and “I know how to design a website” (z = − 4,177, p < 0.001). Specifically, students answered whether they possessed the above skills after the intervention. Also, the instructional content included information about the Creative Commons license, which students had to integrate into their blogs. As a result, statistically significant results were reported for the item, “I know which different types of licenses apply to online content” (z = − 4.030, p < 0.001). Mainly, students answered confidently after the intervention that they possessed the above skill. One of the factors that made it challenging for students to create more advanced multimedia formats, such as video, was the time frame of the meetings, which was limited for editing content and practicing with the tools. Therefore, there was no statistically significant difference in students' self-assessment before and after the intervention for the item, “I would feel confident putting video content I have created online” (z = − 1,732, p = 0.083).

To conclude, there was a need to emphasize authentic activities related to students' interests and multimedia content creation. Also, the authors thought it necessary to utilize cloud services for students to store and access their work at any time. Although the authors addressed this topic in the middle of the semester, it was evident that it would be helpful for students to utilize these services at the beginning of the course. Another key finding was that students were looking for examples of how they could design and develop their blogs. For essence, when asked to create their welcome message for their blog readers, some students looked for exemplary texts to compose their message.

Similarly, they searched for patterns in the blog's structure and publishing posts. As it turned out, Wikis did not work as effectively as the use of blogs. Some other challenges were the lack of shared interests among the group members and the difficulty of clarifying each student's work and active participation.

Concerning Critical skills, there were misunderstandings regarding copyright and a lack of fundamental understanding of the ethical issues of using online content. Specifically, in some cases, students used images from websites where the sharing of material was prohibited. In these cases, they downloaded images from search engines without using search filters or services that provide free multimedia content. Additionally, there were cases where a partial understanding of the rights to fair use of images had emerged. To conclude, participants found it challenging to evaluate the publicly available online content. All the above, together with the lack of relevant instructional material, resulted in no significant results for all relevant items: “I am confident in selecting search results” (z = − 1.897, p = 0.058), “Sometimes I find it hard to verify information I have retrieved” (z = − 1.069, p = 0.285), and “I carefully consider the information I find online” (z = − 1,734, p = 0.083). However, a positive, statistically significant difference was reported for the item, “I know which apps/software are safe to download” (z = -3.090, p = 0.002). Specifically, students answered with higher confidence after the intervention that they possessed the above skill. Although students could not download software and practice the skill in the University's laboratory due to regulations prohibiting downloading any software from the Internet, discussing the above topic by exchanging personal experiences enhanced their confidence that they possessed the above skill after the course.

In terms of Critical skills, it was necessary to enrich the instructional content with criteria for evaluating online resources and the fair use of multimedia resources available on the Internet. It was also essential to refer to procedures, strategies, and tools to support students in the information retrieval process to obtain quality results from reliable sources. Additionally, the authors discussed issues related to personal responsibility for disseminating information and knowledge.

The data collected from the open-ended questionnaire revealed that students' prior experience with digital technologies, knowledge level, and readiness to meet the learning objectives were heterogeneous. Most of them seemed to use information search engines and social media services quite often. According to some of their answers: “I use the internet at least 12 h a day, so this course is useful to me,” and “From the tools we learned during the course, I have used before about twenty percent of them.” Regarding the instructional strategies that can support learning, students preferred the link of theory to practice and the use of tutorials about the basic functionalities of the tools. Participants reported: “What I liked most was the practical part mainly because we could better understand what we learned in theory;” “The fact that we had to integrate some tools into our blog was an incentive to work from home;” and “It was very easy for me that the tutorials included everything, so I could later find what I forgot.” They also suggested using gamification techniques, such as quizzes. For example, a student said: “The process was very organized and much easier than I thought. It could also include activities such as quiz games.”

Regarding using digital tools that supported instruction, students stated that they preferred the ones that allowed them to create multimedia material for academic and professional purposes. In addition, blogs seemed to be preferable to the use of wikis. One student said: “What I liked the most is that we created our blog for free. Creating multimedia seemed quite valuable, offering an opportunity to create professional presentations. On the other hand, […] did not find wikis exciting and preferred blogs in which [..] posted our work or text and videos about our work.”

Regarding the main factors that enhanced students' motivation to participate in the learning process, their answers revealed the usefulness of the skills to their studies and their subsequent professional career. According to some of their responses: “In many jobs, you use internet tools. If the employer sees that I know and am aware of various tools, it will benefit me.” Furthermore, students found Creative Commons licenses to protect their work particularly interesting. For instance, a student stated: “I think it is necessary to know [the Creative Commons license] as it concerns protecting our rights.” Lastly, students stressed the need to make meaningful connections between different tools. According to their answers: “I find it very useful that we learn about the use of each tool separately, but I think it would have been more useful to learn how all these tools could be connected. This strategy can make the course more meaningful,” and “It would be particularly effective to have constant and frequent interaction with the tools to become more familiar with them.” The above findings informed the design decisions for implementing the next research cycle.

Research cycle 2

The results from the previous Cycle suggested the need to redesign the instructional practice to gradually familiarize students with various internet tools and their capabilities through a series of tasks and interconnected activities that they could implement at their own pace. For this purpose, the authors applied a project-based learning approach to emphasize autonomous learning and participation in authentic learning experiences through progressive skills development. Project-based learning is a common practice in HE for producing artifacts using computer technologies, and its benefits for skills development have been documented in several previous studies (Gülbahar and Tinmaz, 2006; Guo et al. 2020; Lee et al. 2014). To align the tools to the skills indicators of the ISS and their affordances for supporting academic studies based on students' perceptions, the authors applied a technology mapping process (Angeli and Valanides, 2013). Based on the student's preference for the use of blogs, the authors were able to identify several activities and tools that could be linked to their use and support the progressive development of students’ skills. Notably, creating a blog requires developing technical skills, such as using a browser to access the service, filling out an online registration form, or changing the privacy settings (Operational skills). Also, it requires the organization of the blog’s content, which is essential for the reader's successful navigation. In this regard, the blog administrator learns all possible ways to navigate a web page (Navigation skills). Also, blog posts can include all types of information that resulted from searching on various websites (Information skills). It is also an excellent communication tool that allows the administrator to share information, connect to other blogs, and interact with other Internet users (Social skills).

Furthermore, as a personal expression tool, it allows posting multimedia content, such as text, images, and video (Creative skills). Finally, the sharing of information as a means of self-expression and creativity also implies its responsible use by the administrator in terms of accuracy and reliability (Critical skills). Online Appendix 4 presents the course structure designed and implemented during the second research cycle.

Findings of research cycle 2

In general, the development of Operational skills was relatively easy for students. Several references linked concepts and practices to students' experiences during the activities and motivated them to engage in learning actively. For example, managing browsing history and security settings were identified as skills applied to various environments, either personal or academic. During the creation of their blogs, students had to complete several online forms to access tools and services. Additionally, they were prompted to bookmark tools, websites, and services to easily retrieve them afterward and complete their projects. Before using online registration forms, students had to complete a gamified activity about creating secure passwords. The purpose of the activity was to attract students' attention and ensure that they would apply this knowledge during their registration to online services. The results about the development of students' skills showed statistically significant differences for the items: “I know how to bookmark a website” (z = − 2.965, p = 0.003), “I know how to adjust privacy settings” (z = − 2.958, p = 0.003), and “I know how to complete online forms” (z = − 2333, p = 0.020). Specifically, students answered with higher confidence after the intervention that they possessed the above skills, even though no statistically significant difference was found for the item “I know how to use shortcut keys (e.g., CTRL-C for copy, CTRL-S for save)” (z = − 1.730, p = 0.084).

Concerning the Information-Navigation skills, during the development of their blogs, students chose different templates in terms of structure and appearance (navigation, main menu, posts, sidebar). The above fact implied that they were familiar with various interfaces and, therefore, navigation. Furthermore, managing the blogs' structure using widgets (e.g., adding the function of search and recent posts) helped them get an idea of website design and navigation. Consequently, there were statistically significant differences for the relevant items: “All the different website layouts make working with the Internet difficult for me” (z = − 2.121, p = 0.034), “I find how many websites are designed confusing” (z = − 2.070, p = 0.038), “Sometimes I end up on websites without knowing how I got there” (z = − 1.998, p = 0.046), and “I find it hard to find a website I visited before” (z = 2.041, p = 0.041). Specifically, after the intervention, students answered confidently that the above skills statements were not factual, meaning they did not experience navigation difficulties.

Additionally, during this Cycle, students were asked to include informational material in their blogs. Thus, the instructional content was enriched with activities related to using information search strategies (e.g., using filtering options, broadening and narrowing the search results) and the utilization of search and meta-search engines. Such activities allowed students to compare search tools, understand their differences, implement strategic actions to take immediate quality results, and justify using various search tools to determine their added value. For example, students had to complete an online quiz regarding how well they could operate the search engine they were using daily. Such reference to their daily activities enhanced their positive attitudes toward the subject. In general, the use of quizzes enabled immediate feedback and encouraged discussions about the practical implications of this knowledge in both personal and academic settings. Therefore, statistically, significant differences were reported for the items: “I should take a course on finding information online” (z = − 2.280, p = 0.023), “I find it hard to decide what the best keywords are to use for online searches” (z = − 2.989, p = 0.003). Specifically, students answered with higher confidence after the intervention that the skills mentioned above were not actually of them, meaning that they were confident that they developed the relevant information skills. However, the analysis did not show a statistically significant difference for the item “I get tired when looking for information online” (z = − 2.209, p = 0.46). Specifically, the activity used for applying information retrieval strategies included a lot of information and questions for the strategy and tools, requiring much time and mental effort from students to experiment and assimilate the newly acquired knowledge. As a result, many students felt overloaded. Accordingly, there was a need to distinguish general from academic search tools, provide explicit step-by-step instructions for the use of the strategy, and give students the time to explore and become familiar with the functions of each tool.

Concerning Social skills, the creation of students' blogs was based on examples drawn from activities structured around authentic case study scenarios. The researchers prompted students to manage their digital reputation and formulate rules for sharing information in their blogs during the activities. Additionally, students were asked to apply their knowledge by providing feedback to their peers regarding the netiquette rules applied to their blogs. Consequently, statistically, significant differences were found for the relevant items: “I know when I should and should not share information online” (z = − 2.251, p = 0.024), “I know how to change whom I share content with (e.g., friends, friends of friends or public)” (z = − 2.070, p = 0.038), “I am confident about writing a comment on a blog, website or forum” (z = − 2.070, p = 0.038), “I know which information I should and should not share online” (z = − 2.460, p = 0.014), and “I would feel confident writing and commenting online” (z = − 2.121, p = 0.034). Specifically, students answered with higher confidence after the intervention that they possessed the above skills. However, there were no statistically significant differences for the items: “I am careful to make my comments and behaviors appropriate to the situation I find myself in online” (z = − 1.414, p = 0.157), and “I feel comfortable deciding whom to follow online (e.g., on services like Twitter or Tumblr)” (z = − 1.633, p = 0.102). Such findings indicated that the training materials did not cover all the rules of netiquette. Discussions made during the course revealed the need to focus on how to formally communicate through emails (email netiquette), as this tool was one of the main tools used by students during their studies.

Additionally, at a technical level, students were familiar with the “Follow” function; they all followed their fellow students' blogs. However, in some cases, the decisions to follow other blogs were merely made based on the relevant content without considering evaluation criteria (e.g., if the blog's content is reliable). Therefore, it was essential to extend the evaluation process and apply it to blogs to promote a more general perception of evaluation and the choices that media users make when deciding to follow someone online. The good practices identified included the references to students' online social practices (e.g., memes with humorous characters) and how to apply netiquette rules. Such practices enhanced students’ positive attitudes and strengthened their motivation toward the learning subject. An additional theme that emerged during the intervention was the need to include professional networking tools with the possibility of linking the blog to a professional profile.

The development of Creative skills was based on creating multimedia materials used to enrich students' blog content. Students' engagement in content creation, apart from learning about each tool's capabilities for the course purposes, encouraged them to explore new possibilities, such as creatively expressing their views and suggesting alternative uses of the tools themselves. Additionally, prompting students to transfer their skills to other academic activities (e.g., creating multimedia presentations required from other courses) was essential to helping them understand the clear connection of tools to their academic work. Additionally, highlighting the importance of using Creative Commons licenses helped students understand their role and responsibilities in communicating information ethically and critically. The results showed statistically significant differences for all related skills items: “I would feel confident putting video content I have created online” (z = − 1,732, p = 0.043), “I know how to create something new from existing online images, music, or video” (z = − 2.428, p = 0.015), “I know how to make basic changes to the content that others have produced” (z = − 2.701, p = 0.007), “I know how to design a website” (z = − 2.994, p <  = 0.003), and “I know which different types of licenses apply to online content” (z = − 3.115, p = 0.002). Namely, students answered with higher confidence after the intervention that they possessed the above skills. Some good practices identified in this Cycle are presenting examples for each tool and the corresponding product during the content creation process. As part of the teaching process, the presentation of examples was helpful in terms of communicating the expectations regarding the final artifact. Also, evaluation criteria were distributed to students to ensure the quality of the produced projects. For example, students had to incorporate Creative Commons licenses into the presentation and use images free from copyright restrictions. One critical issue which emerged during content creation was the lack of consistent blog posting. Thus, the researchers created a checklist with possible items published constantly on students' blogs to keep them updated.

Regarding the development of Critical skills, students were prompted to post reliable online resources on their blogs. They had to search for relevant material and apply evaluation criteria to assess their results. For this purpose, they were asked to evaluate original online reliable and non-reliable sources, including their blogs and troll news media websites familiar to younger audiences. However, students experienced difficulties evaluating online information included in websites for which it was not apparent from their URL addresses whether they had reliable/non-reliable information. For such cases, the application of evaluation criteria based on the website content was necessary. It is worth noting that students could relate their own experiences to the course content. In particular, they referred to phishing emails circulated to their email accounts and suggested relevant content as part of the course material. Students were also prompted to participate in gamified activities regarding which online applications were safe to download. The activity included reward elements and produced immediate feedback. The results from the activity prompted students to link to their own experiences regarding malware protection, and some of them who were already familiar with such practices advised their peers on how to keep their computers and applications updated safely.

Students positively perceived such practices. For example, the results from students' skills assessment showed statistically significant differences for the following items: “I know which Αpps/software are safe to download” (z = − 3.125, p = 0.002), “I am confident in selecting search results” (z = − 2.511, p = 0.012), and “I carefully consider the information I find online” (z = − 2.850, p = 0.004). Specifically, students answered with higher confidence that they developed the above skills. Additionally, a statistically significant difference was found for the item: “Sometimes I find it hard to verify the information I have retrieved” (z = − 2.491, p = 0.013). Mainly, students answered with higher confidence after the intervention that did not find it hard to verify the information they had retrieved.

To conclude, the results of the second research cycle were encouraging. The presented course material created favorable conditions for preparing students to participate actively throughout the learning experience. The curriculum, organized holistically, supported the systematic transfer of knowledge and the progressive development of students' skills. Additionally, students were prompted to activate their prior knowledge and experiences. The variety of activities and the creative approach of the blog production allowed them to develop their autonomy and strengthen their decision-making skills. The use of case study scenarios emphasized creative thinking and provided stimuli for reflection and action. The variety of tools connected to tasks for the blog creation enhanced students' motivation and engagement in the learning process throughout the course.

Research cycle 3

Based on the research results of Cycle 2, the authors put effort into further developing students' understanding of internet tools' affordances for academic study. Online Appendix 4 highlights the additions made to the course structure used in Cycle 3.

Results of research cycle 3

Regarding Operational skills, the main design addition included a keyboard-shortcut scaffold in the form of a notepad, which was available on every computer to allow immediate access to keyboard functionalities and allow students to retain new knowledge. Such scaffold facilitated the repetition and transfer of new knowledge to use all the tools. Additionally, the affordances of each tool to support learning in academic environments were communicated to students, who were also prompted to identify several potential uses of the tools in their academic lives. Such affordances included (a) managing academic work using bookmarks, (b) minimizing the time it takes to complete an assignment using keyboard shortcuts, (c) using cloud technology services to organize and store assignments and other study-related documents efficiently, and (d) customizing browser settings to adjust to individual practices and protect user accounts. All the above resulted to positive statistically significant differences for all items: “I know how to use shortcut keys (e.g., CTRL-C for copy, CTRL-S for save)” (z = − 3.673, p = 0.001), “I know how to bookmark a website” (z = − 3.516, p = 0.001), “I know how to adjust privacy settings” (z = − 3.689, p = 0.001), and “I know how to complete online forms” (z = − 3.276, p = 0.001). After the intervention, students answered that they possessed the skills above.

Regarding the Information-Navigation skills, the information retrieval activity was redesigned to include fewer questions and a step-by-step guide on implementing each information search strategy. Such practices helped students better discern the different types of information search. The tools' affordances that were identified and communicated as useful for academic work included the following: (a) navigating academic websites and databases using indexes/directories, (b) using search and meta-search engines to search for multimedia, (c) using different search tools that can meet specific information needs when completing an assignment, (d) defining keywords to search for specific content in search engines, and (e) storing and classifying files and documents in folders. Consequently, there were statistically significant differences for all the skills items: “I should take a course on finding information online” (z = − 3.555, p = 0.001), “I find it hard to decide what the best keywords are to use for online searches” (z = − 3.869, p = 0.001), “All the different website layouts make working with the Internet difficult for me” (z = − 2.797, p = 0.005), “I find the way in which many websites are designed confusing” (z = − 3.448, p = 0.001), “Sometimes I end up on websites without knowing how I got there” (z = − 3.008, p = 0.003), “I find it hard to find a website I visited before” (z = 2.687, p = 0.007), and “I get tired when looking for information online” (z = − 2.958, p = 0.003). Specifically, students answered with higher confidence after the intervention that all the above statements were not true, meaning that they were confident that they had developed the relevant information search and navigation skills.

Concerning Social skills, the activities' design and content included applying netiquette rules in the academic environment and using different information-sharing options. The identified tools' affordances for communication purposes were as follows: (a) applying netiquette rules when using the institution's email or forums in a course's Learning Management System, (b) communicating information in academic contexts through different types of media (text, image, video), and (c) using social networking services for professional purposes, such as showcasing their online CVs. The results showed statistically significant differences for all the skills: “I know when I should and shouldn't share information online” (z = − 3.448, p = 0.001), “I am careful to make my comments and behaviours appropriate to the situation I find myself in online” (z = − 2.952, p = 0.003), “I know how to change who I share content with (e.g., friends, friends of friends or public)” (z = − 2.179, p = 0.029), “I am confident about writing a comment on a blog, website or forum” (z = − 3.339, p = 0.001), “I feel comfortable deciding who to follow online (e.g., on services like Twitter or Tumblr)” (z = − 2.667, p = 0.008), “I know which information I should and shouldn't share online” (z = − 3.626, p = 0.001), and “I would feel confident writing and commenting online” (z = − 2.972, p = 0.003). Specifically, students answered with higher confidence after the intervention that they possessed the above skills.

Concerning the Creative skills, the design included a checklist with different types of posts that students needed to create and follow throughout their project's development. This practice reinforced students' motivation to work with the tools learned more frequently. In addition, content creation activities strengthened students' motivation and interest in creating their final project. In cases where students were more experienced using similar tools, they were prompted to activate their prior knowledge (e.g., students from the IT department used more sophisticated functions of the tool used to customize their blogs). The tools' affordances that were identified to support learning in academic environments included (a) creating and publishing products with the use of multimedia (e.g., presentations, videos), (b) protecting original work from copyright, and (c) conducting research with the use of online software for developing questionnaires and analyzing data, (d) producing artifacts/publications and communicating the results, and (e) allowing students to create and showcase a professional work-related portfolio. The findings showed statistically significant differences for all skills items: “I would feel confident putting video content I have created online” (z = − 3.654, p < 0.001), “I know how to create something new from existing online images, music, or video” (z = − 3.542, p < 0.001), “I know how to make basic changes to the content that others have produced” (z = − 3.548, p < 0.001), “I know how to design a website” (z = − 3.805, p < 0.001), and “I know which different types of licenses apply to online content” (z = − 4.274, p < 0.001). Specifically, students answered with higher confidence after the intervention that they developed these skills.

Concerning Critical skills, the content was enriched with step-by-step explanations for evaluating web-based information, including Scams, Hoaxes, and Fake News, which students suggested during Cycle 2. The material used during the activities was up to date and directly related to students' experiences (e.g., a real incident of attempted email fraud, real blog owners spreading fake news), including original videos popular with younger people. During the activities, students linked such practices to personal experiences derived from fake news or scams circulated on social media, such as “like” farming, and questioned the reliability of online content. The practices identified as supportive to academic studies were (a) assessing the suitability of online resources and multimedia content for academic purposes and (b) protecting academic identity from online fraud. All the above resulted in statistically significant differences for all items: “I know which apps/software is safe to download” (z = − 3.292, p = 0.001), “I am confident in selecting search results” (z = − 3.808, p < 0.001), and “I carefully consider the information I find online.” (z = − 3.567, p < 0.001). Particularly, students answered with higher confidence after the intervention that they possessed the above skills. Additionally, a statistically significant difference was found for the item “Sometimes I find it hard to verify the information I have retrieved” (z = − 3.999, p < 0.001). Specifically, students answered with higher confidence after the intervention that they did not find it hard to verify the information they had retrieved.

To conclude, Cycle 3 was positive regarding the quality of the learning experience. As it turned out, communicating the affordances of tools to support future academic/professional needs enhanced students' interest and motivation, resulting in positive outcomes regarding developing their skills.

Discussion

Our research results confirmed the literature findings that digital skills are essential for university studies (Brooks 2016; Dahlstrom and Bichsel 2014). Specifically, during the research, students perceived positively the communication of the affordances of the digital tools to support academic work. In some cases, they also highlighted several potential uses of the tools, despite their different academic profiles. This finding was in line with previous research on the affordances of tools to support learning in HE (Margaryan et al. 2011; Ng, 2012). In addition, an exciting result that emerged during the study was that students found value in developing digital skills for their future professional careers.

Regarding their prior knowledge and skills, students were very familiar with the use of digital technologies, and they could very efficiently operate and adjust the functionalities of the tools. We could see evidence of their digital world immersion (Seemiller and Grace 2017). Before the interventions, the most competent use of digital technologies was reported for the Operational and Social skills sets. This finding agrees with the previous research, which emphasized students’ familiarity with Learning Management Systems, search engines, and online communication tools (Bond et al. 2018; Bullen et al. 2011; Kennedy and Fox, 2013; List et al. 2016; Ng 2012). Additionally, although students did not report competent use of digital tools for content creation processes and digital content licensing, a finding reported in previous studies (Kennedy and Fox, 2013; López-Meneses et al. 2020), they adapted very easy to the content creation process. Several challenges emerged in developing Information-Navigation and Critical skills sets, especially regarding searching and selecting credible information sources and considering copyright restrictions. Previous studies also reported similar results (Gudmundsdottir et al. 2020; Hargittai et al. 2010; Head, 2013; Lee et al. 2012; List et al. 2016). It is worth noting that although a pattern can be drawn from the above findings, the results from each Cycle regarding students’ skills levels were distinctive, supporting previous studies that argued that young learners do not constitute a homogeneous generation in terms of digital skills (Corrin et al. 2010; Helsper and Eynon 2010; Ng 2012). The exploration of both students' digital skills levels and their learning preferences before the intervention allowed us to identify potential needs and gaps and adjust our designs accordingly; such an adjustment was critical for the successful implementation of the project-based learning approach.

Regarding the assessment of digital skills in the context of HE, the present study demonstrated the need to include additional skills items to the Internet Skills Scale tool related to the academic environment. Specifically, regarding Operational Skills, future research could consist of statements that will refer to cloud technologies for organizing and storing information. Also, it is suggested to include items related to the safe use of browsers to protect security and privacy in the digital environment. Such statements could refer, for example, to creating secure passwords, managing browser history, and understanding how cookies work. Concerning Information-Navigation skills, it is essential to enrich the Scale with items related to strategic information retrieval, such as finding multimedia content. Regarding Social Skills, it is advised to include items related to netiquette with specific references to frequent communication activities in academic settings, such as the use of email or the use of a discussion forum. Lastly, regarding Critical skills, it is suggested to introduce items regarding the fair use of online content.

Additionally, the triangulation of data from all research cycles made it possible to conclude the content and the design of the learning experience. Specifically, based on the study's findings, we can conclude that the project-based learning intervention and the underpinning learning design contributed to the development of students’ digital skills. Such results align with previous research, which suggested that constructivist learning approaches and project-based learning are beneficial for HE students (Guo et al. 2020; Kivunja 2014; Ng 2015; Wekerle et al. 2022).

Based on our study results, general programs for digital skills education in the context of HE should include

• Advanced use of web browsers (adjusting privacy settings, safe browsing);

• Fundamental understanding of copyright and ethical issues about the use of online content (e.g., use of text, images, videos);

• Cloud services (e.g., organizing and classifying folders and files);

• Information search strategies (search and meta-search engines, academic search, multimedia search);

• Digital footprint (privacy settings, personal and professional identity);

• Netiquette rules (email, forum);

• Content evaluation (Scam, Hoaxes, Fake News); and

• Online fraud (Phishing).

Secondly, the design principles that can support the development of digital skills education programs are summarized as follows.

Digital technologies need to be selected based on their affordances to support learning

The selection of tools should be based on the possibilities they offer to accomplish the learning objectives and should facilitate the transfer and the practical use of knowledge. Before implementing a program, mapping tools' capabilities can support their regular review to ensure they are up to date. In addition, the tool's affordances should be communicated to students.

Content should focus on the development of a range of skills that are interconnected and linked to students' academic and personal environment and prior knowledge

Meaningful connections can be achieved by developing small units that progressively address all skills in different areas. Emphasizing declarative and procedural knowledge in each unit can contribute to skills development within realistic timelines. When necessary, the processes of using digital technologies can be repeated, and their application to different subject areas can be highlighted. Due to students' familiarization with a range of digital technologies, the use of advanced organizers that link instructional content to their prior knowledge and experience can attract their attention and better prepare them for the subject matter. Advance organizers may include information on course organization, examples of good practice, original texts from the Internet, and multimedia material related to students' daily life, such as references to important events, news, and announcements related to the university community. The content should be up to date, and instructors should point out its future usefulness.

Content should stimulate interest in the program/course

The stimulation of students' interest in the program/course can use various tools and activities, which stimulate curiosity, include surprising content, are fun (gamified), are contextual, contradict what students already know, create opportunities for reflection, and bring challenges. Also, it is essential that the content is relevant to students' daily life and experiences and is presented in various forms, especially multimedia.

Active participation should be promoted throughout the learning experience

Active participation can be promoted through a series of activities that strengthen digital skills in the long run through an explicit schedule for implementing activities with opportunities for immediate feedback at regular intervals. Also, active participation can be enhanced by personalizing the learning experience based on students' individual needs and preferences.

The provision of timely support is essential

The progressive skills development in the use of digital tools requires timely support for acquiring new knowledge due to skills' interdependence. Such support can occur by sharing tutorials, such as how to subscribe to various services and use the essential functions of a tool.

Opportunities for personalizing the learning experience should be provided

Students should be enabled to explore and choose how to use digital technologies based on their motivations and expectations. Personalization can also occur by communicating learning opportunities that digital tools offer in the academic environment and by providing opportunities for autonomous learning through clear guidelines and schedules.

Collaboration through the exchange of good practices should be promoted

Providing collaboration opportunities among students can help create a productive atmosphere by sharing good practices. More specifically, the interaction between students can improve the quality of their produced works, as it provides stimuli for further exploration of the learning material. The above practice can work effectively in cases where more advanced students can guide their less advanced fellow students in using digital technologies.

The provision of guidance and examples should be used to support task completion

Guidance can occur through detailed tutorials with visual elements, which provide information on the digital tools' functionality (e.g., subscription, features). Also, activity checklists with completion criteria or “how-to” instructions that are particularly popular for the younger generations can help students keep track of and accomplish their tasks. Presenting examples, such as well-designed websites, exciting and engaging multimedia, and multimodal texts can provide tangible evidence that helps students recognize the value of the acquired skills. Examples can also stimulate students' interest in the program/course and serve as a reference point for their project.

Assessment criteria should be communicated

Assessment processes and standards must be communicated to students in advance to help them understand the expected outcomes and to realize possible improvements. In this regard, the activities should provide evidence of content acquisition.

Implications for practice

The study findings showed that today's students are not fully prepared to embrace technologies in their academic pathways. They need training and support to develop their digital skills in the context of HE. Future research could benefit from the empirical results of our study and develop updated frameworks and instruments to address the need mentioned above, including aspects that are most relevant to HE. Assessment instruments could be used in HE programs to support early students' preparation for using digital technologies during their studies and prevent potential skills gaps. Additionally, the proposed project-based learning approach and the accompanying design principles that emerged during the design process could positively impact digital skills development and support learning in academic settings. Specifically, the proposed method could inform the development of interdisciplinary programs, e.g., in academic libraries, emphasizing the implementation of research work. The design principles presented herein can be used to create or review programs in formal and informal learning environments for the younger generations. Based on our findings, we suggest that HEIs need to reconsider the integration of digital technologies in their programs and courses in terms of a) the affordances of technology to support learning, b) the adoption of instructional practices that favor the development of digital skills, and c) the professional development of teaching staff.

Limitations of the study and future directions

From a research perspective, several limitations need to be addressed by future researchers to draw more concrete conclusions. Firstly, the research participants were students who chose to attend the course. Thus, they were probably more interested in participating and enhancing their skills. Future research could occur in mandatory programs exploring the effectiveness of transferring digital skills from general programs to other courses in the academic environment. Second, the assessment of internet skills took place by students, and as a result, there may be an element of subjectivity in their answers. Although the evaluation of students' final blogs could have indicated skills development, it could not support evaluating all the skills mentioned in the ISS (e.g., I get tired when looking for information, I would feel confident writing and commenting online). Furthermore, the self-assessment was considered more appropriate for the present research regarding time and resources. Finally, it is worth noting that the nature of DBR presupposes the study of a natural environment, which is subjected to research limitations. For example, the University's laboratory settings posed restrictions on online material (e.g., download options). Future research could adopt alternative teaching approaches, such as the BYOD approach or the flipped classroom approach, supporting more flexible research designs.

Conclusion

The present study highlighted the importance of designing and developing digital skills programs to address undergraduate students' needs in HE. This paper makes the case that such programs should be based on empirical investigations to inform relevant policies and practices. Specifically, our findings indicate that the design and development of digital skills programs should be based on relevant skills frameworks, assessment instruments, instructional approaches, and content. Lastly, we hope that the project-based learning approach and the design principles we propose could inform theoretically and methodologically the research community about the design and development of digital skills programs in the context of HE.

Supplementary Information

Below is the link to the electronic supplementary material.

Data availability

The first author is in the process of publishing more papers from this data set, so at this time, it cannot be made available.

Declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.