Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: A literature review

Abstract

Digital technologies have brought changes to the nature and scope of education and led education systems worldwide to adopt strategies and policies for ICT integration. The latter brought about issues regarding the quality of teaching and learning with ICTs, especially concerning the understanding, adaptation, and design of the education systems in accordance with current technological trends. These issues were emphasized during the recent COVID-19 pandemic that accelerated the use of digital technologies in education, generating questions regarding digitalization in schools. Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses. Such results have engendered the need for schools to learn and build upon the experience to enhance their digital capacity and preparedness, increase their digitalization levels, and achieve a successful digital transformation. Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem, there is a need to show how these impacts are interconnected and identify the factors that can encourage an effective and efficient change in the school environments. For this purpose, we conducted a non-systematic literature review. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors that affect the schools’ digital capacity and digital transformation. The findings suggest that ICT integration in schools impacts more than just students’ performance; it affects several other school-related aspects and stakeholders, too. Furthermore, various factors affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the digital transformation process. The study results shed light on how ICTs can positively contribute to the digital transformation of schools and which factors should be considered for schools to achieve effective and efficient change.

Introduction

Digital technologies have brought changes to the nature and scope of education. Versatile and disruptive technological innovations, such as smart devices, the Internet of Things (IoT), artificial intelligence (AI), augmented reality (AR) and virtual reality (VR), blockchain, and software applications have opened up new opportunities for advancing teaching and learning (Gaol & Prasolova-Førland, 2021; OECD, 2021). Hence, in recent years, education systems worldwide have increased their investment in the integration of information and communication technology (ICT) (Fernández-Gutiérrez et al., 2020; Lawrence & Tar, 2018) and prioritized their educational agendas to adapt strategies or policies around ICT integration (European Commission, 2019). The latter brought about issues regarding the quality of teaching and learning with ICTs (Bates, 2015), especially concerning the understanding, adaptation, and design of education systems in accordance with current technological trends (Balyer & Öz, 2018). Studies have shown that despite the investment made in the integration of technology in schools, the results have not been promising, and the intended outcomes have not yet been achieved (Delgado et al., 2015; Lawrence & Tar, 2018). These issues were exacerbated during the COVID-19 pandemic, which forced teaching across education levels to move online (Daniel, 2020). Online teaching accelerated the use of digital technologies generating questions regarding the process, the nature, the extent, and the effectiveness of digitalization in schools (Cachia et al., 2021; König et al., 2020). Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses (Blaskó et al., 2021; Di Pietro et al, 2020). Such results have engendered the need for schools to learn and build upon the experience in order to enhance their digital capacity (European Commission,2020) and increase their digitalization levels (Costa et al., 2021). Digitalization offers possibilities for fundamental improvement in schools (OECD, 2021; Rott & Marouane, 2018) and touches many aspects of a school’s development (Delcker & Ifenthaler, 2021) . However, it is a complex process that requires large-scale transformative changes beyond the technical aspects of technology and infrastructure (Pettersson, 2021). Namely, digitalization refers to “a series of deep and coordinated culture, workforce, and technology shifts and operating models” (Brooks & McCormack, 2020, p. 3) that brings cultural, organizational, and operational change through the integration of digital technologies (JISC, 2020). A successful digital transformation requires that schools increase their digital capacity levels, establishing the necessary “culture, policies, infrastructure as well as digital competence of students and staff to support the effective integration of technology in teaching and learning practices” (Costa et al, 2021, p.163).

Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem (Eng, 2005), there is a need to show how the different elements of the impact are interconnected and to identify the factors that can encourage an effective and efficient change in the school environment. To address the issues outlined above, we formulated the following research questions:

a) What is the impact of digital technologies on education?

b) Which factors might affect a school’s digital capacity and transformation?

In the present investigation, we conducted a non-systematic literature review of publications pertaining to the impact of digital technologies on education and the factors that affect a school’s digital capacity and transformation. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors which affect the schools’ digital capacity and digital transformation.

Methodology

The non-systematic literature review presented herein covers the main theories and research published over the past 17 years on the topic. It is based on meta-analyses and review papers found in scholarly, peer-reviewed content databases and other key studies and reports related to the concepts studied (e.g., digitalization, digital capacity) from professional and international bodies (e.g., the OECD). We searched the Scopus database, which indexes various online journals in the education sector with an international scope, to collect peer-reviewed academic papers. Furthermore, we used an all-inclusive Google Scholar search to include relevant key terms or to include studies found in the reference list of the peer-reviewed papers, and other key studies and reports related to the concepts studied by professional and international bodies. Lastly, we gathered sources from the Publications Office of the European Union (https://op.europa.eu/en/home); namely, documents that refer to policies related to digital transformation in education.

Regarding search terms, we first searched resources on the impact of digital technologies on education by performing the following search queries: “impact” OR “effects” AND “digital technologies” AND “education”, “impact” OR “effects” AND “ICT” AND “education”. We further refined our results by adding the terms “meta-analysis” and “review” or by adjusting the search options based on the features of each database to avoid collecting individual studies that would provide limited contributions to a particular domain. We relied on meta-analyses and review studies as these consider the findings of multiple studies to offer a more comprehensive view of the research in a given area (Schuele & Justice, 2006). Specifically, meta-analysis studies provided quantitative evidence based on statistically verifiable results regarding the impact of educational interventions that integrate digital technologies in school classrooms (Higgins et al., 2012; Tolani-Brown et al., 2011).

However, quantitative data does not offer explanations for the challenges or difficulties experienced during ICT integration in learning and teaching (Tolani-Brown et al., 2011). To fill this gap, we analyzed literature reviews and gathered in-depth qualitative evidence of the benefits and implications of technology integration in schools. In the analysis presented herein, we also included policy documents and reports from professional and international bodies and governmental reports, which offered useful explanations of the key concepts of this study and provided recent evidence on digital capacity and transformation in education along with policy recommendations. The inclusion and exclusion criteria that were considered in this study are presented in Table 1.

Table 1.

Inclusion and exclusion criteria for the selection of resources on the impact of digital technologies on education

Inclusion criteria	Exclusion criteria
• Published in 2005 or later • Review and meta-analysis studies • Formal education K-12 • Peer-reviewed articles • Articles in English • Reports from professional/international bodies • Governmental reports • Book chapters	• Ph.D. dissertations and theses • Conference poster papers • Conference papers without proceedings • Resources on higher education • Resources on pre-school education • Individual studies

To ensure a reliable extraction of information from each study and assist the research synthesis we selected the study characteristics of interest (impact) and constructed coding forms. First, an overview of the synthesis was provided by the principal investigator who described the processes of coding, data entry, and data management. The coders followed the same set of instructions but worked independently. To ensure a common understanding of the process between coders, a sample of ten studies was tested. The results were compared, and the discrepancies were identified and resolved. Additionally, to ensure an efficient coding process, all coders participated in group meetings to discuss additions, deletions, and modifications (Stock, 1994). Due to the methodological diversity of the studied documents we began to synthesize the literature review findings based on similar study designs. Specifically, most of the meta-analysis studies were grouped in one category due to the quantitative nature of the measured impact. These studies tended to refer to student achievement (Hattie et al., 2014). Then, we organized the themes of the qualitative studies in several impact categories. Lastly, we synthesized both review and meta-analysis data across the categories. In order to establish a collective understanding of the concept of impact, we referred to a previous impact study by Balanskat (2009) which investigated the impact of technology in primary schools. In this context, the impact had a more specific ICT-related meaning and was described as “a significant influence or effect of ICT on the measured or perceived quality of (parts of) education” (Balanskat, 2009, p. 9). In the study presented herein, the main impacts are in relation to learning and learners, teaching, and teachers, as well as other key stakeholders who are directly or indirectly connected to the school unit.

Findings

The study’s results identified multiple dimensions of the impact of digital technologies on students’ knowledge, skills, and attitudes; on equality, inclusion, and social integration; on teachers’ professional and teaching practices; and on other school-related aspects and stakeholders. The data analysis indicated various factors that might affect the schools’ digital capacity and transformation, such as digital competencies, the teachers’ personal characteristics and professional development, as well as the school’s leadership and management, administration, infrastructure, etc. The impacts and factors found in the literature review are presented below.

Impacts of digital technologies on students’ knowledge, skills, attitudes, and emotions

The impact of ICT use on students’ knowledge, skills, and attitudes has been investigated early in the literature. Eng (2005) found a small positive effect between ICT use and students' learning. Specifically, the author reported that access to computer-assisted instruction (CAI) programs in simulation or tutorial modes—used to supplement rather than substitute instruction – could enhance student learning. The author reported studies showing that teachers acknowledged the benefits of ICT on pupils with special educational needs; however, the impact of ICT on students' attainment was unclear. Balanskat et al. (2006) found a statistically significant positive association between ICT use and higher student achievement in primary and secondary education. The authors also reported improvements in the performance of low-achieving pupils. The use of ICT resulted in further positive gains for students, namely increased attention, engagement, motivation, communication and process skills, teamwork, and gains related to their behaviour towards learning. Evidence from qualitative studies showed that teachers, students, and parents recognized the positive impact of ICT on students' learning regardless of their competence level (strong/weak students). Punie et al. (2006) documented studies that showed positive results of ICT-based learning for supporting low-achieving pupils and young people with complex lives outside the education system. Liao et al. (2007) reported moderate positive effects of computer application instruction (CAI, computer simulations, and web-based learning) over traditional instruction on primary school student's achievement. Similarly, Tamim et al. (2011) reported small to moderate positive effects between the use of computer technology (CAI, ICT, simulations, computer-based instruction, digital and hypermedia) and student achievement in formal face-to-face classrooms compared to classrooms that did not use technology. Jewitt et al., (2011) found that the use of learning platforms (LPs) (virtual learning environments, management information systems, communication technologies, and information- and resource-sharing technologies) in schools allowed primary and secondary students to access a wider variety of quality learning resources, engage in independent and personalized learning, and conduct self- and peer-review; LPs also provide opportunities for teacher assessment and feedback. Similar findings were reported by Fu (2013), who documented a list of benefits and opportunities of ICT use. According to the author, the use of ICTs helps students access digital information and course content effectively and efficiently, supports student-centered and self-directed learning, as well as the development of a creative learning environment where more opportunities for critical thinking skills are offered, and promotes collaborative learning in a distance-learning environment. Higgins et al. (2012) found consistent but small positive associations between the use of technology and learning outcomes of school-age learners (5–18-year-olds) in studies linking the provision and use of technology with attainment. Additionally, Chauhan (2017) reported a medium positive effect of technology on the learning effectiveness of primary school students compared to students who followed traditional learning instruction.

The rise of mobile technologies and hardware devices instigated investigations into their impact on teaching and learning. Sung et al. (2016) reported a moderate effect on students' performance from the use of mobile devices in the classroom compared to the use of desktop computers or the non-use of mobile devices. Schmid et al. (2014) reported medium–low to low positive effects of technology integration (e.g., CAI, ICTs) in the classroom on students' achievement and attitude compared to not using technology or using technology to varying degrees. Tamim et al. (2015) found a low statistically significant effect of the use of tablets and other smart devices in educational contexts on students' achievement outcomes. The authors suggested that tablets offered additional advantages to students; namely, they reported improvements in students’ notetaking, organizational and communication skills, and creativity. Zheng et al. (2016) reported a small positive effect of one-to-one laptop programs on students’ academic achievement across subject areas. Additional reported benefits included student-centered, individualized, and project-based learning enhanced learner engagement and enthusiasm. Additionally, the authors found that students using one-to-one laptop programs tended to use technology more frequently than in non-laptop classrooms, and as a result, they developed a range of skills (e.g., information skills, media skills, technology skills, organizational skills). Haßler et al. (2016) found that most interventions that included the use of tablets across the curriculum reported positive learning outcomes. However, from 23 studies, five reported no differences, and two reported a negative effect on students' learning outcomes. Similar results were indicated by Kalati and Kim (2022) who investigated the effect of touchscreen technologies on young students’ learning. Specifically, from 53 studies, 34 advocated positive effects of touchscreen devices on children’s learning, 17 obtained mixed findings and two studies reported negative effects.

More recently, approaches that refer to the impact of gamification with the use of digital technologies on teaching and learning were also explored. A review by Pan et al. (2022) that examined the role of learning games in fostering mathematics education in K-12 settings, reported that gameplay improved students’ performance. Integration of digital games in teaching was also found as a promising pedagogical practice in STEM education that could lead to increased learning gains (Martinez et al., 2022; Wang et al., 2022). However, although Talan et al. (2020) reported a medium effect of the use of educational games (both digital and non-digital) on academic achievement, the effect of non-digital games was higher.

Over the last two years, the effects of more advanced technologies on teaching and learning were also investigated. Garzón and Acevedo (2019) found that AR applications had a medium effect on students' learning outcomes compared to traditional lectures. Similarly, Garzón et al. (2020) showed that AR had a medium impact on students' learning gains. VR applications integrated into various subjects were also found to have a moderate effect on students’ learning compared to control conditions (traditional classes, e.g., lectures, textbooks, and multimedia use, e.g., images, videos, animation, CAI) (Chen et al., 2022b). Villena-Taranilla et al. (2022) noted the moderate effect of VR technologies on students’ learning when these were applied in STEM disciplines. In the same meta-analysis, Villena-Taranilla et al. (2022) highlighted the role of immersive VR, since its effect on students’ learning was greater (at a high level) across educational levels (K-6) compared to semi-immersive and non-immersive integrations. In another meta-analysis study, the effect size of the immersive VR was small and significantly differentiated across educational levels (Coban et al., 2022). The impact of AI on education was investigated by Su and Yang (2022) and Su et al. (2022), who showed that this technology significantly improved students’ understanding of AI computer science and machine learning concepts.

It is worth noting that the vast majority of studies referred to learning gains in specific subjects. Specifically, several studies examined the impact of digital technologies on students’ literacy skills and reported positive effects on language learning (Balanskat et al., 2006; Grgurović et al., 2013; Friedel et al., 2013; Zheng et al., 2016; Chen et al., 2022b; Savva et al., 2022). Also, several studies documented positive effects on specific language learning areas, namely foreign language learning (Kao, 2014), writing (Higgins et al., 2012; Wen & Walters, 2022; Zheng et al., 2016), as well as reading and comprehension (Cheung & Slavin, 2011; Liao et al., 2007; Schwabe et al., 2022). ICTs were also found to have a positive impact on students' performance in STEM (science, technology, engineering, and mathematics) disciplines (Arztmann et al., 2022; Bado, 2022; Villena-Taranilla et al., 2022; Wang et al., 2022). Specifically, a number of studies reported positive impacts on students’ achievement in mathematics (Balanskat et al., 2006; Hillmayr et al., 2020; Li & Ma, 2010; Pan et al., 2022; Ran et al., 2022; Verschaffel et al., 2019; Zheng et al., 2016). Furthermore, studies documented positive effects of ICTs on science learning (Balanskat et al., 2006; Liao et al., 2007; Zheng et al., 2016; Hillmayr et al., 2020; Kalemkuş & Kalemkuş, 2022; Lei et al., 2022a). Çelik (2022) also noted that computer simulations can help students understand learning concepts related to science. Furthermore, some studies documented that the use of ICTs had a positive impact on students’ achievement in other subjects, such as geography, history, music, and arts (Chauhan, 2017; Condie & Munro, 2007), and design and technology (Balanskat et al., 2006).

More specific positive learning gains were reported in a number of skills, e.g., problem-solving skills and pattern exploration skills (Higgins et al., 2012), metacognitive learning outcomes (Verschaffel et al., 2019), literacy skills, computational thinking skills, emotion control skills, and collaborative inquiry skills (Lu et al., 2022; Su & Yang, 2022; Su et al., 2022). Additionally, several investigations have reported benefits from the use of ICT on students’ creativity (Fielding & Murcia, 2022; Liu et al., 2022; Quah & Ng, 2022). Lastly, digital technologies were also found to be beneficial for enhancing students’ lifelong learning skills (Haleem et al., 2022).

Apart from gaining knowledge and skills, studies also reported improvement in motivation and interest in mathematics (Higgins et. al., 2019; Fadda et al., 2022) and increased positive achievement emotions towards several subjects during interventions using educational games (Lei et al., 2022a). Chen et al. (2022a) also reported a small but positive effect of digital health approaches in bullying and cyberbullying interventions with K-12 students, demonstrating that technology-based approaches can help reduce bullying and related consequences by providing emotional support, empowerment, and change of attitude. In their meta-review study, Su et al. (2022) also documented that AI technologies effectively strengthened students’ attitudes towards learning. In another meta-analysis, Arztmann et al. (2022) reported positive effects of digital games on motivation and behaviour towards STEM subjects.

Impacts of digital technologies on equality, inclusion and social integration

Although most of the reviewed studies focused on the impact of ICTs on students’ knowledge, skills, and attitudes, reports were also made on other aspects in the school context, such as equality, inclusion, and social integration. Condie and Munro (2007) documented research interventions investigating how ICT can support pupils with additional or special educational needs. While those interventions were relatively small scale and mostly based on qualitative data, their findings indicated that the use of ICTs enabled the development of communication, participation, and self-esteem. A recent meta-analysis (Baragash et al., 2022) with 119 participants with different disabilities, reported a significant overall effect size of AR on their functional skills acquisition. Koh’s meta-analysis (2022) also revealed that students with intellectual and developmental disabilities improved their competence and performance when they used digital games in the lessons.

Istenic Starcic and Bagon (2014) found that the role of ICT in inclusion and the design of pedagogical and technological interventions was not sufficiently explored in educational interventions with people with special needs; however, some benefits of ICT use were found in students’ social integration. The issue of gender and technology use was mentioned in a small number of studies. Zheng et al. (2016) reported a statistically significant positive interaction between one-to-one laptop programs and gender. Specifically, the results showed that girls and boys alike benefitted from the laptop program, but the effect on girls’ achievement was smaller than that on boys’. Along the same lines, Arztmann et al. (2022) reported no difference in the impact of game-based learning between boys and girls, arguing that boys and girls equally benefited from game-based interventions in STEM domains. However, results from a systematic review by Cussó-Calabuig et al. (2018) found limited and low-quality evidence on the effects of intensive use of computers on gender differences in computer anxiety, self-efficacy, and self-confidence. Based on their view, intensive use of computers can reduce gender differences in some areas and not in others, depending on contextual and implementation factors.

Impacts of digital technologies on teachers’ professional and teaching practices

Various research studies have explored the impact of ICT on teachers’ instructional practices and student assessment. Friedel et al. (2013) found that the use of mobile devices by students enabled teachers to successfully deliver content (e.g., mobile serious games), provide scaffolding, and facilitate synchronous collaborative learning. The integration of digital games in teaching and learning activities also gave teachers the opportunity to study and apply various pedagogical practices (Bado, 2022). Specifically, Bado (2022) found that teachers who implemented instructional activities in three stages (pre-game, game, and post-game) maximized students’ learning outcomes and engagement. For instance, during the pre-game stage, teachers focused on lectures and gameplay training, at the game stage teachers provided scaffolding on content, addressed technical issues, and managed the classroom activities. During the post-game stage, teachers organized activities for debriefing to ensure that the gameplay had indeed enhanced students’ learning outcomes.

Furthermore, ICT can increase efficiency in lesson planning and preparation by offering possibilities for a more collaborative approach among teachers. The sharing of curriculum plans and the analysis of students’ data led to clearer target settings and improvements in reporting to parents (Balanskat et al., 2006).

Additionally, the use and application of digital technologies in teaching and learning were found to enhance teachers’ digital competence. Balanskat et al. (2006) documented studies that revealed that the use of digital technologies in education had a positive effect on teachers’ basic ICT skills. The greatest impact was found on teachers with enough experience in integrating ICTs in their teaching and/or who had recently participated in development courses for the pedagogical use of technologies in teaching. Punie et al. (2006) reported that the provision of fully equipped multimedia portable computers and the development of online teacher communities had positive impacts on teachers’ confidence and competence in the use of ICTs.

Moreover, online assessment via ICTs benefits instruction. In particular, online assessments support the digitalization of students’ work and related logistics, allow teachers to gather immediate feedback and readjust to new objectives, and support the improvement of the technical quality of tests by providing more accurate results. Additionally, the capabilities of ICTs (e.g., interactive media, simulations) create new potential methods of testing specific skills, such as problem-solving and problem-processing skills, meta-cognitive skills, creativity and communication skills, and the ability to work productively in groups (Punie et al., 2006).

Impacts of digital technologies on other school-related aspects and stakeholders

There is evidence that the effective use of ICTs and the data transmission offered by broadband connections help improve administration (Balanskat et al., 2006). Specifically, ICTs have been found to provide better management systems to schools that have data gathering procedures in place. Condie and Munro (2007) reported impacts from the use of ICTs in schools in the following areas: attendance monitoring, assessment records, reporting to parents, financial management, creation of repositories for learning resources, and sharing of information amongst staff. Such data can be used strategically for self-evaluation and monitoring purposes which in turn can result in school improvements. Additionally, they reported that online access to other people with similar roles helped to reduce headteachers’ isolation by offering them opportunities to share insights into the use of ICT in learning and teaching and how it could be used to support school improvement. Furthermore, ICTs provided more efficient and successful examination management procedures, namely less time-consuming reporting processes compared to paper-based examinations and smooth communications between schools and examination authorities through electronic data exchange (Punie et al., 2006).

Zheng et al. (2016) reported that the use of ICTs improved home-school relationships. Additionally, Escueta et al. (2017) reported several ICT programs that had improved the flow of information from the school to parents. Particularly, they documented that the use of ICTs (learning management systems, emails, dedicated websites, mobile phones) allowed for personalized and customized information exchange between schools and parents, such as attendance records, upcoming class assignments, school events, and students’ grades, which generated positive results on students’ learning outcomes and attainment. Such information exchange between schools and families prompted parents to encourage their children to put more effort into their schoolwork.

The above findings suggest that the impact of ICT integration in schools goes beyond students’ performance in school subjects. Specifically, it affects a number of school-related aspects, such as equality and social integration, professional and teaching practices, and diverse stakeholders. In Table 2, we summarize the different impacts of digital technologies on school stakeholders based on the literature review, while in Table 3 we organized the tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript.

Table 2.

The impact of digital technologies on schools’ stakeholders based on the literature review

Impacts	References
Students
Knowledge, skills, attitudes, and emotions
• Learning gains from the use of ICTs across the curriculum	Eng, 2005; Balanskat et al., 2006; Liao et al., 2007; Tamim et al., 2011; Higgins et al., 2012; Chauhan, 2017; Sung et al., 2016; Schmid et al., 2014; Tamim et al., 2015; Zheng et al., 2016; Haßler et al., 2016; Kalati & Kim, 2022; Martinez et al., 2022; Talan et al., 2020; Panet al., 2022; Garzón & Acevedo, 2019; Garzón et al., 2020; Villena-Taranilla, et al., 2022; Coban et al., 2022
• Positive learning gains from the use of ICTs in specific school subjects (e.g., mathematics, literacy, language, science)	Arztmann et al., 2022; Villena-Taranilla, et al., 2022; Chen et al., 2022b; Balanskat et al., 2006; Grgurović, et al., 2013; Friedel et al., 2013; Zheng et al., 2016; Savva et al., 2022; Kao, 2014; Higgins et al., 2012; Wen & Walters, 2022; Liao et al., 2007; Cheung & Slavin, 2011; Schwabe et al., 2022; Li & Ma, 2010; Verschaffel et al., 2019; Ran et al., 2022; Liao et al., 2007; Hillmayr et al., 2020; Kalemkuş & Kalemkuş, 2022; Lei et al., 2022b; Condie & Munro, 2007; Chauhan, 2017; Bado, 2022; Wang et al., 2022; Pan et al., 2022
• Positive learning gains for special needs students and low-achieving students	Eng, 2005; Balanskat et al., 2006; Punie et al., 2006; Koh, 2022
• Oportunities to develop a range of skills (e.g., subject-related skills, communication skills, negotiation skills, emotion control skills, organizational skills, critical thinking skills, creativity, metacognitive skills, life, and career skills)	Balanskat et al., 2006; Fu, 2013; Tamim et al., 2015; Zheng et al., 2016; Higgins et al., 2012; Verschaffel et al., 2019; Su & Yang, 2022; Su et al., 2022; Lu et al., 2022; Liu et al., 2022; Quah & Ng, 2022; Fielding & Murcia, 2022; Tang et al., 2022; Haleem et al., 2022
• Oportunities to develop digital skills (e.g., information skills, media skills, ICT skills)	Zheng et al., 2016; Su & Yang, 2022; Lu et al., 2022; Su et al., 2022
• Positive attitudes and behaviours towards ICTs, positive emotions (e.g., increased interest, motivation, attention, engagement, confidence, reduced anxiety, positive achievement emotions, reduction in bullying and cyberbullying)	Balanskat et al., 2006; Schmid et al., 2014; Zheng et al., 2016; Fadda et al., 2022; Higgins et al., 2012; Chen et al., 2022a; Lei et al., 2022b; Arztmann et al., 2022; Su et al., 2022
Learning experience
• Enhance access to resources	Jewitt et al., 2011; Fu, 2013
• Opportunities to experience various learning practices (e.g., active learning, learner-centred learning, independent and personalized learning, collaborative learning, self-directed learning, self- and peer-review)	Jewitt et al., 2011; Fu, 2013
• Improved access to teacher assessment and feedback	Jewitt et al., 2011
Equality, inclusion, and social integration
• Improved communication, functional skills, participation, self-esteem, and engagement of special needs students	Condie & Munro, 2007; Baragash et al., 2022; Koh, 2022
• Enhanced social interaction for students in general and for students with learning difficulties	Istenic Starcic & Bagon, 2014
• Benefits for both girls and boys	Zheng et al., 2016; Arztmann et al., 2022
Teachers
Professional practice
• Development of digital competence	Balanskat et al., 2006
• Positive attitudes and behaviours towards ICTs (e.g., increased confidence)	Punie et al., 2006,
• Formalized collaborative planning between teachers	Balanskat et al., 2006
• Improved reporting to parents	Balanskat et al., 2006
Teaching practice
• Efficiency in lesson planning and preparation	Balanskat et al., 2006
• Facilitate assessment through the provision of immediate feedback	Punie et al., 2006
• Improvements in the technical quality of tests	Punie et al., 2006
• New methods of testing specific skills (e.g., problem-solving skills, meta-cognitive skills)	Punie et al., 2006
• Successful content delivery and lessons	Friedel et al., 2013
• Application of different instructional practices (e.g., scaffolding, synchronous collaborative learning, online learning, blended learning, hybrid learning)	Friedel et al., 2013; Bado, 2022; Kazu & Yalçin, 2022; Ulum, 2022
Administrators
Data-based decision-making
• Improved data-gathering processes	Balanskat et al., 2006
• Support monitoring and evaluation processes (e.g., attendance monitoring, financial management, assessment records)	Condie & Munro, 2007
Organizational processes
• Access to learning resources via the creation of repositories	Condie & Munro, 2007
• Information sharing between school staff	Condie & Munro, 2007
• Smooth communications with external authorities (e.g., examination results)	Punie et al., 2006
• Efficient and successful examination management procedures	Punie et al., 2006
Home-school communication
• Support reporting to parents	Condie & Munro, 2007
• Improved flow of communication between the school and parents (e.g., customized and personalized communications)	Escueta et al., 2017
School leaders
Professional practice
• Reduced headteacher isolation	Condie & Munro, 2007
• Improved access to insights about practices for school improvement	Condie & Munro, 2007
Parents
Home-school relationships
• Improved home-school relationships	Zheng et al., 2016
• Increased parental involvement in children’s school life	Escueta et al., 2017

Table 3.

Tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript

Technologies/tools/practices/policies	References
ICT general – various types of technologies	Eng, 2005 (review) Moran et al., 2008 (meta-analysis) Balanskat et al., 2006 (report) Punie et al., 2006 (review) Fu, 2013 (review) Higgins et al., 2012 (report) Chauhan, 2017 (meta-analysis) Schmid et al., 2014 (meta-analysis) Grgurović et al., 2013 (meta-analysis) Higgins et al., 2012 (meta-analysis) Wen & Walters, 2022 (meta-analysis) Cheung & Slavin, 2011 (meta-analysis) Li & Ma, 2010 (meta-analysis) Hillmayr et al., 2020 (meta-analysis) Verschaffel et al., 2019 (systematic review) Ran et al., 2022 (meta-analysis) Fielding & Murcia, 2022 (systematic review) Tang et al., 2022 (review) Haleem et al., 2022 (review) Condie & Munro, 2007 (review) Underwood, 2009 (review) Istenic Starcic & Bagon, 2014 (review) Cussó-Calabuig et al., 2018 (systematic review) Escueta et al. (2017) (review) Archer et al., 2014 (meta-analysis) Lee et al., 2020 (meta-analysis) Delgado et al., 2015 (review) Di Pietro et al., 2020 (report)
Practices/policies on schools’ digital transformation	Bingimlas, 2009 (review) Hardman, 2019 (review) Hattie, 2008 (synthesis of multiple meta-analysis) Trucano, 2005 (book-Knowledge maps) Ređep, 2021 (policy study) Conrads et al, 2017 (report) European Commission, 2019 (EU report) Elkordy & Lovinelli, 2020 (book chapter) Eurydice, 2019 (EU report) Vuorikari et al., 2020 (JRC paper) Sellar, 2015 (review) European Commission, 2020 (EU report) OECD, 2015 (international paper)
Computer-assisted instruction, computer simulations, activeboards, and web-based learning	Liao et al., 2007 (meta-analysis) Tamim et al., 2011 (meta-analysis) Çelik, 2022 (review) Moran et al., 2008 (meta-analysis) Eng, 2005 (review)
Learning platforms (LPs) (virtual learning environments, management information systems, communication technologies and information and resource sharing technologies)	Jewitt et al., 2011 (report)
Mobile devices—touch screens (smart devices, tablets, laptops)	Sung et al., 2016 (meta-analysis and research synthesis) Tamim et al., 2011 (meta-analysis) Tamim et al., 2015 (systematic review and meta-analysis) Zheng et al., 2016 (meta-analysis and research synthesis) Haßler et al., 2016 (review) Kalati & Kim, 2022 (systematic review) Friedel et al., 2013 (meta-analysis and review) Chen et al., 2022b (meta-analysis) Schwabe et al., 2022 (meta-analysis) Punie et al., 2006 (review)
Digital games (various types e.g., adventure, serious; various domains e.g., history, science)	Wang et al., 2022 (meta-analysis) Arztmann et al., 2022 (meta-analysis) Martinez et al., 2022 (systematic review) Talan et al., 2020 (meta-analysis) Pan et al., 2022 (systematic review) Chen et al., 2022a (meta-analysis) Kao, 2014 (meta-analysis) Fadda et al., 2022 (meta-analysis) Lu et al., 2022 (meta-analysis) Lei et al., 2022a (meta-analysis) Koh, 2022 (meta-analysis) Bado, 2022 (review)
Augmented reality (AR)	Garzón & Acevedo, 2019 (meta-analysis) Garzón et al., 2020 (meta-analysis and research synthesis) Kalemkuş & Kalemkuş, 2022 (meta-analysis) Baragash et al., 2022 (meta-analysis)
Virtual reality (VR) Immersive virtual reality (IVR)	Villena-Taranilla et al., 2022 (meta-analysis) Chen et al., 2022b (meta-analysis) Coban et al., 2022 (meta-analysis)
Artificial intelligence (AI) and robotics	Su & Yang, 2022 (review) Su et al., 2022 (meta review)
Online learning/elearning	Ulum, 2022 (meta-analysis) Cheok & Wong, 2015 (review)
Blended learning	Grgurović et al., 2013 (meta-analysis)
Synchronous parallel participation	Friedel et al., 2013 (meta-analysis and review)
Electronic books/digital storytelling	Savva et al., 2022 (meta-analysis) Quah & Ng, 2022 (systematic review)
Multimedia technology	Liu et al., 2022 (meta-analysis)
Hybrid learning	Kazu & Yalçin, 2022 (meta-analysis)

Additionally, based on the results of the literature review, there are many types of digital technologies with different affordances (see, for example, studies on VR vs Immersive VR), which evolve over time (e.g. starting from CAIs in 2005 to Augmented and Virtual reality 2020). Furthermore, these technologies are linked to different pedagogies and policy initiatives, which are critical factors in the study of impact. Table 3 summarizes the different tools and practices that have been used to examine the impact of digital technologies on education since 2005 based on the review results.

Factors that affect the integration of digital technologies

Although the analysis of the literature review demonstrated different impacts of the use of digital technology on education, several authors highlighted the importance of various factors, besides the technology itself, that affect this impact. For example, Liao et al. (2007) suggested that future studies should carefully investigate which factors contribute to positive outcomes by clarifying the exact relationship between computer applications and learning. Additionally, Haßler et al., (2016) suggested that the neutral findings regarding the impact of tablets on students learning outcomes in some of the studies included in their review should encourage educators, school leaders, and school officials to further investigate the potential of such devices in teaching and learning. Several other researchers suggested that a number of variables play a significant role in the impact of ICTs on students’ learning that could be attributed to the school context, teaching practices and professional development, the curriculum, and learners’ characteristics (Underwood, 2009; Tamim et al., 2011; Higgins et al., 2012; Archer et al., 2014; Sung et al., 2016; Haßler et al., 2016; Chauhan, 2017; Lee et al., 2020; Tang et al., 2022).

Digital competencies

One of the most common challenges reported in studies that utilized digital tools in the classroom was the lack of students’ skills on how to use them. Fu (2013) found that students’ lack of technical skills is a barrier to the effective use of ICT in the classroom. Tamim et al. (2015) reported that students faced challenges when using tablets and smart mobile devices, associated with the technical issues or expertise needed for their use and the distracting nature of the devices and highlighted the need for teachers’ professional development. Higgins et al. (2012) reported that skills training about the use of digital technologies is essential for learners to fully exploit the benefits of instruction.

Delgado et al. (2015), meanwhile, reported studies that showed a strong positive association between teachers’ computer skills and students’ use of computers. Teachers’ lack of ICT skills and familiarization with technologies can become a constraint to the effective use of technology in the classroom (Balanskat et al., 2006; Delgado et al., 2015).

It is worth noting that the way teachers are introduced to ICTs affects the impact of digital technologies on education. Previous studies have shown that teachers may avoid using digital technologies due to limited digital skills (Balanskat, 2006), or they prefer applying “safe” technologies, namely technologies that their own teachers used and with which they are familiar (Condie & Munro, 2007). In this regard, the provision of digital skills training and exposure to new digital tools might encourage teachers to apply various technologies in their lessons (Condie & Munro, 2007). Apart from digital competence, technical support in the school setting has also been shown to affect teachers’ use of technology in their classrooms (Delgado et al., 2015). Ferrari et al. (2011) found that while teachers’ use of ICT is high, 75% stated that they needed more institutional support and a shift in the mindset of educational actors to achieve more innovative teaching practices. The provision of support can reduce time and effort as well as cognitive constraints, which could cause limited ICT integration in the school lessons by teachers (Escueta et al., 2017).

Teachers’ personal characteristics, training approaches, and professional development

Teachers’ personal characteristics and professional development affect the impact of digital technologies on education. Specifically, Cheok and Wong (2015) found that teachers’ personal characteristics (e.g., anxiety, self-efficacy) are associated with their satisfaction and engagement with technology. Bingimlas (2009) reported that lack of confidence, resistance to change, and negative attitudes in using new technologies in teaching are significant determinants of teachers’ levels of engagement in ICT. The same author reported that the provision of technical support, motivation support (e.g., awards, sufficient time for planning), and training on how technologies can benefit teaching and learning can eliminate the above barriers to ICT integration. Archer et al. (2014) found that comfort levels in using technology are an important predictor of technology integration and argued that it is essential to provide teachers with appropriate training and ongoing support until they are comfortable with using ICTs in the classroom. Hillmayr et al. (2020) documented that training teachers on ICT had an important effecton students’ learning.

According to Balanskat et al. (2006), the impact of ICTs on students’ learning is highly dependent on the teachers’ capacity to efficiently exploit their application for pedagogical purposes. Results obtained from the Teaching and Learning International Survey (TALIS) (OECD, 2021) revealed that although schools are open to innovative practices and have the capacity to adopt them, only 39% of teachers in the European Union reported that they are well or very well prepared to use digital technologies for teaching. Li and Ma (2010) and Hardman (2019) showed that the positive effect of technology on students’ achievement depends on the pedagogical practices used by teachers. Schmid et al. (2014) reported that learning was best supported when students were engaged in active, meaningful activities with the use of technological tools that provided cognitive support. Tamim et al. (2015) compared two different pedagogical uses of tablets and found a significant moderate effect when the devices were used in a student-centered context and approach rather than within teacher-led environments. Similarly, Garzón and Acevedo (2019) and Garzón et al. (2020) reported that the positive results from the integration of AR applications could be attributed to the existence of different variables which could influence AR interventions (e.g., pedagogical approach, learning environment, and duration of the intervention). Additionally, Garzón et al. (2020) suggested that the pedagogical resources that teachers used to complement their lectures and the pedagogical approaches they applied were crucial to the effective integration of AR on students’ learning gains. Garzón and Acevedo (2019) also emphasized that the success of a technology-enhanced intervention is based on both the technology per se and its characteristics and on the pedagogical strategies teachers choose to implement. For instance, their results indicated that the collaborative learning approach had the highest impact on students’ learning gains among other approaches (e.g., inquiry-based learning, situated learning, or project-based learning). Ran et al. (2022) also found that the use of technology to design collaborative and communicative environments showed the largest moderator effects among the other approaches.

Hattie (2008) reported that the effective use of computers is associated with training teachers in using computers as a teaching and learning tool. Zheng et al. (2016) noted that in addition to the strategies teachers adopt in teaching, ongoing professional development is also vital in ensuring the success of technology implementation programs. Sung et al. (2016) found that research on the use of mobile devices to support learning tends to report that the insufficient preparation of teachers is a major obstacle in implementing effective mobile learning programs in schools. Friedel et al. (2013) found that providing training and support to teachers increased the positive impact of the interventions on students’ learning gains. Trucano (2005) argued that positive impacts occur when digital technologies are used to enhance teachers’ existing pedagogical philosophies. Higgins et al. (2012) found that the types of technologies used and how they are used could also affect students’ learning. The authors suggested that training and professional development of teachers that focuses on the effective pedagogical use of technology to support teaching and learning is an important component of successful instructional approaches (Higgins et al., 2012). Archer et al. (2014) found that studies that reported ICT interventions during which teachers received training and support had moderate positive effects on students’ learning outcomes, which were significantly higher than studies where little or no detail about training and support was mentioned. Fu (2013) reported that the lack of teachers’ knowledge and skills on the technical and instructional aspects of ICT use in the classroom, in-service training, pedagogy support, technical and financial support, as well as the lack of teachers’ motivation and encouragement to integrate ICT on their teaching were significant barriers to the integration of ICT in education.

School leadership and management

Management and leadership are important cornerstones in the digital transformation process (Pihir et al., 2018). Zheng et al. (2016) documented leadership among the factors positively affecting the successful implementation of technology integration in schools. Strong leadership, strategic planning, and systematic integration of digital technologies are prerequisites for the digital transformation of education systems (Ređep, 2021). Management and leadership play a significant role in formulating policies that are translated into practice and ensure that developments in ICT become embedded into the life of the school and in the experiences of staff and pupils (Condie & Munro, 2007). Policy support and leadership must include the provision of an overall vision for the use of digital technologies in education, guidance for students and parents, logistical support, as well as teacher training (Conrads et al., 2017). Unless there is a commitment throughout the school, with accountability for progress at key points, it is unlikely for ICT integration to be sustained or become part of the culture (Condie & Munro, 2007). To achieve this, principals need to adopt and promote a whole-institution strategy and build a strong mutual support system that enables the school’s technological maturity (European Commission, 2019). In this context, school culture plays an essential role in shaping the mindsets and beliefs of school actors towards successful technology integration. Condie and Munro (2007) emphasized the importance of the principal’s enthusiasm and work as a source of inspiration for the school staff and the students to cultivate a culture of innovation and establish sustainable digital change. Specifically, school leaders need to create conditions in which the school staff is empowered to experiment and take risks with technology (Elkordy & Lovinelli, 2020).

In order for leaders to achieve the above, it is important to develop capacities for learning and leading, advocating professional learning, and creating support systems and structures (European Commission, 2019). Digital technology integration in education systems can be challenging and leadership needs guidance to achieve it. Such guidance can be introduced through the adoption of new methods and techniques in strategic planning for the integration of digital technologies (Ređep, 2021). Even though the role of leaders is vital, the relevant training offered to them has so far been inadequate. Specifically, only a third of the education systems in Europe have put in place national strategies that explicitly refer to the training of school principals (European Commission, 2019, p. 16).

Connectivity, infrastructure, and government and other support

The effective integration of digital technologies across levels of education presupposes the development of infrastructure, the provision of digital content, and the selection of proper resources (Voogt et al., 2013). Particularly, a high-quality broadband connection in the school increases the quality and quantity of educational activities. There is evidence that ICT increases and formalizes cooperative planning between teachers and cooperation with managers, which in turn has a positive impact on teaching practices (Balanskat et al., 2006). Additionally, ICT resources, including software and hardware, increase the likelihood of teachers integrating technology into the curriculum to enhance their teaching practices (Delgado et al., 2015). For example, Zheng et al. (2016) found that the use of one-on-one laptop programs resulted in positive changes in teaching and learning, which would not have been accomplished without the infrastructure and technical support provided to teachers. Delgado et al. (2015) reported that limited access to technology (insufficient computers, peripherals, and software) and lack of technical support are important barriers to ICT integration. Access to infrastructure refers not only to the availability of technology in a school but also to the provision of a proper amount and the right types of technology in locations where teachers and students can use them. Effective technical support is a central element of the whole-school strategy for ICT (Underwood, 2009). Bingimlas (2009) reported that lack of technical support in the classroom and whole-school resources (e.g., failing to connect to the Internet, printers not printing, malfunctioning computers, and working on old computers) are significant barriers that discourage the use of ICT by teachers. Moreover, poor quality and inadequate hardware maintenance, and unsuitable educational software may discourage teachers from using ICTs (Balanskat et al., 2006; Bingimlas, 2009).

Government support can also impact the integration of ICTs in teaching. Specifically, Balanskat et al. (2006) reported that government interventions and training programs increased teachers’ enthusiasm and positive attitudes towards ICT and led to the routine use of embedded ICT.

Lastly, another important factor affecting digital transformation is the development and quality assurance of digital learning resources. Such resources can be support textbooks and related materials or resources that focus on specific subjects or parts of the curriculum. Policies on the provision of digital learning resources are essential for schools and can be achieved through various actions. For example, some countries are financing web portals that become repositories, enabling teachers to share resources or create their own. Additionally, they may offer e-learning opportunities or other services linked to digital education. In other cases, specific agencies of projects have also been set up to develop digital resources (Eurydice, 2019).

Administration and digital data management

The digital transformation of schools involves organizational improvements at the level of internal workflows, communication between the different stakeholders, and potential for collaboration. Vuorikari et al. (2020) presented evidence that digital technologies supported the automation of administrative practices in schools and reduced the administration’s workload. There is evidence that digital data affects the production of knowledge about schools and has the power to transform how schooling takes place. Specifically, Sellar (2015) reported that data infrastructure in education is developing due to the demand for “information about student outcomes, teacher quality, school performance, and adult skills, associated with policy efforts to increase human capital and productivity practices” (p. 771). In this regard, practices, such as datafication which refers to the “translation of information about all kinds of things and processes into quantified formats” have become essential for decision-making based on accountability reports about the school’s quality. The data could be turned into deep insights about education or training incorporating ICTs. For example, measuring students’ online engagement with the learning material and drawing meaningful conclusions can allow teachers to improve their educational interventions (Vuorikari et al., 2020).

Students’ socioeconomic background and family support

Research show that the active engagement of parents in the school and their support for the school’s work can make a difference to their children’s attitudes towards learning and, as a result, their achievement (Hattie, 2008). In recent years, digital technologies have been used for more effective communication between school and family (Escueta et al., 2017). The European Commission (2020) presented data from a Eurostat survey regarding the use of computers by students during the pandemic. The data showed that younger pupils needed additional support and guidance from parents and the challenges were greater for families in which parents had lower levels of education and little to no digital skills.

In this regard, the socio-economic background of the learners and their socio-cultural environment also affect educational achievements (Punie et al., 2006). Trucano documented that the use of computers at home positively influenced students’ confidence and resulted in more frequent use at school, compared to students who had no home access (Trucano, 2005). In this sense, the socio-economic background affects the access to computers at home (OECD, 2015) which in turn influences the experience of ICT, an important factor for school achievement (Punie et al., 2006; Underwood, 2009). Furthermore, parents from different socio-economic backgrounds may have different abilities and availability to support their children in their learning process (Di Pietro et al., 2020).

Schools’ socioeconomic context and emergency situations

The socio-economic context of the school is closely related to a school’s digital transformation. For example, schools in disadvantaged, rural, or deprived areas are likely to lack the digital capacity and infrastructure required to adapt to the use of digital technologies during emergency periods, such as the COVID-19 pandemic (Di Pietro et al., 2020). Data collected from school principals confirmed that in several countries, there is a rural/urban divide in connectivity (OECD, 2015).

Emergency periods also affect the digitalization of schools. The COVID-19 pandemic led to the closure of schools and forced them to seek appropriate and connective ways to keep working on the curriculum (Di Pietro et al., 2020). The sudden large-scale shift to distance and online teaching and learning also presented challenges around quality and equity in education, such as the risk of increased inequalities in learning, digital, and social, as well as teachers facing difficulties coping with this demanding situation (European Commission, 2020).

Looking at the findings of the above studies, we can conclude that the impact of digital technologies on education is influenced by various actors and touches many aspects of the school ecosystem. Figure 1 summarizes the factors affecting the digital technologies’ impact on school stakeholders based on the findings from the literature review.

Fig. 1.

Factors that affect the impact of ICTs on education

Discussion

The findings revealed that the use of digital technologies in education affects a variety of actors within a school’s ecosystem. First, we observed that as technologies evolve, so does the interest of the research community to apply them to school settings. Figure 2 summarizes the trends identified in current research around the impact of digital technologies on schools’ digital capacity and transformation as found in the present study. Starting as early as 2005, when computers, simulations, and interactive boards were the most commonly applied tools in school interventions (e.g., Eng, 2005; Liao et al., 2007; Moran et al., 2008; Tamim et al., 2011), moving towards the use of learning platforms (Jewitt et al., 2011), then to the use of mobile devices and digital games (e.g., Tamim et al., 2015; Sung et al., 2016; Talan et al., 2020), as well as e-books (e.g., Savva et al., 2022), to the more recent advanced technologies, such as AR and VR applications (e.g., Garzón & Acevedo, 2019; Garzón et al., 2020; Kalemkuş & Kalemkuş, 2022), or robotics and AI (e.g., Su & Yang, 2022; Su et al., 2022). As this evolution shows, digital technologies are a concept in flux with different affordances and characteristics. Additionally, from an instructional perspective, there has been a growing interest in different modes and models of content delivery such as online, blended, and hybrid modes (e.g., Cheok & Wong, 2015; Kazu & Yalçin, 2022; Ulum, 2022). This is an indication that the value of technologies to support teaching and learning as well as other school-related practices is increasingly recognized by the research and school community. The impact results from the literature review indicate that ICT integration on students’ learning outcomes has effects that are small (Coban et al., 2022; Eng, 2005; Higgins et al., 2012; Schmid et al., 2014; Tamim et al., 2015; Zheng et al., 2016) to moderate (Garzón & Acevedo, 2019; Garzón et al., 2020; Liao et al., 2007; Sung et al., 2016; Talan et al., 2020; Wen & Walters, 2022). That said, a number of recent studies have reported high effect sizes (e.g., Kazu & Yalçin, 2022).

Fig. 2.

Current work and trends in the study of the impact of digital technologies on schools’ digital capacity

Based on these findings, several authors have suggested that the impact of technology on education depends on several variables and not on the technology per se (Tamim et al., 2011; Higgins et al., 2012; Archer et al., 2014; Sung et al., 2016; Haßler et al., 2016; Chauhan, 2017; Lee et al., 2020; Lei et al., 2022a). While the impact of ICTs on student achievement has been thoroughly investigated by researchers, other aspects related to school life that are also affected by ICTs, such as equality, inclusion, and social integration have received less attention. Further analysis of the literature review has revealed a greater investment in ICT interventions to support learning and teaching in the core subjects of literacy and STEM disciplines, especially mathematics, and science. These were the most common subjects studied in the reviewed papers often drawing on national testing results, while studies that investigated other subject areas, such as social studies, were limited (Chauhan, 2017; Condie & Munro, 2007). As such, research is still lacking impact studies that focus on the effects of ICTs on a range of curriculum subjects.

The qualitative research provided additional information about the impact of digital technologies on education, documenting positive effects and giving more details about implications, recommendations, and future research directions. Specifically, the findings regarding the role of ICTs in supporting learning highlight the importance of teachers’ instructional practice and the learning context in the use of technologies and consequently their impact on instruction (Çelik, 2022; Schmid et al., 2014; Tamim et al., 2015). The review also provided useful insights regarding the various factors that affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the transformation process. Specifically, these factors include a) digital competencies; b) teachers’ personal characteristics and professional development; c) school leadership and management; d) connectivity, infrastructure, and government support; e) administration and data management practices; f) students’ socio-economic background and family support and g) the socioeconomic context of the school and emergency situations. It is worth noting that we observed factors that affect the integration of ICTs in education but may also be affected by it. For example, the frequent use of ICTs and the use of laptops by students for instructional purposes positively affect the development of digital competencies (Zheng et al., 2016) and at the same time, the digital competencies affect the use of ICTs (Fu, 2013; Higgins et al., 2012). As a result, the impact of digital technologies should be explored more as an enabler of desirable and new practices and not merely as a catalyst that improves the output of the education process i.e. namely student attainment.

Conclusions

Digital technologies offer immense potential for fundamental improvement in schools. However, investment in ICT infrastructure and professional development to improve school education are yet to provide fruitful results. Digital transformation is a complex process that requires large-scale transformative changes that presuppose digital capacity and preparedness. To achieve such changes, all actors within the school’s ecosystem need to share a common vision regarding the integration of ICTs in education and work towards achieving this goal. Our literature review, which synthesized quantitative and qualitative data from a list of meta-analyses and review studies, provided useful insights into the impact of ICTs on different school stakeholders and showed that the impact of digital technologies touches upon many different aspects of school life, which are often overlooked when the focus is on student achievement as the final output of education. Furthermore, the concept of digital technologies is a concept in flux as technologies are not only different among them calling for different uses in the educational practice but they also change through time. Additionally, we opened a forum for discussion regarding the factors that affect a school’s digital capacity and transformation. We hope that our study will inform policy, practice, and research and result in a paradigm shift towards more holistic approaches in impact and assessment studies.

Study limitations and future directions

We presented a review of the study of digital technologies' impact on education and factors influencing schools’ digital capacity and transformation. The study results were based on a non-systematic literature review grounded on the acquisition of documentation in specific databases. Future studies should investigate more databases to corroborate and enhance our results. Moreover, search queries could be enhanced with key terms that could provide additional insights about the integration of ICTs in education, such as “policies and strategies for ICT integration in education”. Also, the study drew information from meta-analyses and literature reviews to acquire evidence about the effects of ICT integration in schools. Such evidence was mostly based on the general conclusions of the studies. It is worth mentioning that, we located individual studies which showed different, such as negative or neutral results. Thus, further insights are needed about the impact of ICTs on education and the factors influencing the impact. Furthermore, the nature of the studies included in meta-analyses and reviews is different as they are based on different research methodologies and data gathering processes. For instance, in a meta-analysis, the impact among the studies investigated is measured in a particular way, depending on policy or research targets (e.g., results from national examinations, pre-/post-tests). Meanwhile, in literature reviews, qualitative studies offer additional insights and detail based on self-reports and research opinions on several different aspects and stakeholders who could affect and be affected by ICT integration. As a result, it was challenging to draw causal relationships between so many interrelating variables.

Despite the challenges mentioned above, this study envisaged examining school units as ecosystems that consist of several actors by bringing together several variables from different research epistemologies to provide an understanding of the integration of ICTs. However, the use of other tools and methodologies and models for evaluation of the impact of digital technologies on education could give more detailed data and more accurate results. For instance, self-reflection tools, like SELFIE—developed on the DigCompOrg framework- (Kampylis et al., 2015; Bocconi & Lightfoot, 2021) can help capture a school’s digital capacity and better assess the impact of ICTs on education. Furthermore, the development of a theory of change could be a good approach for documenting the impact of digital technologies on education. Specifically, theories of change are models used for the evaluation of interventions and their impact; they are developed to describe how interventions will work and give the desired outcomes (Mayne, 2015). Theory of change as a methodological approach has also been used by researchers to develop models for evaluation in the field of education (e.g., Aromatario et al., 2019; Chapman & Sammons, 2013; De Silva et al., 2014).

We also propose that future studies aim at similar investigations by applying more holistic approaches for impact assessment that can provide in-depth data about the impact of digital technologies on education. For instance, future studies could focus on different research questions about the technologies that are used during the interventions or the way the implementation takes place (e.g., What methodologies are used for documenting impact? How are experimental studies implemented? How can teachers be taken into account and trained on the technology and its functions? What are the elements of an appropriate and successful implementation? How is the whole intervention designed? On which learning theories is the technology implementation based?).

Future research could also focus on assessing the impact of digital technologies on various other subjects since there is a scarcity of research related to particular subjects, such as geography, history, arts, music, and design and technology. More research should also be done about the impact of ICTs on skills, emotions, and attitudes, and on equality, inclusion, social interaction, and special needs education. There is also a need for more research about the impact of ICTs on administration, management, digitalization, and home-school relationships. Additionally, although new forms of teaching and learning with the use of ICTs (e.g., blended, hybrid, and online learning) have initiated several investigations in mainstream classrooms, only a few studies have measured their impact on students’ learning. Additionally, our review did not document any study about the impact of flipped classrooms on K-12 education. Regarding teaching and learning approaches, it is worth noting that studies referred to STEM or STEAM did not investigate the impact of STEM/STEAM as an interdisciplinary approach to learning but only investigated the impact of ICTs on learning in each domain as a separate subject (science, technology, engineering, arts, mathematics). Hence, we propose future research to also investigate the impact of the STEM/STEAM approach on education. The impact of emerging technologies on education, such as AR, VR, robotics, and AI has also been investigated recently, but more work needs to be done.

Finally, we propose that future studies could focus on the way in which specific factors, e.g., infrastructure and government support, school leadership and management, students’ and teachers’ digital competencies, approaches teachers utilize in the teaching and learning (e.g., blended, online and hybrid learning, flipped classrooms, STEM/STEAM approach, project-based learning, inquiry-based learning), affect the impact of digital technologies on education. We hope that future studies will give detailed insights into the concept of schools’ digital transformation through further investigation of impacts and factors which influence digital capacity and transformation based on the results and the recommendations of the present study.

Data availability statement

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Declarations

Conflict of interest

None