Exploring AR and VR Tools in Mathematics Education Through Culturally Responsive Pedagogies

Abstract

Augmented reality (AR) and virtual reality (VR) have been noted to enhance student learning by supporting spatial reasoning and visualization, long-term memory, engagement, and increased motivation. The researchers situated the exploration of these tools for learning within the culturally responsive pedagogy (CRP) in mathematics education. The researchers conducted a qualitative case study interlinked with design-based research (DBR). Data were collected using questionnaires, interviews, observations, and documentation. Sixty-five students in grades three to eight and 10 adults participated in the context of a STEAM camp. Students used tools such as block-based coding and digital and crafting design materials to learn, understand, and apply mathematical concepts (i.e., for mathematical thinking and modelling). The researchers designed and taught the learning activities which used a game-design-like coding software and Cospaces Edu app. The AR and VR activities were within the learning contexts of storytelling and cultural artifacts. The researchers report the study’s results, analyzing mathematics concepts learned through the activities. Specifically, students’ motivation was boosted when students used game-design-like software within the storytelling context of the app.

Introduction

As technology rapidly increases in educational settings, scholars are exploring its potential benefits for learning (Schutera et al., 2021). Augmented reality (AR) and virtual reality (VR) digital technologies can be used to create a learning space for students where they interact with digital visualization and simulations. AR learners engage with a physical world perceptually altered using digital software or devices (Ardiny & Khanmirza, 2018). For example, computer-generated content can be embedded such that a hologram image is projected off the surface of a book, allowing learners digitally to interact with (or manipulate) that digital image. VR, by contrast, through sensory stimuli, immerses learners completely in a virtual environment in which the physical world is replaced by a simulated one (Ardiny & Khanmirza, 2018). For example, AR presents a unique learning space in a “game-like environment with 3D models” (Schutera et al., 2021, p. 2) that allows students to visualize, interact with, and manipulate (i.e., engage with) the object as if it is in three dimensions, providing them with a new, different, or better understanding of mathematical concepts such as geometry ideas represented.

Mathematics “is one of the disciplines that need computational thinking (CT) abilities [… to promote and develop] logical, imaginative, and critical thinking” (Angraini et al., 2023, p. 1034). One of the most common ways students learn CT skills is through computer programming or coding (Yang et al., 2023). For example, students learn CT skills such as abstraction (i.e., dissecting a problem and breaking it down to its most essential components), debugging (i.e., finding and fixing coding errors), remixing (i.e., modifying or altering the code), and iteration (i.e., revising and improving the design) for problem solving (Brennan & Resnick, 2012; Resnick et al., 2009; Wing, 2008; Yang et al., 2023).

Visual programming languages, such as block-based coding, are more suitable for novice programmers and can aid students in learning, understanding, and applying CT skills (Lye & Koh, 2014; Yang et al., 2023), and curricular content. For example, some students may communicate their ideas and thoughts through coding stories illustrated multimodally such as with sounds, pictures, and movements rather than solely through text. Students use block-based coding to express their ideas using digital animations and storytelling. Pöllänen and Pöllänen (2019) noted that CT concepts equip students to select appropriate representations, which can significantly improve intellectual skills and enhance digital competency and literacy (diSessa, 2000, 2018).

Researchers argue that AR and VR representations improve learning by supporting visualization, boosting long-term memory, increasing engagement, and motivation (Ardiny & Khanmirza, 2018; Schutera et al., 2021), fostering student interaction, idea exchange, and collaboration by reducing anxiety (Buentello-Montoya et al., 2021). Chen (2010) argued that AR and VR technologies are “merely [… tools which] by themselves do not teach. They have to be carefully and effectively implemented to assist in the learning process” (p. 73).

Quah and Ng (2022) noticed that “there has been a shift in technology trends focusing more on [a tangible user interface] mixing physical and digital interactions in story making” (p. 851), through technologies such as those for augmented and virtual reality. To improve students’ motivation and engagement in learning, “technology-supported storytelling activities have been used as an effective method of achieving these goals […] AR is one such new technology that has attracted attention and can be used in storytelling” (Yilmaz & Goktas, 2017, p. 75). Digital storytelling can be used as an educational method to enhance students’ expression, communication, and digital skills (Del-Moral-Pérez et al., 2019).

Specifically, digital storytelling has been used to promote literacy and cross-curricular skills through the integration of science, mathematics, and social science (Quah & Ng, 2022). Through digital storytelling, students can also explore their individual, group, and community identity or other identities different than their own (Wake, 2012). Digital storytelling may provide students with the opportunity to explore “multicultural and cross-cultural stories […] developing creativity and social and cultural awareness” (as cited in Quah & Ng, 2022, p. 855; Sylla et al., 2019). For example, Hill and Grinnell (2014) noted that digital storytelling humanizes technology as students explore their lived experiences and self-identity through personal and relational interactions with friends, family, and community. Research on digital storytelling builds on earlier studies on the use of technology, in support of student learning in the context of culture (e.g., Eglash et al., 2013) and on the use of storytelling (Zazkis & Liljedahl, 2009) in mathematics teaching.

Context of the Study

According to Schutera et al. (2021), “there are only a few studies on the benefits of AR for teaching mathematics [… and] AR is rarely used in mathematic classes” (p. 2). Specifically, Quah and Ng (2022) found a limited number of studies on digital storytelling using AR and VR technologies in mathematics education. The use of AR and VR tools in teaching needs further investigation to explore how activities support learning, such as through improved engagement and increased motivation, how these activities are better designed to support learning, and how the cognitive learning challenges associated with the use of these as tools for learning may be addressed. With this context in mind, this article will address the following research questions: How can students’ storytelling using AR and VR technologies support students’ engagement and motivation in the context of computational thinking and modelling skills of coding and designing? In which ways does digital storytelling using AR and VR apps and tangibles (tools) support students’ learning, understanding, and application of mathematics concepts?

Theoretical Framework

In this study, the researchers adopted two theoretical frameworks: Aguirre and del Rosario Zavala’s culturally responsive mathematics teaching (CRMT) tool and Borba and Villarreal’s conceptualization of humans-with-technology and humans-with-media.

When defining culture as dynamic and nested, Nolan and Xenofontos (2023) maintain that culturally responsive pedagogy (CRP) is an approach toward equitable and socially just mathematics teaching. Certain researchers such as Nicol et al. (2013) define CRP in the context of cultures of groups of people and as an approach that facilitates, honors, builds on, and draws from the “culture, knowledge, and language of students, teachers, and local community” (p. 76). Drawing from Ladson-Billings (1995), the CRMT tool was developed by Aguirre and del Rosario Zavala (2013) as a professional development tool for implementing CRP in the context of new teachers learning to teach mathematics. Although the literature describes CRMT as an effective tool used in many studies, it has been noted that (a) implementing culturally responsive practices in mathematics classrooms is a complex multidimensional process, and (b) over time, teachers have struggled with creating meaningful connections between their students’ lived experiences and the mathematics knowledge taught in schools (Rousseau Anderson, 2021).

Aguirre and del Rosario Zavala (2013) argue that the CRMT tool is “essential to student learning” (p. 164), and it provides a culturally responsive approach to teaching mathematics in the classroom (Abdulrahim & Orosco, 2020; Bonner, 2021; Celedón-Pattichis et al., 2018). In our study, first, the researchers specifically used the CRMT tool to design the classroom activities for the study. Second, researchers selected six of the eight CRMT tool dimensions related to the focus of the study to interpret and analyze the data collected: intellectual support, depth of student knowledge and understanding, subject analysis, communication and discourse, student engagement, and the integration of the students’ funds of knowledge, cultures, and backgrounds (Aguirre & del Rosario Zavala, 2013). One of the dimensions is directly related to students’ culture and background, and the other five dimensions of CRMT, according to Aguirre and del Rosario Zavala (2013), support this dimension by enhancing academic achievement through instruction (i.e., scaffolded intellectual support for all students), promoting understanding (i.e., depth of student knowledge/understanding, subject analysis, and student discourse), and promoting overall student engagement and motivation (i.e., enhanced interaction between human actors and non-human digital tools).

Sum et al. (2024) noted that “despite the important work related to CRMT and storytelling in mathematics education, the two research studies have remained largely separated, and little is known about if and how storytelling supports students’ mathematics learning in […] sociocultural contexts” (pp. 637–638). Third, similar to Sum et al. (2024), who employed the CRMT tool when they investigated the efficacy of storytelling in developing fraction concepts through a CRMT approach, in this study, CRMT guided the selection of contexts used to encourage students to tell their own stories (Billinghurst & Duenser, 2012; Zazkis & Liljedahl, 2009), which were technology-supported as they coded and animated their characters using the Cospaces Edu app. Fourth, the researchers in this study also used the CRMT tool to study non-human actors, such as the cultural products and artifacts that students designed or selected.

According to Borba and Villarreal (2005), “knowledge is produced together with a given medium or technology of intelligence” (p. 23) through a collective process which they describe as humans-with-media, humans-with-technology, or humans-with-technology-of-intelligence, where humans and non-human actors both have agency. Borba (2021) explained that non-human actors or things could be natural (e.g., environment) or cultural (e.g., symbols or images that represent a student’s culture/background), or they could be artifacts (e.g., software, hardware, internet, apps). Thus, the researchers also conceptualized the AR and VR activities in the study through a conceptual lens of humans-with-technology and humans-with-media within the dimensions of the CRMT tool, specifically the interaction between human (students) and non-human (technology/digital tools/media/cultural artifacts) actors.

Methodology

The researchers conducted a qualitative case study interlinked with design-based research (DBR). A case study “is a qualitative approach in which the investigator explores a real-life, contemporary bounded system […] over time, through detailed, in-depth data collection involving multiple sources of information […] and reports a case description and themes” (Cresswell, 2012, p. 97). DBR, like the case study method, occurs in the natural or “real environment where education and practice take place” (Vaezi et al., 2019, p. 29). The researchers adopted the following features of DBR: it is designed to support learning (mathematics, coding, CT, and other STEM concepts), has a prospective (i.e., remembering and articulating what needs to be done) and reflective component (i.e., reflecting and documenting stage), and is cyclic in nature (see the iterative design cycle in Fig. 1).

Fig. 1.

Iterative design cycle used in this study adapted from Cobb et al. (2003)

In particular, the researchers used an “iterative design process featuring cycles of invention and revision” (Cobb et al., 2003, p. 10), “when creating and implementing a curriculum by trying it out and refining the design based on the instructor and student feedback” (Bertrand et al., 2023, pp. 49–50). The interlinking of the two research methods, case study with DBR, was necessary to implement and redesign the curriculum which integrates technology through an iterative design cycle. See Fig. 1 and Table 1 for more details about the iterative design cycle used in this study.

Table 1.

Summary of iterations of the design cycle and processes used to design and refine the curriculum from years 1 to 4

	Designing and refining	Creating and implementing	Reflecting and documenting
Year 1	Students design their dream space using block-based coding in Cospaces Edua	Curriculum created for the STEAM Camp with kids grades 3–8 in an online platform Zoom (due to the COVID-19 pandemic). The activities in Cospaces Edu were created to encourage exploration, experimentation, and creativity in design	Documented through informal observations and reflections. Students need an in-person learning platform to assist and increase students’ overall engagement. Students also need a more in-depth understanding of mathematical and coding concepts
Years 2 and 3	With the addition of the MERGE cubeb and AR/VR technologies, students used the MERGE cube virtually to interact with the design created in Cospaces Edu	To create a more meaningful and engaging learning experience, the curriculum was modified and adapted for two in-person STEAM Camps with kids grades 3–8. The focus of the camp was integrating digital tools in teaching mathematics concepts through a culturally responsive lens	Data was collected through student questionnaires, formal observations using a template, and teacher interviews. This lesson/activity on AR/VR seemed to be rushed. Students need to have more time to develop their skills and explore Cospaces Edu using different digital tools and platforms
Year 4	Students imported into Cospaces Edu, 360o virtual backgrounds they created using Skybox AI.c They then further designed scenarios to view and interact using VR goggles	The curriculum for in-person learning was modified to integrate AI education and digital tools. This was to connect to the new Mathematics and Science curriculum. In-person STEAM Camps with kids grades 3–8 and Maker Ed Workshops with Preservice Teachers. Cospaces Edu activity/lesson was expanded to 2 days to allow students to fully develop a story that represented their culture, interests, and backgrounds	Data was collected through student questionnaires, formal observations using a template, and teacher interviews. For future iterations, we need to reduce the cognitive overload with the three different platforms in Cospaces Edu (Regular 3D, 360o, & MERGE cube mode). Introduce the three different platforms, but get them to design and create mainly in one of those platforms. The AI education component of the curriculum seemed to overshadow the mathematics teaching and learning

^aCospaces Edu is an online virtual program that enables students to design their own virtual content, create animations with code, and explore their digital creations with AR and VR technologies. https://cospaces.io/edu/

^bMERGE cube is a physical foam cube that, through its outer design, allows various MERGE Cube apps to interact with the cube, projecting realistic graphics and 3D content onto the cube’s surface (i.e., holographic projection). Students can create AR, 3D environment, in Cospaces Edu and interact with/view it on the MERGE cube. https://mergeedu.com/

^cSkybox AI is an online platform that allows students to create their own 360^o virtual environment via text prompts in a chat box. This environment can be downloaded and imported into Cospaces Edu and viewed using VR goggles. https://skybox.blockadelabs.com/

Drawing from the Ontario Mathematics curriculum (OME, 2020), a STEAM (Science, Technology, Engineering, Arts, and Mathematics) summer outreach camp was planned, implemented, and studied in an out-of-school context. The camp integrated lessons and activities in which mathematics, coding, and other STEAM concepts (disciplinary and interdisciplinary concepts) were specified as learning outcomes. A CRP approach to teaching and learning mathematics was used for cultural diversity, equity, inclusion, ethnomathematics (Ascher & Ascher, 1986), critical aspects, and social justice. The AR and VR lessons/activities were integrated into Years 2, 3, and 4 of the designed learning and research activities.

Participants, Research Sites, and Digital Tools

The STEAM camps were publicized on social media (X formerly Twitter and Facebook) and in the University’s newsletter in the local Southwestern Ontario community where this study took place. In line with the funding for the project, the camp was intended for students whose parents identified as Black, Indigenous, and People of Color (BIPOC) individuals. A counterpart camp was publicized and offered to all the other students in Years 2, 3, and 4 of the study. From 2021 to 2024, a total of 91 elementary students attended the camp. During the camp registration, 76% of student participants indicated that they had learned coding skills or had varying degrees (either intermediate or advanced) of prior coding experience before attending the camp. Among these participants, 65 out of 92 elementary students and their parents consented to participate in the study. The student participants were labelled using codes and the year of the study (e.g., P01, Year 1). The demographic breakdown of the participants is as follows: 1 student in Grade three, 20 in Grade four, 12 in Grade five, 16 in Grade six, 9 in Grade seven, 6 in Grade eight, and 1 not indicated. There were ten adult participants, also part of the research team over time, who consented to participate: six instructors and four observers.

In 2021, the camp was conducted virtually over 3 days (via Zoom), with each day dedicated to a specific digital tool and a curriculum integrating coding, CT, and global competencies. Subsequently, in 2022, 2023, and 2024, the camp shifted to an in-person format, hosted at an inner-city urban community center designed to offer multiple learning and recreational experiences to community members of various ages.

In the in-person format, students had the opportunity to experiment with different AR and VR-integrated activities during the fourth and fifth days of the camp. In the earlier days, they learned and used coding and design technologies to obtain skills and knowledge of coding animations, programming robots, designing scenarios, creating 3D digital designs, mathematical modelling, crafting with physical materials, and remixing and reimagining their design. Students explored and experimented with various premade AR and VR technologies, such as MERGE cube, AR Books, and VR goggles. Students designed 3D objects and digital animations using block-based coding in Cospaces Edu and interacted with these animated characters using handheld personal devices (e.g., smartphone or tablet) and the premade AR and VR technologies.

Data Collection and Analysis

The researchers conducted questionnaires, interviews, and observations. They also took photographs of the students’ projects. In the first iteration of the analysis, the researchers used pre-existing themes identified in the literature on AR and VR technologies (Hubert, 2014). These themes included six dimensions of the CRMT tool and AR and VR technologies enhancing students’ visual and spatial reasoning. The themes were used further to analyze the data. In the second iteration, the researchers found emerging themes from the data and used both existing and emergent themes to re-analyze the data (Hubert, 2014; Parker et al., 2017). After the second round of coding, the researchers were able to establish relationships that emerged from the codes, which resulted in categories and subcategories being created (Parker et al., 2017). After reflecting on the findings, it was evident that the research questions needed to be modified to represent one of the main emerging themes: student learning through digital storytelling.

The researchers triangulated data from multiple sources (e.g., written questionnaires, observational notes and reflections, video/audio recorded interviews, photographs of students’ individual and group work, and projects) to understand thickly the preliminary results, to corroborate research findings, and to ensure the study’s trustworthiness (Arthur et al., 2012). From the questionnaires, interviews, and observation data, the researchers identified keywords, codes, and themes that emerged through the CRMT analytical lens. Overarching themes emerged from the data, such as student motivation and engagement, communication and discourse through digital storytelling, and students’ funds of knowledge, cultures, backgrounds, and interests.

Results

In this section, the researchers present the results of the study analyzed with Aguirre and del Rosario Zavala’s (2013) CRMT tool and Borba’s (2012, 2021) conception of humans-with-technology and humans-with-media. The researchers also explored the data through an analytical lens of humans-with-technology and humans-with-media within the dimensions of the CRMT tool, specifically the interaction between human (students) and non-human (technology/digital tools/media/cultural artifacts) actors in the AR and VR activities. In this study, students experienced spatial understanding through the affordance by VR activities of immersion, whereas AR activities helped make abstract concepts tangible and fostered engagement through interactive overlays. Further, the researchers modelled different levels used in Cospaces Edu (Regular 3D, 360^o, and MERGE cube mode) with the aim of increasing students’ level of engagement (e.g., using interactive apps and tools such as Skybox AI, MERGE cube, and VR goggles) and expanding their development of skills (e.g., coding, digital literacy, spatial reasoning, mathematical modelling, and communication).

Intellectual Support

First, as a class, students discussed the differences between AR and VR technologies. Then, students explored and experimented with different coding, design, and digital reality materials and apps, such as the augmented tangible materials—the MERGE cube, the VR goggles, and AR books (i.e., premade technologies). Next, they created their own augmented reality in Cospaces Edu (i.e., scaffolded instruction from premade to creating their own AR). Additionally, students designed a character or virtual robot to be simulated in an AR setting for the iRobot book, and they controlled and manipulated (e.g., increased/decreased the scale) the characters (e.g., dinosaurs) in the AR setting while using the Jurassic World book. They also had to put into practice their problem-solving skills to figure out how the technology worked and how to move the iRobot or dinosaur character so that it followed a linear path in a specific direction. Through this activity, the students learned about scale factors, ratios, and the relative position of objects as they moved the iRobot or dinosaur closer or further away in proximity using the touch screen device (i.e., smartphone or tablet).

Depth of Student Knowledge and Understanding

When designing the 3D objects in Cospaces Edu, students applied mathematical thinking and considered mathematical concepts, specifically movements, measurements, and rates such as clockwise and counter-clockwise rotations, angles, circumference, radius, distance, and speed when assembling or writing code for animating characters. These concepts manifest through the manipulation of 3D shapes, coding for object control, and exploration of various angular orientations within the virtual environment. For example, one student coded the two characters (rooster and mouse) in their story to rotate in a circular motion to simulate a chase, one animated character running after another (see Fig. 2). The student explained, “there’s a rooster and a mouse, running after each other and fighting” (P31, Year 2). To code this animation, the student needed to understand that a repeated turn by 360° with a displacement of a radius of 1 m per second per turn would create navigation along a circular path (see Fig. 2).

Fig. 2.

A student created a simple sequence of events using repetition (a forever loop) to code the rooster and mouse to move around the circumference of the circle (rotated clockwise 360° followed by a displacement measured by a time-lapse of 1 s)

The student demonstrated their understanding of circles, radii, and clockwise rotations and movements by writing code for the characters in their story to simulate the rooster chasing after the mouse as they travelled through space as depicted with the sun, moon, and planets on the MERGE cube (see Fig. 2). The sequence of the code was a key factor in determining which character comes first and which one follows (i.e., the rooster was first and then the mouse second: the rooster was running after the mouse and not the other way around). Similarly, the order of events in their digital story matched (paralleled) the sequences of the coding blocks. Because AR (MERGE cube) and VR (Skybox AI) simulations are 3D in nature, the students had to rotate the characters in the x, y, and z axes or planes. When creating a 3D model of the MERGE cube, there were some digital constraints related to the story illustration. The nature and magnitudes of the movements had to consider the dimensions, measurement, shape, surface area, and volume so that the animated characters fit within the dimensions/constraints of the cube. As such, students were speaking about the measurements they chose to ensure that the characters were not seen to run off the edges or lift off the surface of the plane of the cube.

Subject Analysis

As a class, students discussed the question: In what ways is video game designing related to mathematics constructions? Students specifically examined how different drawings, graphs on the x–y plane, and the patterns and rules for their relations—linear, circular, and curved (e.g., exploring linear, quadratic, and exponential functions taught in the grades 8 to 11 mathematics curriculum)—are used when writing the code for their animated characters, so that they could see the characters move across the screen or a physical cube. Some students used several digital tools all at once, such as a hand-held connected device (i.e., smartphone or tablet) and a physical object (i.e., MERGE cube), with a potential for augmentation through a wireless connection using an app (i.e., Cospaces Edu). Some students created a simulation of characters within a story on a screen, while others created an augmented reality that they could interact with using the MERGE cube and handheld device.

Students were asked to reflect on their learning in different subject areas. A student explained, “I learned that coding, math, animation, and 3D are all types of math that can lead to different activities like games and programming robots” (P38, Year 3), “When you code it is like real life, I learned you can use art to code, I feel good” (P60, Year 4), “To program games you need coding and coding comes from math” (P54, Year 3). Students said the tools and process of animating and simulating inspired them to create and animate game-like characters, some of which they felt embedded in the game as one of the characters. For example, one student said, “I am at the top right here and I am the king of the monkeys and dragons, and they are all dancing” (P28, Year 2).

Students used mathematical concepts when programming their animations, such as “rotation and geometry during creating in Cospaces” (P26, Year 2) and “angles” (P31, Year 2). Students also had to consider the distance and relative position of the animated character from the cube, so that it was easier to locate the virtual image on the smartphone or tablet screen (i.e., projection of their visual story on the MERGE cube). Other students created code that incorporated abstract concepts, such as the x, y, and z axes in which their characters would rotate three-dimensionally on top of the x–y axis and lift off in the x–y–z space or plane (see Fig. 3). In Fig. 3, the student coded one character, Raccoon 1, to rotate at a speed of 4.5 m forward in a time-lapse of 1 s (4.5 m/s) around the x, y, and z axes compared with the other character, Racoon 2, which took 2 s to rotate around the axes. In the sequence of code, the raccoons followed a linear path (4.5 m forward in 1 s) and a non-linear path (turn 90° clockwise around the x, y, and z axes starting at the coordinate x = 90, y = 0, and z = 1). Students were able to use mathematical thinking (e.g., geometric and spatial reasoning) and modelling (e.g., representing movement and functions in the code such as linear, exponential, and quadratic paths that the animated character followed) to understand abstract mathematical concepts better when coding their animated characters in Cospaces Edu.

Fig. 3.

A student assembled blocks of code for two characters, raccoons, to move at a 4.5 m/s speed and to rotate clockwise around the x, y, and z axes

In the STEAM camp, students learned about both coding and technology in the context of animations of 3D objects on an x–y–z plane. One student said, “I learned how to animate stuff in Cospaces. I could do my own MERGE cube in Cospaces [and] I felt good through the process” (P26, Year 2). Similarly, another student expressed, “when I was creating an animation in Cospaces, I used coding to allow my characters to move around” (P39, Year 3). Another student explained, “I learned how to add codes to move my character like how to make it speak and animate it to do moves” (P47, Year 3). Some students made the connection between mathematics and coding, e.g., “I learned that coding, math, animation and 3D are all types of math that can lead to different activities like games and programming robots” (P38, Year 3).

Communication and Discourse Through Digital Storytelling

Beyond the curriculum content, digital storytelling appeared to be integrated into the design and animation of the 3D environment in Cospaces Edu. One student reflected, “My favorite time about it was creating my story in a 3D link called Cospaces. It’s sooooo fun!!!!!” (P66, Year 4). Another student explained further, “I learned how to add codes to move my character like how to make it speak and animate it to do moves and I felt good through the process, and it was fun” (P47, Year 3). One student said, “I learned how to animate [code] stuff in Cospaces. I could do my own MERGE cube in Cospaces [and] I felt good through the process” (P26, Year 2).

For example, in one design scenario, a student crafted a narrative focused on the rescue of baby animals (a baby pig and a baby sheep) who were being pursued by enigmatic creatures in a dense forest. To create a sense of urgency, the student coded these screen-based characters so they could, in the simulation, cover 30 m in 5 s, ensuring their swift movement in unison (toward safety in this scenario). The student had to consider mathematical concepts such as speed, direction, position on the coordinate plane, linear, and non-linear movements, angle of rotation, and radius when animating the characters in this story. Another student created a story around a common fantasy animal and a magical unicorn, using her vernacular language (slang and colloquial words and phrases) to express the animated characters’ ideas and thoughts (represented in words coded for the character to say). The student wove in verbal expressions composed of non-formal terms and phrases (see Fig. 4), with slang such as “cool,” “rudee,” and “bro you have to know you are human and you can think,” “If you think you’re so cool. What’s the magical spell to turn the unicorn blue?”.

Fig. 4.

A student created a dream world in Cospaces Edu with fantasy animals such as unicorns and assembled appearance coding blocks to show subtitles of a dialogue between characters in the simulation that tell a story

One instructor noted that most “students seemed to interact with each other more during the AR and VR centers, they formed small groups with different students who they had not previously worked with throughout the [STEAM] camp and explored the tools together” (Instructor 1, Year 2). Students who felt more comfortable with the technology supported others who were less familiar with using the digital tools and software. One instructor explained, “some students, who figured out the features of the programs/apps/tools earlier aided their peers” (Instructor 2, Year 2). For example, “during the Cospaces activity, one student had problems with how to move his 3D model, and his friend saw [him struggling] and showed him how to do it” (Observer 1, Year 3). Another student explained to his peers how to add aesthetic features, such as music and sound effects, that could play in the background as the animated characters moved. Students were proud of their work and wanted to share it with others. One researcher noted that “each student was positive and confident in presenting all of their work to their parents [… Similarly, another] student showed his Cospaces work and his MERGE cube work to all the instructors after he finished them” (Observer 1, Year 3).

In the final iteration of the camp (Year 4), students were asked to develop an AI prototype and to design a detailed story that explored the interaction between the human and the non-human AI robot. Students were asked to sketch their AI prototype (see Fig. 5), create a digital prototype of their model, and then import the 3D model into Cospaces Edu (see Fig. 6). When students designed their AI prototypes, they had to think about mathematics concepts such as machine learning (i.e., visual-spatial reasoning and image recognition built into the sensors).

Fig. 5.

A student developed a prototype for a Chef Bot that would sense when her dad was tired and help her father

Fig. 6.

A student created her digital story in Cospaces Edu to illustrate some of the potential issues that may occur with the Chef Bot (AI prototype in Fig. 5) in a real-world scenario when cooking in the kitchen at her father’s restaurant

Student Sensorial and Gesture Engagement

Overall, students appeared to be more engaged during the AR and VR integrated activities compared with earlier lessons, which were solely screen-based and did not include either digital or physical tangibles. One student expressed, “I learned how to animate 3D stuff. It made me happy. […] I like doing Cospaces. Because it allows you to create virtual worlds and stuff” (P26, Year 2). Similarly, another student said, “I was excited to try the AR/VR and excited to make a mini world” (P48, Year 3), and another commented, “I was excited [… about] Cospaces because I find it really fun and cool to create your own codes” (P47, Year 2). One researcher reflected when “compared to previous days [of the camp,] they were more engaged in the activities. They completed multiple activities on the same day” (Observer 1, Year 3). One instructor noted that the “students who engaged with the AR books, hologram projector, and MERGE cubes were highly engaged, asking questions, and enjoyed the activities” (Instructor 4, Year 3).

Further, the researchers observed how the students became excited about creating and simulating their own augmented reality, AR, in Cospaces Edu, and interacting with it using the MERGE cube. One instructor elaborated on why the students felt this way:

For the most part […] they’re very engaged, like, they’re very interested in the stuff they were doing, especially when they knew that they got to go into the computer room and start building stuff just out of their imagination. Like, they were very excited about that (Instructor 4, Year 3).

The students also used their senses (sight, sound, and touch) to query and interact with the AR and VR tools. Using the Cospaces Edu app on handheld devices (smartphone or tablet), the students were able to interact with the digital animation that was projected on the MERGE cube (see Fig. 7) and VR goggles (see Fig. 8). Students could physically rotate the MERGE cube in their hand and look at their animated characters from different perspectives on the cube’s surface (i.e., top, front, and side views). They could also increase or decrease the scale of the 3D objects projected on the MERGE cube depending on how close or far away the MERGE cube was from the handheld device. Students were also able to increase or decrease the scale by using their fingers and moving them on the surface of the tablet.

Fig. 7.

Students were able to interact physically when simulating digital animation which they had designed and assembled code for, using a handheld tablet or smartphone, the Cospaces app, and the MERGE cube

Fig. 8.

Students viewed and virtually immersed themselves in the virtual world they created using VR goggles: They explored the VR environment through body movements to navigate through and see the 360^o environment from different angles and perspectives

Similarly, students could explore the 360^o virtual environment they created kinaesthetically through movement (moving their head, neck, and body) and the VR goggles (see Fig. 8). Students appeared to embody mathematical concepts through their movements, such as angles, rotations, and translations in the x, y, and z axes. Several students decided to add sound (i.e., music or sound effects) and gestures when writing code such as animating their characters to speak. Several students also added actions when writing code for animating their characters to dance, jump, and run following a geometrical path such as a circle. For example, one student programmed the MERGE cube to play cricket noises in the background of the 3D campfire scene they had created.

Students’ Funds of Knowledge, Cultures, Backgrounds, and Interests

In some instances, students appeared to make a connection between their culture and the mathematics taught in the lessons/activities:

When you think about math, and how it relates to art, like some of the artwork, or some of the architecture was in mosques, a lot of the students were able to kind of see that and relate because they go to their mosques every week. And so, they’re able to recognize that there is math within our world. [… and making] those connections […] being able to believe that math can be intertwined with their culture and like they’re not two completely separate things (Instructor 4, Year 3).

The instructors reflected on how students could make other connections to their culture and explore their own identity. One instructor expressed, “they were able to think about culture and other people’s cultures and recognise that culture does play a really big part in people’s lives. And stories reflect people’s cultures […] and how it kind of forms their own personal identity” (Instructor 4, Year 3). Similarly, another instructor noted, “The students were able to […] reflect on their culture, and their identity and their likes and their dislikes, and how they saw themselves […] bringing in like different symbols that were representative of their [culture]” (Instructor 3, Years 2 and 3).

Specifically, as students designed in Cospaces Edu, they incorporated images, symbols, and cultural monuments from their family’s culture and background, as well as historical buildings and places of interest. A student commented, “I saw a mosque which really reminded me of my culture [beliefs] because I am Muslim and Muslims pray in the mosque” (P22, Year 2) and another said, “[I chose] Dubai because it is my culture [country of origin]” (P36, Year 3). Others wanted to explore historical and architectural monuments and places of interest, such as “castles in Portugal, and mosques in India” (P44, Year 3). Additionally, another student expressed that she wanted “to explore [new countries, traditions, and cultures] something that [her] family is not connected to at all, China” (P37, Year 3). The students appeared to realize what they envisioned as they created and animated their MERGE cube in Cospaces Edu. To do that, they had to integrate some mathematical aspects, such as lines, measurements, symmetry, area, and volume into the design of the monuments and manipulate the 3D objects by increasing/decreasing the scale and rotating it three-dimensionally, among other transformations.

The students appeared to incorporate seamlessly the mathematics and cultural aspects of their digital creations as they shared their projects with the instructors/researchers. For example, one student described her digital creation in Cospaces Edu and the mathematical concepts represented in that creation:

The plane takes a full spin around the earth. It stops at different places such as New York, Rome and Dubai. The angle is 360 because it takes a full circle [to rotate around the MERGE cube]. The low number [i.e., radius] makes it faster. For example, if it’s one, the plane moves super-fast [as seen in Fig. 9] (P54, Year 3).

Fig. 9.

Students incorporated cultural monuments, such as the Taj Mahal, Eiffel Tower, and Colosseum, as well as mathematical concepts, such as rotation, angles, circles, radius, and speed, into the design of scenarios to project on the MERGE cube

The instructors found that “some students [succeeded in incorporating their culture and background into their digital designs while others] struggled to make those connections […] so then some of them would like, struggle to go more in-depth,” during the cultural component of the activity (Instructor 3, Years 2 and 3). This made it difficult for timing because, “some students would put so much time and effort into it, and then others maybe, like finished more quickly” (Instructor 3, Years 2 and 3). To address this issue, students were able to create multiple digital worlds in Cospaces Edu and explore them with AR or VR using a smartphone, tablet, and/or goggles.

Discussion

In this section, the researchers discuss the research findings, aiming for insights into the implications of, and connections within the context of, teaching mathematics concepts through AR and VR activities for a diverse student population. In this section, the researchers will also highlight some of the dimensions of the CRMT tool (student engagement, depth of student knowledge and understanding, communication and discourse, and students’ funds of knowledge), as well as the interaction between human and non-human actors (humans-with-technology and human-with-media).

Student Motivation and Engagement

Students were drawn to and interested in creating their own digital animations in Cospaces Edu. Their motivation and engagement appeared to be enhanced by the interaction with the 3D and game-like environment and the opportunity to create their own augmented reality (Schutera et al., 2021). This is echoed by Ardiny and Khanmirza’s (2018) findings that “AR/VR systems could present educational content in attractive ways and enhance students’ motivation and interest [… and] engage them more in learning processes” (pp. 485–486). For instance, students’ motivation and engagement appeared to be enhanced with the AR and VR activities as they designed their digital stories in Cospaces Edu. In order to engage students on multiple levels, three different platforms in Cospaces Edu were used: regular 3D mode (screen-based), 360^o mode (VR goggles and smartphone), and MERGE cube mode (smartphone or tablet and MERGE cube). This provided students with the opportunity to engage with both the physical and digital tools simultaneously through these tangible user interfaces.

Technological innovations such as these have the potential to change the way students learn and experience mathematics (Borba, 2021). Borba and Villarreal (2005) stated that “humans are constituted by technologies that transform and modify their reasoning and, at the same time, these humans are constantly transforming these technologies” (p. 22). As the students interacted with the MERGE cube, smartphone, or tablet, and Cospaces Edu app, they had multiple opportunities to engage with the AR and VR tools using their senses (i.e., sight, sound, and touch) to understand better mathematical concepts, as they manipulated the conditions of (e.g., changed the scale, angle, and distance) and interacted with the 3D virtual environment. The AR and VR integrated activities appeared to motivate and engage students more in the learning process and provided opportunities for them to experiment with and model their mathematical thinking while interacting with the MERGE cube and animating their characters in Cospaces Edu.

On the other hand, “Virtual reality has its limitations. Despite its ability to provide [… an] immersive experience, it blocks users’ interaction with the surroundings” (Abdul Hanid et al., 2022, p. 9487). The researchers found this to be true in that, while it was exciting for students to view the 360^o virtual world through the VR goggles, they did not spend as much time interacting with the VR tools, because they were not able to manipulate or interact directly with the 3D objects and animated characters. Although this seems to be a limitation of the VR technologies, students appeared to be highly engaged when they were designing their digital story in the 360^o platform and creating their own virtual world using Skybox AI.

Visual–Spatial Reasoning and Mathematical Learning

AR and VR technologies also appeared to enhance students’ visual and spatial reasoning skills. This visual and spatial reasoning can help students understand a complex mathematical concept by allowing them to visualize and manipulate a 3D model in a virtual environment and “experience abstract concepts more concretely” (Ardiny & Khanmirza, 2018, p. 485). This sentiment is echoed by Borba’s (2012) findings that digital tools can be used as a catalyst to promote collaboration and exploration of “more complex mathematical ideas” (p. 804). This is in line with our findings that students were able to explore more complex mathematical concepts through AR and VR, such as transformations involving linear (translations) and non-linear movements (rotations), scale factors, functions (linear, quadratic, and exponential), limits (volume and surface area), and 3D rotations around the x, y, and z axes. Through observations, it seemed that AR technologies lead to more “engaging and immersive interactions” (Angraini et al., 2023, p. 1035), as the students manipulated the scale factors and relative position of the virtual objects by zooming in and out using their fingers on the touch screen device and rotating the MERGE cube itself (see Fig. 7). Thus, they enhanced their visual–spatial reasoning.

Communication, Discourse, and Peer-Assisted Learning

The incorporation of AR and VR technologies in the learning environment provided many opportunities for student interactions with each other and with the technologies. Students seemed to interact more actively during all AR and VR activities. A heightened level of communication and collaboration was notable particularly when students were engaged in animating their 3D environments in Cospaces Edu. According to Buentello-Montoya et al. (2021), the incorporation of AR and VR technologies provides many opportunities for student interactions such as “allowing exchanges of ideas, teamwork, and collaboration” (p. 6).

An intriguing discovery emerged as students who quickly grasped the features of AR and VR programs and tools voluntarily assumed mentoring roles. These students willingly offered assistance to peers who encountered challenges with technology navigation or required additional support. Chen (2008) also indicated that students believed that collaborating with peers would enhance their understanding of the content, even though it might require increased effort on their part. This phenomenon of peer-assisted learning highlights the collaborative potential inherent in immersive digital environments within an educational context. It showcases how students can leverage their digital literacy and interests to support one another and foster a more inclusive learning atmosphere.

Digital Storytelling in the Mathematics Classroom

Beyond enhancing the overall engagement and learning experience, the AR and VR activities also facilitated communication and discourse among students as they collaboratively developed and shared their digital narratives with others. Borba (2012) discussed the power of discourse among students when “sharing their ideas in multimedia environments [through images, words, and sounds this] multimodal discourse becomes a new means of expression [… and new avenue] in the production of knowledge” (p. 803). Further, digital storytelling humanizes technology as students explore their culture, background, and interests through personal and relational lived experiences (Hill & Grinnell, 2014). For example, one student created a narration about a Chef bot (AI robot), sensing her father’s energy levels drop when he is tired and responding to this need by assisting in the kitchen at her father’s restaurant (see Figs. 5 and 6). This student was invested in this narrative when she explored possible solutions to reduce her father’s workload and how the Chef bot could lighten his load and alleviate some of the pressures of running a restaurant.

Digital storytelling can be perceived as an educational strategy that significantly boosts expressive and communication skills as well as enhances digital competencies (Del-Moral-Pérez et al., 2019). In this study, the researchers found that digital storytelling provided students with a conducive learning environment for developing their narrative, literacy, and other skills (e.g., critical thinking, creativity, problem solving) when supported by a range of AR and VR technologies. This is echoed by Quah and Ng’s (2022) results that digital storytelling promotes literacy skills, information, and communication technology literacy, social and cultural awareness, critical thinking and problem solving, and communication.

In this study, the researchers found that digital storytelling not only supports a deeper understanding of mathematical concepts by representing those ideas in a story but also encourages active student participation by enriching the overall learning experience. This is in line with Sum et al.’s (2023) conclusions that the value of a well-structured storytelling activity can increase student performance and provide support and “rich linguistic and non-linguistic resources for deepening their understanding” (p. 646) in mathematics education. Furthermore, it serves as a “pedagogical bridge between home and school contexts” (p. 648).

Students’ Cultures, Backgrounds, and Interests

When the researchers asked students about cultural connections with respect to the AR and VR activities, students highlighted the religious significance of the cultural monuments and reflected on their own culture, background, and interests. One student expressed, “I saw a mosque [in the cultural and historical monuments] which really reminded me of my culture” and another explained that he chose “Dubai because it is my culture.” During the AR and VR activities, most students appeared to associate culture simply with their faith-based beliefs or country of origin, while other students incorporated symbols and images that represented their customs and practices. Culture is complex and represents customs (not just beliefs, but may include ceremonies, rituals, myths, legends, traditions, celebrations, foods, symbols, and images), social institutions (transcend beyond a country, such as political, educational, economic, family, and religious institutions), and achievements of a particular group of people (Birukou et al., 2013). The connection students made to their culture and background through their beliefs and country of origin may be the first step to a more in-depth exploration of their own culture (self-identity) and others. A “tangible user interface [such as AR and VR] promotes collaborative creation of multicultural and cross-cultural stories, [… and may result] in children developing creativity and social and cultural awareness” (as cited in Quah & Ng, 2022, p. 855; Sylla et al., 2019). Several students mentioned personal interests, such as “castles in Portugal and mosques in India,” revealing their curiosity and exploratory nature. Additionally, another individual expressed a desire to venture beyond their own culture and background, stating she wished to “explore something that […] is not connected to [her] at all.” The above responses underscore the diverse interests of the students and their motivations to explore cultural and personal experiences through AR and VR technologies.

According to Aguirre and del Rosaria Zavala (2013), “CRMT tool provides some specific guideposts to meet the diverse mathematics learning needs of their students” (p. 178) by integrating mathematical thinking, language, and culture into the curriculum. In some cases, AR and VR activities appeared to offer rich opportunities for individuals from various cultural backgrounds to explore images or artifacts of different countries, cultures, and personal interests in the context of mathematics education. When students were asked about the mathematical representations in their culture and background, “some students struggled to make those connections [in the AR and VR activities between culture and mathematics] more than others” (Instructor 3, Years 2 and 3). This is in line with Rousseau Anderson’s (2021) findings that it can be difficult to make a connection between the student’s lived experiences and the mathematics knowledge being taught. Some students were able to see the mathematics in the “architecture in mosques […] And so, they’re able to recognise that there is math within our world [… and make] those connections” (Instructor 4, Year 3).

Conclusion

The integration of AR and VR technologies into educational settings has garnered significant attention, driven by the potential to enrich learning experiences and foster student engagement. Our study highlights the potential of digital storytelling and coding through AR and VR technologies as effective tools for students to articulate their cultural identity, background, and interests. These tools also concurrently immerse students in a learning environment where they can use mathematics, coding, CT, and other STEM concepts. These findings underscore the promising outcomes of employing AR and VR in teaching both simple and complex, mathematical topics, including abstract concepts (e.g., rotation around the x, y, and z axes or planes, scale factor, maximizing and minimizing volume). Notably, AR and VR activities show promise in enhancing students’ visual and spatial reasoning skills, as evidenced by our findings.

Beyond its impact on individual learning, the integration of AR and VR technologies fosters a collaborative and inclusive educational environment. The results of our study reveal that AR and VR technologies facilitate increased communication, discourse, and peer-assisted learning, promoting collaboration and inclusivity. Furthermore, our study sheds light on the application of digital storytelling in mathematics education, offering a means to provide a deeper and more meaningful learning experience. Further, AR and VR tools have the potential to “foster abstraction of concepts in a palpable way [… through] student interaction, […] exchanges of ideas, teamwork, and collaboration” (Buentello-Montoya et al., 2021, p. 6). Additionally, our study emphasizes the advantages of tailoring AR and VR activities to students’ cultural backgrounds and interests, thereby providing opportunities for exploring diverse cultures and encouraging personal discovery (self-identity).

While these immersive technologies offer unique affordances and opportunities, they also pose challenges such as technical issues, hardware limitations, and navigation challenges (e.g., using the touch screen device and accessing the app). Addressing these challenges is crucial to ensure an uninterrupted learning experience with AR and VR technologies. Looking ahead, several avenues for future research on AR and VR in schools and mathematics classrooms emerge. Our research team aims to investigate further the training of pre-service and in-service teachers, focusing on the integration of AR and VR technologies into the mathematics classroom and curriculum. These initiatives are driven by the crucial need for adequately trained educators equipped with the requisite knowledge and tools to leverage fully the potential of these immersive technologies, such as AR and VR tools, within the realm of mathematics education.

Author Contribution

The first author, Marja Gabrielle Bertrand, contributed solely to the conceptualization, visualization, and writing —original draft preparation. The first and second authors, Marja Gabrielle Bertrand and Hatice Beyza Sezer, contributed equally to the following tasks: methodology, investigation, data curation, and formal analysis. All three authors, Marja Gabrielle Bertrand, Hatice Beyza Sezer and Immaculate Kizito Namukasa, contributed to the validation of the results, writing, reviewing, editing, and resources used in this study. Finally, the supervision, project administration, and funding acquisition were done by the third and final author Immaculate Kizito Namukasa.

Funding

This work has been funded by the following agency: The Social Sciences and Humanities Research Council (SSHRC), Canada.

Data Availability

No datasets were generated or analysed during the current study.

Declarations

Competing interests

The authors declare no competing interests.