Integrating augmented reality into inquiry-based learning approach in primary science classrooms

Abstract

Notwithstanding the advantages of incorporating Augmented Reality (AR) in education, AR’s concrete uses as compared to other technologies are not fully recognised. Moreover, many of the existing studies have neglected to examine the impact of pedagogy and its corresponding instructional models, whilst implementing AR in teaching and learning. In leveraging the affordances of AR, an inquiry-based learning framework, referred to as QIMS, was proposed in this study. A learning package was developed on the topic of plant reproduction for primary 5 students (aged 11–12) based on the QIMS framework. Using a quasi-experimental approach, this study evaluated three conditions (AR and QIMS; QIMS; Non-AR and Non-QIMS) for a series of science lessons in a primary school. 117 students took part in this study. The quantitative results showed that although there was no statistically significant difference in students’ academic performance when AR was used, students’ self-directed learning and creative thinking skills increased significantly after partaking in the QIMS inquiry-based lessons. The usage of AR and QIMS had a significant effect in increasing students’ critical thinking and knowledge creation efficacy skills. Moreover, in view of students’ academic outcomes, the integration of QIMS and AR proved to be more beneficial to low-progress students. Qualitative analysis of the interview data from teachers and students aids in accounting for the quantitative results and indicate productive implementation strategies. The findings of this study will guide the design of future AR interventions, by providing insights for both researchers and practitioners on how to integrate and implement AR with pedagogical approaches.

Introduction

Augmented reality (AR), as an emerging technology, overlays augmented virtual objects onto the user’s physical environment (Akçayır & Akçayır, 2017). AR adds to reality and assimilates virtual information in an authentic environment where information is processed and created in real time and helps to provide learners new learning experiences that are more authentic, interactive, and engaging (Altinpulluk, 2019; Klopfer & Yoon, 2004). Moreover, AR applications can reinvent teaching practices, which appeal to the increasing attention on pedagogical innovation. In terms of pedagogical strategies, inquiry-based learning (IBL) is one of the most common pedagogies in AR interventions (Garzón, et al., 2020; Wen & Looi, 2019). AR-enabled inquiry activities could engage learners in an immersive context that enhances scientific investigations—one where students can collect data outside the classroom, interact with an avatar, or communicate face-to-face with peers in a more authentic setting (Dunleavy et al., 2009).

Despite the potential advantages of using AR in education and considering its suitability for supporting science learning through simulation activities, AR’s concrete users are not as fully recognised as compared to that of other technologies (Joseph & Uther, 2009; Wen, 2021). Many AR studies have reported on the positive effects on learners’ motivation and learning effectiveness but have overlooked the importance of pedagogical strategies in implementing AR in teaching and learning (Garzón, et al., 2020). Additionally, as Sanabria and Arámburo-Lizárraga (2017) stated, AR has unquestionable strengths in promoting learners’ twenty-first century learning skills, such as collaborative learning, critical thinking, or creative thinking involved in STEAM studies, but the applications should be governed by pedagogical strategies for promoting these soft skills. To respond to this need, this study investigates how the IBL approach can be integrated with AR-supported formal science learning, and if and how the integration can help to improve learning outcomes by considering students’ academic performance and twenty-first century learning skills.

Literature review

AR in teaching and learning

AR offers tremendous possibilities for students in facilitating sensory immersion, navigation, and manipulation, and have been reported to promote positive emotions while learning and may create more efficient and better learning outcomes (Cheng & Tsai, 2013; Wu et al., 2013). With proven capabilities to engage students in learning activities, AR can foster collaboration, as well as critical thinking, problem-solving, and self-directed learning skills, which are notably 21st Century learning skills (Adams Becker et al., 2017). Unlike Virtual Reality, AR is not limited to a particular device, and this makes its applications in schools more flexible. The combination of mobile AR technology and inquiry activities has been shown to be effective in promoting students’ understanding of science content (Chiang et al., 2014; Nielsen et al., 2016). In social sciences, fostering historical reasoning through the integration of AR-based inquiry-supported elements has been evidenced to improve learning as well (Efstathiou et al., 2018). AR enables students to observe and experience comparisons between the past and the present, as it helps to immerse students into the past and the present by blending real-life settings with virtual information (Chang et al., 2015). Even so, the application of the essence of AR is yet to be fully understood compared to other forms of technologies (Garzón et al., 2020; Joseph & Uther, 2009).

Inquiry-based learning

Inquiry-based learning (IBL), considered a prominent form of active learning in education, is a learning approach in which students act and think like scientists, in the pursuit of constructing knowledge (Keselman, 2003). This is particularly important for the science discipline, as inquiry is a key component of acquiring science concepts, especially for deeper reasoning and scientific thinking (Liu et al., 2021). In the inquiry process, students pose questions, gather information, conduct investigations, and test hypotheses, on the grounds of uncovering evidence that reveals new learning discoveries (Liu et al., 2021; Pedaste et al., 2015; Markant et al., 2016). IBL is also essential in supporting students to improve twenty-first century learning skills (Chu et al., 2017).

A number of studies have highlighted that the value of IBL lies in guided inquiry rather than unstructured inquiry or more didactic teaching methods (Kyza & Georgiou, 2018; Lazonder & Harmsen, 2016). For instance, in a meta-analysis that compared IBL to direct instruction and unstructured student-led activities, Furtak et al. (2012) reported outcomes of better science learning in favour of the inquiry approach. Similarly, in a meta-analysis by Lazonder and Harmsen (2016), the authors concluded that whilst there should be sufficient guidance during IBL, excessive expository forms of instruction will hamper the inquiry process. Indeed, for IBL to make a key impact on learning, its design must be meaningful and should help to establish learning opportunities that can relate to theories or complex concepts (Gómez & Suárez, 2020). There is consensus that IBL should have sufficient scaffolding and guidance for it to be successful (e.g., Alfieri et al., 2011; De Jong et al., 2014; Furtak et al., 2012; Kyza & Georgiou, 2018; Lazonder & Harmsen, 2016). Additionally, the use of technology in IBL should create a productive balance between a learner’s agency and sufficient scaffolding (Suárez et al., 2018).

Design of AR activities integrated with inquiry-based learning

Educators have contended that leveraging AR is promising in scaffolding students in inquiry (Cheng & Tsai, 2013; Efstathiou et al., 2018; Kyza & Georgiou, 2018). The scaffolding ability of AR tools is effective for students to acquire accurate knowledge during periods of exploration and inquiry (Yoon et al., 2017) and allows for the possibility of developing a narrative that promotes IBL that is not easily supported in a typical classroom environment (Squire & Jan, 2007). AR could be an engaging mechanism for initial stimulus material, and when incorporated with inquiry tasks, could be productive in promoting students' mastery of science content (Chang et al., 2013). It is also argued that AR-based inquiry could be a point of entry into Splitter’s (1991) communities of inquiry, by reforming classrooms that can maintain “dimensions of inquiry and wonderment” at the centre of the learner’s daily learning activities (p. 98). A design that demonstrates careful consideration, combined with image augmentation and visualisation of complex science concepts afforded by AR technology, could result in a diverse spectrum of questions that learners can examine and explore.

Furthermore, Nielsen et al. (2016) have emphasised the need to consider how AR can mediate student learning, and its design should be pedagogically meaningful. For instance, it is important to explicitly incorporate features highlighted in inquiry-based science education into the AR activities. Students could then actively construct their own hypotheses before the collecting and investigating of relevant data. When designed purposefully in AR activities, this can then enhance and transform student questions to drive self-directed inquiry (Nielsen et al., 2016). Nevertheless, introducing AR in classrooms are not without its risks (Ibáñez et al., 2016). In an inquiry activity designed with AR, researchers reported that students were so preoccupied with the AR tool that they failed to complete the critical components of the activity (Dunleavy et al., 2009). A similar problem occurred in a study by Ibáñez et al. (2015) where although the students felt motivated while using AR tools, almost 25% of the simulation tasks were unsuccessfully completed, leading to a lower-than-expected learning effectiveness.

Summary

Taken together, the potential of AR in classroom learning has been increasingly acknowledged (Akçayir & Akçayir, 2017; Abdinejad et al., 2021; Gnidovec et al., 2020; Plunkett, 2019). Currently, a significant number of AR-based inquiry learning studies are centred on the achievement of cognitive and motivational learning goals (Pedaste & Jürivete, 2020). These studies evaluate students’ conceptual understanding and knowledge in terms of learning achievement (e.g., Chiang et al., 2014; Chiu et al., 2015; Kyza & Georgiou, 2018; Pedaste & Jürivete, 2020). In comparison, there have been fewer studies focused on analysing the impact of pedagogical approaches, as well as the pedagogical strategies of implementing AR in teaching and learning (Garzón et al., 2020; Pedaste & Jürivete, 2020). There is a need for research to explore how AR-based learning can be integrated with pedagogical approaches to understand the impact of AR in learning outcomes according to students’ characteristics (Ibáñez & Delgado-Kloos, 2018). While investigating learning outcomes, the impact on 21st Century learning skills should also be considered in addition to academic performance (Sanabria & Arámburo-Lizárraga, 2017). Also, it remains unclear whether effectiveness is mainly due to the integration of AR with IBL, or the pedagogical approach itself; or the effect of the combination of diverse factors such as learners’ capability, teachers’ enactment, and the learning environment. To fill these gaps, this quasi-experimental study compared the effectiveness of AR-supported IBL learning environment on students’ academic performance and twenty-first century learning skills and how they responded to the various learning conditions. The findings of the study seek to provide insights for both researchers and practitioners on how to integrate and implement AR with pedagogical approaches.

AR-based inquiry learning design

QIMS inquiry framework

In this study, we conceived a framework, termed QIMS (Questioning, Investigating, Making, and Synthesising) to guide IBL that leveraged AR technologies. The QIMS framework consists of the intertwined processes: Questioning, Investigating, Making, and Synthesising (see Fig. 1). Drawing from the framework, learning packages were designed to actively engage student in the AR-based learning activities: (1) Questioning: students will be guided to pose questions in the subject they are learning about; (2) Investigating: students engage in a concrete experience of the phenomenon through AR simulations and reflect on that experience; (3) Making: students will apply what they have learned in the new context by creating artefacts; (4) Synthesising: by presenting their artefacts, students will explain their conclusions and review their acquired knowledge (Wu & Wen, 2021). This framework is built on IBL and the idea of learner-generated contexts which emphasises the shift of learners from being consumers to being context creators (Luckin et al., 2011). The use of AR technology helps to underscore learner-generated contexts by moving toward the combination of interactions the learner experiences, across multiple physical spaces and time, beyond a physical location (Wen, 2021). Context does not only determine learning as an external source, but it is also created by learners. Taking account of the affordances of AR, the QIMS framework proposes to enhance the use of IBL and operationalise learner-generated contexts.

Fig. 1.

QIMS inquiry framework

Learning packages integrating AR and QIMS

In this study, the QIMS framework formed the basis of the lesson design and instruction of the topic of plant reproduction. In the first lesson, the teacher described a story about some aliens who had travelled to the students’ school because their planet was in trouble. The teacher then explained the problem statement, which was that the aliens needed help with growing plants, as and their plants were almost extinct and unable to thrive. After being presented with the problem, the students were tasked to help the aliens, using their newly acquired knowledge of plants. This background story would be constantly highlighted in the QIMS cycle.

In (1) Questioning: Students were encouraged to ask questions about the alien’s planet and on the topic of plants; (2) Investigating: Students concretised their experience of the phenomena by using AR tools and conducted experiments related to the topic. They also noted down their reflections and completed some worksheets on the subject; (3) Making: Their main task was to create a plant to save the aliens’ planet. They developed hypotheses on why their creations could save the planet by applying the concepts they had learnt; (4) Synthesising: They concluded by presenting their plant creations with supporting explanations on why their plants were feasible.

Designed AR app for the topic of plant reproduction

The Plant Lifecycle AR App was developed for the topic of plant reproduction, built using the Unity3D engine, with AR functionality provided by the Vuforia plugin. The Plant Lifecycle AR app supports three activities: (1) Flower Anatomy, (2) Seed Dispersal, and (3) Plant Engineer. The selection of activities arose out of the needs of the school that we collaborated with.

The core design tenet of the Plant Lifecycle AR app is to leverage real space for learning. For this reason, the app was designed in the context of spatial visualisation. One example of this occurs in the scanning function, which is available in two out of the three activities. This function allows the user to interact with virtual objects by aiming the device at the desired object and waiting for a meter to fill up (typically between one and three seconds). By leveraging the spatial relationship between the device and the virtual object, we aim to engage the learner’s kinaesthetic sense in a way that is not possible without the use of AR.

The Flower Anatomy activity requires learners to examine a virtual flower and “scan” its various parts by aiming the mobile device’s camera at them (Fig. 2a). Upon scanning each part, the learner is presented with a textual description of the part, as well as a “Thinking Time” question for them to answer. They can type in their answers to these questions using the standard iPad onscreen keyboard. Learners are not compelled to answer the questions immediately and can return to them later in the activity. The onscreen display includes a list of “Missions” which is similar to that of many popular games, and thus familiar to the majority of learners (Fig. 2b). This Mission list is updated every time the user makes progress towards one of the objectives on the list. In doing so, students are naturally be inclined to examine all the activity content, but the order in which they encounter the content is under their own control.

Fig. 2.

Flower anatomy activity

The Seed Dispersal activity is a simple simulation of seed dispersal in plants. It presents learners with a miniature virtual island, upon which they can plant seeds of three different fictional plant species. Each species has a different method of dispersal—one by wind, one by water, and the last by explosive action. Learners can speed up and/or slow down the simulation time using an onscreen menu. They can also control the direction and speed of the wind by manipulating two AR markers, one representing the island and the other the wind, to change their relative positions. Finally, learners can scan a particular plant to “track” it. By slowing down time when the plant disperses its seeds, learners can easily observe the dispersal process; by speeding it up, they can see how the plants spread across the landscape over the course of weeks or months (see Fig. 3).

Fig. 3.

Seed dispersal activity

The Plant Engineer activity is an unstructured activity intended as a “capstone” summation of learners’ botanical knowledge. This activity is built on a fictional narrative where two students, representing the learners, have to help aliens to “re-green” their planet, which is an environment with very specific conditions (e.g., low water table; high winds). As such, the capstone activity consists of designing a plant to survive and reproduce in these conditions. To complete the activity, learners are presented with an empty virtual/real space centred on the AR marker; they can then select from an onscreen menu of plant parts and place those parts in the virtual space. Although the parts are categorised according to type (roots, stems, leaves, flowers), there are no restrictions on the positioning or number of the parts; they can be placed on the virtual “ground” (e.g., the marker) or floating in mid-air. The learners could capture screenshot images depicting their model plants together with the plants (see Fig. 4). Thereafter, the students need to present their creations and explain why they choose certain parts.

Fig. 4.

Example of a completed plant model

Methodology

A quasi-experimental design was adopted in this study to investigate if the integration of AR and IBL can help to promote students’ academic performance and soft skills more effectively, and how they should be integrated and enacted. As QIMS framework was proposed to guide IBL that leveraged AR technologies, the following research questions were addressed:

1. What effects did the AR-supported inquiry-based learning activities, as guided by the QIMS framework, have on students’ academic performance, and what was the discrepancy among students with different academic abilities?

2. What effects did the AR-supported inquiry-based learning activities, as guided by QIMS the framework, have on students’ twenty-first century skills, and what was the discrepancy among students with different academic abilities?

3. How did the QIMS framework help to design and enact AR-supported inquiry-based science learning?

Participants

The study was conducted in a Singapore government primary school. In Singapore, reforms focusing on the promulgation of the twenty-first century learning skills and the drive towards lifelong learning have been actively promoted to provide students with holistic education since the 2000s (Tan et al., 2017). A total of 117 5th grade students (10–12 years, 57 boys and 60 girls) from 3 classes and their science teachers participated in the study (see Table 1). To investigate whether the learning efficacy was caused by the pedagogical approach itself or the integration of AR with the pedagogical design, the 3 classes participated in different learning design conditions. Class A used AR apps on iPads and all the learning activities were designed and implemented with the guide of the QIMS framework. Class B also used iPads, but they did not use AR apps, and iPads were only used for facilitating online discussions. Class C as the control class was taught without intervention. These 3 classes were of similar academic abilities in the school. Students worked in groups of two during their learning activities.

Table 1.

Participants and learning design conditions

Class	No. of students	Learning design conditions	Teacher
Class A	39	QIMS framework + AR	Teacher A
Class B	38	QIMS framework + iPads without using AR	Teacher B
Class C	40	No QIMS and without using iPad and AR	Teacher C

Classroom procedure

The topic for the research study was on Plant reproduction and there were three sub-topics—Plant anatomy, Pollination and Seed dispersal. The lessons for these sub-topics were conducted for a period of over three weeks. All three classes had the same learning outcomes to adhere to, based on the school’s official curriculum. They also had to complete tasks in the official textbook and workbook that the school used. The difference, however, was in the mode of teaching and in the ways the science concepts were presented. Only Class A used the AR apps comprising the following modules—“Plant life cycle”, which gives an overview on parts of plants, “Seed Dispersal”, which covers seed dispersal concepts and “Plant Engineer”, which is a synthesis of all three sub-topics.

For class A, the Teacher A began by introducing a story with a problem scenario—aliens had visited the students’ school for help. Upon setting the context, the teacher then got students to ask questions that they were curious about (Questioning). For investigating, the students were given various tasks and encouraged to “think like a scientist”. During the introductory lesson, they learnt about the structure of plants on their own using the Plant Lifecycle AR app, where they worked on The Flower Anatomy activity. This was done without any formal instruction from the teacher. Their task was to examine a virtual flower using the app, identify and scan parts of the virtual flower according to their “missions”. After they had successfully scanned the correct virtual flower part, the app displayed pop-up descriptions of the plant. At the subsequent lesson, the teacher brought real-life flower specimens to the class to review what they learnt, so that they could “verify” that the virtual plant had the same structure as a real-life plant. The AR app allowed for students to self-direct their own learning through self-discovery and active participation instead of just relying primarily on the teacher for information.

The second activity was framed through the same problem scenario, aimed at illustrating the concept of seed dispersal. Students had to learn about the various methods of seed dispersals via a simulation activity so that they could teach the aliens how to help their plants pollinate. The students used the AR app, where they worked on the Seed Dispersal activity that simulates seed dispersal in plants. Students worked in pairs to plant virtual seeds on a virtual island and observed the plants' dispersal process, either by wind, water, or explosive action (Investigating).

The last two activities were meant to review and consolidate what students had learnt for the plant reproduction topic synthesising. The teacher got students to relate what they had during learnt during the plant reproduction lessons to a previous topic, Plant transport systems and explained how the two topics were related. Then, using the AR app, students had to design a hybrid plant that could survive and reproduce in the harsh conditions of the aliens’ planet. After creating the hybrid plant, the students then presented the hybrid plant to their classmates and were encouraged to be as creative as they could.

Similarly, the students in class B did similar activities with that of class A but without the use of AR. Teacher B in class B began by framing the lesson with the same story of the aliens. She did not use the AR app but led students in a cycle of questioning and investigating. Students used the iPads in class to engage with their classmates in online discussions on their online learning portal.

Teacher B engaged students both in classroom discussions, as well as online discussions. First, she would start off by explaining the task or main discussion topic and scaffold it by introducing the main concepts through stimulating a class discussion. Next, she would get the students in work in groups of two for more in-depth peer discussion. Thereafter, students would submit their answers online. Then, students would engage in peer review before the teacher reviewed their responses in class. This allowed the students to have a good mix of physical discussions and online discussions. Online discussions were particularly useful to engage students in active learning, especially for more reserved students who were too shy to speak in class. Additionally, this activity helped students to consolidate, organise and process what they learnt and communicate their findings to each other in the online platform.

Teacher B likened the process of learning science to that of an investigator and introduced the steps of a scientific method to the class. She gave the students more room for group discussions and used an indirect instruction approach. In addition to classroom discussions, students could continue their peer discussions on the learning portal, where they exchanged ideas, thoughts, and opinions. Teacher B asked students what questions they would ask if they were conducting an investigative protocol and used that approach to elicit questions from students. She also got students to think about what sorts of questions students would ask if they met with the aliens. She then used students' questions to steer and scaffold the next activity—the investigations. She brought different types of seeds to class and got them to touch and feel the seeds and got them to think about why and how the seeds could help the aliens learn more about pollination and seed dispersal methods. After the investigations, the teacher recapped all the concepts and related it back to the story of the aliens, as well as to the previous topic of plant transport systems.

Class C conducted investigations similar to those of Class A and B, but without the use of AR. Teacher C in class C did not narrate the story of the aliens, nor did she utilise AR or the QIMS framework. The teacher provided an overview of plant reproduction and seed dispersal before students observed the specimens but did not use students' questions to steer the investigation. She employed a more traditional, teacher-led and direct instructive approach. The students did conduct investigations but were given more explicit instructions.

Data collection and analysis

The data reported in this study is drawn from the following sources: post-intervention interviews with teachers and students; pre-and post-tests on the learning topic, and a post survey on students’ perceptions of their twenty-first century learning skills. Additionally, the final year science examination scores of all the students prior to the intervention were used to classify students into higher-ability and lower-ability groups.

Pre-and post-tests

The pre-test checks students’ existing conceptual knowledge of the topic of plant reproduction before the intervention. The students did not have any lessons on plant reproduction prior to the intervention. The test was designed by the teachers. It was a pen-and-paper test with a total score of 16 marks. The test had ten questions and the students were given about 20 min to complete it. Six of the questions were multiple choice (6 marks) and four questions were scenario-based open-ended questions (10 marks). It comprised questions about plant reproduction, in the areas of pollination, plants’ life cycle processes and seed dispersal methods. An example an open-ended question is as follows: "Ming saw some brightly-coloured and fleshy fruits growing in her garden even though she did not plant them there. The edible fruits contained many small, inedible, and indigestible seeds. How do you think these fruits are dispersed? Please provide explanations”. Open ended questions require critical thinking and application. The students completed the post-test after the intervention and the test questions were the same as the pre-test.

Students’ perceptions on their twenty-first century learning skills

The survey questionnaire adapted from Chai et al. (2015) measures students’ perceptions of twenty-first century learning practices and their knowledge creation self-efficacy after the intervention. The post-survey, consisting of 6 scales, was administrated online in the week of the last intervention. A total of 112 students’ responses were collected and analysed. The Cronbach’s alpha reliability coefficients for the survey was .966. The Cronbach’s alpha reliability coefficients of each scale are shown in Table 2.

Table 2.

The Cronbach’s alpha for each scale

Scale	Number of items	Cronbach’s alpha
Self-directed learning	5	.862
Creative thinking	4	.874
Critical thinking	3	.828
Authentic problem-solving	5	.903
Collaborative learning	5	.891
Knowledge creation efficacy	5	.879

Semi-structured interviews for teachers and students

We conducted online interviews with teachers and students. The three teachers were interviewed individually via zoom. Each interview lasted about an hour. The teachers’ interviews were conducted in a semi-structured format and included questions such as—how they perceived students’ attitudes towards the use of AR and their opinions on the use of the QIMS framework. Student participants from the class A and class B were randomly selected and interviewed in groups after the intervention via zoom. From Class A, we selected 5 groups of 2 students each and from Class B, we selected 3 groups of 2 students each. The interviews with each group of students lasted for about 20 min.

The semi-structured interviews with students focused on students’ perceptions about twenty-first century learning skills and the learning activities they had attended for this study. Thematic analysis was conducted to analyse the transcribed focus group discussions and teachers’ interviews to explain the quantitative findings and to answer the second research question.

Results

Impact on academic performances

We did an analysis of covariance (ANCOVA) to examine the significance of the mean differences among the 3 learning design conditions. The post-test score was used as the dependent variable and the pre-test score was used as the covariant. The homogeneity of the regression coefficients was examined and the result (F = .798; p = .453) did not reach the level of significance. Then, ANCOVA was performed, and the results indicated no significant difference in learning outcomes among the three conditions, F (2, 111) = .303, p = .739 (see Table 3). Though the adjusted mean of the class using AR with the QIMS framework was higher than the other classes, the LSD post-hoc comparisons showed that there were no significant differences among three classes.

Table 3.

Descriptive statistics of students’ pre- and post-test scores and ANCOVA on three learning design conditions

Class	No. of students	Pre-test		Post-test		Mean (adjusted)	ANCOVA
		Mean	SD	Mean	SD		SD	F	p
Class A	37	10.405	4.416	16.270	2.047	16.207	.389	.303	.739
Class B	36	10.250	3.545	15.972	2.613	15.934	.394
Class C	39	9.449	3.788	15.692	2.613	15.787	.380

To examine whether a significant difference exists in the academic performance of students with different academic ability, ANCOVAs by using the pre-test score as the covariate were conducted. The homogeneity of regression coefficients was examined first. The results did not reach the level of significance in all conditions: Class A (F = .132, p = .719), Class B (F = .349, p = .558), and Class C (F = 3.071, p = .089). Then, the ANCOVAs were proceeded, and the results are shown in Table 4. A significant difference only could be found between the higher-ability students and lower-ability students in the control class (Class C), but the p-value of the difference is marginal (see Table 4).

Table 4.

ANCOVA on students with different academic ability

Class	Academic ability	No. of students	Mean (adjusted)	SD	F	p
Class A	Lower	15	16.007	.633	.002	.961
	Higher	22	16.390	.526
Class B	Lower	12	15.722	.687	1.332	.257
	Higher	23	15.981	.498
Class C	Lower	23	14.913	.514	4.126	.050*
	Higher	16	16.975	.598

*p ≤.05

Impact on students’ twenty-first century learning skills

To interpret whether and in which aspects AR and QIMS had an impact on students’ twenty-first century learning skills, we conducted separate analyses of variance (ANOVA) for each scale. Before ANOVA, the homogeneity of regression coefficients was conducted first. The homogeneity of regression coefficients was confirmed for the scales of self-directed learning (F = .192, p = .388), creative thinking (F = .262, p = .770), critical thinking (F = 1.519, p = .224), authentic problem-solving (F = 1.605, p = .206), collaborative learning (F = .556, p = .575), knowledge creation efficacy (F = .892, p = .413). Then, the ANOVAs were conducted. As shown in Table 5, there are significant differences among classes in self-directed learning [F (2,109) = 4.040, p = .020], critical thinking [F (2,109) = 3.142, p = .047] and creative thinking [F (2,109) = 4.079, p = .020], but no significant differences among classes could be found in other scales.

Table 5.

ANOVA comparing twenty-first century learning skills among classes

Twenty-first century learning skill scale	Class						F	p
	Class A		Class B		Class C
	Mean	SD	Mean	SD	Mean	SD
Self-directed learning	4.032	.625	4.000	.595	3.595	.956	4.040	.020*
Creative thinking	4.094	.698	3.765	.742	3.595	.908	4.079	.020*
Critical thinking	4.153	.655	4.064	.539	3.727	1.045	3.142	.047*
Authentic problem-solving	3.741	.881	3.950	.743	3.467	1.017	2.787	.066
Collaborative learning	4.157	.600	4.094	.687	3.810	.950	2.225	.113
Knowledge creation efficacy	3.995	.654	3.883	.746	3.600	.124	2.411	.095

*p ≤.05

LSD Post-hoc tests further revealed that there were no significant differences between the QIMS classes with or without using AR in all the scales. However, the pairwise LSD comparisons revealed that there were significant differences in mean values between the class using AR with QIMS and the control class in self-directed learning (p = .012), creative thinking (p = .007), and critical thinking (p = .018), and knowledge creation efficacy (p = .036). Significant differences in mean values between the class using AR without QIMS and the control class also could be found in self-directed learning (p = .021) and creative thinking (p = .048).

Taking the students' abilities into consideration, we further found that different learning modes did not have significant differences on self-perceived twenty-first century skills for students with higher capability but had significant difference for students with lower capability. The perceptions of self-directed learning and creative thinking skills of students with lower ability in Class C were significantly lower than that of students in the experimental class using AR and QIMS (Class A), but no significant differences could be found among higher capability students in different classes (see Tables 6 and 7).

Table 6.

The results of post-hoc analysis on Self-directed learning

Group	Class	Academic ability	1	2	3	4	5	6
1	A	Lower	–
2	A	Higher	.561	–
3	B	Lower	.420	.742	–
4	B	Higher	.708	.813	.596	–
5	C	Lower	.016*	.040*	.161	.021*	–
6	C	Higher	.148	.320	.584	.223	.371	–

*p ≤.05

Table 7.

The results of post-hoc analysis on creative thinking

Group	Class	Academic ability	1	2	3	4	5	6
1	A	Lower	–
2	A	Higher	.733	–
3	B	Lower	.633	.844	–
4	B	Higher	.397	.576	.786	–
5	C	Lower	.015*	.020*	.079	.072	–
6	C	Higher	.158	.231	.396	.484	.348	–

*p ≤.05

Merits of QIMS guided learning activities

Providing learners with autonomy in questioning and making

According to the teachers who used QIMS, the learning activities based on the QIMS framework provided the students the opportunity to be creative, as students were able to do their own questioning and generate their own ideas on how to solve a question or a problem. This finding is consistent with the quantitative findings drawn from students’ surveys. Even though IBL has been adopted to guide the science curriculum, both teachers pointed out that the use of QIMS promoted more opportunities for autonomous learning.

Teacher B who was using QIMS without AR, mentioned in the post-interview that “I think that’s what I appreciate about the task, and the nature of this framework. Because it’s not just about the knowledge that we want to relay, but the life skills in a sense… usually, we just teach them to the test, so that’s something they’re used to. For this, it’s not teaching to the test.” She further explained that “usually we will either provide them the concept, and then (students) do the activity to reinforce what they needed to learn for that particular concept. QIMS, we sort of withheld the concepts first, through presenting the problem scenario to them.”

Initially, students felt a lot of uncertainty as they usually had the concepts taught explicitly to them. Instead, Teacher B encouraged students to generate their own ideas first before revealing the concepts and solutions to the problem scenarios. As students were not quite used to this manner of teaching, they kept asking the teacher if they were on the right track. Teacher B recounted “I kept saying there’s no right or wrong, you don’t know you just try, see what the rest of the class says, or ask your partner, don’t keep asking me, I’m not going to tell you the right or wrong answer. I kept reiterating there’s no fixed solution, and they weren’t very comfortable with that.”

With the teacher’s encouragement, the students adapted to autonomous learning and understood what the teacher’s intentions were. In response to which teaching strategy they preferred, “Definitely not teacher telling us the answers. Because if your teacher tells you the answers, we can’t really learn ourselves. We just listen to what she says, and we just copy down everything, so you won’t learn that way.” [Student 1 in class B; higher ability]. In a certain degree, this reflects the development of students’ sophisticated epistemologies during the designed learning activities.

Similarly, Teacher A using AR pointed out that “with the use of AR, we need to be able to give students autonomy for their own learning.” This is consistent with the feedback given by the students in his Class. The following excerpts exemplify how the students who used AR perceived their learning experiences:

“In YouTube you see other people do (sic), but in AR you’re doing it yourself, so you feel better. In YouTube you’re just seeing other people do stuff, but if you’re doing AR, you’re doing it yourself. So, the difference is in YouTube other people do it, but in normal stuff you do it.” [Student 1 in class A; lower ability].

“Also, teacher talk too boring and very long time. If you use AR, teacher talks lesser and you get to do more hands-on activities, you get to scan, you get to see virtually…It was quite fun because we worked with our group members and we came up with our own kind of plants, which we felt was very fun.” [Student 2 in class A; lower ability].

Encouraging diverse ways of representing knowledge to improve creative thinking

Both Teacher A and Teacher B emphasised the importance of multimodal representation on improving creative thinking. For one learning activity, students were supposed to solve a challenge by moving some styrofoam balls without using their hands. This was intended to convey the concept of pollination which was not explicitly stated or explained initially by the teacher. Teacher B got students to present their solution through any medium that they felt most appropriate. Although Class B did not use AR, Teacher B mentioned that the learning activity design provided other ways of representing knowledge. She pointed out “I think it made me realise there are more ways of getting them to represent their knowledge. I got them to do some videos—you know how it is with social media and this generation, so I was surprised that many of them had their own ways of doing up the videos. There was one group who actually did slips of papers as subtitles, captions, for the various processes involved in plant reproduction. There were also some video memes that they created. …That element of creativity could be seen because it was not something that you can represent on pen and paper.” Teacher B highlighted the need to keep students’ mind open when facilitating the activities. She said “… along the way, I realised that [teachers] had to also be open-minded to their ways of representing. So, I think that was me trying to adapt at these certain junctures, so as not to hold them back from creating.”

The same strategy was also emphasised by Teacher A. At the stage of synthesising, students in the AR class were asked to create and present their plant creations. They concluded by presenting their plant creations with supporting explanations on why their plants were feasible. Teacher A expressed delight at his students’ creativity “I’m actually quite pleased and surprised to know that they’re quite creative. I didn’t even help them one bit, I just told them to be creative and can use any platform to showcase or present your hybrid plant. So, some of them started to make a skit on their own ideas and their own initiatives; they made songs to present; using a poster.”

When students are given the chance to present and learn knowledge in various modes, it can help learners understand new knowledge more efficiently. Students from Class A reflected that it was fun “creating ideas” as it was “a different way of learning” and it gave them opportunities to “create plants that you really can’t create in real life. And you get to see the plant you created”. One student shared that making the hybrid plant was useful because “you get a clearer view of what’s going on—it just pops more ideas into our mind.” Together, these qualitative excerpts demonstrate the ways the teachers and students perceive the diverse ways of representing knowledge.

Facilitating learning activities with advanced epistemological beliefs

Adopting new forms of technology-mediated learning can extend teachers’ own understanding of themselves as educators by promoting self-reflection and critical evaluation (Wen & Wu, 2017). During the teacher interviews, both teachers stated that they believed in autonomous learning strategies and perceived themselves more as facilitators rather than taking on the traditional role of content experts. Teacher B felt that the way lessons conducted were “something that is very positive, something that will value add to what [students] already have, or what they are exposed to.” Teacher A shared that “it makes me as a teacher to form certain connections, which I otherwise would not think about, and very intentionally have the children be involved in the creating process, and the thinking process.”

At the same time, they also revealed that they were weighed down by practical concerns like time constraints and having the complete the syllabus on time. Teacher A shared that teachers would be required to troubleshoot the app on their own, and even then, there would be hiccups in class—so that would take up a lot more time and effort on the teacher’s part. Thus, that could be a “limitation that may prevent my teaching peers to adopt the app.” As Teacher B reflected:

“Teachers have to be very adaptable and flexible in preparing lessons […] My advice to them, to have an open mind, to be able to think on their feet in terms of always unexpected, not in a bad way—it’s more of being flexible enough, to approach the lessons, to be able to answer the students’ queries. Because when they’re using AR, using the tablet, the questions they ask may not pertain to the lesson itself.”

In a similar vein, Teacher A contemplated:

“I think there still needs to be a lot of skill from the teacher in the sense you have to react, you have to respond accordingly. There’s no fixed template or lesson plan that can prepare you for what they will come up with or ask you. That’s why I’m saying that’s very uncomfortable for me.”

Discussion

The findings of the study suggested that either using AR or the QIMS framework had little impact on students with higher ability in terms of learning results. Little difference could be found between the class A using AR with QIMS and class B with QIMS without AR. However, the use of AR and QIMS might help lower-ability students improve their learning results more significantly, as no statistically significant differences could be found in the higher-ability and lower-ability students in both classes A and B, while it could be found in the control class (Class C).

With regards to students’ soft skills, there was a significant difference in the constructs of self-directed learning, critical thinking, and creative thinking. Yet, there was still no significant difference in class A (QIMS and AR) and class B (QIMS without AR). Integrating AR with QIMS had a comparatively more significant effect on students’ self-directed learning and creative thinking skills, especially for the students with lower academic performance. Ibáñez et al., (2015, 2016) have emphasised the need for scaffolding in AR-based IBL activities, especially for students with low levels of self-regulation skills or motivation. The findings of this study resonate with their study, by highlighting the importance of having scaffolding when integrating IBL with AR.

It is worth noting that the use of AR did not affect students’ perceptions on authentic problem-solving obviously. This finding might indicate that the immersion of paper-based AR was not sufficient. On the other hand, this suggested that instead of replicating real-world experiments or being limited in the classroom, our following AR activity design could further strengthen the link with the living environment. Besides, although the AR class had a higher self-rating in the scale of collaborative learning, there was no significant difference among the three classes. This finding is not line with existing AR studies (e.g., Fidan & Tuncel, 2019), in which the advantages of AR environment for promoting collaborative learning have been evidenced. This might be due to the Covid-19, only two students could sit together to complete the activities. This also suggests that the collaboration script embedded in the AR and QIMS-based learning activities could be further improved.

The success of using AR in teaching and learning is the result of multiple factors (Garzón et al., 2020). On the premise of recognising the importance of integrating AR and IBL, another purpose of this study was to investigate how to guide teachers to implement the integration more intuitively. Three strategies about implementing QIMS framework in AR-enhanced learning were proposed based on the findings of the study: (1) providing learners with autonomy in questioning and making, (2) encouraging diverse ways of representing knowledge to improve creative thinking, and (3) facilitating learning activities with advanced epistemological beliefs. These strategies are also interrelated. As stated in an earlier study by Tsai (2004), more empirical evidence should be gathered to examine the cyclical relationship between the development of sophisticated epistemologies and the practices of using advanced technologies. The findings of this study resonate the importance of advanced epistemological beliefs of teachers and show the evidence that its influence on the development of students sophisticated epistemologies in an AR-enhanced learning environment.

Conclusion, limitations and future directions

The paper reports an empirical study to demonstrate how the QIMS framework was used to guide an AR learning activity design and its impact on the learning outcomes and development of soft skills of primary school students in fifth grade. The findings indicate that the class using AR did not outperform the class who used QIMS without AR. However, the two classes with QIMS-guided activities outperformed the class who did not use QIMS. Though students did not demonstrate statistically significant improvement in their academic performance, their self-directed learning and creative thinking skills did significantly improve via experiencing the QIMS-guided learning activities. Moreover, the findings suggest that using AR with QIMS-guided framework could contribute to the improvement of academic performance of primary school students who are academically weaker in science. One reason could be that the design of the learning environment provides every learner with autonomy in learning activities. Therefore, students of different academic abilities could have the opportunity to participate in more creative learning activities. When integrating IBL with AR and implementing it, practitioners should (1) provide learners with autonomy in questioning and making, (2) encourage diverse ways of representing knowledge to improve creative thinking, and (3) facilitating learning activities with advanced epistemological beliefs.

This study has several important limitations. Firstly, as the AR app is self-developed, the user interface and 3D animations still need further polishing to better engage learners. Our initial hypothesis was that the class using both AR and QIMS could outperform the other two classes. However, this is not supported by the data of this study. It may be caused by the intervention time and/or the AR app design. We also take into account the potential for app malfunctions and lag, as it can result in a poor user experience and cause frustration for the users. The AR app will be further upgraded, and relevant work has been planned and will be conducted in more schools. The subsequent studies will also explore the impact of AR-based QIMS learning packages on different topics. Secondly, the teacher who used AR and QIMS may need more time to adapt to the design learning environment, which may be another reason why significant difference between the AR and QIMS classes were not identified, as we had assumed. Another limitation is that three different teachers conducted the intervention in the three classes. Ideally, another around of intervention with the same teachers would be implemented in the school with different batches of classes. Lastly, due to the constraints of data collection during the Covid-19 period, students’ learning process data could not be collected in its entirety. In the following round of interventions, we will pay attention to how students interact during their learning process to further explain how individual differences influence the success of the AR activities and how to design AR activities to benefit diverse learners.

Biographies

Yun Wen

is an Assistant Professor in the Learning Sciences and Assessment Academic Group at the National Institute of Education (NIE), Nanyang Technological University (NTU Singapore). Her research investigates how people learn through interaction and conversations in multimodal environments, and how to use technology (e.g., representational tools or augmented reality techniques) to spark and support collaborative learning.

Longkai Wu

is a professor at the National Engineering Research Centre for E-Learning & MOE (PRC) Educational Informatisation Strategy Research Base, Faculty of Artificial Intelligence in Education, Central China Normal University. He has been working on classroom inquiry to enable teachers and students for a deeper understanding of science subjects, promoting paradigms of immersive learning, inquiry-based learning in different classroom endeavours.

Sujin He

is a Research Assistant at NIE. Prior to this, she was involved in curatorial research for arts and heritage organisations. She has worked on research projects on school autonomy and leadership, informal learning spaces and making-centred learning.

Nathanael Hsien-Ern Ng

is a lecturer and freelance software developer. His research interests include the use of interactive media in teaching and learning.

Beng Chong Teo

is an Associate Professor in the Mathematics and Mathematics Education Academic Group at NIE. His research interests include design and interest-based learning contexts.

Chee Kit Looi

is the Co-Director of the Centre for Research and Development in Learning (CRADLE) at NTU. He was the Founding Head of Learning Sciences Lab, the first research centre devoted to the study of the sciences of learning in the Asia–Pacific region. His research in education is characterised by producing outcomes, processes or artifacts that impact practice.

Yiyu Cai

is senior faculty with NTU. He heads the Computer-aided Engineering Labs, and directs the Strategic Research Program of VR & Soft Computing. He is also deputy director of The Center of Augmented & Virtual Reality.

Funding

This study is funded by the National Institute of Education, Nanyang Technological University, Singapore, with project reference number: OER 26/19 WLK. This research is also supported by the Ministry of Education, Singapore, under its Tertiary Research Fund (Project MOE2018-TRF-006).

Data availability

Nanyang Technological University (NTU) owns all research data produced by this project. There will be no restriction to sharing final research data. All the research data will be achieved at the National Institute of Education (NIE) research data repository after the end of the project.

Declarations

Conflict of interest

The author declares that she has no conflict of interest.

Ethical approval

The project has research ethics approval from the Institutional Review Board of Nanyang Technological University. The NTU IRB reference number for this study is IRB-2020-04-037.

Consent to participate

Written informed consent was obtained from all individual participants/their parents.

Consent to publish

The author affirms that all participants provided informed consent for the publication of the images.