Abstract
The aim of this study is to design and validate an instrument that allows the evaluation of educational experiences and formative assessment in the metaverse from a holistic perspective. Hence, a research design based on the development of a scale has been used. Three hundred and sixty-two Spanish secondary school students participated in the study, selected through purposive sampling. The instrument created has been subjected to an analysis of content validity, validity by expert judgment, construct validity, and reliability. For data analysis, the SPSS and AMOS programs have been used. An exploratory factor analysis and a confirmatory factor analysis have been performed to determine the construct validity. In addition, the Cronbach's alpha has been calculated to verify the internal consistency of the tool. The results reveal several findings that position the resulting questionnaire as a valid and reliable instrument to evaluate educational experiences and practices in the metaverse. In short, this study has led to the development of a comprehensive evaluation tool at the service of educators or any institution interested in implementing its educational praxis within the metaverse, a field of research that has yet to be explored.
Keywords: Validation, Instrument, Validity, Reliability, Metaverse, Educational innovation, Educational technology
Validation; Instrument; Validity; Reliability; Metaverse; Educational innovation; Educational technology.
1. Introduction
Technology-enhanced learning is becoming more popular as more educators are trying to implement technological applications within their teaching activities (Bayne, 2015). This fact can assist in meeting students’ new educational needs and requirements (Moreno-Guerrero et al., 2021). More engaging learning experiences that promote meaningful learning and enable students to actively participate are sought after (Crisol-Moya et al., 2020). With new digital devices and emerging technological applications being rapidly integrated into the educational process (Zawacki-Richter and Latchem, 2018), it is essential to create and apply proper validation tools and instruments to assess their impact, usability and accessibility in educational contexts.
The topic of metaverse is gaining ground. Metaverse is based upon the Generation Z social value that there is no difference between one's offline and online self (Park and Kim, 2022). Metaverse can be regarded as a next-generation internet application, a social form and virtual world which utilizes novel technologies to create a virtual living environment which can be produced and edited by users and integrates economic, social and identity systems (Ning et al., 2021). These shared, perpetual, concurrent virtual environments are combined to create a perceived and integrated virtual universe (Lee et al., 2021) which is characterized by its hyper spatiotemporal, fully immersive and self-sustaining nature (Wang et al., 2022).
Metaverse capitalizes on virtual reality and augmented reality technologies in order to offer users a virtual presence in shared and parallel computer-generated three-dimensional (3D) virtual worlds (Lemos, 2007). As this virtual environment overcomes the boundaries of time and distance differences and allows real-time interactions and socialization (Falchuk et al., 2018), it can be described as a lifelike digital extension of the real world (Stephenson, 1992) which has interactivity, embodiment and persistence at its core (Castronova, 2001). As metaverse combines “virtually-enhanced physical reality” with “physically persistent virtual space”, it enables daily tasks and interactions to move from the physical world and be carried out in the virtual one (Collins, 2008). This fact can greatly affect daily life, human societies and cultures. Moreover, users can share their experiences, thoughts, emotions and culture in these environments (Park et al., 2021) and as a result, they can create social identities within virtual communities that promote a sense of belonging and an improved social presence.
The findings of the recent studies, which were analyzed during the identification of the main and most frequent dimensions used to assess the effectiveness of metaverse in education, showed positive results regarding its use in education, the overall educational experience and learning outcomes. In particular, based on these findings, it can be said that metaverse aligns with the participatory and collaborative model and consequently, allows teachers to implement novel pedagogical approaches and didactic methods (Diaz, 2020). When applied in teaching and learning activities, metaverse has the potential to yield several benefits and affect the overall educational process positively by providing meaningful learning opportunities (Baynat and López, 2020). As metaverse constantly evolves both technically and culturally, it can bring about changes to both e-learning and conventional learning experiences in an innovative way (Abeles, 2007). Particularly, metaverse can create immersive, flexible, scalable, diverse and interactive learning environments that enhance students' motivation, active involvement and engagement and promote collaborative and hands-on experiences (Diaz, 2020; Tarouco et al., 2013; Vaca Barahona et al., 2016). Additionally, these environments can create synchronous and asynchronous learning experiences of high quality (Diaz, 2020; Tarouco et al., 2013) as they promote affective learning, help abolish social and identity barriers (Calongne et al., 2013), increase students’ soft skills (Nurhidayah et al., 2020), creativity (Garcia, 2011) and overall communication and interaction (Vaca Barahona et al., 2016) as well as enhance subject comprehension and academic performance (Garrido-Inigo & Rodriguez-Moreno, 2013; Reyes, 2020; Schaf et al., 2012). It is worth noting that even though students might face some technical difficulties in the beginning, they are able to quickly familiarize themselves with the metaverse and its various features as they are accustomed to handling digital devices and media (Arcila, 2015). This fact in combination with the game-like nature of the metaverse (Schaf et al., 2012), the opportunity to hone their skills in real-world scenarios (Schlemmer et al., 2009) and to engage in interactive problem-solving activities (Clark, 2009) are the reasons why the use of metaverse in education is positively viewed and widely accepted by students.
Despite this fact, in order for metaverse to be more widely implemented within educational settings, there still remain some open issues and concerns regarding security, ethics and privacy that should be addressed (Chukwunonso et al., 2022; Fernandez and Hui, 2022; Wang et al., 2022). Additionally, interoperability, scalability, confidentiality, immersive realism and access and identity ubiquity are four of the core areas of metaverse that need to be further examined and developed (Dionisio et al., 2013). Particularly in the case of education, areas such as privacy issues, health concerns, students’ protection, access inequality, specific metaverse laws, desensitization and identity hacking remain some main concerns and drawbacks that need to be dealt with (Pereira, 2022). If these issues are not solved, they might hinder the widespread use and the inclusion of all students in some educational activities.
As it can be seen, the implementation of metaverse in education can affect the teaching and learning activities positively and create student-centered meaningful learning experiences and allow the educational process to take place in purely digital environments. Additionally, due to the immersive and interactive nature, the creation of safe and controlled environments and adaptation to the learners’ characteristics, immersive technologies and metaverse specifically can facilitate and increase the inclusion of students in educational activities (López-Belmonte et al., 2022). Nonetheless, in order for metaverse to be more widely adopted and used in educational contexts, there is a clear need for validation and assessment tools to be developed. To evaluate the impact and successful integration of new technologies, applications and approaches within the educational domain, the development, adoption and use of appropriate, sound and reliable validation and assessment tools and instruments are essential (Caeiro et al., 2013; Shepard, 2000). Consequently, the need to have valid and reliable tools that allow the assessment of the educational practices conducted in the metaverse arises. The expert literature on the state of the issue involves the use of various instruments to perform such a task. However, no instrument has been reported that covers the dimensional spectrum contemplated by the metaverse in the educational field. Therefore, the aim of this study to design, validate, and create a reliable assessment tool, called METAEDU, which allows the evaluation of the educational experiences carried out in the metaverse from a holistic perspective. Under the concept of educational experiences, all those actions designed, planned and implemented by the teacher are included so that the students can achieve the different objectives and didactic contents.
The evaluation tool uses key dimensions that were identified in the literature. These dimensions can be related to students’ interaction with the metaverse, its virtual contents as well with other participants (teachers and students), to its accessibility and management, to its intrinsic possibilities as well as to its ability to create fun, interesting, engaging and motivating learning experiences (Pozo-Sánchez et al., 2020). An additional dimension that is essential to be included in the current era is that of netiquette, ethics and digital literacy (Soler-Costa et al., 2021).
2. Method
2.1. Research design
To achieve the scope of the proposed objective, a research design based on the development of a scale has been used (Ato et al., 2013; Parra-González et al., 2022). Additionally, the design is transversal in terms of time and correlational with respect to the stated objective. This type of research design is used to study the psychometric properties of an assessment tool (Hernández et al., 2014).
2.2. Participants
Three hundred and sixty-two Spanish students participated in the study. Of this number, 39.5% are men (n = 143) and the rest (60.5%) are women (n = 219). These students are between the ages of 14 and 16 (M = 15.3; SD = 1.226), enrolled in the 3rd (n = 208; 57.5%) and 4th (n = 154; 42.5%) year of Compulsory Secondary Education of the Spanish Educational System. These subjects were selected through purposive sampling. This technique is justified by researchers in the choice of pilot centers in which innovative educational practices are carried out in the metaverse.
2.3. Instrument
During the literature review phase, a search was made for instruments used in preliminary research. The purpose of the search was to acquire a range of tools with which to explore how the educational experiences carried out in the metaverse by other researchers have been evaluated. After this process, various instruments were reported which were then grouped into the following, most prominent categories: Interviews (Jaffurs, 2011; Schlemmer et al., 2009); Ad hoc questionnaires (Clark, 2009; Reyes, 2020; Tarouco et al., 2013); Observation records (García, 2011; Nurhidayah et al., 2020); Mixed-method tools (Peña, 2014; Díaz et al., 2020); Validated questionnaires (Park et al., 2021).
After analyzing the dimensions and variables of the various tools, it was decided to use the instruments designed by Peña (2014) and Tarouco et al. (2013). Furthermore, to complement these instruments, the European Digital Competence Framework was taken as a reference to incorporate the netiquette dimension and the adaptation of the Motivated Strategies for Learning Questionnaire (MSLQ) to Spanish to measure student motivation (Segura-Robles et al., 2021).
This implied that after the analysis of the previous literature and the validation and reliability process were carried out, the final design of the METAEDU questionnaire consisted of a total of 68 items, grouped into the following 8 dimensions: 1) Interaction with technology; 2) Intrinsic possibilities; 3) Accessibility and handling; 4) Interaction; 5) Interest; 6) Motivation; 7) Learning; 8) Netiquette (Appendix 1).
The final tool developed uses a six-point Likert-type rating scale (0 = never; 1 = rarely; 2 = occasionally; 3 = frequently; 4 = many times; 5 = always).
In the result section of this manuscript, the different validation and reliability processes carried out in this instrument are detailed with greater precision, with the aim of achieving a valid and reliable tool for the evaluation of educational experiences in the metaverse.
2.4. Procedure
This research is part of the Project called: We arrived at the META: teaching methodology for the transformation of learning in the metaverse, with code 22–115 and endorsed by the University of Granada (Spain).
In this study, a strict process has been followed, structured in different actions: a) review of the literature; b) identification of previous instruments; c) tool design and development; d) content validity; e) construct validity; f) reliability of the instrument. This study was approved by an ethics committee (MTV22). Moreover, informed consent was obtained from participants in this research. The participation of the subjects was completely voluntary. The anonymity of the students was respected at all times. Additionally, the participants had the right to withdraw from the study at any given time if they considered it appropriate.
Once the questionnaire was validated through the judgment of various experts, it was applied to the students virtually through the institutional platforms of the educational centers. A period of 14 days was established for the students to respond to the questions raised. The teachers of these educational centers collaborated so that the students responded to the tool objectively and satisfactorily. Once a relevant number of respondents was reached, the statistical analysis was performed to continue with the validation of the instrument.
2.5. Data analysis
Statistical analysis was conducted with version 25 of the SPSS and AMOS programs. Descriptive statistics were used to provide information on the sociodemographic characteristics of the participants. An exploratory factor analysis (EFA) and subsequent confirmatory factor analysis (CFA) were performed to determine construct validity. Cronbach's alpha (α) was used to verify the internal consistency of the tool. In addition, confidence intervals utilizing boostrap or resampling techniques have been applied, using McDonald's Omega (Romero-Rodríguez et al., 2022).
3. Results
3.1. Content validity
Content validity refers to the degree to which an instrument reflects a specific content domain of what is measured. It is the degree to which the measurement represents the concept or variable measured (Hernández et al., 2014). In this case, the authors of this manuscript reviewed more than 64 documents. To determine the choice of these documents, a search equation was performed in Web of Science (WOS) and SCOPUS, having applied a PRISMA protocol (Figure 1) with the search equation related to "metaverse" for this purpose. Of these documents, 16 were research articles and 1 was a book chapter on the subject of the study. The type of studies analyzed were systematic reviews (n = 2), meta-analyses (n = 0), qualitative studies (n = 5), quantitative studies (n = 6) and mixed studies (n = 4).
Figure 1.
PRISMA protocol phases.
After completing the review, the most frequent dimensions were established: 1) Interaction with technology; 2) Intrinsic possibilities; 3) Accessibility and handling; 4) Interaction; 5) Interest; 6) Motivation; 7) Learning; and 8) Netiquette.
Subsequently, bearing these dimensions in mind, a detailed search was carried out in the scientific literature to determine the possible items that could constitute these dimensions. In this case, the dimensions already established in the studies related to metaverse were used as a basis, but to complement and enrich the instrument, the action presented below was carried out as indicated above.
3.2. Validity of expert judgment
Once the content validity was completed, we proceeded to the expert validity (Escobar-Pérez and Cuervo-Martínez, 2008). For this purpose, questions were asked about the appropriateness of each item. The adequacy criteria were related to quality, coherence, and relevance (Dorantes et al., 2016). Each item was answered by the experts on a 5-point Likert-type scale. The levels were distributed as follows: 1 (Does not meet the criterion), 2 (Low level), 3 (Medium level), 4 (High level) and 5 (Meets the criterion perfectly). In each of the items, the possibility of establishing comments and/or proposals was offered.
The selection of experts was made bearing in mind the following premises:
-
•
Experience in the subject matter: for this purpose, the WOS and SCOPUS databases were visualized to identify experts of the subject in the Spanish context. As the instrument was evaluated and validated in the Spanish context, it was essential for the experts to be familiar with it.
-
•
Experience in the validity of instruments in the educational field: the WOS and SCOPUS databases were visualized to identify the most relevant instruments in both Spanish and international contexts.
-
•
Experience in the educational field: this was done by searching on Twitter, using the hashtag #metaverso and #metaverse. In this case, the aim was to identify the most relevant international educational experts.
For each case, three candidates were selected. Their profiles are displayed in Table 1 of the expert judgment protocol (Romero-Rodríguez et al., 2022):
Table 1.
Expert judgment protocol.
| Purpose | Confirm the appropriateness of each item on the scale. |
|---|---|
| Experts |
|
| Validation mode | Single method. The experts had no connection with each other. Three have been selected according to the type of experience (theme, instrument validity and educational field). |
From the answers given by the experts, the data was compiled taking into account the mean and standard deviation statistics. In addition, the concordance index of each item was taken into consideration in relation to the criteria of clarity, coherence, and relevance (Table 2). The calculation of the concordance index was made taking into account the frequency, represented as a percentage based on the scores given by the authors to each item according to the established Likert scale (1–5).
Table 2.
Mean, standard deviation and concordance index as a function of frequency.
| Items | M/SD (%) |
||
|---|---|---|---|
| Clarity | Coherence | Relevance | |
| 1.- I play network role-playing games in my everyday life | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 2.- I interact in virtual worlds on a daily basis | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 3.- I use social networking social media daily | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 4.- I make video-conferences with other people | 4.22/.441 (84.4) | 5/0 (100%) | 5/0 (100%) |
| 5.- I receive virtual education through platforms | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 6.- Work collaboratively through content management platforms or the cloud. | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 7.- I teleport to other places in the metaverse | 5/0 (100%) | 4.78/.441 (95.6) | 4.33/.500 (86.6) |
| 8.- I modify my physical appearance in the metaverse | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 9.- I usually change the way I dress in the metaverse | 4.11/.333 (82.2) | 4.44/.527 (88.8) | 5/0 (100%) |
| 10.- I handle different objects that I have in my personal inventory | 4.78/.441 (95.6) | 5/0 (100%) | 5/0 (100%) |
| 11.- I open boxes that I find to expand my inventory of objects | 4.56/.527 (91.2) | 5/0 (100%) | 4.22/.441 (84.4) |
| 12.- I communicate with other people in the metaverse in different formats (e.g. voice, text) | 5/0 (100%) | 5/0 (100%) | 4.11/.333 (82.2) |
| 13.- I take and save photos from places that I like in the metaverse | 5/0 (100%) | 5/0 (100%) | 4.67/.500 (93.4) |
| 14.- I use vehicles or other means of transportation to move around the metaverse | 5/0 (100%) | 5/0 (100%) | 4.22/.441 (84.4) |
| 15.- I enjoy interacting with other people in the metaverse | 5/0 (100%) | 4.89/.333 (97.8) | 5/0 (100%) |
| 16.- I interact daily in the metaverse | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 17.- I overcame the difficulties of access and use thanks to my teachers | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 18.- I overcame the difficulties of access and use thanks to my colleagues | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 19.- I overcame the difficulties of access and use thanks to the help of tutorials | 5/0 (100%) | 5/0 (100%) | 4.22/.441 (84.4) |
| 20.- I overcame the difficulties of access and use thanks to personal research | 4.11/.333 (82.2) | 5/0 (100%) | 5/0 (100%) |
| 21.- I overcame the difficulties of access and use thanks to the suggestions and help services of the system | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 22.- The teachers used appropriate methodologies to encourage the creative activity of the metaverse | 4.78/.441 (95.6) | 5/0 (100%) | 5/0 (100%) |
| 23.- The tutorials were useful to resolve doubts | 4.11/.333 (82.2) | 5/0 (100%) | 5/0 (100%) |
| 24.- The tutorials were useful to improve control of the system as a user | 2.33/.421 (46.6) | 2.77/.449 (55.4) | 2.22/.411 (44.4) |
| 25.- The interaction in the metaverse has modified my habits of organizing free time | 4.22/.441 (84.4) | 5/0 (100%) | 5/0 (100%) |
| 26.- The interaction in the metaverse has modified my sleeping habits | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 27.- I would like to carry out academic activities in the metaverse | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 28.- Interacting through my avatar has allowed me to relate easily to other people | 5/0 (100%) | 5/0 (100%) | 4.11/.333 (82.2) |
| 29.- Interacting with other people in the metaverse has allowed me to relate better in real life | 5/0 (100%) | 5/0 (100%) | 4.78/.441 (95.6) |
| 30.- Interacting with other people in the metaverse has allowed me to feel freer to express my ideas in real life | 4.11/.333 (82.2) | 5/0 (100%) | 5/0 (100%) |
| 31.- Interacting from my virtual avatar has motivated me to learn in the metaverse | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 32.- Interacting with other avatars has been familiar to me due to real-life experiences | 2.77/.421 (55.4) | 2.11/.339 (42.2) | 1.99/.749 (39.8) |
| 33.- Interacting in the metaverse has increased my feeling of belonging to the training community | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 34.- Attending classes in the metaverse has been interesting to me | 4.78/.441 (95.6) | 5/0 (100%) | 5/0 (100%) |
| 35.- Solving problems in the metaverse has been interesting to me | 5/0 (100%) | 5/0 (100%) | 4.22/.441 (84.4) |
| 36.- Exchanging ideas among colleagues in the metaverse has helped me strengthen my digital competence | 5/0 (100%) | 5/0 (100%) | 4.78/.441 (95.6) |
| 37.- Interacting in the metaverse has improved my ability to communicate in digital environments | 5/0 (100%) | 5/0 (100%) | 4.11/.333 (82.2) |
| 38.- Interacting with colleagues in the metaverse has fostered my socialization | 4.67/.500 (93.4) | 4.78/.441 (95.6) | 5/0 (100%) |
| 39.- The visual communication used during the course was motivating | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 40.- The three-dimensional visual effect (3D) of the environment and the characters of the metaverse have favored my motivation to interact | 5/0 (100%) | 5/0 (100%) | 4.11/.333 (82.2) |
| 41.- The immersive experience of the metaverse has generated my interest to continue learning about other metaverses in the educational field | 3.11/.413 (62.2) | 2.88/.296 (57.6) | 2.44/.779 (48.8) |
| 42.- The immersive experience of the metaverse has generated my interest in learning about the metaverse in other contexts | 2.99/.366 (59.8) | 2.11/.227 (2.11) | 2.22/.285 (44.4) |
| 43.- Communicating through chat in the metaverse was interesting to me | 4.22/.441 (84.4) | 5/0 (100%) | 5/0 (100%) |
| 44.- Communicating with a voice in the metaverse was interesting to me | 4.78/.441 (95.6) | 5/0 (100%) | 5/0 (100%) |
| 45.- I prefer class material that is really challenging to learn new things | 5/0 (100%) | 5/0 (100%) | 4.22/.441 (84.4) |
| 46.- I prefer class material that arouses my curiosity, even if it is difficult to learn | 5/0 (100%) | 5/0 (100%) | 4.78/.441 (95.6) |
| 47.- The most satisfying thing for me is to understand the contents in the best possible way | 4.11/.333 (82.2) | 5/0 (100%) | 5/0 (100%) |
| 48.- When I have the opportunity to choose, I choose to do tasks in which I can learn even if they do not guarantee a good grade | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 49.- Getting a good grade in class is the most satisfying thing for me at the moment | 5/0 (100%) | 5/0 (100%) | 4.22/.441 (84.4) |
| 50.- The most important thing for me at the moment is to improve the average of my grades, so my main concern is to get good grades | 4.78/.441 (95.6) | 5/0 (100%) | 5/0 (100%) |
| 51.- If I can, I want to get better grades than most of the other students | 5/0 (100%) | 5/0 (100%) | 4.11/.333 (82.2) |
| 52.- I want to do well in class because it is important to show my ability to my family and social environment (friends, teachers or other people) | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 53.- Immersion in the metaverse has favored my ability to learn conceptual content (facts, data, and concepts) | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 54.- Immersion in the metaverse has favored my ability to learn procedural content (know how) | 4.11/.333 (82.2) | 5/0 (100%) | 5/0 (100%) |
| 55.- Immersion in the metaverse has favored my ability to learn attitudinal content (values, attitudes, and norms) | 5/0 (100%) | 5/0 (100%) | 4.78/.441 (95.6) |
| 56.- Immersion in the metaverse has improved my level of communicative competence | 4.11/.333 (82.2) | 5/0 (100%) | 5/0 (100%) |
| 57.- Immersion in the metaverse has improved my ability to process information and digital competence | 4.67/.500 (93.4) | 5/0 (100%) | 5/0 (100%) |
| 58.- Immersion in the metaverse has improved my level of social and civic competence | 5/0 (100%) | 5/0 (100%) | 4.78/.441 (95.6) |
| 59.- Immersion in the metaverse has improved my level of competence in autonomy and personal initiative | 5/0 (100%) | 5/0 (100%) | 5/0 (100%) |
| 60.- Immersion in the metaverse has improved my level of competence in learning to learn | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 61.- Immersion in the metaverse has allowed me to know and put into practice the proper use of multimedia communication tools within the teaching and learning process | 5/0 (100%) | 5/0 (100%) | 4.67/.500 (93.4) |
| 62.- Immersion in the metaverse has favored my academic performance | 5/0 (100%) | 4.78/.441 (95.6) | 5/0 (100%) |
| 63.- Immersion in the metaverse has improved my considerations about my self-esteem and my self-concept | 2.11/.441 (42.2) | 2.11/.349 (42.2) | 2.22/.334 (44.4) |
| 64.- Immersion in the metaverse has improved my family and social environment perception of my performance | 2.88/.221 (57.6) | 2.77/.278 (55.4) | 2.88/.471 (57.6) |
| 65.- I know and use some conventions or rules of written and iconic communication among Internet users | 5/0 (100%) | 5/0 (100%) | 4.11/.333 (82.2) |
| 66.- I try to write my messages respectfully and without offending others | 4.22/.441 (84.4) | 5/0 (100%) | 5/0 (100%) |
| 67.- I am aware that there are dangers arising from the use of the Internet | 5/0 (100%) | 5/0 (100%) | 4.67/.500 (93.4) |
| 68.- I define and characterize the different inappropriate uses of the Internet and its negative effects | 4.78/.441 (95.6) | 5/0 (100%) | 5/0 (100%) |
| 69.- I am able to identify and act in the event of cyber-bullying | 5/0 (100%) | 4.11/.333 (82.2) | 5/0 (100%) |
| 70.- I feel discomfort and rejection towards any type of discrimination, harassment or inappropriate use of technology | 4.67/.500 (93.4) | 5/0 (100%) | 5/0 (100%) |
| 71.- I know the basic rules of education when I communicate with my peers. | 5/0 (100%) | 5/0 (100%) | 4.78/.441 (95.6) |
| 72.- I avoid using words or images that may be offensive or misunderstood by the recipients of my messages | 4.22/.441 (84.4) | 5/0 (100%) | 5/0 (100%) |
From the initial 72 items, four items were eliminated and three items were not considered due to their having a mean of less than 3. For those items with mean scores between 4 and 5, the comments of the experts were taken into account to modify and improve them. It should be noted that the experts' ratings below 5 were mainly due to questions of wording or inadequacy for the objectives of the instrument.
3.3. Construct validity of questionnaire
Once the content validity and expert judgment validity had been carried out, a total of 362 Spanish adolescents from different schools in Spain, were selected. This sample was the basis for proceeding to construct validation.
First, an exploratory factor analysis (EFA) was performed. For this purpose, the Kaiser-Meyer-Olkin sample adequacy measure (KMO = 0.836), which allows the assessment of the degree to which each of the variables is predictable from the other variables and Bartlett's test of sphericity (X2 = 1805.531; df = 342); p-value = 0.000) were calculated. The resulting values show the appropriateness of the relevance of conducting the EFA. The community analysis shows that the values are above 0.6, varying between 0.603 and 0.912, being adequately explained by the factor structure. In the principal component analysis with Quartimax rotation with Kaiser, the 68 items were grouped into 8 components that explained 84.56% of the variance. As for the distribution of items per factor, the first factor explained 12.31% of the variance and included six items, those referring to interaction with technology. The second factor accounted for 11.62% and included ten items related to intrinsic possibilities. The third factor represented 10.53% and included seven items, corresponding to accessibility and handling. The fourth factor represented 10.42% and included eight items, related to interaction. The fifth factor represented 10.31% and included nine items related to interest. The sixth factor represented 10.11% and included eight items related to motivation. The seventh factor represented 9.92% and included ten items related to learning. The eighth and last factor represented 9.34% and included eight items related to netiquette. The factor model consisted of appropriate construct indicators (Table 3).
Table 3.
Rotated component matrix.
| Items | Factor 1 | Factor 2 | Factor 3 | Factor 4 | Factor 5 | Factor 6 | Factor 7 | Factor 8 |
|---|---|---|---|---|---|---|---|---|
| 1.- I play network role-playing games in my everyday life | 0.731 | |||||||
| 2.- I interact in virtual worlds on a daily basis | 0.634 | |||||||
| 3.- I use social networking social media daily | 0.812 | |||||||
| 4.- I make video-conferences with other people | 0.607 | |||||||
| 5.- I receive virtual education through platforms | 0.711 | |||||||
| 6.- Work collaboratively through content management platforms or the cloud. | 0.912 | |||||||
| 7.- I teleport to other places in the metaverse | 0.603 | |||||||
| 8.- I modify my physical appearance in the metaverse | 0.785 | |||||||
| 9.- I usually change the way I dress in the metaverse | 0.662 | |||||||
| 10.- I handle different objects that I have in my personal inventory | 0.872 | |||||||
| 11.- I open boxes that I find to expand my inventory of objects | 0.650 | |||||||
| 12.- I communicate with other people in the metaverse in different formats (e.g. voice, text) | 0.709 | |||||||
| 13.- I take and save photos from places that I like in the metaverse | 0.635 | |||||||
| 14.- I use vehicles or other means of transportation to move around the metaverse | 0.883 | |||||||
| 15.- I enjoy interacting with other people in the metaverse | 0.619 | |||||||
| 16.- I interact daily in the metaverse | 0.713 | |||||||
| 17.- I overcame the difficulties of access and use thanks to my teachers | 0.699 | |||||||
| 18.- I overcame the difficulties of access and use thanks to my colleagues | 0.769 | |||||||
| 19.- I overcame the difficulties of access and use thanks to the help of tutorials | 0.714 | |||||||
| 20.- I overcame the difficulties of access and use thanks to personal research | 0.881 | |||||||
| 21.- I overcame the difficulties of access and use thanks to the suggestions and help services of the system | 0.701 | |||||||
| 22.- The teachers used appropriate methodologies to encourage the creative activity of the metaverse | 0.634 | |||||||
| 23.- The tutorials were useful to resolve doubts | 0.661 | |||||||
| 24.- The tutorials were useful to improve control of the system as a user | 0.315 | |||||||
| 25.- The interaction in the metaverse has modified my habits of organizing free time | 0.771 | |||||||
| 26.- The interaction in the metaverse has modified my sleeping habits | 0.827 | |||||||
| 27.- I would like to carry out academic activities in the metaverse | 0.609 | |||||||
| 28.- Interacting through my avatar has allowed me to relate easily to other people | 0.901 | |||||||
| 29.- Interacting with other people in the metaverse has allowed me to relate better in real life | 0.802 | |||||||
| 30.- Interacting with other people in the metaverse has allowed me to feel freer to express my ideas in real life | 0.667 | |||||||
| 31.- Interacting from my virtual avatar has motivated me to learn in the metaverse | 0.747 | |||||||
| 32.- Interacting with other avatars has been familiar to me due to real-life experiences | 0.418 | |||||||
| 33.- Interacting in the metaverse has increased my feeling of belonging to the training community | 0.613 | |||||||
| 34.- Attending classes in the metaverse has been interesting to me | 0.775 | |||||||
| 35.- Solving problems in the metaverse has been interesting to me | 0.893 | |||||||
| 36.- Exchanging ideas among colleagues in the metaverse has helped me strengthen my digital competence | 0.704 | |||||||
| 37.- Interacting in the metaverse has improved my ability to communicate in digital environments | 0.691 | |||||||
| 38.- Interacting with colleagues in the metaverse has fostered my socialization | 0.887 | |||||||
| 39.- The visual communication used during the course was motivating | 0.613 | |||||||
| 40.- The three-dimensional visual effect (3D) of the environment and the characters of the metaverse have favored my motivation to interact | 0.730 | |||||||
| 41.- Communicating through chat in the metaverse was interesting to me | 0.771 | |||||||
| 42.- Communicating with a voice in the metaverse was interesting to me | 0.615 | |||||||
| 43.- I prefer class material that is really challenging to learn new things | 0.611 | |||||||
| 44.- I prefer class material that arouses my curiosity, even if it is difficult to learn | 0.802 | |||||||
| 45.- The most satisfying thing for me is to understand the contents in the best possible way | 0.838 | |||||||
| 46.- When I have the opportunity to choose, I choose to do tasks in which I can learn even if they do not guarantee a good grade | 0.673 | |||||||
| 47.- Getting a good grade in class is the most satisfying thing for me at the moment | 0.669 | |||||||
| 48.- The most important thing for me at the moment is to improve the average of my grades, so my main concern is to get good grades | 0.715 | |||||||
| 49.- If I can, I want to get better grades than most of the other students | 0.728 | |||||||
| 50.- I want to do well in class because it is important to show my ability to my family and social environment (friends, teachers or other people) | 0.822 | |||||||
| 51.- Immersion in the metaverse has favored my ability to learn conceptual content (facts, data, and concepts) | 0.649 | |||||||
| 52.- Immersion in the metaverse has favored my ability to learn procedural content (know how) | 0.616 | |||||||
| 53.- Immersion in the metaverse has favored my ability to learn attitudinal content (values, attitudes, and norms) | 0.673 | |||||||
| 54.- Immersion in the metaverse has improved my level of communicative competence | 0.782 | |||||||
| 55.- Immersion in the metaverse has improved my ability to process information and digital competence | 0.648 | |||||||
| 56.- Immersion in the metaverse has improved my level of social and civic competence | 0.793 | |||||||
| 57.- Immersion in the metaverse has improved my level of competence in autonomy and personal initiative | 0.662 | |||||||
| 58.- Immersion in the metaverse has improved my level of competence in learning to learn | 0.672 | |||||||
| 59.- Immersion in the metaverse has allowed me to know and put into practice the proper use of multimedia communication tools within the teaching and learning process | 0.701 | |||||||
| 60.- Immersion in the metaverse has favored my academic performance | 0.661 | |||||||
| 61.- I know and use some conventions or rules of written and iconic communication among Internet users | 0.886 | |||||||
| 62.- I try to write my messages respectfully and without offending others | 0.664 | |||||||
| 63.- I am aware that there are dangers arising from the use of the Internet | 0.748 | |||||||
| 64.- I define and characterize the different inappropriate uses of the Internet and its negative effects | 0.619 | |||||||
| 65.- I am able to identify and act in any case of cyber-bullying that arises | 0.649 | |||||||
| 66.- I feel discomfort and rejection towards any type of discrimination, harassment or inappropriate use of technology | 0.705 | |||||||
| 67.- I know the basic rules of education when I communicate with my peers | 0.667 | |||||||
| 68.- I avoid using words or images that may be offensive or misunderstood by the recipients of my messages | 0.793 |
Note: Dimension 1. Interaction with technology; Dimension 2. Intrinsic possibilities; Dimension 3. Accessibility and management; Dimension 4. Interaction; Dimension 5. Interest; Dimension 6. Motivation; Dimension 7. Learning; Dimension 8. Netiquette.
For the confirmatory factor analysis (CFA), goodness-of-fit indices were collected that were adequate for the model established in the validation of the instrument. Thus, the root mean square error of approximation (RMSEA = 0.015) indicated the anticipated fit with the total population value. The standardized root mean square root (SRMR = 0.048) revealed measures of the size of the model error. The goodness-of-fit index (GFI = 0.917) and adjusted goodness-of-fit index (AGFI = 0.872) indicated the absolute best performing rates. The Normalized Fit Index (NFI = 0.914) evaluated the decrease in the χ2 statistic of the adopted model with respect to the base model. And the Comparative Fit Index (CFI = 0.973) reflected the percentage representativeness of the covariance that could be reproduced by the model.
On the other hand, the correlations between the dimensions were positive in all cases, with the correlation established between interaction-label (R = 0.221) and interaction with technology-learning (R = 0.201) and motivation-learning (R = 0.209) being significant.
The factor weights of each of the dimensions showed the adequacy of each item with respect to the dimension of which they are part. Additionally, the dimension "interaction with technology" was composed of items 1,2,3,3,4,5 and 6 with factor weights ranging from 0.36 to 0.89. The dimension "intrinsic possibilities" was composed of items 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16 with factor weights ranging between 0.31 and 0.79. The "accessibility and handling" dimension was composed of items 17, 18, 19, 20, 21, 22 and 23 with factor weights ranging from 0.37 to 0.86. The "interaction" dimension consisted of items 24, 25, 26, 27, 28, 29, 30 and 31 with factor weights ranging from 0.41 to 0.92. The "interest" dimension was composed of items 32, 33, 34, 35, 36, 37, 37, 38, 39 and 40 with factor weights ranging from 0.37 to 0.81. The "motivation" dimension consisted of items 41, 42, 43, 44, 45, 46, 47 and 48 with factor weights ranging from 0.39 to 0.82. The "learning" dimension comprised of items 49, 50, 51, 52, 53, 54, 55, 55, 56, 57 and 58 with factor weights ranging from 0.37 to 0.81. The "netiquette" dimension consisted of items 59, 60, 61, 62, 63, 64, 65 and 66 with factor weights ranging between 0.32 and 0.71 (Figure 2 and Table 4).
Figure 2.
Confirmatory factor analysis.
Table 4.
Interdimensional correlation index of the confirmatory factor analysis.
| Dimensional Correlations | Correlation index |
|---|---|
| Interaction with technology-accessibility and management | r = .167 |
| Interaction with technology-interaction | r = .091 |
| Interaction with technology-interest | r = .112 |
| Interaction with technology-motivation | r = .132 |
| Interaction with technology-learning | r = .201 |
| Interaction with technology-netiquette | r = .164 |
| Intrinsic possibilities-interaction | r = .183 |
| Intrinsic possibilities-interest | r = .056 |
| Intrinsic possibilities-motivation | r = .103 |
| Intrinsic possibilities-learning | r = .117 |
| Intrinsic possibilities-netiquette | r = .128 |
| Accessibility and handling-interest | r = .102 |
| Accessibility and handling | r = .199 |
| Accessibility and driving-learning | r = .124 |
| Accessibility and handling-netiquette | r = .099 |
| Interaction-motivation | r = .155 |
| Interaction-learning | r = .138 |
| Interaction-netiquette | r = .221 |
| Interest-learning | r = .114 |
| Interest-netiquette | r = .172 |
| Motivation-netiquette | r = .101 |
3.4. Reliability analysis
Reliability was calculated using Cronbach's alpha coefficient (α) and McDonald's Omega (W), being the most commonly used indices to calculate the reliability of the instruments. The overall reliability of the instrument was acceptable (α = 0.81; W = 0.82). On the other hand, for each of the dimensions the reliability was: interaction with technology (α = 0.79; W = 0.80); intrinsic possibilities (α = 0.78; W = 0.79); accessibility and handling (α = 0. 73; W = 0.75); interaction (α = 0.71; W = 0.72); interest (α = 0.70; W = 0.71); motivation (α = 0.69; W = 0.70); learning (α = 0.67; W = 0.69); netiquette (α = 0.65; W = 0.66).
4. Discussion
The analysis of the scientific literature carried out in this research shows that the metaverse as an educational environment is still under development (Crisol-Moya et al., 2020; Diaz, 2020). This is due to the fact that research into metaverse is still at its initial and exploratory stages, finding a significant gap in the scientific literature on the subject, especially in the educational field (Baynat and Lopez, 2020; Nurhidayah et al., 2020; Park et al., 2021). Thus, the proliferation of studies that analyze this new socio-educational reality on the rise is essential to exploit all its potentials for interactivity in virtual environments that are controlled and adaptable to the needs of students (Abeles, 2007; Garrido-Inigo and Rodriguez-Moreno, 2013; Reyes, 2020; Schaf et al., 2012; Schlemmer et al., 2009; Tarouco et al., 2013).
This research has addressed the need for valid and reliable tools to assess educational practices in the metaverse. Consequently, the design, validation, and reliability of an instrument to assess the educational experiences carried out in the metaverse from a holistic perspective has been conducted. The analysis of the scientific literature reflected the existence of research that has addressed this topic of study in a preliminary way, focusing on various instruments for data collection (Clark, 2009; Díaz et al., 2020; García, 2011; Jaffurs, 2011; Nurhidayah et al., 2020; Park et al., 2021; Reyes, 2020; Schlemmer et al., 2009).
Although the evaluation of learning is a common topic among questionnaires validated and published in the scientific literature, great difficulties have been found in obtaining instruments that specifically analyze educational practices in the metaverse. The educational metaverse is a field of study in an exploratory phase that requires new contributions to lay the foundations for a field of research that is still expanding.
Some instruments propose a dimensional structuring similar to the one proposed in this work (Peña, 2014; Tarouco et al., 2013), especially in the motivational dimension developed in the MSLQ (Segura-Robles et al., 2021). The interaction with technology dimension has been addressed as a fundamental aspect by authors such as Erturk and Reynolds (2020) and Estudante and Dietrich (2020), stating that digital resources are not well designed and adapted to the metaverse and consequently, their use must be optimally analyzed. Authors such as González-Crespo et al. (2013), Kanematsu et al. (2010), and Lucas et al. (2013) addressed the importance of analyzing the dimension related to intrinsic possibilities in their research. These authors have verified that, despite the adaptation in time, design, and practice of the metaverse, its intrinsic possibilities are enormous, achieving a higher quality in learning and allowing endless possibilities in educational contexts. In a similar line, addressing the learning dimension, authors such as Barry et al. (2015) and Kanematsu et al. (2010) considered that the use of the metaverse achieved a better quality of learning based on its enormous potentialities derived from the multitude of utilities that virtual learning spaces present and the sensation of face-to-face assistance with a high degree of immersion.
Starting from the proposal to analyze the interest and motivation dimension as the backbone of the instrument addressed in this study, this improvement in learning in the metaverse leads to increased student motivation and engagement and enriches traditional learning by providing experiences that would otherwise be impossible. This fact is also supported by the findings of previous studies carried out by Barry et al. (2015) and Erturk and Reynolds (2020). The dimension related to interaction is also especially important as authors such as Estudante and Dietrich (2020) and Erturk and Reynolds (2020) highlighted the significance of new mobile devices in the daily lives of students. Additionally, they pointed out that despite encouraging a greater degree of personalization and facilitating virtual interaction it can inevitably generate a reduction in students’ team skills and communication.
On the other hand, in the line supported by the dimensional proposal of the instrument of this research, authors such as Belei et al. (2011) and Tlili et al. (2022) highlighted the importance of the accessibility and management dimension as a fundamental element for a suitable implementation of the educational metaverse, by verifying that at a general level, students do not have enough knowledge of technology to apply what is addressed in the classroom.
5. Conclusions
Based on this review, content validity, expert judgment validity, construct validation and reliability analyses were carried out, with the instrument obtaining positive results in all cases. In accordance with the systematization established for the validation of the instrument, the dimensions that make up the instrument were delimited from the filtering and analysis of the documents reported in the scientific literature according to the results obtained from the search equation and the PRISMA protocol. The application of the Delphi method made it possible to adjust the concordance index of each item in relation to the criteria of clarity, coherence and relevance, optimizing the instrument based on the assessments of researchers with experience in the subject, in the educational field and in the validation of instruments in this field. In terms of construct validity, positive values were obtained which supported the relevance of the exploratory factor analysis. The community analysis and the principal component analysis with Quartimax rotation with Kaiser showed positive values, and the factor structure was adequately explained. The correlations between the dimensions were positive in all cases and the factor weights of each of the dimensions showed the adequacy of each item with respect to the dimension of which it is part. Finally, the reliability analysis of the instrument based on Cronbach's alpha and McDonald's Omega coefficient yielded acceptable results for overall reliability as well as for each of the dimensions that make up the instrument.
This work presents a set of practical implications related to its added value and its prospective, having generated a validated and reliable instrument for the evaluation of educational experiences carried out in the metaverse. On a theoretical level, this study represents an increase in the scientific literature on educational metaverse, a field of research that has yet to be explored and exploited. In this way, it will make it possible to adapt teaching and learning processes to the new times and to the rapid pace at which technology is evolving, taking advantage of all its techno-pedagogical potentials. On a practical level, this work culminates in the creation of an instrument that has a double intrinsic purpose. On the one hand, it will encourage the development of educational practices in the metaverse. The dissemination of the metaverse as a learning space will allow students with attendance problems related to events that prevent them from attending the face-to-face class to be adapted. The metaverse will also allow the exploitation of digital resources that facilitate attention to diversity: instant translation, generation of subtitles, individualized use of curricular content or accessibility for students with specific difficulties. On the other hand, such instructional actions deployed in this virtual environment can be effectively assessed due to the design of this questionnaire. A tool has been designed that provides a holistic perspective by covering a broad dimensional spectrum in which the dimensions most in demand and used by the teaching community in their usual educational evaluation practices are integrated. Consequently, this study has led to the development of a comprehensive assessment tool at the service of educators or any institution interested in this field of study.
The contextual restriction derived from the generalization of the results and the exploratory phase of the research field can be mentioned as the main limitations of this study. Specifically, the research is limited by its contextual restriction, since this questionnaire has been initially validated in a Spanish context and adolescent population. Therefore, for the tool to maintain its relevant psychometric properties of validity and reliability, its application is limited to this group. This contextual limitation creates an opportunity as a line of future research, through which we intend to carry out the process of translation and adaptation to other contexts to internationalize the tool and contribute to the study of educational experiences carried out in the metaverse in different regions and cultures. Regarding the existing extrinsic limitations in the potential use of this instrument as an evaluation tool, it is necessary to highlight the complexity of the metaverse and the exploratory phase in which we find ourselves within this field of study. These limitations overlap with the need for broader holistic and interdisciplinary perspectives that approach the analysis of the metaverse and its particularities from a global and integrative perspective. The metaverse is a field of study with multiple uses in various fields, which requires the integration of various study disciplines so that research can be carried out in a unified way that complements its results from different perspectives. This complexity also requires a detailed analysis of the feasibility, effectiveness, and suitability of the various methodological strategies required to address such a complex issue, as well as the creation and design of new metaverse-specific methodologies arising from the principles of the new paradigms of learning spaces.
Declarations
Author contribution statement
Jesús López-Belmonte: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.
Santiago Pozo-Sánchez: Performed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.
Georgios Lampropoulos: Contributed reagents, materials, analysis tools or data; Wrote the paper.
Antonio-José Moreno Guerrero: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.
Funding statement
This work was supported by the GOAL! Teaching Methodology for the Transformation of Learning in the METAVERSE (reference 22-115).
Data availability statement
Data included in article/supp. material/referenced in article.
Competing interest statement
The authors declare no conflict of interest.
Additional information
No additional information is available for this paper.
Appendix 1. METAEDU questionnaire in its original language for the Spanish context.
| Ítems | Escala de valoración |
|||||
|---|---|---|---|---|---|---|
| 0 | 1 | 2 | 3 | 4 | 5 | |
| Interacción con la tecnología | ||||||
| 1.- Juego a juegos de rol en red en mi día a día | ||||||
| 2.- Interactúo en mundos virtuales cotidianamente | ||||||
| 3.- Uso redes sociales a diario | ||||||
| 4.- Realizo videoconferencias con otras personas | ||||||
| 5.- Recibo una enseñanza virtual mediante plataformas | ||||||
| 6.- Trabajo de forma colaborativa mediante plataformas de gestión de contenidos o nube. | ||||||
| Posibilidades intrínsecas | ||||||
| 7.- Me teletransporto a otros lugares del metaverso | ||||||
| 8.- Modifico mi apariencia personal en el metaverso | ||||||
| 9.- Suelo cambiar mi forma de vestir en el metaverso. | ||||||
| 10.- Manejo distintos objetos que dispongo en mi inventario personal | ||||||
| 11.- Abro cajas que me encuentro para ampliar mi inventario de objetos | ||||||
| 12.- Me comunico con otras personas del metaverso en distintos formatos (voz, texto) | ||||||
| 13.- Hago fotos y las guardo de lugares que me gustan del metaverso | ||||||
| 14.- Monto en vehículos u otros medios de transporte para desplazarme por el metaverso. | ||||||
| 15.- Disfruto interactuando con otras personas en el metaverso | ||||||
| 16.- Interactúo diariamente en el metaverso | ||||||
| Accesibilidad y manejo | ||||||
| 17.- Superé las dificultades de acceso y uso gracias a mis tutores | ||||||
| 18.- Superé las dificultades de acceso y uso gracias a mis colegas | ||||||
| 19.- Superé las dificultades de acceso y uso gracias a la ayuda de tutoriales | ||||||
| 20.- Superé las dificultades de acceso y uso gracias a través de la investigación personal | ||||||
| 21.- Superé las dificultades de acceso y uso gracias a las sugerencias y servicios de ayuda del sistema | ||||||
| 22.- Los tutores utilizaron metodologías adecuadas para fomentar la actividad creativa del metaverso | ||||||
| 23.- Las tutorías fueron de utilidad para solventar las dudas | ||||||
| 24.- Las tutorías fueron de utilidad para mejorar el control del sistema como usuario | ||||||
| Interacción | ||||||
| 25.- La interacción en el metaverso ha modificado mis hábitos de organización de tiempo libre | ||||||
| 26.- La interacción en el metaverso ha modificado mis hábitos de sueño | ||||||
| 27.- Me gustaría realizar actividades académicas en el metaverso | ||||||
| 28.- Interactuar por medio de mi avatar me ha permitido relacionarme con facilidad con otras personas | ||||||
| 29.- Interactuar con otras personas en el metaverso me ha permitido relacionarme mejor en la vida real | ||||||
| 30.- Interactuar con otras personas en el metaverso me ha permitido sentirme más libre para expresar mis ideas en la vida real | ||||||
| 31.- Interactuar desde mi avatar virtual me ha motivado a aprender en el metaverso | ||||||
| 32.- Interactuar con otros avatares me ha resultado familiar a las experiencias de la realidad | ||||||
| 33.- Interactuar en nuestro metaverso ha aumentado tu sensación de pertenencia a la comunidad formativa | ||||||
| Interés | ||||||
| 34.- Asistir a clase en el metaverso me ha resultado de interés | ||||||
| 35.- Resolver problemas en el metaverso me ha resultado de interés | ||||||
| 36.- Intercambiar ideas entre colegas en el metaverso me ha ayudado a fortalecer mi competencia digital | ||||||
| 37.- Interactuar en el metaverso ha mejorado mi capacidad de comunicación en entornos digitales | ||||||
| 38.- Interactuar con los colegas en el metaverso ha fomentado mi sociabilización | ||||||
| 39.- La comunicación visual utilizada durante el curso fue motivadora | ||||||
| 40.- El efecto visual tridimensional (3D) del ambiente y de los personajes del metaverso ha favorecido tu motivación por interactuar | ||||||
| 41.- Comunicarte a través del chat en el metaverso te resultó de interés | ||||||
| 42.- Comunicarte con voz en el metaverso te resultó de interés | ||||||
| Motivación | ||||||
| 43.- Prefiero material de clase que realmente sea un reto para poder aprender cosas nuevas | ||||||
| 44.- Prefiero material de clase que despierte mi curiosidad, aunque sea difícil de aprender | ||||||
| 45.- Lo más satisfactorio para mí es entender los contenidos de la mejor forma posible | ||||||
| 46.- Cuando tengo oportunidad de elegir, elijo hacer tareas en las que puedo aprender aunque no garanticen una buena nota | ||||||
| 47.- Obtener una buena nota en clase es lo más satisfactorio para mí en este memento | ||||||
| 48.- Lo más importante para mí en este memento es mejorar la media de mis notas, así que mi principal preocupación es conseguir una buena nota | ||||||
| 49.- Si puedo, quiero obtener mejores notas que la mayoría de los otros estudiantes | ||||||
| 50.- Quiero hacerlo bien en clase porque es importante mostrar mi habilidad a mi entorno (familia, amigos, profesores u otras personas) | ||||||
| Aprendizaje | ||||||
| 51.- La inmersión en el metaverso ha favorecido mi capacidad para aprender contenidos conceptuales (hechos, datos y conceptos) | ||||||
| 52.- La inmersión en el metaverso ha favorecido mi capacidad para aprender contenidos procedimentales (saber hacer) | ||||||
| 53.- La inmersión en el metaverso ha favorecido mi capacidad para aprender contenidos actitudinales (valores, actitudes y normas) | ||||||
| 54.- La inmersión en el metaverso ha mejorado mi nivel de competencia comunicativa | ||||||
| 55.- La inmersión en el metaverso ha mejorado mi capacidad de tratamiento de la información y de competencia digital | ||||||
| 56.- La inmersión en el metaverso ha mejorado mi nivel de competencia social y ciudadana | ||||||
| 57.- La inmersión en el metaverso ha mejorado mi nivel de competencia en autonomía e iniciativa personal | ||||||
| 58.- La inmersión en el metaverso ha mejorado mi nivel de competencia para aprender a aprender | ||||||
| 59.- La inmersión en el metaverso me ha permitido conocer y poner en práctica el buen uso de las herramientas de comunicación multimedia dentro del proceso de enseñanza y aprendizaje | ||||||
| 60.- La inmersión en el metaverso ha favorecido mi rendimiento académico | ||||||
| Netiqueta | ||||||
| 61.- Conozco y utilizo alguna convención o regla de comunicación escrita e icónica entre usuarios de internet | ||||||
| 62.- Procuro escribir mis mensajes de forma respetuosa y sin ofensas hacia los demás | ||||||
| 63.- Soy consciente de que existen peligros derivados del uso de internet | ||||||
| 64.- Defino y caracterizo los distintos usos inadecuados de internet y sus efectos negativos | ||||||
| 65.- Soy capaz de identificar y actuar ante algún caso que se presente de ciberacoso | ||||||
| 66.- Siento malestar y rechazo hacia cualquier tipo de discriminación, acoso o uso inadecuado de la tecnología | ||||||
| 67.- Conozco las reglas básicas de educación cuando me comunico con mis iguales | ||||||
| 68.- Evito utilizar palabras o imágenes que puedan ser ofensivas o malinterpretadas por los destinatarios de mis mensajes | ||||||
References
- Abeles T.P. Education unbound. Horizon. 2007;15(4):199–203. [Google Scholar]
- Arcila J.B.P. Metaversos para el máster iberoamericano en educación en entornos virtuales. Etic@net. 2015;14(2):227–248. [Google Scholar]
- Ato M., López-García J., Benavente A. Un sistema de clasificación de los diseños de investigación en psicología. An. Psicolog. 2013;29(3):1038–1059. [Google Scholar]
- Barry D.M., Ogawa N., Dharmawansa A., Kanematsu H., Fukumura Y., Shirai T., Yajima K., Kobayashi T. Evaluation for students’ learning manner using eye blinking system in Metaverse. Procedia Comput. Sci. 2015;60:1195–1204. [Google Scholar]
- Baynat M.M.E., López S.M. La machinima entrevista de trabajo/entretien d’embauche: une ressource numérique pour l’apprentissage de la langue française en contexte universitaire. Synerg. Eur. 2020;15:163–179. [Google Scholar]
- Bayne S. What's the matter with ‘technology-enhanced learning. Learn. Media Technol. 2015;40(1):5–20. [Google Scholar]
- Belei N., Noteborn G., De Ruyter K. It’s a brand new world: teaching brand management in virtual environments. J. Brand Manag. 2011;18(8):611–623. [Google Scholar]
- Caeiro S., Leal Filho W., Jabbour C., Azeiteiro U. Springer International Publishing; 2013. Sustainability Assessment Tools in Higher Education Institutions: Mapping Trends and Good Practices Around the World. [Google Scholar]
- Calongne C., Sheehy P., Stricker A. The Immersive Internet. Palgrave Macmillan UK; 2013. Gemeinschaft identity in a gesellschaft metaverse; pp. 180–191. [Google Scholar]
- Castronova E. Virtual worlds: a first-hand account of market and society on the cyberian frontier. SSRN Electron. J. 2001 [Google Scholar]
- Chukwunonso A.G., Njoku J.N., Lee J.M., Kim D.S. Security in metaverse: a closer look. Proc. Korea Inform. Proc. Soc. Conf. 2022:199–200. [Google Scholar]
- Clark M.A. Genome island: a virtual science environment in second life. Innovate J. Online Educ. 2009;5(6) [Google Scholar]
- Collins C. Looking to the future: higher education in the metaverse. Educ. Rev. 2008;43(5):51–63. [Google Scholar]
- Crisol-Moya E., Romero-López M.A., Caurcel-Cara M.J. Active methodologies in higher education: perception and opinion as evaluated by professors and their students in the teaching-learning process. Front. Psychol. 2020;11 doi: 10.3389/fpsyg.2020.01703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Díaz J.E.M., Saldaña C., Avila C. Virtual world as a resource for hybrid education. Int. J. Emerg. Technol. Learning (iJET) 2020;15(15):94–109. [Google Scholar]
- Dionisio J.D.N., Burns W.G., III, Gilbert R. 3D virtual worlds and the metaverse: current status and future possibilities. ACM Comput. Surv. 2013;45(3):1–38. [Google Scholar]
- Diaz J.E.M. Virtual world as a complement to hybrid and mobile learning. Int. J. Emerg. Technol. Learning (iJET) 2020;15(22):267. [Google Scholar]
- Dorantes J.A., Hernández J.S., y Tobón S. Juicio de expertos para la validación de un instrumento de medición del síndrome de Burnout en la docencia. Ra Ximhai. 2016;12(6):327–346. [Google Scholar]
- Erturk E., Reynolds G.B. International Conference on E-Learning. 2020. The expanding role of immersive media in education; pp. 191–194. [Google Scholar]
- Escobar-Pérez J., y Cuervo-Martínez A. Validez de contenido y juicio de expertos: una aproximación a su utilización. Avan. Med. 2008;6:27–36. [Google Scholar]
- Estudante A., Dietrich N. Using augmented reality to stimulate students and diffuse escape game activities to larger audiences. J. Chem. Educ. 2020;97(5):1368–1374. [Google Scholar]
- Falchuk B., Loeb S., Neff R. The social metaverse: battle for privacy. IEEE Technol. Soc. Mag. 2018;37(2):52–61. [Google Scholar]
- Fernandez C.B., Hui P. 2022. Life, the Metaverse and Everything: an Overview of Privacy, Ethics, and Governance in Metaverse. arXiv preprint arXiv:2204.01480. [Google Scholar]
- García F.C. El uso de metaversos en el mundo educativo: gestionando conocimiento en second life. REDU. Rev. de Doc. Univ. 2011;8(2):147–159. [Google Scholar]
- Garrido-Inigo P., Rodriguez-Moreno F. The reality of virtual worlds: pros and cons of their application to foreign language teaching. Interact. Learn. Environ. 2013;23(4):453–470. [Google Scholar]
- González Crespo R., Escobar R.F., Joyanes Aguilar L., Velazco S., Castillo Sanz A.G. Use of ARIMA mathematical analysis to model the implementation of expert system courses by means of free software OpenSim and Sloodle platforms in virtual university campuses. Expert Syst. Appl. 2013;40(18):7381–7390. [Google Scholar]
- Hernández R., Fernández C., y Baptista P. McGraw-Hill; 2014. Metodología de la investigación. [Google Scholar]
- Jaffurs S.E. In: Learning, Teaching and Musical Identity: Voices across Cultures. Green L., editor. Indiana University Press; 2011. SIMPhonic island: exploring musical identity and learning in virtual space; pp. 295–308. [Google Scholar]
- Kanematsu H., Fukumura Y., Barry D.M., Sohn S.Y., Taguchi R. International Conference on Knowledge-Based and Intelligent Information and Engineering Systems. Springer; 2010. Multilingual discussion in Metaverse among students from the USA, Korea and Japan; pp. 200–209. [Google Scholar]
- Lee L.H., Braud T., Zhou P., Wang L., Xu D., Lin Z., Bermejo C., Hui P. 2021. All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda. arXiv preprint arXiv:2110.05352. [Google Scholar]
- Lemos A. Imagem, Visibilidade e Cultura Midiática. Livro Da XV COMPÓS. Sulina; Porto Alegre: 2007. Ciberespaço e tecnologias móveis: processos de territorialização e desterritorialização na cibercultura; pp. 277–293. [Google Scholar]
- López-Belmonte J., Moreno-Guerrero A.-J., Marín-Marín J.-A., Lampropoulos G. The impact of gender on the use of augmented reality and virtual reality in students with ASD. Educ. Knowl. Society. 2022;23:1–14. [Google Scholar]
- Lucas E.P., Benito J.C., Gonzalo O.G. 2nd International Workshop on Evidence-Based Technology Enhanced Learning. Springer; 2013. Usalsim: learning and professional practicing in a 3d virtual world; pp. 75–82. [Google Scholar]
- Moreno-Guerrero A., Soler-Costa R., Marín-Marín J., López-Belmonte J. Flipped learning and good teaching practices in secondary education. Comunicar. 2021;29(68):107–117. [Google Scholar]
- Ning H., Wang H., Lin Y., Wang W., Dhelim S., Farha F., Ding J., Daneshmand M. 2021. A Survey on Metaverse: the State-Of-The-Art, Technologies, Applications, and Challenges. arXiv preprint arXiv:2111.09673. [Google Scholar]
- Nurhidayah N.N., Halim N., Basri M. Analyzing student’s learning outcome using systemic approach. Asian EFL J. Res. Articles. 2020;27(4):230–247. [Google Scholar]
- Park S.M., Kim Y.G. A Metaverse: taxonomy, components, applications, and open challenges. IEEE Access. 2022;10:4209–4251. [Google Scholar]
- Park S., Min K., Kim S. Differences in learning motivation among bartle’s player types and measures for the delivery of sustainable gameful experiences. Sustainability. 2021;13(16):9121. [Google Scholar]
- Parra-González M.E., Segura-Robles A., Moreno-Guerrero A.J., López-Belmonte J. Elaboration and validation of the scale to measure the experience on gamification in education (EGAMEDU) J. Technol. Sci. Educ. 2022;12(1):217–229. [Google Scholar]
- Peña J. Metaverses for the master degree iberoamerican in education in virtual environments. Etic@net. 2014;14(2):227–248. [Google Scholar]
- Pereira I. O metaverso e a sua influência no posicionamento das marcas. The Trends Hub. 2022;2 [Google Scholar]
- Pozo-Sánchez S., López-Belmonte J., Moreno-Guerrero A.-J., Sola-Reche J.M., Fuentes-Cabrera A. Effect of bring-your-own-device program on flipped learning in higher education students. Sustainability. 2020;12(9):1–11. [Google Scholar]
- Reyes C.E.G. Percepción de estudiantes de bachillerato sobre el uso de metaverse en experiencias de aprendizaje de realidad aumentada en matemáticas. Pixel-Bit, Rev. de Medios y Educación. 2020;58:143–159. [Google Scholar]
- Romero-Rodríguez J.M., Kopecký K., García-González A., y Gómez-García G. Sharing images or videos of minors online: validation of the sharenting evaluation scale (SES) Child. Youth Serv. Rev. 2022;136:1–7. [Google Scholar]
- Schaf F.M., Paladini S., Pereira C.E. Proceedings of the 2012 IEEE Global Engineering Education Conference (EDUCON) 2012, April. 3D AutoSysLab prototype. [Google Scholar]
- Schlemmer E., Trein D., Oliveira C. The metaverse: telepresence in 3D avatar-driven digital-virtual worlds. @tic. Rev. Innovació Educ. 2009;2:26–32. [Google Scholar]
- Segura-Robles A., Moreno-Guerrero A.-J., Parra-González M.-E., López-Belmonte J. Adaptation and validation of the motivated strategies for learning questionnaire for Spanish adolescents. Europ. J. Inv. Health, Psychol. Educ. 2021;11(1):142–153. doi: 10.3390/ejihpe11010012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shepard L.A. The role of assessment in a learning culture. Educ. Res. 2000;29(7):4–14. [Google Scholar]
- Soler-Costa R., Lafarga-Ostáriz P., Mauri-Medrano M., Moreno-Guerrero A.-J. Netiquette: ethic, education, and behavior on interneta systematic literature review. Int. J. Environ. Res. Publ. Health. 2021;18(3):1212. doi: 10.3390/ijerph18031212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stephenson N. Bantam Books; 1992. Snow Crash: A Novel. [Google Scholar]
- Tarouco L., Gorziza B., Correa Y., Amaral E.M.H., Muller T. IEEE Global Engineering Education Conference (EDUCON) 2013. Virtual laboratory for teaching calculus: an immersive experience. [Google Scholar]
- Tlili A., Huang R., Shehata B., Liu D., Zhao J., Metwally A.H.S., Burgos D. Is Metaverse in education a blessing or a curse: a combined content and bibliometric analysis. Smart Learning Environ. 2022;9(1):1–31. [Google Scholar]
- Vaca Barahona B., Cela Ranilla J., Gallardo Echenique E.E. 2016. The Communication in Simulated Learning Environments. [Google Scholar]
- Wang Y., Su Z., Zhang N., Liu D., Xing R., Luan T.H., Shen X. 2022. A Survey on Metaverse: Fundamentals, Security, and Privacy. arXiv preprint arXiv:2203.02662. [Google Scholar]
- Zawacki-Richter O., Latchem C. Exploring four decades of research in computers & education. Comput. Educ. 2018;122:136–152. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data included in article/supp. material/referenced in article.