Abstract
Gamification is an innovative pedagogical approach to addressing problems related to social behaviour, student motivation and academic performance at different educational stages. Therefore, this research aimed to analyse its impact on the motivations and academic performances of university students. The research was carried out in the training of future teachers specialising in physical education during two academic courses. In total, 127 students participated in the study, divided into a gamified experimental group (n = 62) and a control group (n = 65). The participants completed a questionnaire to assess motivation in physical education before and after the intervention and performed a final exam to assess academic performance. The results indicated an increase in external regulation in the experimental group only. Furthermore, this group achieved significantly better academic performance. The findings of this study suggest that gamified implementation is beneficial for academic performance at the university stage, even though intrinsic motivation does not change. Furthermore, the nature of rewards or punishments, as characteristic of this pedagogical approach, could play an important role in the expected results, since external regulation increased significantly after the intervention.
Keywords: active methodologies, educational innovation, psychosocial factors, motivation, improvement of indicators, academic performance, Classcraft, sport learning
1. Introduction
Gamification has been gaining in popularity within the educational field. Its use as an active methodology has been growing in recent years by creating a novel educational approach to address problems related to social behaviour and student motivation [1,2,3,4,5]. This emerging pedagogical approach is based on the utilisation of the potential of games in order to redesign methodological and didactic elements to increase motivation and adherence to the teaching–learning process, significantly changing the social behaviours of students to ultimately improve their academic performances [6,7,8]. Gamification, an innovative solution inspired by the idea of the use of recreational elements applied in non-recreational contexts [5,9], has been integrated into an educational context from the moment such elements and recreational experiences became part of the design and planning of the educational process [10,11]. These playful elements, that gamify educational contexts, must be tangible for the students (avatars, points, levels, achievements, insignia, etc.), they must create a sense of action and progression (rules, challenges, competition, cooperation, feedback, etc.) and they must regulate the group’s social–emotional competencies (motivations, emotions, obligations, social relationships, etc.) and, consequently, modulate academic performance [6,12].
Within the area of Physical Education (PE), games have traditionally been used, since any motor, popular, traditional or sport game that is proposed in class is configured as the foundation from which to structure the entire teaching–learning process of the curricular contents [13,14,15]. However, the emerging methodological approach of gamification is even more transformative, since it proposes that the class itself, taking advantage of the fact that we are immersed in a digital society, really transforms itself into an active game with digital elements, based on role-playing games and in video games, within which both teachers and students have the opportunity to grow [4,16,17,18,19,20,21,22]. Inquiry into the possible psychological benefits and the effectiveness of the gamification approach within PE is relatively new, scarce and somewhat controversial [11,23]. Previous quantitative and qualitative research among university students show that gamified classes help create a more ideal classroom climate for learning by arousing more interest in and commitment to the subject, so that greater learning, enjoyment and performance are achieved, compared with traditional methodologies that do not use, among other things, new technologies or game-based elements [24,25,26]. However, other studies show a lower degree of motivation and academic performance in other university students when using gamified classes [27,28,29]. In the field of PE in compulsory education, various studies have been carried out that investigate the psychological effects on students using this methodology as a motivational strategy for learning curricular content specific to the area [4,6,30,31]. Some of these studies, using a mixed quantitative–qualitative design with validated and non-validated questionnaires or interviews, showed that gamification fostered students’ enjoyment and motivation, creating meaningful experiences in PE, despite increasing teacher workload [4,30,31]. Other studies, based on experimental or quasi-experimental designs with scales and validated questionnaires showed that gamification impacted positively on students’ dispositional flows and motivations, improving their basic psychological needs and academic performances at school. However, these studies pointed out that gamification is not effective per se, since other game elements could trigger different motivational outcomes [6]. As it can be understood, although the relationship between these psychological and performance aspects has been analysed, the results of these studies are not conclusive. Therefore, more rigorous research is necessary to continue evaluating the real impact of gamification on student motivation and academic performance within the scope of PE.
Motivation is an essential factor for student success. For this reason, self-determination theory (SDT) is frequently used as a macro-theory to understand the role of motivation within the educational environment [32,33,34]. In fact, this theoretical framework has been successfully used in various studies in gamified contexts within PE [4,6,35]. Essentially, this theory recognises three types of regulatory motivations for human behaviour: intrinsic motivation, based on the performance of a behaviour or activity for its inherent satisfaction; extrinsic motivation, referring to the performance of the activity by external recognition or as a means to achieve something; amotivation, that is, the absence of motivation towards the activity [32]. From highest to lowest, self-determination, intrinsic motivation, extrinsic motivation and amotivation are located on a continuum [33]. Thus, intrinsic motivation, recognised as high quality motivation in SDT, is identified with different positive effects within the educational context, such as greater ability to concentrate on the task [36] or higher levels of commitment to the subject [37]. For its part, in relation to extrinsic motivation, different forms of behaviour regulation have been established within it: external regulation, introjected regulation and identified regulation [33]. External regulation includes those behaviours regulated by external incentives, such as rewards or punishments. Introjected regulation is identified with actions associated with the expectations of personal recognition or actions aimed at avoiding certain personal feelings of guilt and anxiety. The identified regulation implies the carrying out of activities voluntarily because the individual considers them important and beneficial for him or her, even if he or she does not enjoy them. Finally, the behaviours based on amotivation refer to actions carried out with no intention of completing the action or with the relative absence of personal motivation towards that behaviour [32]. Gamification has been associated with improved task achievement and higher academic performance by fostering changes in different manifestations of student motivation from a SDT approach [8,38]. In this sense, gamification creates an adequate gamified atmosphere which impacts positively on students’ learning motivations and dispositional flows, thereby supporting their basic psychological needs and ultimately improving their academic performances [6,39,40]. Thus, it could be suggested that this methodological approach might be incorporated into PE classes when looking for an improvement in such psychological and learning aspects in students [41,42]. However, so far there have been few studies that empirically support the effectiveness of gamification in student motivation and performance, so new studies are needed to confirm such claims [43].
Despite the fact that PE is usually a motivating and fun subject for most students, previous studies have shown that quite a few students, especially in the adolescent period, feel amotivation towards the subject, considering it even frustrating or embarrassing because, on many occasions, teachers do not consider the socio–emotional and psychological needs of students in a specific period of pre-adolescent and adolescent development [44,45,46,47]. Based on this, it is reasonable to propose that one of the main challenges for PE teachers would be to provide sufficient motivation for students in their classes, incorporating new methodological approaches to meet the psycho–emotional and social needs of the students. In this sense, gamification is being configured as a key tool to positively influence the different manifestations of motivation, the classroom climate and school performance [4,5,6]. However, it is evident that, currently, empirical research on the use of gamification within the context of PE is still scarce. Additionally, it is essential to propose interventions using these new active methodologies with university students who will be future teachers, in order to determine how it affects these socio–emotional aspects and so that they experience the use of gamification as a pedagogical resource using information and communication technologies (ICT).
For all of the above, the present study was proposed, the primary objective of which was to analyse the impact that the implementation of gamification has on motivation in university students. The second objective was to analyse and compare their academic performances in the subject, evaluating their learning through an objective test of knowledge evaluation. Consequently, it was initially hypothesised that:
Hypothesise (H1).
The group of students experiencing gamification would increase their intrinsic motivation compared to the control group.
Hypothesise (H2).
The group of students experiencing gamification would increase their external regulation compared to the control group.
Hypothesise (H3).
The group of students experiencing gamification would decrease their amotivation compared to the control group.
Hypothesise (H4).
The experimental group would obtain higher scores in the final exam compared to the control group.
2. Materials and Methods
The research was carried out within the context of physical education teacher education (PETE), in the subject “Physical activity in the natural environment”. This subject falls within the curriculum leading to obtaining the “Degree of Primary Education Teacher”, with reference to the specialty of PE at the University of Alicante (Spain) during two academic courses in 2018 and 2019. The present investigation is based on a quasi-experimental design and with a non-equivalent control group [48]. To contrast the hypotheses presented, two groups of students in the subject were assigned to the control group, CON, and two other groups of students to the experimental group, GAM. The class schedule was 3:00−5:00 p.m. for the first group and 5:00−7:00 p.m. for the second, along the two academic years. The sample was counterbalanced according to the academic course (GAM vs. CON; CON vs. GAM), avoiding possible bias related to the different schedules. The class groups were established by the university itself, following general criteria, such as ratio of students by group and gender.
2.1. Sample
Initially 212 university students participated (Table 1). After applying the exclusion criteria, a final sample of 127 students remained (73 boys: 57.5% and 54 girls: 42.5%; average age: 22 ± 3.5 years). These criteria were: (1) regular attendance of class and being evaluated through an ongoing assessment process, that is ≥ 80% of the lessons (CON; n = 29, GAM; n = 34); (2) not completing the questionnaires pre or post (CON; n = 11, GAM; n = 8); (3) having an injury (CON; n = 2, GAM; n = 1). All the participants were informed about the study, approved their participation and agreed to the publishing of the results anonymously after signing a confidentiality agreement.
Table 1.
Descriptive data of the sample (n) regarding year, gender and treatment.
| Academic Year | Gender | Treatment | Total | |
|---|---|---|---|---|
| Control | Gamification | |||
| 2018/2019 | Male | 11 | 14 | 25 |
| Female | 14 | 17 | 31 | |
| Total | 25 | 31 | 56 | |
| 2019/2020 | Male | 30 | 18 | 48 |
| Female | 10 | 13 | 23 | |
| Total | 40 | 31 | 71 | |
2.2. Instruments
2.2.1. Self-Determined Motivation
The questionnaire on Motivation in Physical Education Classes (CMEF) was used to assess motivation [49]. This questionnaire was applied twice: prior to intervention (pre-test) and after intervention (post-test). In the pre-test, the questionnaire was applied before the project explanation, as shown in Figure 1, and therefore before performing the group distribution (GAM and CON), thus avoiding possible biases in the sample. The main statement was adapted to the context of the subject: “I practice physical activity in the natural environment because”. This questionnaire consisted of 20 items, grouped into five factors, which measured intrinsic motivation (e.g., because it is fun), identified regulation (e.g., because I can learn skills that I could use in other areas of my life), introjected regulation (e.g., because it is what I must do to feel good), external regulation (e.g., because it is seen as good by the teacher), and amotivation (e.g., I do not understand why we should do it). These items had a closed response option, following a Likert scale from 1 to 5, with 1 (= totally disagree) to 5 (= totally agree). CMEF was shown to be an excellent fit to the data (CFI = 0.96; TLI = 0.95; GFI = 0.95; SRMR = 0.04; RMSEA = 0.05), and the scale showed satisfactory internal consistency (α > 0.70) [49].
Figure 1.
Project explanation and pre-test questionnaire (CMEF-pre).
2.2.2. Final Exam
One week after the practical intervention, a theoretical final exam was given to evaluate the academic performance. This exam consisted of a test of 40 multiple choice questions related to the contents studied (identical for both treatment groups), with five possible predetermined answers to choose from. All questions were of equal value. The rating was calculated using the following equation: Rating = Hits × 0.25.
2.3. Procedure
A five-week programme with a total of 30 h was performed, as shown in Table 2. During the first week, the questionnaire (CMEF-pre) was administered to the students and an initial class was held with all the participants to explain the operation of the classes for the following weeks so that, in the case of the experimental group, the students were familiar with the completion of the different questionnaires and the use of the ClassCraft® interface (Classcraft Studios Inc., New York, NY, USA). During weeks two, three and four, the students received different practical classes comprising a total of over 18 h (orienteering, climbing, aquatic skills and natural gymnastics), as shown in Figure 2. Both the GAM and CON groups underwent exactly the same grouping and content learning activities, except for the familiarisation lessons with the gamified resource ClassCraft®, which was done as homework for the GAM group. Additionally, the teaching style followed the same criteria for both groups, since the entire intervention programme was carried out by the same teacher (the main author), in order to guarantee that the intervention programme was similarly applied within each group (GAM vs. CON). In the last week (week five), after the practical intervention, the questionnaire (CMEF-post) was administered again. Finally, the participants took the final exam.
Table 2.
Research design outline.
| Weeks | Class | Content | Time (min) | Group | Place |
|---|---|---|---|---|---|
| Week 1 | 1 | Motivation questionnaire (pre) | 10 | Both | Class |
| Project explanation | 100 | Both | Class | ||
| HW | ClassCraft® familiarisation 1 | 120 | GAM Only | Class | |
| 2 | Introduction to subject content | 110 | Both | Class | |
| Week 2 | 3 | Orienteering (Introduction to symbology) | 75 | Both | Campus |
| 4 | Orienteering (micro-sprint) | 75 | Both | Campus | |
| 5 | Climbing (clothes and material) | 75 | Both | Climbing wall | |
| 6 | Orienteering (relay) | 75 | Both | Campus | |
| 7 | Aquatic Skills (familiarisation and individual activities with flotation material) | 50 | Both | Swimming pool | |
| Week 3 | 8 | Adventure raid (adapted activities) | 75 | Both | Campus |
| 9 | Orienteering (score) | 75 | Both | Campus | |
| 10 | Climbing (Boulder) | 75 | Both | Climbing wall | |
| 11 | Orienteering (normal race) | 75 | Both | Campus | |
| 12 | Aquatic skills (individual and group activities with and without flotation material) | 50 | Both | Swimming pool | |
| Week 4 | 13 | Orienteering (drawing maps) | 75 | Both | Campus |
| 14 | Natural gymnastics (moving on four legs and jumping) | 60 | Both | Campus | |
| 15 | Natural gymnastics (balance, flexibility and turns) | 60 | Both | Campus | |
| 16 | Climbing (knots and rapel) | 75 | Both | Climbing wall | |
| 17 | Natural gymnastics (complex game without material) | 75 | Both | Campus | |
| Week 5 | 18 | Motivation questionnaire (post) | 10 | Both | Class |
| Final exam | 110 | Both | Class | ||
| 19 | Presentation of results and final discussion | 110 | Both | Class |
Note: HW = Homework.
Figure 2.
Different lessons carried out during the intervention.
For the development of the classes with the experimental group, the Classcraft® educational platform (https://www.classcraft.com) was used. This tool includes a gamified and collaborative learning methodology in the educational proposal. All the participants had their own personal and non-transferable account, which gave them access to create their personalised avatar. With the choice of this avatar, the students could choose between one of three characters: Wizard, Warrior or Healer. Each of these characters had different powers which could help their team or clan. Each team was made up of male and female, and had a maximum of 6 components, which had to include all existing roles. In addition, all the participants had to sign the hero’s pact, a commitment to accepting the rules and decisions of the Grandmaster (the teacher).
Through this gamification tool, it is possible to evaluate certain behaviours or conducts linked to the objectives of the subject. These behaviours define progress through the achievement of points of a different nature: points of experience, health, power, etc., as shown in Table 3.
Table 3.
Equivalences between behaviours and gamified points.
| Behaviours | (+) Points of Experience | (-) Health Points | ||
| +135 Xp | Class attendance | −10 Hp | Arriving to class late and/or wearing inappropriate clothing | |
| +270 Xp | Giving constructive criticism to improve work sessions | −20 Hp | Any disruptive behaviour that prevents the class from developing normally | |
| +540 Xp | Actively collaborating in group work | −30 hp | Lack of respect for teachers, classmates or materials. | |
| +1080 Xp | Internalising desirable behaviours for other colleagues | |||
Each character has strengths and weaknesses and different powers. On the one hand, the healer is suggested or chosen by students who like to help others and can heal by restoring their own health points and those of other members of the team or clan. On the other hand, the warrior has a more offensive character, and therefore is more vulnerable to losing health points more easily. Warriors can take away damage from others and heal themselves. Finally, the magician helps other team members by granting action points and is usually chosen by students who do not usually lose health points easily, since they have less of those points compared to the other characters. Finally, for the development of the classes with the gamified experimental group, the following equivalences shown in the table below were established between behaviours and points, both positive and negative. The equivalences between points and rewards (powers) or punishments (sentences) are shown below in Table 4.
Table 4.
Equivalences for rewards and punishments and gamified points.
| Rewards/Punishments | (+) Academic Powers | (-) Judgments * | |
| 1080 Xp + 30 Ap | Five extra minutes to finish the theoretical exam of the subject | Five minutes less to finish the theoretical exam on the subject | |
| 1080 Xp + 30 Ap | The teacher discards 50% of multiple non-true answers in a test question | Editing a short video of the subject | |
| 1080 Xp + 30 Ap | The teacher points to the correct answer in a test question | Delivery of a didactic unit before December | |
| Delivery of a module in a school before the month of December | |||
Note: * The sentences are applied when the students run out of HP points.
2.4. Data Analyses
All continuous variables in the data set were subjected to a normality test (Kolmogorov–Smirnov). The data were further subjected to a chi-square analysis and univariate statistical analysis for non-parametric samples, specifically the Mann–Whitney U test, to assess the differences between the groups (GAM vs. CON) on two occasions: pre and post intervention. Next, the Wilcoxon test was applied to observe the intra-group differences (pre vs. post). The level of significance was established at p < 0.05 in all cases. The statistical programmes Statistics Product and Service Solutions (IBM® SPSS® Statistics Version 24.0.0.0) (International Business Machines Corp., Madrid, Spain) and Microsoft Excel® in its 2016 version (Microsoft Corp., Redmond, WD, USA) were used.
3. Results
3.1. Sample Equivalence
Firstly, a Mann–Whitney U test found no significant differences between the groups in terms of age (Z = −0.200; p = 0.841) and attendance (Z = −0.354; p = 0.723). A chi-square analysis revealed no differences in the distribution by gender (χ2 = 1.706; p = 0.191).
3.2. Motivation
Before submitting this results section, it was reported that the values represented by the motivational variables assessed were calculated from the sum of the items of the questionnaire (e.g., intrinsic motivation value = item 1 + item 6 + item 11 + item 16), where the minimal result could be 4 and the maximal result could be 20 in each of the variables measured.
The different groups did not show initial differences in the motivational variables evaluated, thus, initially the groups were considered homogeneous, as shown in Figure 3.
Figure 3.
Box diagram of the motivational variables measured in a questionnaire CMEF (pre-test). Note: GAM = experimental group; CON = control group; IM = Intrinsic motivation; IR = Identified Regulation; INR = Introjected Regulation; ER = External regulation; AM = Amotivation.
On the one hand, no significant differences were observed, as shown in Table 5, regarding pre-test and post-test comparisons (p < 0.05) in either of the two educational groups, GAM vs. CON, except for the variable of external regulation (ER) in the GAM group, whose post-test value was higher—that is, the experimental group increased its ER (Z = −2.667; p = 0.008. ES = 0.34).
Table 5.
Results for each educational group in the Wilcoxon test (pre-test vs. post-test).
| Treatment Groups | Wilcoxon Test | IM | IDR | INR | ER | AM |
|---|---|---|---|---|---|---|
|
GAM
(n = 62) |
Negative ranges | 19 | 17 | 24 | 15 | 19 |
| Positive ranges | 19 | 21 | 29 | 33 | 13 | |
| Draws | 24 | 24 | 9 | 14 | 30 | |
| Z | −0.05 | −1.17 | −1.12 | −2.67 | −1.22 | |
| Asymptotic sig. (bilateral) | 0.96 | 0.24 | 0.26 | 0.01 | 0.22 | |
| Effect size | 0.34 | |||||
|
CON
(n = 65) |
Negative ranges | 24 | 21 | 30 | 21 | 28 |
| Positive ranges | 18 | 23 | 23 | 27 | 16 | |
| Draws | 23 | 21 | 12 | 17 | 21 | |
| Z | −0.52 | −0.22 | −0.7 | −1.3 | −1.24 | |
| Asymptotic sig. (bilateral) | 0.61 | 0.83 | 0.48 | 0.19 | 0.22 |
Note: IM = Intrinsic motivation; IR = Identified Regulation; INR = Introjected Regulation; ER = External regulation; AM = Amotivation.
On the other hand, although the sample distribution was not parametric, Table 6 shows the means ± SD of all the motivational variables, including minimum and maximum scores pre- and post-test.
Table 6.
Mean ± standard deviation for each educational group in motivational variables.
| TG | V | IM | IDR | INR | ER | AM |
|---|---|---|---|---|---|---|
| GAM | Pre | 17.11 ± 1.9 | 17.21 ± 1.9 | 11.87 ± 2.7 | 10.10 ± 3.4 | 5.47 ± 1.7 |
| Min | 11 | 12 | 4 | 4 | 4 | |
| Max | 20 | 20 | 19 | 16 | 9 | |
| Post | 17.06 ± 1.9 | 17.48 ± 1.9 | 12.18 ± 3.2 | 10.81 ± 3.4 | 5.23 ± 1.5 | |
| Min | 13 | 13 | 4 | 4 | 4 | |
| Max | 20 | 20 | 19 | 18 | 8 | |
| CON | Pre | 16.95 ± 1.7 | 16.83 ± 1.6 | 12.34 ± 3.1 | 10.74 ± 3.1 | 6.38 ± 1.8 |
| Min | 14 | 13 | 5 | 4 | 4 | |
| Max | 20 | 20 | 18 | 17 | 11 | |
| Post | 16.75 ± 2.3 | 16.80 ± 1.7 | 12.28 ± 3.2 | 11.22 ± 3.3 | 6.09 ± 1.9 | |
| Min | 10 | 14 | 4 | 4 | 4 | |
| Max | 20 | 20 | 19 | 19 | 11 |
Note: TG = Treatment groups; V = Variable; IM = Intrinsic motivation; IR = Identified Regulation; INR = Introjected Regulation; ER = External regulation; AM = Amotivation; Min = Minimum score; Max = Maximum score.
Lastly, a Mann–Whitney U test test showed no post-test differences between the groups (GAM vs. CON) in the motivational variables evaluated after treatment (p > 0.05, Figure 4).
Figure 4.
Box diagram of the motivational variables measured in a questionnaire CMEF (post-test). Note: GAM = experimental group; CON = control group; IM = Intrinsic motivation; IR = Identified Regulation; INR = Introjected Regulation; ER = External regulation; AM = Amotivation.
3.3. Academic Performance
The Mann–Whitney U test showed that the GAM group obtained statistically better scores than the CON group in the results shown in the final exam results (Z = −3417; p = 0.001; ES = 0.30), as shown in Figure 5).
Figure 5.
Box diagram of the qualifications obtained in the final exam. Note: GAM = experimental group; CON = control group.
A deeper analysis of the data was performed for a comparison of the percentile of the qualifications obtained by both treatment groups, as shown in Table 7. The GAM group obtained a higher rating in all ranks except for the highest percentiles (90 and 95). In addition, these higher qualifications were mostly obtained by males, especially in the CON group, as shown in Table 8. Otherwise, the lowest scores in the GAM group were all observed among females.
Table 7.
Percentiles of qualifications in final exam (GAM vs. CON).
| Treatment Group | Percentile | ||||||
|---|---|---|---|---|---|---|---|
| 5 | 10 | 25 | 50 | 75 | 90 | 95 | |
| GAM | 4.83 | 5.00 | 5.50 | 5.90 | 6.50 | 7.00 | 7.17 |
| CON | 3.12 | 4.00 | 4.45 | 5.20 | 6.25 | 7.12 | 7.91 |
Note: GAM = experimental group; CON = control group.
Table 8.
Score and gender of best and worst marks in final exam.
| Rank | GAM | CON | |||
|---|---|---|---|---|---|
| Gender | Score | Gender | Score | ||
| Higher | 1 | Female | 8.00 | Male | 8.00 |
| 2 | Male | 7.30 | Male | 8.00 | |
| 3 | Male | 7.20 | Male | 8.00 | |
| 4 | Male | 7.00 | Male | 7.70 | |
| 5 | Male | 7.00 | Male | 7.50 | |
| Lower | 1 | Female | 4.50 | Male | 2.00 |
| 2 | Female | 4.70 | Female | 3.00 | |
| 3 | Female | 4.80 | Male | 3.00 | |
| 4 | Female | 5.00 | Female | 3.40 | |
| 5 | Female | 5.00 | Male | 3.80 | |
Note: GAM = experimental group; CON = control group.
4. Discussion
Carrying out an intervention based on gamification is based on the idea that any student who participates in this type of proposal can more effectively develop a degree of intrinsic and extrinsic motivation or, conversely, of amotivation [50]. This will have a series of effects on an individual’s psychological attitude and behaviour towards a subject and towards their own learning process, ultimately influencing their academic results [51], among other things. In line with this, Kiesler et al. [17] highlighted gamification as a process that combines two relevant problems of education related to academic results, such as motivation and commitment to the learning process. According to these authors, gamification can not only improve the learning process through better results but will also support three fundamental areas in teaching—emotional, social and cognitive areas [17].
According to the first objective of the research, the obtained results in the questionnaire on Motivation in Physical Education Classes (CMEF) [49] showed a significant increase in the values of extrinsic motivation, specifically ER, only in the group that underwent gamification. This allows us to accept the proposed research hypothesis (H2). As previous research has shown, ER is understood as a powerful form of motivation, although it is difficult to maintain since it describes behaviours regulated by contingencies external to the subject [32,33]. Good and desirable behaviours, as well as undesirable behaviours, by the students in the experimental group were regulated by contingencies external to the subject (rewards and punishments) through the dynamics of the game itself (with experience points -XP- or health points -HP-), detailed in Table 3 and Table 4, while studying the teaching–learning process of the contents of the subject (orienteering, climbing, natural gymnastics, etc.). This feedback, as Bogost [52] points out, must be part of the gamified structure and translate the process that is being developed in the students, so that the students themselves can interpret and control the situation in which they are immersed, activating and improving achievement motivation by receiving information or feedback about its execution. Taking into consideration that university students are already adults and that their education is not compulsory, these rewards or punishments could change their ER as they experience an increase in personal value to those contingencies of the game, as was the case in the group that experienced gamification. A priori, this argumentation could reinforce the opposite idea, that students actively participate in the class for reasons intrinsic to the game, which could be synonymous with enjoyment and, by definition, lead to an increase in intrinsic motivation [33] according to the first hypothesis raised. However, Fernandez-Rio, et al. [4] stated that tangible rewards are a key element for a successful implementation of gamification. Even though the experience points do not represent a functional reward in themselves, the accumulation of these points becomes an academic power, and that could justify the modification of this variable. Unlike similar studies carried out with children and adolescents [42], university life promotes an increase in personal independence and responsibility that could directly relate these academic powers to their qualification [53]. According to this, Hanus and Fox [28] suggested that, in terms of university students, additional rewards are interpreted as controlling, causing students to feel less confident, be less satisfied with the course and have less motivation to engage with the material. Unlike the Hanus and Fox study, the present investigation carried out a gamified intervention from a MDA approach (Mechanic–Dynamic–Aesthetic). However, both rewards and punishment are reflected in the same way—that is, in a public leaderboard (XP, HP and AP) which all students participating in the class can see. This could have a negative effect on social comparisons [54,55]. For these reasons, the features design of the rewards or punishments could become an important attribute for the effective application of gamified treatment. Thus, professionals who apply such interventions must seek to reduce social comparisons and see that the nature of rewards or punishment is inherent in the game itself and not extrapolatable to tangible interests.
Another aspect to highlight in the results of this study, directly related to the first hypothesis, is the absence of change in the IM of the group implemented with gamification. This is somewhat in line with other studies carried out with university students where IM decreased [28]. Despite the fact that gamification was an innovative teaching technique for the students who participated in this study, which could have contributed to improving this variable [56], IM remained without significant change. This allows us to reject the proposed research hypothesis (H1) and accept the null hypothesis (H0).
Despite the fact that the use of external rewards is related to a decrease in intrinsic motivation [34], a recent study in a non-educational context reported that external rewards improved IM as long as the feedback was immediate [57]. During the entire intervention of this research, the researcher used the application Classcraft® for smartphones in order to assign the rewards and punishments immediately through systematic observation, although the students were not aware of this feedback until they started the application at the end of class or when they got home. Therefore, even though the teacher used concurrent feedback for practical purposes, for the student it could be considered delayed or terminal feedback, which would not have the expected effect of the teacher on intrinsic motivation in the gamified group [58]. Other authors, who only assessed this type of intrinsic motivation after gamified treatment [4,28], claimed that the decrease observed in this variable may be due to students who are bored and do not wish to be there, and that rewards and incentives might increase intrinsic motivation. However, for students who are innately interested in the material and already motivated to attend, efforts to gamify the classroom might harm their intrinsic motivation [4,28]. This claim cannot be verified in the present study, nor does it support the fact that no differences have been found between groups at the amotivation levels, whereby we can reject H3. However, the least qualified student observed in the gamified group scored higher than the least qualified student observed in the control group, as shown in Table 7, which seems to support the abovementioned idea that gamification has a greater effect in less interested students. Thus, gamification in PE may be a double-edged sword since, for most students, it is a very interesting and funny curricular subject in itself [4,44].
In line with the previous idea, recent studies that applied gamification in physical education in primary and secondary education [4,42] during a time period greater than the present study (approximately 30 h) reported an improvement in IM through quantitative and qualitative results. According to these results, and those found in the present study, it could be that the age factor, educational level [7,59] or even the level of initial motivation [1,2,4,28] are relevant factors when observing an effective change in IM. However, it is necessary to deepen the knowledge of each of these aspects pertaining to motivation, while increasing investigational requirements, in spite of their complexity, since, in most studies examined, they simply analyse motivation in general.
One of the most applicable results of the present investigation, related to the fourth hypothesis, was that the group that complemented the gamified sessions obtained significantly better scores in the final exam compared to the group that carried out the same sessions without the use of this gamified tool (Z = −3417; p = 0.001; ES = 0.30). It also rejects the null hypothesis (H0) and accepts the fourth research hypothesis (H4). Despite the fact that, in the present study there was no change in the IM that could somehow justify this result [1,2,3,44,60,61], other studies with university students who received teaching with gamified techniques also obtained improvements in performance and, therefore, better academic qualifications [20,21,22]. It could create a sense of control and responsibility within students, which brings them closer to the opportunity to replace the usual goal of learning in order to pass with a more simple desire to learn [26]. Accordingly, not only can there be an improvement in academic results or performance, but students can perceive an increase in the retention and acceptance of knowledge [19] and improvements in learning the contents of the subject [18] in following a gamified approach. Nevertheless, it is necessary to delve further into these aspects in future inquiry, especially in PE.
It seems that the effect of a gamified experience does not affect all students equally [29]. When comparing the distribution of academic performances among all of the participants in the different treatment groups, it was observed that the gamified group, on average, was better than the control group in all percentiles, as shown in Table 7, except for the highest ranks (90 and 95 percentile). This suggests that the gamified group achieved higher academic performance in the sample distribution (compared to the control group), although the highest marks were obtained by the control group. In fact, the worst percentile of the gamified group got marks very close to the limit of approval. Accordingly, it is necessary to emphasise that, within these best scores, most were observed among males in both treatment groups, as shown in Table 8. Moreover, also in line with this analysis, the lowest scores in the experimental group were observed among females. Because this study cannot posit any hypothesis on gender differences and no study related to gamification was found regarding a possible different effect on boys and girls in PE, future studies should investigate the effect of gamification and gender interaction in a physical education context.
Some practical implications may be extracted from this study. The use of a gamified teaching approach using a technological resource enables students to participate from an MDA perspective. The configuration of rewards and punishments must be intrinsic and connected to the game itself, avoiding an approach that encourages less autonomous motivation. In addition, the resource used must be in a mature phase and the interface used must be familiar to the students. Finally, meta-analytic evidence suggests that combinations of cooperation and competition are likely to be an effective gamification strategy [50], which can invite us to use points, badges and leader boards (PBL).
5. Limitations and Future Lines of Research
The use of Classcraft® as a tool for gamified educational intervention allows both groups (Intervention and Control) to receive exactly the same content with the same methodology (grouping, design of activities, length of classes, types of feedback in class, etc.). At the same time, this becomes a major limitation, since combining an innovative pedagogical approach, such as gamification, and the use of a technological resource to carry it out makes it less possible to discern whether the results are attributed to one (gamification) or the other (technological resource), or both (gamification–technological resource) factors. In addition, in terms of the results where motivation plays a prominent role, the absence of measuring other motivational variables, such as integrated regulation [62,63], is another limitation in which future studies with university students should be considered. Although the present study sample may be sufficient, given the research design, future research could extend the sample by allowing for the analysis of possible divergent effects on boys and girls. Another limitation in the present study is its lack of qualitative data, which could shed more light on the students’ experiences. Future studies in the field of physical education should also investigate the optimal length of intervention as compared to the one applied in the present study.
6. Conclusions
As reflected in the literature, it has been observed in the present paper that the pedagogical approach of gamification, implemented in future university teaching PE specialists, shows no variation in intrinsic motivation, but variation in extrinsic motivation is evident. These results are discussed in depth, highlighting the importance of the role that the nature of rewards or punishments could occupy in the implementation of this pedagogical approach to control the change in the extrinsic motivational regulation of students. Gamification is an innovative education approach, which seems to improve learning experiences by improving variables such as motivation. However, this approach seems to have different effects among students, being more effective in less motivated students. Regarding the fourth hypothesis, there is a difference in the scores obtained in academic performance. Therefore, these findings suggest that the implementation of gamification is beneficial for academic performance in university students, even though their intrinsic motivation is not enhanced.
Acknowledgments
The authors appreciate the participation of all university students in this research.
Author Contributions
Conceptualization, A.F.-V. and M.G.-J.; methodology, A.F.-V. and O.Ø.; formal analysis, A.F.-V. and S.G.-M.; investigation, A.F.-V.; S.G.M. and M.G.-J.; writing—original draft preparation, A.F.-V., M.G.-J., S.G.-M. and O.Ø.; writing—review and editing, O.Ø.; supervision, A.F.-V., M.G.-J., S.G.-M. and O.Ø.; project administration A.F.-V.; funding acquisition, A.F.-V. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the University of Alicante, grant number “XARXES-I3CE-2018-4416”.
Conflicts of Interest
The authors declare no conflict of interest.
References
- 1.López-Belmonte J., Segura-Robles A., Fuentes-Cabrera A., Parra-González M.E. Evaluating activation and absence of negative effect: Gamification and escape rooms for learning. Int. J. Environ. Res. Public Health. 2020;17:2224. doi: 10.3390/ijerph17072224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Dicheva D., Dichev C., Agre G., Angelova G. Gamification in education: A systematic mapping study. Educ. Technol. Soc. 2015;18:75–88. [Google Scholar]
- 3.Gómez-Ejerique C., López-Cantos F. Application of innovative teaching-learning methodologies in the classroom. Coaching, flipped-classroom and gamification. Case Study Success. 2019;6:46–70. [Google Scholar]
- 4.Fernandez-Rio J., de las Heras E., González T., Trillo V., Palomares J. Gamification and physical education. Viability and preliminary views from students and teachers. Phys. Educ. Sport Pedagog. 2020:1–16. doi: 10.1080/17408989.2020.1743253. [DOI] [Google Scholar]
- 5.Morford Z.H., Witts B.N., Killingsworth K.J., Alavosius M.P. Gamification: The intersection between behavior analysis and game design technologies. Behav. Anal. 2014;37:25–40. doi: 10.1007/s40614-014-0006-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Quintas A., Bustamante J.-C., Pradas F., Castellar C. Psychological effects of gamified didactics with exergames in Physical Education at primary schools: Results from a natural experiment. Comput. Educ. 2020;152:103874. doi: 10.1016/j.compedu.2020.103874. [DOI] [Google Scholar]
- 7.De Sousa Borges S., Durelli V.H.S., Reis H.M., Isotani S. Proceedings of the 29th Annual ACM Symposium on Applied Computing. Association for Computing Machinery; Gyeongju, Korea: 2014. A Systematic Mapping on Gamification Applied to Education; pp. 216–222. [Google Scholar]
- 8.Kapp K.M. The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. Pfeiffer; San Francisco, CA, USA: 2012. [Google Scholar]
- 9.Deterding S., Dixon D., Khaled R., Nacke L. Du game design au gamefulness: Définir la gamification. Sci. Du Jeu. 2014;2 doi: 10.4000/sdj.287. [DOI] [Google Scholar]
- 10.Dichev C., Dicheva D. Gamifying education: What is known, what is believed and what remains uncertain: A critical review. Int. J. Educ. Technol. High. Educ. 2017;14:9. doi: 10.1186/s41239-017-0042-5. [DOI] [Google Scholar]
- 11.Gómez-Gonzalvo F., Alventosa J.P.M., Devís J.D. Los videojuegos como materiales curriculares: Una aproximación a su uso en Educación Física. Retos. 2018;34:305–310. [Google Scholar]
- 12.González C.S.G., Navarro V.A. Gamification and Active Games for Physical Exercise: A Review of Literature; Proceedings of the 1st International Workshop on Gamification and Games for Learning; Tenerife, Spain. 5–6 June 2017. [Google Scholar]
- 13.Castañer M., Camerino O. Manifestaciones Básicas de la Motricidad. Edicions de la Universitat de Lleida; Lleida, Spain: 2006. [Google Scholar]
- 14.Extremera A.B., Montero P.J.R. El juego motor como actividad física organizada en la enseñanza y la recreación. Emásf Rev. Digit. Educ. Física. 2016;38:73–86. [Google Scholar]
- 15.Kirk D. Physical Education Futures. Routledge; London, UK: 2010. [Google Scholar]
- 16.Fernandez-Rio J. Gamificando la Educación Física. Servicio de Publicaciones de la Universidad de Oviedo; Oviedo, Asturia: 2019. [Google Scholar]
- 17.Kiesler S., Kraut R.E., Koedinger K.R., Aleven V., Mclaren B.M. Gamification in education: What, how, why bother? Acad. Exch. Q. 2011;15:1–5. [Google Scholar]
- 18.Escutia R.R., Pamplona S.R. IV Congreso Internacional sobre Aprendizaje, Innovación y Competitividad (CINAIC 2017) Universidad de Zaragoza; Zaragoza, Spain: 2017. Gamification in a roman law course: A case study. [Google Scholar]
- 19.Nevin C.R., Westfall A.O., Rodriguez J.M., Dempsey D.M., Cherrington A., Roy B., Patel M., Willig J.H. Gamification as a tool for enhancing graduate medical education. Postgrad. Med. J. 2014;90:685–693. doi: 10.1136/postgradmedj-2013-132486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Osorio I.M.V. La gamificación en el aprendizaje de los estudiantes universitarios. Rastros Rostros. 2016;18:27–38. [Google Scholar]
- 21.Sheldon L. The Multiplayer Classroom: Designing Coursework as a Game. 2nd ed. CRC Press; Boca Raton, FL, USA: 2020. [Google Scholar]
- 22.Apostol S., Zaharescu L., Alexe I. Gamification of learning and educational games; Proceedings of the 9th International Scientific Conference “eLearning and Software for Education”, Editura Universitatii Nationale de Aparare “Carol I”; Bucharest, Romania. 25–26 April 2013; pp. 67–72. [Google Scholar]
- 23.Escaravajal Rodríguez J.C., Martín-Acosta F. Análisis bibliográfico de la gamificación en Educación Física. Rev. Ib. Cc. Act. Fís. Dep. 2019;8:97–109. doi: 10.24310/riccafd.2019.v8i1.5770. [DOI] [Google Scholar]
- 24.Charles D., Charles T., McNeill M., Bustard D., Black M. Game-based feedback for educational multi-user virtual environments. Br. J. Educ. Technol. 2011;42:638–654. doi: 10.1111/j.1467-8535.2010.01068.x. [DOI] [Google Scholar]
- 25.Bellotti F., Berta R., De Gloria A., Lavagnino E., Antonaci A., Dagnino F.M., Ott M. A gamified short course for promoting entrepreneurship among ICT engineering students; Proceedings of the 2013 IEEE 13th International Conference on Advanced Learning Technologies; Beijing, China. 15–18 July 2013; pp. 31–32. [Google Scholar]
- 26.Pérez-López I.J., Rivera García E., Trigueros Cervantes C. The prophecy of the chosen ones: An example of gamification applied to university teaching. Int. J. Med. Sci. Phys. Act. Sport. 2017;17:243–260. [Google Scholar]
- 27.De-Marcos L., Domínguez A., Saenz-de-Navarrete J., Pagés C. An empirical study comparing gamification and social networking on e-learning. Comput. Educ. 2014;75:82–91. doi: 10.1016/j.compedu.2014.01.012. [DOI] [Google Scholar]
- 28.Hanus M.D., Fox J. Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput. Educ. 2015;80:152–161. doi: 10.1016/j.compedu.2014.08.019. [DOI] [Google Scholar]
- 29.Domínguez A., Saenz-de-Navarrete J., de-Marcos L., Fernández-Sanz L., Pagés C., Martínez-HerráIz J.-J. Gamifying learning experiences: Practical implications and outcomes. Comput. Educ. 2013;63:380–392. doi: 10.1016/j.compedu.2012.12.020. [DOI] [Google Scholar]
- 30.Arufe-Giráldez V. Fortnite EF, un nuevo juego deportivo para el aula de Educación Física: Propuesta de innovación y gamificación basada en el videojuego Fortnite. Sportis. 2019;5:323–350. doi: 10.17979/sportis.2019.5.2.5257. [DOI] [Google Scholar]
- 31.González L.E.Q., Jiménez F.J., Moreira M.A. Más allá del libro de texto. La gamificación mediada con TIC como alternativa de innovación en Educación Física. Retos Nuevas Tend. Educ. Física Deporte Recreación. 2018;34:343–348. [Google Scholar]
- 32.Ryan R.M., Deci E.L. Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. The Guilford Press; New York, NY, USA: 2017. [Google Scholar]
- 33.Ryan R.M., Deci E.L. Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemp. Educ. Psychol. 2020 doi: 10.1016/j.cedpsych.2020.101860. [DOI] [Google Scholar]
- 34.Deci E.L., Koestner R., Ryan R.M. Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Rev. Educ. Res. 2001;71:1–27. doi: 10.3102/00346543071001001. [DOI] [Google Scholar]
- 35.Sailer M., Hense J.U., Mayr S.K., Mandl H. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Comput. Hum. Behav. 2017;69:371–380. doi: 10.1016/j.chb.2016.12.033. [DOI] [Google Scholar]
- 36.Hagger M.S., Chatzisarantis N.L., Culverhouse T., Biddle S.J. The processes by which perceived autonomy support in physical education promotes leisure-time physical activity intentions and behavior: A trans-contextual model. J. Educ. Psychol. 2003;95:784–795. doi: 10.1037/0022-0663.95.4.784. [DOI] [Google Scholar]
- 37.Aelterman N., Vansteenkiste M., Van Keer H., Van den Berghe L., De Meyer J., Haerens L. Students’ objectively measured physical activity levels and engagement as a function of between-class and between-student differences in motivation toward physical education. J. Sport Exerc. Psychol. 2012;34:457–480. doi: 10.1123/jsep.34.4.457. [DOI] [PubMed] [Google Scholar]
- 38.Kam A.H., Umar I.N. Fostering authentic learning motivations through gamification: A self-determination theory (SDT) approach. J. Eng. Sci. Technol. 2018;13:1–9. [Google Scholar]
- 39.Mekler E.D., Brühlmann F., Tuch A.N., Opwis K. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput. Hum. Behav. 2017;71:525–534. doi: 10.1016/j.chb.2015.08.048. [DOI] [Google Scholar]
- 40.Van Roy R., Zaman B. Unravelling the ambivalent motivational power of gamification: A basic psychological needs perspective. Int. J. Hum. -Comput. Stud. 2019;127:38–50. doi: 10.1016/j.ijhcs.2018.04.009. [DOI] [Google Scholar]
- 41.Sheehan D.P., Katz L. The practical and theoretical implications of flow theory and intrinsic motivation in designing and implementing exergaming in the school environment. Load. 2012;6:53–68. [Google Scholar]
- 42.Monguillot M.H., González C.A., Zurita C.M., Almirall L.B., Guitert M.C. Play the Game: Gamificación y hábitos saludables en educación física. Apunt. Educ. Fis. Deportes. 2015;119:71–79. doi: 10.5672/apunts.2014-0983.es.(2015/1).119.04. [DOI] [Google Scholar]
- 43.Sun H. Impact of exergames on physical activity and motivation in elementary school students: A follow-up study. J. Sport Health Sci. 2013;2:138–145. doi: 10.1016/j.jshs.2013.02.003. [DOI] [Google Scholar]
- 44.Ntoumanis N. A self-determination approach to the understanding of motivation in physical education. Br. J. Educ. Psychol. 2001;71:225–242. doi: 10.1348/000709901158497. [DOI] [PubMed] [Google Scholar]
- 45.Portman P.A. Who is having fun in physical education classes? Experiences of sixth-grade students in elementary and middle schools. J. Teach. Phys. Educ. 1995;14:445–453. doi: 10.1123/jtpe.14.4.445. [DOI] [Google Scholar]
- 46.Smith A., Parr M. Young people’s views on the nature and purposes of physical education: A sociological analysis. Sport Educ. Soc. 2007;12:37–58. doi: 10.1080/13573320601081526. [DOI] [Google Scholar]
- 47.Gutiérrez M., Pérez L.M.R., López E. Clima motivacional en educación física: Concordancia entre las percepciones de los alumnos y las de sus profesores. Rev. Psicol. Deporte. 2011;20:321–335. [Google Scholar]
- 48.Campbell D.T., Stanley J.C. Experimental and Quasi-Experimental Designs for Research. Rand-McNally; Chicago, IL, USA: 1963. [Google Scholar]
- 49.Sánchez-Oliva D., Marcos F.M.L., Amado D., Alonso I.G.-P., García-Calvo T. Desarrollo de un cuestionario para valorar la motivación en educación física. Rev. Iberoam. Psicol. Ejerc. Deporte. 2012;7:227–250. [Google Scholar]
- 50.Sailer M., Homner L. The gamification of learning: A meta-analysis. Educ. Psychol. Rev. 2020;32:77–112. doi: 10.1007/s10648-019-09498-w. [DOI] [Google Scholar]
- 51.Ryan R.M., Deci E.L. Brick by brick: The origins, development, and future of self-determination theory. Adv. Motiv. Sci. 2019;6:111–156. [Google Scholar]
- 52.Bogost I. Persuasive Games. Volume 5 MIT Press; Cambridge, MA, USA: 2007. [Google Scholar]
- 53.Sánchez de Miguel M., Lizaso I., Hermosilla D., Alcover C.M., Goudas M., Arranz-Freijó E. Preliminary validation of the Perceived Locus of Causality scale for academic motivation in the context of university studies (PLOC-U) Br. J. Educ. Psychol. 2017;87:558–572. doi: 10.1111/bjep.12164. [DOI] [PubMed] [Google Scholar]
- 54.Christy K.R., Fox J. Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women’s math performance. Comput. Educ. 2014;78:66–77. doi: 10.1016/j.compedu.2014.05.005. [DOI] [Google Scholar]
- 55.Dijkstra P., Kuyper H., Van der Werf G., Buunk A.P., van der Zee Y.G. Social comparison in the classroom: A review. Rev. Educ. Res. 2008;78:828–879. doi: 10.3102/0034654308321210. [DOI] [Google Scholar]
- 56.González-Cutre D., Sicilia Á. The importance of novelty satisfaction for multiple positive outcomes in physical education. Eur. Phys. Educ. Rev. 2019;25:859–875. doi: 10.1177/1356336X18783980. [DOI] [Google Scholar]
- 57.Woolley K., Fishbach A. It’s about time: Earlier rewards increase intrinsic motivation. J. Personal. Soc. Psychol. 2018;114:877–890. doi: 10.1037/pspa0000116. [DOI] [PubMed] [Google Scholar]
- 58.Huéscar E., Moreno-Murcia J.-A. Relación del tipo de feed-back del docente con la percepción de autonomía del alumnado en clases de educación física. Infanc. Y Aprendiz. 2012;35:87–98. doi: 10.1174/021037012798977449. [DOI] [Google Scholar]
- 59.Bicen H., Kocakoyun S. Perceptions of students for gamification approach: Kahoot as a case study. Int. J. Emerg. Technol. Learn. 2018;13:72–93. doi: 10.3991/ijet.v13i02.7467. [DOI] [Google Scholar]
- 60.Ntoumanis N., Taylor I.M., Standage M. Testing a model of antecedents and consequences of defensive pessimism and self-handicapping in school physical education. J. Sports Sci. 2010;28:1515–1525. doi: 10.1080/02640414.2010.511650. [DOI] [PubMed] [Google Scholar]
- 61.Standage M., Duda J.L., Ntoumanis N. A test of self-determination theory in school physical education. Br. J. Educ. Psychol. 2005;75:411–433. doi: 10.1348/000709904X22359. [DOI] [PubMed] [Google Scholar]
- 62.Ferriz R., González-Cutre D., Sicilia Á. Revisión de la Escala del Locus Percibido de Causalidad (PLOC) para la inclusión de la medida de la regulación integrada en educación física. Rev. Psicol. Deporte. 2015;24:1–10. [Google Scholar]
- 63.González-Cutre D., Sicilia Á., Fernández A. Hacia una mayor comprensión de la motivación en el ejercicio físico: Medición de la regulación integrada en el contexto español. Psicothema. 2010;22:841–847. [PubMed] [Google Scholar]