Skip to main content
PMC home page
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2022 Apr 12;27(7):10207–10238. doi: 10.1007/s10639-022-11048-x

Gamification in education: A scientometric, content and co-occurrence analysis of systematic review and meta-analysis articles

Somayyeh Nadi-Ravandi 1,2, Zahra Batooli 3,4,✉
PMCID: PMC9002043  PMID: 35431599

Abstract

This study seeks scientometric, content and co-occurrence analysis of systematic review and Meta-analysis articles in the field of gamification in education. In terms of purpose, this is an applied study and regarding type, it is a scientometric and co-occurrence analysis. The researchers conducted a search in WoS, Scopus and PubMed databases. The abstract and full text of 25 out of 71 articles were selected to be included in the study. Then, the citation and altmetrics indicators were investigated. In addition, VOSviewer software was utilized to analyze and visualize keywords and map of articles. Finally, the full texts of all articles were analyzed to be provided more information about the types of analyses in these articles. The findings showed that 25 articles were published between 2016 and 2021. Co-occurrence map of articles showed that the three variables of motivation, learning, and engagement have been considered in gamified education studies and most studies have examined gamification in the e-learning environment. Finally, the content analysis of the articles showed that 344 articles were included and analyzed in these 25 systematic reviews and meta-analyses. The types of analyzes performed on these 344 articles categorized them in 7 categories including Country/Territory, Duration of intervention, Lessons/content and the level of gamified educational course, the number of learners, platforms, the game elements and the theories. The results of the study illustrate that different dimensions of the gamification in articles in the field of Education have been considered by the researchers.

Keywords: Gamification, Education, Content analysis, Co-occurrence analysis

Introduction

In the current age, computer games are one of the things that digital culture has brought to modern life. The latest concept in this field is termed gamification, which acts as a broad umbrella for the use of video game components in order to improve the experience and increase users' participation in environments and contexts that are not related to the game (Hamari et al., 2014).

Gamification, which is related to the field of digital media industry (Schönen, 2014), means "the use of game design elements in non-gaming spaces" (Deterding et al., 2011). In fact, gamification is the use of tools and mechanism, aesthetic aspects and game thinking to make people more engaged and motivated to behave specifically, and encourage them to learn and solve problems (Kapp, 2012). Now, it covers a very wide and diverse range such as education and learning, health, e-commerce, environment, and hotel management (Deterding et al., 2011). Thus, gamification is the use of game-like thinking and characteristics in areas that are not inherent in the game (Huotari & Hamari, 2012); however, it uses game structures such as foundations, stimuli and components of game to solve life problems (Chou, 2016).

The word gamification was coined in 2002, but the concept first became popular in the scientific literature in the second half of 2010, and gained popularity among researchers in 2011. Gartner predicted that by the end of 2015, more than 40% of the world's top 1,000 organizations will benefit from gamification components in terms of customer orientation and product quality improvement (Schönen, 2014) and in the near future, there will be significant progress in the field of internalization of internal processes as well as external interactions, i.e. attracting more users and customers (Burke, 2012). Currently, there are a lot of websites and experts in this field and many articles have been written on this topic and its sub-categories. This generated knowledge can inherently provide valuable information about the role ofof gamification in various aspects of life. A simple Google search for new teaching and learning methods shows that gamification is a fascinating method alongside other methods such as flipped learning, project-based learning, cooperative learning, problem-based learning, design thinking, thinking-based learning, and competency-based learning (Realinfluencers, 2019). Interestingly, this method itself has the ability to be integrated with other methods so that, an inverted learning method, for instance, can be linked to a collaborative learning using gamification. Discovering and using this information in the field of learning and teaching requires a look through the literature in this field., Systematic review and meta-analysis articles are most probably the best, shortest and fastest ways to obtain valid information in this regard. These types of articles aim to evaluate, select, and synthesize quality studies in a specific field to provide more accurate results, which can not only provide high-quality evidence but also make decisions about reviewing original studies easier and faster. On the other hand, the results of the research by Hamri, Quisto and Sarsa testify to the claim that the most widely used concept of gamification has been in the field of "teaching and learning" (Hamari et al., 2014). Therefore, an analysis of systematic review and meta-analysis articles in this subject area can provide valuable information for researchers and those interested in the status of studies related to the gamification in the field of education.

In addition, altmetric analysis of high-quality articles in a subject area contributes to our greater knowledge of research, topics, and trends. This type of analysis shows the process of dissemination, the evolution of knowledge and the evidence-based practice of a subject. Therefore, due to the increasing applications of scientometrics in the evaluation and measurement of scientific products, the purpose of this study is scientometric analysis of the systematic review and meta-analysis articles related to gamification (Mostafavi & Bazrafshan, 2011). Examination of citation and altmetric indices of articles shows their scientific and social impact (Lora et al., 2020).

Co-occurrence analysis is also one of the types of scientometric analysis namely content analysis, which is obtained through the co-occurrence of words with the concepts in texts and sources, which can be used to identify the main concepts of a field or scientific field. As a result, patterns and conceptual events, scientific structure, conceptual network, hierarchical relationships of concepts and conceptual categories of the field under study are discovered, plotted and managed (Zhang et al., 2016). This conceptual network is drawn by counting the number of thematic words in the text and its association with other topics. In other words, if two terms are used together in a document and are repeated as much as possible, it means that these two words are more semantically related. The co-occurrence of two terms or two words is also used to discover the connection between two topics in a field of research, and in this way the development and progress of that field of science can be traced (Ahmadi & Osareh, 2017).Among the studies that have dealt with the co-occurrence of words in the scientific productions of different subject areas are Covid-19 (Al-Zaman, 2021), Coronavirus (Atlasi et al., 2021), Artificial Intelligence (Chen et al., 2020), Dentistry (Ghaffari et al., 2019), Diabetes (Makkizadeh et al., 2016), Blockchain technology (Niknejad et al., 2021), Child abuse (Tran et al., 2018). Therefore, considering that the use of gamification in education has caught many researchers' attention and numerous systematic review and meta-analysis studies has been done in this regard, this study seeks scientometric, content and co-occurrence analysis of systematic review and meta-analysis articles in the field of gamification in education.‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬

Method

The purpose of this study is a scientometric, content and co-occurrence analysis of systematic review and meta-analysis articles in the field of gamification in education. In terms of purpose, this is an applied study and regarding type, it is a scientometric and co-occurrence analysis.

Literature search strategy

In order to retrieve systematic review and meta-analysis articles related to gamification, using related keywords (Systematic review, meta-analysis, Game, Games, Gamification, Gameful), a search was conducted in WoS, Scopus and PubMed databases. Education-related keywords were not added to the search strategy, and after reviewing the titles and abstracts of the articles, articles in the field of education were selected.

Inclusion and exclusion criteria

Inclusion criteria

Articles of systematic review and meta-analysis in the field of Gamification in Education.

Exclusion criteria

Articles which were not based on the definition of gamification (Deterding et al., 2011; Kapp, 2012; Nicholson, 2015) (using game elements in non-game environments to change behavior and solve problems) and were not related to the use of Gamification in Education.

Identification, selection, and coding

In the identification phase, 421 articles were retrieved with our search strategy. After removing duplicates, 208 articles remained for further review. Afterwards, the articles were screened, out of which71 articles were selected. Finally, the abstract and full text of selected articles were reviewed and only 25 articles (based on inclusion and exclusion criteria) selected to be included in the study. Figure 1 shows the flowchart of the present study process.

Fig. 1.

Fig. 1

Open in a new tab

Flowchart of present study process

Data analysis

Citation and Altmetric analysis

In order to investigate the scientific and social impact of the articles, their citation and altmetrics indices were extracted from WoS and Scopus. Citation indicators such as WOS and Scopus Citation, the scientific impact of articles and altmetrics indicators such as Mendeley Read, Facebook Share, Tweets, Scopus View, WOS Usage Count Since 2013, and Altmetric Attention Score, show their social impact.

Co-occurrence analyses

Finally, using VOS viewer software, the word map of articles was drawn. The mapping of a scientific field is a technique that provides a structured overview of the science. One of the techniques used to map science is the visualization of similarities (VOS). By drawing a graphic of each discipline, science maps have paved the way for a better and more accurate identification of that branch of human knowledge and the transformation of the abstract concept of the discipline into a more objective concept. These maps are drawn with various techniques and methods. One of the purposes and applications of co- occurrence analysis is to draw the structure of science or to draw scientific maps. In a keyword co-occurrence map, the size of the circles shows the number of repetitions of the keywords. In other words, the larger the circle, the more frequent the vocabulary of that domain. In a keyword co-occurrence map analysis, the relative distance of one concept to another is emphasized. The proximity of keywords at the point means that more concepts are related to each other. In addition, the thicker and shorter the lines are, the stronger the connection between the words exists.

Content analysis

In order to analyze the content of the articles, the following items were extracted from 25 reviewed articles: Subject, Number of articles reviewed in each article, Names of databases searched in 25 articles for resource extraction (ERIC, Science Direct, ACM Digital Library, IEEE Xplore, Scopus, Springer Link, Wiley Online Library, Google Scholar), Extraction period of resources reviewed in 25 articles, Language of resources reviewed in 25 articles (English, Spanish, etc.), and type of publication reviewed in 25 articles (journal article, conference article, dissertation, book).

Results

Citation and Altmetric analysis

In this study, 25 systematic reviews (18 articles) and meta-analyses (seven articles) in the field of gamified educational environments were studied. All articles were published between 2016 and 2021 that 6 articles were conference proceedings and the others were journal articles. Table 1 shows the findings of top articles regarding the altmetric and citation indicators.

Table 1.

Top articles in terms of citation and altmetric metrics

Metrics No. Of Metrics Author Title Year Journal
WoS Citation 104 Subhash & Cudney, 2018 Gamified learning in higher education: A systematic review of the literature 2018 Computers in Human Behavior
Scopus Citation 142
Scopus View 522
Usage Count Since 2013 202
Mendeley Read 835
FWCI 33.57 Kalogiannakis et al., 2021 Gamification in science education. A systematic review of the literature 2021 Education Sciences
Facebook share & Tweets 413 Osatuyi, Osatuyi et al., 2018 Systematic review of gamification research in IS education: A multi-method approach 2018 Communications of the Association for Information Systems
AAS Sailer & Homner, 2020 The Gamification of Learning: a Meta-analysis 2020 Educational Psychology Review
Open in a new tab

It can be seen from Table 1, in terms of the number of citations and views in WoS and Scopus and the number of read in Mendeley, the article by Subhash & Cudney (2018) is the best one among all articles. The highest number of FWCI belongs to the article by Kalogiannakis et al. (2021).

Co-occurrence analyses

Next, the co-occurrence map was comprisedof 64 keywords from authors and database keywords by using VOSviewer software, which is shown in Fig. 2. The size of the circles indicates the amount of knowledge available in each concept. Nodes represent concepts and lines show how they are related.

Fig. 2.

Fig. 2

Open in a new tab

co- occurrence maps of articles

Figure 2 shows that gamification keyword is at the center of the map. The keywords Motivation, student learning, computer aided instruction, engagement, learning achievement, learning performance are also keywords that have a larger node. Figures 3, 4, 5 and 6 highlights the relationship between the four most frequent keywords with other keywords.

Fig. 3.

Fig. 3

Open in a new tab

Relationship of Motivation with Gamification

Fig. 4.

Fig. 4

Open in a new tab

Relationship of Engagement with Gamification

Fig. 5.

Fig. 5

Open in a new tab

Relationship of Computer aided instruction with Gamification

Fig. 6.

Fig. 6

Open in a new tab

Relationship of Student learning with Gamification

As can be seen from the maps (3-6), the three variables, including motivation, learning, and engagement have been considered in gamified education studies. It is also clear that most studies have examined gamification in the e-learning environment.

Content analysis

In the next step, the full texts of all 25 articles were examined and information such as databases, period, language, type of included resources (conference article, journal article, thesis and book) and the number of included articles were extracted. Table 2 shows that the included articles in these 25 systematic reviews and meta-analyses were indexed in 48 databases, the most repeated one belongs to WoS with 12 cases and ERIC, Science Direct, ACM Digital Library, IEEE Xplore, Scopus, Springer Link, Wiley Online Library, Google Scholar are in the next ranks. In addition, the most and least number of articles in these 25 studies are 148 and 6 cases, in turn. Most of the which are in English. However, there were some articles in Dutch, Portuguese and Spanish in some studies.

Table 2.

Bibliographic information on articles reviewed in systematic reviews and meta-analyses (Database, Date, No. of Article, Publication Type, and Language)

Citation Database date No. of Article Publication Type Language
Malicki et al., 2020 PubMed, CINAHL, ERIC, and Cochrane to 2018 23 articles Journal English
Bai et al., 2019 ACM Digital Library, EBSCO, IEEE Xplore Digital Library, INSPEC, ProQuest, Scopus, Web of Science to 2018 13 articles Journal, conference proceedings English
Huang et al., 2020 ACM Digital Library, ACS Publications, DOAJ, EBSCOhost, Gale Databases, HEINONLINE, IEEE Xplore Digital Library, LearnTechLib, NCBI Databases, Ovid, ProQuest, Sage Journals, ScienceDirect, SpringerLink, Taylor & Francis Online, Web of Science, Wiley Online Library, World Cat 2009-2018 Qualitative: 118 Quantitave: 30 Journal, Dissertation/thesis, Conference proceeding English
Zainuddin et al., 2020 ScienceDirect, EBSCOhost Web, Emerald Insight, Taylor & Francis Online, Wiley Online Library and SpringerLink 2016-2019 46 articles Journal English
Indriasari et al., 2020 ACM Digital Library, IEEE Xplore, ScienceDirect, Springer Link, Scopus, Web of Science Core Collection, and ERIC 2011-2018 39 articles Journal, conference proceedings, book chapter English
Subhash & Cudney, 2018 Academic Search Complete, ACM Digital Library, Education Full Text, ASEM Digital Collection, IEEE Xplore, PsychINFO, and Scopus to September 2017 41 articles Journal English
Ortiz-Rojas et al., 2017 Web of Science 2000–2016 23 articles Journal Spanish, English
Kim & Castelli, 2021 The Academic Search Complete, Communication & Mass Media Complete, Education Source, ERIC, Library Information Science & Technology Abstracts, and PsycINFO 2010-2019 18 articles Journal, conference proceedings English
Garcia et al., 2020 Springer Link, ACM Digital Library, Science Direct and IEEE Xplore the last five years 45 articles Journals English, Portuguese
Kalogiannakis et al., 2021 Science Direct, Eric,Wiley Online Library, SpringerLink, Sage Journals, Taylor & Francis Online, and JSTOR. Google Scholar 2012-2020 24 articles Journal, conference proceedings English
Bai et al., 2020 ACM Digital Library, EBSCO host research databases (including Academic Search Premier, British Education Index, ERIC, TOC Premier), Emerald Insight, Science Direct, Scopus, and Web of Science 2010-2018 quantitative: 24 Qualitative: 32 Journal, conference proceedings English
Yıldırım & Şen, 2019 Web of Science and Google Scholar 2010-2016 40 articles Journal, Book chapter, Thesis/ Dissertation Conference proceeding, all languages
Dehghanzadeh et al., 2019 Scopus, ERIC, and Web of Science 2008-2019 22 articles Journal, conference proceedings English
Van Gaalen et al., 2021 Academic Search Premier; CINAHL; EMBASE; ERIC; Psychology and Behavior Sciences Collection; PsychINFO, PubMed and the Cochrane Library to April 2018 44 articles Journal Dutch or English
Alomari et al., 2019 Google scholar, Springer, ERIC (education resources information system), IEEE Xplore and Science Direct 2016-2018 40 articles Journal, conference proceedings English
Dos Santos et al., 2020 ACM, IEEE Xplore, Springer Link, Science Direct, in conjunction with the CAPES Periodicals Portal from the Brazilian Ministry of Education 2013-2018 43 articles Journal English
Ortiz et al., 2016 Web of Science 2000-2016 30 articles Journal, conference proceedings Spanish, English
Sailer & Homner, 2020 ACM Digital Library, ERIC, IEEE Xplore, JSTOR, PubMed, ScienceDirect, and SpringerLink, Google Scholar to March 2017 38 articles Journal, conference proceedings English
Osatuyi et al., 2018 Journals: ISR, JAIS, JCIS, JISE And major IS conference proceedings available in the Association for Information Systems 2008-2017 41 articles Journal, conference proceedings
Ekici, 2021 The Web of Science, Scopus, Wiley Online Library, ERIC and Science Direct 2010-2019 22 articles Journal English
Brick et al., 2020 ERIC, Emerald and Elsevier 2014-2018 6 articles Journal -
Da Silva et al., 2019 ISI Web of Science and Scopus to December 2017 104 articles Journal -
Manzano-León et al., 2021 Wos, scopus, dialent 2016–2020 14 articles Journal English, Spanish
Tenório et al., 2018 high quality journal; learning as primary field; computers in education as secondary field. Selected journals to this review.: IJDET, JEMH, JITE, JSET, C&E,CE, RLTE all year 14 articles Journal English
Pinto et al., 2021 ACM, IEEE Xplore, Mary Ann Liebert, Scopus,Wiley and Web of Science 1990–2020 32 articles Journal, conference proceedings English, Portuguese
344 articles
Open in a new tab

In 25 systematic reviews and meta-analysis articles, 344 articles were included and analyzed. Having read the full text of all 25 articles, the researchers extracted the types of analyzes performed on these 344 articles and categorized them in 7 categories including:

  1. Country/Territory related to studies in the field of gamified education studies

  2. Duration of intervention

  3. Lessons/content and the level of gamified educational course

  4. The number of learners

  5. The used platforms

  6. The game elements

  7. The used theories in the studies

Country/territory

Cultural differences can create different expectations and attitudes in learners about gamified learning in different countries. Table 3 shows that only 11 out of 25 studied articles reported country/territory where intervention was conducted.

Table 3.

Country/Territory, where gamified educational interventions were conducted

Citation Country/Territory Citation Country/Territory
Subhash & Cudney, 2018 Spain, US, Germany, United Kingdom; Ortiz et al., 2016 Europe, America, Asia , Africa and not specified
Ortiz-Rojas et al., 2017 Europe:12, America: 5; Asia: 4, not specified: 2 Bai et al., 2019 Asia, Europe: 4, America:3 Countries: Spain, Hong Kong and Turkey
Bai et al., 2020 East Asia:15, Western Asia: 5; Northern America: 2, South America: 2; Southern Europe: 2 Ekici, 2021 Spain, Great Britain, Taiwan, Indonesia, Hong Kong, Turkey and Cyprus, Singapore, Mexico, Germany
Yıldırım & Şen, 2019 USA: 9, Turkey: 9, Spain: 4, England: 3; Taiwan: 3, Hong Kong: 2 Osatuyi et al., 2018 USA 53, Germany 44, Australia 22, Switzerland 12 Hong Kong 8, UK 8, Finland 7
van Gaalen et al., 2021 USA or Canada Malicki et al., 2020 United States (6), Canada (4), Australia (2), Finland (2), Norway, Portugal, Spain (2)
Dos Santos et al., 2020 Spain
Open in a new tab

As it is clear from Table 3, most of the studies in the field of gamified education were conducted in the US, Canada and Spain, respectively.

Duration of gamified educational intervention

One of the most important items in the field of gamified education considered by researchers is course effectiveness based on the length of the intervention. Table 4 shows that only in 8 studies, the duration of gamified intervention were reported.

Table 4.

Duration parameter in gamified educational interventions

Citation Length of Gamified Interventions
Ortiz-Rojas et al., 2017 <1 month: 4, 2—4 months: 5, 1 semester: 5, Not Stated: 6, < 1 semester: 3
Kim & Castelli, 2021 less than 1 h: 5, 2–16 weeks: 11, 1–2 years: 2
Bai et al., 2020 1 month-3 months: 10, <1 weeks: 6, ≥1 semesters: 5, 3 months-1 semester: 4, 1 week-1 month: 3, No data reported:2
Dehghanzadeh et al., 2019 < 1 Hour: 6, 1 Hour: 2, < 3 weeks: 4, 3-6 weeks: 4, 6 weeks: 2, 2 months: 1, 3 months: 1, 6 months: 1
Ortiz et al., 2016 1 semester: 16, 1-14 weeks: 7, 1-4 months: 2, 1-24 h: 1, ≥ 1 year: 1, Not specified: 1, Other: 2(14)
Bai et al., 2019 three-quarters of a term: 4, one quarter of a term: 3, two-quarters of a term: 3, more than one term: 1, No data: 2
Sailer & Homner, 2020 1 day or less, 1 week or less (but longer than 1 day), 1 month or less (but longer than 1 week), half a year or less (but longer than 1 month), more than half a year
Ekici, 2021 between 0 and 4 months long: 16, six months long: 2, shorter than a month: 3
Open in a new tab

Table 4 indicates that the maximum and minimum lengths of each course are "less than one hour" and "one to two years", in turn.

Participants (lesson/content/discipline and grade)

Another important item in systematic reviews and meta-analyses was the Lesson/content and the grade of gamified educational course for the participants. Table 5 shows the results of 19 articles examined this item in their studies.

Table 5.

Discipline and Participants’ levels of education in gamified educational interventions*

Citation Lesson/content/ Discipline of the intervention Participants’ levels of education
Huang et al., 2020 Social science (Psychology, Education): 10; Engineering Computing: 6; Arts and Humanities: 5, Science (Biology, Physics): 4, Math: 2; Health (Nursing and Medicine): 1, Business, nformation Systems: 1; Other: 1 Undergraduate: 13; K-12: 10; Higher education: 3; Graduate: 2; Mixed higher education: 1; Mixed higher education and K-12:1
Zainuddin et al., 2020 - Adult/Higher education: 36; Primary schools: 5; Secondary school: 5
Indriasari et al., 2020 Physical Science, Mathematics & Computer Science: 10; Engineering: 8, Education: 7, Social Sciences: 3; Business: 3; Health Professions: 2, Other majors:1, Unknown: 6 Elementary; High school; University
Subhash & Cudney, 2018 Computing: 14, Business, science: 5, Science: 4; Academic Distribution: 3, Entrepreneurship: 2, Communication: 2; Civil: 1, Language: 1, Manufacturing Engineering: 1; Mechanical Engineering: 1, Nursing:1, Pedagogy: 1, Psychology:1 Higher education
Ortiz-Rojas et al., 2017 Science, Technology, Engineering and Mathematics (STEM): 19; Communication: 1, Financing: 1, Use of Photoshop: 1; Designing Questionnaires: 1 Higher Education: 19; High School: 2; Middle School: 2
Kim & Castelli, 2021 - College students: 10, Adults: 6, k-12: 2
Garcia et al., 2020 Mathematics, Teacher:2, Student:33, Both: 1, Unknown: 9 Higher Education: 19, Middle School:16, Both: 1, unknown :8
Kalogiannakis et al., 2021 Biology or Health: 8, Physics: 5, Chemistry: 2, Natural Sciences: 9 Higher education: 10; Secondary education: 9; Primary education: 5
Bai et al., 2020 Arts: 3, Computer & information: 9, Language: 7, Research methodology: 3, Science: 3, Health education: 2, Mathematics: 2 Undergraduate: 10; Elementary school: 9; High school: 5; Postgraduate: 3; No data reported: 2
Yıldırım & Şen, 2019 Non-technology: 30; Technology: 15; Overal: 45 Primary school: 5; Secondary school: 6; High school: 4; University: 30; Overall: 45
Dehghanzadeh et al., 2019 English, Learning vocabularies: 15, Grammar:5: Pronunciation:4 Speaking:5, Writing: 3, Listening: 4 High schools: 10; Higher education:7; Elementary schools: 4
Van Gaalen et al., 2021 Heallth, Surgery, Anatomy & physiology, Anatomy learning Attitudes towards aging, Auscultatory skills, Biology Breast imaging, Clinical reasoning, Critical care, Hypertension treatment, Internal medicine , Learning about, Learning critical thinking, Life-support training, Microbiology , Pediatric knowledge Pharmaceuticals, Physiology, Quality improvement, Radiology Resuscitation principles, Scientific writing, Urine catheterization Residents: 11, Medical students: 14 Nursing: 4, Allied health students: 1 Specialists:3, Osteopathic students:1 Pharmacy students: 1 Nursery: 3, Primary care physicians:1 Speech-Language and Hearing Science: 1, Medicine (mixed): 3, Dentistry:1
Dos Santos et al., 2020 - High/technical school:18; General education: 11; Higher education: 7; Elementary school:5; Early childhood education: 2
Ortiz et al., 2016 STEM, computer science: 25, Science/Technology: 2, Math: 1 Chemistry:1, Not specified: 1 Higher Education
Bai et al., 2019 - Undergraduate:9, Elementary school:1; High school: 1; High school + undergraduate + postgraduate: 1,Postgraduate: 1
Sailer & Homner, 2020 - School setting, Higher education setting
Ekici, 2021 STEM, information science and information and communication technologies: 12, educational sciences: 2 English learning: 2 personal and professional development: 2 College: 18; High school: 3; Primary school: 1
Manzano-León et al., 2021 Physical education, Foundations of the curriculum and physical Education, Sciences, Industrial technology, MSc in Software Engineering for the Web, Advanced quantum mechanics course Online seminar of psychology, Second language(English), Ethical education, Matter and Energy subject in Primary, Education Degree Psychology courses, Math University: 6; School: 3; High School: 3; School and high school: 2
Pinto et al., 2021 Language: English: 21; German: 3; Chinese: 2; Japanese: 1; Basque: 1; Mandarin: 1; Amazigh: 1; Danish: 1 Primary education or first stage of basic education: 9; Lower secondary or second stage of basic education: 5; Pre-primary: 2; Secondary: 2; First stage of tertiary: 2; Post-secondary non-tertiary: 1; Second stage of tertiary: 1; Participants recruited from the university community: 1; 21 years old: 1; 21 to 55 years old: 1; Students from an higher education foreign language course: 1; Do not inform: 9
Open in a new tab

* The numbers in the table refer to the number of articles with this situation in each systematic or meta-analysis(Citation)

Table 5 shows that some studies focused only on one lesson or content such as English or Mathematics while the majority of systematic reviews and meta-analyses investigated studies with intervention on different contents. Based on the findings, Science, Technology and Mathematics (STEM) accounted for a significant number of studies. Furthermore, in terms of educational grade, according to Table 4, the gamified educational interventions were conducted in all grades from pre-primary to postgraduate that most of them were conducted in Higher education.

The number of learners

Table 6 indicates that only 5 systematic reviews and meta-analyses examined the number of learners participating in interventions in included articles.

Table 6.

Samples size in gamified educational interventions

Citation Sample size
Ortiz-Rojas et al., 2017 21-100 students: 12; 101-200 students: 3; 201-300 students: 2; ≥301 students: 5; Not mentioned: 1
Bai et al., 2020 <50 students: 13; 50–100 students: 8; ≥150 students:8
Dehghanzadeh et al., 2019 <50 students: 11; 50–100 students: 7; >100 students: 3
Ortiz et al., 2016 <10 students: 1; 11-60 students: 13; 61-110 students: 5; 111-470 students: 9; 2263 students: 1; Not mentioned: 1
Ekici, 2021 <60 students: 8 l <120 students: 14; Not mentioned: 2
Open in a new tab

According to Table 6, the lowest and the highest sample size were less than 10 and 2263 participants, respectively. The sample size in most of the articles was less than 100 learners.

The used platforms in the gamified educational interventions

Table 7 indicates that four studies reported the name of the used platforms in included articles. Findings show that the majority of examined articles in systematic reviews and meta-analyses used some of the most exciting gamification platforms such as Cahoot and Quizziz.

Table 7.

Platforms used in gamified educational interventions*

Citation Platforms
Zainuddin et al., 2020 Adapted gamification platforms: ClassDojo and ClassBadges, Ribbonhero of Microsoft Rain classroom, Quizbot, Duolingo Kahoot and Quizizz, Math Widgets, Google + CommunitiesiSpring Learn LMS learning management system: MOOCs (Coursera, Udacity, and edX), wiki platforms, moodle platforms or institutional LMS
Kalogiannakis et al., 2021 Pre-existed gamified platform: Kahoot, ClassDojo, Socrative, Quizziz, Zondle, and 3D GameLab
Dehghanzadeh et al., 2019 WordBricks, Duolingo, Kahoot, Babbel, Jeopardy, ClassDojo, Lifeline, Feelbot, Brainscap
Ekici, 2021 Moodle, Kahoot, Blackboard, Socrative, iSpring Learn LMS, The Minimum Learning Judgement System, VoiceTube, Quizziz, Khan Academy LMS, Electronic Book
Open in a new tab

Table 7 shows that the most popular platforms are Kahoot, ClassDojo, Duolingo, Moodle, Quizziz and Khan.

The game elements

In every study in the field of the gamified educational environment, one or more game elements have been used. The game elements in educational interventions are one of the most important items that systematic review and meta-analyses articles reported them. Table 8 shows that 17 out of 25 studies examined the game elements in the included articles.

Table 8.

Game elements used in gamified educational interventions*

Citations Game elements (the number of studies that used the element) and the number of articles using these elements in every citation
Huang et al., 2020 Points/experience (24), Leaderboards (23), Badges/awards (22), Competition (21 ), Responsive feedback (19), Advancement/levels (14), Quests/missions/modules (12), Collaboration (9), Avatars/customization (8), Timed activity (6), Performance graphs (6), Non-linear navigation (5), Adaptivity/personalization (5), Narrative/storytelling (5)
Zainuddin et al., 2020 Point (38), Leaderboard (33), Badges (33), Levels (21), Trophies (7), Avatars (6), Gift (5), Progress bar (5), ranking (5)
Indriasari et al., 2020 Points (27), Leaderboards (22), Badges (26), Progress Bar (5), Virtual Gift (5), Level (4), Mission/ Quest (2), Prize (1)
(Subhash & Cudney, 2018) Point, leaderboard, badge, level, feedback, collaboration, graphics, design (goals, rules, time limit, competition), narrative, freedom to fail, real reward, role play
Ortiz-Rojas et al., 2017 Badges (13), Leaderboard (10), Points (6), Levels (4), Ranking (4), Challenges (3), Trophies (3), Virtual Currency (1), Feedback (1), Hearts (1), Quests (1), Scoring (1), Achievements (1), Avatars (1), Awards (1)
Kim & Castelli, 2021 Badges 15, leaderboard 14, points 13, progress bar 5, Challenge2, levels 2, avatar2, goals 1, peer assessment 1 , storytelling1, prize1
Garcia et al., 2020 Feedback (29), Pontuation (29), Levels (25), Rewards (21), Goals (19), Cooperation (15), Narrative (15), Real time (8), Objective History (1)
Kalogiannakis et al., 2021 Competition (15), points (13), levels (12), Leaderboard (12), Progression (11), Badges (6), Time- pressure (5), Rewards (4), Cooperation (4), Storytelling (3), Quizzes (3), Avatar (3), Score (2), Story-based (2), Narrative (2), Challenges (2), Collaboration (2), Stats (1), Repeat-testing (1), Puzzle (1), prizes (1), Goals/objectives (1), Feedback (1)
Dehghanzadeh et al., 2019 Feedback (22), Challenge (12), Reward (11), Point (11), Leaderboard (8), Level (7), Time pressure (6), Progress bar (6), Badge (5), Score system (3), Like or dislike (3), Narration (2), Answer question (2), Quest (2), Story (2),Achievement (2), Avatar (2), Character system (2), Curiosity (2), Emoticon (1),Fantasy (1), Mission (1), Virtual credit(1), Medal (1), Performance graph (1), Use of social media (1), Warning signal (1), Wall (1), Control (1), Appreciation (1), Freedom to fail (1), Chatting with users (1), Message (1), User guidance (1), Status (1), Rule (1), Specific phrases (1), Competition (1), Uploading (1), Chunking(1), Correctness bar (1), Peer assessment (1), Error typing (1), Profile (1), New feed (1)
Van Gaalen et al., 2021 Scoring/Points (15),Competition (13), Rewards (7), Time (6), Teams (4), Levelling (3), Crossword puzzle (2), Spaced-learning (2), Social network (2), Surprise (2), Role playing (1), Avatar (1), curiosity (1), Progress (1), signposting (1) , Mystery character (1), Awards (1), Badges (1), Chance (1)
Alomari et al., 2019 Points (30), Badges (27), leaderboards (25), Levels (14), Progress bar (5), Challenge (4), Feedback (4), Achievement rewards (3), Avatars (3), Quests (2), Ranking (2), Rewards (2), Social engagement (1), Storyline (1), Thumbs –ups (1), Trophies (1), Win-state (1), Real gifts (1), Reputation (1), Narrative (1), Progressive levels (1), group competition (1), Comparisons (1), Constraints (1), Cards (1), Awards (1)
Ortiz et al., 2016 Combination (18), Badges (7), Leaderboard (2), Points (1), Challenge (1), Quests (1)
Ekici, 2021 Points (17), Badges (14), Leaderboard (8), Levels (2), Progress bar (1), Virtual coins (1), Virtual Objects (1), Rewards (1)
Manzano-León et al., 2021 Points (10), Narrative (8), Badges (7), Ranking (6), Rewards (6), Challenge (4), Prize (3), Levels (3), Playful activities (2), Tasks (2), Events (1), Roles (1), Feedback (1), Choices (1), Competition (1), Achievements (1)
Tenório et al., 2018 Badges (9), Points (8), Leaderboard (6), Level (5), Avatar image (4), Teams (4), Avatar in 3D (3), Social Graph (3),Virtual Goods (2)
Bai et al., 2020 Badges + leaderboard/rank + points (8), Badges + leaderboard/rank + levels/unlock + points (6), Badges + points (4), Points (3), Badges + levels/unlock + points (3), Avatar+leaderboard/rank+levels/unlock +points+progress bar+team (collaboration, competition) (3)
Bai et al., 2019 Badges + leaderboard/rank + level/unlock + points: (4), Badges + leaderboard/rank + points (2), Badges + points + progress bar (1), Badges + leaderboard/rank + level/unlock + progress bar (1), Badges + leaderboard/rank + level/unlock (1), Badges (1), Badges + leaderboard/rank (1), Level/rank + points (1), Avatar + badges + leaderboard/rank + level/unlock + points + word notification (1)
Open in a new tab

* The numbers in the table refer to the number of articles with this situation in each systematic or meta-analysis(Citation)

Table 8 shows that the game elements used in educational programs are very different at different levels, but some elements are used more than others. The majority of the used game elements are Point, Leaderboard, Badge, Level, Feedback, Progress bar, Challenge and Avatar.

The theories applied in the gamified educational interventions

Theories that are the basis of designing gamified learning environments are among the cases that have been studied in these types of articles.Table 9 shows the titles of these theories.

Table 9.

Theories used in gamified educational interventions

Citation Theories
Zainuddin et al., 2020 Self-determination theory; flow theory; The goal-setting theory; Cognitive evaluation theory; Cognitive load theory; Behaviour reinforcement theory; Social comparison theory; Theory-driven gamification design model: goal, access, feedback, challenge and collaboration; Theory of reasoned action; Rational choice theory; Taxation theory; Information systems success model/information systems theory; Presence pedagogy model; Eisenkraft's 7E instructional Model; Felder-Silverman learning style model; Unified Modelling Language; Fogg's behavior model; Merrill's first principles of instruction design theory; Landers' theory of gamified learning; Social development theory: zone of proximal development and scaffolding; Self-efficacy theory; Constructivist learning theory; Technology-enhanced training effectiveness model
Kalogiannakis et al., 2021 self-determination theory; flow theory; goal-setting theory; cognitive theory of multimedia learning; motivation theory to learn
Osatuyi et al., 2018 Self-determination theory; Flow theory; Situated learning theory; Experiential learning theory; Uses andgratifications theory; Zone of proximal development; Achievement goal theory; Activity theory; Andragogy theory of adult learning; Cognitive evaluation theory; Cognitive load theory; Constructivist theories of learning; Grounded theory; social capital theory; Social cognitive theory; Social exchange theory; The frame model; The organismic integration theory (OIT); The SNAP: model of motivation; Trans-theoretical model of behavior change (TTM)
Van Gaalen et al., 2021 Experiential Learning Theory; Reinforcement Learning Theory; Social Comparison Theory; Self-Directed Learning; Deliberate Practice Theory
Open in a new tab

As the findings in Table 9 shows, the two theories of Self-determination theory and Flow theory in three studies and four theories, each in two studies have been reported as the most frequent theories.

Discussion

This study is a scientometrics, systematic, and co-occurrence analyses of systematic review and meta-analysis articles in the field of gamified education. According to or findings, 7 out of 24 articles were systematic reviews and the rest were meta-analyses with publication dates from 2000 to 2020.

Co-occurrence analysis of words indicated that motivation, learning and engagement are the most important concepts studied in articles in the field of gamified education. The results of a study showed that performance, participation, attitude, motivation, pleasure, perceived learning, satisfaction, practical skills, and increased learner competition are some benefits observed in studies related to gamification in education (Subhash & Cudney, 2018). In fact, learning engagement and motivation, learning achievement, interaction and social connection are some effects of these kinds of intervention. Gamified tests at the beginning and the end of each class increase learners' mastery of lesson content and engagement during class activities, as well as improve their cognitive, emotional, and behavioral engagement (Zainuddin et al., 2020). Gamification can be directly related to increasing learners' learning performance. However, some studies reflect weaker statistical differences between on-game and off-game environments (Ortiz-Rojas et al., 2017). The results of the studies indicate that in some gamified educational interventions, no improvement was observed in final exam scores, but perceived learning was widely concluded as a positive effect of gamification learning. Improving learners' performance in presenting higher quality projects, improving learning outcomes, reducing failure rates and higher average scores are also observed in game-based learning groups (Subhash & Cudney, 2018). The results of another study also showed that the level of participation had a higher effect size than the test score. Therefore, gamification has a greater effect on the level of learners' participation than the test score. Increasing the level of participation can develop learning skills and academic achievement. Thus, educators are expected to improve learners' participation levels using gamification strategies (Kim & Castelli, 2021).

In terms of content analysis, researchers extracted 7 fundamental categories. In the following, we have discussed every category.

Country/territory

Because of cultural differences in every country, learners' attitude and expectations might be different about learning via gamification (Subhash & Cudney, 2018). In addition, based on educational subjects, learners in different countries have different tendencies to gamified learning. For instance, in the field of higher education, Spain is the first country in regard with the highest number of studies in gamified learning and United States, Germany, and the United Kingdom are in the next ranks, respectively (Ortiz-Rojas et al., 2017). To measure the student learning outcomes, East Asia with 15 and Western Asia with 5 articles are in the first and second ranks (Bai et al., 2020). About the effect of gamification on academic success in students, both USA and Turkey (9 articles) and Spain (4 articles) have the most studies (Yıldırım & Şen, 2019). A systematic review by Gaalenet al. showed that in the field of medical education, the majority of studies were conducted in the USA and Canada (Van Gaalen et al., 2021).However, in terms of the use of gamification in collaborative learning, Spain had had conducted the highest number of studies (Dos Santos et al., 2020). European countries are pioneers in research on the application of gamification in Science, Technology, Engineering and Mathematics (STEM), followed by America, Asia and Africa, respectively (Ortiz et al., 2016). Moreover, researchers in countries such as Spain, Hong Kong and Turkey have shown great interest in gamified learning methods in measuring learners' learning performance in this field (Bai et al., 2019). Flipped learning is another field that has attracted gamification. While the studies in this field have been conducted in 12 different countries, Spanish researchers have the first rank and more than half of studies have been conducted in European countries (Ekici, 2021). The investigation of studies in the field of gamified education in information systems (IS) showed that Americans,Australians and German' researchers published 53, 44 and 22 articles, in turn (Osatuyi et al., 2018). Eventually, in the gamified nursing education, the United States and Canada have the highest number of publications (6 and 4, respectively) (Malicki et al., 2020).

Duration

An important parameter in gamified learning is how long the intervention s has taken place. In fact, this is important whether the effects of gamification last long time or not (Dichev & Dicheva, 2017; Hamari et al., 2014; Seaborn & Fels, 2015). Because duration of gamified course is considered as a potential modifier of effects on the results of cognitive, motivational and behavioral learning. However, there are conflicting findings in this regard. According to Wouters et al. (2013) when the participants participate in several sessions and play for longer period of time, the effects of games are greater, while findings of Kim and Castelli (2021) about the effect of gamification on behavioral change showed that gamified interventions lasting some days is more effective than those lasting one or two years. Thus, the studies recommend the short courses rather than longer ones in gamified learning. Another analysis carried out by Sailer and Homner indicated that both long and short-term interventions are useful in cognitive and behavioral learning. However, the interventions that lasted for half a year or less (but more than 1 month) have a moderate effect on motivational learning outcomes, while the effectiveness of interventions of one-day courses or less were negligible. In fact, for motivational outcomes, it may even take longer time to affect motivation. However, this does not lead to any conclusions about the durability of the effects obtained (Sailer & Homner, 2020). Based on Ekici (2021), the duration of gamification used in learning is up to 4 months. Hanus and Fox's findings highlights the negative effect of long-term gamification courses on intrinsic motivation, academic achievement and satisfaction while many studies have been conducted in a relatively short period of time (less than four months) (Hanus & Fox, 2015). For example, Hung (2017) used Kahoot for gamifying their education course in the control group, and the results showed that it was effective on increasing motivation and academic achievement in the short term. In another study, Chen and Hwang (2019) used Kahoot for only six weeks.

Game elements

There are three important issues with game elements including the type of element, the number of elements and the type of combination of game elements.

Type of used element

Our findings show that the game elements used in learning interventions do not have the same effects on learners' learning. For example, Huang et al. (2020) found that using a timed activity element produces a smaller effect size than other elements, while environments that do not use this element have a larger effect size. The same is true about the leaderboard element, though the difference is not significant. Even some studies showed that the most controversial element is the leaderboard that may harm learners' motivation when they are doing an explicit competition. Howeverr, in almost all articles, leaderboards were introduced as the most attractive elements of the game (Zainuddin et al., 2020). One study showed that the main game elements used in learning are points, medals, rankings, and narratives (Manzano-León et al., 2021). Nevertheless, a large number of studies showed that the most frequent elements are points, badges, and leaderboard (Ekici, 2021; Indriasari et al., 2020; Subhash & Cudney, 2018; Tenório et al., 2018; Zainuddin et al., 2020). On the other hand, few studies have reported quest, virtual goods or gifts as the game elements (Indriasari et al., 2020).Collaboration was also one of the most common game elements used in the form of teammates and discussion boards.The results of Subhash and Cudney' study illustrated that points, badges, leaderboard, levels, feedback, and graphics, as the most important game elements, are suitable for higher education environments (Subhash & Cudney, 2018).

Number of elements

There are also challenges regarding the number of elements used in gamification. Ekici's findings showed that most studies used more than one game element (Ekici, 2021) because when only one or two game elements such as points or badges are used in educational interventions, the effects on students' motivation are becoming less or even negative. Manzano-León et al.'s research reinforces the idea that a diverse gaming environment is more motivational and can meet the needs of its players according to their characteristics, a result consistent with that of the Kocadere and Çaglar (Manzano-León et al., 2021).According to Indriasari et al., 62% of studies used a combination of game elements, while only 15 studies reported using only one element (Indriasari et al., 2020). However, Manzano-León et al. (2021) showed that there was no significant difference between the effects of interventions that used more elements of the game with other interventions. Based on their findings, in different interventions, four elements, three elements, two elements, one element and six game elements have been used, respectively.

Combination of game elements

One of the important issues is the combination of game elements. While the greatest effect size for the gamification design feature was observed in the use of quests/missions/modules in the interventions (Huang et al., 2020), the results of several studies revealed that in most of the gamified educational interventions, the combination of badges+leaderboards+points is often used (Ekici, 2021; Kalogiannakis et al., 2021; Ortiz-Rojas et al., 2017). Bai et al. (2019) indicated that in some studies, the most used combination include badges, leaderboard/rank, level/unlock and points, followed by badges, leaderboard/rank, and points. The ranking of the impact of elements in terms of effect size alone or in combination with other elements in Huanget al.'s study is as follows: Quests/missions/modules, Collaboration, Avatars/customization, Adaptivity/personalization, Non-linear navigation, Responsive feedback, Advancement/levels, Narrative/storytelling, Points/experience, Badges/awards, Competition, Leaderboards, Performance graphs, Timed activity (Huang et al., 2020).

Participants (field of study and grade)

Participants’ field of study

The study by Bai et al. (2020) showed that there is no significant difference between gamification in different fields of study. They found that the effect size was not affected by student'gradelevel of education (e.g., elementary, high school and college students) and subject disciplines (e.g., computer and information science, math, science). However, various studies indicated that in some areas of science, gamification is more highlighted. The study by Huang et al. (2020) illustrated that most of the fields in which gamification was introduced were "social sciences" and "engineering and computer", both of which had a statistically significant effect size. In contrast, in subject areas such as "arts and humanities" with the effect size, was not statistically significant. Subject areas such as Math, Health care, and Business used less gamification in their educational settings. However, the study by Indriasari et al. indicated that most types of the gamification was applied in the fields of Physical sciences, Mathematics and Computer science, Engineering and Education. In addition, Science, Technology, Engineering, and Math (STEM) are disciplines in which most of the peer-to-peer review activities (Indriasari et al., 2020) and flipped gamification learning (Ekici, 2021) were reported. Subhash and Cudney also showed that 14 out of the 37 studies in the field are computing (Kim & Castelli, 2021).

Participants’ degree

In terms of participants, the results are somewhat contradictory. According to a study by Bai et al. (2020) the effect size in the high school environment was significantly larger than those in undergraduate and graduate levels. Their findings showed that the effect size was not affected by the participants' degree. Even Yıldırım and Şen (2019) showed that the effect of gamification on students' progress in different degrees of education was not different. Bai et al. (2020) showed that most of the studies were conducted with undergraduate students with the highest effect size and K-12 students were in the next rank but there was no significant difference. Yet, it is not surprising to see more studies on undergraduate education because they are more accessible for researchers in their institutions. However, the results of Bai et al. showed that the effect size of undergraduate students is almost twice as much as that of K-12 students. Zainuddin et al. (2020) also illustrated that most of the articles studied were related to adult learners or higher education students. The results of the study by Indriasari et al. (2020) also confirmed that a small number of studies were related to high school and primary school, and most of the articles were conducted in the university level, even in the studies of flipped education (Ekici, 2021). Perhaps, this is why, despite the results of some studies (Bai et al., 2020; Sailer & Homner, 2020; Yıldırım & Şen, 2019), gamification is not statistically significant in high school level (Yıldırım & Şen, 2019) and the effects of gamified interventions are much more effective for adults than K-12 and college students. In fact, it is possible that younger people and adults are more interested in the gamified factors in education than the age groups of college students. Because adults showed the highest participation rate compared to college students and K-12 students (Kim & Castelli, 2021). Only Sailer and Homner's study showed that gamified cognitive learning in school was better than other educational environments (Sailer & Homner, 2020). What is certain is while the majority of the research were related to students, a small number of studies in this field have provided solutions that are directly aimed at teachers (Garcia et al., 2020).More information about study field of the intervention and the participants’ level of education is shown in Table 6.

Gamification platforms

The platforms and applications used in gamified learning research are other considerable issues. The results of one of these studies by Zainuddin et al (2020) showed that most of the articles used existing platforms from different sources such as ClassDojo and ClassBadges, Ribbonhero of Microsoft Rain classroom, Quizbot, Duolingo, Kahoot and Quizizz, Math Widgets, Google + CommunitiesiSpring Learn LMS. The most common of these is Kahoot (Kalogiannakis et al., 2021). The integration of game elements in the Learning Management System (LMS) is also used. For example, by integrating gamification using Web 2 tools, new functions are created for MOOCs (Coursera, Udacity, and edX), wiki and moodle platforms, and enterprise learning management systems (Aparicio et al., 2019; Huang et al., 2019; B. Huang & Hew, 2018; Jurgelaitis et al., 2019; Özdener, 2018). In addition, the National Budget Forecasting project is another platform used (Buckley & Doyle, 2017). Some researchers have developed their game development platforms to prioritize user-centric needs and help to provide an effective online experience for a diverse range of users. Their goal is to improve the performance and participation of inclusive learning (8, 9) and to participate in online discussions using the tools of the game (Bouchrika et al., 2019; Ding, 2019; Ding et al., 2017, 2018). In terms of applying gamification in teaching English, different types of digital learning environments such as WordBricks, Duolingo, Kahoot, Babbel, Jeopardy, ClassDojo, Lifeline, Feelbot, Brainscap have been used to play LESL (Dehghanzadeh et al., 2019). The predominant environment/tool of gamification in flipped game education research are learning management systems such as Moodle, iSpring LMS, Blackboard, The Minimum Learning Judgment System, and Khan Academy LMS. Moodle was used in 9 studies and Kahoot in 7 studies, which were ranked first and second (Ekici, 2021).

Theories in gamified learning

The theories used in the design of gamified educational environments are the other important element. Self-determination and Flow theories (Kalogiannakis et al., 2021; Osatuyi et al., 2018) are the most frequent theories used in gamified studies. These two theories have been widely used in gamified studies of educational environments (Zainuddin et al., 2020). According to Kalogiannakis et al. (2021) that conducted a systematic review of articles related to gamification in science education, most of the articles included in the systematic review had no theoretical basis. Of the 24 studies reviewed, only six articles implicitly stated their theoretical framework, which self-determination theory is one of the most comprehensive and significant one. The results of a study by Osatuyi et al. (2018) showed that only 17 out of 41 existing articles were theoretically based. The results of this study showed that the following theories were dominant among the theoretical frameworks used in game development research:

  • Social theories (such as theories that support psychological processes such as social exchange theory, social capital theory, social cognitive theory);

  • Cognitive theories (such as cognitive evaluation theory, cognitive load theory, Kolb’s experiential learning theory, Lave’s situated learning theory, andconstructivist theories of learning);

  • Behavioral theories (such as self-determination theory (SDT) and flow theory).

Conclusion

The results showed that 344 articles in the field of gamified learning and education were reviewed in 25 systematic review and meta-analysis articles, most of which were in English. Therefore, there is a lack of systematic review research for articles in other languages. Content analysis showed that these 25 articles can be categorized in 7 categories based on the most important elements in the field of gamification and learning, including country/territory, duration of intervention, lessons/content, the number of learners, platforms, the game elements, theories. Based on results, all these items were not analyzed in all 25 articles. Therefore, it is suggested that these seven items be considered in subsequent systematic reviews studies and meta-analyses. In addition, the results showed that most of these studies have implemented gamification in online learning environments. There is a need for more research to gamify face-to-face classes.

On the other hand, most of the review articles were in the field of "social sciences" or "engineering and computer". It is suggested that studies be conducted to examine interventions in other disciplines and courses. The results showed that in most studies, due to time and cost issues, they preferred to use existing platforms and LMS.

The results of some studies showed that educational interventions were effective in promoting learning, motivation and participation of learners, but in most of these studies, the definite effect of gamification was not mentioned and among their research suggestions, the need for further studies was suggested. The results of some studies also reflected weaker statistical differences between gamified and non-gamified environments. Therefore, it is suggested that the higher quality studies (two groups with pre-test and post-test) be performed to determine the effect of gamification on variables. Finally, due to the inconsistency of the results of these studies, it is suggested that systematic review and meta-analysis studies focusing on the seven variables proposed in the present study.

Author contributions

Author 1: Contributed to conception, design, data acquisition and interpretation, performed all analyses, drafted and critically revised the manuscript.

Author 2: Contributed to conception, design, data acquisition and interpretation, performed all analyses, drafted and critically revised the manuscript.

Funding

Funding for this project was provided by the by Kashan university of medical sciences, Kashan, Iran, with No. 400062 (IR.KAUMS.NUHEPM.REC.1400.032).

Declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Ethical standards

This research did not involve Human Subjects.

All authors gave their final approval and agree to be accountable for all aspects of the work.

Footnotes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Somayyeh Nadi-Ravandi, Email: snadi2006@gmail.com.

Zahra Batooli, Email: batooli91@gmail.com.

References

  1. Ahmadi H, Osareh F. Co-word Analysis Concept, Definition and Application. National Studies on Librarianship and Information Organization. 2017;28(1):125–145 . [Google Scholar]
  2. Al-Zaman MS. A bibliometric and co-occurrence analysis of COVID-19–related literature published between December 2019 and June 2020. Science Editing. 2021;8(1):57–63. doi: 10.6087/kcse.230. [DOI] [Google Scholar]
  3. Alomari, I., Al-Samarraie, H., & Yousef, R. (2019). The role of gamification techniques in promoting student learning: A review and synthesis. Journal of Information Technology Education: Research, 18, 395–417. 10.28945/4417
  4. Aparicio M, Oliveira T, Bacao F, Painho M. Gamification: A key determinant of massive open online course (MOOC) success. Information and Management. 2019;56(1):39–54. doi: 10.1016/j.im.2018.06.003. [DOI] [Google Scholar]
  5. Atlasi R, Noroozi Chakoli A, Ramezani A, Tabatabaei-Malazy O, Larijani B. Scientometric analyzing the output of researchers and organizations on COVID-19 for better conducting the scientific efforts: With a glance to endocrinology. Journal of Diabetes and Metabolic Disorders. 2021;20(1):107–118. doi: 10.1007/s40200-020-00718-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bai S, Hew KF, Huang, B. (2019). Examining the effect of gamification in information science, computer and engineering education: A Meta-analysis of Student Learning Performance. Icce 2019 - 27th International Conference on Computers in Education, Proceedings, 1, 515–524. https://journament.com/show_manuscript/98748.
  7. Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. In Educational Research Review (Vol. 30, p. 100322). Elsevier Ltd. 10.1016/j.edurev.2020.100322.
  8. Bouchrika I, Harrati N, Wanick V, Wills G. Exploring the impact of gamification on student engagement and involvement with e-learning systems. Interactive Learning Environments. 2019 doi: 10.1080/10494820.2019.1623267. [DOI] [Google Scholar]
  9. Brick, M., Setyosari, B., Ulfa, P., & Kuswandi, S. (2020). A Meta-Analysis of Selected Studies on the Effectiveness of Gamification Method for Children. International Journal of Instruction, 13(1), 845–854. 10.29333/iji.2020.13154a
  10. Buckley P, Doyle E. Individualising gamification: An investigation of the impact of learning styles and personality traits on the efficacy of gamification using a prediction market. Computers and Education. 2017;106:43–55. doi: 10.1016/j.compedu.2016.11.009. [DOI] [Google Scholar]
  11. Burke B. amification 2020: What is the future of gamification. Gartner; 2012. [Google Scholar]
  12. Chen, J., Lu, Y., Zhang, T., & Ouyang, Z. (2020). Artificial Intelligence in Medicine in the United States, China and India. PervasiveHealth: Pervasive Computing Technologies for Healthcare, 257–264. 10.1145/3429889.3429938
  13. Chen P, Hwang G. An IRS-facilitated collective issue-quest approach to enhancing students’ learning achievement, self-regulation and collective efficacy in flipped classrooms. British Journal of Educational Technology. 2019;50(4):1996–2013. doi: 10.1111/bjet.12690. [DOI] [Google Scholar]
  14. Chou Y. Actionable gamification: Beyond points, badges, and leaderboards. Octalysis Media; 2016. [Google Scholar]
  15. Da Silva, R. J. R., Rodrigues, R. G., & Leal, C. T. P. (2019). Gamification in management education: A systematic literature review. BAR - Brazilian Administration Review, 16(2). 10.1590/1807-7692bar2019180103
  16. Dehghanzadeh, H., Fardanesh, H., Hatami, J., Talaee, E., & Noroozi, O. (2019). Using gamification to support learning English as a second language: A systematic review. Computer Assisted Language Learning,1–24,. 10.1080/09588221.2019.1648298
  17. Deterding, S., Khaled, R., Nacke, L., & Dixon, D. (2011). Gamification: Toward a definition, CHI 2011 gamification workshop proceedings. 2011 Annual Conference on Human Factors in Computing Systems (CHI’11).
  18. Dichev, C., & Dicheva, D. (2017). Gamifying education: what is known, what is believed and what remains uncertain: a critical review. In International Journal of Educational Technology in Higher Education (Vol. 14, Issue 1, p. 9). Springer Netherlands. 10.1186/s41239-017-0042-5
  19. Ding L. Applying gamifications to asynchronous online discussions: A mixed methods study. Computers in Human Behavior. 2019;91:1–11. doi: 10.1016/j.chb.2018.09.022. [DOI] [Google Scholar]
  20. Ding L, Er E, Orey M. An exploratory study of student engagement in gamified online discussions. Computers and Education. 2018;120:213–226. doi: 10.1016/j.compedu.2018.02.007. [DOI] [Google Scholar]
  21. Ding L, Kim CM, Orey M. Studies of student engagement in gamified online discussions. Computers and Education. 2017;115:126–142. doi: 10.1016/j.compedu.2017.06.016. [DOI] [Google Scholar]
  22. Dos Santos, L. S., De Lima Sobreira, P., Santiago, L. M. S., Abijaude, J. W., El Guemhioui, K., & Wahab, O. A. (2020, March 1). Gamification-Supported Collaborative Learning: A Systematic Literature Review. EDUNINE 2020 - 4th IEEE World Engineering Education Conference: The Challenges of Education in Engineering, Computing and Technology without Exclusions: Innovation in the Era of the Industrial Revolution 4.0, Proceedings. 10.1109/EDUNINE48860.2020.9149543
  23. Ekici M. A systematic review of the use of gamification in flipped learning. Education and Information Technologies. 2021;26(3):3327–3346. doi: 10.1007/s10639-020-10394-y. [DOI] [Google Scholar]
  24. Garcia FYH, Rangel EGH, Mera NAG. Gamification in mathematics education: A systematic review. Telos-Revista Interdisciplinaria En Ciencias Sociales. 2020;22(1):62–75. doi: 10.36390/telos221.05. [DOI] [Google Scholar]
  25. Ghaffari, S., Zakiani, S., Mirnejad, F., & Akbari, P. (2019). Assessment and evaluation of scientific Output of Iranian Dental researchers during 2010–2017 in ISC. Journal of Research in Dental Sciences, 16(4), 295–302. 10.29252/jrds.16.4.295
  26. Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? - A literature review of empirical studies on gamification. Proceedings of the Annual Hawaii International Conference on System Sciences,3025–3034,. 10.1109/HICSS.2014.377
  27. Hanus MD, Fox J. Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers and Education. 2015;80:152–161. doi: 10.1016/j.compedu.2014.08.019. [DOI] [Google Scholar]
  28. Huang B, Hew KF. Implementing a theory-driven gamification model in higher education flipped courses: Effects on out-of-class activity completion and quality of artifacts. Computers and Education. 2018;125:254–272. doi: 10.1016/j.compedu.2018.06.018. [DOI] [Google Scholar]
  29. Huang B, Hew KF, Lo CK. Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement. Interactive Learning Environments. 2019;27(8):1106–1126. doi: 10.1080/10494820.2018.1495653. [DOI] [Google Scholar]
  30. Huang R, Ritzhaupt AD, Sommer M, Zhu J, Stephen A, Valle N, Hampton J, Li J. The impact of gamification in educational settings on student learning outcomes: A meta-analysis. Educational Technology Research and Development. 2020;68(4):1875–1901. doi: 10.1007/s11423-020-09807-z. [DOI] [Google Scholar]
  31. Hung HT. Clickers in the flipped classroom: Bring your own device (BYOD) to promote student learning. Interactive Learning Environments. 2017;25(8):983–995. doi: 10.1080/10494820.2016.1240090. [DOI] [Google Scholar]
  32. Huotari, K., & Hamari, J. (2012). Defining gamification - A service marketing perspective. Proceedings of the 16th International Academic MindTrek Conference 2012: “Envisioning Future Media Environments”, MindTrek 2012, 17–22. 10.1145/2393132.2393137
  33. Indriasari TD, Luxton-Reilly A, Denny P. Gamification of student peer review in education: A systematic literature review. Education and Information Technologies. 2020;25(6):5205–5234. doi: 10.1007/s10639-020-10228-x. [DOI] [Google Scholar]
  34. Jurgelaitis M, Čeponienė L, Čeponis J, Drungilas V. Implementing gamification in a university-level UML modeling course: A case study. Computer Applications in Engineering Education. 2019;27(2):332–343. doi: 10.1002/cae.22077. [DOI] [Google Scholar]
  35. Kalogiannakis M, Papadakis S, Zourmpakis A-I. Gamification in Science Education. A Systematic Review of the Literature. Education Sciences. 2021;11(1):22. doi: 10.3390/educsci11010022. [DOI] [Google Scholar]
  36. Kapp, K. M. (2012). The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. In International Journal of Gaming and Computer-Mediated Simulations (Vol. 4, Issue 4). John Wiley & Sons. 10.4018/jgcms.2012100106
  37. Kim, J., & Castelli, D. M. (2021). Effects of gamification on behavioral change in education: A meta-analysis. In International Journal of Environmental Research and Public Health (Vol. 18, Issue 7). MDPI AG. 10.3390/ijerph18073550 [DOI] [PMC free article] [PubMed]
  38. Lora AV, Andreu MGN, Lo. Alternative metrics for assessing the social impact of tourism research. Sustainability (switzerland) 2020;12(10):4299. doi: 10.3390/su12104299. [DOI] [Google Scholar]
  39. Makkizadeh, F, Hazeri, A., & Keikhaee, F. (2016). The Study of Thematic Structure of Documents related to the Treatment of Diabetes Mellitus, Type 2 in PubMed from 2005–2014. Journal of Health Administration, 19(64), 43–55.‬‬‬‬‬
  40. Malicki A, Vergara FH, Van de Castle B, Goyeneche P, Mann S, Scott MP, Seiler J, Meneses MZ, Whalen M. Gamification in nursing education: An integrative literature review. Journal of Continuing Education in Nursing. 2020;51(11):509–515. doi: 10.3928/00220124-20201014-07. [DOI] [PubMed] [Google Scholar]
  41. Manzano-León A, Camacho-Lazarraga P, Guerrero MA, Guerrero-Puerta L, Aguilar-Parra JM, Trigueros R, Alias A. Between Level Up and Game Over: A Systematic Literature Review of Gamification in Education. Sustainability. 2021;13(4):2247. doi: 10.3390/su13042247. [DOI] [Google Scholar]
  42. Mostafavi E, Bazrafshan A. A Scientometric Overview of 36 Years of Scientific Productivity by Pasteur Institute of Iran in ISI SCIE. The Journal of Health Administration. 2011;14(45):7–10. [Google Scholar]
  43. Nicholson, S. (2015). A recipe for meaningful gamification. Gamification in Education and Business,1–20,. 10.1007/978-3-319-10208-5_1
  44. Niknejad, N., Ismail, W., Bahari, M., Hendradi, R., & Salleh, A. Z. (2021). Mapping the research trends on blockchain technology in food and agriculture industry: A bibliometric analysis. Environmental Technology and Innovation,21,. 10.1016/j.eti.2020.101272
  45. Ortiz-Rojas, M., Chiluiza, K., & Valcke, M. (2017). Gamification and Learning Performance: A Systematic Review of the Literature. European Conference on Games Based Learning. https://www.proquest.com/openview/e88cc51e3bc6c29b694a9a4851f520ce/1?cbl=396495&pq-origsite=gscholar
  46. Ortiz, M., Chiluiza, K., & Valcke, M. (2016). Gamification in Higher Education and Stem: a Systematic Review of Literature. EDULEARN16 Proceedings, 1, 6548–6558. 10.21125/edulearn.2016.0422
  47. Osatuyi, B., Osatuyi, T., & De La Rosa, R. (2018). Systematic review of gamification research in is education: A multi-method approach. Communications of the Association for Information Systems, 42(1), 95–124. 10.17705/1CAIS.04205
  48. Özdener, N. (2018). Gamification for enhancing Web 2.0 based educational activities: The case of pre-service grade school teachers using educational Wiki pages. Telematics and Informatics, 35(3), 564–578. 10.1016/j.tele.2017.04.003
  49. Pinto RD, Peixoto B, Melo M, Cabral L, Bessa M. Foreign Language Learning Gamification Using Virtual Reality—A Systematic Review of Empirical Research. Education Sciences. 2021;11(5):222. doi: 10.3390/educsci11050222. [DOI] [Google Scholar]
  50. Realinfluencers, R. (2019). 8 methodologies that every 21st century teacher should know. Blinklearning. https://www.realinfluencers.es/en/2019/05/09/8-21st-century-methodologies/.
  51. Sailer M, Homner L. The Gamification of Learning: A Meta-analysis. Educational Psychology Review. 2020;32(1):77–112. doi: 10.1007/s10648-019-09498-w. [DOI] [Google Scholar]
  52. Schönen, R. (2014). Gamification in change management processes (Issue Fk 07). http://www.enterprise-gamification.com/index.php?option=com_content&view=article&id=196:bachelor-thesis-gamification-in-change-management-processes&catid=4:blog&Itemid=251&lang=en
  53. Seaborn K, Fels DI. Gamification in theory and action: A survey. International Journal of Human Computer Studies. 2015;74:14–31. doi: 10.1016/j.ijhcs.2014.09.006. [DOI] [Google Scholar]
  54. Subhash S, Cudney EA. Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior. 2018;87:192–206. doi: 10.1016/j.chb.2018.05.028. [DOI] [Google Scholar]
  55. Tenório MM, Reinaldo FAF, Góis LA, Lopes RP, Santos Junior DOS, G. Elements of gamification in virtual learning environments: A systematic review. Advances in Intelligent Systems and Computing. 2018;716:86–96. doi: 10.1007/978-3-319-73204-6_12. [DOI] [Google Scholar]
  56. Tran, B. X., Van Pham, T., Ha, G. H., Ngo, A. T., Nguyen, L. H., Vu, T. T. M., Do, H. N., Nguyen, V., Le Nguyen, A. T., Tran, T. T., Truong, N. T., Hoang, V. Q., Ho, T. M., Van Dam, N., Vuong, T. T., Nguyen, H. Q., Le, H. T., Do, H. T., Moir, M., … Ho, R. C. M. (2018). A bibliometric analysis of the global research trend in child maltreatment. International Journal of Environmental Research and Public Health, 15(7). 10.3390/ijerph15071456 [DOI] [PMC free article] [PubMed]
  57. Van Gaalen AEJ, Brouwer J, Schönrock-Adema J, Bouwkamp-Timmer T, Jaarsma ADC, Georgiadis JR. Gamification of health professions education: A systematic review. Advances in Health Sciences Education. 2021;26(2):683–711. doi: 10.1007/s10459-020-10000-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Wouters P, Van Nimwegen C, Van Oostendorp H, Van Der Spek ED. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology. 2013;105(2):249–265. doi: 10.1037/a0031311. [DOI] [Google Scholar]
  59. Yıldırım İ, Şen S. The effects of gamification on students’ academic achievement: A meta-analysis study. Interactive Learning Environments. 2019 doi: 10.1080/10494820.2019.1636089. [DOI] [Google Scholar]
  60. Zainuddin Z, Chu SKW, Shujahat M, Perera CJ. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review. 2020;30:100326. doi: 10.1016/j.edurev.2020.100326. [DOI] [Google Scholar]
  61. Zhang K, Wang Q, Liang QM, Chen H. A bibliometric analysis of research on carbon tax from 1989 to 2014. Renewable and Sustainable Energy Reviews. 2016;58:297–310. doi: 10.1016/j.rser.2015.12.089. [DOI] [Google Scholar]

Articles from Education and Information Technologies are provided here courtesy of Nature Publishing Group

RESOURCES