Abstract
Background
Artificial intelligence (AI) and game-based methods such as serious games or gamification are both emerging technologies and methodologies in health care. The merging of the two could provide greater advantages, particularly in the field of therapeutic interventions in medicine.
Objective
This scoping review sought to generate an overview of the currently existing literature on the connection of AI and game-based approaches in health care. The primary objectives were to cluster studies by disease and health topic addressed, level of care, and AI or games technology.
Methods
For this scoping review, the databases PubMed, Scopus, IEEE Xplore, Cochrane Library, and PubPsych were comprehensively searched on February 2, 2022. Two independent authors conducted the screening process using Rayyan software (Rayyan Systems Inc). Only original studies published in English since 1992 were eligible for inclusion. The studies had to involve aspects of therapy or education in medicine and the use of AI in combination with game-based approaches. Each publication was coded for basic characteristics, including the population, intervention, comparison, and outcomes (PICO) criteria; the level of evidence; the disease and health issue; the level of care; the game variant; the AI technology; and the function type. Inductive coding was used to identify the patterns, themes, and categories in the data. Individual codings were analyzed and summarized narratively.
Results
A total of 16 papers met all inclusion criteria. Most of the studies (10/16, 63%) were conducted in disease rehabilitation, tackling motion impairment (eg, after stroke or trauma). Another cluster of studies (3/16, 19%) was found in the detection and rehabilitation of cognitive impairment. Machine learning was the main AI technology applied and serious games the main game-based approach used. However, direct interaction between the technologies occurred only in 3 (19%) of the 16 studies. The included studies all show very limited quality evidence. From the patients’ and healthy individuals’ perspective, generally high usability, motivation, and satisfaction were found.
Conclusions
The review shows limited quality of evidence for the combination of AI and games in health care. Most of the included studies were nonrandomized pilot studies with few participants (14/16, 88%). This leads to a high risk for a range of biases and limits overall conclusions. However, the first results present a broad scope of possible applications, especially in motion and cognitive impairment, as well as positive perceptions by patients. In future, the development of adaptive game designs with direct interaction between AI and games seems promising and should be a topic for future reviews.
Keywords: artificial intelligence, AI, games, serious games, gamification, health care, review
Introduction
Background
Artificial intelligence (AI) and serious games are both relevant topics in the health sector, and the body of studies and literature is continuously growing. Interestingly, in terms of the research landscape, the 2 topics are not connected; rather, existing research views them independently.
The use of games for educational and serious purposes is nearly as old as the history of humankind and is an integral part of our culture [1]. In 1970, Abt [2] used the term “serious games” for the first time in his book with the same name. Sawyer and Smith [3] take a broad definition and consider serious games as “any computerized game whose chief mission is not entertainment and all entertainment games which can be reapplied to a different mission other than entertainment.” What serious games have in common is that they pursue a concrete (pedagogical) intention and aim to provide information on a specific topic (eg, health) that is accessible in an entertaining and interactive way to deepen competencies or to achieve a change in behavior [4].
Serious games for health can be used in the fields of medical diagnostics, therapy, and prevention, as well as health promotion and medical or patient education [5]. From a didactic and learning psychology perspective, the effect of serious games is based on the integration of the motivating and multimedia aspects of computer and video games. Serious games can increase engagement, motivation, enthusiasm, and interest [6,7]. There are several existing use cases in health contexts [8-11]. One example is the game EndeavorRx. In 2020, the US Food and Drug Administration permitted its marketing as the first game-based digital therapeutic device to improve attention function in children with attention-deficit/hyperactivity disorder (ADHD) [12]. The game Re-Mission was developed for children with cancer and showed good results regarding compliance and the understanding of disease-related issues in the target group [13]. EMERGE is a simulation game that recreates an emergency department in real time to improve the clinical reasoning skills of physicians [14].
Next to serious games, gamification has emerged as a major trend in the health sector, which is reflected in a growing number of publications, including several meta-analyses [15-17]. The most used definition of this concept is “the use of game design elements in non-game contexts” [18]. The motivational effect of the game elements can be explained in different ways. Sailer et al [19] established the link between various gamification elements (eg, points, leaderboards, and badges) and the self-determination theory proposed by Ryan and Deci [20]. As a theory of motivation, this defines three universal psychological basic needs that determine human action: (1) competence, (2) autonomy, and (3) social inclusion. If ≥1 of these needs are addressed (eg, through gamification elements), this has positive effects on behavior and its determinants [19]. In the health sector, there are numerous studies that have demonstrated the effects of using gamification on motivation, performance, engagement, health, and well-being status [5,21,22].
According to Westera et al [23], computer games have been linked with AI since the first computer was programmed to play chess [24]. New AI methods have been used in computer games, for instance, to generate levels, scenarios, and storylines; to balance complexity; or to add intelligent behaviors to nonplayer characters (NPCs) [25]. However, over the years, various authors have pointed at the marginal penetration of academic game AI methods in industrial game production [26]. AI techniques will become indispensable to coordinate the ever-growing complexity and dynamics of games [23]. AI-driven adaptation and assessment systems are used to offer learner-centered environments [27]. As an example, NPCs controlled by AI can adapt to the behavior of the gamer and can enrich immersive and challenging experiences within the game play.
When transferring these principles to health care, the interaction between AI and games could provide a benefit, especially in the management of chronic diseases, which most game designs already target. The possibility to quickly adapt to new game-generated data or performance and provide live feedback could lead to more individual and thus more patient-centered game design in both illness detection and treatment. This could increase motivation and engagement for patients, leading to higher therapy adherence through more personal involvement. The vast body of evidence in the field of serious games and gamification, along with the growing body of evidence in the use of AI, may thus form a new field of research.
Scope
Therefore, this scoping review sought to generate an overview of the currently existing literature on the interaction of AI and game-based approaches in health care. At this point, to the best of our knowledge, this is the only review that targets this interaction.
The primary objective was to analyze the current body of evidence based on (1) the disease or health issue being evaluated, (2) the process of care in which these projects are located, and (3) the kind of AI and type of game-based approach used and the interaction of both techniques.
A secondary objective was to obtain an overview of publications on the interaction of AI and game-based approaches such as serious games, gamification, commercial games, and game periphery. Another secondary objective was to analyze the quality of the existing studies in this field regarding their grade of evidence and the conducted study types.
Methods
Overview
For this scoping review, we applied the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols; Figure 1) guidelines [28]. Furthermore, we used the recommendations of the Cochrane Consortium for conducting systematic reviews and the RefHunter website as guidance [29].
Before starting the review process, we defined the inclusion and exclusion criteria (Textbox 1).
Inclusion and exclusion criteria.
Inclusion criteria
Article type: original study, journal article, or conference paper
Article scope: articles report the use of artificial intelligence (AI), machine learning, and deep learning in combination with game-based approaches (serious games, gamification, and game-based-learning)
Health profession: medicine
Area of application: articles that conducted research in the field of education, therapy, and health
Language: English
Publication period: last 20 years
Exclusion criteria
Article type: opinion, commentary, or letter to the editor
Article scope: not related to AI and game-based approaches
Health profession: other than medicine
Area of application: not related to health and medicine
Language: not in English
Publication period: published >20 years ago
For this review, we conducted the following steps:
Literature search
Title and abstract screening
Content screening
Further in-depth screening (snowballing method and asking colleagues)
Step 1: Literature Search
We applied the search terms primarily in the database PubMed on February 2, 2022. We tested and honed different search terms and Boolean operators (Multimedia Appendix 1) until sufficiently fitting results seemed to have been obtained (n=305). The final search term was defined as follows:
[“game” OR “gamification”] AND “artificial intelligence”
The search was extended to more open databases to assess studies that target AI and serious games in medicine-related research areas or interprofessional approaches that may include medical professions and in more technically oriented databases such as IEEE Xplore [30] to include papers from informatics and engineering with a focus on technical issues.
The same search term ([“game” OR “gamification”] AND “artificial intelligence”) AND “artificial intelligence”) was used for IEEE Xplore (n=98), Cochrane Library [31] (n=25), and PubPsych [32] (n=89). In Scopus, the search term used in the other databases showed fewer results and were modified to extend the range of hits (“serious” AND “game” AND “artificial intelligence”; n=41).
In addition, we conducted a search in Google Scholar [33]. However, the results from Google Scholar were not precise enough for inclusion in a review, which is consistent with the results of several studies [34-36].
After deduplication using Rayyan software (Rayyan Systems Inc), the combined search in these databases identified 545 (97.7%) publications out of the initial 558 identified. We then performed a manual deduplication, which resulted in 60 (11%) of the 545 publications being excluded; thus 485 (89%) publications remained (Multimedia Appendices 1 and 2; Multimedia Appendix 3 [37-51]).
Step 2: Title and Abstract Screening
For the second step of the scoping review (title and abstract screening), we used Rayyan software. The results from the literature search were transferred to the citation software Zotero (version 5.0.85; Rayyan) and to Rayyan software [52]. This software automatically identified duplicates. After iterative deduplication, the publications were subjected to manual screening. The first screening step was conducted using Rayyan and permitted publication inclusion based on their titles and abstracts. Given the volume of the publications to be screened, the title and abstract screening was distributed among 2 authors of this paper (JS and DT). To ensure the uniformity of the screening, the authors conducted several training sessions in Rayyan with the coreviewers. In addition, the authors randomly double-checked some of the excluded publications (25/485, 5.2%) to warrant the consistency of the screening by the other reviewer. This step was conducted independently by both researchers and was followed by a discussion of the results between the 2. As all analysis steps were conducted independently by the 2 researchers, a discussion of differently categorized literature (marked as “Conflict” in Rayyan) and subsequent adaptation were necessary in this step. Overall, only a few adjustments were necessary, and a good agreement between the 2 researchers could be reached. Possible conflicts and all included articles were discussed with a third team member (SK). The data generated by Rayyan can be found in Multimedia Appendix 2.
Step 3: Content Screening
After the primary screening, full-text publications were screened by the 2 lead authors. Toward this end, a table was prepared to compile relevant information (Textbox 2).
Information compiled for full-text screening.
Relevant details obtained
Authors, year, title, journal, and digital object identifier (DOI)
Study type (according to Röhrig et al [53])
Population, intervention, comparison, and outcomes (PICO) criteria
Subject (topic of the study)
Level of evidence (according to the Oxford Centre for Evidence-Based Medicine: Levels of Evidence [54])
Disease or health issue
Level of care (prevention, diagnostics, therapy, rehabilitation, nursing, organization or monitoring, and other [55])
Game variants (serious games, gamification, games, and game controller or periphery)
Artificial intelligence (AI) technology (machine learning, deep learning, and AI [not further specified])
Function type (promoting health literacy, analysis and cognition, indirect intervention, direct intervention, documentation of health and medical history, organization and administration, and purchasing and supply) [56]
Some of the studies (4/16, 25%) showed an overlap among different categories (eg, in level of care). In these cases, double classifications were performed. All eligible studies were categorized and coded in detail (Multimedia Appendix 3 [37-51]).
Step 4: Further In-Depth Screening (Snowballing Method and Asking Colleagues)
After conducting the scoping review, we additionally used the “snowballing” approach described by Greenhalgh and Peacock [57], who have stated that in reviews of complex and heterogeneous evidence, formal protocol-driven search strategies may fail to identify important evidence. Informal approaches such as browsing and asking colleagues can substantially increase the efficiency of search efforts. Snowballing methods such as pursuing references of references and electronic citation tracking are very useful for identifying high-quality sources in obscure locations. Therefore, to validate the results of the review, the 2 reviewers searched the literature references used in meta-analyses, reviews, and papers that were closely related to the topic of the search. In addition to using the snowballing method, the method of asking colleagues, as recommended by Greenhalgh and Peacock [57], was applied as a last step.
Results
Overview
When applying the aforementioned search terms, we initially identified 335 studies on the topic of games and the use of AI in health care in the last 20 years. The subsequently performed step of title and abstract screening reduced the number of the initially identified studies from 335 to 47 (14%). In the next step, assessing the actual full-text literature, of the 47 papers, 3 (6%) were excluded because their full text was not in English, and the aforementioned inclusion and exclusion criteria were applied to the remaining 44 (94%). After the full-text screening, 10 (23%) of the 44 papers met all inclusion criteria. Using the snowballing method, 1 additional paper could be identified. Asking colleagues revealed 5 additional papers, which led to an overall total of 16 eligible papers (Figure 1). Not all criteria showed hits (eg, function type showed hits only in 2 categories, whereas level of care showed no hits in nursing).
Categories
Overview
The eligible papers showed a clear emphasis on certain categories (Table 1). Regarding the targeted diseases, the field of motion impairment was investigated the most (5/16, 31%). Cognitive impairment was targeted in 19% (3/16) of the studies, phantom limb pain or limb absence in 19% (3/16), rheumatoid arthritis in 13% (2/16), cancer in 6% (1/16), and ADHD in 6% (1/16). The primary focus on rehabilitation (10/16, 63%) was the most compelling. Of the 16 studies, 5 (31%) took place in the field of prevention, 4 (25%) in the field of diagnostics, and 1 (6%) in a nonrehabilitation therapeutic context (4 double assignments).
Table 1.
Category | Studies, n (%) | |
Disease or health topic (n=16) | ||
|
Motion impairment | 5 (31) |
|
Phantom limb pain or limb absence | 3 (19) |
|
Cognitive impairment | 3 (19) |
|
Rheumatoid arthritis | 2 (13) |
|
Cancer | 1 (6) |
|
Attention-deficit/hyperactivity disorder | 1 (6) |
|
Other | 1 (6) |
Function type (n=16) | ||
|
Direct intervention | 9 (56) |
|
Analysis and cognition | 7 (44) |
Level of care (n=20a) | ||
|
Prevention | 5 (25) |
|
Diagnostics | 4 (20) |
|
Therapy | 1 (5) |
|
Rehabilitation | 10 (50) |
AIb technology (n=16) | ||
|
Machine learning | 12 (75) |
|
Deep learning | 2 (13) |
|
AI (not further specified) | 2 (13) |
Game variant (n=16) | ||
|
Serious games | 11 (69) |
|
Gamification | 1 (6) |
|
Games | 3 (19) |
|
Game periphery | 1 (6) |
aA total of 4 studies showed an overlap between prevention and diagnostics and were double classified, resulting in an overall total of 20 studies.
bAI: artificial intelligence.
Most of the studies (12/16, 75%) applied machine learning as the AI technology, and 13% (2/16) used deep learning, whereas the remaining studies (2/16, 13%) did not specify the AI technology. Most of the studies (11/16, 69%) used a serious game, whereas 19% (3/16) used a commercial games approach. Despite the highly increased use of gamification in health and education, of the 16 studies, only 1 (6%) specifically used gamification to improve the motivation of patients, and 1 (6%) used game design–like interactions.
We further clustered and outlined the eligible papers according to the targeted disease (Table 2). A more detailed description of every included publication with a more specific outline of the use of AI and game variant can be found in Multimedia Appendix 4 [37-51].
Table 2.
Authors; year | Target group (participants, n) | Subject | Study design | Level of evidencea | Level of care | Function type | AIb technology | Game variant | |||||||||
Disease or health topic: motion impairment | |||||||||||||||||
|
Yeh et al [43]; 2014 | Patients (48) | Noninvasive balance training | Case control study | 3b | Therapy | Direct intervention | Machine learning | Games | ||||||||
|
Lyu et al [44]; 2019 | Healthy individuals (8) | Electromyography-controlled knee exoskeleton | Quantitative, proof of concept | 5 | Rehabilitation | Analysis and cognition | Deep learning | Games | ||||||||
|
Nasri et al [45]; 2020 | Patients (15) | Real-time hand gesture recognition | Case series | 4 | Rehabilitation | Direct intervention | Deep learning | Serious games | ||||||||
|
Burdea et al [42]; 2021 | Healthy individuals (2) | Game controller–based telerehabilitation | Proof of concept | 5 | Rehabilitation | Direct intervention | AI | Game controller or periphery | ||||||||
|
Zhang et al [46]; 2021 | Patients (5) | Gait analysis and waist motion capture | Case series | 4 | Rehabilitation | Direct intervention | Machine learning | Serious games | ||||||||
Disease or health topic: phantom limb pain or limb absence | |||||||||||||||||
|
Ortiz-Catalan et al [47]; 2016 | Patients (14) | Phantom motor execution | Quantitative clinical trial | 4 | Rehabilitation | Direct intervention | Machine learning | Games | ||||||||
|
Lendaro et al [48]; 2019 | Patients (4) | Phantom motor execution | Quantitative clinical trial | 4 | Rehabilitation | Analysis and cognition | Machine learning | Serious games | ||||||||
|
Kristofferson et al [49]; 2021 | Patients (4) | Prosthesis system | Explorative study | 4 | Rehabilitation | Direct intervention | Machine learning | Serious games | ||||||||
Disease or health topic: cognitive impairment | |||||||||||||||||
|
Valladares-Rodriguez et al [50]; 2018 | Healthy individuals or patients (16) | Early detection of mild cognitive impairment | Proof of concept | 5 | Prevention or diagnostics | Analysis and cognition | Machine learning | Serious games | ||||||||
|
Valladares-Rodriguez et al [51]; 2019 | Patients (74) | Early detection of mild cognitive impairment | Case series | 4 | Prevention or diagnostics | Analysis and cognition | Machine learning | Serious games | ||||||||
|
Jung et al [37]; 2019 | Patients (12) | Mini-Mental State Examination | Case series | 4 | Rehabilitation | Direct intervention | Machine learning | Serious games | ||||||||
Disease or health topic: cancer | |||||||||||||||||
|
Good et al [39]; 2014 | Registered players (1077) | Gene Selection for breast cancer survival prediction | Quantitative study | 5 | Prevention | Analysis and cognition | Machine learning | Serious games | ||||||||
Disease or health topic: attention-deficit/hyperactivity disorder | |||||||||||||||||
|
Keshav et al [40]; 2019 | Patients (7) | Digital attention-related augmented reality game | Case series | 4 | Prevention or diagnostics | Analysis and cognition | AI | Serious games | ||||||||
Disease or health topic: rheumatoid arthritis | |||||||||||||||||
|
Varga et al [38]; 2021 | Healthy individuals (7) | Virtual arthritis rehabilitation app | Proof of concept | 5 | Rehabilitation | Direct intervention | Machine learning | Serious games | ||||||||
|
Varga et al [58]; 2022 | Patients (10) | Virtual arthritis rehabilitation app | Case series | 4 | Rehabilitation | Direct intervention | Machine learning | Serious games | ||||||||
Disease or health topic: other | |||||||||||||||||
|
Pinto et al [41]; 2019 | Older adults (11) | Active and assisted living for monitoring daily life activities | Case series | 4 | Prevention or diagnostics | Analysis and cognition | Machine learning | Gamification |
aAccording to the Oxford Centre for Evidence-Based Medicine: Levels of Evidence [54].
bAI: artificial intelligence.
Motion Impairment
Almost one-third of the studies (5/16, 31%) targeted the objective of motion impairment. Studies included upper- and lower-limb rehabilitation with a broad range of possible medical indications, ranging from poststroke to vestibular dysfunction. Games were used to enhance motivation and provide a user-friendly at-home training experience. Some of the studies (5/16, 31%) achieved this through an integration of virtual reality and artificial reality. AI was integrated in different ways. Some of the studies (4/16, 25%) used games as a training tool and then analyzed and classified the collected data with AI. Other studies (3/16, 19%) first processed sensor data via AI to improve the quality of an associated game. Direct interaction between the AI and the games component was shown in 2 (40%) of the 5 studies, in which AI adapted the game design and difficulty to the ability level of the patient.
Only 1 (20%) of the 5 studies tested the design using a control group analyzing patient improvements in clinical parameters. All other studies demonstrated the functionality and usability of their technical approach in pilot studies.
Phantom Limb Pain or Limb Absence
Of the 16 studies, 3 (19%) targeted the topic of phantom limb pain or limb absence, where a game environment can support at-home therapy and provide enhanced visual feedback. Of the 3 studies, 2 (67%) by the same research group targeted phantom motor execution with similar approaches. Machine learning was used to improve the quality of electromyography sensor data and thus provide better data input for training. Different training methods in the spectrum of virtual reality and augmented reality and serious games were tested. Of the 3 studies, 1 (33%) focused on ethnographic user–type analysis, and 1 (33%) effected a decrease in phantom pain. The third study tested a machine learning–aided prothesis, comparing 2 different training approaches—1 conventional and 1 via a serious game—to collect electromyography data. Testing was only conducted on 4 patients; however, the results were insignificant.
Cognitive Impairment
In cognitive impairment, the included studies used a set of games covering different cognitive functions as diagnostic instruments. Data were then processed by machine learning techniques to further improve outcome quality. Of the 3 studies, 1 (33%) focused on evaluating patients with cognitive impairment after stroke. Scores acquired from a game set were analyzed by AI and compared with the clinically widely used Mini-Mental State Examination (MMSE) [37]. Of the 3 studies, 2 (67%) used a game set for predicting the future development of mild cognitive impairment, using AI to automatically distinguish between healthy individuals and individuals who were possibly affected. In both fields, pilot studies were conducted with patients, showing high motivation to participate and good usability of the game sets.
Rheumatoid Arthritis
In rheumatoid arthritis, a serious game for hand rehabilitation was developed. Neural networks for processing data and machine learning for testing the accuracy of hand movements for individually adapting difficulty were integrated. Two small pilot studies, 1 with healthy individuals and 1 with patients, showed high accuracy of the machine learning algorithm and good usability, whereas clinical benefits have not yet been measured [38].
Cancer
In cancer, a crowdsourcing campaign was set up via an open web-based game that captured inputs from players regarding their estimation of 5 specific genes, which can be used as predictors of breast cancer survival. Gene selections were processed by machine learning to identify the best prediction models. When only including inputs from people with a self-proclaimed Doctor of Medicine degree, a Doctor of Philosophy degree, or expertise in cancer, the resulting models performed similarly to clinically established gene sets [39].
ADHD Severity
A set of smartglasses was developed to assess ADHD severity through playing an attention-related augmented reality game designed as a social-emotional communication aid. AI was used to analyze video and audio as well as affective and behavioral data and provided users with in-game rewards based on their performance. The study showed significant correlation of the game score to validated clinical gold standard assessments for ADHD [40].
Other
To improve the prevention of cognitive and physical decline, an at-home innovative system consisting of remote monitoring and neurocognitive games was developed. Feedback to the user, including badges or benefits for real-life events, is provided via machine learning analysis. Older adult users indicated “great acceptability” of the system [41].
Discussion
Principal Findings
Currently, there are only a limited number of studies involving a combination of game-based methods and AI in health. Almost one-third of the included studies (5/16, 31%) were centered on addressing motion impairment. The primary emphasis of the research was on rehabilitation. In addition, most of the studies (9/16, 56%) focused on prevention and diagnostics. In terms of AI technology, machine learning was the most commonly used approach (12/16, 75%). Furthermore, serious games were used in most of the studies (11/16, 69%).
When analyzing the studies by disease category, most of the studies (5/16, 31%) used a rehabilitation approach for different aspects of motion impairment (eg, in poststroke conditions, phantom limb pain or limb absence, and rheumatoid arthritis). In this field, studies have a focus on providing individual, at-home, and complex training opportunities for improving motoric limb function, in which therapeutic concepts rely on long-term and self-guided exercising. Games take the role of a training tool, enhancing at-home training motivation and providing multidimensional and exercises compared with the current standard of care. In addition, in some of the studies (5/16, 31%), the integration of virtual reality and augmented reality provided an immersive experience. The role of AI in this context is diverse, sometimes to analyze and classify collected data to improve game setup and level, sometimes to analyze data resulting from game play itself.
In a second cluster, studies for neurological diseases, including those handling cognitive impairment in older adults as well as 1 study for ADHD in younger patients, there was a clear focus on diagnostic evaluations. Here, different sets of games were used to assess various cognitive subdomains, with AI processing these different data inputs and calculating scores and predictions. The advantages in this field are the wide range of possible game designs and the feasibility to play these games individually at home. This could reduce health professionals’ time in assessing cognitive function during face-to-face visits or supplement them by enabling longitudinally acquired data sets and trajectories. The first results show promising results in comparison with standard clinical scores obtained using, for instance, the MMSE.
The direct interaction between the games approach and AI technology was only described in 3 (19%) of the 16 studies. Most of the time, the 2 entities follow each other, with the AI technology not analyzing live in-game playing data. However, direct interaction holds a promise of benefit through an AI-enabled assessment of the patient’s ability during game play and individualized live adjustments of game design and difficulty. Examples using this approach showed good technical functioning and positive user feedback [42]. Even so, the limited number of published studies suggest that the potential of this integrated approach has so far not been fully used yet. This is rather surprising, given the fact that the direct link between AI and games is widely prevalent in the commercial games sector. The reasons for this are purely speculative. The transition of findings from 1 field to another is still pending, perhaps because studies in the commercial games field have a different scope than those in medicine and health. Another reason could be the resource-intensive nature of research. However, the future potential of this interaction seems promising, with the stimulation of user motivation by game design and gamification elements and with AI being used to process large and multimodal data sources and to perform individualized adaptations.
When analyzing further categories, our review shows that the studies so far have produced very limited quality evidence (all studies have an evidence level of 4 or 5, except for 1 study that has an evidence level of 3b), with most of the studies presenting either a rather technical proof of concept (15/16, 94%) or performing usability testing with a small sample size of healthy individuals and patients (14/16, 88%). Higher-quality studies with control groups and end points focusing on specific clinical outcomes are missing.
Of note, the research field is still young. All studies were conducted in the last 8 years, with 13 (81%) of the 16 studies being published in the last 3 years. All research settings however bear the potential of conducting higher-quality studies with bigger sample sizes and specific medical outcomes in the near future.
However, the studies in this review already show promising results, with overall well-functioning technical implementation of the game elements and high accuracy and usefulness of the AI integration. From the patients’ and healthy individuals’ perspective, generally high usability, motivation, and satisfaction were found, mostly assessed by established usability questionnaires and qualitative interviews. This is an encouraging perspective for the future because individualized patient-driven at-home diagnostic and therapeutic approaches are increasingly relevant in all fields of medicine.
All 16 studies identified in this review have a relatively low level of evidence (3b: n=1, 6%; 4: n=10, 63%; and 5: n=5, 31%). The risks of bias in these studies are multifaceted. Pilot studies, often conducted to assess the feasibility of a full-scale study, typically featured small sample sizes and often lacked rigorous methodology, randomization, and blinding procedures. As a result, they are susceptible to a range of biases, including selection bias, performance bias, and detection bias. Studies were characterized by weaker methodologies, which can lead to biases in data collection, analysis, and reporting. Nonrandomized studies were prone to selection bias, confounding, and other methodological flaws. The high heterogeneity of the identified studies encompassed a wide range of disease or health issues, populations, and interventions. This heterogeneity makes it challenging or impossible to integrate data and limits overall conclusions.
Limitations
First, as described earlier, the field of research is still very interdisciplinary, and the studies carried out are very diverse based on the vast variety of game-based approaches and therapeutic interventions.
This review only covered original studies in English, which were found in the PubMed, Scopus, IEEE Xplore, Cochrane Library, and PubPsych databases and published in the last 20 years. Although these are widely recognized and commonly used databases in the field of health care research, restricting the review to these 5 databases may have resulted in the exclusion of relevant studies published in other databases owing to this high interdisciplinarity. However, efforts have been made to minimize this limitation using comprehensive search strings, snowballing, and asking colleagues to identify additional relevant literature. In addition, this review also includes interdisciplinary databases such as the more technical-oriented IEEE Xplore and the more pedagogical-oriented PubPsych.
It especially remains unclear whether all projects conducted especially with a more technical focus have been published in scientific journals at all. For future reviews, a more holistic approach should be taken to assess more results from projects that may not have been included in a publication.
In addition, there might be a lack of awareness that research in the engineering, gaming, and fitness spectrum has a direct connection with health-related issues. Thus, it seems possible that certain publications were not fully covered by our already broad search strategy or that promising interventions have not been related to health care yet. This should be mitigated in future studies, considering the growing attention to this young research field.
Another limitation is that this review focused on therapeutic medical interventions rather than on health interventions. AI and game-based approaches in the field of prevention and health promotion have not been included, although this is an important aspect of population health. Game-based approaches especially are used a lot in this field to reach the target groups [8,9,21,22,59-61].
Future Directions
In the near future, the potential of games, which is already established in the commercial games sector, should be applied to the field of serious games and AI. Adaptive game design can be suitable in health care to improve the intervention outcome via AI-driven health care games that assess the skills level of the patient and adapt the difficulty in feedback loops, which could lead to a better harmonization with traditional therapy sessions. NPCs could be used as virtual patients or other health care personnel or relatives to simulate the interprofessional working environment and to improve the interaction and communication with virtual patients [26,27,34].
Finally, the integration of AI and games should carefully consider the ongoing discussions regarding ethical, moral, and data protection issues. In particular, studies describing ethical issues using game-based approaches are scarce [62,63].
Analyzing the currently limited evidence with promising future possibilities in study design and quality, as well as a dynamic research field, it seems, at this stage, that another review should be conducted in the next few years to assess this rapidly growing research field.
Acknowledgments
Open access funding was provided by the open access publishing funds of Philipps-University Marburg and Bielefeld University.
Abbreviations
- ADHD
attention-deficit/hyperactivity disorder
- AI
artificial intelligence
- MMSE
Mini-Mental State Examination
- NPC
nonplayer character
- PRISMA-P
Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols
Search term protocol.
Rayyan data of the review process of all studies.
Categorization of all eligible studies.
Detailed descriptions of all included studies.
PRISMA-P-Checklist.
Footnotes
Authors' Contributions: All authors were responsible for conceptualization, methodology, validation, and visualization. DT and JJS were responsible for data curation and formal analysis. DT and SK were responsible for investigation. SK was responsible for funding acquisition, project administration, resources, software, and supervision. DT and JJS wrote the original draft; all authors reviewed and edited the manuscript (contributions of the authors are based on CRediT [contributor roles taxonomy] [64]).
Conflicts of Interest: SK is the founder and a shareholder of MED.digital. All other authors declare no other conflicts of interest.
References
- 1.Tolks D, Dadaczynski K, Horstmann D. Einführung in die Vergangenheit, Gegenwart und Zukunft von Serious Games (for Health) Prävention Gesundheitsförderung. 2018 Sep 3;13:272–9. doi: 10.1007/s11553-018-0667-9. [DOI] [Google Scholar]
- 2.Abt CC. Serious Games. New York, NY: Viking Press; 1970. [Google Scholar]
- 3.Sawyer B, Smith P. Serious games taxonomy. The Digital Entertainment Alliance. 2008. Feb 19, [2023-12-20]. https://thedigitalentertainmentalliance.files.wordpress.com/2011/08/serious-games-taxonomy.pdf .
- 4.Sostmann K, Tolks D, Buron S, Plener J, Gaedicke G, Gross M. Serious games healthcare: applying games to medical instruction. German Medical Science. 2010. [2023-12-11]. https://www.egms.de/static/en/meetings/cbt2010/10cbt06.shtml .
- 5.Tolks D, Lampert C, Dadaczynski K, Maslon E, Paulus P, Sailer M. [Game-based approaches to prevention and health promotion: serious games and gamification] Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2020 Jun;63(6):698–707. doi: 10.1007/s00103-020-03156-1.10.1007/s00103-020-03156-1 [DOI] [PubMed] [Google Scholar]
- 6.Lujan HL, DiCarlo SE. Too much teaching, not enough learning: what is the solution? Adv Physiol Educ. 2006 Mar;30(1):17–22. doi: 10.1152/advan.00061.2005. https://journals.physiology.org/doi/10.1152/advan.00061.2005?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed .30/1/17 [DOI] [PubMed] [Google Scholar]
- 7.Pitt MB, Borman-Shoap EC, Eppich WJ. Twelve tips for maximizing the effectiveness of game-based learning. Med Teach. 2015;37(11):1013–7. doi: 10.3109/0142159X.2015.1020289. [DOI] [PubMed] [Google Scholar]
- 8.Lu AS, Kharrazi H. A state-of-the-art systematic content analysis of games for health. Games Health J. 2018 Feb;7(1):1–15. doi: 10.1089/g4h.2017.0095. https://europepmc.org/abstract/MED/29293368 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Maheu-Cadotte MA, Cossette S, Dubé V, Fontaine G, Lavallée A, Lavoie P, Mailhot T, Deschênes MF. Efficacy of serious games in healthcare professions education: a systematic review and meta-analysis. Simul Healthc. 2021 Jun 01;16(3):199–212. doi: 10.1097/SIH.0000000000000512.01266021-202106000-00007 [DOI] [PubMed] [Google Scholar]
- 10.Wang Y, Wang Z, Liu G, Wang Z, Wang Q, Yan Y, Wang J, Zhu Y, Gao W, Kan X, Zhang Z, Jia L, Pang X. Application of serious games in health care: scoping review and bibliometric analysis. Front Public Health. 2022 Jun 10;10:896974. doi: 10.3389/fpubh.2022.896974. https://europepmc.org/abstract/MED/35757619 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gentry SV, Gauthier A, L'Estrade Ehrstrom B, Wortley D, Lilienthal A, Tudor Car L, Dauwels-Okutsu S, Nikolaou CK, Zary N, Campbell J, Car J. Serious gaming and gamification education in health professions: systematic review. J Med Internet Res. 2019 Mar 28;21(3):e12994. doi: 10.2196/12994. https://www.jmir.org/2019/3/e12994/ v21i3e12994 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kollins SH, DeLoss DJ, Cañadas E, Lutz J, Findling RL, Keefe RS, Epstein JN, Cutler AJ, Faraone SV. A novel digital intervention for actively reducing severity of paediatric ADHD (STARS-ADHD): a randomised controlled trial. Lancet Digit Health. 2020 Apr;2(4):e168–78. doi: 10.1016/S2589-7500(20)30017-0. https://linkinghub.elsevier.com/retrieve/pii/S2589-7500(20)30017-0 .S2589-7500(20)30017-0 [DOI] [PubMed] [Google Scholar]
- 13.Kato PM, Cole SW, Bradlyn AS, Pollock BH. A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics. 2008 Aug;122(2):e305–17. doi: 10.1542/peds.2007-3134.122/2/e305 [DOI] [PubMed] [Google Scholar]
- 14.Chon SH, Timmermann F, Dratsch T, Schuelper N, Plum P, Berlth F, Datta RR, Schramm C, Haneder S, Späth MR, Dübbers M, Kleinert J, Raupach T, Bruns C, Kleinert R. Serious games in surgical medical education: a virtual emergency department as a tool for teaching clinical reasoning to medical students. JMIR Serious Games. 2019 Mar 05;7(1):e13028. doi: 10.2196/13028. https://games.jmir.org/2019/1/e13028/ v7i1e13028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Bai S, Hew KF, Huang B. Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educ Res Rev. 2020 Jun;30:100322. doi: 10.1016/j.edurev.2020.100322. [DOI] [Google Scholar]
- 16.Zainuddin Z, Chu SK, Shujahat M, Perera CJ. The impact of gamification on learning and instruction: a systematic review of empirical evidence. Educ Res Rev. 2020 Jun;30:100326. doi: 10.1016/j.edurev.2020.100326. [DOI] [Google Scholar]
- 17.Sailer M, Homner L. The gamification of learning: a meta-analysis. Educ Psychol Rev. 2019 Aug 15;32:77–112. doi: 10.1007/s10648-019-09498-w. [DOI] [Google Scholar]
- 18.Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining "gamification". Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments; MindTrek '11: Academic MindTrek 2011; September 28-30, 2011; Tampere, Finland. 2011. [DOI] [Google Scholar]
- 19.Sailer M, Hense JU, Mayr SK, Mandl H. How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. Comput Hum Behav. 2017 Apr;69:371–80. doi: 10.1016/j.chb.2016.12.033. [DOI] [Google Scholar]
- 20.Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000 Jan;55(1):68–78. doi: 10.1037//0003-066x.55.1.68. [DOI] [PubMed] [Google Scholar]
- 21.Johnson D, Deterding S, Kuhn KA, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: a systematic review of the literature. Internet Interv. 2016 Nov 2;6:89–106. doi: 10.1016/j.invent.2016.10.002. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(16)30038-0 .S2214-7829(16)30038-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Schmidt-Kraepelin M, Thiebes S, Baumsteiger D, Sunyaev A. State of play: a citation network analysis of healthcare gamification studies. Proceedings of the 26th European Conference on Information Systems; 26th European Conference on Information Systems; June 23-28, 2018; Portsmouth, UK. 2018. [Google Scholar]
- 23.Westera W, Prada R, Mascarenhas S, Santos PA, Dias J, Guimarães M, Georgiadis K, Nyamsuren E, Bahreini K, Yumak Z, Christyowidiasmoro C, Dascalu M, Gutu-Robu G, Ruseti S. Artificial intelligence moving serious gaming: presenting reusable game AI components. Educ Inf Technol. 2019 Jul 30;25:351–80. doi: 10.1007/s10639-019-09968-2. doi: 10.1007/s10639-019-09968-2. [DOI] [Google Scholar]
- 24.Shannon CE. XXII. Programming a computer for playing chess. Lond Edinb Dublin Philos Mag J Sci. 1950;41(314):256–75. doi: 10.1080/14786445008521796. [DOI] [Google Scholar]
- 25.Yannakakis GN, Togelius J. A panorama of artificial and computational intelligence in games. IEEE Trans Comput Intell AI Games. 2015 Dec;7(4):317–35. doi: 10.1109/tciaig.2014.2339221. [DOI] [Google Scholar]
- 26.Yannakakis GN, Togelius J. Artificial Intelligence and Games. Cham, Switzerland: Springer; 2018. [Google Scholar]
- 27.Nyamsuren E, van der Vegt W, Westera W. Automated adaptation and assessment in serious games: a portable tool for supporting learning. Proceedings of the 15th International Conferences, ACG 2017; 15th International Conferences, ACG 2017; July 3–5, 2017; Leiden, The Netherlands. 2017. [DOI] [Google Scholar]
- 28.Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015 Jan 01;4(1):1. doi: 10.1186/2046-4053-4-1. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053-4-1 .2046-4053-4-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Nordhausen T, Hirt J. Manual zur literaturrecherche in fachdatenbanken. Medizinische Fakultät Institut für Gesundheits- und Pflegewissenschaft. 2018. Sep 10, [2023-12-20]. https://www.hwg-lu.de/fileadmin/user_upload/service/bibliothek/RefHunter.pdf .
- 30.IEEE Xplore. [2023-10-10]. https://ieeexplore.ieee.org/Xplore/home.jsp .
- 31.Advanced search. Cochrane Library. [2023-10-10]. https://www.cochranelibrary.com/advanced-search .
- 32.PubPsych. [2023-10-10]. https://www.pubpsych.eu/
- 33.Google Scholar. [2023-12-20]. https://scholar.google.com/
- 34.Gusenbauer M, Haddaway NR. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods. 2020 Mar;11(2):181–217. doi: 10.1002/jrsm.1378. https://europepmc.org/abstract/MED/31614060 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Giustini D, Boulos MN. Google Scholar is not enough to be used alone for systematic reviews. Online J Public Health Inform. 2013 Jul 01;5(2):214. doi: 10.5210/ojphi.v5i2.4623. https://europepmc.org/abstract/MED/23923099 .ojphi-05-214 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Boeker M, Vach W, Motschall E. Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough. BMC Med Res Methodol. 2013 Oct 26;13:131. doi: 10.1186/1471-2288-13-131. https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-13-131 .1471-2288-13-131 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Jung HT, Daneault JF, Lee H, Kim K, Kim B, Park S, Ryu T, Kim Y, Ivan Lee S. Remote assessment of cognitive impairment level based on serious mobile game performance: an initial proof of concept. IEEE J Biomed Health Inform. 2019 May;23(3):1269–77. doi: 10.1109/JBHI.2019.2893897. [DOI] [PubMed] [Google Scholar]
- 38.Varga G, Stoicu-Tivadar L, Nicola S. Serious gaming and AI supporting treatment in rheumatoid arthritis. Stud Health Technol Inform. 2021 May 27;281:699–703. doi: 10.3233/SHTI210262.SHTI210262 [DOI] [PubMed] [Google Scholar]
- 39.Good BM, Loguercio S, Griffith OL, Nanis M, Wu C, Su AI. The cure: design and evaluation of a crowdsourcing game for gene selection for breast cancer survival prediction. JMIR Serious Games. 2014 Jul 29;2(2):e7. doi: 10.2196/games.3350. https://games.jmir.org/2014/2/e7/ v2i2e7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Keshav NU, Vogt-Lowell K, Vahabzadeh A, Sahin NT. Digital attention-related augmented-reality game: significant correlation between student game performance and validated clinical measures of attention-deficit/hyperactivity disorder (ADHD) Children (Basel) 2019 May 28;6(6):72. doi: 10.3390/children6060072. https://www.mdpi.com/resolver?pii=children6060072 .children6060072 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Pinto M, Pereira M, Raposo D, Simões M, Castelo-Branco M. GameAAL - an AAL solution based on gamification and machine learning techniques. Proceedings of the 2019 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB); 2019 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB); July 9-11, 2019; Siena, Italy. 2019. [DOI] [Google Scholar]
- 42.Burdea G, Kim N, Polistico K, Kadaru A, Grampurohit N, Roll D, Damiani F. Assistive game controller for artificial intelligence-enhanced telerehabilitation post-stroke. Assist Technol. 2021 May 04;33(3):117–28. doi: 10.1080/10400435.2019.1593260. https://europepmc.org/abstract/MED/31180276 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Yeh SC, Huang MC, Wang PC, Fang TY, Su MC, Tsai PY, Rizzo A. Machine learning-based assessment tool for imbalance and vestibular dysfunction with virtual reality rehabilitation system. Comput Methods Programs Biomed. 2014 Oct;116(3):311–8. doi: 10.1016/j.cmpb.2014.04.014.S0169-2607(14)00166-7 [DOI] [PubMed] [Google Scholar]
- 44.Lyu M, Chen WH, Ding X, Wang J. Knee exoskeleton enhanced with artificial intelligence to provide assistance-as-needed. Rev Sci Instrum. 2019 Sep;90(9):094101. doi: 10.1063/1.5091660. [DOI] [PubMed] [Google Scholar]
- 45.Nasri N, Orts-Escolano S, Cazorla M. An sEMG-controlled 3D game for rehabilitation therapies: real-time time hand gesture recognition using deep learning techniques. Sensors (Basel) 2020 Nov 12;20(22):6451. doi: 10.3390/s20226451. https://www.mdpi.com/resolver?pii=s20226451 .s20226451 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Zhang Q, Jin T, Cai J, Xu L, He T, Wang T, Tian Y, Li L, Peng Y, Lee C. Wearable triboelectric sensors enabled gait analysis and waist motion capture for IoT-based smart healthcare applications. Adv Sci (Weinh) 2022 Feb;9(4):e2103694. doi: 10.1002/advs.202103694. https://europepmc.org/abstract/MED/34796695 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Ortiz-Catalan M, Guðmundsdóttir RA, Kristoffersen MB, Zepeda-Echavarria A, Caine-Winterberger K, Kulbacka-Ortiz K, Widehammar C, Eriksson K, Stockselius A, Ragnö C, Pihlar Z, Burger H, Hermansson L. Phantom motor execution facilitated by machine learning and augmented reality as treatment for phantom limb pain: a single group, clinical trial in patients with chronic intractable phantom limb pain. Lancet. 2016 Dec 10;388(10062):2885–94. doi: 10.1016/S0140-6736(16)31598-7.S0140-6736(16)31598-7 [DOI] [PubMed] [Google Scholar]
- 48.Lendaro E, Hermansson L, Burger H, Van der Sluis CK, McGuire BE, Pilch M, Bunketorp-Käll L, Kulbacka-Ortiz K, Rignér I, Stockselius A, Gudmundson L, Widehammar C, Hill W, Geers S, Ortiz-Catalan M. Phantom motor execution as a treatment for phantom limb pain: protocol of an international, double-blind, randomised controlled clinical trial. BMJ Open. 2018 Jul 16;8(7):e021039. doi: 10.1136/bmjopen-2017-021039. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=30012784 .bmjopen-2017-021039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Kristoffersen MB, Franzke AW, Bongers RM, Wand M, Murgia A, van der Sluis CK. User training for machine learning controlled upper limb prostheses: a serious game approach. J Neuroeng Rehabil. 2021 Feb 12;18(1):32. doi: 10.1186/s12984-021-00831-5. https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-021-00831-5 .10.1186/s12984-021-00831-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Valladares-Rodriguez S, Pérez-Rodriguez R, Fernandez-Iglesias JM, Anido-Rifón LE, Facal D, Rivas-Costa C. Learning to detect cognitive impairment through digital games and machine learning techniques. Methods Inf Med. 2018 Sep;57(4):197–207. doi: 10.3414/ME17-02-0011. [DOI] [PubMed] [Google Scholar]
- 51.Valladares-Rodriguez S, Fernández-Iglesias MJ, Anido-Rifón L, Facal D, Rivas-Costa C, Pérez-Rodríguez R. Touchscreen games to detect cognitive impairment in senior adults. A user-interaction pilot study. Int J Med Inform. 2019 Jul;127:52–62. doi: 10.1016/j.ijmedinf.2019.04.012.S1386-5056(18)30163-1 [DOI] [PubMed] [Google Scholar]
- 52.Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016 Dec 05;5(1):210. doi: 10.1186/s13643-016-0384-4. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-016-0384-4 .10.1186/s13643-016-0384-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Röhrig B, du Prel JB, Wachtlin D, Blettner M. Types of study in medical research: part 3 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2009 Apr;106(15):262–8. doi: 10.3238/arztebl.2009.0262. https://europepmc.org/abstract/MED/19547627 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.OCEBM levels of evidence. Centre for Evidence-Based Medicine (CEBM), University of Oxford. [2023-12-20]. https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence .
- 55.Kuhn S, Thranberend T. Medizinische begutachtung digitaler gesundheitsanwendungen im projekt "trusted-health-apps". Bertelsmann Stiftung. [2023-12-07]. https://www.bertelsmann-stiftung.de/de/publikationen/publikation/did/medizinische-begutachtung-digitaler-gesundheitsanwendungen-im-projekt-trusted-health-apps-all .
- 56.AppQ: gütekriterien-kernset für mehr qualitätstransparenz bei digitalen gesundheitsanwendungen. Bertelsmann Stiftung. 2019. Oct, [2023-12-07]. https://www.bertelsmann-stiftung.de/fileadmin/files/BSt/Publikationen/GrauePublikationen/Studienbericht_AppQ_191028.pdf .
- 57.Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005 Nov 05;331(7524):1064–5. doi: 10.1136/bmj.38636.593461.68. https://europepmc.org/abstract/MED/16230312 .bmj.38636.593461.68 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Varga G, Stoicu-Tivadar L, Nicola S. Serious gaming and artificial intelligence in rehabilitation of rheumatoid arthritis. Stud Health Technol Inform. 2022 Jun 29;295:562–5. doi: 10.3233/SHTI220790.SHTI220790 [DOI] [PubMed] [Google Scholar]
- 59.Koivisto J, Hamari J. Gamification of physical activity: a systematic literature review of comparison studies. Proceedings of the GamiFIN Conference 2019; GamiFIN Conference 2019; April 8-10, 2019; Levi, Finland. 2019. [Google Scholar]
- 60.van Gaalen AE, Brouwer J, Schönrock-Adema J, Bouwkamp-Timmer T, Jaarsma AD, Georgiadis JR. Gamification of health professions education: a systematic review. Adv Health Sci Educ Theory Pract. 2021 May;26(2):683–711. doi: 10.1007/s10459-020-10000-3. https://europepmc.org/abstract/MED/33128662 .10.1007/s10459-020-10000-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Cheng VW, Davenport T, Johnson D, Vella K, Hickie IB. Gamification in apps and technologies for improving mental health and well-being: systematic review. JMIR Ment Health. 2019 Jun 26;6(6):e13717. doi: 10.2196/13717. https://mental.jmir.org/2019/6/e13717/ v6i6e13717 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Barnes S, Chandler T, Salazar MG. Help over harm: practical and ethical considerations for the evaluation and deployment of therapeutic games. Proceedings of ICACTCE'23 — The International Conference on Advances in Communication Technology and Computer Engineering; ICACTCE'23; February 24-25, 2023; Bolton, UK. 2023. [DOI] [Google Scholar]
- 63.Kim TW, Werbach K. More than just a game: ethical issues in gamification. Ethics Inf Technol. 2016 May 12;18:157–73. doi: 10.1007/s10676-016-9401-5. [DOI] [Google Scholar]
- 64.CRediT - contributor roles taxonomy. CRediT. [2023-10-11]. https://credit.niso.org/
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Search term protocol.
Rayyan data of the review process of all studies.
Categorization of all eligible studies.
Detailed descriptions of all included studies.
PRISMA-P-Checklist.