A Meta-Analysis of Serious Digital Games for Healthy Lifestyle Promotion

Abstract

Several systematic reviews have described health-promoting effects of serious games but so far no meta-analysis has been reported. This paper presents a meta-analysis of 54 serious digital game studies for healthy lifestyle promotion, in which we investigated the overall effectiveness of serious digital games on healthy lifestyle promotion outcomes and the role of theoretically and clinically important moderators. Findings showed serious games have small positive effects on healthy lifestyles (g=0.260, 95% CI 0.148; 0.373) and their determinants (g=0.334, 95% CI 0.260; 0.407), especially for knowledge. Effects on clinical outcomes were significant, but much smaller (g=0.079, 95% CI 0.038; 0.120). Long-term effects were maintained for all outcomes except for behavior. Serious games are best individually tailored to both socio-demographic and change need information, and benefit from a strong focus on game theories or a dual theoretical foundation in both behavioral prediction and game theories. They can be effective either as a stand-alone or multi-component programs, and appeal to populations regardless of age and gender. Given that effects of games remain heterogeneous, further exploration of which game features create larger effects are needed.

Introduction

Healthy lifestyles comprise an array of potentially modifiable behaviors that can prevent a wide range of diseases, such as some cancers, cardiovascular diseases, stroke, dementia, mental illness, and diabetes (Fratiglioni et al. 2004;Kim et al. 2012;Lopez et al. 2006;Peel et al. 2005). Healthy lifestyle adoption and maintenance, however, are often hindered by motivational issues, lack of time to participate in health promotion programs, and the interventions’ low reach into the target group (Baert et al. 2011;McGuire et al. 2013;Toobert et al. 2002). Computer-delivered and computer-tailored interventions have been successfully designed to overcome these obstacles by tailoring to motivational stage, being accessible whenever the individual has time, and ensuring high availability at lower cost (Krebs et al. 2010;Portnoy et al. 2008).

Serious digital games are a type of computer-delivered intervention considered to be both educational and fun. Games differ from computer-delivered interventions by aspiring to be highly enjoyable, attention-captivating and intrinsically motivating (Graesser et al. 2009;Prensky 2007). Serious games differ from mere entertainment games in their aim to educate or promote behavior change. In the context of health promotion programs, this may be achieved via the provision of health-related information, modeling of positive health behaviors, the creation of opportunities to practice healthy lifestyle skills (Kato 2010), by changing mediators (e.g. self-regulatory skill development), and by applying change procedures (such as e.g. tailoring and goal-setting) (Thompson et al. 2012;Thompson et al. 2010). Serious games may furthermore create sustained effects by being intrinsically motivating to play longer and repeatedly (Sitzmann 2011;Wouters et al. 2013).

Serious games are theorized to derive their learning effects from at least three sources: 1) by creating immersion or transportation, a state in which the player becomes absorbed in the play without disbelief, while creating personally relevant experiences and deep affection for the characters; 2) by establishing flow, a state of high concentration in which the player experiences a balance between skills and challenge; and 3) by meeting the individuals’ needs for mastery, autonomy, connectedness, arousal, diversion, fantasy, or challenge (Annetta 2010;Boyle et al. 2012;Connolly et al. 2012;Kapp 2012;Lu et al. 2012). Several narrative systematic reviews have described health-promoting effects of serious games. These included health games relating to diverse behaviors and populations, including games for treatment, prevention, and professional education (Rahmani and Boren 2012). Reviews on healthy lifestyles mostly focused on a single health behavior (e.g. Guy et al, 2011, on obesity prevention) or focused on one specific age group (e.g. Guse et al, 2012, on sexual health among adolescents). An exception was the review of Baranowski et al. which included games for healthy diet, physical activity and illness self-management (Baranowski et al. 2008). All reviews noted large differences between studies, and concluded that reasons for these differences are as yet unclear (Baranowski, Buday, Thompson, & Baranowski 2008;DeShazo et al. 2010;Gamberini et al. 2008;Guse et al. 2012;Guy et al. 2011;Kato 2010;Kharrazi et al. 2012;Lu, Baranowski, Thompson, & Buday 2012;Papastergiou 2009;Primack et al. 2012;Rahmani & Boren 2012;Wilkinson et al. 2008).

As yet, no meta-analysis of serious games for healthy lifestyle promotion has been reported. Meta-analysis overcomes the problem of small sample sizes in individual studies, which make it hard to determine a treatment’s effectiveness. Furthermore, quantifying and comparing heterogeneity across studies by game characteristics permits tests of hypotheses reported in previous reviews (Borenstein et al. 2009). These insights may guide professionals in developing future evidence-based serious games (Ritterfeld et al. 2009).

This meta-analysis of the effectiveness of serious games for healthy lifestyle promotion addressed the following questions: 1) How effective are serious games in changing health behaviors, their determinants and clinical outcomes?; and 2) What is the influence of moderators, such as study characteristics, sample characteristics, theoretical basis, tailoring, and implementation method, on intervention effectiveness?

Methods

Search Strategy and study selection

Inclusion and exclusion criteria

Healthy lifestyles were defined as the ability to adapt and self manage mental, social and physical health, in line with a recent conceptualization of health (Huber et al. 2011). Four categories of health behaviors were studied: 1) healthy diet and physical activity, 2) health responsibility/maintenance, 3) social behavior, and 4) mental health promotion. Healthy diet and physical activity/exercise were grouped because they frequently co-occurred in games for health promotion. Examples within this category were drinking water, eating vegetables, low sedentary behavior, and daily step counts. The second category, health responsibility/maintenance, was composed of both general preventive actions and self-management of disease and illness. Examples were good dental care, not smoking, participating in health screening, and asthma self-management. Category 3, social behavior, referred to maintaining positive interpersonal relationships. Examples were not bullying, seeking social support, establishing and maintaining friendships. The fourth category, mental health promotion, included reducing mental health risks, promoting well-being and self-actualization (e.g. personal growth, seeking happiness) and stress management. Examples were monitoring mood, using coping skills, and maintaining cognitive functioning for the elderly. These categories were based on the dimensions in the Health-Promoting Lifestyle Profile (Walker et al. 1987), and can be considered different, yet interrelated, healthy lifestyles. For example, physical activity behaviors may also have mental health promoting effects (Asztalos et al. 2009). Behaviors were thus coded in the category of the closest fit, although they may have also been related to other categories.

Although several commercial off-the-shelf games have been effective in obtaining health benefits (Peng et al. 2011), serious games designed specifically for health promotion provide additional benefits compared to these purely leisure games, for example by providing health-related information (Kato 2010). We therefore only focused on games made specifically to promote health while also being fun. Exergames (requiring physical activity when played), for example, were only included when developed specifically with a health promotion purpose.

Studies evaluating effects on behavior or its determinants as the primary outcome were included, provided an intervention and control group were used. Finally, only studies reported in English were retained. Table 1 provides an overview of inclusion and exclusion criteria.

Table 1.

Inclusion and exclusion criteria

Inclusion	Definition	Exclusion
Games	Organized play having a set of rules by which to play and a goal, which creates a challenge, provides feedback or shows outcomes, entails interaction and has a topic (Prensky 2007)	Multimedia programs which are not games (e.g. watching video without any interaction)
Serious games	Made specifically to promote health while also being fun	Commercially available, games only developed for entertainment or leisure purposes (e.g. commercial exergames such as Wii)
Digital game	Includes all games using computer technologies as the delivery media (Mäyrä 2008)	Games not played on digital media (e.g. board games)
For healthy lifestyle promotion	Health behaviors covered in this review were healthy diet, physical activity, social behavior, health responsibility, stress management and self-actualization, based on the Health-Promoting Lifestyle Profile (Walker, Sechrist, & Pender 1987). The study’s primary outcome focused on healthy lifestyle behavior or one of its determinants.	Games which only target increased skill level, but no lifestyle change (e.g. athletic performance); Which are only used in a therapeutic context, with no intent to create a lifestyle change (e.g. treatment support); Which are used for professional education
Effect studies	Games evaluated for their effects and that allowed an effect size to be calculated for behavior or its determinants	Studies that only reported usability evaluations, player experiences or case studies; or which only reported effects on clinical outcomes but no healthy lifestyles
Research designs	The following research designs were included: 1) Pre-test post- test with control group (randomized on individual level); 2) Pre-test post-test with non-equivalent control group (randomized at group level) 3) Post–test only control group design; 4) non-equivalent post-test only control group design (Portney and Watkins 2009).	The following designs are excluded: 1) One group pre-test post-test; 2) One group post-test only design

Search strategy

Pubmed (°1966), Web of Science (°1926), Cinahl (°1937) and PsycInfo (°1887) databases were searched for publications since the start of the journal databases until end of July 2013, with the keywords: (‘games’ or ‘video games’ or ‘interactive multimedia’) and health. Search results were complemented with hand-searching studies reported in above mentioned reviews, examining the table of contents of relevant specialized journals and databases (Computers in Human Behavior, Games for Health Journal, CyberPsychology, Behavior and Social Networking, Telemedicine and E-Health, Health Games Research database) and by requesting qualifying manuscripts from the local DiGRA (Digital Games Research Association) chapter. Authors were contacted for more information when data for coding or effect size calculation were lacking.

Coding frame

The coding frame is included in Appendix A.

Primary and secondary outcomes

The primary outcomes were behavior, knowledge, behavioral intentions, perceived barriers, skills, attitudes, subjective norm, and self-efficacy (Bartholomew et al. 2011). Whatever was mentioned by the authors as attitudes, skills etc. was coded under these outcomes. As a secondary outcome, clinical effects (e.g. weight, depression score) were included, when applicable.

Study characteristics

The following research designs were coded: 1) Pre-test post-test with control group (randomized on individual level); 2) Pre-test post-test with non-equivalent control group (randomized at group level) 3) Post–test only control group design; 4) non-equivalent post-test only control group design (Portney & Watkins 2009).

While the pre-test post-test with control group design offers the highest internal validity, in certain situations randomization at group level (pre-test post-test with non-equivalent control group) is preferred when individual randomization within existing groups increases social threats to internal validity (i.e. when participants in the control group and intervention group are in contact with each other and may be aware of the other group’s circumstances, e.g. in schools) (Portney & Watkins 2009).

The quality of each study was evaluated using the Effective Public Health Practice Project (EPHPP) assessment tool for public health interventions (http://www.ephpp.ca/tools.html), included in Appendix A. Other study characteristics considered were intervention duration (period during which the game could be played), total actual play time, and time between last gameplay and first measurement, or second measurement (if reported). Other indicators of dose-response (e.g. actual frequency of play) were unfortunately so infrequently reported that these could not be included in the coding scheme.

Individual tailoring

Individual tailoring was coded when the game content was adapted to individual user characteristics (e.g. age, gender, sexual preference, motivation). Individual tailoring was categorized as adapting to 1. socio-demographic characteristics, 2. change needs (e.g. risk factors) or stages of adoption (e.g. motivation, attitudes, current level of behavior), or 3. a combination of both. When no tailoring or only group tailoring (Portnoy, Scott-Sheldon, Johnson, & Carey 2008) was used, this was coded as no individual tailoring.

Theoretical basis

Theories were categorized as behavioral prediction/change theories, game-based learning theories, clinical psychology approaches and theory-based methods, or none reported. Theories on determinants of both risk and healthful behavior and that specified methods to promote change in these determinants were coded as behavioral prediction or change theories (e.g. Health Belief Model, Social Cognitive Theory, Theory of Planned Behavior). Game-based learning theories included those enhancing player motivation, attention and retention of the message by manipulating game characteristics (e.g. Elaboration Likelihood Model, Transportation Theory). While many of these theories are not exclusively used or developed for game-based learning, they are commonly applied in game design to enhance the player’s experience (Prensky 2007). Clinical psychology approaches and theory-based methods which were used to understand the problem behavior, but not to provide levers to change the desired behavior were recorded as ‘clinical psychology approaches/theory-based methods’ (e.g. Cognitive Behavioral Therapy). Combinations of these theories were entered as such. If no theory was mentioned, this was coded as ‘no theoretical foundation reported’.

Implementation method

For each study we recorded whether the game was evaluated as a stand-alone intervention, or as part of a broader multicomponent intervention.

Interrater reliability

Two coders (ADS, WVL) independently scored a random selection of one third of the games on all coding dimensions. Both coders had adequate research experience in behavioral science and serious games, and were well-trained in coding game and behavioral change theories and methods. Inter-rater reliability was good (ICC=0.90). Quality of the studies using the EPHPP criteria was also independently coded by two coders sufficiently experienced in using this tool (ADS, SC), yielding excellent inter-rater reliability (ICC=0.98).

Meta-analytic procedure

Random-effects Hedges’ g was applied to indicate effect size, correcting for small sample sizes (Hedges 1981). A negative Hedges’ g or a positive Hedges’ g indicated that the serious game respectively reduced or increased adoption of a healthy lifestyle or its determinant. In cases where the intervention targeted a reduction of unhealthy lifestyles, the computed sign of the effect size was reversed so all positive differences reflected an improvement in healthy lifestyles for the treatment group. Moderator analysis was conducted to explain differences in effect sizes. For all moderator analyses, a mixed-effects model was used and Cochran’s Q test and I² (Higgings et al. 2003) were reported to investigate the degree of heterogeneity in effect sizes. Moderator analyses were only conducted when there were at least 3 studies per category. Meta-regression (methods-of-moments procedure) was performed for continuous moderators (Thompson and Higgins 2002), where the slope (β) and its p–value indicated the importance of this moderator in understanding linear changes in effect sizes. To maintain the independence of the data, whenever necessary, effect sizes were averaged across different outcomes. Sensitivity analyses were performed for possible outliers (threshold +/−3SD), for pre-test-post-test correlations set lower (0.20) and higher (0.80) than the standard assumption of 0.50, and for publication bias via a funnel plot and related statistics. All analyses were performed with Comprehensive Meta-Analysis software, version 2.2.050 (Biostat Inc, Englewood, NJ, USA). Effect sizes of ≥0.80 were considered large, ≥0.50 were considered moderate and ≥0.20 were small effects (Cohen 1988). More information on methods is provided in Appendix A.

Results

The database search yielded 7192 hits, from which 1473 duplicates were removed. Next, 5719 articles were deleted after reading the abstract and title. After reading the full texts, fifty-one games were retained. Sixteen studies were added from other sources, such as a search in specialized journals mentioned above and requests via professional networks, such as the Digital Games Research Association. This resulted in a total of 67 studies. Thirteen studies were removed because they did not include a control condition, resulting in 54 included papers (Fig. 1).

Fig. 1.

Flow chart of study selection process

Study and sample characteristics

In the 54 papers, 52 games were assessed in 61 different game evaluations (using independent samples or different study characteristics). A detailed description of these games is shown in Appendix B. Fifty-nine game evaluations involved a first measurement after the intervention (later referred to as ‘first post-test’), fourteen evaluations included a longer follow-up measurement as after this first measurement (later referred to as ‘follow-up test’) and two evaluations only reported the follow-up measurement. Most studies were published in 2010 or later (44.3%), situated in North America (68.8%), or evaluated games for children (57.4%).

Average play duration was 3.8 hours over the full duration of the intervention. First measurement on average took place 18.6 days after the intervention. The majority of first effect measurements were conducted on the same day as the intervention. Nearly half of the studies were of strong quality, using a representative sample of the population, reliable and valid measures, adjusting for confounders and reporting low drop-out (Table 2). Moderate designs (one weak scoring) and weak designs (more than one weak scoring) mainly struggled with external validity issues in having high drop-outs at post-intervention measurement (12 games, 24.5%) and using a sample not assumed highly representative of the population (10 games, 20.4%).

Table 2.

Study attributes

Characteristic	k studies	%
Publication year
<2000	6	9.8
2000–2004	15	24.6
2005–2009	13	21.3
2010–2013	27	44.3
Region
North America (U.S., Canada, Mexico)	42	68.8
Europe	17	27.9
Asia Pacific (incl. Australia, New Zealand)	2	3.3
Africa	0	0
South America	0	0
Study characteristics
First post-intervention measurement studies	59
Effects measured on same day (1st post) *	29	69.0
Second post-intervention measurement studies	16
Effects measured on same day (2nd post)	1	10.0
Weak study quality (EPHPP scoring)	10	16.9
Moderate study quality (EPHPP scoring)	21	35.6
Strong study quality (EPHPP scoring)	28	47.7
Sample characteristics**
Targeting children (age ≤12y)	35	57.4
Targeting adolescents (age 13–18y)	21	34.4
Targeting adults (age 19–65y)	18	29.5
Targeting elderly (over 65y)	1	1.6
	M	SD
% female (range 0–100)	52.5	23.4
Age in years (range 5.2 – 66.3)	16.6	11.8
Days until 1st measurement (range 0–270)	18.6	48.5
Days until 2nd measurement (range 0–186)	82.9	65.5
Average play duration in hours (range 0.08 – 21.6)	3.8	6.8
Duration of full intervention in days (range 1–365)	54.9	8

^*Information only available for 42 studies at first post measurement and 10 at second post measurement

^**>100% as some games targeted more than one age group

The majority of games targeted health responsibility/maintenance behavior (29 game evaluations) or healthy diet and physical activity (22 game evaluations). Six were aimed at improving social behavior, while only two were designed to promote mental health. In 37 evaluations, games were studied as stand-alone intervention, whereas 22 were evaluated as part of a multicomponent program.

Effectiveness of games in changing healthy lifestyles

First post-test

For one study (Jantz et al. 2002) the effect size of one outcome exceeded the outlier threshold; this outcome was hence removed from the analyses. All other studies and outcomes were retained.

The average effect size for behavior was positive, but small (g=0.260 (95% CI 0.148; 0.373), n=314790, k=21) (Fig. 2), indicating an improvement in healthy lifestyle after playing the serious games. For behavioral determinants, average effect size was similarly small (g=0.334 (95% CI 0.260; 0.407), n=19934, k=50) (Fig. 3), whereas the average effect size on clinical outcomes was very small, but significant (g=0.079 (95% CI 0.039; 0.120), n=9367, k=10) (Fig. 4). <enter Figure 2 here>

Fig. 2.

Forest plot of effect sizes on behavior sorted by size

Fig. 3.

Forest plot of effect sizes on behavioral determinants sorted by size

Fig. 4.

Forest plot of effect sizes on clinical outcomes sorted by size

There was high heterogeneity between games’ effects for behavior and behavioral determinants, both at first post-test and at follow-up test, but not for clinical outcomes. There were no significant differences in average effect size on behavior nor on its determinants or clinical outcome by health domain. No significant heterogeneity was observed between games’ effects in different health domains.

A sensitivity analysis was conducted for lower (r=0.20) and higher (r=0.80) pre-test post-test correlation values, during which effect sizes largely remained within the 95^th percent confidence interval. Since one study had an extremely large sample size, analyses were also run without this study. As for the other sensitivity analyses, effect size here remained within the 95^th percent confidence interval (Table 3).

Table 3.

Average effect sizes for behavior, determinants and clinical outcomes

Outcome	n	k	Hedges’ g (95% CI)	p	Q	p	I2 Index
BEHAVIOR
Mean effect size 1st measurement	314790	21	0. 260 [0.148; 0.373]	<.001	406.96	<.001	95%
Without Kärnä et al 2011a (large sample)	17058	20	0.277 [0.109; 0.445]	<.01	385.97	<.001	95%
-at r=0.20	314790	21	0.240 [0.128; 0.352]	<.001	399.70	<.001	95%
-at r=0.80	314790	21	0.312 [0.195; 0.428]	<.001	436.69	<.001	95%
Health domain (behavior)	314790	21			3.62	0.163
Health responsibility/maintenance	2640	5	0.541 [−0.164; 1.246]	.133	258.86	<.001	98%
Diet & physical activity	5578	13	0.138 [0.085; 0.191]	<.001	8.84	0.717	0%
Social behavior	306573	3	0.098 [0.086; 0.109]	<.001	1.16	0.559	0%
Mental health	0	0	NA		NA
Mean effect size 2nd measurement
Behavior	11565	10	0.110 [−0.009; 0.228]	0.070	36.29	<.001	75%
BEHAVIORIAL DETERMINANTS
Mean effect size 1st measurement	19934	50	0. 334 [0.260; 0.407]	<.001	183.24	<.001	73%
-at r=0.20	19934	50	0.300 [0.231; 0.369]	<.001	159.78	<.001	69%
-at r=0.80	19934	50	0.416 [0.331; 0.501]	<.001	258.11	<.001	81%
Health domain (behavioral determinants)	19854	48			2.98	0.226
Health responsibility/maintenance	7147	29	0.294 [0.211; 0.377]	<.001	61.59	<.001	55%
Diet & physical activity	4184	14	0.460 [0.290; 0.630]	<.001	71.64	<.001	80%
Social behavior	8305	4	0.298 [−0.097; 0.694]	0.139	11.08	<.05	73%
Mental health	80	2	0.235 [−0.110; 0.580]	0.182	0.11	0.744	0%
Mean effect size 2nd measurement
Behavioral determinants	10919	12	0.262 [0.139; 0.384]	<.001	40.50	<.001	73%
CLINICAL OUTCOMES
Mean effect size 1st measurement clinical outcomes	9367	10	0. 079 [0.039; 0.120]	<.001	2.18	0.988	0%
-at r=0.20	9367	10	0.063 [0.023; 0.104]	<.01	1.62	0.996	0%
-at r=0.80	9367	10	0.123 [0.083; 0.164]	<.001	4.14	0.902	0%
Health domain (clinical outcomes)	1161	9			0.07	0.793
Health responsibility/maintenance	689	5	0.101 [−0.049; 0.250]	0.187	0.64	0.958	0%
Diet & physical activity	249	3	0.068 [−0.124; 0.260]	0.487	1.21	0.547	0%
Social behavior	8166	1	0.077 [0.034; 0.120]	<.01	NA
Mental health	40	1	0.181 [−0.236; 0.599]	0.395	NA
Mean effect size 2nd measurement
Clinical outcomes	8928	7	0.101 [0.060; 0.143]	<.001	0.80	0.997	0%

n=combined sample size; k=number of studies; Hedges’ g (random effects); CI= confidence Interval; Q= homogeneity statistic (mixed effects); I²=Inconsistency, a second measure of heterogeneity; NA: not applicable

Funnel plot and Egger’s Test (t(48)=6.042, p<.001) indicated publication bias could not be ruled out for effects on behavioral determinants with a tendency for smaller published studies to report higher effect sizes. Using Duval’s and Tweedie’s Trim and Fill approach to impute missing studies to the left of the mean, adjusted average g for behavioral determinants was reduced to g=0.179 (95% CI 0.104; 0.254) but remained significant. For effects on behavior (t(19)=1.51, p=0.07) and on clinical outcomes (t(8)=0.762, p=0.233), Egger’s Test on publication bias was not significant.

Second post-test

Average effect size for behavior at second post-test measurement was smaller than at first post-intervention measurement and failed to reach significance (p=.07). For behavioral determinants and clinical outcomes, average effect size at second post-test measurement was significant and of similar magnitude as at first post-measurement (Table 3).

Influence of study, sample and game characteristics as moderators of game effectiveness

Outcome of interest

Among specific behavioral determinants, the largest effects were found for improvement in knowledge and attitudes; however, all effect sizes were small. Average effect size for subjective norms was non-significant (measured in only one study) (Table 4).

Table 4.

Average effect sizes for behavioral determinants

Mean effect size 1st measurement	19934	50	0. 334 [0.260; 0.407]	<.001	183.24	<.001	73%
Knowledge	8221	32	0.407 [0.311; 0.504]	<.001	100.77	<.001	69%
Attitudes	10023	8	0.386 [0.159; 0.614]	<.01	61.10	<.001	89%
Self-efficacy	13120	20	0.212 [0.091; 0.333]	<.01	113.91	<.001	83%
Skills	9561	7	0.352 [0.075; 0.628]	<.05	65.05	<.001	91%
Subjective norms	907	1	−0.086 [−0.220; 0.048]	0.207	NA
Perceived barriers	452	1	0.198 [0.014; 0.383]	<.05	NA
Behavioral intentions	1497	6	0.308 [0.072; 0.545]	<.05	15.60	<.01	68%

Mean effect size 2nd measurement	10919	12	0.262 [0.139; 0.384]	<.001	40.50	<.001	73%
Knowledge	2075	8	0.464 [0.289; 0.639]	<.001	20.38	<.01	66%
Attitudes	9237	3	0.070 [0.029; 0.111]	<.01	1.06	0.590	0%
Self-efficacy	10580	8	0.101 [0.044; 0.159]	<.01	8.07	0.327	13%
Skills	8206	2	0.050 [0.006; 0.093]	<.05	0.19	0.663	0%
Subjective norms	907	1	0.186 [0.052; 0.320]	<.01	NA
Perceived barriers	442	1	−0.002 [−0.188; 0.184]	0.986	NA
Behavioral intentions	1071	2	0.129 [0.006; 0.251]	<.05	0.42	0.520	0%

n=combined sample size; k=number of studies; Hedges’ g (random effects); CI= confidence Interval; Q= homogeneity statistic (mixed effects); I²=Inconsistency, a second measure of heterogeneity; NA: not applicable

At second post-test, average effect size remained highest for knowledge and approximated a moderate effect size, but average effects were smaller for other behavioral determinants compared to first post-measurement (with the exception of subjective norms, measured in only one study) (Table 4).

Individual tailoring

Individual tailoring was only a significant moderator for the effects on behavioral determinants, but not for behavior or clinical outcomes (Table 5–7). For behavioral determinants, games tailored only by change needs or stages of adoption had a lower effect size than games which were not tailored or tailored to both socio-demographics and change needs. Since games tailored only by socio-demographic information were not included in moderator analyses (k<3), it is unclear whether these games would have a significantly higher effect than those tailored only by stages of adoption or change needs (Table 6).

Table 5.

Moderator analyses on behavior (first post-test)

Moderator	n	k	Hedges’ g (95% CI)	p	Q	p	I2 Index
EPHPP study quality	312914	20			0.00	0.974
Weak	1876	1	0.123 [0.032; 0.213]	<.01	NA
Moderate	1245	5	0.267 [0.129; 0.404]	<.001	1.50	0.827	0%
Strong	311670	15	0.263 [0.122; 0.404]	<.001	400.28	<.001	97%
Study design	312914	20			5.92	0.052
Non-equivalent post-test only	1876	1	0.123 [0.032; 0.213]	<.01	NA
Non-equivalent pretest- post-test control group	13383	9	0.269 [−0.032; 0.570]	.080	358.17	<.001	98%
Post-test only control group	298100	3	0.099 [0.087; 0.111]	<.001	0.53	0.767	0.000
Pre-test post-test control group	1432	8	0.325 [0.121; 0.529]	<.01	23.79	<.01	71%
Tailoring	7195	16			0.357	0.550
No tailoring	3925	10	0.234 [0.101; 0.368]	<.01	27.79	<.01	68%
Tailoring by socio-demo	675	1	0.194 [−0.041; 0.429]	0.105	NA
Tailoring by stages/needs	3270	6	0.414 [−0.160; 0.989]	0.158	245.82	<.001	98%
Tailoring by socio-demo & stages/needs	907	1	−0.096 [−0.229; 0.038]	0.162	NA
Theoretical foundation	314050	19			3.16	0.206
None mentioned	2243	3	0.131 [0.047; 0.215]	<.01	0.23	0.890	0%
Only behavioral prediction theory	308852	7	0.270 [0.061; 0.480]	<.05	370.17	<.001	98%
Only game theory	66	1	0.496 [0.012; 0.981]	<.05	NA
Game and clinical psychology approaches/theory-based methods	675	1	0.194 [−0.041; 0.429]	0.105	NA
Both behavioral and game theory	2955	9	0.275 [0.104; 0.446]	<.01	29.96	<.001	73%
Implementation method	314790	21			1.22	0.269
Stand-alone	5517	12	0.307 [−0.004; 0.618]	0.053	271.61	<.001	96%
Multi-component	309273	9	0.127 [0.052; 0.202]	<.01	41.17	<.001	81%

^*Categories with fewer than 3 observations were removed from moderator analyses when calculating heterogeneity

Table 7.

Moderator analyses on clinical outcomes

Moderator	n	k	Hedges’ g (95% CI)	p	Q	p	I2 Index
EPHPP study quality	9367	10			0.07	0.794
Weak	0	0	NA
Moderate	499	5	0.102 [−0.070; 0.273]	0.246	1.43	0.838	0%
Strong	8868	5	0.078 [0.036; 0.120]	<.001	0.68	0.954	0%
Study design	9367	10			NA
Non-equivalent pre-test-post-test control group	8389	2	0.077 [0.034; 0.120]	<.001	0.00	0.989	0%
Pre-test post-test control group	978	8	0.099 [−0.025; 0.223]	0.117	2.07	0.956	0%
Tailoring	1086	8			NA
No tailoring	834	6	0.106 [−0.029; 0.241]	0.123	0.80	0.977	0%
Tailoring by socio-demo	0	0	NA
Tailoring by stages/needs	252	2	0.142 [−0.114; 0.398]	0.278	0.22	0.639	0%
Tailoring by socio-demo & stages/needs	0	0	NA
Theoretical foundation	9367	10			NA
None mentioned	119	1	0.221 [−0.140; 0.581]	0.230	NA
Clinical psychology approaches/theory-based methods	40	1	0.181 [−0.236; 0.599]	0.395	NA
Only behavioral prediction theory	8317	2	0.078 [0.035; 0.121]	<.001	0.04	0.851	0%
Only game theory	0	0	NA
Both behavioral and game theory	891	6	0.066 [−0.070; 0.202]	0.339	1.28	0.937	0%
Implementation method	9367	10			0.15	0.700
Stand-alone	475	4	0.113 [−0.062; 0.287]	0.206	0.64	0.888	0%
Multi-component	8892	6	0.077 [0.036; 0.119]	<.001	1.40	0.925	0%

n=combined sample size; k=number of studies; Hedges’ g (random effects); CI= confidence Interval; Q= homogeneity statistic (mixed effects); I²=Inconsistency, a second measure of heterogeneity; NA: not applicable

Table 6.

Moderator analyses on behavioral determinants

Moderator	n	k	Hedges’ g (95% CI)	p	Q	p	I2 Index
EPHPP study quality	19934	50			0.24	0.886
Weak	3172	10	0.317 [0.192; 0.442]	<.001	13.61	0.137	34%
Moderate	3750	17	0.357 [0.223; 0.490]	<.001	50.04	<.001	68%
Strong	13012	23	0.317 [0.206; 0.429]	<.001	97.66	<.001	77%
Study design	18058	49			3.30	0.192
Non-equivalent post-test only	1876	1	0.220 [0.129; 0.311]	<.001	NA
Non-equivalent pre-test-post-test control group	13952	17	0.278 [0.161; 0.395]	<.001	83.87	<.001	81%
Post-test only control group	440	5	0.223 [0.009; 0.438]	<.05	3.11	0.540	0%
Pre-test post-test control group	3666	27	0.391 [0.297; 0.485]	<.001	47.96	<.01	46%
Tailoring	11654	48			11.79	<.01
No tailoring	7228	36	0.381 [0.293; 0.468]	<.001	81.03	<.001	57%
Tailoring by socio-demo	114	1	1.011 [0.622; 1.400]	<.001	NA
Tailoring by stages/needs	3440	9	0.146 [0.037; 0.255]	<.01	16.47	<.05	15%
Tailoring by socio-demo & stages/needs	987	3	0.634 [−0.042; 1.310]	0.066	11.92	<.01	83%
Theoretical foundation	19934	50			17.73	<.01
None mentioned	3150	10	0.403 [0.272; 0.534]	<.001	14.53	0.105	38%
Clinical psychology approaches/theory-based methods	168	3	0.354 [−0.214; 0.921]	0.222	7.49	<.05	73%
Only behavioral prediction theory	12691	12	0.149 [0.068; 0.230]	<.001	26.45	<.01	58%
Only game theory	1112	9	0.432 [0.193; 0.671]	<.001	16.12	<.05	50%
Both behavioral and game theory	2813	16	0.417 [0.260; 0.574]	<.001	58.54	<.001	74%
Implementation method	19934	50			1.06	0303
Stand-alone	7444	31	0.371 [0.263; 0.480]	<.001	106.04	<.001	72%
Multi-component	12491	19	0.293 [0.192; 0.395]	<.001	62.17	<.001	71%

n=combined sample size; k=number of studies; Hedges’ g (random effects); CI= confidence Interval; Q= homogeneity statistic (mixed effects); I²=Inconsistency, a second measure of heterogeneity; NA: not applicable

Theoretical basis

The type of theoretical foundation significantly affected effectiveness for behavioral determinants, but not for behavior or clinical outcomes (Table 5–7). Games based purely on behavioral prediction theories had the lowest average effect size. Their effects were however still significant (Table 6).

Implementation method

Implementation method was not a significant moderator of game effectiveness for either behavior, determinants or clinical outcomes (Table 5–7). Both forms of implementation had significant average effect sizes for behavioral determinants and clinical outcomes. Average effects on behavior of stand-alone games however showed only a trend to significance.

Sample characteristics

Neither the average age of participants nor the percentage of women in the sample was a significant moderator on game effectiveness on behavior, determinants or clinical outcomes (note: only one game for the elderly was included) (Table 8).

Table 8.

Meta-regression analyses

Moderators	n	k	β (95% CI)	p
BEHAVIOR
Time until 1st measurement	6303	14	0.005 [0.002; 0.008]	<.01
Duration of intervention	6556	15	0.000 [−0.001; 0.002]	0.325
Time until follow-up	2492	8	−0.003 [−0.006; −0.001]	<.01
Publication year	314790	21	0.008 [−0.024; 0.040]	0.619
Mean age	314790	21	0.013 [−0.005; 0.031]	0.160
% female	314790	21	0.001 [−0.000; 0.004]	0.242
Play duration (hours)	6245	13	−0.005 [−0.036; 0.026]	0.749
DETERMINANTS
Time until 1st measurement	7668	35	−0.001 [−0.003; 0.001]	0.539
Duration of intervention	9005	44	−0.001 [−0.002; 0.000]	0.156
Time until follow-up	1391	6	−0.002 [−0.004; −0.000]	<.05
Publication year	19934	50	−0.005 [−0.018; 0.008]	0.468
Mean age	19667	46	−0.001 [−0.007; 0.005]	0.823
% female	18199	44	0.002 [−0.001; 0.005]	0.231
Play duration (hours)	7975	39	−0.011[−0.026; 0.005]	0.165
CLINICAL OUTCOMES
Time until 1st measurement	754	5	−0.002 [−0.013; 0.008]	0.647
Duration of intervention	1067	8	0.000 [−0.002; 0.002]	0.830
Time until follow-up	433	4	−0.000 [−0.004; 0.003]	0.878
Publication year	9367	10	−0.004 [−0.022; 0.015]	0.686
Mean age	9157	8	0.001 [−0.006; 0.008]	0.795
% female	9367	10	0.001 [−0.004; 0.006]	0.767
Play duration (hours)	741	6	0.004 [−0.011; 0.012]	0.593

Note: all effect sizes are at first post-measurement, except Time until follow-up

Study characteristics

The quality of the study did not significantly affect the effectiveness for behavior, determinants or clinical outcomes (Table 5–7). For behavior, games evaluated in a post-test only control groups design had a lower average effect size (borderline significant) than games evaluated in other study designs (Table 5).

The elapsed time between the last time the game was played and when the outcome was measured, was a significant moderator in the games’ effects, but only for behavior. More specifically, the greater the amount of time between the last game play and first post-test for behavior, the larger the effect size (Table 8).

This was, however, the opposite at second post-test: here the greater the time elapsed until post-test measurement, the smaller the effect size for behavior, suggesting a curvilinear relation, in which a minimal time is needed to practice the behavior at first post-test to have effects, but where a longer follow-up time showed smaller effects on behavior. This inverse relation between time and effect was also true for behavioral determinants (Table 8). None of the other study characteristics were significant moderators in the average effects on behavior, determinants or clinical outcomes. <enter Table 8 here>

Discussion

This was the first article to conduct a meta-analysis of serious games for healthy lifestyle promotion. Overall, serious games increased healthy lifestyle adoption (g=0.260, 95% CI 0.148; 0.373) and improved antecedents that determine adoption (g=0.334, 95% CI 0.260; 0.407), and this applied across the several included health domains.

Overall, effect sizes were small. The effect sizes on behavior were however in line with findings of two meta-analyses on computer-delivered interventions (range between 0.13 and 0.22 (Krebs, Prochaska, & Rossi 2010); and range between 0.04 and 0.35 (Portnoy, Scott-Sheldon, Johnson, & Carey 2008)). Hence, serious games for the promotion of healthy lifestyles may be considered as effective as other computer-delivered interventions. The effects on clinical outcomes (g=0.08 at first measurement and g=0.10 at second measurement in our meta-analysis) were also comparable to the available literature (d=0.10 in Portnoy et al., 2008). Health professionals and policy makers may therefore consider serious games as an alternative to other computer-delivered interventions. Further insights on time-efficiency and cost-effectiveness of these methods for health promotion may also be needed to inform this choice. This information is, to our knowledge, not yet available for healthy lifestyle promotion. An educational game showed that, possibly, more time may be needed to create learning effects via game-based learning than with traditional methods (Huizenga et al. 2009). Players of educational games, however, also appeared willing to spend more learning time on games than on learning via traditional methods, provided the challenge in the game was feasible (Bourgonjon et al. 2010;Squire 2005).

Furthermore, strategies recommended to overcome low reach and adoption of computer-tailored interventions, such as higher interactivity and visual attractiveness (Crutzen et al. 2008;Crutzen et al. 2011), are inherent in serious games. Coupled with their intrinsically motivating fun aspect (Ritterfeld, Cody, & Vorderer 2009), serious games may prove a better medium than other computer-delivered interventions to reach the target audience in a large scale implementation. This hypothesis, however, awaits further corroboration.

Our meta-analysis revealed a large heterogeneity between studies, which warrants investigation of putative moderators. We were able to identify some moderators.

First, effect sizes varied as a function of type of outcome. The effects were largest on knowledge, which approximated a moderate effect size at follow-up. This is consistent with findings from general health promotion and computer-delivered interventions, that knowledge is easier to influence than other outcomes (Portnoy, Scott-Sheldon, Johnson, & Carey 2008). Regrettably, this is often not the most relevant determinant since the causal relationship between knowledge and behavior change is weak (Bartholomew, Parcel, Kok, Gottlieb, & Fernàndez 2011). The effect on the intention to change health behavior, which is often a strong predictor of behavior, is more promising. Unfortunately, the effect on behavioral intention at follow-up measurement was substantially reduced, as was the case for self-efficacy, which is another determinant with a strong association with behavior change (Bartholomew, Parcel, Kok, Gottlieb, & Fernàndez 2011). Games are thought to provide a good medium for increasing self-efficacy as they offer the opportunity for enactive experience in a safe environment, without real-life consequences to making wrong decisions (Peng 2008;Prensky 2007). In our review, effects on self-efficacy were unexpectedly small. A review of computer-tailored interventions also reported limited effects on self-efficacy (Lustria et al. 2009). Possibly, self-efficacy may have been unrealistically high at the start of the intervention, known as optimistic bias. This may have been corrected when more information and experience became available (Schwarzer 1998).

For computer-tailored interventions, the use of self-regulation theories has been recommended to improve effectiveness (Lustria, Cortese, Noar, & Glueckauf 2009). The few (n=5, k=1071) game evaluations in our review using these theories and measuring self-efficacy outcomes did nevertheless not show significantly higher effects than games without this theoretical foundation (Q=0.711, p=0.399).

Although effects at follow-up measurement were significant and positive for most outcomes, they were smaller than at the immediate post-measurement, and failed to reach significance for behavior. Methods used in non-game healthy lifestyle interventions to maintain a long-term effect on behavior were providing reinforcement throughout follow-up, having booster sessions, sending tailored reminders, using self-regulation and anticipatory coping techniques to deal with potential relapse, integrating Motivational Interviewing techniques and creating healthy-behavior-facilitating changes in the environment via multi-level programs (Jacobs et al. 2004;Lustria, Cortese, Noar, & Glueckauf 2009;Martins and McNeil 2009;Norris et al. 2001;Orleans 2000).

Second, individual tailoring related significantly to the games’ effects on determinants, but not on behavior or clinical outcome. It appeared insufficient to tailor games only to behavioral change needs, instead games are best tailored by both socio-demographic information (e.g. age, gender, body frame) and behavioral change needs (e.g. current level of lifestyle adoption, already acquired knowledge, stages of change, or motivation). Similarly, in computer-tailored interventions, tailoring showed the strongest effects when based on both theoretical concepts (e.g. stages of change, decisional balance, attitudes) and user characteristics (Noar et al. 2007) and performed via dynamic tailoring, in which tailoring is continuously adjusted during the intervention (Krebs, Prochaska, & Rossi 2010). For games, dynamic tailoring has been recommended based on the difficulty level the player can master (e.g. via providing hints when the challenge is too hard), to ensure flow and to provide all players with an optimal level of challenge (Charles et al. 2005;Wilson et al. 2009). In games, not only the educational content but also game rules and environment need to be learned by players. Games with highly immersive features designed to increase game enjoyment were found to increase cognitive load and decrease the performance on the educational task: players were focusing on mastering the game requirements, which competed with the cognitive resources available to grasp the educational content (Schrader and Bastiaens 2012). It may thus be advisable to tailor the game also to the player’s prior gaming experience and adjust this continuously based on their game play proficiency. Only two games in our review used dynamic tailoring. Further experimental research is needed to clarify the benefits of dynamic tailoring in serious games for health promotion.

Third, as hypothesized, the theories used for game development are a significant element in the serious game’s effectiveness, but only on behavioral determinants and not on behavior or clinical outcomes. A strong focus on game-based learning theories (e.g. Transportation Theory) alone or combined with behavioral prediction theories (e.g. Social Cognitive Theory) related to higher game effectiveness than games founded only on behavioral prediction theories. This finding supports hypotheses from previous articles suggesting that a good theoretical blending of the ‘fun’ and ‘educational’ element is critical in game effectiveness (Baranowski et al. 2011;DeShazo, Harris, & Pratt 2010;Kato 2012;Kharrazi, Lu, Gharghabi, & Coleman 2012;Papastergiou 2009). While serious games share some characteristics with computer-tailored programs, the importance of game-based learning theories underlines the specificity of serious games as an intervention mode. Merely using behavioral prediction theories was associated with the smallest effects in our meta-analysis, whereas reviews on computer-tailored interventions have reported moderate effects for programs only based on behavioral prediction theories (Webb et al. 2010). Often game studies briefly mentioned which theoretical foundation was used, but not how theoretical techniques were applied to address the target outcome. Interpretation and application of theories may therefore have been different across game studies. Other studies have indeed reported that the terminology of theories is not used in a consistent manner across interventions and that one term may be used to represent different techniques, as well as that different labels may be used for the same technique (Michie et al. 2011a). More detailed process descriptions on how theory guided change in specific target outcomes, e.g. using the Intervention Mapping Protocol (Bartholomew, Parcel, Kok, Gottlieb, & Fernàndez 2011) or the Behavior Change Wheel (Michie et al. 2011b), are needed.

Some moderators did not explain heterogeneity in the games’ effects. The duration of the full intervention did not relate to a different effectiveness, in contrast with the finding of Primack et al. (Primack, Carroll, McNamara, Klem, King, Rich, Chan, & Nayak 2012). They reported that interventions lasting fewer than 12 weeks more often achieved their objectives than interventions lasting longer. This result was mostly influenced by the solidly positive effects of one-time interventions (Primack, Carroll, McNamara, Klem, King, Rich, Chan, & Nayak 2012). Since they did not assess the time between the intervention and measurement, length of intervention and time between intervention and measurement may be confounded here, as effects of one-time interventions were likely to be measured immediately afterwards. These were coded separately in our meta-analyses, singling out effect of intervention duration versus duration until measurement. While this seems counter-intuitive, a meta-analysis on computer-delivered interventions for health promotion also revealed that the dose of the intervention (ranging from 1 hour for stand-alone interventions to 4–8 hours per component in multi-component programs) did not significantly relate to the average intervention effect (Portnoy, Scott-Sheldon, Johnson, & Carey 2008).

Similarly, game play duration was, contrary to expectations, not significantly related to effectiveness. Game duration in general was low, with a total average of 4 hours, while leisure games are played 7 hours per week on average (Boyle, Connolly, Hainey, & Boyle 2012). Insufficient playing time was previously mentioned as a potential reason for lack of effectiveness (Rahmani & Boren 2012), but this was not supported by our findings. Possibly a floor effect was at play here.

Both games evaluated as part of a multi-component program and as stand-alone games were effective, although the effects on behavior of stand-alone games were only borderline significant (p=0.053), despite a higher average effect size than games that were part of multi-component programs. Effect sizes of stand-alone games showed significant heterogeneity. Many other game characteristics (e.g. use of a narrative, challenge in the game, co-design, scaffolding levels) (Kapp 2012;Lu, Baranowski, Thompson, & Buday 2012) may influence game effectiveness and may be applied differently in stand-alone or multi-component games. Future investigations into these game features are needed to further explore the high heterogeneity within stand-alone and multi-component games.

Differences in effect size were also not found for mean age or gender, confirming earlier findings that serious games can work in various populations (Primack, Carroll, McNamara, Klem, King, Rich, Chan, & Nayak 2012), when their preferences are taken into account (Brox et al. 2011).

Future studies should aim to strengthen the evidence base for interventions with a dual theoretical foundation in game and behavioral prediction theories, clearly describing theoretical frameworks allowing to discern how key features were applied, using rigorous designs with high external validity, longer play durations, tailoring dynamically throughout the game and including a follow-up measurement and techniques on how to maintain the effect in the longer term.

Limitations

Some limitations to this meta-analysis need to be noted. First, ‘no evidence for an effect’ does not equal ‘evidence for no effect’. In some areas, analyses were likely statistically underpowered (e.g. mental health, social behavior, clinical outcomes). Second, other factors (e.g. game features), may partially explain the high heterogeneity which indicates results should be treated with caution. These features were not included here. Third, publication bias is always a concern: reported effect sizes may be overestimated (e.g. behavioral determinants). The analyses were furthermore limited by the information available in the manuscripts and that additionally obtained from the researchers. More detailed descriptions of the games themselves and of the development process are needed to make empirical advances in this field. Lastly, this review included various types of health behaviors, which may be difficult to compare. There was, however, no significant heterogeneity related to the various health domains which were included. Decreasing risky behavior may be harder to achieve than increasing health-promoting behavior (Adriaanse et al. 2011). Our meta-analysis did not distinguish between reducing unhealthy behavior or increasing healthy actions. Future meta-analysis may wish to include these differences and also take into account the health status of the target group.

Conclusions

Findings indicated that serious games had positive effects on healthy lifestyles and their determinants, especially for knowledge, and on clinical outcomes. Long-term effects were maintained for knowledge, but not for behavior. Serious games were best individually tailored to both socio-demographic and change need information and benefited from a theoretical foundation in both behavioral prediction and game theories. They were effective either as a stand-alone or multi-component program and appealed to a variety of populations regardless of age and gender. Given that effects of games remain heterogeneous, further explorations of which game features create larger effects are needed.