In Pursuit of Change: Youth Response to Intensive Goal Setting Embedded in a Serious Video Game

Abstract

Background

Type 2 diabetes has increased in prevalence among youth, paralleling the increase in pediatric obesity. Helping youth achieve energy balance by changing diet and physical activity behaviors should decrease the risk for type 2 diabetes and obesity. Goal setting and goal review are critical components of behavior change. Theory-informed video games that emphasize development and refinement of goal setting and goal review skills provide a method for achieving energy balance in an informative, entertaining format. This article reports alpha-testing results of early versions of theory-informed goal setting and reviews components of two diabetes and obesity prevention video games for preadolescents.

Method

Two episodes each of two video games were alpha tested with 9- to 11-year-old youth from multiple ethnic groups. Alpha testing included observed game play followed by a scripted interview. The staff was trained in observation and interview techniques prior to data collection.

Results

Although some difficulties were encountered, alpha testers generally understood goal setting and review components and comprehended they were setting personal goals. Although goal setting and review involved multiple steps, youth were generally able to complete them quickly, with minimal difficulty. Few technical issues arose; however, several usability and comprehension problems were identified.

Conclusions

Theory-informed video games may be an effective medium for promoting youth diabetes and obesity prevention. Alpha testing helps identify problems likely to have a negative effect on functionality, usability, and comprehension during development, thereby providing an opportunity to correct these issues prior to final production.

Background

Type 2 diabetes has increased in prevalence among youth,¹ paralleling increases in pediatric obesity.² Preventing obesity reduces type 2 diabetes risk.³ Obesity results from a long-term imbalance in energy whereby intake exceeds expenditure.⁴ Thus, obesity prevention efforts should target both decreased energy intake and increased energy expenditure to achieve energy balance.

Energy balance can be achieved by matching calorie intake with expenditure.⁴ Consuming foods low in energy density, such as fruit and vegetables, has been correlated with weight loss.⁵ It is well established, however, that youth consume less than the recommended servings of fruit and vegetables per day.^6,7 Sweetened beverage consumption, such as soft drinks, has increased among youth⁸ and has been associated with excess energy intake and weight gain.⁹ Because there are limits to the amount of fluid a person can consume, increasing water intake should decrease sweetened beverage intake. Youth physical activity, the major source of energy expenditure under volitional control,⁴ is also less than optimal, with decreases occurring throughout adolescence.¹⁰ Alternatively, sedentary behavior, such as television viewing, has been positively associated with body mass index.¹¹ Thus, promoting increased consumption of low-energy dense foods and beverages such as fruit, vegetables, and water; more physical activity; and less sedentary behavior should promote energy balance, thereby reducing the risk of both obesity and type 2 diabetes.

Behavior is often influenced by many factors,^12,13 making behavior change a challenging process.¹⁴ Theory codifies and systematizes what is known about the major influences on a particular behavior.¹⁵ Therefore, using a theoretical framework to design an intervention that specifically targets variables highly related to the behavior of interest (also known as mediating variables) should increase the probability of success.^14,16 Interventions targeting complex behaviors often have more than one mediating variable.^14,16 Mediating variables are typically obtained from behavioral theory.¹⁶ Common theories from which to derive these variables include both social cognitive theory (SCT)¹⁷ and self determination theory (SDT),¹⁸ whereas the elaboration likelihood model (ELM)¹⁹ and behavioral inoculation theory (BIT)²⁰ offer procedures and techniques that facilitate behavior change. A brief review of each of these theories and their theoretical contributions to behavior change is outlined. Figure 1 is a graphical representation of the hypothesized interrelationships among theory-specified variables in promoting behavior change.

Figure 1.

Hypothesized goal setting and behavior change pathway.

Social Cognitive Theory

The social cognitive theory¹⁷ posits that behavior, personal factors, and the environment are interrelated and work together to achieve behavior change. Personal factors, such as behavior-specific knowledge, self-efficacy, and/or self-regulatory skill development (i.e., goal setting, problem solving, and goal review), have been included in interventions attempting to change youth diet and/or physical activity behavior.^21–26

Behavior-Specific Knowledge

Behavior-specific knowledge provides a necessary, but not sufficient, precondition to behavior change.¹⁷ However, attempting to change behavior without possessing basic behavior-specific knowledge can lead to difficulty. For example, when attempting to consume more fruit, it is important to understand that cherry gelatin is not a fruit, but that cherries are. Interventions attempting to change behavior should include basic knowledge regarding skills and behaviors related to the behavior. Thus, knowledge is a mediator of behavior change, because without basic knowledge, successful behavior change is unlikely.

Self-Efficacy

Self-efficacy is one's confidence in personal ability to perform a specific task or behavior successfully.¹⁷ It affects task choice, persistence, and effort, all of which can ultimately impact behavior.²⁷ Experiencing success while attempting a behavior (i.e., personal mastery) has the most dramatic positive effect on self-efficacy,¹⁷ but observing others (i.e., models) perform a behavior successfully may also have an effect.¹⁷

Self-Regulatory Skills

Self-regulatory skills enable an individual to exert willful control over their behavior.¹⁷ Self-regulatory skills, such as goal setting, problem solving, and review of progress to goal attainment through self-monitoring, have been included in youth diet and physical activity behavior change programs.^21–26

Goal setting

Goal setting, a key self-regulatory procedure, provides focus to behavior change efforts.²⁸ Goals, an integral component of goal setting, are statements of behavioral intention.²⁸ Specific, short-term goals are more likely to be enacted, thereby facilitating behavior change.²⁸ For example, having a goal to “make healthy choices” is not specific, but a goal to “drink one more glass of water a day in the afternoon after school for the next 2 days” is specific and short term. It is also measurable, which facilitates self-monitoring and assessment of goal progress.²⁸

Goal setting consists of two phases: goal intentions and implementation intentions.²⁹ Goal intentions are what is intended (i.e., the goal itself), whereas implementation intentions specify how to meet a goal by examining possible situations and selecting the ones most likely to lead to goal attainment.²⁹ Implementation intentions are tightly connected to goal intentions and facilitate goal attainment by creating a mental link between the goal and possible future situations.²⁹ Environmental cues, rather than willful thought, trigger a goal-directed response, thereby automatizing behavior,³⁰ even when cognitive resources are devoted to other activities.³¹ Automatizing the behavior increases the likelihood of goal attainment.³⁰ Implementation intentions are stable, help overcome habitual behavior, and have enhanced goal attainment for a variety of behaviors.³² Implementation intentions can be a specific plan of how to achieve a goal or a plan for overcoming anticipated barriers to goal attainment (Table 1).³² Regardless of type, implementation intentions should be specific (rather than general), possible (i.e., the individual possesses the necessary resources and/or abilities to carry out the plan), and lead to goal attainment if followed (i.e., efficacious).³²

Table 1.

Examples of Implementation Intentions

Goal intention: Eat bananas on my cereal for breakfast on [day/date]

Possible implementation intentions to create a plan of goal achievement: When I see Mom after school today, I will ask her to add bananas to the grocery shopping list When I get home from school today, I will check the pantry to make sure we have bananas for my cereal; if we do not have any, I will call Mom and ask her to get some on her way home from work On my way home from school today, when I pass the corner store, I will stop and buy a banana for my cereal

Possible implementation intentions to overcome an anticipated barrier:

Possible barrier: forget to eat a banana on my cereal When I get home from school today, write myself a note to eat a banana on my cereal and tape it to the bathroom mirror When Mom gets home from work tonight, ask her to remind me to eat a banana on my cereal When I get home from school today, put my cereal, a bowl, a spoon, and a banana on the kitchen counter beside the refrigerator

Problem solving

Barriers impede goal attainment,³³ and it is well documented that youth perceive barriers to eating healthy³⁴ and being physically active.^35,36 Problem solving, an important self-regulatory skill, ideally identifies effective strategies to overcome barriers and attain desired goals.³⁷ For example, a physical activity barrier youth may face is having to stay inside after school because they live in an unsafe neighborhood. Rather than simply giving up and not being physically active, problem solving can generate personally relevant strategies for overcoming this barrier, such as staying inside and dancing to music, climbing interior stairs, or going to the community center.

Goal review

Goal review helps assess progress toward goal attainment. A critical precursor to goal review is self-monitoring, as it provides information about goal progress and supports the behavior change process.¹⁷ It has been suggested that self-monitoring alone may lead to behavior change.³⁸

Self Determination Theory

The self determination theory¹⁸ posits that behavior is intrinsically or extrinsically motivated. Behavior that is intrinsically motivated is performed for the satisfaction it brings (e.g., walking on the beach; reading a favorite novel), whereas extrinsically motivated behavior is performed for a specific reason (e.g., reward; recognition; pressure from others). The SDT contends that there are three basic needs that drive behavior: competence (i.e., skills, ability to perform a behavior successfully), autonomy (i.e., choice, control), and relatedness (i.e., connection to important others or ideals). The degree to which these needs are met determines if a behavior is intrinsically or extrinsically motivated. For example, intrinsically motivated behavior is characterized by high levels of competence, autonomy, and relatedness, whereas the extreme state of extrinsic motivation is characterized by low levels of competence, lack of choice, and isolation. Therefore, SDT-guided interventions seek to develop competence, provide choice, and connect goals to important factors such as personal values.

Elaboration Likelihood Model

The elaboration likelihood model¹⁹ posits that the amount of attention and effort devoted to processing a message is determined by ability and motivation. Ability relates to factors that affect the capacity to process and act on a message (e.g., prior knowledge, comprehension, distraction), whereas motivation is influenced by the desire to act on a message (i.e., personally relevant messages about something of interest to an individual are more likely to be attended to than generic messages). Tailoring, a method for enhancing personal relevance, has been shown to be an effective method for changing behavior.³⁹

Behavioral Inoculation Theory

The behavioral inoculation theory²⁰ posits that “inoculating” or mentally preparing an individual to resist temptation not to perform a behavior, such as eating fruit for dessert, enhances their ability to do so. BIT has two components: identification of a potential threat, followed by “refutation preemption” (i.e., counterargument). Inoculation is thought to work by enhancing attitude accessibility and strength.⁴⁰

Video Games

Video games likely facilitate behavior change by creating an entertaining and engaging environment within which behavior-specific knowledge enhancement, self-regulatory skill acquisition, modeling of behaviors and skills, and tailoring to personal characteristics can occur. Video games have shown promising results at promoting youth dietary change,²¹ food acceptance,⁴¹ and physical activity.⁴² They have also been used to teach diabetes self-management skills.^43–45 Therefore, well-designed, theory-informed video games should be an effective channel for promoting energy balance to reduce the risk of obesity and type 2 diabetes among youth.

Testing during Video Game Development

During video game development, testing with the target audience is needed to observe users interacting with the product.^46–48 Although terminology regarding the early testing phase is inconsistent in the literature (i.e., referred to as either alpha or beta testing^46–48), in this article, it is referred to as alpha testing. Alpha testing provides a critical opportunity to identify and correct potential problems (i.e., technical, usability, comprehension) likely to affect playability (i.e., ability to play a game with ease and enjoyment) prior to full-scale development and final production.^46–48 Based on the results of this testing, small or large changes may be made to the structure and content of the video game. Alpha testing can also be iterative, with multiple rounds of testing.⁴⁶

This article reports key alpha-testing results of early versions of goal setting and review components of two diabetes and obesity prevention video games being developed for preadolescents.

Method

Video Game Descriptions

Escape from Diab (Diab) and Nanoswarm: Attack from Inner Space (Nanoswarm) are both nine-episode video games designed to work as a set. Diab, to be played first, addresses diet and physical activity separately, whereas Nanoswarm emphasizes energy balance. The video games have different story lines, characters, and approaches. The stories, written by professional writers, were developed using comments and suggestions from focus group testing of potential characters and story lines. Episodes 1–8 of both video games end with a cliff-hanger to entice players to return to learn the resolution.

Diab, told from the third-person perspective, uses three-dimensional sets and animated characters to tell the story of DeeJay, an athletic and healthy modern-day youth who falls through the floor of an abandoned building. He awakens to find himself in the dark and dreary land of Diab, where fruit, vegetables, and physical activity are forbidden by the evil King Etes. Upon his arrival, DeeJay is befriended by a group of Diab youth. Together, they develop a plot to escape. To develop the mental acuity and physical strength needed to outwit King Etes and his guards, DeeJay mentors his new friends in healthy eating and physical activity behaviors. The player engages in behavior change activities as part of game play. The ending the player receives (i.e., whether or not the characters have the option to escape) is dependent on his/her personal performance in the behavior change components of the game.

Nanoswarm, told from the first-person perspective, uses live actors and blue screen technology. In a futuristic laboratory, teenage scientists attempt to help Dr. Gunderson (the lead scientist who is an adult) control renegade nanobots, small machines originally designed to keep the environment free of garbage and pollutants that now appear to be attacking people. Dr. Gunderson created the nanobots, and his laboratory is charged with getting them under control. During the crisis, Fred, one of the teenage scientists with poor eating and physical activity habits, falls ill and is near death. To keep Fred alive until the source of his illness can be discovered, the characters (including the player) have to keep Fred's energy meters balanced by eating healthy and being physically active. As in Diab, the player participates in behavior change components embedded in game play.

Behavior Change Goals

The video games are designed to modify behaviors likely to decrease the risk of developing type 2 diabetes and obesity among preadolescents (10–12 year olds). Thus, by the conclusion of each video game, daily behavioral goals are to consume three to five servings of fruit, four to seven servings of vegetables, and five to eight glasses of water; participate in 60 minutes of moderate-to-vigorous physical activity; and engage in ≤2 hours of sedentary activity (e.g., television, video games, telephone).

Video Game Structure

Each multiepisode video game follows the same general structure, although there is variability in how the structure is operationalized within each video game. The story lines reinforce diet and physical activity behavior change objectives. Woven into each episode are activities promoting development and refinement of self-regulatory skills (i.e., goal setting, problem solving, goal review) tailored to each player's personal characteristics via questions answered during the game-player profile setup and at strategic times during game play. Minigames, incorporated into game play, enhance behavior-specific knowledge to support behavior change (e.g., “What counts as a fruit ”). Both video games use fantasy (e.g., awakening in a dark and dreary land where the player's goal is to develop skills to help the characters escape Diab; piloting a miniaturized craft inside the human body to save the life of a colleague), and video game characters convey knowledge and demonstrate self-regulatory skills through dialogue and modeling. Additionally, Nanoswarm includes an emphasis on asking and negotiation skills, whereas Diab does not.

Description of Self-Regulatory Video Game Components

Tailoring

Goal setting is tailored to player characteristics to enhance personal relevance. To obtain information needed for tailoring, players provide personal information as part of the video game player-profile setup and at strategic times during game play. For example, during the player-profile setup, the player provides information related to gender, height, weight, and age. An algorithm uses this information to estimate the player's daily calories,⁴⁹ which is then converted to estimations of servings of fruit and vegetables to consume at specific meals/snacks using “My Pyramid” guidelines of the U.S. Department of Agriculture (USDA)(www.mypyramid.gov). To offer a tailored menu of possible goals from which the player can select, questions are embedded in game play to obtain usual dietary intake, physical activity, or physical inactivity, corresponding to the goals in a particular episode. For example, in Diab, DeeJay, a central character, approaches street vendors and requests fruit, a forbidden food. In response, street vendors ask DeeJay (whose responses are controlled by the player) questions about his fruit and vegetable intake. DeeJay's (i.e., player's) responses appear to be secret passwords that alert the street vendor it is “safe” to give fruit to DeeJay. Algorithms convert the player's responses to tailored options in goal setting.

Goal-Setting Component

The goal-setting component helps the player set specific, short-term, measurable goals to increase the likelihood of behavior change. The number of goals increases across episodes to provide challenge and increase the extent of change. Players are offered choices (i.e., specific foods or activities, days to attempt goals) to enhance autonomy. “Reason” statements (i.e., statements connecting values and goals) are tailored to player-selected values to enhance relatedness. Creating implementation intentions to overcome anticipated problems enhances the likelihood of successful goal attainment, and thereby competence and self-efficacy. Players are inoculated against potential threats to goal attainment (i.e., self-doubts; peer influence) in a tongue-in-cheek segment called “good guy/bad guy” (Table 2).

Table 2.

Goal-Setting Procedures: Sequence of Activities and Theory Derivation^a

Player reports gender, height, weight, birth date, age (tailored goals—ELM) Unseen by player: Computer algorithms estimate appropriate kcal/day and corresponding recommended numbers of servings of foods in food groups by meals and a snack; based on information entered as part of initial video game setup (tailored goals—ELM) Player reports current meal/snack-specific behaviors through a series of interactive questions embedded in game play (current behavior—SCT; tailoring—ELM) Unseen by player: Tailor goal options offered to child by determining differences between height- and weight-specified servings and reported servings consumed (personal relevance—ELM; choice—SDT) Player selects (or confirms in later episodes) most important value (choice; relatedness—SDT; personal relevance—ELM) Player selects most important reason (related to selected value) for doing behavior (choice, relatedness—SDT; personal relevance—ELM) Player selects specific behavior and meal goal from tailored menu list of up to five choices (choice—SDT; goal formation—SCT) Player selects 1/2/3 days in next week when goal will be accomplished (progressively more difficult) (choice—SDT; goal formation—SCT) Player selects specific foods (F, V, W) to increase on each day (from menu tailored to responses above) (personal relevance—ELM; choice—SDT; goal formation, preference—SCT) Player views bad guy temptation to not meet goal (threat—BIT) Player receives good guy-tailored reason-related retort (refutational preemption—BIT) Player selects most likely barrier to achieving their goal—implementation intention (problem identification—SCT) Player selects most likely to be effective solution for overcoming that barrier—implementation intention (solution identification—SCT) Player congratulated! (positive outcome expectancy—SCT)

^aSCT, social cognitive theory; SDT, self determination theory; ELM, elaboration likelihood model; BIT, behavioral inoculation theory.

Goal Review Component

Players engage in goal review in episodes 2–9, wherein they report whether the goals set in the previous episode were attained. For an attained goal, players receive a reinforcing statement, and player-selected values and supporting reasons are reinforced (to enhance relatedness). For a goal not attained, players are routed to a problem-solving routine where they identify the problem that kept them from attaining their goal, select a plan for overcoming it, and decide whether to retry the goal. In the last episode (episode 9), a message related to personal goal attainment across all episodes is delivered to increase that the likelihood behavior change will be maintained over time (i.e., behavioral maintenance).⁵⁰

Sample

Ten to eleven 9- to 11-year-old youth alpha tested early versions of two episodes of each video game. There was variability in testers across episodes. Youth were recruited from the USDA/Agricultural Research Service (ARS) Children's Nutrition Research Center research participant database. All alpha testers provided written informed child assent and parental consent. By design, prior exposure to video games was highly variable, ranging from experienced video gamers to those with little or no video game-play experience. The alpha-testing protocol was approved by the Institutional Review Board at Baylor College of Medicine.

Alpha-Testing Procedure

Alpha testing, a type of qualitative research, was conducted with members of the target audience to identify potential problems with the video games (e.g., technical, usability, comprehension) that may affect playability. During several alpha-testing sessions, members of the video game programming staff quietly observed game play to obtain a first-hand view of problems encountered. This procedure, rather than video-taped sessions with blind review, was selected because the development team wanted to observe how alpha testers played the game at each step, help them out if they ran into difficulties so that game play would not stop, and then ask detailed and specific questions about the game-play experience. This would not have been possible using a video-taped procedure with blind review. During testing, player suggestions for changes were also obtained. Each episode was tested separately in one-on-one sessions, with testing sessions lasting ≤2 hours each. All alpha tests were conducted by one of two staff members trained in alpha-testing procedures, following a structured script (Table 3). Alpha testers played one complete episode of the video game without interruption, with game play observed by staff (approximately 1 hour), followed by a scripted interview (approximately 1 hour). When questions involved rating scales, testers were shown an index card portraying the scale visually. Players then stated their response verbally.

Table 3.

Sample Alpha-Test Questions: Goal Setting

Observation
Rate the amount of attention the player gave to selecting goals from 1 to 3: 1 = paid no attention   2 = paid a little attention   3 = paid close attention
Did the player commit to trying the goals that they chose (commitment screen)? □ YES □ NO
Record any technical problems: □ If the child stopped playing the game and came back, the game started at the wrong place when they returned □ Broken link (game would not accept child's response to a question or statement) □ Clicking a button on the controller did not do what it was supposed to do □ Missing data (something that was supposed to be included was missing) □ Program froze □ Other; explain:
Record any relevant details:
Other comments:

Interview
Remind them which value they chose:
Show them index card with rating scale on it.
For this question, use this scale: 1 is not important, 2 is a little important, and 3 is very important.
How important, if at all, is this value to you?
Probe to determine underlying reasons for their answer.
Remind them which value reason they chose:
Show them index card with rating scale on it.
For this question, use this scale: 1 is no sense, 2 is a little sense, and 3 is a lot of sense.
How much sense, if any, did this reason make to you for eating fruit and vegetables and drinking water?
Probe to determine underlying reasons for their answer.
Were the goals for you to do or for a character in the game to do?
Probe to determine underlying reasons for their answer.
FOLLOW-UP QUESTION if they thought the goals were for a character in the game:The goals are actually for you to do in real life, not for a character in the game. How could we help the player understand that?
Remind them which problem they chose:
Show them index card with rating scale on it.
For this question, use this scale: 1 is not likely, 2 is a little likely, and 3 is very likely.
How likely would you be to have this problem if you try to (insert the one that they chose) [eat more fruit/ eat more vegetables/ drink more water]?
Probe to determine underlying reasons for their answer.
Remind them which solution they chose:
Show them index card with rating scale on it.
For this question, use this scale: 1 is not likely, 2 is a little likely, and 3 is very likely.
How likely would you be to try this solution if you had a problem (insert the one that they chose) [eating more fruit/ eating more vegetables/ drinking more water]?
Probe to determine underlying reasons for their answer.
Show them index card with rating scale on it.
For this question, use this scale: 1 is not clear at all, 2 is a little clear, and 3 is very clear.
In this segment, you set two goals. On a scale of 1 to 3, how clear was it that that you could still meet one goal if you did not meet the other?
Probe to determine underlying reasons for their answer.

Notes:

Italicized statements are directions to the interviewer.

Bolded statements are read by the interviewer to the alpha tester.

This article reports key alpha-testing results from tailoring questionnaires, goal setting, and goal review components of early versions of episodes 2 and 3 of Diab (diet) and episodes 4 (diet) and 5 (physical inactivity) of Nanoswarm.

Results

Questionnaires

Providing answers to questions used to tailor goal setting and review took alpha testers on average 1 to 7 minutes, depending on the number of questions asked and the session being tested (i.e., session 2, 3 minutes, 21 questions; session 3, 2 minutes, 4 questions; session 4, 7 minutes, 33 questions; session 5, 1 minute, 10 questions). Alpha testers generally understood they were to provide responses about themselves rather than about the fictitious video game characters. However, some reported giving “pretend” answers, or answering questions for a game character. To ensure that future players understand they need to provide answers about themselves, an alpha tester suggested simply telling players they were supposed to answer questions about themselves.

While few technical problems were encountered, several key usability and comprehension issues were detected. For example, there were several reports of not understanding how to start the user profile, enter responses, or correct inaccurate responses. Some alpha testers reported not knowing the requested information (i.e., height, weight). In the first episode tested, no alpha testers understood the meaning of “portion.” Useful suggestions were to add a way to modify incorrect responses, allow players to provide more precise responses (i.e., talking on the phone 10, rather than 30, minutes), and to provide a more detailed explanation of portion size.

Goal Setting

Observational data revealed it took alpha testers on average 2 to 4 minutes to complete the goal-setting episodes. Few technical difficulties were encountered. Key comprehension difficulties were related to two reports of not understanding that a “nonschool day” referred to Saturday and Sunday and a report of confusion over Dr. Phen, a doctor, being a “bad guy” in the good guy/bad guy segment of Nanoswarm. Observers rated alpha testers as engaged (mean score 2.9; 3-point scale) and paying attention when selecting values and corresponding reasons (mean score 2.9; 3-point scale), goals (mean score 2.9; 3-point scale), and answering problem-solving questions (mean score 2.8; 3-point scale).

Interview data supported observational data. Alpha testers reported that goal setting was a little easy to complete (mean score 2.1; 5-point scale ranging from “very easy” to “very hard”) and that it was a little fun (mean score 2.0; 3-point scale). They reported selecting personally important values (mean score 3.0; 3-point scale) and said value-supporting reason statements made sense (mean score 2.8; 3-point scale). Alpha testers said if playing the game, they would try to meet the goals they set (mean score 2.7 on a 3-point scale). When creating an implementation intention to overcome an anticipated problem, alpha testers reported it was a little likely (mean score 2.1; 3-point scale) they would encounter the selected problem when trying to meet their goal because they did not typically have problems eating fruit and vegetables. Alternatively, they reported that if they did encounter the problem, it was likely they would try the solution they selected (mean score 2.6; 3-point scale). Alpha testers also reported it was only a little clear they could meet some, but not all, of the goals set in the game (mean score 2.2; 3-point scale). They generally understood the goals they set were for themselves, not a game character, although there was some confusion. A suggestion was to use the player's name during goal setting to minimize confusion. Alpha testers gave suggestions to enhance goal setting, the most frequent of which was to allow players to earn points.

Goal Review

To provide a more authentic representation of goal review, alpha testers participated in goal review during episodes 3 and 5 only, based on goals set in episodes 2 and 4. Observational data revealed it took an average of 2 minutes to complete goal review. Few technical problems were encountered. The most significant usability problem was a report of not being able to locate a specific problem on the problem list. The major comprehension problem was several reports of alpha testers not understanding how to select goals that were met. Alpha testers were observed to be engaged during goal review (mean score 3.0; 3-point scale). They appeared to pay attention to problem solving (mean score 2.7; 3 point scale) and to answering problem-solving questions related to goals they were planning to retry (mean score 3.0; 3-point scale).

During the interview, most alpha testers reported understanding the purpose of goal review in Diab; however, fewer reported understanding its purpose in Nanoswarm. Alpha testers thought goal review was a little easy (mean score 2.1; 5-point scale ranging from “very easy” to “very hard”) and a little fun (mean score 2.0; 3-point scale). They reported that the length was about right (mean score 3.1; 5-point scale ranging from “too short” to “too long”) and that it was helpful in solving problems they might encounter when attempting to achieve a goal (mean score 2.5; 3-point scale). Most (n = 7) reported that the difficulty level was appropriate and should not be changed.

Discussion

This research emphasizes the importance of involving the target audience at strategic times in the video game development process. Based on this initial round of alpha testing, changes were made to address issues related to functionality, usability, and comprehension. For example, personal data questionnaires were reconfigured to be less repetitive. They were also modified to make them more interactive and less “form like.” In Nanoswarm, goal setting was reconfigured to be a minigame to make it more fun and less didactic. Dr. Phen was removed from the good guy/bad guy segment because of confusion caused by a doctor being presented as a “bad guy.” In addition, the information obtained from initial alpha testing was used to guide and inform the development of the remaining episodes of the video games.

These results reinforce the importance of assessing functionality, usability, and comprehension prior to completion of a video game. Problems such as these may decrease exposure and/or attention to the behavior change components, thereby ultimately decreasing behavior change. Additionally, identifying technical, usability, and/or comprehension problems after production can be costly. Therefore, systematic alpha testing early in the development cycle is an essential component of video game development.

The video games were developed using a multicomponent theoretical framework comprised of SCT, SDT, ELM, and BIT. The theoretical framework guided game design decisions and enabled the developers to maintain a tight focus on procedures and techniques likely to facilitate behavior change. The true test of the value of this approach, however, will be the effect of the video games on youth behavior.

Next Steps

Once both video games have been finalized, the next step will be to test them in a randomized control trial using a two-group (treatment, control) design. The treatment group will play Diab immediately followed by Nanoswarm, whereas the control group will receive a package of computer games and Web sites that provide general diet and physical activity information. Data on diet (fruit, vegetable, water), activity (physical activity, sedentary behavior), and psychosocial mediators will be collected at four data collection points (baseline, after Diab, after Nanoswarm, and 2 months later). Changes in diet and activity will be the true tests of the video games' effectiveness. Mediating variable analyses will determine whether the changes occurred through the hypothesized theoretical pathways, as well as the importance of self-regulatory skills to the behavior change process.

Strengths and Limitations

The strength of this research is that it was conducted with trained staff, following a structured script. The primary limitations are the small sample and the limited number of video game episodes tested. Small samples, however, are common in qualitative research, such as alpha testing, because of the intense nature of the experience and the need for rapid turnaround of evaluative information.

Conclusion

Video games, enhanced by behavior-change technology and motivating story lines, offer promise for promoting diet and physical activity change for diabetes and obesity prevention in youth. Alpha testing offers an opportunity to identify problems that may limit functionality, usability, and comprehension. Because these problems could potentially have a negative effect on enjoyment, knowledge, and skill development, it is critical to detect them prior to final production. Therefore, alpha testing is a useful and critical component of video game development.

Abbreviations

ARS

Agricultural Research Service

BIT

behavioral inoculation theory

ELM

elaboration likelihood model

SCT

social cognitive theory

SDT

self determination theory

USDA

U.S. Department of Agriculture