Assessing Feedback in a Mobile Videogame

Abstract

Background: Player feedback is an important part of serious games, although there is no consensus regarding its delivery or optimal content. “Mommio” is a serious game designed to help mothers motivate their preschoolers to eat vegetables. The purpose of this study was to assess optimal format and content of player feedback for use in “Mommio.”

Materials and Methods: The current study posed 36 potential “Mommio” gameplay feedback statements to 20 mothers using a Web survey and interview. Mothers were asked about the meaning and helpfulness of each feedback statement.

Results: Several themes emerged upon thematic analysis, including identifying an effective alternative in the case of corrective feedback, avoiding vague wording, using succinct and correct grammar, avoiding provocation of guilt, and clearly identifying why players' game choice was correct or incorrect.

Conclusions: Guidelines are proposed for future feedback statements.

Introduction

The functionality of providing feedback has been established for enhancing learning and behavior change.¹ Feedback was one of the most common behavior change techniques used in physical activity behavior change applications² and in activity monitoring software that provide specific feedback of performance in regard to past accomplishments, norms of similar groups, and precise goals.³ In one application, feedback involved brief comments on the quality of the recorded behavior and suggestions for change.⁴

Although feedback is generally accepted as a part of videogame play,⁵ the form of feedback best suited for any particular game style, learning goal, or target audience is unclear. In particular, there is little guidance on corrective feedback that is both affirming, so as to not damage the player's self-efficacy,⁶ yet detailed enough to impact learning⁷ in the context of serious mobile games.

“Mommio”^8–10 is a three-dimensional, open-world, first-person adventure videogame designed to teach mothers of preschoolers effective vegetable parenting practices.¹¹ Designed for smartphones and tablets, the game character in “Mommio” is a child who hates veggies. The player's goal is to get her child game character to eat vegetables while also encouraging the child's long-term enjoyment of vegetables. Players “talk” to the child and take actions to encourage the child to eat. Player experience and knowledge gained during gameplay should translate into real-world application through player goal setting at the end of a quest.

Feedback in “Mommio” for a player's wrong choices (i.e., selection of ineffective vegetable parenting practices) was tested in “Oreo” format during early development (i.e., sandwiching a negative feedback statement between two positive feedback items).¹² Although Oreo feedback was received positively, results were mixed by socioeconomic groups, and the statements were too lengthy for smartphone use. This study interviewed mothers of 3–5-year-old children about new condensed vegetable parenting feedback statements focusing on helpfulness, clarity, and self-efficacy support. The purpose of this study was to assess a working collection of feedback statements, as well as to establish guidelines for additional feedback in “Mommio.”

Materials and Methods

Sample

Eligible participants were English-speaking mothers of 3–5-year-old children who reported having difficulty getting their child to eat vegetables. Mothers who reported little or no difficulty with vegetables, as well as mothers who did not live with their child most of the time, were excluded. Participants were recruited from the Children's Nutrition Research Center's volunteer list and through flyers posted throughout the Texas Medical Center. The sample (n = 20) was composed of women mostly in their 30s, who were generally educated with at least some college, and most often white or Hispanic (Table 1). Approval was obtained from Baylor College of Medicine's Institutional Review Board prior to recruitment. Participants provided informed consent.

Table 1.

Sample Demographics (n = 20)

	Frequency	Percentage
Child's gender
Boy	14	70
Girl	6	30
Child's age (years)
3	5	25
4	8	40
5	7	35
Mother's age (years)
20–29	2	10
30–39	15	75
40–49	3	15
Mother's ethnicity
White	7	35
Black	3	15
Hispanic	9	45
Asian	1	5
Highest education completed
Technical school	1	5
Some college	4	20
College graduate	4	20
Postgraduate study	11	55
Annual household income
Less than $30,000	5	25
$30,000–$60,000	3	15
Over $60,000	12	60
Employed
Yes	14	70
No	6	30
Marital status
Married or living with a significant other	14	70
Single, never married	4	20
Divorced, separated, or widowed	2	10

Web survey and interview

A professional writer adapted 36 possible feedback statements that corresponded to 12 vegetable parenting practice categories (four effective, seven ineffective, and one neutral), so that three feedback statements represented each category. These were compiled into 36 “what” statements identifying the selected parenting practice's category, as well as “why” statements describing why the player's game choice was either an effective or ineffective vegetable parenting practice.¹³ Statements were intended to apply to all vegetable parenting statements within a category. Random assignment combined these “what” and “why” statements and interjected a softener that either praised or warned against the vegetable parenting practice (see example statements in Table 2).

Table 2.

Structure and Revisions for Game-Based Feedback Statements Regarding Vegetable Parenting Practices

	Tested feedback			Revised feedback
Vegetable parenting practice	Softener	“What” statement	“Why” statement	Statement	Main revisions
C1. Guilt	Oh no!	This laid on the guilt.	You want [Child's Name] to eat veggies because it will make [him/her] healthy, not because it will make you happy.	Oh no! This might make [Child's Name] feel guilty. You want [Child's Name] to eat veggies because it will make [him/her] healthy, not because it will make you happy. Instead, be a role model and show [CHILD'S NAME] that you enjoy veggies.	Effective alternative added, mommy guilt removed (“might” added)
C2. Anger	Oops!	You lost your temper and got angry.	If you get angry, [Child's Name] might get angry right back. Then the meal is sure to be a disaster.	Uh oh! You lost your temper and got angry. Using this tone might cause anger in [CHILD'S NAME], raising tension for everyone. It might take several attempts; try again at a later meal.	Effective alternative added, inevitability removed, explanation clarified
C3. Force	Oops!	This was demanding.	Take some deep breaths and try again when you are both calmer.	Oops! This was demanding. Statements like these can cause [CHILD's NAME] to not like veggies. Take some deep breaths and try again in a few minutes when you are both calmer.	Effective alternative clarified, explanation added
C4. Threat	Oh no!	You used a threat to get [Child's Name] to eat.	If you serve the veggies with threats, you'll leave a bad taste in his/her mouth that lasts a long time.	Oh no! You used a threat to get [Child's Name] to eat. Linking food to punishment can have lasting, damaging effects on a child. If [NAME] is upset, acknowledge it and then calmly move on to a positive topic.	Effective alternative added, explanation clarified, colloquialism removed
C5. Begging	Uh oh.	You begged [Child's Name] to eat.	If you're begging, you already lost this round. Now [Child's Name] has learned the magic power of “no.”	Uh oh. You begged [Child's Name] to eat. If you're begging, you have lost control of the situation. You can avoid this problem by approaching meals as an opportunity to share time together.	Effective alternative added, casual language removed
C6. Pressure	Oh no!	This put pressure on [Child's Name] to eat.	Try to humor [Child's Name] into eating instead of insisting.	Oh no! This pressured [CHILD'S NAME] to eat. This can cause [CHILD'S NAME] to eat because [he/she] feels [he/she] must and not because [he/she] is learning to like veggies. You may have more luck if you talk to [CHILD'S NAME] about why veggies are healthy.	Effective alternative added, explanation added, casual and vague language removed
C7. Bribe	Uh oh.	This was a bribe.	Eating veggies should be its own reward. The reward is enjoyable flavors and good health.	Uh oh. This was a bribe. One bribe will lead to another, and then [CHILD's NAME] will eat for the bribe instead of learning to like veggies. Next time, tell [CHILD'S NAME] that veggies will give [him/her] energy to play.	Effective alternative added, explanation modified
C8. Neutral		This combination of facial expression, voice tone and statement neither helped nor hurt you.	The statement was not as supportive as it could have been, but you did choose an appropriate voice tone and facial expression.	Not bad! You used a great voice and face, but you could have chosen a better statement. Keep up the tone, but next time try role modeling enjoying veggies.	Effective alternative added, softener added, succinct language
E1. Involvement	Awesome!	This turned mealtime into a group activity.	It also transformed eating veggies from a necessary thing to a fun group activity.	Awesome! You turned mealtime into a group activity. It also transformed eating veggies from a chore into something fun to do together.	Vague language removed
E2. Support	Excellent!	You showed [Child's Name] you care about [him/her] when you said this.	Listening is a form of caring. So is serving veggies. Doing them together helps [Child's Name] understand that.	Excellent! You showed [Child's Name] you care about [him/her] when you said this. Listening is a form of caring, and this helps [Child's Name] understand that you care about [him/her] during meals.	Grammar corrected, succinct language
E3. Encouragement	Excellent!	You asked [Child's Name] to eat veggies because they are appealing.	Praising good choices is a great way to be sure [Child's Name] makes those choices again.	Excellent! You asked [Child's Name] to eat veggies because they are appealing. Praising good choices is a great way to be sure [Child's Name] makes those choices again.	No change
E4. Reason	Good job.	This explained that veggies are healthy in an understandable way.	Now [Child's Name] will understand that food that is good for [him/her] can also taste good.	Good job. This explained that veggies are healthy in an understandable way. Now [Child's Name] might understand that it is important to try food that is good for [him/her].	Broadened to match game choice

Participants completed an online survey, in which they were given a short introduction to “Mommio” and were asked to imagine playing a mealtime scenario. Each feedback statement was presented with a randomly assigned game choice statement, tone, and face. Tone and face were added because in the game, a parent can choose a voice tone (gentle, firm, or harsh) and facial expression (concerned, happy, neutral, or angry) to use to accompany her selected statement. For example, the statement “Try the veggie and you'll get a new toy,” accompanied by a concerned face and gentle voice, was presented to elicit a feedback statement about bribes. Although a harsh face or angry tone would result in an ineffective choice regardless of statement content, this compilation was excluded from the study, which focused on content rather than tone and face. Thus, no statements were paired with tone and face choices that would have changed the feedback for its content. For each feedback statement, participants were asked what they thought the feedback meant, if they thought it could be said better, and if they found the feedback helpful.

Analysis

Two researchers trained in qualitative methods analyzed responses to the Web survey. Answers agreed upon by both researchers as vague or unclear were noted. All participants then completed a phone interview with an interviewer trained in qualitative methods. The interview included further discussion of noted items, which included repeating the survey question of note and the participant's answer and then asking for elaboration. Interviews were transcribed and imported into NVivo software (version 10.0, 2012; QSR International, Doncaster, VIC, Australia). Transcripts were coded by question using thematic analysis techniques.¹⁴

Results

Reactions to positive feedback

Many feedback statements were positive, in that they praised gameplay that reflected effective vegetable parenting.¹³ Reactions to these positive feedback statements had two distinct patterns.

Level of critique

Positive feedback statements were generally liked better and critiqued less than corrective feedback statements. Of 12 positive feedback statements (categories E1–E4 in Table 2), all mothers agreed that feedback could not be “said better” in three instances. Two additional feedback statements had notations that they could be said better by a small number of participants. However, when discussed, these participants addressed changing the game choice that prompted the feedback rather than feedback content. Five feedback statements thus required no alterations as a result of this formative work (see Table 2, E3). This occurred exclusively for positive feedback.

Matching to game choice

The most frequent critique of positive feedback was not the isolated content of the feedback statement, but rather its pairing with the game choice statement. In four instances, at least one participant mentioned that the feedback was “disconnected” or “did not match” the game choice. For example, in one instance, taste was mentioned in the feedback but not in the game choice (see Table 2, E4). Four positive feedback statements were amended as to universally match game choices within their designated vegetable parenting practice categories.

Reactions to corrective feedback

Much of the feedback provided was corrective (i.e., discussed a player's gameplay choice that fell into one of seven ineffective vegetable parenting practice [C1–C7 in Table 2] categories or was deemed neutral with room for improvement [C8 in Table 2]). A variety of themes, outlined below, emerged from their discussion.

Effective alternatives

The most common reaction to corrective feedback was the desire for a suggested effective alternative parenting strategy. Six participants mentioned this in a general way, for example, stating, “If it's not going to work, then give mom an alternative for what would work” and “I think for me it would help me with alternatives that instead of just saying what I did wrong, give me alternatives. And that's for all the statements.” Of 24 corrective feedback statements, 15 were specifically described as unhelpful because they did not provide an effective alternative strategy (see Table 2, C1–C8).

Guilt and inevitability

Although corrective feedback inherently pointed out an ineffective gameplay, parents reported they interpreted this as guilt inducing or “talking down” to the player. Two mothers mentioned that all parenting feedback statements should be encouraging because “in the whole mommy world, there's so much guilt and pressure as it is. Moms don't need additional guilt.” Two feedback statements were specifically noted to contribute to this “mommy guilt” (see Table 2, C1).

Other statements were labeled with constructs similar to guilt. Two feedback statements produced a feeling in some mothers that failure was inevitable, given the feedback's use of definitive clauses such as “he will avoid [vegetables].” Mothers who felt this way suggested using “might” or “may” instead in the inevitable “will.” Three additional feedback statements were perceived as threatening or too harsh. For example, one mother thought that the suggestion that one statement said with anger would turn the meal into a disaster was too harsh and “presented the worst-case scenario,” deeming it unhelpful (see Table 2, C2).

Explanation

When participants saw a game choice labeled as incorrect, they not only wanted to know what was incorrect, but also why it was ineffective. Small explanations, such as “it's nice that your child wants to please you, but that's not why you want him to like veggies,” were deemed as sufficient (one participant said, “it tells me what is wrong and why”). Feedback statements lacking simple explanations were identified (see Table 2, C2–C4, C6, C7). Participants generally requested explanations be added to increase helpfulness.

Writing style

A few themes, detailed below, occurred universally throughout reactions to all types of feedback. These themes involved the structure of the feedback as grammatically and stylistically presented rather than its content.

Succinct and clear writing

Clarity and conciseness were themes throughout responses. Use of words such as “too,” “some,” and “thing” were labeled as vague and suggested to be either replaced for clarity or discarded (see Table 2, C6, E1). Parents also said that “wordiness does not help.” Longer statements were perceived as too long or containing jargon (see Table 2, C8, E2). Shorter, simpler feedback statements were praised as “clear to the point.” Statements deemed largely helpful were also thought to be concise.

Grammar

Original feedback was intentionally written in a casual and welcoming style, with correct grammar traded for common spoken language structure, such as “Probably seemed like a good idea” and “So is serving veggies” (see Table 2, E2). Where incorrect grammar existed, mothers voiced their opposition to its use. This was universally true but was reported as more important for constructive feedback, as one mother stated, “if someone's giving me criticism or negative feedback, I prefer it to be in proper English.”

Use of casual voice

Every instance of colloquialism or idiom was noted as distracting, vague, or inconsistent by at least one participant. Examples of such included “don't hang your hat on this,” “if you serve veggies with threats, you'll leave a bad taste in his mouth,” and “Now he understands you're a mealtime friend” (see Table 2, C4). Other statements were noted to be “too casual” and generally inconsistent with the authoritative voice of more helpful feedback statements (see Table 2, C5, C6). In all instances, this type of voice was met with requests to be either replaced with a sentence that expressed the concept plainly or removed completely.

Discussion

Corrective feedback inferring parental guilt or the inevitability of failure was criticized. This is similar to findings in the general feedback literature supporting self-efficacy as key to providing successful feedback,⁶ as well as that failure to do so can alienate target users.

Although the tested statements were not in “Oreo” format¹² because of the limited space of a smartphone user interface, participant requests that corrective feedback also contain positive statements support earlier findings.¹² Positivity, even in the form of an effective alternative that could have been done in place of a less desirable game choice, helps with acceptance of negative feedback.

Although learning was not tested in the current study, mothers were asked whether each feedback statement would be helpful to them. This often inspired personal reflection, and statements reported as well liked were also reported as personally helpful. This supports the notion that once amended to fit guidelines outlined in Table 3, the feedback could support learning and consequently behavior change.

Table 3.

Feedback Guidelines for Use in a Serious Game Regarding Vegetable Parenting Practices

Feedback should include	Feedback should exclude
Explanation as to why a game choice is effective or ineffective	Dramatic “worst-case-scenario” effects of ineffective strategies
Effective alternative strategies accompanying constructive feedback	Open-ended or rhetorical questions
Correct grammar	Colloquialisms or idioms
Self-affirming language such as “good job” and “excellent”	Language that blames the player
Words of possibility such as “may” and “could”	Words of inevitability such as “will” and “must”
Concise, clear wording	Vague, unquantifiable terms like “too” and “some”
Universal language that matches advice for all game actions in its category	Specifics that do not match the game choice for which it is prompted

Based on these findings, guidelines were generated for both amending the feedback statements studied here and developing similar feedback in the future (Table 3).

This study has limitations. A small sample was asked to imagine playing a game, although no game was actually played. If the sample were larger or if participants had been asked to play the actual “Mommio” game (currently under development) and react to feedback about actual game choices, results may have been different. For example, a population differing from or larger than the sample of this study may enjoy colloquialisms or appreciate robust explanations within feedback, perhaps because of unfamiliarity of the subject matter or distaste for authoritarian curtness. Although these results are intriguing, the suggested guidelines should be cautiously applied to similar feedback statements in consideration of these limitations. Research that is more naturalistic in inclusion of gameplay or diverse in its game content and sample would expand this study's limited beginning.

Acknowledgments

This work was funded by grant HD075521from the National Institute of Child Health and Human Development. This work is also a publication of the U.S. Department of Agriculture (USDA/ARS) Children's Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, TX, and had been funded in part with federal funds from the USDA/ARS under Cooperative Agreement Number 58-3092-5-001.

Author Disclosure Statement

No competing financial interests exist.