Incorporating Prototyping and Iteration Into Intervention Development: A Case Study of a Dining Hall-Based Intervention

Abstract

Background

Previous research from the fields of computer science and engineering highlight the importance of an iterative design process (IDP) to create more creative and effective solutions.

Objective

This study describes IDP as a new method for developing health behavior interventions and evaluates the effectiveness of a dining hall-based intervention developed using IDP on college students’ eating behavior and values.

Participants

458 students (52.6% Female, age M=19.6±1.5).

Methods

The intervention was developed via an IDP parallel-process. A cluster-randomized controlled study compared differences in eating behavior among student in 4 university dining halls (2 intervention, 2 control).

Results

The final intervention was a multi-component, point-of-selection marketing campaign. Students in the intervention dining halls consumed significantly less junk food and high-fat meat and increased their perceived importance of eating a healthful diet relative to the control group.

Conclusion

IDP may be valuable for the development of behavior change interventions.

INTRODUCTION

Improving dietary behavior for disease prevention is challenging and typical approaches show limited success ¹. Current intervention development practices usually involve a mixture of qualitative research (e.g., focus groups, semi-structured interviews) ^2,3 and theory (e.g., social cognitive theory and the transtheoretical model are popular theoretical models) ⁴ to develop full intervention “packages” (i.e., interventions with multiple behavioral components). Intervention “packages” are then traditionally tested in a randomized controlled trial (RCT) ⁵. This method of intervention development and testing is limited because it is difficult without conducting post hoc/secondary analyses to determine (a) which components are the “active” ingredients and (b) if each intervention component is functioning as designed.

Previous research from human computer interactions and other fields of computer science and engineering have highlighted the importance of user centered design for which a key part is utilizing an iterative design process (IDP) to create tangible prototypes and multiple iterations that test concepts and ideas, early in the process ⁶. This iterative approach informs future design with the intention of reducing risks by helping designers discover previously unknown attributes, issues and opportunities that may not have been conceived of a priori ⁶. Research shows that IDP can result in more creative and effective solutions ^6–8. At present, little if any research has been conducted within the field of behavioral science exploring the utility of IDP and, in particular, the utility of simple prototypes for optimizing health behavior interventions.

The aims of this paper are to (a) describe the IDP process as illustrated by an intervention developed using this approach in a dining hall setting among university students; and (b) to report results from an initial trial testing the efficacy of the intervention developed. The intervention was designed to be implemented before and during the final examination period, a period of high stress for students. Research shows a tendency for university students to increase unhealthy dietary intake during final examinations ^9,10 and other times of stress ^11–16. Therefore, it was hypothesized that students in the control group would consume more unhealthy foods, whereas students in the intervention group would maintain or improve their dietary intake patterns.

METHODS

Intervention Development Using the Iterative Design Process

The 4-week, short-term, multi-component, point of selection intervention was developed via user experience design which includes an iterative design process (IDP) ⁸. User experience design (also called user centered design) ¹⁷ places the people who will ultimately be impacted by the intervention, (i.e., the “user”) at the center of the intervention design process. A key a priori focus for the IDP work was an explicit emphasis for reducing meat consumption based on advantages both for health and environmental sustainability, a combination of motives that fits with our previous research ¹⁸. Our IDP utilized rapid and informal forms of qualitative research methods to gain an understanding of the “needs” and “point of view” of the potential “users” of the intervention as quickly as possible to foster the development of prototypes ^19,20 Within our study, insights were gained from the user via rapidly conducted qualitative methods including ethnographic interviews and observation. These “needs finding” tasks were conducted in the participant’s natural environment (i.e. dining halls) and were used to empathize with and understand the students’ activities within the context of their social cultural world ^20–22. Multiple teams observed students’ behavior and had informal conversations with students to inquire about food choices in the dining halls. The information from this informal needs finding and ethnography approach was processed using post-it notes to aggregate information and themes until potential insights merged (see Stanford D-School Bootleg Bootcamp) ¹⁹ and in particular the “Story Share and Capture”, “Saturate and Group” and other idea organization methods in the document for specific references to techniques used for organizing the qualitative data. These potential insights were then further processed and turned into a “point of view” defined as a “…reframing of a design challenge into an actionable problem statement” (pg 21, Stanford D-School Bootleg Bootcamp). These points of view were meant to frame the problem and guide intervention development efforts. These preliminary steps of user experience design provide a rapid method for intervention development similar to that provided by more rigorous qualitative methods and theory.

Following this qualitative needs finding and point of view development work, we then utilized the IDP by cyclically working through (a) idea generation, (b) developing prototypes, (c) receiving user feedback on the prototypes, and (d) repeating the process until each component is believed to be functioning as designed based on user feedback. Ideas were generated to explore multiple solutions for users that fit into the points of view. Then, these ideas were developed into sketches ²³ and prototypes (e.g., functional examples of potential intervention components) that were shared with potential users for feedback, reactions, and preliminary conceptual testing (e.g., if a sign is thought to increase one’s awareness of eating healthy, a sign could be put up in the dining hall and then individuals who walked by it would be asked if they noticed it and what it meant to them to see if it elicited the appropriate response). Specifically, once these prototypes were developed, they were brought to the dining halls and multiple teams observed students’ behavior and had informal conversations with students to inquire about the reaction to the multiple prototypes. Based on the results of this “user testing,” new sketches and prototypes, or possibly even earlier parts in the process (e.g., “needs finding” work) were revisited until each intervention component appeared to elicit the appropriate behavioral responses from potential users.

Efficacy Trial Study Design

Following intervention development, a cluster-randomized study with a repeated cross-sectional assessment strategy was used to compare differences in eating behavior among students in four university dining halls (two intervention, two control), before and after a 4-week intervention period; the post intervention data collection period coincided with final exams week in the spring quarter of 2011. Students purchase a prepaid meal plan and obtain foods from various food stations, therefore the costs of food is not a factor in this study.

Participants and Dining Halls

The university had a total undergraduate population of 3,554 in 2010 with 21.6% of undergraduates from self-reported minority groups (African American, Hispanic, Native American, Asian American, and/or Pacific Islander). All students living in undergraduate University resident halls are required to enroll in a dining hall meal plan. There are 9 dining halls on campus, and among those 4 dining halls were selected for the current study. We targeted dining halls that had the largest proportion of freshman students because research suggests that the freshman year of college results in significant differences to individual’s eating patterns and is a critical period of risk of weight gain among young-adults ^24,25. Each of the four dining halls that were selected serves ~500–600 people at any given lunch/dinner period.

This research was approved by the university’s Institutional Review Board. Data were collected by a team of researchers onsite during the lunch and dinner hours over the course of 7 days in February 2011 (pre-survey), followed by a 4-week intervention and again over the course of 7 days in May 2011 (post-survey). The projected target sample was 100 surveys in each of the four dining halls, at each of the two time points - pre and post intervention. Every person who entered the dining hall was eligible to participate both during the pre- and post-survey phases; actual participation was determined by the practical logistics of a limited number of surveyors approaching as many students as they could in a limited time frame; i.e., convenience sampling.

Both the pre and the post surveys were cross-sectional, but the nature of the data collection method (convenience sampling among all students eating in the dining halls) meant that some students would contribute both pre and post-surveys. We utilized a repeated cross-sectional assessment strategy primarily because our primary area of interest was dining hall-wide food consumption. To ensure individuals that were measured both pre and post did not create an unintentional bias in results, we conducted analyses both with and without these students who provided both pre-post data and found no significant differences. As such, all analyses are reported with an emphasis on the repeated cross-sectional approach as our primary level of analysis was on overall changes in eating patterns between the two dining halls.

Each dining hall was visited for a minimum of three and a maximum of four days. By the fourth day of data collection, most students who were interested in participating had been approached by the research team. Participants were required to be between the ages of 18 and 23, have a meal plan with the residence dining hall, and eat at the specified dining hall for at least 3 days/week to ensure they were adequately exposed to the intervention.

Survey Measures

Dietary intake was assessed with items adapted from the Harvard Food Frequency Questionnaire (FFQ) and scored as servings per week within food categories (vegetables, fruits, high-fat dairy, high-fat meats, and “junk food” defined as fast food and other processed foods and beverages such as sodas, candy, or high-fat desserts), which has been previously shown to be effective at detecting changes in eating habits among college students ¹⁸. This food frequency questionnaire assessed student’s self-reported intake of the targeted healthful foods provided in the dining halls at lunch and dinner every day. Students also rated the perceived importance of eating a healthful diet and engaging in physical activity using measures of satisfactory reliability and validity among college students (unpublished data).

Statistical Analyses

Descriptive statistics were used to describe demographic information. Mixed effect modeling was used to analyze group differences across time and the effect of the intervention on eating behavior and values (Singer & Willet, 2003). Two-level multilevel modeling was fitted to account for participants nested within dining halls. All models were adjusted for gender and age. In addition, differences between groups in the dining halls on key demographic data were tested both pre and post. Any variables that were shown to differ between groups were included as covariates to control for the impact of these differences to reduce the potential biases involved in the repeated cross-sectional assessment strategy used.

RESULTS

User-experience design

This study intended to specifically reduce meat consumption and improve overall healthful eating among students. From the ethnography and needs finding work, two points of view stood out. First, students felt that the dining hall food was overly-mass produced and little care or thought went into preparing their foods (i.e. Care Point of View). Second, students felt over-loaded with schoolwork and needed simple ways to know how to pick healthier foods. Specifically, students wanted to know what foods provided them with the energy to stay up late and study (i.e. Energy Point of View). Subsequently, multiple prototypes were ideated and created in parallel to address these two points of view.

Iterative Design Process

Care Point of View

From the Care Point of View, four preliminary prototypes were developed: (1) Students were given menus upon entry into the dining hall to help them decide their meal. (2) A “dim-sum” style vegetable cart was pushed throughout the dining hall. (3) Prepared balanced meals were placed on display at the dining hall’s entrance. (4) A Chef’s ‘Pic’ of the day that included a portrait of the chef and a plated vegetarian meal were placed on display at the front entrance of the dining hall (see Figure 1).

Figure 1.

Images of Prototypes to Represent the Care Point of View

The feedback from the initial prototyping for the Care Point of View indicated that students did not find the menus helpful and did not want to hold anything in their hands as they waited in line and chose their food. The students felt the dim-sum style vegetable carts made the food look less mass-produced. However, the students felt that they already put enough food on their plate and did not want to add anything else. Lastly, students were impressed with the plated balanced meals and the Chef’s ‘pic’ of the day and thought the prototypes were helpful in knowing how to put together a meal. The students also enjoyed knowing who was preparing their food. The plated meals reminded the students to eat their vegetables and encouraged them to try eating other foods in the dining hall.

This prototype was then reworked into a second stage of prototyping. Prototype II involved (a) a Chef’s “Pic” of the day; (b) plating of a complete vegetarian meal; and (c) informational placards on tables and food stations that highlighted vegetarian options. During this second stage of prototyping, students continued to praise knowing who was preparing the meals and stated that they were more likely to make healthy choices. Visual observations showed that students mimicked the prepared meals and there was chef-buy in. In fact, because the chefs stated that they knew their names and faces were going beside the vegetarian meal they put more effort into the vegetarian meals that was their pick of the day.

Energy Point of View

From the Energy Point of View, three preliminary prototypes were developed (a) An “energy” station was created that was filled with energy-sustaining foods; (b) A station was developed that showed students how to prepare a balanced meal; (c) Foods were labeled as “brain”, “mood”, or “energy” foods at point-of-selection and had complimentary table tents that described what made up a “brain”, “mood”, or “energy” food. The food label categories were based on recommendations from the Academy of Nutrition and Dietetics (eatright.org) and various scientific articles that describe the effect of nutrients and foods on the likelihood of enhancing memory and brain function ²⁶, decreasing depression ^27,28, and sustaining energy ²⁹ throughout the day.

The energy station was unsuccessful because students felt that they already put enough food on their plate and did not want to add anything else. The station that taught students to prepare healthy meals was unsuccessful because students found it difficult to find all of the foods in the dining hall. The students thought that the foods labeled as “brain”, “mood”, or “energy” foods were very intuitive. Students stated that this labeling system was more helpful than using nutrition statistics. Additionally, the labeling system was easy to implement and required minimal infrastructure changes.

Combined Point of View for Intervention

Subsequently, the point of views and final prototypes were combined. The final intervention was a marketing campaign implemented around finals entitled “Food for the Homestretch”, and included sample plates that were promoted by a staff “pic” of the day, healthy choice indicators (1” × 2” placards,), large signs (34” × 44”’ posters), table tents (8 ½” × 11”), flyers and colorful photographs with benefit-based (e.g., “brain”, “mood”, and “energy” foods) messages for promoting healthy food choices (i.e., more fruits, vegetables and fish, and less high-fat meat and processed/junk food) (see Figure 2–3). Benefit-based indicators were distributed throughout the dining halls on walls, at food stations, and at dining tables.

Figure 2.

Final Prototype - a poster describing the meaning of the healthy choice indicators around the dining hall

Figure 3.

Final Prototype - a plate with healthy choice, benefit-based indicators to promote healthy food choices in the dining hall

Efficacy Trial Study

At pre-intervention, 402 surveys were collected, of which 312 were complete (i.e. did not have missing data). At post-intervention, 286 surveys were collected, of which 213 were complete. The breakdown by intervention vs. control dining halls is presented in Table 2. The total sample was made up of 458 university students (pre- and post-surveys combined); 15% of the sample completed both the baseline and follow-up survey.

Table 2.

The effect of the intervention on health values across time

	Value of Eating a Healthful Diet		Value of Being Physically Active
	F Value	P	F Value	P
Intervention	2.34	0.13	4.10	0.04
Time	1.77	0.18	4.60	0.03
Intervention* Time	5.10	0.03	4.46	0.04
Gender	0.88	0.35	0.14	0.71
Age	0.79	0.38	0.08	0.78

Longitudinal mixed modeling was used to analyze the effect of the intervention on health values.

Demographic information for student participants is displayed in Table 1. There were significant differences in age and class across time (p<.0001). Therefore, because age and class are highly correlated (P<.01), only age was controlled for in all analyses.

Table 1.

Demographics, values, and outcome variables statistics, % unless otherwise indicated

Variables	Control Halls		Intervention Halls
Demographics	Pre	Post	Pre	Post
N	157	121	155	92
Female	54.7	47.5	51.7	57.3
Age (M±SD)	19.1±1.3	19.5±1.3	19.5±1.6	20.6±1.5
Race/Ethnicity
White	24.0	26.3	18.5	25.6
Latino/Hispanic	13.6	18.6	17.1	14.0
African American/Black	7.8	6.8	13.7	7.0
Asian/Pacific Islander	48.7	45.0	41.8	47.7
Other	5.8	3.4	8.9	5.8
Grade Level
Freshman	61.7	53.8	47.0	22.5
Sophomore	18.2	23.5	23.8	21.4
Junior	14.3	11.8	13.3	16.9
Senior	4.6	10.1	13.3	39.3
Other	1.3	0.8	2.7	----
Dining Hall Frequency (days)	6.4±1.2	6.3±1.4	6.8±1.4	6.7±1.3
Values variables (M±SD) a
Importance of eating a healthful diet	3.6±1.1	3.2±1.0	3.5±1.1	3.6±1.1
Importance of staying physically fit	3.7±1.1	3.2±1.0	3.7±1.2	3.6±1.1
Servings per week of dietary intake variables (M±SD) b
Fruit	15.4±5.6	14.8±5.1	15.4±5.6	15.2±6.3
Vegetables	23.9±8.3	22.7±7.9	23.7±7.5	23.9±8.0
High-fat meat	10.8±3.7	11.7±3.7	12.2±4.0	11.3±4.0
Junk food	14.4±4.3	16.2±5.7	13.9±3.8	13.8±4.1

^aThese survey items were phrased as Compared to other things in your life, [issue] is: Not at all important compared to other things in our life (coded=0); Less important (1); About as important (2); More important (3); Just about the most important (4); the very most important (5).

^bHigh-fat meat includes bacon, hot dogs, hamburgers, processed meats (e.g. sausage, salami, and bologna), beef, pork, or lamb in a sandwich or as a main dish. Junk food includes chocolate, candy without chocolate, pie or cakes cookies, fast food, soda, French fries and processed snack food.

Table 2 shows the longitudinal mixed models predicting study group assignment, time and the interaction of the two on health values. The control dining halls had significantly lower perceived value for eating a healthful diet from baseline to post intervention (F[1,493]=5.10, P=.02). Tables 3 and 4 show the longitudinal mixed models predicting dietary intake. The control dining halls had significantly greater junk food (F[1,494]= 4.56, P=.03) and high-fat meat (F[1,494]=4.114, p=.04) consumption from baseline to post intervention relative to the intervention dining hall (See Figure 4).

Table 3.

The effect of the intervention on healthy dietary intake across time

	Fruits		Vegetables
	F Value	P	F Value	P
Intervention	0.25	0.62	0.031	0.86
Time	0.45	0.50	0.02	0.89
Intervention* Time	0.13	0.13	1.20	0.27
Gender	12.16	<0.0005	5.36	0.02
Age	0.21	0.65	0.01	0.91

Longitudinal mixed modeling was used to analyze the effect of the intervention on healthy dietary intake.

Table 4.

The effect of the intervention on unhealthy dietary intake across time

	High fat meat consumption		Junk food consumption
	F Value	P	F Value	P
Intervention	2.29	0.13	4.62	0.03
Time	0.11	0.74	6.88	<0.01
Intervention* Time	4.11	0.04	4.56	0.03
Gender	62.99	<0.0001	0.69	0.41
Age	0.03	0.87	7.06	0.01*

Longitudinal mixed modeling was used to analyze the effect of the intervention on unhealthy dietary intake.

Figure 4.

Pre-Intervention to Post-Intervention Difference in Servings of Food Group Per Week ** p<.01; *** p<.0001; All mixed models adjust for gender and age over time.

COMMENT

Overall, results from this study suggest that the iterative design process may be a valuable new method to use for developing effective health behavior interventions. Specifically, the intervention development process, which focused on decreasing meat consumption and improving overall healthful eating among university students, resulted in the rapid identification of two core factors to focus on among the University’s dining hall students; students wanted to perceive the connection between their food and where it came from and students wanted to know more about which foods to eat to keep them focused and healthy through finals. Based on this, two phases of prototypes were developed that highlighted these key “needs”. The final design was then tested and results suggest that the intervention dining halls maintained levels of healthful eating whereas the control dining halls ate more poorly during finals, a period marked by high levels of stress. The control dining halls increased junk food and high-fat meat consumption and decreased their value for eating healthy compared to the intervention dining halls.

The finding that the control dining halls were more likely to eat more junk food and high-fat meats during finals week is consistent with previous research showing a tendency for students to eat a less healthful diet during finals ^9,10 and other times of stress ^11–16. Students awaiting an exam report higher emotional stress and an increased tendency to eat in order to distract themselves from stress⁹. Additionally, high-stressed students are more likely to consume sweet, high-fat foods ^11,15,30.

The intervention was conceived, iterated upon and tested all within the period of 6 months at very little cost. The speed and relatively low costs of this intervention development process coupled with a focus on constant feedback from “users” provides a valuable framework to be used within behavioral science research for quickly and cost-effectively developing interventions. In addition, the iterative design process is also likely a valuable method for creating targeted and tailored interventions. As discussed by others, targeting (i.e., interventions developed for specific types of individuals based on characteristics that can be measured prior to delivery of an intervention) and tailoring (i.e., altering an intervention based on how a person responds; see King et al ³¹ for a full discussion) interventions are an important next steps for improving the overall impact of health behavior change interventions ^31,32. The iterative design process is well suited for the creation of targeted and tailored interventions as the techniques are refined several times through prototyping and testing with the target audience prior to full implementation.

Limitations

There were several limitations to the efficacy study. Related to the intervention development process, we did not use more conventional qualitative research methods for our “needs finding” and development of our “points of view.” As such, we do not have specific numbers on reactions or other metrics often available for more sophisticated qualitative research methods. Despite the lack of quantitative data for our formative research, results from our cluster-randomized controlled study using a repeated cross-sectional assessment strategy suggest that our intervention was efficacious at influencing healthful eating at a dining hall level, therefore providing support to the technique. Future research should explore doing a cost-effectiveness style study comparing the utility user experience design compared to more traditional qualitative research methods for intervention development to determine the most resource-efficient fashion for developing interventions. Previous research testing design principles could be used as possible starting models for creating the appropriate study paradigms⁸. With regard to our cluster-randomized trial, although dining halls, the primary level of analysis, were randomized, we only had two dining halls within each grouping, thus greatly limiting the potential generalizability of these results. As there were only 4 dining halls randomized, the results may have occurred because of some statistical anomaly, although the large samples in both the control and intervention dining halls at both pre and post suggest the likelihood of relatively stable metrics for the 4 dining halls studied, therefore lending further general support to the results. Although all eligible students were contacted to participate, those that filled out the survey represent only a subsample of the total student population resulting in a convenience sample. Therefore, the primarily repeated cross-sectional study design represents another significant limitation. All participants were undergraduates at University dining halls. It is not known if this intervention would be as effective in other settings. Finally, there were constraints due to the self-report nature of our measurements as assessed by food frequency questionnaire. While alternate methods of diet assessment are available, these other methods typically involve higher participant burden and staff involvement that would have been prohibitive for this particular study. Although FFQ’s involve inherent limitations and lead to some level of intake misclassifications, these errors tend to make it more difficult to detect differences when they truly exist, not less difficult. In this study, the FFQ diet data offered a feasible and valuable starting point for this first generation study exploring the utility of IDP for promoting healthful eating among college students.

Conclusion

This paper described the development of an intervention developed using an iterative design process in a dining hall setting among university students. The final intervention was a marketing campaign implemented at the last half of an academic quarter that included sample plates and healthy choice indicators with benefit-based (e.g., “brain”, “mood”, and “energy” foods) messages for promoting healthy food choices. This study found that an intervention developed using an iterative design process resulted in the intervention dining halls maintaining their eating practices during finals week while the control dining halls ate more poorly during finals week. Specifically, students in the control dining hall increased junk food and high-fat meat consumption and decreased their value for eating healthy compared to the intervention group.

IMPLICATIONS

This study suggests that incorporating an iterative design process into behavioral science practices may be a valuable starting point to help identify components within an intervention for optimization. An iterative design process may help ensure that intervention components are being used appropriately as operational definitions of theoretical constructs to help develop evidence-informed, tailored intervention strategies for promoting population health.