Using Participant Feedback to Improve Two Selective Eating Disorder and Obesity Prevention Programs

Abstract

Objective:

To examine qualitative feedback from participants of Healthy Weight and Project Health eating disorder/obesity prevention programs to guide refinements to increase program efficacy.

Method:

Feedback was collected from college students with weight concerns randomly assigned to one of these interventions (N=364, 72% female).

Results:

Healthy Weight participants reported greater program satisfaction (p=.02); no other quantitative differences emerged in opinions of the program or leaders. Project Health participants most valued goal setting (27%), the group setting (23%), and the provided information (16%). Healthy Weight participants most valued the group setting (21%), goal setting (19%), and accountability for behavior change (18%). Project Health participants reported home exercises most frequently as least valuable (22%), followed by role-plays (13%). Healthy Weight participants most frequently reported “nothing” as least valuable (24%), followed by food/exercise logs (9%). The top suggestion from participants from both groups was to add interactive activities.

Conclusions:

Weight gain preventive effects of Project Health were stronger, but Healthy Weight participants were more satisfied, perhaps because Project Health incorporates dissonance-induction strategies which produce psychological discomfort that promotes internalization of health lifestyle goals. Both interventions could be made more interactive by adding group exercises, encouraging group connection via social media, and adding icebreakers.

Healthy Weight, an eating disorder and obesity prevention intervention in which women with body image concerns make gradual lifestyle refinements that bring dietary intake and output into balance, is the only prevention program to significantly reduce future onset of both eating disorders and obesity in multiple trials (Stice, Marti, Spoor, Presnell, & Shaw, 2008; Stice, Rohde, Shaw, & Marti, 2012, 2013; Stice, Shaw, Burton, & Wade, 2006), and has consistently produced significantly greater reductions in body dissatisfaction than assessment only control conditions and alternative interventions, with effects persisting through 3-year follow-up (Stice et al., 2006, 2008, 2012, 2013). We developed a new 6-hr intervention (Project Health) because we wanted to add activities designed to produce dissonance about engaging in behaviors that contribute to weight gain (e.g., discussing health costs of overeating). Both Healthy Weight and Project Health participants showed significantly greater eating disorder symptom reductions through 2-year follow-up, and marginally lower onset of future eating disorders over 2-year follow-up relative to educational video controls. Project Health participants had significantly smaller increases in BMI through 2-year follow-up and lower overweight/obesity onset over 2-year follow-up than Healthy Weight and controls (Stice, Rohde, Shaw, & Gau, 2018).

We collected qualitative data regarding ways to improve the interventions, with the hope this information would allow us to improve the efficacy of these programs. Considering input from key stakeholders fosters a sense of ownership, which is critical for promoting sustainability (Johnson, Hays, Center, & Daley, 2004). A prior paper examining participant feedback to three different delivery methods of the Body Project eating disorder prevention program found that group membership was key to participant satisfaction (Shaw, Rohde, & Stice, 2016). The current study is the first to examine qualitative feedback from participants in two programs that have reduced eating disorder and obesity onset. Programs producing effects for both eating disorder and obesity are rare, thus continuing to refine these interventions should facilitate the design of more effective dual-focused interventions.

Methods

Recruitment and Descriptive Information

Participants (N=364) were 261 women (M age=19.1, SD=1.2; M BMI=23.2, SD=2.7) and 103 men (M age=19.2, SD=1.2; M BMI=24.2, SD=2.4); 3% were African American, 3% American/Alaskan Native, 15% Asian/Pacific Islander, 68% Caucasian, 0.5% Hawaiian/Pacific Islander, and 11% were Hispanic. College students ages 17-23 with weight concerns were recruited into a study designed to promote healthy lifestyle choices and prevent weight gain. Participants were randomized to Project Health (N=119), Healthy Weight (N=122), or an educational video control condition (N=123) via a random number table. Both interventions consisted of 6 weekly 1-hr group sessions with 6-10 participants and 2 group leaders. Both interventions consisted of 6 weekly 1-hour group sessions with 6-10 participants and 2 facilitators. Twenty clinical graduate students or college mental health staff (17 women and 3 men, of various ethnicities) facilitated 29 groups (15 Healthy Weight, 14 Project Health; mean number of groups led by a facilitator = 2.9, range = 1-6). If a participant missed a session, a brief (10-15 minute) individual session was conducted to review material when possible. Facilitators read reports regarding the interventions and the intervention script, and attended a 4-hour workshop to role-play sessions and discuss process issues, including homework compliance and retention.

Interventions

Healthy Weight is described to participants as a body acceptance intervention and focuses on making weekly small, sustainable changes to diet and exercise to balance caloric intake/output. Sessions include a brief review, educational handouts, in-session discussions, review of behavior change goals, and developing healthy behavior change plans for next session. Home exercises include implementing participant-selected diet and exercise goals, and keeping food/exercise logs.

Project Health sessions begin with a verbal commitment to participate, include discussions of home exercises and in-session writing exercises, discussions and role-plays concerning the costs and negative effects of obesity, and conclude with home exercises. Participants are videotaped reading their homework to increase accountability. Activities were designed to produce cognitive dissonance to promote internalization of healthy lifestyle goals. Participants also commit to small gradual improvements to diet and activity. Educational Video participants were encouraged to watch a free 51-minute video, called The Weight of the World focusing on costs of the obesity epidemic.

Measures

Participants completed a written survey post-intervention including the following three questions: (1) How satisfied were you with your experience in this program; (2) How effective do you think this program would be for someone who has weight concerns; and (3) How much do you think this program will help someone deal with or avoid weight concerns; 1-5 scale), and 10 questions regarding perceptions of group leaders (intelligence, clarity, trustworthiness, skillfulness, preparedness, experience, respectfulness, sincerity, confidence, and topical expertise; 1-5 scale). These 10 questions formed a reliable scale (α=.92), so an overall composite score was analyzed (mean composite score for Project Health =4.45 (SD=.57); mean composite score for Healthy Weight=4.59 (SD=.50).

Three open-ended questions asked: (1) What aspects of the program did you find most valuable; (2) What aspects of the program did you find less valuable; and (3) How could we make the program more interesting and enjoyable for future participants? Evaluation data were provided by 107 Project Health participants (90%) and 116 Healthy Weight participants (95%).

Data Analysis

The frequency distribution, mean and standard deviation of scores are provided for the quantitative items assessing reactions to the intervention, as are the means and SDs for perception of group leaders. Differences in quantitative scores for participants in the two conditions were examined using t-tests. Content analysis was used to analyze the qualitative data as it permits quantifying the frequency of categories and themes (Vaismoradi, Turunen, & Bondas, 2013). The first two authors conducted the analysis, which consisted of reading the open-ended survey responses (which usually consisted of 1-2 sentences), independently generating codes and resolving coding discrepancies through discussion and consensus, developing final codes, and coding the data. Rank order correlations of categories suggested inter-coder agreement: (1) most valuable aspects (Spearman rho=.93), (2) least valuable aspects (Spearman rho=.79), and (3) suggested improvement (Spearman rho=.96). Average of coding frequencies of the two authors are reported in Tables 1 and 2.

Table 1:

Participant ratings regarding program satisfaction, effectiveness, and whether programs could help deal with weight concerns

	Group	Percent Endorsed					Descriptive Stats		T-Tests
		1	2	3	4	5	Mean	SD	t-value	p-value
Program satisfaction	PH	1.0	6.0	26.0	40.0	27.0	3.86	0.92	2.34	.020
	HW	0.9	3.8	17.9	33.0	44.3	4.16	0.92
Program effectiveness	PH	0.0	2.0	27.0	45.0	26.0	3.95	0.78	0.90	.368
	HW	0.0	1.9	20.8	48.1	29.2	4.05	0.76
Help w/weight concerns	PH	1.0	1.0	30.0	51.0	17.0	3.82	0.76	1.81	.072
	HW	0.0	4.7	17.9	48.1	29.2	4.02	0.82

PH = Project Health, HW = Healthy Weight.

Table 2.

Qualitative feedback from Project Health Participants

Q1: What aspects of the program did you find most valuable?	Frequency
Goals	40
Group setting	33
Information provided	24
Accountability	16
Awareness of food/eating habits	13
Role-plays	8
Facilitators	3
Home exercises	3
Other aspects	6
Total Responses	146
Q2: What aspects of the program did you find least valuable?	Frequency
Home exercises	21
Role-plays	12
Nothing	11
Course material	11
Information obvious/repetitive	7
Goal review too long, etc.	7
Too formal/structured/impersonal	5
Verbal commitment	3
Other aspects	17
Total Responses	94
Q3: How could we make the program more interesting and enjoyable for future participants?	Frequency
More interactive exercises and interaction generally	28
More information/visual aids	19
Nothing	11
Less scripted	6
Add icebreakers in beginning	5
Have groups with similar others	4
Longer sessions/duration	2
Other suggestions	22
Total responses	97

Results

Quantitative Ratings of Intervention and Group Leaders

Most participants in both interventions reported positive rating regarding program satisfaction (either a 4 or 5 on a 1-5 scale ranging from not very to very) (77% Healthy Weight; 67% Project Health); program effectiveness (77% Healthy Weight; 71% Project Health); and how they felt the program would help them deal with or avoid weight concerns (77% Healthy Weight; 68% Project Health). One of these differences, regarding satisfaction, was statistically significant; (t[1 73]=2,34, p=.02); e.g., 44% of Healthy Weight participants reported being “very satisfied” with their experience in the program compared to 27% of Project Health participants. Table 1 shows the breakdown in responses to these questions (and means and standard deviations) by intervention.

Regarding rated perceptions of group leaders, average scores for Healthy Weight (M=4.59, SD=.50) and Project Health (M=4.45, SD=.57) suggested a trend-level difference favoring Healthy Weight; (t[173]=1.81, p=.073).

Qualitative Data: Valuable Aspects and Suggested Improvements to Groups

Tables 2 and 3 report responses to the 3 open-ended questions. Most responses consisted of 1-2 sentences and had strong face validity (i.e., when asked what aspects of the program they liked/disliked, they would state “role-play” or “home assignment”). A “miscellaneous aspects” responses category was created for the questions to capture those that were unique to a participant. Some aspects were also unique to the interventions, such as role-plays for Project Health, and food/exercise logs for Healthy Weight. Participants could list more than one response to these questions. Ten percent of Project Health and 8% of Healthy Weight participants did not respond to the question, “What aspects of the intervention did you find most valuable”. Ten percent of Project Health and 5% Healthy Weight participants did not respond to the question, “What aspects of the intervention did you find least valuable”. Nine percent of Project Health and 5% of Healthy Weight participants did not respond to the question, “How could the group be made more interesting/valuable for future participants”. Post-intervention evaluation data were provided by 107 Project Health participants (90% of total) and 116 Healthy Weight participants (95%).

Table 3.

Qualitative feedback from Healthy Weight Participants

Q1: What aspects of the program did you find most valuable?	Frequency
Group setting	25
Goals	23
Accountability	21
Information provided	21
Food/exercise logs	10
Facilitators	8
Other aspects	11
Total Responses	119
Q2: What aspects of the program did you find least valuable?	Frequency
Nothing	26
Food/exercise logs	10
Facilitators	9
Handouts/Homework	9
Goal review too long, etc.	7
Other group members	5
Sessions too long	4
Exercise aspect not helpful	4
Repetitive/obvious or not enough info	4
Too much info/need more time for discussing/sharing	3
Other aspects	26
Total Responses	107
Q3: How could we make the program more interesting and enjoyable for future participants?	Frequency
More interactive exercises/more interaction generally	20
Nothing	20
More information/tip sharing	17
Less robotic/scripted/repetitive	9
Add icebreakers in beginning	6
Add visual aid	4
Have groups with similar others	3
Other suggestions	21
Total responses	100

Aspects of the group found to be most valuable by Project Health participants in order of frequency were: lifestyle change goals (setting/maintaining goals) (reported by 27%), the group setting (23%), information provided (16%), accountability for lifestyle changes (to the group, facilitators, and oneself) (11%), awareness of food/eating habits (9%), role-plays (5%; practicing responding to pressures to eat unhealthy food, etc.), miscellaneous aspects (4%; e.g., “cognitive appraisals”, “talking about how the media portrays women”), facilitators (2%), and home exercises (2%).

Aspects of the group found to be least valuable by Project Health participants in order of frequency were: home exercises (22%; e.g., “Some assignments seemed pointless”, “Researching health topics”), miscellaneous aspects (18%; e.g., “it just didn’t grab my attention”, “people’s ideas on how to deal with temptation”), role-plays (13%; e.g., “I don’t think the role-playing had much of an effect”, “The one on one role-playing”), “nothing” (12%), course material (e.g., “explaining bad eating habits”) (12%), information was obvious/repetitive (7%), goal review too long (7%), too formal/structured/impersonal (5%), and the verbal commitment (3%).

When asked how to improve the intervention, Project Health participants suggested increased group interaction (e.g., role-plays) (29%), miscellaneous aspects (23%; e.g., “suggest exercise classes we could attend”, “keep adding more motivational activities such as the ‘things you’ll change’ list”), more information and/or visual aids (20%), “no suggestions” (11%), less scripted (6%), more icebreakers (5%), groups with similar others (4%), and longer sessions/group duration (2%).

Aspects mentioned as most valuable by Healthy Weight participants were: group setting (21%), goals (19%), accountability (18%), information provided (18%), miscellaneous aspects (9%; e.g., “it was nice to see that other people were just regular college students”, “scientific and simple approach”), food/exercise logs (8%), and facilitators (7%).

Aspects of the Healthy Weight group found least valuable were: “Nothing” (24%), miscellaneous aspects (24%; e.g., “I was the only guy”, “the fact that we didn’t weigh ourselves/have access to BodPod info”), food/exercise logs (9%), facilitators (8%), handouts/homework (8%), goal review (too long, etc.) (7%), other group members (5%), sessions too long (4%), the exercise aspect not helpful (4%), information was repetitive/obvious (4%), and that there was too much time for sharing (3%).

When asked how to improve the intervention, Healthy Weight participants mentioned miscellaneous aspects (21%; e.g., “assign buddies to workout with”, “partner people up in the 6 week session”), that they liked it as is (no suggestions, 20%), making it more interactive (20%), adding more information/tip sharing (17%), making it less scripted (9%), adding icebreakers (6%), visual aids (4%), and having groups with similar others (3%).

Discussion

This paper examined (a) ratings regarding program benefit and facilitator characteristics, (b) intervention aspects reported as most and least valuable and suggested improvements, and (c) whether ratings differed by group. The purpose was to solicit information to refine these interventions and potentially illuminate why Project Health produced superior weight gain prevention effects.

Healthy Weight participants reported slightly higher satisfaction than Project Health participants. Additionally, twice as many Healthy Weight participants reported “nothing” as least valuable versus Project Health participants, suggesting that dissonance-inducing activities in Project Health may have contributed to lower satisfaction. Consistent with these acceptability data, mean session attendance was slightly lower in Project Health than in Healthy Weight (M = 4.0 vs. 4.6; Stice et al., 2018). Project Health participants mentioned role-plays second most frequently as a least valuable (13%) aspect of the program, and they were ranked sixth in order of aspects mentioned as most valuable (5%), which might reflect their psychologically conflicting nature. All participants found the group and goal setting aspects of the group most valuable. Thus, results support the therapeutic value of group modalities in the development of prevention programs, consistent with a prior paper that found participants placed a higher value on a group versus online version of the Body Project (Shaw et al., 2016).

Results also suggest that home exercises might be somewhat difficult to complete for college students. In Project Health, home exercises were reported most frequently as the least valuable aspect of the intervention, and in Healthy Weight they were reported as the second least valuable intervention component. However, reducing the home exercises might weaken intervention effects, particularly for Project Health, as increased effort is thought to increase cognitive dissonance (e.g., Axsom & Cooper, 1981). Food and exercise logs were mentioned the second most frequently in Healthy Weight as least valuable, suggesting that reducing either the amount of detail or frequency required for these logs should be considered. Facilitators were mentioned the third most frequently in Healthy Weight as the least valuable aspect, with participants saying “…1 felt the instructors would have known better what to change than me”, “No one was a dietitian/nutritionist…etc., suggesting that Healthy Weight participants expected more guidance with food and exercise change goals. Yet, too much guidance could reduce the participant-driven nature that we consider crucial to this program’s success. However, when possible, it would be good to work with facilitators who are knowledgeable about nutrition to provide useful information in a way that does not take the ownership of these lifestyle changes away from participants.

The top suggestion for improving both interventions was more group interaction (29% Project Health; 20% Healthy Weight). This suggests that there is something inherently valuable about the group setting, which might enhance effectiveness. It would be useful to encourage group members to support each other’s goals and brainstorm solutions to implementation challenges. Including more interaction in the group-based activities and encouraging participants to engage in lifestyle change behaviors together outside the group might also be useful. Future versions of these programs could also use social media to increase accountability and group support via ‘virtual’ connections, both during and after the intervention. Participants in both programs also suggested adding icebreakers to foster greater cohesion. Both programs already include an icebreaker in Session 1 so it might be useful to add a more interesting icebreaker to this session, as well as a second icebreaker to Session 2 (e.g., “What type of exercise have you always wanted to try, but haven’t?”).

Another frequently mentioned suggestion for both interventions is that participants would like more information and sharing on healthy eating and exercise. The former suggests that distributing additional educational handouts would be good for the next intervention iteration. This is also in line with employing facilitators who are knowledgeable on nutrition and exercise, but minimally, suggests that it would be useful to provide more information to help guide participants’ healthy change goals. Regarding the suggestion for “more time to share tips”, it might be helpful to allow more time for brainstorming healthy lifestyle changes, which would also increase interaction.

Collectively, results indicate that group cohesion, goal setting, accountability, and delivering information are important aspects of successful eating disorder/obesity prevention programs, and future research should further evaluate their relative importance. Results also suggest the interesting possibility that inducing cognitive dissonance might inadvertently reduce satisfaction while also driving stronger effects (i.e., weight gain prevention). Balancing the degree of perceived satisfaction in a group that induces some degree of psychological discomfort between one’s stated values and actual behaviors is a topic that to our knowledge has not been examined. It could also be that some degree of weight stigmatization could lead to psychological discomfort and greater dissatisfaction with Project Health, a possibility that future research should examine. However, it is important to note that the intervention script expressly states not to discuss anything negative regarding people with obesity, but rather the costs of lifestyle behaviors that contribute to obesity, as well as the medical, psychological, and social costs associated with obesity, although some group participants may not have grasped this important distinction. Findings also indicate that Healthy Weight and Project Health interventions, which have both successfully reduced eating disorder symptoms and obesity onset, have also achieved high acceptability among key stakeholders. However, it should be noted that these findings are limited by the fact that data were collected via a short survey that tended to elicit brief phrases or 1-2 sentence answers rather than a qualitative interview, which likely would have provided additional feedback with which to refine these two interventions.

Highlights.

• Healthy Weight participants identified greater program satisfaction than Project Health participants despite weight gain preventive effects being stronger for Project Health

• Both Healthy Weight and Project Health participants reported the group setting and goals as the top 2 most valuable program components

• Participants in both groups suggested that additional interactive activities be included