Process Evaluation of the Project SHINE Intervention for African American Families: An Integrated Positive Parenting and Peer Monitoring Approach to Health Promotion

Abstract

This study describes the process evaluation of Project SHINE, a randomized family-based health promotion intervention that integrated parenting and peer monitoring for improving sedentary behavior, physical activity, and diet in African American families. Adolescent-parent dyads (n=89) were randomized to a six-week behavioral, positive parenting, and peer monitoring skills intervention or a general health education comparison condition. Process evaluation included observational ratings of fidelity, attendance records, psychosocial measures, and qualitative interviews. Results indicated the intervention was delivered with high fidelity based on facilitator adherence (>98% of content delivered) and competent use of theoretically-based behavior change and positive parenting skills (100% of ratings >3 on a 1-4 scale). Although only 43% of peers attended the “bring a friend” session, overall attendance was high (4.39±1.51 sessions) as was the retention rate (88%). Parents in the intervention condition reported significant improvements in communication related to adolescents’ engagement in health behaviors both on their own and with peers. These findings were supported by qualitative themes related to improvements in family communication and connectedness. This study provides an innovative example of how future family-based health promotion trials can expand their process evaluation approaches by assessing theoretically-relevant positive parenting variables as part of ongoing monitoring.

Process evaluation methods are increasingly being integrated into pediatric health promotion interventions targeting improvements in weight, sedentary behavior (SB), physical activity (PA), and diet (Branscum, Sharma, Wang, Wilson, & Rojas-Guyler, 2013; Knowlden & Sharma, 2014; Robbins, Pfeiffer, Wesolek, & Lo, 2014; Wilson et al., 2009). While this increase is encouraging, a recent review of 87 interventions in this area found that only 40% reported at least one indicator of process evaluation (Branscum et al., 2013). Because youth health behavior change involves multiple components and is implemented within existing social systems (e.g., families, peers), understanding the quality of intervention delivery is a necessary first step in determining program success. The purpose of this study is to describe the process evaluation of Project SHINE (“Supporting Health Interactively through Nutrition and Exercise”), a randomized family-based health promotion intervention that tested the effects of behavioral skills plus positive parenting and peer monitoring on SB, PA, and diet in African American adolescents and their parents (St. George, Wilson, Schneider, & Alia, 2013).

The theoretical framework used to develop Project SHINE integrated behavioral strategies from Social Cognitive Theory (Bandura, 2004) (e.g., self-monitoring, goal-setting), elements involved in fostering intrinsic motivation from Self-Determination Theory (Ryan & Deci, 2000) (i.e., autonomy, competence, belongingness), and positive parenting practices from Family Systems Theory (Broderick, 1993) (e.g., parent-adolescent communication, parental monitoring and management of peers). Positive parenting practices for adolescents, including open communication, cooperative problem-solving, and monitoring of peer relationships, support an adolescents’ need for autonomy, competence, and belongingness (Ackerman, Kashy, Donnellan, & Conger, 2011; Joussemet, Landry, & Koestner, 2008; Mounts, 2008; Robin & Foster, 2002). Autonomy-supportive parenting is particularly important for promoting youth behavioral skill-building and well-being and involves communication practices such as offering choices, providing a rationale for requests, and recognizing the youth’s perspective (Joussemet et al., 2008). Similarly, ongoing parental monitoring and management of adolescent peer relationships allows parents to aid adolescents in fostering friendships that reinforce healthy behaviors (Mounts, 2008). Although positive parenting practices have been associated with greater youth PA (Arredondo et al., 2006), less screen time (Jago et al., 2011), and higher fruit and vegetable intake (Pearson, Atkin, Biddle, Gorely, & Edwardson, 2010), few health promotion interventions have used these strategies for integrating parent and peer systems to improve adolescent SB, PA, and diet. Because ethnic minority youth, including African Americans, are at increased risk for engaging in unhealthy behaviors (Iannotti & Wang, 2013), innovative health promotion interventions are urgently needed to reduce existing health disparities. Overall, Project SHINE was designed to bridge family and peer systems through promoting positive parenting practices for managing adolescents’ SB, PA, and diet on their own and in the presence of peers (St. George et al., 2013). Given the novelty of the approach, the goal of this study is to describe the extent to which theoretical essential elements related to behavioral skill-building, positive parenting, and peer monitoring were practically delivered within the context of the Project SHINE intervention.

Previous process evaluations of evidence-based positive parenting interventions (e.g., Parent Management Training - Oregon, Family Check-up) have systematically assessed intervention fidelity, or the extent to which an intervention is delivered as planned, by rating how well facilitators follow session content, demonstrate knowledge of positive parenting skills, and use general clinical skills to engage with families (e.g., joining the family’s storyline, asking questions that lead to openness, encouraging skill development) (Forgatch, Patterson, & DeGarmo, 2005; Smith, Dishion, Shaw, & Wilson, 2013). Across these studies, high ratings of fidelity have been positively associated with increased parent engagement as well as improvements in observed positive parenting practices and youth outcomes (Forgatch et al., 2005; Smith et al., 2013). Contrastingly, family-based health promotion interventions for improving adolescent weight, SB, PA, or diet have historically limited their process evaluation efforts to an assessment of program attendance and participant satisfaction (Baranowski et al., 2003; Golley, Magarey, Baur, Steinbeck, & Daniels, 2007; Kitzman-Ulrich et al., 2011; Morgan et al., 2011; Pearson, Atkin, Biddle, & Gorely, 2010; Robinson et al., 2003; Robinson et al., 2010). The present study thus expands on previous research by describing the development and use of a measure designed to assess facilitator fidelity in delivering the Project SHINE intervention in accordance with theoretical essential elements (e.g., behavioral skills, positive parenting). Additional process data, including participant attendance and retention, changes in family communication and parental management of peers from baseline to post intervention, and qualitative themes related to perceived program effects on family relationships are also presented.

Method

Participants

Participants were 89 African American adolescent-parent dyads, most of which were recruited at local community events. These events were held in predominantly low-income areas and were either health-related (e.g., health fairs), family-related (e.g., “family fun day” at local churches or parks), or school-based functions (e.g., middle school carnival, parent orientation). Families were eligible if they had an 11-15 year old African American adolescent and one parent (or legal guardian) willing to participate. Families were excluded if they were enrolled in a health or weight loss program or the adolescent had a chronic disease (e.g., type 2 diabetes), psychiatric condition or developmental delay.

Study Design and Procedure

This study was approved by the University of South Carolina Institutional Review Board prior to data collection. Participants were informed that the purpose of the study was to promote well-being in African American adolescents. Study staff further explained that families would be randomized to either a “specialized nutrition and exercise program,” which also included parenting strategies for promoting positive adolescent health behaviors (i.e., behavioral skills plus positive parenting and peer monitoring intervention), or a “comprehensive health program” covering information on a broad range of health topics (i.e., general health education comparison condition; see Footnote 1 for list of topics covered).

Project SHINE was implemented across ten groups (5 intervention, 5 comparison). Families were stratified by adolescent sex and weight status and randomly assigned to an evening (i.e., Tuesdays or Thursdays). Families then participated in a two-week run-in period, a strategy which has been previously shown to improve retention by allowing families to demonstrate their commitment to the program (attendance) prior to randomizing them to a condition (Ulmer, Robinaugh, Friedberg, Lipsitz, & Natarajan, 2008). Run-in sessions included an orientation and review of national health behavior recommendations. Each evening was then randomized to a condition. To encourage participant attendance and punctuality across conditions, door prizes (e.g., $5 gift cards to local stores) were raffled at the beginning of all sessions and free childcare was provided for younger siblings of study participants. Additionally, healthy snacks (e.g., smoothies, fruits) were served at all sessions. Finally, participants were provided with colorful handouts tailored to each session’s topic.

SHINE Intervention

A detailed intervention curriculum matrix, summarizing weekly session content and activities, has been published elsewhere (St. George et al., 2013). The intervention was conducted in a university setting across six 1.5-hour weekly sessions jointly attended by parents and adolescents. “Essential elements” (see Table 1) based on constructs from the guiding theories informed intervention development, facilitator training, and the corresponding measure of fidelity.

Table 1.

Project SHINE Theories, Theoretical Constructs, and Essential Elements

Theory	Theoretical Constructs	Description of Intervention Essential Elements
Behavioral Skills
SCT	Self-Monitoring	• Parents and adolescents monitor their health behaviors, using a tool of their choice.
SCT	Goal-setting	• Parents and adolescents set specific and measurable long and short term health behavior change goals together. • Parents and adolescents are reinforced for effort towards goals and goals achieved.
SCT	Self-regulation	• Parents and adolescents review goals and are provided with feedback. • Parents and adolescents identify barriers and supports for health behavior change. • Parents and adolescents revise behavior change contracts and make action plans to achieve goals.
Positive Parenting
SDT, FST	Communication skills	• Parents use communication strategies to elicit input from adolescents. • Adolescents use communication strategies to elicit social support from parents and peers. • Parents and adolescents use problem-solving skills to set family rules around health behaviors. • Parents use problem-solving skills to navigate adolescent-peer relationships around health behaviors. • Parents and adolescents self-evaluate family communication/climate changes.
SCT, SDT, FST	Social support	• Parents provide adolescents with social support for changing health behaviors. • Adolescents elicit social support for health behaviors from their parents and peers.
SDT, FST	Autonomy support	• Parents seek input from adolescents and negotiate rules and behavior changes together. • Adolescents have choices and are provided with opportunities to provide input.
Peer Monitoring
FST	Parental monitoring and management of peer relationships	• Parents keep track of adolescent health behaviors. • Parents manage peer relationships around health behaviors through monitoring, supporting, and guiding.
Adolescent Intrapersonal Variables
SCT	Self-efficacy	• Adolescents feel confident that they can change health behaviors. • Adolescents feel confident communicating about health with their family and friends. • Adolescents have support from family and friends for changing behaviors.
SDT	Motivation	• Intervention social climate fulfills adolescent autonomy, competence, and belongingness.

Note. SCT = Social Cognitive Theory; SDT = Self-Determination Theory; FST = Family Systems Theory.

Behvaioral skills for health behavior change.

Families were challenged to work on improving a health behavior of their choice (SB, PA, or diet) by self-monitoring, goal-setting, and self-evaluating their progress. Each week, participants completed a family health behavior contract detailing their goals, self-monitoring strategies, reinforces, and sources of social support. Facilitators provided brief individualized feedback on a weekly basis that allowed participants to identify barriers to change, problem solve accordingly, and revise their goals as needed.

Positive parenting and peer monitoring skills.

Positive parenting and peer monitoring skills were a major focus of weekly session content and activities meant to facilitate family health behavior change. The first two weeks of the intervention, for example, focused largely on positive communication and support skills. Corresponding session activities included a discussion of strategies for effective communication (e.g., active listening, using “I” statements, taking turns making brief statements, using a neutral tone of voice), the collaborative establishment of group “ground rules” (e.g., “allow time for everyone to express their point,” “listen with an open mind”), and a brief “Family Walk and Talk” session wherein parents and adolescents discussed family SB, PA, and dietary behaviors. Parents also spent time reviewing strategies for providing social support (e.g., reinforcement, praise) and autonomy support (e.g., offering choice, eliciting input, engaging in shared decision-making, negotiating) while adolescents rehearsed skills for eliciting social support from both their parents and peers. During the third week, families continued strengthening their communication skills through engaging in a group problem-solving activity related to establishing healthy family household rules or engaging in healthy behvaiors with friends. Because a novel element of this intervention was the integration of parent and peer systems through the use of peer management skills, this concept was further discussed with parents as were specific skills for facilitating adolescent access to similarly “health-minded” peers (e.g., guiding, supporting) (Mounts, 2011) during the fourth week. Adolescents were encouraged to bring a friend to this session, and activities included goal setting and planning for healthy activities with friends. “Family Bonding Activities” were assigned on a weekly basis (e.g., negotiate family health rules, engage in a healthy activity together with family and friends) to reinforce session content.

Strategies for enhancing cultural relevance.

Strategies for enhancing the cultural relevance of the intervention for African American families (e.g., peripheral strategies, such as featuring photos of African American families in study materials; evidential strategies, such as providing national data related to African American youth; constituent-involving strategies, such as having an African American dietitian and community leader as outside speakers) were woven into intervention content and delivery (Kreuter, Lukwago, Bucholtz, Clark, & Sanders-Thompson, 2003).

Intervention Facilitators

Facilitators were graduate students in clinical psychology or public health (n=6, 5 female, 3 African American, 2 Caucasian, 1 Hispanic) who followed a structured guide with weekly topics/activities. Prior to leading sessions, facilitators attended 10 hours of training comprised of didactics, group discussions, role-plays, and activities related to the theoretical model. Study essential elements were simplified into a user-friendly acronym and corresponding pyramid-shaped illustration referred to as “LITE” (i.e., Lifestyle, Interactions, Together, Engaged; see Figure 1). This tool was developed so facilitators could quickly assess intervention sessions by noting whether families were engaged and having fun (“Together/Engaged”), using positive parenting skills (“Interactions”), and working on behavior change (“Lifestyle”). Facilitators were instructed to view each level of the pyramid as a necessary prerequisite for the next. For example, to effectively cover parenting and behavioral skills, facilitators were trained to first recognize and maintain a fun, autonomy-supportive climate. Facilitators met on a weekly basis to discuss intervention delivery and relevant clinical and/or administrative issues.

Figure 1.

Simplified essential elements “LITE” (Lifestyle, Interactions, Together, Engaged) used to train facilitators

Process Evaluation Measures

Intervention fidelity.

Quantitative checklists and rating scales were developed to assess fidelity, including facilitator adherence (i.e., the degree to which an intervention is implemented according to the overarching framework) and competence (i.e., the level of clinical skill involved in promoting participant behavior change) (Forgatch et al., 2005). Although these checklists were designed to capture theoretical constructs specific to this study, similar checklists were used to guide their development. For example, in a randomized school-based after school PA trial, fidelity checklists captured the extent to which intervention leaders gave students opportunities to vote on activities, encouraged participation based on behavioral goals, and created an interactive environment (Wilson et al., 2009). To validate the use of these measures, a senior behavioral scientist (second author) who provided input on the development of process evaluation materials for this study listened to audio recordings of intervention sessions for the first study cohort while completing the quantitative checklists/ratings scales. A trained, independent rater systematically observed all weekly sessions in person. Prior to rating sessions, the evaluator completed four hours of training consisting of didactics, an overview of the study framework/content, and a detailed review of the coding dictionary.

Facilitator adherence.

The measure for facilitator adherence (see Table 2) was comprised of 10 yes/no responses based on whether facilitators followed the structured guide in delivering session content (e.g., ground rules displayed, key topics/skills demonstrated, “Family Bonding Activity” assigned). Percentages of “yes” responses were used to summarize results. Achieving adequate adherence was defined a priori as ≥ 90% of the intended intervention actually delivered to each cohort.

Table 2.

Process Evaluation Form for Assessing Facilitator Adherence and Competence

Essential Element	Items	Response Format
Facilitator Adherence
	1. Facilitators start session on time  2. Snack offered  3. Door prize raffled  4. Ground rules displayed  5. Session agenda reviewed with participants  6. Key topics/skills explained  7. Key topics/skills demonstrated (or examples provided/elicited from participants)  8. Participants engage in interactive activity  9. “Family Bonding Activity” assigned  10. Summary/closure	0 = No; 1 = Yes
Facilitator Competence
Behavioral skills (self-monitoring, goal-setting, self-regulation)	Facilitators…  1. Reinforce national recommendations for target health behaviors  2. Provide feedback on weekly short term goals and assignments  3. Aid participants in linking short to long term goals  4. Reinforce progress towards achieving short and long term goals  5. Allow participants to self-evaluate progress and revise personal/family goals as needed  6. Encourage families to support one another in setting/meeting goals through comments or questions	1 = None; 2 = Some; 3 = Most; 4 = All
Communication skills	Facilitators…  1. Review session agenda with all participants  2. Ask if participants have questions  3. Fully answer participants’ questions  4. Ensure participants follow ground rules  5. Call participants by name  6. Use a neutral tone of voice  7. Get on participants’ level/avoid jargon  8. Make good eye contact  9. Orient towards participants when they are speaking  10. Use expressive facial expressions  11. Listen to participants  12. Verbally respond to participants when prompted  13. Respond with reflective listening statements	1 = None; 2 = Some; 3 = Most; 4 = All
Social support	Facilitators…  1. Set a welcoming, positive atmosphere by making small talk with participants  2. Are upbeat and enthusiastic  3. Acknowledge positive within-family interaction  4. Reinforce positive within-family interaction  5. Acknowledge positive between-family interaction  6. Reinforce positive between-family interaction	1 = None; 2 = Some; 3 = Most; 4 = All
Autonomy support	Facilitators…  1. Provide participants with choices  2. Elicit participant input  3. Reinforce participant input	1 = None; 2 = Some; 3 = Most; 4 = All

Facilitator competence.

The measure of facilitator competence (see Table 2) was developed based on intervention essential elements including behavioral skills, communication skills, social support, and autonomy support. “Competent” intervention delivery was based on how well facilitators utilized behavior change and positive parenting skills. Ratings were made on each of four components using a 4-point scale ranging from 1=low to 4=high competence: behavioral skills (6 items), communication skills (13 items), social support (6 items), and autonomy support (3 items). Items were intended to mirror positive parenting skills reviewed with families throughout the course of the intervention. For example, items related to communication assessed how well facilitators used a neutral tone of voice, made good eye contact, and responded to participants with reflective listening statements. Averages for individual components across cohorts were calculated. Adequate competence was defined a priori as a value of ≥3 for each component.

Participant attendance.

Attendance was coded as either 0=family not in attendance, 0.5=only one member of the adolescent-parent dyad in attendance, or 1=family in attendance. Although the importance of attending in-person sessions was emphasized, make-up sessions were permitted. Attendance was calculated both including and excluding make-up sessions. The number of families who brought guests to the “bring a friend” session was also recorded.

Participant retention.

Retention was calculated based on the percent of families who participated in intervention and/or comparison group sessions and completed post measures. An independent study staff member not directly involved in intervention delivery contacted the families who discontinued study participation to assess their reasons for drop-out.

Participant psychosocial variables.

Aside from the use of fidelity checklists, differences between groups on key theoretical constructs (i.e., parent-adolescent communication regarding health behaviors, parental management of peers) were examined to determine how well theoretical essential elements had been delivered to participants in the intervention group. Because these constructs were not targeted specifically in the comparison condition, it was hypothesized that relative to those in the comparison condition, those in the intervention would display greater improvements in both communication and parental monitoring of peers at post intervention.

Parent-adolescent communication around health behaviors.

To assess parent-adolescent communication regarding adolescent engagement in health behaviors on their own and with peers, a 16-item measure of communication was developed (St. George et al., 2013). At baseline and post intervention, parents indicated the frequency (8 items) and quality (8 items) with which they discussed eight topics (e.g., “decreasing how much TV you watch,” “being physically active with your friends”). Frequency of communication was assessed with a 4-point scale ranging from 0=“Never” to 4=“Many times.” Quality of communication was assessed on a 4-point scale ranging from 0=“Do not discuss this topic” to 4=“Usually talk about it openly and each say what we think.” All items were averaged to obtain a single score of communication. Cronbach’s alpha indicated adequate internal consistency (α=0.91). In addition, its construct validity was supported through its negative association with self-reported adolescent SB (St. George et al., 2013).

Parental management of peers.

Adolescents responded to 18 items from the Parental Management of Peers Inventory assessing the extent to which their parents guided and/or provided them with direction regarding their peer relationships (Mounts, 2004, 2011). Response options ranged from 1 = “Strongly disagree” to 4 = “Strongly agree.” Sample items include, “My parents encourage me to do activities with kids they like,” and “My parents talk to me about the pros and cons of hanging around certain people.” The scale demonstrated adequate internal consistency in this study (α = 0.79).

Data analyses.

Multiple imputation methods (m=20) were used for handling missing data (6% of psychosocial data missing due to drop out). Given the nested study design (individuals within groups), two random intercept multilevel regression models were used to test intervention effects on parent-adolescent communication and parental management of peers, respectively. Models controlled for demographic variables (e.g., adolescent sex, age), cohort, and baseline values. Variables were either contrast coded (sex, cohort), mean centered (age, zBMI, income, baseline value), or dummy coded (treatment; 0 = comparison, 1 = intervention) to facilitate model interpretation, with the intercept representing the mean value across groups for the average adolescent in the intervention condition.

Post-program interviews.

Following completion of study post measures, a subset of families randomized to the intervention condition (n=18 parents, n = 16 adolescents) completed brief (5-10 minute) follow-up phone interviews assessing their overall experience in the program. A study staff member not directly involved in intervention delivery conducted the interviews with parents and youth (separately). Parents and adolescents were asked four open-ended questions: (1) What were some of the most positive aspects of the program? (2) What could be improved about the program? (3) How (if at all) has the program affected your family? (4) Is there anything else you’d like to add about your overall experience with Project SHINE? Interviews were audio recorded and transcribed by a professional transcription agency. A general inductive approach, or a systematic procedure that entails multiple readings and interpretations of transcripts reflective of participant responses, was used to derive study themes (Pope, Ziebland, & Mays, 2000; Thomas, 2006). Given the brief nature of the interviews, all study transcripts were read in detail by one author (SMS) and used to develop a coding scheme. The coding scheme included three main categories based on the questions (what participants liked, what they would have changed, how the program impacted their relationships) and corresponding sub-categories derived directly from participant responses. This same author (SMS) coded participant responses to determine the most commonly endorsed themes.

Results

Intervention Fidelity

Adherence.

Facilitator adherence ratings were consistently high across items and cohorts, with the average ranging from 98% to 100%.

Competence.

Average facilitator competence ratings indicated a priori goals were consistently met for all components across all study cohorts (Behavioral skills=3.88±0.11; Communication skills=3.97±0.02; Social support=3.69±0.06; Autonomy support=3.96±0.10).

Participant Attendance

Average attendance across the six sessions was 4.39±1.51 for intervention families (not including make-up sessions) and 4.92±1.48 (including make-up sessions). All make-up sessions lasted between 10-50 minutes, with an average duration of 28 minutes. Forty-three percent of families in the intervention condition (n=21) brought a “friend” to the “bring a friend” session. Of those, 52% were friends from school or neighbors; the other 48% were family members (e.g., siblings, cousins) of the target adolescent.

Participant Retention

A total of 124 families were enrolled in the study, 35 of which discontinued study participation before or during the run-in period. Notably, 80% of the these 35 families did not attend either of the run-in sessions, 17% attended one run-in session only, and 3% attended both run-in sessions. Of the 89 families randomized to a condition, 88% were retained and completed post measures. Follow-up phone interviews were conducted with 64% of parents (n=7) and 45% of adolescents (n = 5) from families who left the study. Health-related reasons (n = 6) and/or time conflicts (n = 5) were the most common reasons for study discontinuation. Other reasons included resource/transportation issues (n = 2), perceptions that the program was not beneficial (n = 2), family emergencies (n = 1), and/or school-related issues (n =1).

Participant Psychosocial Variables

Parent-Adolescent communication around health behaviors.

There was a significant intervention effect on parent-adolescent communication at post intervention (Table 3), such that parents in the intervention condition reported more positive communication at post-intervention than did those in the comparison condition (b=0.51, SE=0.15, t=3.40, p < .01). The only significant covariate in this model was parent-adolescent communication at baseline (b=0.37, SE=0.08, t=4.63, p < .01).

Table 3.

Multilevel Model Predicting Parent-reported Communication at Post-Intervention

Parameter	Estimate	(SE)	df	p	FMI	Lower CI	Upper CI
Intercept	1.59**	(0.11)	89	0.00	0.19	1.37	1.81
Female	0.01	(0.15)	89	0.94	0.13	−0.28	0.30
Age	−0.00	(0.05)	89	0.93	0.18	−0.10	0.10
Income	−0.05	(0.04)	89	0.19	0.23	−0.14	0.03
zBMI	−0.06	(0.06)	89	0.36	0.17	−0.18	0.06
Cohort1.con	−0.22	(0.23)	89	0.34	0.14	−0.68	0.24
Cohort2.con	−0.24	(0.22)	89	0.27	0.12	−0.67	0.19
Cohort3.con	−0.08	(0.27)	89	0.75	0.23	−0.61	0.44
Cohort4.con	−0.05	(0.23)	89	0.83	0.04	−0.49	0.39
Baseline Communication	0.37**	(0.08)	89	0.00	0.12	0.23	0.52
Treatment	0.51**	(0.15)	89	0.00	0.13	0.22	0.79

Note. SE = Standard error of the parameter estimate adjusted for the use of multiple imputations; df = Estimated degrees of freedom adjusted for use of multiple imputations and capped at sample size; FMI = fraction of missing information; CI = 95% confidence intervals; zBMI = Age and sex-standardized Body Mass Index

^**p < .01

^*p < .05

Parental management of peers.

There was no significant intervention effect for parental management of peers. The only covariate that significantly predicted parental management of peers at post was parental management of peers at baseline (b=0.31, SE=0.07, t=4.43, p < .01)

Post Program Interviews

Parents reported that among the most positive aspects of the program were the opportunity to interact with other families (61%), the health-related topics covered (61%), and the family-like atmosphere (50%). For example, one parent noted, “They put together a very friendly, family-like atmosphere for everybody so I think that was, that may have been the best thing.” With regard to what could be changed, parents generally said they liked the program the way it was (44%); those who provided suggestions, however mentioned their desire for a longer program (38%) and also for PA time to be built into sessions (17%). The majority of parents (72%) reported the program had a positive impact on their communication and overall relationship with their adolescent. Parents reported an increased use of listening skills and noted the program brought their family closer as a result of improved communication. For example, one parent noted, “You know, we were communicating before but I think I’m more in a listening kind of mode and digger deeper into what he’s saying.” Similarly, another parent said, “It became more of a, we discuss things and he would tell me how he feel, even though in the end my decision was final but he got to get his input in and it, and it made him feel more a part of what, what’s going on, you know.”

Adolescent participants reported that among the most positive aspects of the program were the health-related topics covered (56%) and the interactive nature of the sessions and activities (50%). For example, one adolescent noted “The best part, well I really like is when we do the ‘walk and talk.’ Usually sometimes we’ll just walk around the building or up and down the stairs and just talk about what, how’s our day going.” Like parents, they also reported liking the program how it was (44%) and had few suggestions for improvement, among them more time for PA or games during sessions (17%). Adolescents reported that the program improved family relationships by improving the family’s joint participation in PA and healthy dietary behaviors (56%) and by bringing family members closer together, such as through increased communication (38%). For example, one adolescent shared, “It has us eating healthier and doing more and being more interactive with each other. We kind of talk a little bit more. And we help each other work out and stay in shape.” Other representative comments included, “We came closer. Like we’re talking more,” and “Well, between me and my mom, we spending more time together, like exercising, you know… healthy eating, getting fit together.” Finally, one-third of adolescents reported self-improvements in their ability to make healthy choices when around their friends, such as “doing different things with my friends, like being more active and playing different sports with them.”

Discussion

Because findings of health promotion studies rely on interventions being delivered with fidelity, the identification and use of process evaluation tools is critical for future investigators. This study expands on past research by describing the process evaluation of Project SHINE, an innovative family-based health promotion intervention that integrated parent and peer systems through the use of behavioral, positive parenting, and peer monitoring skills for improving SB, PA, and diet in African American families. Results indicated the intervention was delivered with high fidelity based on facilitator adherence to program content and competent use of theoretically-based behavior change and positive parenting skills. Average participant attendance across the six intervention sessions (4.39±1.51) was high as was the 88% study retention rate; however, only 43% of youth brought a peer to the designated “bring a friend” session. Although there were no significant improvements in adolescent-reported parental management of peers, parents in the intervention condition reported significant improvements in communication with adolescents regarding engagement in health behaviors on their own and with peers. Qualitative themes related to improvements in family interactions (e.g., family communication, closer relationships, and joint engagement in healthy behaviors) also emerged in post-program interviews. Overall, this descriptive study serves as an example of how future family-based health promotion trials can expand their process evaluation approaches by assessing theoretically-relevant positive parenting variables as part of on-going monitoring.

Both parents and peers have been identified as primary socializing agents in the lives of adolescents, making them particularly influential in promoting positive health behaviors. Although evidence-based family-centered preventive interventions for reducing adolescent health risk behaviors (e.g., drug use) teach parents skills for enhancing family communication and monitoring adolescent peer relationships (Prado & Pantin, 2011), these strategies have been underutilized in health promotion. Because Project SHINE is one of the first to test the effects of this type of approach on youth SB, PA, and diet, examining process evaluation data related to program delivery as well as participant retention, engagement, and perceived changes in key psychosocial variables, may be particularly important to drawing accurate study conclusions. For example, because fidelity data suggest the intervention was delivered as planned, the probability of making a Type III error in this study (i.e., concluding a program is ineffective, when in fact it was inconsistently implemented) is low (Harachi, Abbott, Catalano, Haggerty, & Fleming, 1999). Significant improvements in parent-reported communication as well as qualitative themes related to improved family relationships were similarly suggestive of successful intervention delivery. In addition, and despite low engagement and retention rates often observed in interventions with ethnic minority families (Baranowski et al., 1990; Zeller et al., 2004), high participant attendance and retention rates suggest Project SHINE was well-received by participants.

Data from this study might also be used to inform larger trials that attempt to integrate families and peers for adolescent health promotion. For example, the low peer attendance rate at the “bring a friend” session indicates this component may not have been feasible for families. Similarly, the fact that nearly half of “peers” who attended this session were actually relatives (e.g., siblings, cousins) of the target adolescent suggests expanding the definition of “peers” to include extended family members may be a novel and potentially more practical way to involve other developmentally relevant systems. Future studies may additionally benefit from developing validated measures of health-related parent-peer constructs. For example, the lack of improvement in adolescent-reported parental management of peers may be explained by the fact that this measure assesses general peer management versus peer management as it relates to adolescent SB, PA, and diet. Finally, it will be important for larger trials to link process data more directly to psychosocial variables and study outcomes to further elucidate the role of different parent and peer-related intervention “ingredients.” Due to the small sample size and lack of variability in fidelity ratings, more sophisticated analyses with the process data were not feasible for this study. Another limitation of the present study includes the use of only one independent rater (rather than multiple raters) to assess intervention fidelity across sessions. Finally, although the communication scale developed for and used in this study had high internal consistency, further investigation of its psychometric properties is needed. Despite these limitations, however, the present study expands on previous literature by describing how an integrated theory may be used to inform study design and process evaluation within family-based approaches for health promotion.

Implications for Practice

Overall, this study provides a novel example of how to develop and utilize process evaluation tools that capture theoretically meaningful variables as they relate to intervention delivery. In particular, the tool developed to train facilitators may serve as an example for simplifying complex theoretical constructs into easy-to-recognize intervention elements. Similarly, the measure used to assess intervention fidelity may be readily adapted by practitioners in real-world settings who work with diverse populations. Although only African American families participated in the present study, because the measure of fidelity was used to assess facilitator behaviors, this tool may be relevant for other family-based health promotion interventions that target similar constructs. Specifically, process evaluation tools developed for this study may be useful in assessing how well positive parenting skills are implemented in health promotion programs that target parents as key agents of change for facilitating healthy adolescent family and peer contexts.