Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2007 Apr 1.
Published in final edited form as: J Spec Pediatr Nurs. 2006 Apr;11(2):80–89. doi: 10.1111/j.1744-6155.2006.00050.x

Enhancing Intervention Fidelity: A Means of Strengthening Study Impact

Sharon Horner 1, Lynn Rew 1, Rosamar Torres 1
PMCID: PMC1474027  NIHMSID: NIHMS10307  PMID: 16635187

Abstract

PURPOSE

Intervention studies that address the goals of the Healthy People initiative are increasing, and many focus on health concerns of children and adolescents. The purpose of this paper is to present strategies and discuss the rationale for including intervention fidelity in study design.

CONCLUSION

Researchers need to verify that their interventions were delivered as designed (intervention fidelity), and that variations from the design can be assessed. Infidelity can result in non-significant findings that are not due to the study design but rather to elements that affected the intervention delivery.

IMPLICATIONS

Addressing intervention fidelity in study design supports the researcher's conclusion about associations between intervention and outcomes and helps reduce premature abandonment of potentially useful interventions.

Search terms

Experimental studies, fidelity, interventions, research

Healthy People is a major public health initiative that was launched in the late 1970s to guide activities for promoting health and preventing disease (Ory, Jordan, & Bazzarre, 2002). In 1998, the National Institutes of Health implemented a policy to increase the inclusion of children in research studies of health concerns that affect them. As a result of these initiatives, intervention studies have been designed to address the different focal areas in the Healthy People reports, and many include children and adolescents in their samples. Integral to these interventions is the need to enhance intervention fidelity to ensure that internal validity of the study is maintained and external validity is enhanced. Two such health promotion intervention studies with school-age children and adolescents are being conducted by the first two authors who incorporated intervention fidelity strategies in their studies' designs.

Stated simply, intervention fidelity means that the intervention was conducted as planned. Adherence to the intervention design advances the study aims (Nigg, Allegrante, & Ory, 2002), supports the researcher's conclusions about the association between the intervention and the study outcomes (Calsyn, 2000), and helps reduce premature abandonment of potentially useful interventions (Nigg et al.). Bellg et al. (2004) acknowledged that “methodologic procedures for preserving internal validity and enhancing external validity in studies…are not emphasized in research-training curricula, and their relative lack of perceived importance is also evidenced by the scant reporting of treatment fidelity practices in journal articles” (p. 443).

While treatment fidelity was identified as early as the 1970s, it was during the 1990s that concern about the internal validity of intervention studies gained recognition (Bellg et al., 2004). Internal validity refers to what the investigator asserts about the effects of the intervention on the outcomes. In other words, are the outcomes observed due to the experimental manipulation, or are they due to other extraneous variables or processes (Polit & Hungler, 1999; Trochim, 2001)? When reviewing literature one finds that most of the articles that report on the intervention fidelity of their research are in the area of psychiatric mental health; more recently, however, researchers who study health promotion to prevent physical health problems are beginning to address the issue of fidelity as well. Examples of such health promotion studies with children and adolescents in which intervention fidelity has been incorporated as a measured variable include substance abuse prevention (Orwin, 2000), family grief therapy (Chan, O'Neill, McKenzie, Love, & Kissane, 2004), children's disruptive bedtime behavior (Burke, Kuhn, & Peterson, 2004), serious emotional disturbance in children (Epstein et al., 2003), promoting competence and reducing risk for conduct disorders in school-age children (Dumas, Lynch, Laughlin, Smith, & Prinz, 2001), and improving school children's nutrition (Davis et al., 2000). The purpose of this paper is to present strategies and discuss rationale for including intervention fidelity in the study design. Examples of intervention fidelity strategies from the first two authors' studies will be provided.

Threats to Internal Validity

When researchers draw conclusions about their studies' findings, they are basing these conclusions on inferences that the internal validity was maintained through procedures and processes built into the studies' designs. Internal validity refers to the degree to which a study's findings can be attributed to the intervention rather than other factors (Polit & Hungler, 1999; Santacroce, Maccarelli, & Grey, 2004). Conversely, researchers should acknowledge when other factors might explain the study results. These other factors are known as threats to internal validity (Polit & Hungler). Threats to internal validity can affect how participants respond to the intervention or to measures used in data collection. These include (a) history or the influence of events that are external to the study design on participants, (b) maturation that takes place within the participant over the course of the study, (c) testing effects where the participant learns from the instruments used or gains new insights through the process of responding to questions, (d) instrumentation effects occur when data collection procedures or measurement tools change over the course of the study, (e) selection effects refer to preexisting differences between groups, and (f) mortality, which refers to differential loss of participants or attrition from the groups being compared in the study (Polit & Hungler).

In intervention research one should also consider the social threats to internal validity. These arise when the intervention is carried out in a real-world context where participant interactions can also influence the outcomes (Trochim, 2001). These social threats to validity include diffusion or imitation of treatments, compensatory rivalry, resentful demoralization, and compensatory equalization of treatment (Trochim). The first three social threats occur in those study designs in which the participants in the control group know about or become aware of the intervention. They may respond by imitating what they believe happens in the treatment group (i.e., diffusion or imitation of treatment), or increase their efforts to perform well in comparison to the treatment group (i.e., compensatory rivalry), or they may become resentful and demoralized and thus perform worse on the outcomes than they otherwise would (Trochim). The fourth social threat occurs when participants pressure the researchers to assign them to the opposite group (treatment or control group) from the one to which they were originally assigned. This has the effect of equalizing the two groups and minimizing treatment effects (Trochim).

Researchers design their studies to minimize the impact of or control for these internal threats to validity. Variance in how the treatment or intervention is delivered is a significant threat to internal validity (Calsyn, 2000). The risk of variance, or deviations from the planned study design, increases as the intervention design becomes more complex (when multiple interventionists [e.g., group leaders, health educators, lay or professional volunteers] are needed to implement the intervention) and when the intervention is offered many times to different groups of participants.

Simple interventions are those that can be provided in one session to the participants. Interventions become more complex as the number of sessions per intervention increases, as the intervention is offered multiple times to more participants, as different group configurations are incorporated in an intervention, as the group becomes more heterogeneous (Kerns & Prinz, 2002), and as location of the intervention delivery varies (Soldano & Markell, 1997). In general, interventions are designed as a series of sessions to allow time for the participants to develop skills and increase knowledge and comprehension (Kerns & Prinz). As the intervention design becomes more complex, the risk for variation in the intervention delivery increases as well (Santacroce et al., 2004). An intervention can be delivered to individuals, families, small groups of non-related individuals, or any combination of these participant configurations (Kerns & Prinz; Santacroce et al.). Variations across the sessions can occur as the interventionist makes adjustments in the protocol based on assessment of earlier sessions or, conversely, unplanned changes that take place as the interventionist “drifts” from the protocol (Bellg et al., 2004; Nigg et al., 2002).

Other sources of variation include participants' interactions in the sessions that may be disruptive and contextual factors such as the location of the intervention site and the schedule of sessions (Soldano & Markell, 1997). For example, unruly, overly demanding, or highly garrulous participants can disrupt group sessions, resulting in a lack of participation by other group members (Morgan, 1997; Orwin, 2000). As the interventionist attempts to address the needs of these individuals, the objectives of the session can be subverted, resulting in a loss of intervention session or delivery of a different treatment from what was planned (Morgan).

The contextual factors of location, schedule, and transportation are especially pertinent when one is working with underserved populations. The authors' studies focus on rural families of children with asthma and homeless adolescents — both are underserved populations. Issues of transportation availability, schedule limitations based on employment patterns, inaccessible clinics, and participant perception of safety in traveling to and from the clinic site all influence the participants' willingness and availability to attend intervention sessions (Price, Kirkland, & Kreisler, 1994; Soldano & Markell, 1997). In an effort to make their interventions as inclusive and available as possible, the authors travel to their participants' locations (e.g., homeless clinic or shelter, rural homes and schools) and offer the interventions at different times (e.g., afternoons, evenings) and days of the week to meet their study aims of improving the health of these populations.

To summarize, internal validity is generally seen as contributing to the soundness of the study findings (Santacroce et al., 2004), and as such, can be enhanced by careful attention to maintaining the integrity of the intervention delivery across multiple sessions or between different groups (Dumas et al., 2001). Confidence in study findings is increased when strategies for improving internal validity have been incorporated into the study design.

Strategies for Enhancing Intervention Fidelity

Investigators incorporate strategies into their study designs to ensure that all members of the treatment group receive all the essential elements of the intervention (Bellg et al., 2004) and that the intervention is delivered in a comparable manner to the participants (Dumas et al., 2001; Kerns & Prinz, 2002). The primary focus of intervention fidelity strategies is on the interventionist. Researchers who are designing intervention fidelity strategies should address the training provided to the research team members who will deliver the intervention and incorporate monitoring procedures to evaluate intervention fidelity.

Intervention fidelity strategies will be discussed, and examples from the first two authors' studies will be presented. The first author is conducting a health promotion study to improve family asthma management. The intervention consists of 16 small-group sessions provided in schools to rural school-age children who have asthma and followed by one individualized home-visit session to the family. Hereafter this first study will be referred to as the asthma management intervention. The second author is conducting a health promotion study to reduce risky sexual behaviors among homeless adolescents. The intervention consists of eight sessions provided to gender-specific groups of homeless adolescents at a community health clinic that provides a variety of services to homeless youth. Hereafter this second study will be referred to as the sexual health intervention.

Intervention fidelity was incorporated into these studies' designs from the beginning. Strategies for enhancing intervention integrity included developing detailed manuals of intervention content and activities, providing training to interventionists, and monitoring the ongoing intervention sessions.

Manuals

Developing manuals that spell out the program purpose, goals, session objectives, and essential or critical elements, such as behaviors that are to be role modeled by interventionists or the content that must be covered, can standardize the training of interventionists (Bellg et al., 2004). Well-developed intervention manuals or protocols are the gold standard of intervention therapies (Kerns & Prinz, 2002). The elements covered in the manual form the basis for evaluation of fidelity. A manual should contain detailed information about each session that includes the objectives, specific content, and materials used in each session. The manual should also indicate how much time is allotted to cover each bit of content, what behaviors are to be demonstrated or role-played, and what strategies are used to check participants' understanding during the session. Each of these elements can then be monitored to insure fidelity.

Manuals were developed by the investigators for the small group sessions for both of these projects. The authors' manuals identify each study's purpose, goals or objectives for each session, a list of topics to cover, behaviors or skills to be practiced in the individual sessions, and materials needed (Kerns & Prinz, 2002; Santacroce et al., 2004). Specific content for the respective manuals was based on the conceptual frameworks guiding each of these studies (Nigg et al., 2002).

The conceptual framework for an intervention study clearly identifies the concepts or constructs included in the intervention. Such concepts may include specific content that is to be delivered and specific outcomes that are to be achieved. For example, a number of interventions that are designed to change people's behaviors are based on social cognitive theory (Bandura, 1986). According to this theory, human behavior is goal-directed and is influenced by watching other people. Behaviors are also influenced by how one reflects on one's ability to perform that behavior. This concept is known as self-efficacy and may be a measurable outcome of a behavioral intervention based on social cognitive theory. Using social cognitive theory (SCT) as a framework for an intervention, the investigator would clearly identify in the study manual the content, instructional strategies, and behavioral outcomes consistent with the concepts that comprise SCT.

For example, in the asthma management intervention, the focus is on children's skills, knowledge, self-efficacy, and problem-solving as they face commonly occurring situations (e.g., asthma episodes on the school bus, when playing sports, or in the classroom). The manual is divided into the content to be provided over the course of the intervention and includes problem-solving scenarios, and skills practice with metered dose inhalers and peak flow meters; self-efficacy and knowledge about asthma are incorporated throughout the intervention. The manual has a list of supplies needed for each of the children's sessions, the session objectives, and specific content to be covered in each session. Woven throughout the manuals are directions (specific instructions) or suggestions for handling different interactions in the group. Ways to handle disruptive behaviors are identified and are developmentally appropriate for the participants: Redirecting the disruptive child by assigning the child a specific task like helping gather materials or writing down the class responses on the board are appropriate strategies for this age group.

In the sexual health intervention, the focus is on developing healthy sexual behaviors, including assertive communication, self-care, and personal decision-making. The manual has specific instructions for each of the eight sessions, including providing specific information about a topic such as assertive communication, activities to personalize the information to the participant's experience, and role-playing to reinforce the learning. The content and activities delineated in the manual are clearly aligned with the major variables in a conceptual framework developed to guide sexual health practices in homeless adolescents (Rew, 2001). Guidelines for handling disruptive behaviors of the homeless adolescents that might occur during the group sessions are built on a respect for their autonomy and accountability for their own behaviors. Each intervention, consisting of eight sessions, begins with the group identifying ground rules for group behavior. At the beginning of each session, group leaders review the ground rules.

Training

Training of the interventionists was provided in both studies. Research team members bring a wide range of skills and experiences to a study and this can influence the manner in which the intervention is delivered. The study planners must consider the interventionists' skill sets — what they bring to the project (Bellg et al., 2004). Variance in intervention delivery can be reduced by having a high degree of structure in the intervention design (Kerns & Prinz, 2002). Well-defined protocols can increase the comfort of less experienced staff in implementing the intervention. In contrast, highly experienced staff or those who have considerable knowledge in the problem area may feel overly constrained by a highly structured protocol and “itch” to modify the intervention to fit their philosophy or prior training (Bellg et al.; Kerns & Prinz). Strategies for achieving conformity across multiple interventionists include training for intervention delivery in which (a) the philosophy of the program is discussed, (b) objectives and intervention procedures are covered in detail, and (c) the interventionists have an opportunity to practice the necessary skill sets needed for the intervention (Bellg et al.; Kerns & Prinz).

Asthma Management Intervention Training

The asthma management intervention had two different types of interventionists: registered nurses provided the home visit education, and lay health educators provided the small class sessions in schools to the children.

Lay persons, nominated by the school principals or nurses because they served as tutors and substitute teachers in their respective schools, taught the 16 small-group children's classes. The decision to use lay persons for this component of the intervention was to improve the potential for translation of the intervention into a school program (Leventhal & Friedman, 2004). The manual for the in-school classes contained specific directions and content for each day's session, including the amount of time needed to cover the topics or practice skills. Training of the class teachers included reviewing the manuals, presenting each class session while using the appropriate class materials, time for self-study, and then holding a final follow-up meeting for clarification of content and return demonstration of different elements of the classes.

The training of the registered nurses for the home visit education consisted of educating the nurses about asthma pathophysiology, treatment, environmental management, problem-solving for common asthma-related situations, and the use of asthma action plans. Required elements of the home visit education included (a) reviewing the asthma action plan and assisting the families to fill in the written plan with their prescribed treatments, (b) teaching peak flow meter assessment and discussing how to use this information when deciding what steps to take based on the asthma action plan, and (c) reviewing environmental management for reducing asthma triggers. The home visit education was further individualized by reviewing specific aspects of care that affect the individual child and by answering the families' questions.

Sexual Health Intervention Training

The second author's study had group leaders who were educated in the fields of public health, nursing, social work, and psychology. The group leaders not only had to be knowledgeable about sexual health topics but comfortable discussing them. Training consisted of didactic instruction that addressed the uniqueness of the study population and how to effectively communicate with homeless adolescents. Role-play sessions were included in the training and were instrumental in providing the interventionists with strategies for dealing with difficult participants and improving their facility in discussing sensitive topics such as the disclosure of childhood sexual abuse or drug use during pregnancy. Interventionists evaluated the training sessions through a post-training evaluation questionnaire. The attention to the training the interventionists received laid the groundwork for intervention fidelity monitoring.

Monitoring Intervention Fidelity

Ongoing process evaluation was included in both interventions. Regular monitoring of intervention delivery by using a theoretically based measure of fidelity allows researchers to identify variance from the studies' planned procedures or content and make corrections over the course of intervention delivery (Baranowski & Stables, 2000; Nigg et al., 2002). Another advantage of monitoring intervention fidelity with a structured instrument is that the resultant measure can be used as a variable to evaluate the program outcome (Calsyn, 2000; Nigg et al.).

Two health promotion projects with school children funded by the National Cancer Institute exemplify the impact of monitoring intervention fidelity on study outcomes. Both projects involved elementary school children who were provided with curricula designed to increase fruit and vegetable consumption. The Gimme 5 project attained a 0.2 increase in daily servings of fruits and vegetables consumed as compared to the 1.68 increase in daily servings attained by the High 5 project (Baranowski et al., 2000; Davis et al., 2000; Reynolds et al., 2000). Classroom monitoring revealed that only 51% of the fourth grade curriculum and 46% of the fifth grade curriculum was actually delivered in the Gimme 5 project (Davis et al.), which is in stark contrast to the 90% of the curriculum delivered in the High 5 project (Reynolds et al.). By using standardized evaluation instruments and procedures that incorporate essential components of the interventions, these research teams were able to evaluate the efficacy of their interventions to effect behavioral change in the children and the school environments (Calsyn, 2000).

Sexual Health Intervention Monitoring

Every session of the sexual health intervention was monitored by a group co-leader who audiotape-recorded the session. In addition, the co-leader took notes and completed a checklist that included the specific topics to be covered in that day's session. The checklist was developed to reflect the conceptual framework and to insure that each topic was covered as detailed in the intervention manual. After the session, the co-leader listened to the audiotape and reviewed the checklist to verify that topics had been covered as outlined in the manual. In this manner, variance from the planned intervention could be quickly identified before the next session where it could be rectified as needed (Dumas et al., 2001).

Participant evaluation for the group sessions was included in the process evaluation by having the participants complete an evaluation form after every two sessions. The participant evaluation also served to track the content delivered by the interventionists. Team meetings were held weekly after the first and second intervention sessions to provide feedback on the interventionists' adherence to the intervention manual (Waltz, Addis, Koerner, & Jacobsen, 1993). As the interventionists displayed greater ease and skill with leading groups, these team meetings tapered off to monthly rather than weekly meetings.

Intervention-Monitoring Instruments

Intervention-monitoring instruments can use quantitative and qualitative criteria with general process items, specific content items, or a combination of both (Dumas et al., 2001). These fidelity measures can have dichotomous scales (present/absent), ordinal scales (none, adequate, excellent), or rating scales that require considerable inter-rater training and evaluation of inter-rater reliability (Dumas et al.; Orwin, 2000; Santacroce et al., 2004). For example, in the asthma management intervention, an intervention-monitoring guide was used to verify that the children's class content was covered and that the interactions that took place in each session were supportive of the children's learning. The guide has 12 items that address (a) the class topic, including the content to be covered, materials used, and length of time for topics; and (b) the children's responses in terms of their interest in the topic, participation in session activities, and whether their questions were asked and answered. Each item was assessed with a 3-point ordinal scale with 1 for “not covered,” 2 for “adequately covered,” and 3 for “excellently covered” as the rating scale. Ratings of “adequately covered” and “excellently covered” both indicate that the content presented was accurate and complete; the distinction between the two ratings is a quality difference in the instructor's communication style in presenting information and responding to the children in the class. Such a qualitative assessment is very useful for providing feedback to the instructor.

The procedure for intervention monitoring had a member of the research team visit 5 of the 16 class sessions for every lay teacher and observe those sessions. The monitor would score the observed session for class-objective adherence. The scores for each item were summed (possible range of 12–36) and yielded an instructor's score for that day. All items that the monitor rated as “not covered” were reviewed with the teacher before the next class session. The scores for the five monitored sessions can be summed and averaged for each teacher and this becomes an intervention score for each child in the teacher's group. This intervention score becomes one of the variables in evaluating the study outcomes (Santacroce et al., 2004). In this way variance between interventionists can be addressed as another element of intervention efficacy. This is critical if the researcher wants to assert that significant study outcomes were due to the intervention and not due to some other unmeasured factor (Bellg et al., 2004).

Including measures of intervention fidelity in the outcome analyses enables the researcher to sort out the variance in outcomes that are due to non-adherence or partial adherence to the intervention design (Bellg et al., 2004). These are measures of the dosage effect or how much of the intervention was actually delivered to the different participants (Santacroce et al., 2004). As such, the intervention fidelity data can contribute to refining the intervention design in future studies (Bellg et al.; Nigg et al., 2002). Monitoring measures may, themselves, constitute data that can be analyzed to determine dosage effects. Information from monitoring instruments can also be used in the data interpretation phase of an intervention study and for helping the investigator to interpret expected or unexpected outcomes of the intervention. Monitoring data may provide an objective measure of the validity of the intervention.

Conclusion

Evaluation of intervention fidelity is a key element of the study design. It must be addressed early in study development to yield useful processes and instruments for ongoing evaluation of the intervention, both during and after the study is implemented. As Nigg and colleagues (2002) point out, “verification of treatment fidelity is integral to both the interpretation and generalization of the research findings” (p. 674). These early decisions about the use to be made of intervention fidelity evaluation (e.g., monitoring and feedback for interventionists, or statistical factor) will enhance the quality of the study (Bellg et al., 2004). Furthermore, as the study proceeds and data are gathered and analyzed, this analysis can contribute to the planning of future interventions. For example, the number of sessions attended by a participant (Santacroce et al., 2004), the amount of content delivered (number of topics covered: Baranowski & Stables, 2000), and even participant satisfaction with the intervention can be measured and incorporated into the outcome evaluation (Burke et al., 2004).

Intervention fidelity is an integral component of the study design. Validity of the outcomes of intervention studies is based in large part on the degree to which the intervention fidelity is maintained and evaluated (Kerns & Prinz, 2002). Attention to the internal validity of the study procedures can strengthen confidence that the study outcomes are indeed related to the intervention.

How Do I Apply This Information to Nursing Practice?

This information is pertinent for research studies and clinical service projects where interventions are provided to participants or consumers and data are collected pertaining to the impact of the intervention on some selected outcomes. As nurses plan their interventions, manuals or instructor guidelines need to be developed, training sessions for the interventionists planned, and intervention-monitoring tools selected or adapted for use with the project. Generally, many people are involved in implementing these projects, and the nurse planners need to demonstrate the effectiveness of their interventions to the persons or groups who funded the project, to their colleagues and supervisors, and other healthcare professionals. Incorporating strategies for maintaining and enhancing intervention fidelity as the intervention is delivered standardizes the intervention, reduces deviations from the plan, and increases confidence in the integrity of the project outcomes. Equally important is the need to report procedures used and findings of intervention fidelity monitoring in presentations and written reports of these projects. Reports that include results of intervention monitoring are useful to consumers of research as they evaluate the efficacy of an intervention and assist them in deciding on whether or not to adopt the intervention for their clinical practice.

Acknowledgment

Funding for this manuscript was provided from the National Institutes of Health, National Institute for Nursing Research to the first author (R01 NR007770), to the second author (R01 NR04820), and to the third author (T32 NR07106).

References

  1. Bandura A. Social foundations of thought and action: A social cognitive theory. Prentice Hall; Englewood Cliffs, NJ: 1986. [Google Scholar]
  2. Baranowski T, Davis M, Resnicow K, Smith M, Lin L, Baranowski J, et al. Gimme 5 fruit, juice, and vegetables for fun and health: Outcome evaluation. Health Education Research Quarterly. 2000;27:96–111. doi: 10.1177/109019810002700109. [DOI] [PubMed] [Google Scholar]
  3. Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Education & Behavior. 2000;27:157–166. doi: 10.1177/109019810002700202. [DOI] [PubMed] [Google Scholar]
  4. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from NIH Behavior Change Consortium. Health Psychology. 2004;23:443–451. doi: 10.1037/0278-6133.23.5.443. [DOI] [PubMed] [Google Scholar]
  5. Burke RV, Kuhn BR, Peterson JL. Brief report: A “storybook” ending to children's bedtime problems — The use of a rewarding social story to reduce bedtime resistance and frequent night waking. Journal of Pediatric Psychology. 2004;29:389–396. doi: 10.1093/jpepsy/jsh042. [DOI] [PubMed] [Google Scholar]
  6. Calsyn RJ. A checklist for critiquing treatment fidelity studies. Mental Health Services Research. 2000;2:107–113. doi: 10.1023/a:1010109205676. [DOI] [PubMed] [Google Scholar]
  7. Chan EKH, O'Neill I, McKenzie M, Love A, Kissane DW. What works for therapists conducting family meetings: Treatment integrity in family-focused grief therapy during palliative care and bereavement. Journal of Pain and Symptom Management. 2004;27:502–512. doi: 10.1016/j.jpainsymman.2003.10.008. [DOI] [PubMed] [Google Scholar]
  8. Davis M, Baranowski T, Resnicow K, Baranowski J, Doyle C, Smith M, et al. Gimme 5 fruit and vegetables for fun and health: Process evaluation. Health Education & Behavior. 2000;27:167–176. doi: 10.1177/109019810002700203. [DOI] [PubMed] [Google Scholar]
  9. Dumas JE, Lynch AM, Laughlin JE, Smith EP, Prinz RJ. Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the Early Alliance prevention trial. American Journal of Preventive Medicine. 2001;20:38–47. doi: 10.1016/s0749-3797(00)00272-5. [DOI] [PubMed] [Google Scholar]
  10. Epstein MH, Nordness PD, Kutash K, Duchnowski A, Schrepf S, Benner GJ, et al. Assessing the wraparound process during family planning meetings. Journal of Behavioral Health Services & Research. 2003;30:352–362. doi: 10.1007/BF02287323. [DOI] [PubMed] [Google Scholar]
  11. Kerns SEU, Prinz RJ. Critical issues in the prevention of violence-related behavior in youth. Clinical Child and Family Psychology Review. 2002;5:133–160. doi: 10.1023/a:1015411320113. [DOI] [PubMed] [Google Scholar]
  12. Leventhal H, Friedman MA. Does establishing fidelity of treatment help in understanding treatment efficacy? Health Psychology. 2004;23:452–456. doi: 10.1037/0278-6133.23.5.452. Comment on Bellg et al., 2004. [DOI] [PubMed] [Google Scholar]
  13. Morgan DL. Focus groups as qualitative research. 2nd ed. Sage; Thousand Oaks, CA: 1997. [Google Scholar]
  14. National Institutes of Health http://grants1.nih.gov/grants/guide/notice-files/not98–024.html. Policy and guidelines on the inclusion of children as participants in research involving human subjects. 1998 Retrieved August 11, 2005, from.
  15. Nigg CR, Allegrante JP, Ory M. Theory-comparison and multiple-behavior research: Common themes advancing health behavior research. Health Education & Research. 2002;17:670–679. doi: 10.1093/her/17.5.670. [DOI] [PubMed] [Google Scholar]
  16. Orwin RG. Assessing program fidelity in substance abuse health services research. Addiction. 2000;95(Suppl 3):S309–S327. doi: 10.1080/09652140020004250. [DOI] [PubMed] [Google Scholar]
  17. Ory MG, Jordan PJ, Bazzarre T. The behavior change consortium: Setting the stage for a new century of health behavior-change research. Health Education & Research. 2002;17:500–511. doi: 10.1093/her/17.5.500. [DOI] [PubMed] [Google Scholar]
  18. Polit DF, Hungler BP. Nursing research: Principles and methods. 6th ed. Lippincott; Philadelphia: 1999. [Google Scholar]
  19. Price PC, Kirkland R, Kreisler AL. Access to health care for Texas children. Texas Medicine. 1994;90(6):71–78. [PubMed] [Google Scholar]
  20. Rew L. Sexual health practices of homeless youth: A model for intervention. Issues in Comprehensive Pediatric Nursing. 2001;24(1):1–18. [PubMed] [Google Scholar]
  21. Reynolds KD, Franklin TA, Leviton LC, Maloy J, Harrington KF, Yaroch AL, et al. Methods, results, and lessons learned from process evaluation of the High 5 school-based intervention. Health Education & Behavior. 2000;27:177–186. doi: 10.1177/109019810002700204. [DOI] [PubMed] [Google Scholar]
  22. Santacroce SJ, Maccarelli LM, Grey M. Intervention fidelity. Nursing Research. 2004;53:63–66. doi: 10.1097/00006199-200401000-00010. [DOI] [PubMed] [Google Scholar]
  23. Soldano C, Markell G. Parent involvement in health concerns in youth: The issue of adolescent immunization. Journal of School Health. 1997;67:292–293. doi: 10.1111/j.1746-1561.1997.tb03455.x. [DOI] [PubMed] [Google Scholar]
  24. Trochim WMK. The research methods knowledge base. 2nd ed. Cornell University; Cincinnati, OH: 2001. [Google Scholar]
  25. Waltz J, Addis ME, Koerner K, Jacobsen NS. Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology. 1993;61:620–630. doi: 10.1037//0022-006x.61.4.620. [DOI] [PubMed] [Google Scholar]

RESOURCES