Understanding "Agency” in the Translation of a Health Promotion Program

Abstract

Health promotion interventions conducted under “ideal conditions” to prove their efficacy are often difficult to translate and disseminate for utilization in “real-world” settings. This article retrospectively integrates and analyzes the experience of three related projects. We investigate how the development and dissemination of a school-based nutrition and physical activity curriculum for American Indian elementary school children inspired the implementation of an across-the-lifespan train-the-trainer program that has trained more than 600 trainers in American Indian communities nationwide. This process provides an opportunity to explore how individuals in the community and the context in which the research was conducted affected project outcomes in ways which were not anticipated. Results challenge the use of “internal validity” as the primary measure of success in translation–dissemination–utilization research.

Introduction: The Troubling Gap Between Research and Practice

Health promotion and disease prevention efficacy trials are usually conducted under ideal research conditions with grant funding, trained staff, and university resources. Translating research into real-world settings that often are resource poor, understaffed, and required to address competing priorities is challenging (Glasgow and Emmons 2007; Green et al. 2009; Trickett et al. 2011; Rychetnik et al. 2012). Estimates that less than half of health research findings are ultimately utilized outside of controlled conditions have led what was previously referred to as a “gap” between research and practice, to be recognized as a “chasm” (Glasgow and Emmons 2007). Given weak results being obtained in getting research-based strategies for prevention into use in communities, there are calls for re-conceptualizing the current research paradigm in public health (Glasgow and Emmons 2007; Green et al. 2009). Increasingly, it has been recognized that approaches that have been developed and that are useful for investigation in the biological and clinical/medical realm do not sufficiently accommodate the role of people and communities in public health (Green and Glasgow 2006). While “internal validity” may provide evidence for potential “best practice” design of community interventions, seemingly positive outcomes may actually ignore contextual factors and human dynamics that affect dissemination and use in the real world. Thus, attention to internal validity does not necessarily speak to “ecological validity” and as such, may not tell us what we need to know in order to develop meaningful public health interventions. However, the clinical/medical model prioritizing internal validity and eschewing the flexibility often required to make things work “in the real world” continues to be the foundation for much design, development, implementation, and assessment of outcomes in the research translation, dissemination, and utilization enterprise.

Re-conceptualizing research to address the challenge presented by this contradiction requires new and more sophisticated approaches to expand the boundaries of existing scientific focus to be more meaningful for turning knowledge derived from research into action in communities (Woolf 2008). Rychetnik et al. (2012) suggest that we need a more comprehensive understanding of the program adoption process to understand the interface between knowledge and evidence-based practice. We believe that this involves moving beyond the project-specific lens that is the conventional focus in public health research. In this paper, we argue that a holistic conceptual approach that uses theory to integrate the way we think about the research process can assist in this transformation. We use an “agency”-based theoretical framework to understand how people influence the dynamics and outcomes of research across three separate projects. We focus on the interplay between individual and structural factors involved in the design and implementation of research as a way of unifying our analysis. This theoretical framework allows us to conceptually integrate processes and outcomes across artificial project-specific boundaries. Such an approach can help us move toward re-conceptualization of the research process with the potential to improve our understanding and valuation of ecologically valid outcomes.

The discussion here is an ex post facto case study to explore how the development of Pathways, a school nutrition and physical activity intervention with American Indian elementary school children, and its dissemination through Participatory Action for Healthy Lifestyles (PAHL) is related to the subsequent unanticipated evolution of the Pathways intervention, and the adaptation, institutionalization, and widespread dissemination and utilization of the Physical Activity Kit (PAK), a train-the-trainer program based on the original Pathways components. Evidence from this experience demonstrates a novel way to think about research outcomes, including unexpected consequences, and to explain roles and contexts that affect translational processes (see Fig. 1). Embedding analysis of three related projects within one conceptual framework is a novel approach that we believe enhances our understanding of processual factors influencing prevention research. A unified analytical framework integrates multiple levels of data and provides for a more refined analysis of dynamics that would normally be considered separately within the context of distinct projects seen as having discrete processes and outcomes. This framework allows us to conceptualize the ways that the “agency” of stakeholders involved in implementation of an intervention can influence outcomes and provides a novel way of thinking about expectations and “success.” Using this expansive view of the research, synthesized interpretations of dynamics, processes, and outcomes were developed and conclusions are proposed.

Fig. 1.

Conceptual schematic for using a holistic, agency-based theoretical framework to integrate our understanding of the research process

Theoretical Framework: Acknowledging the Agency of People in the Process

Part of the challenge of re-weighting validity in research is the nature of contextual factors and human dynamics themselves. Health promotion programs are especially challenging to translate in community settings because of the complex nature and “unpredictability” of social and contextual factors that affect outcomes. Green (2001) suggested that it is necessary to recognize that “people” have to be factored in when considering the process of getting people to actually use the research—an apparently self-evident fact that seems to be dismissed as an artifact in much translational research with a focus on internal validity. He argues that to be successful, public health practitioners need to develop best practices that “emphasize a process of enabling people to command their own unique or tailored interventions to fit their own perceptions of need and their own circumstances, and to develop their own capabilities” (p. 174). This necessarily requires incorporating community-based or community-engaged participatory approaches that involve community members in designing, implementing, and evaluating the research (Trickett et al. 2011). Later, Green et al. (2009) further proposed that if we intend to break out of the current mold, it is incumbent upon researchers to package scientific knowledge in a manner more explicitly based on the needs, interests, and desires of the people who are ostensibly the end users.

One of the unique challenges posed by a participatory approach is that the process involves active engagement and “agency” (Giddens 1984) of community members, and requires researchers to acknowledge and incorporate community member input. The “agency”/”structure” debate in social theory examines the extent to which individuals or collectivities are constrained by social–structural contextual forces (such as cultural dynamics, socioeconomic factors, or existing institutional policies and practices) or are able to exert influence to change or transform both structural constraints and specific outcomes. In public health research, this means that the “agency” of individuals involved in the research shapes the intervention in ways that are not entirely controlled by researchers, nor is the process necessarily predictable, a fact not explicit or sufficiently accounted for in the current research paradigm.

At the same time, over the past decade, participatory and community-engaged research has gained acceptance as a strategy for involving community members in the design, implementation, and analysis of research in a manner that can positively influence the rate and depth of intervention uptake (Trickett et al. 2011; Wallerstein and Duran 2010). Participatory research design such as that used in Community Based Participatory Research (CBPR), can address powerlessness and lack of control over destiny that have emerged as risk factors for health disparity (Wallerstein 1992). However, despite a certain enthusiasm for community engagement and participation, there are significant challenges to incorporating this approach in health research (Hicks et al. 2012). Although a culture of participatory research has been developed in health promotion and disease prevention research, questions remain about the consequences of knowledge production in participatory health research (Mantoura and Potvin 2012) and about the nature of the participatory enterprise itself (Ponic and Frisby 2010). A fuller understanding is needed regarding how participatory processes are actualized, and in terms of the “agency” of people and groups that affect outcomes. Participatory research has matured to a point where further development of crucial issues and processes is both possible and necessary.

This paper explores these issues through the Giddensian “structurational” (Giddens 1984) interplay between the “agency” of individuals involved in the research setting and the structural contingencies of research design. Structuration posits that in any social context, both agency, and structure operate in a dual but dialectical relationship. The dynamics of this relationship influence social processes and the nature of social practices. Research studies in public health traditionally prioritize structural considerations consistent with an emphasis on internal validity. We propose that analyses in research can be improved by incorporating an agency-based theoretical framework to produce a more integrated and holistic analysis. This approach captures the dialectical interaction between specific local-level processes and broader structural forces, providing for a deeper understanding of dynamics internal to research at multiple levels and across time. This enhanced clarity is valuable for analyzing the research process and for developing future research approaches that further acknowledge and value ecologically valid outcomes.

Development of an Agency-Based Theoretical Framework

The information presented in this paper is based on a synthesis of data from the PAHL project with ex post facto researcher and institutional partner critical analysis of experience with Pathways, PAHL, and PAK. Because the discussion involves multiple acronyms, we have created an acronym key to assist the reader (see Table 1). Pathways and PAHL had both been significant projects for the Prevention Research Center at the University of New Mexico (UNM PRC), but turnover of key project staff had made it difficult to publish research results from PAHL (2002–2005). In 2010, the UNM PRC increased capacity for qualitative research with the addition of an anthropologist to our team and we were interested in further developing our analysis of PAHL data. We recognized that although often under-appreciated in health research, an anthropological lens can offer innovative perspectives (Krumeich et al. 2001; Lambert and McKevitt 2002; Rychetnik et al. 2012; Unger and Schwartz 2012). The hallmark of anthropology is the holistic approach epitomized in ethnography. This holism keeps both macro and micro-level factors in focus, often allowing for a rethinking of boundaries that define a particular problem. In addition, anthropology tends to use theory to create a conceptual understanding of social processes that are often unacknowledged, believed to be unconnected, or poorly understood in health research. An anthropological perspective, therefore, seemed promising for examining the experience and outcomes of PAHL.

Table 1.

Acronym key

Acronym	Referent
CATCH	Coordinated Approach to Child Health
CBPR	Community-based participatory research
DOH	New Mexico Department of Health
IHS	Indian Health Service
PAHL	Participatory Action for Healthy Lifestyles
UNM PRC	University of New Mexico Prevention Research Center
SP Team	Strategic Planning Team
RCT	Randomized Control Trial
PAK	Physical Activity Kit

Because a significant amount of time had elapsed since the end of the project, it was necessary to re-acquaint or introduce ourselves to the data. We began with a process of iterative analytical discussions with current and former members of the research team which included investigators and staff from the UNM PRC and institutional partners at Indian Health Service (IHS) to examine their experience with and perspectives on PAHL. We contextualized our discussions with a review of existing preliminary analyses of unpublished project data from site visits to schools, observations of trainings and classroom implementation, field reports from schools, research team debriefings and interviews with teachers, principals, Department of Health (DOH) staff, and other stakeholders, project meeting notes, and supporting documents. We were also privy to data from Pathways and knowledge of the evolution of the PAK initiative that occurred subsequent to the conclusion of the PAHL research as described below.

Insights from this process of review and the holistic theoretical anthropological perspective we applied to our understanding of the data led us to reconsider the project-specific lens we had been using to explore the experience and outcomes of PAHL. The deeper we went into consideration of the data, the more it became clear that the three projects were connected, not merely as part of a natural, sequential flow of research and implementation, but conceptually. This intrigued us since our inclination was to strictly confine our analysis to stay within the boundaries of project data. Although we attempted to use existing conceptual models to structure our understanding and integrate data across the three projects, we ultimately realized that the most salient way to conceptualize this data was by using a theoretical framework. The integrated discussion presented below is predicated on our sense that the most interesting and important contribution of PAHL is less about the specifics of disseminating a school-based obesity prevention intervention in American Indian schools than about what we can learn from the process. We use the concept of “agency” from social theory to co-locate these projects and to explore how people and context influence health promotion research (Frolich and Potvin 2010). We believe that this integrated, theoretically framed approach provides a novel perspective on the ongoing epistemological debates regarding internal and ecological validity in public health research.

The Pathways RCT

Pathways was a large-scale, multi-site randomized control trial (RCT) funded by the National Heart, Lung, and Blood Institute, to prevent obesity by promoting healthful eating habits and increasing physical activity through a four-component nutrition and physical activity curriculum for elementary schools teaching American Indian children. The 8-year study, from 1992 to 2000, involved 7 Indian Nations, 5 universities including the UNM PRC, and 41 elementary schools with 1,705 American Indian third- through fifth-grade students (Davis et al. 2003; Stone et al. 2003). Pathways promoted behavioral and environmental approaches to preventing childhood obesity through four, integrated components (school curriculum, physical activity enhancements, school food service improvements, and a family component). Although Pathways was an efficacy trial, its design was structured to integrate features that would facilitate translation, including a feasibility study (Gittelsohn et al. 1998) and a comprehensive process evaluation component (Steckler et al. 2003). Pathways required an extensive formative assessment that was conducted in partnership with tribal communities and was designed to fit these communities (Gittelsohn et al. 1998; Davis et al. 2003). The study's Steering Committee included two American Indian community representatives, and a separate “Seven Nations Committee” was formed to serve as a forum for American Indians involved in the project to have an opportunity for input into program design, content, and implementation protocols. Community members were important contributors to the final form and format of Pathways (Davis and Reid 1999).

Pathways resulted in statistically significant changes in health knowledge, cultural identity, and healthful eating behaviors on the part of third-, fourth-, and fifth-grade school children in comparison to control schools (Gittelsohn et al. 2003), and generated a great deal of interest in continuing or expanding Pathways in and beyond the communities where the project was implemented and among public health workers at IHS. In response, following the end of the Pathways RCT, the Pathways curriculum was made available through the UNM PRC website (http://hsc.unm.edu/chpdp/projects/pathways.htm). Subsequent activity on the Pathways webpage on the site was more extensive than anticipated, and interest from tribes other than those in the original Pathways study and at IHS remained high.

Participatory Action for Healthy Lifestyles

Responding to continued interest in Pathways, researchers from the UNM PRC obtained funding in 2002 from the Centers for Disease Control for Participatory Action for Healthy Lifestyles (PAHL), a 3-year qualitative research study to examine participatory processes in the dissemination of the Pathways intervention at three levels (regional, state, and local) using a community-based participatory research (CBPR) approach. PAHL emphasized participation as part of the research design in terms of engaging individual and institutional stakeholders on multiple levels, with an expansive definition of community that included organizations and institutions “partnering” in the dissemination process. The idea was that increasing the level of community participation in the process of disseminating Pathways would increase implementation. The objective of the project was to document the process through which evidence-based research findings were utilized by schools, public health professionals, agencies, the communities of study, and by other systems.

Researchers at the UNM PRC who had been involved in Pathways were interested in sharing the curriculum developed in Pathways and studying the process of dissemination and utilization. The New Mexico DOH and IHS agreed to partner with the UNM PRC to bring Pathways to American Indian schools in New Mexico. PAHL created a Strategic Planning Team (SP Team) as a mechanism for community members to work with DOH and IHS officials and UNM PRC researchers to identify community health priorities and coordinate training logistics. However, as discussed below, developing a realistic design for incorporating participation required evolution of the structure of relationships between supporting institutions. Individual schools, teachers, and school staff were intended users of the Pathways curriculum. The UNM PRC provided tools and infrastructure for Pathways through training and technical support to the users. DOH connected resources to potential community users through the institutional infrastructure of the state health apparatus. The participatory framework of PAHL was to provide for integration of users, technical assistance, and connection to resources through active engagement and involvement of all partners in the process and with the creation of the SP Team.

The SP Team concept, as defined in PAHL, was based on an approach commonly used in participatory research to create an explicit structure for local stakeholders to contribute to project planning and project design. As Wallerstein and Duran (2010) recommend, community participation generally requires the development of a specific structure, forum, or organization (e.g., committee, board, network, etc.) to incorporate community members into the research. However, in the specific context of PAHL, although the concept of the SP Team would have provided a formal, concrete mechanism for engaging community members and for integrating the work, realities on the ground conflicted with participation requiring ongoing coordination and time commitment from multiple stakeholders. Accounting for the needs of community members and research partners required that PAHL develop less structured mechanisms to allow for community participation. As a result, rather than working via the SP Team, a PAHL project coordinator was hired to solicit input and guidance for disseminating and implementing the Pathways curriculum in different local contexts from individual schools. The PAHL project coordinator worked collaboratively with DOH to conduct presentations to potential adopters, and to identify interested schools and individuals. Connections were built with opinion leaders, including teachers and school staff, community health representatives, tribal leaders, and stakeholders from IHS. In this way, the PAHL project coordinator created systematic linkages between researchers and the community in a manner that allowed PAHL to overcome “on the ground” realities inhibiting the capacity of the project to involve people in developing and planning the research and in setting agendas for action. The dynamic nature of the translation process required all stakeholders to operate with a degree of flexibility that is often difficult to accommodate in the research process.

The key contact person for PAHL at DOH was in charge of state diabetes prevention efforts. Her responsibilities included the Coordinated Approach to Child Health (CATCH) program that provides teachers with a framework for incorporating nutrition education and physical activity into the classroom curriculum. For several years, DOH had provided funding to schools to implement CATCH through a small grant program that included support for CATCH “champions” at each CATCH school to encourage program implementation. As a partner in PAHL, DOH agreed to allow American Indian schools to use state funds to implement the Pathways curriculum instead of CATCH. DOH adapted the existing application process for schools to request CATCH funds to also be used for schools interested in adopting Pathways. Although DOH had been an active partner during the planning phase of PAHL and had provided the infrastructure and funding mechanism for encouraging schools to use Pathways, actual DOH engagement with PAHL during the dissemination phase of PAHL was minimal. PAHL data suggest that although the key contact person at DOH had agreed to use Pathways as an alternative to CATCH in schools with a large American Indian student population, DOH field coordinators responsible for developing and implementing CATCH trainings may not have sufficiently “bought-in” to the Pathways model. Based on statements from interviews conducted as part of PAHL, coordinating trainings and technical support for two separate, seemingly competing programs was a burden for DOH field coordinators in terms of time and effort. As a result, working through the DOH field coordinators to develop connections to schools to implement Pathways and to serve as research sites proved challenging. In the end, the PAHL project coordinator recruited schools directly on an individual basis. Ultimately, five schools (two Bureau of Indian Affairs schools and three public schools serving Pueblo students.) applied for and received funds from DOH to implement Pathways, and served as research sites for PAHL.

DOH did provide a funding mechanism for engaging schools in the possibility of implementing Pathways. In itself, the agreement by DOH to leverage the existing funding process for CATCH to include Pathways represented a potentially major development towards institutionalizing Pathways as an option for American Indian schools in New Mexico in a permanent fashion beyond the period of the grant. However, the role of DOH in PAHL evolved. Hoffman et al. (2010) identify stakeholder “buy-in” as one of five key principles for effective engagement in research. Without the “buy-in” of state personnel involved in coordinating trainings and providing technical support for recipients of state monies, while DOH continued to serve a linking function, the state mechanism alone did not provide the capacity for successfully disseminating Pathways. Connectivity provided by DOH at the systems level was an important factor in the successful dissemination of Pathways, but cultivation of more personal connections in the community shifted to a the PAHL project coordinator.

In the fall of 2003, three direct training sessions in the Pathways curriculum were conducted by the PAHL Research Team with 46 individuals representing the five participating schools. Participants were primarily classroom teachers, but also included PE teachers, food service staff, a health educator, an educational aide, a nurse, a recreational therapist, and a parent–teacher organization president. Following initial training sessions, the UNM PRC identified an individual at each school to serve as a “Pathways Champion.” Program “champions” have been shown to be key for effective implementation of health promotion strategies (Dilworth et al. 2013; Elliot et al. 2011; O'Loughlin et al. 1998). Similar to the model used by DOH to support school-level champions to implement CATCH, each of the Pathways champions received a small stipend from DOH funds to cultivate interest and enthusiasm for implementing Pathways at their school and to coordinate a school-level Pathways “Action Team.” This “train-the-trainer” structure became the primary mechanism utilized for disseminating Pathways during the remainder of the project. The fluid structure and flexible roles of the principal stakeholders in PAHL supported and encouraged dissemination of the Pathways curriculum. DOH provided resources for PAHL, but the PAHL project coordinator served as a structural link between the state resources and school/teachers users. The Pathways champion at each school received funding from the DOH and training from the UNM PRC in order to cultivate and enhance utilization of the Pathways curriculum among school staff. This inter-digitation between systems and stakeholders highlights the complexity involved in prevention research, and demonstrates the fluid mechanisms that develop in real-world settings in the context of unique local conditions.

Following training sessions, the selection and training of Pathways champions, and site-specific train-the-trainer sessions provided by the Pathways champions at their schools, all of the schools involved in PAHL made positive changes. Four out of the-five schools indicated using parts of the curriculum component, three schools hosted family engagement events, two schools reported improvements to school food service policy, and three schools reported using exercise break box activities and modified American Indian games. At the same time, a number of factors were identified in PAHL data as hindering more successful and intensive implementation of Pathways components. Some teachers indicated that they were not entirely clear on implementation and did not feel confident about their ability to implement and integrate Pathways into their classroom curriculum. In post-training evaluation documents, some who attended the training reported that they left with what was really a rather cursory understanding of the overall Pathways approach and developed skills at a mechanical rather than a conceptual level. As one participant phrased it, “coverage seemed rushed.”

Part of the challenge seems to have been related to the design of Pathways as an efficacy trial with a conceptual framework. Pathways was developed as an intensive, in-depth RCT intervention using a sequential, integrated interaction with teachers in their classrooms over multiple years, supported by a significant investment of resources and provision of materials. The Pathways framework as developed in the RCT required more than one or two trainings, so in evaluation documents from the trainings, few teachers, and staff reported developing the capacity to implement Pathways as a “package” given the training they received through PAHL. This meant that although an entire conceptual framework underpins the Pathways curriculum, the transfer being accomplished in PAHL was not necessarily of the conceptual package, but of individual components. In the “real world”, people “take” only what they want, have interest in, or are able to use. Some members of the research team believe that the inability of trainings to convey the bigger conceptual framework weakened the capacity of teachers and staff to understand how to incorporate discrete components into the curriculum in a meaningful way.

In addition, a more overt challenge recognized during the project by teachers and principals had to do with schedule limitations and competing priorities for classroom activities. PAHL was predicated on the idea that following the training, and with technical support provided by the researchers, teachers would implement Pathways with fidelity into their existing school curriculum. PAHL interview data indicates that there was a perception among people who took the training that Pathways would require a great deal of time to implement and a significant effort on the part of the teacher. Natural time constraints that exist when working with schools may have been exacerbated by the requisites imposed by “No Child Left Behind” legislation that began at the same time as PAHL in 2002. Teachers had begun to face new demands on curriculum time such that anything that did not directly address preparation for standardized testing was increasingly being excluded from the school day. Not only had there begun to be a reduction of art and music in schools but also PE programming and recess were also being eroded. In this context, where even traditional cornerstones of the school day were being deleted in favor of expanding standards-based academic instruction, teachers expressed the opinion that they found it difficult to justify incorporating nutrition education and physical activity into the required curriculum. One of the teachers who took the Pathways training said, “I have to be selective on components—I may have to bump another subject to use Pathways.” The reality was that teachers did not have the freedom and flexibility to creatively enhance instructional time in a way that was assumed in a school-based adoption of the Pathways curriculum. As a result, PAHL data suggests that the actual implementation of Pathways tended to involve only a few specific activities that could be integrated without much effort or time (such as exercise break activities).

Given these challenges, involvement and “buy-in” from principals and school administration became a key factor in the extent of implementation. At schools where the principal made Pathways a priority, teachers were more likely to actively engage in implementing components in the classroom. In all cases, however, it was a small core of staff who were the prime movers adopting the intervention. Many of those who attended the trainings were not actually committed to the ongoing process of implementation. The experience of PAHL demonstrates how teachers and schools actively exerted agency in the process. Some merely did not use the curriculum following the trainings. Others transformed the intervention “package” to fit with limitations imposed by the content and context of training, the capacity of teachers, and instructional requirements. Teachers inspired by the training but with little time in the instructional day were able to make select components of Pathways fit with the required daily routines in their classrooms. In this context, supportive principals and committed staff made implementation more likely.

Despite the challenges, interest in the Pathways conceptual framework continued to motivate researchers and people in the community, and to capture the attention of stakeholders at IHS who had been involved in the original Pathways intervention. It was understood from the results of PAHL and more directly from people working in the communities that a “next generation” of Pathways with more convenient and easier to access resources would not only better serve the needs of teachers, but would increase the capacity for use of Pathways materials and activities at community venues outside of school settings. In 2006, researchers from the UNM PRC and representatives from IHS met to brainstorm. While modifying the entire Pathways curriculum was not possible, there was interest in and infrastructure for developing the physical activity component. At that time, it was increasingly being recognized that community settings offer potential opportunities for increasing levels of physical activity among community members (Paez et al. 2012) and it has since been demonstrated that inserting short bouts of physical activity into routine daily schedules can be beneficial (Barr-Anderson et al. 2011). The idea that it is important for lifestyle interventions to recognize the differences in skill and ability that affects people's levels of physical activity, especially as this relates to age and past life experiences and history, was of interest (Meyer and Gullotta 2012; Malina 1996; Marcus et al. 2001). This “lifespan” approach fit with community and IHS interest in activities for different ages outside of elementary school settings, and seemed like a good idea given the challenges for implementation in the classroom that had been demonstrated in PAHL. The UNM PRC and IHS decided to work together to combine physical activity components of Pathways with “Healthy Body Awareness” (a program for Navajo Elders also developed by the UNM PRC; Cunningham-Sabo 2000) to create a Physical Activity Kit (PAK) focusing on physical activity called, “Staying on the Active Path in Native Communities: A Lifespan Approach.” Directors of a variety of IHS National Public Health and Wellness programs were interested in PAK.

Although the transformation embodied in the development of PAK was not a predicted outcome of Pathways or PAHL, PAK can be understood using a conceptual framework that integrates the three projects. Stakeholders involved in the implementation of Pathways via PAHL contributed to the development of a modified set of activities better reflecting the interests and needs of the community. Users (schools, teachers, community members) had identified the need for modification or adaptation of the Pathways/PAHL approach in order to achieve better outcomes in terms of dissemination and implementation. Researchers at the UNM PRC channeled this feedback and worked with IHS on a new design that would address user concerns. At the same time, IHS emerged as a key player, able to support the process of modifying and adapting the intervention. These changes engendered a shift from individual teachers/schools to a nationwide network of IHS National Public Health and Wellness programs, their members, and the communities they serve as the intended users. The fact that users were now embedded in an institution interested in adopting Pathways components further influenced success described below in the next stage. The agency of stakeholders contributed to the capacity for community input to actively transform and adapt the original intervention to more completely meet community needs.

The Physical Activity Kit

The partners agreed to develop the PAK concept despite lack of specific funding for the effort. The UNM PRC focused on modifying, condensing, and unifying the physical activity components of the Pathways/Healthy Body Awareness frameworks into a single binder including a train-the-trainer manual and a PowerPoint training presentation. IHS established an infrastructure to disseminate and implement PAK on a large scale. The interested IHS Program Directors identified and recruited PAK pilot sites around the country from among their agencies. In 2007, participating IHS programs provided funding for individuals from each pilot site to attend a 3-day training. Thirty-five people from 11 teams attended the training, and over the following year, participated in monthly conference calls to discuss their experience implementing PAK and training others in their communities. In 2008, IHS sponsored the same group to attend a summit to further evaluate experiences with PAK. During the evaluation summit, the results reported to IHS by trainers were positive, with participants describing enthusiastic reception in their communities. The principal critique that emerged from the summit was that people found the binder format too cumbersome, and the train-the-trainer manual and PowerPoint overly complicated and academic. The idea of creating a more convenient, self-contained format was proposed. Over the next year, these recommendations were integrated into a re-design of PAK as a toolkit with discrete, small, spiral-bound books contained within a box. In 2009, production of the boxed toolkit and funding for training events were incorporated as part of the IHS and UNM PRC annual budgets.

The development, dissemination, and institutionalization of train-the-trainers sessions using components of Pathways through PAK could not have been anticipated at any point in the PAHL project. Following the end of PAHL in 2005, although enthusiasm for the Pathways curriculum continued, further dissemination by the state CATCH program was unlikely, making the possibility of institutionalizing use of the Pathways curriculum in American Indian schools low. Yet positive “relationality” between the UNM PRC and IHS became a driving force for developing new strategies and formats to meet the needs of the new users (IHS National Public Health and Wellness programs, IHS program members, and communities). The UNM PRC contributed expertise and staff time to develop and implement PAK, and IHS as an institution devoted infrastructure with staff and committed funds. IHS connected PAK to frontline community health workers in communities across the country. Because the users were now embedded in IHS as an institution, the dynamics of overlap between the systems operated to facilitate institutionalization. In 2009, IHS sponsored a national roll-out of PAK and since has sponsored trainings for more than 600 trainers from communities across the country. Following the trainings, participants have had access to a Sharepoint website for ongoing technical assistance. In 2010, the UNM PRC collaborated with IHS on a “PAK in Action” informational video featuring clips of selected games from each book to assist in training.

Although national data has not been collected or evaluated regarding the utilization of PAK in community settings beyond training events, institutionalization of PAK trainings within IHS created new channels for dissemination that have allowed PAK to be diffused and replicated nationwide to a new network of users. The extent to which PAK retains “fidelity” to the efficacious components of the original Pathways intervention has not been investigated, but PAK has proven to have a synergy that continues to develop interconnections between the key stakeholders. The result demonstrates important ways in which prevention research can incorporate community participation and influence, and achieve outcomes that, while not predictable, can be understood as “successful.”

Conclusion

We use an agency-based theoretical framework to unify analysis of three related projects by thinking about how community participation and input influenced the research process in a coherent way. This approach integrates our understanding of the three projects while accommodating the interplay between individual “agency” and structural dynamics. Despite efforts to embrace community participation in research, the tension of scientific rigor and fidelity versus adaptability, responsiveness, and re-invention, continues to structure the way that interventions are designed and the way that we define success. One of the people involved in PAHL lamented that PAK could be seen as an ideal outcome, if it were not for a concern with fidelity in the implementation of the Pathways curriculum. Using an integrated and theoretical framework to analyze the evolution of intervention components from Pathways to PAHL and then to PAK illustrates that using community input and participation as a gauge, in fact, PAK can be seen as a very successful, even ideal outcome inspired by the original Pathways intervention. The ultimate “success” of PAK was achieved precisely because community participation at different levels and in multiple phases of the research influenced dissemination and utilization outcomes. Pathways incorporated community and contextual factors in project design to develop a culturally appropriate intervention for American Indian schools. This is what members of the community indicated that led them to be interested in Pathways even after the RCT funding and implementation had ended. Individuals involved in PAHL, including researchers, community partners, and community stakeholders learned to allow the work to evolve to reflect realities encountered in the “real life” experience of disseminating the intervention. PAHL's flexible and dynamic approach to research utilization helped to test what aspects of Pathways would work in the community and what aspects would not. Because people involved in PAHL at all levels were receptive to community needs and willing to adapt project materials, structures, and strategies, the result has been unexpected success on an extensive scale with IHS adoption of PAK trainings as a tool to enhance existing work.

In Table 2, we identify a number of key dimensions of how community agency influences the research process. Integrating the three projects within one analytic framework increases our capacity to explain processes involved in translation of research to practice beyond that of each project examined independently. An integrated approach allows us to expand our vision of PAHL backwards in time to include the RCT Pathways and forwards in time to include the development of PAK that actually occurred following the conclusion of formal PAHL project activities. Moreover, understanding that these “phases” of the research were part of a recursive and iterative system allows more explicit consideration of feedback loops generated by community input as part of the research translation, dissemination, and utilization process. This more comprehensive vision provides for analysis of results from a dynamic “big picture” perspective that allows deeper valuation of the participatory processes at work and more flexibility in interpreting different types of outcomes as representing “success” (see Table 2). Incorporating community input in the translation–dissemination–utilization of research has been recognized as an essential component for successfully getting research to be used in practice by people in the real world; however, the uncertainty entailed by this process has yet to be fully embraced by the research community. Our analysis of the experience of three related interventions using a theoretically based, conceptually integrated framework demonstrated that as the research paradigm shifts to more fully value the importance of ecological validity, gauging community participation, and “agency” in the research process can be a different, but equally meaningful measure of success.

Table 2.

Lessons learned from using a holistic, agency-based theoretical lens to understand the research process: understanding how agency influences health promotion research

Participation takes multiple forms: There is a tendency to approach CBPR with a rigid, structured vision of what participation looks like. Our experience follows that of Ponic and Frisby (2010) who found that participants may make decisions that do not fit with the vision of the researchers. It is important to create diverse mechanisms for participation (a SP Team, individual input conveyed to the project coordinator, project champions, train-the-trainer events, etc.). The value of fluid structure and flexible roles of principal stakeholders: A fluid structure for implementing research and flexibility for incorporating stakeholder roles can allow an intervention to better reflect the needs of participants and the realities of the research context, in the long run making the intervention more appropriate, valuable, and successful.

The importance of intangibles: Individuals at all levels of the process (project team members, teachers, principals, community members, DOH staff), especially those with charismatic personalities or those in positions of leadership, can have an exaggerated influence on the process and the outcomes of health promotion—both positively and negatively.

The key role of stakeholder agency: The data presented here demonstrate the extent to which stakeholders actively and passively transform both the research context and the research itself in both direct and indirect ways. This fact is often treated as an artifact of the research rather than acknowledged, accounted for or embraced.

The significance of context: Social and cultural dynamics impact the research process itself and cannot be seen merely as something external that exists to be transformed or changed, a common underlying assumption in health research.

The need for an Integrated/recursive conceptual framework: Our analysis demonstrates that we need to take a broader view of the research process in order to develop a more comprehensive understanding of community health. This analysis demonstrates a perspective on the research itself and on the nature of the outcomes that would not have emerged with a project-specific, non-theoretical approach.

Limitations to the Analysis

At the same time that we believe that there is great value in using the framework presented here, our analysis is, admittedly, a novel approach. We acknowledge that there are potential competing or alternative interpretations of the data from these three projects. While the amount of time that had elapsed between the writing of this article and the end of PAHL provided a unique opportunity for “hind sight” and reflection, had the issues explored here been of interest at the time that PAHL was operational, additional data might have been gathered that could attenuate our interpretation of the key role that community input played in the evolution of the research process over time. Moreover, analysis of PAK outcomes beyond tracking the number of trainings conducted and the number of trainers who were trained has not been examined. A useful next step would be to conduct a rigorous, structured evaluation of the actual use of PAK tools in community settings to determine the extent to which the toolkit is being implemented and whether there are measurable health outcomes that can be discerned.