It’s Your Game…Keep It Real: Can innovative public health prevention research thrive within a comparative effectiveness research framework?

Abstract

The federal comparative effectiveness research (CER) initiative is designed to evaluate best practices in health care settings where they can be disseminated for immediate benefit to patients. The CER strategic framework comprises four categories (research, human and scientific capital, data infrastructure, and dissemination) with three crosscutting themes (conditions, patient populations, and types of intervention). The challenge for the field of public health has been accommodating the CER framework within prevention research. Applying a medicine-based, research-to-practice CER approach to public health prevention research has raised concerns regarding definitions of acceptable evidence (an evidence challenge), effective intervention dissemination within heterogeneous communities (a dissemination and implementation challenge), and rewards for best practice at the cost of other promising but high-risk approaches (an innovation challenge). Herein, a dynamic operationalization of the CER framework is described that is compatible with the development, evaluation, and dissemination of innovative public health prevention interventions. An effective HIV, STI, and pregnancy prevention program, It’s Your Game…Keep It Real, provides a case study of this application, providing support that the CER framework can compatibly coexist with innovative, community-based public health prevention research.

Introduction

The federal comparative effectiveness research (CER) initiative is designed to evaluate best practices in health care settings where they can be disseminated for immediate benefit to patients (Agency for Healthcare Research and Quality, 2010). The CER framework has evolved from both the need for high-quality evidence for effectiveness of medical treatment and the need to translate this into practice for consumers. Applying a medicine-based, research-to-practice CER approach to public health prevention research has raised concerns regarding a narrow conception of what constitutes acceptable evidence (an evidence challenge), a dissemination approach that attends insufficiently to the needs of heterogeneous communities (a dissemination and implementation challenge), and a framework that rewards best practice at the cost of other promising but high-risk approaches, thereby stifling innovation (an innovation challenge). The purpose of this article is to describe the CER framework, acknowledging the challenges of applying it to the public health context, yet demonstrating that its dynamic operationalization can be compatible with the development, evaluation, and dissemination of innovative public health prevention interventions. An effective HIV, STI, and pregnancy prevention curriculum, It’s Your Game…Keep It Real (IYG), provides a case study of this application.

The Emerging CER Strategic Framework

In 2010, the U.S. Congress, through the American Recovery and Reinvestment Act, allocated $1.1 billion through the Federal Coordinating Council for CER to the Department of Health and Human Services, the Agency for Healthcare Research and Quality, and the National Institutes of Health (Fig. 1). The Federal Coordinating Council defined CER as “the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings” (Federal Coordinating Council, 2009, p. 5). The purpose of CER is “to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances” (ibid.). The emphasis of CER is on pragmatic clinical research trials that measure the effectiveness of a treatment (i.e., the benefit it produces) in routine clinical practice rather than its efficacy (i.e., whether it works or not). CER has been called “patient-centered health research” or “patient-centered outcomes research” to emphasize its focus on patient needs (Federal Coordinating Council, 2009).

Figure 1.

Agency distribution of ARRA CER funds

Adapted from Federal Coordinating Council for Comparative Effective Research (2009)

Exemplars of CER to inform patient and clinician decisions and improve health outcomes are provided in the Report to the President and the Congress on CER and include the National Institute of Diabetes and Digestive and Kidney Diseases–funded Diabetes Prevention Program Trial, which demonstrated that lifestyle change was superior to metformin and placebo in reducing incidence of new cases of metabolic syndrome, and the Veterans Affairs COURAGE trial, which demonstrated that patients treated with optimal medical therapy alone did just as well as patients who received percutaneous coronary intervention plus medical therapy in preventing heart attack and death (Federal Coordinating Council, 2009).

The comprehensive, coordinated approach to priorities and strategies for CER are depicted in a CER conceptual framework (Fig. 2). As shown, CER priorities and activities comprise four categories (research, human and scientific capital, data infrastructure, and dissemination and translation) with three crosscutting themes (priority conditions, priority populations, and types of intervention).

Figure 2.

CER Strategic Framework

Adapted from Federal Coordinating Council for Comparative Effective Research (2009)

Applying CER in Prevention Research: Evidence, Dissemination, and Innovation Challenges

The Report to the President and the Congress on CER acknowledged goals to improve the health of U.S. communities and the performance of the U.S. health care system. In this context, CER can help to identify interventions that yield the greatest health improvement in a targeted population, and to examine their health impact relative to feasible alternatives in other populations. Results of CER, then, should be translatable and scalable into public health programs, practice, and policy in states and communities throughout the nation in a time- and cost-efficient manner (Federal Coordinating Council, 2009). However, the CER initiative was focused primarily on the diagnosis and treatment of diseases and injuries in individuals, consistent with an emphasis on health spending in the United States where most of the gross domestic product (an estimated 17.6% in 2009) is devoted to clinical care (Martin et al., 2011). Given that the greatest drivers of individuals’ overall health are found in their social and physical environments, there was a call for the CER agenda to focus on the public health issues responsible for the highest overall illness and death levels (e.g., increasing high school graduation rates), to spotlight efforts to address widely recognized social and environmental determinants of health (e.g., improving access to early childhood development programs and education), and to invest in further development of methods to compare public health interventions and to use those methods to conduct CER studies (Teutsch & Fielding, 2011).

The Centers for Disease Control and Prevention’s (CDC) CER Program (U48) attempted to address the prevention research challenge. The program defined broad parameters for CER projects such that they attend to issues represented in at least one of three core domains from Institute of Medicine recommendations, Agency for Healthcare Research and Quality–identified evidence gaps, and Medicare Modernization Act Priority Conditions that were not currently addressed (CDC, 2010). The initiative in CER is based on the belief that the approach provides advantages in mitigating challenges for higher quality evidence and more effective dissemination without stifling innovation. The place of CER in prevention research has been challenged in relation to what constitutes acceptable evidence, effective dissemination, and a fertile incubator for innovation.

Evidence Challenge

Decisions about how to promote health or provide care are most effective when they integrate “the best available evidence with practitioner expertise and other resources, and with the characteristics, state, needs, values and preferences of those who will be affected” (Council for Training in Evidence-Based Behavioral Practice, 2008, p. 3). This is a worthy ideal; however, the reality is that, across the United States, clinicians, patients, and the general public all confront important health care decisions without adequate information or evidence-based guidance. Thus, questions regarding therapy (e.g., “What is the best pain management regimen for disabling arthritis in an elderly African-American woman with heart disease?”) or prevention (e.g., “What interventions work best to prevent obesity or tobacco use?”) are met with answers that are not definitive (Federal Coordinating Council, 2009). Even the most thoughtfully conceived and sophisticated practice guidelines lacked what the Institute of Medicine described as adequate evidence (Institute of Medicine, 2007). For example, in the best-practice guidelines developed by the American College of Cardiology and the American Heart Association, most recommendations were based solely on expert opinion, individual case studies, or standard of care. From the CER perspective, this is contributory but insufficient for definitive evidence-based practice. This translates to instances where additional health services, which are often costly and may not be established as effective, fail to yield improved health outcomes. This is exemplified globally when Medicare enrollees in the highest-spending regions of the country, receiving 60% more health services than those in the lowest-spending regions, do not have better health outcomes (Fisher et al., 2003, 2009). Under the auspices of CER, recommendations would need to be based on more rigorous inquiry methods such as randomized controlled trials (RCTs; Tricoci et al., 2009), where evidence informing best practice comprises research findings from the “systematic collection of data through observation and experiment and the formulation of questions and testing of hypotheses” (Council for Training in Evidence-Based Behavioral Practice, 2008, p. 3). Within the CER context, it follows that a goal for health promotion, disease prevention, and disease treatment is to provide rigorously tested, evidence-based interventions. Commentators have expressed concerns regarding the representation of rigorous evidence at the expense of the broad array of possible evidence sources, suggesting that “evidence-based” is not synonymous with “experimental-based” (Schorr & Farrow, 2011). This perspective cautions for a balanced approach to assessing, weighing, understanding, and acting upon interventions without requiring “proof” through experimental methods. This is particularly salient in the assessment of complex interventions in situations that do not allow for comparison groups, and to methods that are responsive to achieving pragmatic results that are “feasible, relevant, and politically acceptable solutions to real-world health problems intrinsically embedded in the widely varying complexities of behavioral, social, and cultural settings” (Livingood et al., 2011, p. 526). Nevertheless, the experimental approach advocated within the CER framework has an acknowledged place in “determining with certainty whether or not a program or a component of a complex intervention, focused on a specific outcome, is achieving desired results” (Schorr & Farrow, 2011, p. iv). In prevention practice, though, effective programs represent a point of departure, not a final destination, which must encompass effective dissemination and implementation to enable optimal reach.

Dissemination and Implementation Challenge

The existence of evidence-based practices and guidelines is no guarantee that they will be disseminated, adopted, and implemented (Hagland, 2010). To date, dissemination has been “trickle down,” following a “very protracted and uneven and erratic path” to adoption by health care providers and patients, making the implementation of standards for decision-making challenging (Hagland, 2010, p. 2). The CER initiative was designed to emphasize a mechanism for disseminating comparative findings, mitigating erratic dissemination, and providing greater immediate utility to health care practitioners, patients, and the general public (Hagland, 2010). Here, successful dissemination implies that the core components of the original intervention be faithfully transported to the real-world setting and adapted within the local context (Rabin et al., 2008). Commentators have challenged this contention, arguing that the research-to-practice paradigm, which is dominant in the context of linear science, is flawed in the context of public health. This paradigm is not as effective in the public health context as in the linear science context because it applies evidence-based interventions tested only in single, highly defined settings or populations to a wide range of settings and populations. Such an approach emphasizes internal validity (i.e., program efficacy) at the expense of external validity (i.e., program generalizability) thus compromising successful, future broad adoption. A suggested shift toward an applied research paradigm has been advocated that may allow for a more holistic integration of the full complexity of social determinants and health disparities (Livingood et al., 2011). Nevertheless, the proponents for this shift also acknowledge the strengths of linear research-to-practice models (e.g., CER and translation approaches) in the development of medical innovations including vaccines, antibiotics, and medical equipment (Livingood et al., 2011), though the role of CER as a catalyst for innovation has been contested.

Innovation Challenge

Innovation is fundamental for the continued resilience of public health and the continued success of the U.S. health care system (Ness, 2011, 2012). As CER is positioned to be “a critical tool for improving health care decision making and health outcomes, there is a need to balance its implementation with its potential impact on the all-important force of innovation in health care” (New England Healthcare Institute [NEHI], 2009, p. 23). In 2009, the NEHI drafted a white paper, titled “Balancing Act: Comparative Effectiveness Research and Innovation in U.S. Health Care,” in response to concerns about the federal CER initiative “winnowing out” some interventions and promoting others, thus stifling the development of new technologies and approaches that are critical to advancing health care and improving health outcomes (NEHI, 2009). The CDC exemplified innovation in the context of their CER U48 grant mechanism in pragmatic terms to include (but not be limited to): adaptations of established health promotion or disease prevention interventions that have not been tested; testing of strategies or interventions in settings or populations that have not been the target for such interventions; and development of novel combinations of adapted interventions, and application of such interventions to new settings and new target populations (CDC, 2010). This operationalization is in keeping with the linear research-to-practice model and is crafted to broaden the adaptation and dissemination possibilities of existing effective or promising interventions. However, it does not allay concerns of a winnowing effect of less rigorously tested interventions and a reduced focus on emerging new interventions.

IYG: Innovative Public Health Prevention Research within the CER Framework

IYG is an HIV, STI, and pregnancy prevention curriculum for middle school youth originally developed and evaluated within the context of sequential National Institute of Mental Health–funded R01 and CDC–funded Special Interest Project CER trials (Markham et al., 2012; Tortolero et al., 2010a). The contributions of IYG and related products/initiatives emanating from it since 2003 provide a case study exemplifying how prevention research might compatibly coexist within a dynamic instance of the CER framework as a stepped, iterative process (Fig. 3) and thereby meet the challenges of evidence, dissemination and implementation, and innovation.

Figure 3.

Stepped CER strategic framework for IYG with cyclical feedback loop

IYG Priority Condition, Population, and Intervention

CER framework priority condition: Adolescent HIV, STI, and pregnancy prevention

IYG is designed to provide a unique contribution to the national priority of adolescent HIV, STI, and pregnancy prevention.

Adolescent pregnancy

In the United States, births to teen mothers (10–19-year-olds) account for over 10% of all births, and 86% of teen births are to unmarried mothers (Hamilton et al., 2009). In 2005, compared with other U.S. states, Texas had the third highest teen birth rate (Martin et al., 2009) and the highest percentage of repeat births to teen mothers (23%; Child Trends, 2008). Teen pregnancy is particularly prevalent among African-American and Hispanic populations (Martin et al., 2009), and is costly to U.S. taxpayers (over $10.9 billion in 2008; National Campaign to Prevent Teen and Unplanned Pregnancy, 2012) and society as a whole (Card, 1999).

Adolescent STIs

In 2000, adolescents (15–24-year-olds) accounted for 9.1 million (48%) of all new STI cases, with an estimated medical cost of $6.5 billion (Chesson et al., 2004; Weinstock et al., 2004). Today, one in four adolescent females (14–19-year-olds) is reportedly infected with an STI (Forhan et al., 2009). It is estimated that over one-third of all new HIV infections occur among those under the age of 29 (CDC, 2009). Through 2006, African-American and Hispanic youth accounted for 88% of cumulative AIDS cases among 13–19-year-olds (CDC, 2008a). Texas currently ranks fourth among U.S. states for the estimated number of AIDS cases among youth aged 13–19 years (CDC, 2008b).

CER framework priority population: Middle school children

IYG is designed to impact STI, HIV, and pregnancy prevention in middle school students by delaying sexual initiation or mitigating the consequences of early sexual activity. The early initiation of sexual intercourse is associated with an increased risk of STIs and pregnancy (Flanigan, 2003; Kaestle et al., 2005). The 2005 Middle School Youth Risk Behavior Survey indicated that 14.7% of 7th graders and 18.1% of 8th graders have engaged in sexual intercourse (Shanklin et al., 2007). Texas data are consistent with national data, which indicate that 15% of 7th graders, 18% of 8th graders, and 33% of 9th graders have reported lifetime sexual activity (CDC, 2008b; Shanklin et al., 2007). Compared with youth who delay initiation of sexual intercourse until age 15 years or older, youth who initiate sexual intercourse at age 14 years or younger are more likely to have multiple lifetime sexual partners, to engage in frequent sexual intercourse, to use alcohol or drugs before sex, and to have sex without a condom (Coker et al.,1994; Flanigan, 2003; O’Donnell et al., 2001). Youth who become pregnant have an increased risk of adverse pregnancy outcomes (Menacker et al., 2004).

The sexual health paradox in the United States is that while this disconcerting, rapid acquisition of sexual behavior demonstrates the imperative for early sexual health education, most students are not receiving sexual health education or are receiving it too late to fully protect themselves against HIV, STIs, and pregnancy (Abma et al., 2004). HIV, STI, and pregnancy prevention programs for middle school populations have been rigorously evaluated, demonstrating improvement in one or more sexual behaviors for periods of up to 1 year (Kirby, 2007), although only three middle school programs have taken advantage of being delivered within the context of school curricula (Coyle et al., 2004; Flay et al., 2004; O’Donnell et al., 1999).

In the spirit of the community-based participatory research foundation for IYG, the priority population was also an active partner in the research process (Israel et al., 2005). Parents, students, community members, and school district personnel contributed to extensive qualitative work addressing youth sexual risk behaviors in a large district in southeast Texas at the middle school level. A Teen Advisory Board of 24 eighth graders from the target community help to guide the development of the overall look and feel of the IYG curriculum and specific activities (i.e., program name, scripts, stories, and role-plays reflecting urban, real-life situations and topics) and the inclusion of “protection messages.”

CER framework priority intervention: Interactive health communication within a hybrid curriculum

IYG provided an innovative contribution to health intervention curriculum design and implementation through the novel integration of interactive health communication delivery platform and strategies (Science Panel on Interactive Communication and Health, 1999). IYG uses technology-based strategies to deliver sexual health education, capitalizing on the pervasiveness and motivational appeal of technology among youth (Lenhart et al., 2007, 2008; Oblinger & Oblinger, 2005; Roberts et al., 2005). Prior research had demonstrated the impact of technology-based approaches on knowledge, attitudes, self-efficacy, intentions, and self-reported behavior (Evans et al., 2000; Kann, 1987; Thomas et al., 1997), as well as health behavioral change in multiple health domains and in multiple settings (Murray et al., 2005; Street et al., 1997). Prior research had also shown that computer-based HIV, STI, and pregnancy prevention programs have a positive impact on adolescent sexual behavior in various settings (Di Noia et al., 2004; Kiene & Barta, 2006; Lightfoot et al., 2007; Noar et al., 2009; Roberto et al., 2005). Prior to IYG, however, no effective middle school program had incorporated any substantial technological innovation, other than video components, into their learning activities.

CER Framework Categories: Research, Human and Scientific Capital, Data Infrastructure, and Dissemination and Translation

CER framework category 1: Research

Within the CER research category, work on IYG has provided three innovative contributions: (1) demonstrated effectiveness of a hybrid, middle school sexual health curriculum in two rigorous RCTs (Markham et al., 2012; Tortolero et al., 2010a); (2) design of innovative behavioral change software that uniquely operationalizes behavioral science theory and methods to impact behavior change (Shegog et al., 2007); and (3) application of innovative tracking methods within longitudinal CER trials (Markham et al., 2012; Tortolero et al., 2010a) (Fig. 3).

Demonstrated effectiveness in rigorous randomized CER trials

The Office of Adolescent Health of the Department of Health and Human Services lists the IYG curriculum as an effective, evidence-based intervention, which is an acknowledgement of its contribution to CER (Office of Adolescent Health, 2012). Since 2004, IYG has been evaluated in two RCTs in Texas middle schools with diverse populations, including Hispanic and African-American youth (Markham et al., 2012; Tortolero et al., 2010a). IYG intervention conditions were compared with usual care, which was defined as the typical sexual health education received within the school district (generally limited in scope and taught from a text book). Results demonstrated IYG’s effectiveness in delaying the initiation of sexual activity and reducing other risky sexual behaviors among 7th and 8th graders (12–14-year-olds) for up to 24 months. Results also indicated that students receiving IYG were 1.29 to 1.5 times more likely than students not receiving IYG to delay sexual initiation (Markham et al., 2012; Tortolero et al., 2010a). Furthermore, IYG positively impacted other important factors including intentions, beliefs, perceived norms, and knowledge about sex; self-control and confidence; refusal skills; rationale for not engaging in sexual behavior; and exposure to situations that put them at risk (Markham et al., 2009a; Tortolero et al., 2010a).

Design of innovative behavioral change software

IYG and associated dissemination platforms (the IYG Web site and IYG–Tech described below) have made innovative contributions to sexual health curriculum design, receiving the 2010 American Public Health Association PHPEP digital media award (Shegog et al., 2010) and being named semifinalist for the 2011 Adobe Design Achievement Awards (Brady, 2011). Developed using an Intervention Mapping framework (Bartholomew et al., 2006), IYG incorporates innovative educational software to operationalize behavioral science theory and methods (social cognitive theory; Bandura, 1985), social influence models (Komro et al., 2001; Perry, 1999; Story et al., 2002), and the theory of triadic influence (Flay & Petratis, 1994) to impact sexual behavior change (Coton et al., 2010; Shegog et al., 2007). IYG consists of 24 lessons (twelve 50-min lessons delivered in both 7th and 8th grade) that integrate group-based classroom activities with personalized journaling, individually tailored activities delivered on laptop computers, and parent–child homework activities to facilitate home dialogue on curriculum content. An innovative life-skills decision-making paradigm (Select, Detect, Protect) provides the unifying metacognitive framework for the curriculum, teaching students to select personal limits regarding risk behaviors, to detect signs or situations that might challenge these limits, and to use refusal skills and other tactics to protect these limits. Broad content domains of IYG include characteristics of healthy friendships and healthy dating relationships, setting personal limits and practicing refusal skills in these relationship contexts, and reproduction and STIs (Tortolero et al., 2010a).

Innovative technology-based educational strategies included: (1) a 3-D virtual world interface featuring an entertainment complex motif; (2) tailored educational activities including interactive 2-D exercises, quizzes, animations, peer videos, and fact sheets that target determinants of sexual risk-taking behavior; and (3) “real world”–style teen serials with online student feedback allowing for real-time group discussion in the classroom (Coton et al., 2010; Shegog et al., 2007; Tortolero et al., 2010a). Educational strategies included modeling of skilled behavior (e.g., saying “no” to having unprotected sex) and situational behaviors (e.g., choosing to avoid high-risk situations) that influence normative perceptions as well as skills; confidential and personalized management of sensitive and potentially embarrassing issues; individualized (tailored) intervention messages to specific characteristics of the users such as gender and sexual experience; and gaming formats and video and animated characters to provide a rich motivational experience that capitalized on the previously reported impact of serious games for health (Healthgamesresearch.org, n.d.; Lieberman, 2001; socialimpactgames.org, n.d.) and provided a contemporary approach to early work on the use of games for pregnancy prevention (i.e., “The Baby Game!” and “Romance!”; Paperny & Starn, 1989). Prior to IYG, no effective middle school program included strategies for tailoring activities, which are important for effective interactive health promotion programs (Krueter et al., 1999) and critical for providing sexual health programs that are appropriate to specific characteristics of these youth that have been reported (Brown et al., 1992; Cooper et al., 1994; Kotchick et al., 2001; Luster & Small, 1994; Reitman et al., 1996; Romer et al., 1994; Shrier et al., 1997; Tubman et al., 1996), suggesting the need for different information and skills training regarding sexual health (Eaton et al., 2006; Nahom et al., 2001). Tailoring content to students’ sexual behaviors and intentions to have sex allowed emphasis on continued abstinence for those who were not sexually experienced or had few intentions to engage in sexual activity, alleviating concerns associated with providing explicit information to all students who are at very different developmental stages, and being responsive to predominant local sexual health education school policies that focused more on abstinence-based approaches and less on teaching students about contraception (Lindberg et al., 2006; Sexuality Information and Education Council of the United States, n.d.).

Innovative tracking and data collection methods for CER

Evaluating IYG necessitated the application of innovative data collection and tracking methods within longitudinal, randomized CER trials. During the IYG CER trials, most students remained at the same middle school for baseline, 6-month, and 18-month follow-ups (i.e., in 7th and 8th grade), with retention challenges including dropouts, juvenile hall detention, and migration to other schools. However, by the 24-month follow-up (i.e., 9th grade), most students entered high school, necessitating final data collection in various high schools within multiple school districts. In the initial IYG CER trial, 92% of the 981-youth cohort was maintained at the 24-month follow-up. This high retention rate has been replicated in subsequent CER trials by triangulating varied innovative strategies adapted or created to track and retain this highly mobile, urban, public school population (Gwadz & Rotheram-Borus, 1992; Markham et al., 2012; Tortolero et al., 2010a). Strategies included collection of comprehensive contact information at recruitment including unique identifiers (school district ID number); initiation of a consistently updated confidential computerized tracking system; periodic participant contact throughout follow-up, with letters (including the use of overnight and certified mail) as needed; use of (computerized) directory assistance, online databases and social media (Facebook, Twitter, MySpace), newsletters, and reminders; incentives; and home visits or meetings in neutral public locations (public libraries and restaurants) as required. Data collection using audio computer-assisted self-interview delivered on laptops (with headphones) maintained confidentiality, provided branching to tailor sensitive survey items (such that only sexually experienced students were exposed to questions on sexual behaviors), and provided audio to those with reading difficulties (Romer et al., 1997; Tortolero et al., 2010a).

CER framework category 2: Human and scientific capital

Developing human and scientific capital in the context of CER refers to training of researchers dedicated to conducting CER or to developing new methods to further the advancement of CER inquiry.

Innovative approaches to training new CER researchers in sexual health

Human and scientific capital in the context of IYG CER trials included on-the-job skills development of students and fellows within IYG research teams (Fig. 3). The commitment to building capital for CER also exists at a broader institutional level, including hiring of faculty devoted to CER in sexual health (University of Texas School of Public Health [UTSPH], 2010b); training researchers in the field of HIV, STI, and pregnancy prevention through existing doctoral programs; and training of practitioners of evidence-based programs (EBPs) within existing Master of Public Health programs, including formal coursework devoted to research practice and adolescent sexual health (University of Texas Prevention Research Center [UTPRC], 2012e; UTSPH, 2010a).

Innovative approaches to train community organizations and practitioners in evidence-based practice

Dissemination of an effective program requires that the program be maintained and that training activities be conducted to build an informed and skilled professional and practitioner base. Within the CER framework, it is important that accredited practitioners ensure fidelity of the effective program to achieve dissemination. IYG training programs (1–4 days and webinar) have been developed for teachers and community partners to understand the components of IYG and gain mastery in its delivery (UTPRC, 2012d). In the context of public health prevention research, the development of human capital extends to multidisciplinary professional training of community health practitioners, teachers, counselors, researchers, and administrators to enable competence in adoption, implementation, and further evaluation of evidence-based practices in sexual health. Workshops (ranging from 30 min to 4 hr) are available to train participants, which include individuals as well as organizations (UTPRC, 2012d).

Innovative approaches to training youth in advocacy

Original conceptions of human capital in the context of CER focused more on practitioners and researchers and less on patients and target recipients of programs. Unique to community practice and community-based participatory research is the human capital afforded by the target population (i.e., youth) to agitate for EBP adoption (UTPRC, 2012b). Innovative strategies have been employed to engage youth in the most high-risk communities to help battle the birth rate and STI epidemic. Teen Scene Investigators is a youth participatory action research project that trains middle school and high school students in high-risk communities to investigate topics related to sexual health and plan action-oriented steps for community change (Johnson et al., 2011). Participating youth have created a research-based advocacy campaign (“Bring It Up”) to increase communication on sexual health topics in schools, families, and the community. Youth were trained in survey development, community mapping, photovoice, and social marketing, resulting in a community-informed advocacy campaign targeted to peers, parents, and schools. In addition to providing skills development, Teen Scene Investigators has the potential to shape career choices in aspiring research practitioners (Johnson et al., 2011).

CER framework category 3: Data infrastructure

Data infrastructure in the context of CER provides for repositories of stored knowledge to guide decision-making that can include networks for distributing practice-based data among practitioners, longitudinally linked electronic medical records, or patient registries. The preventive approach in the public health setting related to sexual health is based on the need for organizations to not only access data but also visualize complex data sets containing information essential to their activities and core goals in a timely and cost-efficient manner that overcomes limited resources and skill sets (Lefer et al., 2008; Myers et al., 2011; Revere et al., 2007). Through the UTPRC, data have been purposed not only to serve the dissemination of IYG but also to extend to the dissemination of any sexual health EBPs. Linkage to existing databases on teen births, as well as access to summary fact sheets, is available through the UTPRC for community practitioners (UTPRC, 2012a). Additional innovative strategies to increase the utility of data translation include geospatial data maps and the “We Can Do More” (WCDM) presentation materials on adolescent sexual health (Fig. 3).

Geospatial data maps

Within U.S. states, there is substantial local variation in teen birth rates, and these rates often correlate with sociodemographic factors such as income and race (Gould et al., 1998). Data are being translated for community partners and practitioners to assist policy makers, advocacy organizations, and public health researchers in understanding the distribution of teen births across Texas to advocate for the adoption of sexual health EBPs. The Texas Teen Birth (TTB) mapping tool is a visual representation of data that allows community users to view teen birth rates by various geopolitical boundaries within Texas (Tucker et al., 2011). This InstantAtlas–based interface allows users to generate custom maps using data provided by the UTPRC (InstantAtlas.com). The underlying database translates raw data from the Texas Department of State Health Services to rates that can assist with decision-making for school districts and state policy makers. These TTB maps have been cited as the most persuasive evidence that a more effective sexual health education curriculum is necessary (Tucker et al., 2011). The UTPRC receives many requests for these custom maps, so much so that it is difficult to meet demand (Tucker et al., 2011). The TTB mapping tool responds to the need for a low-cost, intuitive, Web-based system capable of displaying geospatial data at a granularity below county level. The TTB visualizations are available to the public via the Web (UTPRC, 2012c). They display univariate representations of the teen birth rate in Texas by creating dynamic statewide views, which allow users to drill down from the state to local level, while overlaying various geopolitical boundaries such as counties, school districts, and congressional districts. This framework meets 11 open-source software criteria (Tucker et al., 2011).

WCDM presentation

The WCDM presentation materials use statistics and imagery (including the geospatial maps described above) to define the issue of sexual health in public schools, and utilize these data to mobilize teachers, parents, and administrators to adopt sexual health EBPs in local schools (Ratliff et al., 2011). Participant observation of school health advisory council and school board meetings in 2009 and 2010 (n = 42) and semi-structured interviews with administrators and parents from five school districts (n = 10) have demonstrated the success of the WCDM campaign in forming community academic partnerships and mobilizing school districts around the topic of adolescent sexual health. Thirteen of 23 school districts in Harris County have adopted and are planning for the implementation of EBPs as a result of this and other communication strategies (Ratliff et al., 2011).

CER framework category 4: Dissemination and translation

In CER, dissemination and translation represents the development of tools and methods to disseminate CER findings to health care practitioners and patients and the translation of CER into practice. In the context of prevention research, dissemination is targeted to the community and translation represents the broad adoption of the effective program. With this intent, an innovative Sexual Health Toolkit has been developed (2008–present) to provide the leading adolescent sexual health resource for researchers, policy makers, health educators, service providers, parents, and youth in Texas. It is rare that an integrated toolkit strategy has been arrayed to address a public health problem. A series of scientifically based health communication tools/interventions are being “triangulated” to improve the sexual health outcomes of youth and to facilitate the adoption and implementation of sexual health EBPs (Fig. 4). The “hub” of the toolkit is the UTPRC Web site, which delivers technology-based communication tools comprising: (1) linkage to the IYG Web site (curricula and training information); (2) emerging IYG curricula (IYG–Tech, Native IYG); IYG–informed, clinic-based training for HIV-positive youth (+CLICK); (3) Web-based decision support to adopt and implement any sexual health EBP; (4) data-based advocacy tools including geospatial maps and WCDM presentation materials; (5) social media network outreach through Twitter and Facebook; and (6) online linkage for expert UTPRC support (UTPRC, 2012e). IYG dissemination activities are a reflection of the previously discussed CER categories, representing an almost cyclical unfolding of the CER framework (Fig. 3). Dissemination is engendered through further research activities that develop and test advanced digital versions of IYG (i.e., the IYG Web site and IYG–Tech) and versions to reach underserved populations (i.e., Native IYG) and families (IYG–Family) that not only facilitate broader dissemination but also include further CER evaluation trials, by dissemination research to investigate optimal strategies to disseminate EBPs like IYG (i.e., the Office of Adolescent Health Teen Pregnancy Prevention replication initiative), and by evolving an array of methods and tools to support the adoption, implementation, and maintenance of these EBPs as they are developed (i.e., the UTPRC toolkit) (Fig. 5).

Figure 4.

UT Prevention Research Center sexual health toolkit: An integrated health communication strategy

Figure 5.

IYG-related CER (2003-present)

Products and Initiatives Emanating from IYG since 2003

The IYG Web site: Adaptation of IYG to a Web-based Platform for Dissemination

The IYG Web site was developed within a CER study of varied dissemination strategies for IYG curriculum adoption and implementation (Peskin et al., 2009). The importance of providing IYG on a Web-based dissemination platform and the importance of facilitator training programs were demonstrated as critical dissemination components (Flores et al., 2010). Pilot study results of Web-based IYG lessons with a sample of 33 ethnically diverse students (58% female, M_age = 13.98 years) demonstrated an impact on psychosocial factors related to risky sexual behavior with high ratings on all usability parameters from most students (Shegog et al., 2012b). The IYG Web site provided a dissemination platform in response to broad domestic and international interest in IYG access and is the access hub for participants in large Office of Adolescent Health Tier 1 dissemination trials taking place in Texas, South Carolina, and California (Tortolero et al., 2010b).

IYG–Tech: An Online Interactive Multimedia Sexual Health Curriculum

IYG Tech is a completely online interactive curriculum, adapted from the original IYG, to mitigate individual and organizational challenges that hamper dissemination of conventional classroom-based curricula including reach, lesson fidelity, staff training, barriers related to sensitive material, ongoing support, and cost (Shegog et al., 2011b). Content analysis was used to identify and create new lessons and to adapt existing classroom lessons for the Web. The resulting 13-lesson curriculum comprises over 250 activities including original and modified IYG computer activities (66%), adapted classroom activities (10%), and new activities (24%). Critical added elements included refusal skills training, avoidance of unhealthy relationships, values clarification, and negotiating risk reduction strategies. IYG–Tech is in field-testing in a large CER trial in a large urban school district. Research results will be important to demonstrate the relative efficacy of this innovative technology-based curriculum compared with usual care, as well as to determine the cost-effectiveness implications of this implementation and dissemination strategy.

Native IYG: A Cultural Adaptation of IYG to Reach Underserved Populations

A culturally adapted version of IYG for online use by American Indian/Alaska Native (AI/AN) middle school youth is being developed through a CDC– and Administration for Children and Families funded collaborative between the UTPRC and tribal partners in Alaska, the Pacific Northwest, and Arizona (Shegog & Markham, 2011; Shegog et al., 2011a). Compared with youth of other racial/ethnic backgrounds, AI/AN youth are disproportionately affected by HIV, STIs, and pregnancy (National Campaign to Prevent Teen and Unplanned Pregnancy, 2009), and they experience significant and persistent HIV/STI disparities (CDC, 2009). Native IYG is aimed at promoting health equity and reducing disparities by providing those working in Indian Country with a culturally tailored prevention program for AI/AN youth, and by generating a cost-effective, highly reliable, Web-based sexual health curriculum for widespread access. This is an innovate approach for AI/AN youth, who consume media technology at even higher rates than other U.S. teens, to create social networks and share their culture within and beyond their local community (Craig-Rushing, 2010; Morris & Meinrath, 2009). Usability assessments and feedback from AI/AN youth and adult stakeholders on the original IYG’s cultural relevance, acceptability, and appeal is informing the development of Native IYG, which will be tested in a randomized CER trial with 1,200 AI/AN youth (12–14-year-olds) in regional tribal middle schools and Boys and Girls Clubs. The study will assess if youth who receive Native IYG are more likely than those who receive a comparison program to delay sexual initiation by a 24-month follow-up. If found to be effective, Native IYG will be made available to AI/AN communities and schools nationwide.

+CLICK: IYG informed, Clinic-based Skills Training for HIV-positive Youth

Youth account for almost half of all new HIV infections in the United States (CDC, 2005). Adherence to antiretroviral treatment and sexual risk reduction is essential for successful management, yet reported adherence rates for youth are low (Panel on Antiretroviral Guidelines for Adults and Adolescents, 2012). “+CLICK” is an innovative, Web-based adherence intervention for HIV-positive youth to be used as an adjunct to traditional clinic-based, self-management education (Markham et al., 2009b; Shegog et al., 2012a). The theory-based application, developed for HIV-infected youth aged 13–24 years, provides tailored activities addressing attitudes, knowledge, skills, and self-efficacy related to antiretroviral treatment adherence and sexual risk reduction in separate modules. Single-group, pre-/post-test study pilot tests have been conducted in hospital-based pediatric clinic and community settings. High usability ratings, impact on short-term psychosocial outcomes, and feasibility of +CLICK for use in clinic settings has been established (Markham et al., 2009b; Shegog et al., 2012a). Further CER on long-term and behavioral effects are indicated prior to broader dissemination in clinical practice.

An Emerging Model for Community Adoption of Evidence-based Sexual Health Curricula (CHAMPSS) and Associated Online Decision Support System (iCHAMPSS)

Implementation of sexual health EBPs is a complex and controversial issue for school districts and little guidance exists on the process for finding, adopting, and implementing EBPs. A unique collaborative with adolescent health stakeholders from 14 school districts has led to the development of a practical and realistic model (CHAMPSS) to help districts find, adopt, and implement EBPs (Hernandez et al., 2011), and of an online decision support system (iCHAMPSS) to operationalize this model (Li et al., 2011). The CHAMPSS model provides seven steps, tailored for school-based settings, which encompass phases of adoption, implementation, and maintenance (Hernandez et al., 2011). Advocacy and eliciting support for adolescent sexual health are also core elements of the model. iCHAMPSS provides guidance on each step which contains a set of critical tasks, support tools, and a final deliverables demarcating successful completion of each step (Li et al., 2011).

Conclusion

Innovation, in the context of public health, encompasses collaboration, creation of new ideas or adaptations, demonstrated impact, practical application, broad societal support, and sustainability (UTSPH Innovation Incubation Committee, 2012). Innovation can be the creation of the “new” or an adaptation of the “old” where processes or products are cross-pollinated to serve new domains or disciplines. The experience of IYG demonstrates that the CER framework can facilitate innovation in the context of prevention research, while simultaneously reminding us of the inherent constraints of this approach.

Evidence Challenge

The CER framework exemplifies the research-to-practice model, a prevailing linear scientific approach that generalizes from specific parameters. Efficacy is established in the context of intervention development and testing and, when resources permit, the RCT represents the optimal strategy to maximize internal validity. Innovation implies relative advantage over current best practice, providing greater impact without sacrificing feasibility and utility. For IYG, the initial focus has been, and continues to be, on developing interventions and establishing their effectiveness through RCTs. This is consistent with the promotion of innovation through relative advantage (i.e., greater impact) over comparative approaches. The development of discrete interventions such as a curriculum or session-based treatment is well positioned for this traditional scientific approach.

The research component of the CER framework promotes the collection of high-quality evidence. However, it is necessary to acknowledge that acquiring evidence through the application of scientific methods and processes to seek generalizable best practices does not present the only approach, or even the best approach, when considering public health prevention research that involves complexity of social determinants, health disparities, and “natural” communities where controlled studies are untenable. In such circumstances, traditions of “evaluation in practice” and applied approaches that triangulate numerous sources of evidence, principles, and theories, all within unique communities and that span both research and practice have been advocated (Livingood et al., 2011). In this regard, CER has been criticized for “miss[ing] the mark of producing feasible, relevant, and politically acceptable solutions to real-world health problems intrinsically embedded in the widely varying complexities of behavioral, social, and cultural settings” (Livingood et al., 2011, p. 526). Evidence collated in an RCT, while robust in terms of internal validity (i.e., program effectiveness), may not be as robust in terms of external validity (i.e., program generalizability), casting in doubt the utility of the evidence to lead to innovative solutions. The IYG case study supports the CER approach in facilitating new innovative developments and practices and in establishing evidence of effectiveness. Such evidence, however, does not guarantee generalizability to other contexts nor successful dissemination. Even more poignant, IYG is listed as an effective program, but it has a differential impact on youth exposed to it. At an individual level, while a significantly greater number of youth exposed to IYG will delay initiation of sexual behavior, compared with those not exposed to IYG, not every teen exposed to IYG will delay sexual initiation. The statistical significance obtained from an RCT represents rigorous evidence, but individual impact also indicates the need for other evidence to further understand determinants of program effectiveness. The IYG experience supports the view that the research component of the CER process is the beginning of the process, not its endpoint.

Dissemination Challenge

Even with RCT evidence, an emerging intervention can only be considered truly innovative if there is feasibility and utility for use by society, manifest in effective broad dissemination and sustainability. However, the notion that dissemination of programs deemed “effective” in RCTs to other community settings has been described as flawed, generally underpowered in terms of external validity. In keeping with a prevention research orientation, research and dissemination of IYG involved a broader community-based concept of human capital and data infrastructure that involved the “end users.” This serves to facilitate dissemination and translation by generating ground root support for program adoption “in situ.” Technological adaptations of IYG are designed to mitigate organizational (e.g., personnel training, cost) and individual (e.g., discomfort, access, fidelity) barriers through easy Web-based accessibility to all components of the program, individualized and standardized lessons, and facilitator training to ensure optimal implementation and fidelity, thereby helping to overcome some of the external validity shortfalls inherent in the research-to-practice model. Despite this, there is tension in the translation of an effective program to new communities because the RCT-based effectiveness cannot be guaranteed in the new milieu. Adaptation of the imported program becomes necessary from an “ecological” perspective, while the same adaptation may invalidate the program’s effectiveness from a “translational” perspective. Even enabling dissemination of IYG through the Web has necessitating adaptive changes to overcome bandwidth and maintenance challenges including conversion of original 3-D world interface features to 2-D representation, video compression, and cloud-based streaming. However, adoption of IYG into geographically and culturally disparate locations as California, South Carolina, New York, and Texas suggests that the feasibility and utility required for innovation can be addressed by CER, even in the context of prevention research.

Broadening the research agenda of CER to include prevention research is one of the recommendations for sustaining innovation in this framework (NEHI, 2009). The experience with IYG and ensuing programs indicated that the CER framework could be reconceptualized in the context of prevention research. In this context, the CER framework is a dynamic, constantly spiraling, progression from research to dissemination and translation mediated by human and scientific capital and data infrastructure (Fig. 3). Subsequent to dissemination and translation, there is a feedback loop to further research (i.e., development and evaluation) (Fig. 3).

Innovation Challenge

Innovation is often regarded in reductionist terms specific to local issues that may not produce a “sea change” in thinking and action on a broader scale. The warning is that we do not mistake small “technological steps” for huge “technological leaps” (Gertner, 2012). IYG has been cited as an innovative contribution to prevention research on the strength of discrete creative developments. It provides new approaches in content, channel, and scope, yet we must remind ourselves that for innovation to be true innovation, it needs utility, scalability, and acceptance that manifest as adoption and sustainability within the public health community at large. Innovation ushers large-scale change: “an important new product or process, deployed on a large scale and having a significant impact on society and the economy, that can do a job… ‘better, or cheaper, or both’” (Gertner, 2012, para. 22). Livingood et al. (2011) remind readers that applied (not research-to-practice) science provided one of the greatest accomplishments of 20th-century public health: the reversal of the tobacco epidemic. The dynamic complexity of community and society and policy changes (not to be found amongst the individual behavioral intervention studies that constitute “evidence-based medicine”) defied traditional scientific enquiry (i.e., RCT testing). Despite this, the IYG case study demonstrated that through a CER-based approach IYG has been identified as a public health branding success story (Shegog et al., 2011c) and is nationally recognized as an effective program that is beginning to establish itself in the broader public health domain. An important caution is that it is premature to adequately gauge the innovation of IYG on the basis of large-scale societal change.

IYG has demonstrated the seamlessness of fit with “product” development (similar to that for vaccines, antibiotics, and medical equipment) apropos to the research-to-practice model. The IYG case study has demonstrated a blossoming of innovative development and testing that is projected to continue with the development of vertical complementation of curriculum components from elementary, middle, and high school sexual health curricula; Spanish versions of IYG curricula; emerging modules targeting related topics such as substance use; and versions targeting particular settings including teen clinics, housing projects, and correctional institutions. The contributions of IYG and products/initiatives emanating from it since 2003 provide a case study exemplifying how innovation in prevention research might compatibly coexist within a CER framework and, conversely, how CER can be conceptualized to positively influence prevention research in compliment with applied approaches. It will be of continued importance to weigh the relative contributions of research-to-practice approaches (CER and translation research) with “applied science approaches” in decisions of allocation of resources, formulating policies, and designing and assessing interventions as we continue the pursuit of innovation in prevention research.