Enhancing Dissemination, Implementation, and Improvement Science in CTSAs through Regional Partnerships

Abstract

Background and Importance

Challenges in healthcare policy and practice have stimulated interest in dissemination and implementation science. The Institute of Medicine Committee on the Clinical Translational Science Award (CTSA) program recommended expanding the CTSA program's investment and activity in this domain. Guidance is needed to facilitate successful growth of DII science infrastructure, activity and impacts.

Objectives

Several CTSAs in Southern California collaborated to identify and respond to local challenges and opportunities to expand dissemination, implementation and improvement research by strengthening capacity and relationships between DII researchers and community, health system, and population health partners.

Main outcomes

Planning and outreach by the Southern California CTSAs increased awareness and interest in DII research and generated recommendations for growth. Recommendations include: increasing strong partnerships with healthcare and population health systems to guide policy research agendas and collaborative DII science; promoting multi‐sector partnerships that involve researchers and delivery systems throughout DII processes; bringing together multiple disciplines; and addressing national and international barriers as well as opportunities in DII science.

Implications

CTSAs through regional collaboration can increase their contributions to improved community health via skill‐building, partnership development and enhanced outreach to local healthcare and public health agencies and delivery systems.

Introduction

It is commonly noted that it may take as long as 17 years for a small proportion of new scientific discoveries to enter day‐to‐day practice and benefit patients.1, 2, 3 Multiple national strategic efforts have characterized the gap between knowledge and practice. From 2000 to 2005, the Institute of Medicine (IOM) Clinical Research Roundtable (CRR) worked to identify challenges facing the clinical research enterprise, and proposed several approaches to create a more supportive environment and infrastructure for accelerating progress across the research pipeline.4, 5 The National Institutes of Health (NIH) Roadmap initiative established in response to CRR recommendations developed new programs and funding initiatives to accelerate translation of basic research into studies in humans and to support research into implementation barriers impeding the integration of research findings into routine practice and delivery systems.2

The NIH Clinical and Translational Science Award (CTSA) program operationalizes these goals.6 Launched in 2006, the CTSA program encompasses approximately 60 academic institutions across the United States.7 The CTSA program aims to enhance the benefits of research by accelerating the translation of discoveries from basic science (bench) to bedside and community.6, 7 Phases of translation are conceptualized somewhat differently by Westfall, Mold, and Fagnan;2 Dougherty and Conway;8 and others9 but all emphasize two broad phases encompassing (1) translation of basic science discoveries into effective clinical treatments and strategies and (2) appropriate implementation of effective innovations in medical and healthcare via dissemination, implementation and improvement (DII) science.

Challenges in healthcare policy and practice have further stimulated growing interest in DII science. Continued growth in health expenditures is coupled with recognition that higher US costs of care are not yielding better outcomes than other developed countries. The gap between knowledge and practice has been described by AHRQ, NCQA, and other stakeholders.10 The Affordable Care Act (ACA) is driving changes in the financing and organization of care and has generated new competitive pressures on health systems to improve quality and value, resulting in new interest in research‐based approaches to guide these improvements. New research funding from the Centers for Medicare & Medicaid Services (CMS) Innovation Center, from NIH program announcements in dissemination and implementation science and from additional agencies such as AHRQ and PCORI are helping to increase research attention and activity in dissemination, implementation and improvement science.

The NIH Roadmap and CTSA program recognize the importance of two types of translational roadblocks and the need for research to accelerate translation of knowledge to practice (in addition to translation of basis science findings into clinical innovations). However, national statistics indicate that the bulk of investment in research funding, activity and publication has focused on early phase translation (from bench to human) rather than latter stages of translation related to implementation.2 Full achievement of the Roadmap and CTSA program mission and, indeed, the broader NIH mission to develop and apply knowledge “to enhance health, lengthen life, and reduce illness and disability” will require significantly greater focus and investment in dissemination, implementation and improvement science to ensure that the results of health research are appropriately implemented to benefit patients, communities and populations.

Much of the required DII research and policy/practice activity occurs regionally and locally in healthcare and public health systems. This research requires intensive collaboration between the research, policy and practice communities and novel approaches to planning, designing and conducting research. To date only a handful of CTSAs have created programs to promote and support DII research.11, 12 In response to interest in expanding DII activity within and across CTSAs in Southern California we designed and launched a portfolio of activities to increase DII awareness, interest and capacity through education, outreach and additional strategies. This article describes the foundations and framework for this initiative, including the results of our assessment of barriers and facilitators to expanded DII research activity, and presents the key elements of the initiative. The article also reports results of one of the key activities within the initiative, a one‐day symposium sponsored by multiple Southern California CTSAs to prioritize gaps, needs and opportunities.

Conceptual Framework: Goals, Barriers and Facilitators

In 2012, the UCLA Clinical Translational Science Institute (CTSI) External Advisory Board recommended an expansion of DII science within the CTSI mission and portfolio, with a charge to define the role of the Community Engagement and Research Program (CERP) in the dissemination and implementation of research‐derived, evidence‐based interventions and in transferring knowledge from all forms of CTSA research to benefit community stakeholders and population health in a sustainable manner.13 This recommendation evolved into a CTSI vision containing several key elements:

• Results of clinical studies are put into practice in Southern California and benefit diverse populations;

• Local providers take part in studies on how to innovate, implement, and spread what is found to work;

• Research, dissemination, implementation and improvement occur seamlessly, by design;

• Healthcare and population health systems can readily locate enthusiastic researchers to help them develop system solutions to improve performance;

• Researchers partner with delivery systems and/or provider networks on implementation‐ and improvement‐oriented funding proposals and studies.

The UCLA CTSI and its partner Southern California CTSAs incorporated dissemination, implementation and improvement to reflect NIH CTSA and NIH funding announcement terminology and to enhance and augment current conceptualizations and programs by leveraging the distinct value of each of the three approaches.14 Dissemination science is the study of communication strategies that are designed to increase awareness and understanding of innovative, effective policies and practices to facilitate their widespread adoption.15, 16, 17 Implementation science is the systematic study of planned and active approaches to increase the uptake of effective practices.15, 18, 19 Improvement science uses iterative quasi‐experimental and experimental learning methods, using experiential learning to change processes and systems to achieve better outcomes reliably and at scale in healthcare and public health systems and services.20, 21, 22, 23

The field of improvement science offers a wealth of approaches and tools that complement those of dissemination and implementation science, emphasizing experiential, iterative experimentation and learning for innovation and for improving evidence‐based practice. Improvement science approaches are increasingly employed in practice‐based research networks, academic health systems and in health services research. The Southern California CTSA collaboration embraced improvement science for adapting what works to a range of practice settings and populations.22 Our inclusion of improvement science as a core component of the Southern California CTSA DII initiative is a novel contribution to research infrastructure.

As illustrated in Table 1, the UCLA CTSI developed a DII science initiative to address (1) lack of awareness of DII science and the rapid growth in funding opportunities, conferences and journals; (2) lack of skill and expertise and limited training opportunities; (3) limited tools and resources to facilitate efficient planning, design and conduct of DII studies; and (4) insufficient knowledge of the expertise, capacity and resources that does exist locally. Activities in the initiative include:

1. Education, Training and Capacity Building to enhance DII awareness, interest and skills within the academic research workforce, and to build CTSI institution and community partner capacity to conduct and use DII science research.

2. Consultation activities to support researchers interested in conducting DII research but lacking adequate expertise in DII study design, conduct and publication and thus with insufficient capacity to submit successful responses to DII‐related funding announcements (e.g., the Patient‐Centered Outcomes Research Institute [PCORI], Center for Medicare and Medicaid Services [CMS], foundations).

3. Tools and Resources were compiled, including resources such as templates for DII funding applications and institutional review board (IRB) applications.

4. Synergy of Functions so that all components of the CTSI infrastructure support DII science.

Table 1.

UCLA CTSI DII science expansion strategies.

	Contribution to strategy by
Goal areas	University (CTSI)	DHS & DPH	Delivery systems
1. Education, Training and Capacity Building
A. Train DII science researchers within academic institutions
Establish a Master of Science (MS) in Improvement & Implementation Science	X
Design and offer DII content and modules for the CTSI Training Program in Translational Science (TPTS) (formerly the K30 mechanism) and other fellowship programs	X
Pursue research training and other professional development support for enhanced curriculum development and teaching of DII methods	X
Stimulate interest in DII science in the university	X
Expand or leverage pilot funding for DII science	X
Strengthen recruitment of DII‐oriented fellows, trainees and faculty	X
B. Enhance CTSI institutions and community partner capacity for activity in DII science
Develop DII research components/supplements in ongoing/planned clinical research	X	X	X
Increase DII research that leverages goals and activities of quality improvement, programs and projects within CSTI partner institutions	X	X	X
Increase DII research that leverages the goals and activities of programs and projects outside of the university (e.g., county departments of healthcare, public health, and mental health; healthcare delivery systems; and other institutions)	X	X	X
Improve data capture and reporting capacity within CTSI institutions and community partners to facilitate more efficient practice‐based research	X	X	X
Identify practice‐level incentives for diverse healthcare providers to participate in DII science	X	X	X
Offer professional development (continuing education) programs to build DII research skills in practicing healthcare professionals	X	X	X
Collaborate with professional organizations (e.g., family medicine, pediatrics, community clinic associations) to stimulate interest in DII research and practice	X	X	X
2. Consultation and Technical Assistance
Facilitate successful DII funding applications, projects and publications via expert consultation in DII study design, conduct and publication	X	X	X
3. Development of Tools and Resources
Identify and catalogue DII science tools and resources (e.g., funding announcements, conferences, journals, training programs and resources)	X
Create templates for key portions of DII funding applications, model IRB applications and “Plain English” consent forms, and other resources	X
4. Interface with CTSI and other CTSI cores
A. Coordinate DII activities across CTSI functions (including Community Engagement, Education, Biostatistics, Informatics, Regulatory, Pilot, Evaluation)
Identify key DII science impacts and develop appropriate measures and tracking methods for DII science program activities and accomplishments	X	X	X
Identify funding opportunities and offer grant preparation technical assistance/consultation to expand support to DII opportunities	X	X
Expand use of a multi‐institution research data repository for DII science applications	X	X	X
Strengthen DII science content in training activities	X	X	X
Identify opportunities for harmonizing CTSI partner institutional review boards (IRBs) with local healthcare and public health departments	X	X
Address real and perceived obstacles for investigators undertaking DII research	X	X	X
B. Support enhanced public dissemination of research
Work with existing public affairs/research dissemination infrastructure	X	X	X

The table illustrates shared responsibility for key activities across the academic institutions plus key stakeholders such as the Los Angeles Departments of Public Health (DPH) and Health Services (DHS) and public and private health delivery systems. The CTSI crafted these strategies in consultation with these stakeholders and community partners.

Leveraging Regional CTSA Resources and Strategies in DII

Given the importance of cross‐sector partnerships for effective DII science, the UCLA CTSI partnered with the Southern California Clinical and Translational Science Institute (SC CTSI) from the University of Southern California, and Kaiser Permanente Southern California, to identify opportunities for expanding and improving partnered DII research. The partners hosted annual one‐day symposia in 2014 and 2015 to explore challenges and opportunities in DII science within Southern California. Rather than only promoting what works well with the intent of doing more of the same, each symposium posed the question of how research could be prioritized, planned, conducted and disseminated differently to improve impact. Each symposium provided learning and interactions between researchers and community partners to facilitate stronger collaborations and find ways that research could be conducted more effectively.

The two day‐long symposia sought to expand the quantity and quality of regional DII science activity by (1) identifying and addressing barriers to meaningful, responsive DII research that meets the needs of community partners, (2) sharing knowledge and information regarding current DII science‐related activity and expertise in Southern California, and (3) creating networking opportunities and stimulating new collaboration between experienced researchers, academics who are new to the field, and community partners. The planning and processes for both symposia were similar; the following paragraphs describe the 2014 Symposium.

DII science symposium agenda and target audience

A planning committee comprised of faculty and staff from each sponsor institution designed the symposium agenda to discuss regional and national priorities for DII research, identify federal funding opportunities, illustrate exemplary DII science projects, and stimulate discussions among local stakeholders to identify key gaps, barriers, and opportunities to meaningful DII science. The Symposium targeted junior as well as experienced DII science researchers and community partners from throughout the region to facilitate rich debate, exchange, and networking with the ultimate goal of fostering expanded cross‐institutional collaborations.

Framing the opportunity: Keynote addresses and panel discussions

Keynote speakers representing PCORI and the Association of American Medical Colleges (AAMC) described the critical role of DII science in fulfilling the promise of research in delivering benefits to all patients. They described opportunities for junior and senior researchers to obtain funding for projects that address the missions of academic health systems, PCORI, NIH and that meet the needs of patients.

Experienced DII scientists described exemplary community partnered DII projects in areas encompassing preventive medicine, psychiatry and behavioral sciences, implementation policy in the Veterans Health Administration (VA) healthcare system, and regional public health and delivery systems. Presentations also discussed key challenges to designing and executing implementation and improvement science research that meets the needs of stakeholders in healthcare, public health, and in communities.

Symposium speakers included:

• Ann Bonham, PhD, Chief Scientific Officer, Association of American Medical Colleges

• Tom W. Valente, PhD, Professor, Department of Preventative Medicine, USC

• Kenneth B. Wells, MD, MPH, Professor‐in‐Residence, Psychiatry & Biobehavioral Sciences, UCLA

• Elizabeth Yano, PhD, MSPH, Director, Center for the Study of Healthcare Innovation, Implementation and Policy, VA Greater LA HSR&D

• Jean Slutsky, PA, MSHS, Program Director, Communication and Dissemination Research, Patient‐Centered Outcomes Research Institute

• Jeff Gunzenhauser, MD, MPH, Medical Director, Quality Improvement Division, Los Angeles County Department of Public Health

• Marguerite Koster, MA, MFT, Practice Leader, Evidence‐Based Medicine Services, Southern California Permanente Medical Group, Kaiser Permanente

• Tony Kuo, MD, MSHS, Director, Office of Senior Health, Los Angeles County Department of Public Health

• Jeff Lazarus, MBA, Health Science Consultant, Merck Vaccines

• Catherine MacLean, MD, PhD, Vice President, Quality Improvement, Center for Quality Measures and Improvement, WellPoint

Sharing DII science in progress: Poster session

A call for poster abstracts was issued with emphasis on: (1) relevance to DII science; (2) quality, completeness and significance of the research; (3) implications for the continued development of DII science as a field, and/or DII science in Southern California; and (4) abstract quality. The Symposium planning group reviewed all submitted abstracts for poster presentation.

Barriers, opportunities and brainstorming: Discussion sessions

Parallel breakout/discussion sessions (10–30 participants each) were moderated by a faculty member and a fellow or trainee with DII expertise selected from the participating CTSA institutions. Rather than achieving consensus, the group sessions sought to characterize and understand the key challenges and identify potential pragmatic solutions. The discussions were recorded and developed into papers to serve as a resource for other CTSAs and similar research groups interested in expanding DII science activity.

Symposium Products/Results

The 2014 Symposium was attended by a diverse group of 129 participants including senior and junior researchers, research fellows, leaders of local healthcare delivery systems and public health agencies, and research partners in community‐based organizations in Los Angeles. All attendees were listed in a registry booklet including names, organizations, and biographies to foster potential networking among attendees having common research, policy and practice improvement interests.

A total of 22 posters were presented covering a wide array of DII science topics including hospital readmission prevention, evidence‐based health technology in managed care, rapid HIV testing for the homeless population, research‐operation partnerships to improve quality of care, and novel methods in DII science.

Network analysis and symposium evaluation surveys were collected from attendees at the end of the Symposium. The network analysis will enable the participating CTSAs to track relationships and collaborations among academics and community partners on research projects and grant applications. A post‐Symposium evaluation questionnaire assessed knowledge gain, originality of concepts and ideas presented and their influence in fostering practice change, recommendations for follow‐up activities including webinar series, and interest in participating in future DII science collaborative efforts.

The Symposium attendance included diverse representation of organizations comprising academic research institutions and medical centers (UCLA, USC, Charles Drew University, RAND Corporation, Cedars‐Sinai Medical Center), public and private health systems (Los Angeles County Department of Health Services and Department of Public Health, VA Greater Los Angeles Healthcare System), Kaiser Permanente Southern California) and community organizations (Table 2). Most Symposium attendees reported being well‐versed in DII science. In evaluations, participants reported positive knowledge gain and exposure to new concepts, and participants prioritized proposed actions (such as webinar series, thematic poster sessions) to further increase their interest, exchange and partnerships in dissemination, implementation and improvement science (Table 3). The registration list produced a searchable database of those interested or conducting DII research in Southern California, including joint projects and areas of interest, to augment researcher profiles already maintained by the CTSAs.

Table 2.

Symposium participants

Sector (attendees could list more than one sector)*	N (%)
Academics	47 (63.5)
Healthcare system	36 (48.7)
Public health	8 (10.8)
Medical group	4 (5.4)
Other	6 (8.1)
Affiliation (attendees could list more than one affiliation) †
UCLA	52 (40.3)
Dept. of Veterans Affairs	18 (14.0)
USC/Children's Hospital LA	15 (11.6)
Kaiser Permanente Southern California	15 (11.6)
Community organization	8 (6.2)
Los Angeles County Department of Health Services or Public Health	7 (5.4)
RAND	6 (4.7)
Cedars‐Sinai	4 (3.1)
Charles Drew University	3 (2.3)
Pharmaceutical	2 (1.6)
Other	7 (5.4)
Self‐reported pre‐Symposium DII knowledge
Very knowledgeable	42 (37.5)
Somewhat knowledgeable	65 (58.0)
No knowledge	5 (4.5)

*Sector assessed by event evaluation (n = 74), †affiliation and DII knowledge assessed from registration (n = 129).

Table 3.

Symposium outcomes^*

	N (%)
Knowledge of DII science
Increased knowledge about DII science (n = 72)	58 (80.5)
Increased knowledge of DII activities in the Los Angeles Region (n = 73)	64 (87.7)
Exposure to new ideas and concepts (n = 73)	61 (83.5)
Increased interest in conducting or participating in DII science (n = 72)	53 (73.6)
Symposium was valuable to attendee's work (n = 73)	56 (76.7)
Agenda relevant to attendee's needs/interest (n = 72)	55 (76.4)
Plans to change or augment practice/research based upon symposium topics (n = 71)	33 (46.5)
Likely to participate in future DII activities
Webinars on featured DII research (n = 60)	37 (61.7)
Webinars on DII methods (n = 63)	42 (66.7)
Working groups or collaborative on DII research and/or methods (n = 65)	39 (60.1)
Webinars on DII funding opportunities (n = 63)	34 (57.7)
Webinars on other DII opportunities (n = 62)	34 (54.9)
Listserv on DII funding and/or other DII opportunities (n = 63)	48 (76.2)
Symposia or workshops on DII topics (n = 64)	46 (71.9)

*Response rate differed between questions.

Synthesis of Themes and Conclusions

The UCLA CTSI DII Initiative and the Southern California Regional CTSA Symposia developed strategies for the Southern California CTSAs to employ in expanding DII science activity. These strategies may prove valuable for other CTSAs throughout the United States interested in similar expansion of DII science interest and activity.

Enhanced contributions of DII science to population health

Key challenges to greater societal benefits and impact of research include a mismatch between the narrow research questions posed by most academics and the needs and realities of health systems and public health agencies. Relatively few researchers use methods for iterative learning that would help community partners design and test innovations more efficiently in (and for) a range of population characteristics, practice settings and conditions. Greater use of improvement science could bridge the current tensions between research and operations in terms of rigor, relevance and timeliness. Pragmatic steps that local health departments and CTSA programs can take to facilitate DII research in diverse practice settings include employing study designs to study “combination effectiveness” rather than comparative effectiveness, training scientists and health professionals in these methods, identifying and securing resources to advance this work, and sparking team science to solve complex systems issues.24

DII science and community partners

Key challenges to enhanced collaboration between researchers and policy/practice leaders and stakeholders include communities traditionally having little influence over the selection of problems pursued by health researchers; exclusion of community stakeholders from the research development process; cultural differences and health disparities limiting community engagement in current research; and lack of targeted, customized dissemination of research findings to the communities of study. Pragmatic local steps include ongoing forums to bring together communities and researchers, increasing synergy by aligning community‐academic working groups, encouraging community‐responsive methods such as quality improvement and practical trials, greater use of inter‐professional research teams, and greater clarity and specificity in the role of partners.25

Cross‐disciplinary collaboration and learning

Key challenges to the rich interdisciplinary collaboration required for successful DII studies include identifying and stimulating supportive leadership across disciplines; the unique and separate venues used by each discipline for sharing methods and findings; the need for equal voice and valuing team members with different disciplinary training and experience with theoretical and conceptual methods; and the challenge of ideological silos. Pragmatic steps include bringing together the diverse DII science workforce to tackle shared problems; sharing or developing new interdisciplinary conceptual frameworks for DII science; and sharing outcome measures within DII research that will validate the outcomes in diverse contexts, increase comparability of findings across studies and settings, and spreading the focus on the implementation process across multiple disciplines.26

Our experience suggests that regional collaborations involving multiple CTSAs and community partners can enhance the scope and quality of DII science. Key strategies include (1) sharing effective implementation and improvement science practices, (2) fostering partnership and collaborations in areas ranging from professional development/training to co‐design of shared research efforts, and (3) using feedback from community partners to ensure that the right methods are being applied to address their policy and practice priorities. In Southern California, the contributing CTSAs now sponsor a monthly webinar series to share innovative study designs, methods and DII project approaches among local researchers (faculty, fellows, and staff) and community partners. A new email LISTSERV and DII science website were established to share information about funding and collaboration opportunities. The annual Regional Symposium is now co‐sponsored by local health systems and other community partners as a means of building partnership and collaborations. UCLA CTSI consultation services provide expertise to researchers and practitioners. Shared ownership of DII science activity by CTSAs, healthcare and public health partners, and community partners ensures that all partners are invested in supporting DII in service of population health and the CTSA program mission.

Conflict of Interests

The manuscript authors declare that they have no conflict of interests, financial or other to declare.

Previous Presentations

The work is based on preliminary findings, stakeholder engagement efforts, and forum discussions presented at the 2014 Southern California Regional Dissemination, Implementation and Improvement Science Symposium sponsored, in part, by the National Institutes of Health/National Center for Advancing Translational Sciences through UCLA CTSI Grant Number UL1TR000124, the Southern California CTSI Grant Number UL1TR000130, and Kaiser Permanente Southern California.