White Paper on CTSA Consortium Role in Facilitating Comparative Effectiveness Research

Executive Summary

In 2006, the National Institutes of Health (NIH) initiated the Clinical and Translational Science Awards (CTSAs) as part of the NIH Roadmap Initiative, in order to improve the conduct and impact of NIH's clinical and translational research portfolio. ¹ The CTSA program is intended not only to transform the training programs and research infrastructure at individual academic institutions, but also to create a nation‐wide collaborative consortium to transform the biomedical research enterprise. In January 2009, the NIH CTSA National Consortium adopted Strategic Goals to maximize the CTSAs’ impact on the Nation's healthcare and health. Of these, the CTSA Strategic Goal 4 is to promote the translation of the results of clinical and translational research into practice and public policy. To advance this goal, a committee was constituted to focus on the organization and development of the CTSA Consortium's comparative eflEctiveness research (CER) capacity, an increasingly important component of research translation into practice and policy. This Committee's Workgroups took on a number of deliverables in service of this objective, including producing this White Paper on how the CTSA Consortium might best facilitate CER, for NIH's Institutes and Centers (ICs), other Federal agencies, outside stakeholders, and the healthcare system overall. This White Paper offers some specific suggestions for how the CTSA Consortium might support this emerging and crucial national effort to generate, synthesize, and disseminate CER in order to improve healthcare decision‐making and health outcomes.

Important points of reference for this White Paper are two Congressionally mandated reports on CER released at the end of June 2009, one by the Institute of Medicine (IOM) ² and another by the Federal Coordinating Council for CER (FCC‐CER). ³ The definitions of CER by each report, and their recommendations for the CER enterprise, are highly germane to the purpose of this CTSA Consortium White Paper. The CER definition used in the IOM Report was, “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist patients, clinicians, purchasers, policy makers, and the public to make informed decisions that will improve health care at both the individual and population levels.” The definition from the FCC‐CER Report was, “CER is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in’real world’ settings. Thpurpose of this research is to improve health outcomes by developing and disseminating evidence‐based information to patients, clinicians, and other decision‐makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances…” These definitions represent essentially the same concepts and are used together as the basis for this White Paper.

Considering their core resources and functions, the CTSAs offer an efficient and powerful platform for furthering a national effort toward CER. First, each CTSA has extensive clinical research infrastructure in the form of study design and methods expertise, statistics, epidemiology, and regulatory support. These resources are critical to support the newly emerging area of comparative effectiveness research. Second, each CTSA has extensive formal programs in education, training, and career development, often including the domains central to CER. Third and critical to the needed link between the public and CER, each CTSA has a program in community engagement, both of the general public and of clinical practice communities. Both the IOM and the FCC‐CER reports emphasize the importance of engaging stakeholders in the CER enterprise, including consumers, patients, clinicians, and other community members. An important part of community engagement is the development of research networks of community hospitals, practices, and providers that will provide an efficient infrastructure for conducting CER. Fourth, each CTSA has expertise and capacities in informatics related to research and clinical practice, including the use of clinical information systems and databases that are fundamental to CER. Beyond these specified components of CTSAs, the CTSA Consortium has a mandate to link different disciplines and to link and coordinate with other CTSAs, a function that could help support cross‐linking and collaboration nationally in the development and conduct of CER. Fifth, CTSA Centers have leading roles in facilitating the development and productivity of new research teams that cross disciplines, departments, and institutions. Current estimates suggest that to meet the needs for CER, new research teams that incorporate broad interdisciplinary collaborations will need to be developed. The CTSAs are positioned to support established researchers and cultivate new investigators who want to join or lead new research teams focused on CER.

Taken together, with sufficient focus and resources, CTSAs could be substantial facilitators of CER. This White Paper concludes that CTSAs should be leveraged by the creation of CTSA CER programs composed of multiple extant CTSA components. These programs would serve as the CTSA focus for rapidly facilitating or directly providing high‐quality CER in this country, both at their own institutions and as part of the National CTSA Consortium. Commensurate with the scale of need for CER, creating multi‐component CTSA CER programs will require substantial energy and resources to have real impact. Thus, this Paper makes six recommendations as steps forward for the CTSA Consortium in support of CER:

• 1The CTSA Consortium should organize and support clinical trial infrastructure capacities to be responsive to the needs of CER, including especially the ability to link these capacities across institutions to support multi‐site CER clinical trials. This should result in more efficientgeneralizable comparative effectiveness trials, with shorter time required for trial preparation and execution, thereby generating results more rapidly. If CER studies do not provide results in an acceptable time period, the value of the research will be severely compromised.
• 2The CTSA Consortium should support education, training, and career development in CER by funding NIH T (training), K (career development), and other appropriate awards sufficiently to contribute substantially to the needed CER workforce. This should help develop the workforce of scientists and disciplines of all types needed to conduct CER on the scale required for real support of healthcare improvement.
• 3The CTSA Consortium should develop capacities using CTSA Community Engagement components to incorporate input from the public and the clinical practice communities into generating, prioritizing, and developing CER questions and research protocols; interpreting CER results; and implementing CER results into practice. Also, this should include addressing disparities in health and healthcare in agenda‐setting and conducting CER. This should increase the relevance of CER to community health and healthcare needs and the implementation of these results for the benefit of the community and nation.
• 4The CTSA Consortium should develop the capacity to generate new research methods for evidence generation (studies and trials), evidence synthesis (systematic reviews and modeling studies), and evidence translation of CER to enhance current research and to provide approaches for the new opportunities and needs of CER. This should result in higher‐quality data analyzed in a more sophisticated and useful manner to better inform clinical practice and policy decisions.
• 5The CTSA Consortium should participate in the implementation of electronic health records (EHR) and health information technology (HIT) to ensure the resulting data can be used for the conduct of CER and to study which types of EHR‐based interventions are successful in changing practice and which are not. The proper leveraging ofEHRs and HIT will allow for efficient wide‐scale conduct of CER and the embedding of information from CER to disseminate EHR‐based clinical decision support to improve care.
• 6The CTSA Consortium should organize a CTSA Network for CER that links to agencies and entities outside CTSAs. This will provide direct access to the CTSA resources for CER among CTSA researchers nationally, and also to the other NIH ICs and key federal agencies such as AHRQ, CDC, FDA, other agencies, other entities, and stakeholders in the healthcare community and industry and the public.

As detailed in the final section of this White Paper, these recommendations closely parallel the recommendations of the IOM and FCC‐CER reports. The IOM report lists four recommendations as essential priorities for a robust CER enterprise; Recommendations 5–8 are parallel to this White Paper's recommendations. The IOM Recommendation 5 includes the need for a mechanism for prioritization and coordination of participating entities. This is outside of the direct functions of CTSAs—although CTSAs could certainly contribute to this work. IOM Recommendation 6 focuses on community engagement of the same sort as this White Paper's Recommendation 3. IOM Recommendation 7 emphasizes the need for CER methods development, as does this White Paper's Recommendation 4. IOM Recommendation 8 focuses on need to develop large‐scale, clinical and data networks to facilitate CER, as does this White Paper’ s Recommendations 1, 5, and 6. The FCC‐CER Report recommendations for a framework for CER include: (1) the prioritization and support for CER; (2) the growth of human capital (workforce) and scientific capital (methods) for conducting CER; (3) the needs for CER data infrastructure including informatics; and (4) the need for dissemination of CER findings into practice. This nearly complete correspondence of the IOM and FCC‐CER recommendations with recommendations of this White Paper based on the capacities of CTSAs to facilitate CER reinforces the idea that CTSAs have a substantial role to play in supporting the needed growth and development of CER. The specific ways CTSAs might do this are outlined further in this White Paper.

A broad array of existing and new resources will need to be mobilized to implement the IOM and FCC‐CER recommendations and respond to the challenges of healthcare reform. This will require substantial organizational and financial investments, especially in a time of resource constraints. Nonetheless, the recent allocation of American Recovery and Reinvestment Act (ARRA) funds for CER and CER's anticipated central role in upcoming Healthcare Reform suggest that the Federal Government understands that substantial investments will be needed. The CTSA Consortium represents potentially cost‐effective and already available resources that should be an important component of the CER investment portfolio. This will be particularly so if CTSAs support the roles of the other NIH ICs and other agencies and entities in leveraging and growing the national capacity for CER.

Preface

In January 2009, the NIH Clinical and Translational Science Award (CTSA) Consortium adopted a set of Strategic Goals to maximize the CTSAs’ impact on the Nation s healthcare and health. One of the four Strategic Goals, to promote the translation of the results of clinical and translational research into practice and public policy, included the development of capacity and methods for the translation of research results into practice across the healthcare system, including comparative effectiveness research (CER). A Committee was formed to address a variety of issues related to CTSAs and CER, and Deliverable Workgroups were formed to undertake the following tasks:

• 1Compile a capacity and needs assessment for CER and related resources across CTSA
• 2Articulate in a White Paper how the CTSA Consortium can act as a portal to facilitate CER for NIH Institutes and Centers (ICs), other federal agencies, outside stakeholders, and the healthcare system
• 3Make recommendations for a research agenda, including methods development, and training to fulfill the CTSA CER Strategic Goal
• 4Facilitate links and networking among the potential participants of CER (NIH ICs, other agencies, and other entities and individuals) to conduct CER
• 5Frame the workforce needs in CER and the roles by which CTSAs should have training and career development programs in CER

This report constitutes the second of these deliverables, aimed at describing options for CTSAs to facilitate, support and participate in the growing importance of CER. In doing so, this report builds upon early findings related to the first deliverable, a CTSA Consortium‐wide CER needs and capacities assessment, and is intended to help inform the third, fourth, and fifth deliverables, with the overall goal being to maximize the CTSA Consortium's support of CER and its impact on the US healthcare system.

To understand CTSAs’ potential roles in CER, the Workgroup engaged CTSAs nationally, especially those with ongoing CER activities, and those agencies and entities with which CTSAs might partner in support of CER, such as the Agency for Healthcare Research and Quality (AHRQ), and others already involved in CER. Early discussions were held December 17, 2008, at a meeting hosted by Tufts CTSI (https://research.tufts‐nemc.org/ctsi/cervideo/default.asp≥), and CER was elevated to a Strategic Goal by the CTSA Consortium in January 2009, when the above deliverables were mandated. This report is the product of the CER White Paper Workgroup, whose membership is on the title page.

Introduction

Background

Appreciating the need to better translate biomedical research into improved clinical care and health, the NIH Roadmap was initiated in 2003 to focus on this by strengthening NIH's central role in clinical research and clinical research training. As part of the needed re‐engineering, in 2006 NIH began the CTSA program. At this writing (Summer 2009), there are 39 CTSAs at leading universities across the United States, each having a clinical research graduate program and extensive resources for promoting clinical and translational research. Each of these, and the CTSA Consortium as a whole, are committed to transforming clinical and translational research to maximize impact on health.

The CTSAs bring to this challenge an array of capacities, extensive research infrastructure, formal training and career development programs, community engagement of the public and of clinical practice communities, and informatics related to research and clinical practice. Various CTSAs have different areas of emphasis, including along the spectrum of translational research from bench to patient bedside (the first translational step, “Tl”), from the bedside to general use in clinical practice (“T2”), and from clinical practice to public benefit and policy (“T3”). This broad array of activities has allowed the national consortium to be responsive to needs all along this path of translation of biomedical research into practice and public benefit. Although the activities of CER can be seen as primarily involving T2 and T3 research, they also can be seen as focusing on the optimal translation of Tl research into effective treatment, and thus the very epitome of work in which CTSAs should be deeply involved. This appreciation was reinforced by CER's selection as a key Strategic Goal focus of the CTSA Consortium.

In this context, the CER Strategic Goal of the CTSA Consortium is to promote the translation of the results of clinical and translational research into practice and public policy. In service of this, in two sections below, we review the span of CER resources across the CTSA Consortium and propose potential roles for CTSAs in supporting CER.

As a point of reference for this White Paper, below are two important definitions of CER from reports released at the end of June 2009, from the IOM and from the FCC‐CER:

Definition from the IOM Report on Priorities for CER

The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist patients, clinicians, purchasers, and policy makers, and the public to make informed decisions that will improve healthcare at both the individual and population levels.

Definition from the FCC‐CER Report

CER is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings. Th purpose of this research is to improve health outcomes by developing and disseminating evidence‐based information to patients, clinicians, and other decision‐makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.

• • To provide this information, CER must assess a comprehensive array of health‐related outcomes for diverse patient populations and subgroups.
• • Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, diagnostic testing, behavioral change, and delivery system strategies.
• • This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results.

In their primary intent, these two definitions represent essentially the same concepts and are used together as the basis for this White Paper. (The specific clinical and health areas emphasized in these reports, and the clear need to address health disparities, are beyond the scope of this White Paper but it is understood that these would need to be addressed in the implementation of a CER plan.)

The two‐fold challenge for CER

The emergence and development of CER as a focus for clinical and translational research introduces new challenges for the professionals and organizations involved in conducting, supporting, and disseminating this work. As implied in the IOM definition above, CER will require innovative and strategic expansions in at least two major areas:

• 1developing and synthesizing evidence that compares the benefits and harms of alternative approaches to care, and
• 2informing patients, clinicians, purchasers, policy makers, and the public about the available evidence, seeking to modify practice and thereby improve healthcare at the individual and population levels.

CER projects will be launched to answer questions about the effectiveness and appropriateness of two or more alternative available treatments: which is best for what types of patients and under what circumstances? New research and analytic methods, extensive infrastructures, a sophisticated multidisciplinary research workforce, and many other resources will be critical to addressing such questions. The importance of these questions is matched by the great needs for scaling‐up the capacity to do such research. How can CTSAs support and assist?

Implementation of the lessons of CER will be the pay‐off; without this, its impact will be naught and critical needs for improvement of healthcare will go unmet. Effective strategies are needed to translate scientifically supported best practices into actual healthcare knowledge and practices. How can CTSAs contribute to narrowing the gap between research results and actual healthcare and health?

CTSAs house extensive multidisciplinary clinical research infrastructures and educational and training programs, with links to their communities and local practice networks, and to research networks and national consortia. Their focus is the translation of the fruits of biomedical research into the improvement of health and healthcare. Thus, it is germane to explore how they might best support CER in this common mission.

CTSA CER Capacities

General capacities of CTSAs for CER

In general terms, the research infrastructure and support and training programs for clinical research should make CTSAs potential resources and likely sources of support for CER. First, each CTSA has extensive research infrastructure and support in the form of study design and methods expertise, statistics, epidemiology, regulatory support, and other resources that can be critical for performing high‐quality CER. CTSAs possess extensive ability to conduct clinical trials which, especially when combined through the network of CTSAs, could be a very useful resource for CER. Second, each CTSA has extensive formal programs in education, training, and career development programs that include the domains central to CER. Third, each CTSA has a formal and often extensive program in community engagement, both of the public and of clinical practice communities. Fourth, each CTSA has expertise and capacities in informatics related to research and clinical practice, including the use of clinical information systems and databases that are fundamental to CER. Besides the specified responsibilities CTSAs have in these domains, by virtue of their mandate to link different disciplines and to link and coordinate with other CTSAs, they are also already engaged in the kind of activities that will be key to establishing the national CER networks that will be needed to address the very central role being envisioned in future healthcare.

Having the components that could be so enlisted is not equivalent to being engaged in CER, and thus an assessment was undertaken to determine the extent to which CTSAs were already demonstrating such involvement, outlined below.

Relationship of CTSAs to AHRQ CER programs

To understand the possible roles for CTSAs in facilitating CER, the CTSA CER Strategic Goal Committee started by assessing the CER capacities of CTSAs. Mandated by the Medicare Modernization Act of 2003 to conduct CER, AHRQ has established centers that conduct CER. These programs act as highly developed CER infrastructures in terms of expertise and capacity, and serve AHRQ and many other organizations as sites for high‐quality CER for large and small projects and assignments. Thus, as a first step, the Committee assessed the extent to which CTSA institutions already had established AHRQ CER programs. Spread across the country, these included Evidence‐based Practice Centers (EPC), centers within the Developing Evidence to Inform Decisions about Effectiveness (DeCIDE) Network, and Centers for Education and Research on Therapeutics (CERTs). Fifteen CTSA institutions have one or more of these AHRQ grant programs, four have two AHRQ CER Centers, and two have all three types of CER Centers. Even this list is incomplete, as it ignores the many additional interlocking programs created by subcontracts on these center grants. This reflects that CTSAs and their institutions and their faculty and investigators already play very substantial roles in CER research infrastructure.

Besides these CER centers, AHRQ, like NIH, sponsors T32 National Research Service Awards (NRSAs) that support research training. Although not of identical content, many AHRQ NRSA training programs include clinical epidemiology, decision analysis, cost‐effectiveness, and evidence‐based medicine skills that are core to CER. Of the total of 28 AHRQ institutional T32 NRSA training programs in health services research, fully 25 are associated with 22 CTSAs (three have two AHRQ NRSA awards). Many of the NIH National Library of Medicine Informatics T32 training programs are also at CTSAs. Some of these T32s have the capability for training in CER. Having the capacity to train in CER does not mean it is being done, but this does suggest considerable overlap in CTSAs and AHRQ training activities, and reflects substantial CER training capacity among CTSAs.

In addition to these AHRQ training programs, the CTSAs themselves have major training components. In addition, CTSAs also provide an environment that supports many NIH T32 NRSA and K Career Development awards that also provide training in clinical epidemiology and the many methodological areas related to CER, further reinforcing CTSAs’ importance as extant major venues for potential training and career development in CER.

Survey of CTSA CER capacities and needs

To supplement understanding of the capacities and needs around CER among CTSAs, in June 2009, the CTSA CER Committee Deliverable 1 (Needs/Capacities Assessment) Workgroup assessed the CER capacities and needs of 33 of 39 CTSA institutions. A complete report is in progress by that workgroup, but its preliminary results inform this paper and are summarized below.

All 39 CTSAs existing at that time were contacted, and their principal investigators (Pis) were asked to provide answers to questions about CER capacity and needs of their CTSA (either by themselves or by someone else knowledgeable about CER at their institution). They were reminded to consider the broad university and associated schools when answering the questions.

In considering their CTSAs’ past 3 years research activities across several categories of CER, three types of CER were most frequently indicated as having extensive activity: long‐term observational cohort studies of CER (38%); analysis of outcomes data beyond their own institution (38%); and development of outcomes/quality measures (35%). In contrast, the three types of CER activities most often rated as having no or minimal activities were: use of electronic health records for CER (56%); clinically based registries for CER (53%); and including stakeholders in planning and/or implementing CER (53%). The most common areas CTSAs cited as needing to increase their capacity were: use of electronic health records for CER (55%); clinically based registries for CER (49%); clinical trials in CER (36%); and use of practice‐based research networks (30%).

Related to training, the three most common CER areas in which CTSAs reported having complete courses were: health services research (70%), health economics (61%), and health informatics (56%). Of note, nearly half of the CTSA institutions did not have any complete courses on these common CER topics. When asked in what areas of CER the CTSA Pis would like to increase the number and quality of training they offered, there were a wide range of responses, but the three most common were: conducting CER clinical trials (seven institutions: 21%); use of electronic health records/informatics (five institutions: 15%); and comprehensive programs across all areas of CER (four institutions: 12%).

The CTSA PIs were asked what they would do were they provided with one million dollars a year for 3 years to enhance CER. There was a broad range of responses. Some stated that they would hire new faculty or fund CER pilots. For Pis who mentioned specific infrastructure, the most common were building practice‐based research networks; helping faculty access administrative databases for secondary analyses; enhancing utility of electronic health records (EHRs) for CER; and/or expanding their knowledge about practice change and dissemination.

The Pis were asked how their CTSA currently supported CER. Almost half stated they provided extensive or moderate amounts of coordination including supporting conferences, core research units, and practice‐based research units. However, only 27% reported their CTSA provided moderate or extensive funding for CER, with pilot funds and practice‐based research units being the most frequent recipients of this support. Two CTSAs indicated no role in CER.

At the end of the survey, PIs were asked to provide additional comments on CER at their institutions. Many thought CTSAs could serve as a key forum for CER at their institution. Other comments included the need for a clearer definition of CER, the opportunity for more integration with community engagement efforts, and the importance of relationships with CER‐related AHRQ centers.

In summary, there is very substantial CER expertise and capacity at CTSAs. This is not surprising since a large proportion of CER‐relevant AHRQ Centers are at CTSA institutions, as are nearly all of the AHRQ NRS A health services research training programs, and there is substantial overlap between the AHRQ CER centers and these T32 programs. Training in CER is also included in NIH sponsored NRSA T training programs and K career development awards, including those supported by the CTSA Education and Career Development Components. These strengths in infrastructure and training in CER are consistent with the stated goal of CTSAs in advancing clinical and translational research and translating biomedical scientific advances into community benefit. Thus, these institutions have great expertise relevant to CER and are well suited to play an important role in developing and promoting this emerging discipline. Nonetheless, although CTSAs have a wide variety of components relevant to CER, few CTSAs have all the infrastructure and resources needed to support a comprehensive CER portfolio across multiple clinical questions. Most CTSAs have infrastructure to coordinate some CER studies, but not capacity to undertake additional CER unaided. Growing these CTSA capacities for CER would require strategic redeployment of existing resources or new funding, to take best advantage of individual and consortium capacities for CER.

CTSA Roles for Facilitating CER

Given the broad and deep CER expertise across the CTSAs, including extensive clinical trial expertise and also many AHRQ supported CER research programs and training programs, the CTSA Consortium seems well poised to provide the nation much needed CER research infrastructure capacity and scientific workforce preparation. In specific, CTSAs can provide special resources in five areas directly linked to their current structures and functions, and a sixth one based on their consortium functions:

• 1CER‐relevant clinical trials infrastructure
• 2CER scientific workforce development through formal education, training, and career development programs
• 3community engagement of the public and of clinical practice communities around CER questions and answers
• 4enhanced methods relevant to clinical and translational research, including CER
• 5informatics to support comparative effectiveness studies
• 6cross‐institutional and cross‐regional linking and coordination of CER‐related efforts

Research infrastructure

Research infrastructure to conduct CER is needed across the nation in the many organizations that will be involved in CER in many scientifically related areas. CTSAs are well positioned to provide expertise and to help link those most involved in CER, including AHRQ, NIH, CDC, and other agencies, professional societies, scientific organizations, and stakeholders.

For example, CTSAs are well positioned as core institutional resources, to support access to the institutional, national, and international claims and electronic health record data resources needed for CER. Many such databases are already on site at some of the CTSAs. Additional support would be needed though to assist in their wider use and access, providing programming resources, pilot funding for small research projects using these data, and personnel time for assisting investigators with the special expertise needed in using these data properly. Additional support would permit access to more such databases as well. Indeed, through the CTSAs, access to these data could be shared among the CTSAs. The net result would be wide access to these data resources, which is so critical for retrospective CER studies.

Expertise for conducting clinical trials and other studies, including familiarity with human subject research issues including Institutional Review Board (IRB) approval, potentially via consortia for efficiency in IRB approvals, also will be major assets. Indeed, the plan for CTSAs to form networks of clinical research networks, and CTSA Consortium‐wide IRBs, along with widespread infrastructure for clinical effectiveness trials could be enormously useful for the very large clinical trials that need to be conducted to answer some questions of CER.

Once again, to adequately supply all these resources and consultation, especially to a substantial new group of CER investigators and related individuals and groups, additional resources would need to be made available for this to be viable, and/or prioritization of existing CTSA resources would be needed.

Education, training, and career development programs

The competencies relevant to be a skilled researcher in CER are clearly different from those needed for the typical Tl researcher. It is widely understood that there is a great deficit in the skilled and diverse workforce needed to conduct CER. This is especially so given the needs for massive expansion of CER as is intended in support of healthcare reform that includes very substantial improvements in the generation and delivery of effective treatments.

Training of researchers is a central mission for the CTSAs. Moreover, unlike the training programs of other NIH Institutes and Centers (ICs), CTSA research training programs are not directed at specific organ systems or diseases, but rather at research training in general. Thus, their involvement in CER research already, and their involvement in training in the components of CER, make CTSAs very well suited to provide the CER training programs needed to expand the workforce and to meet the expected need of the coming years.

Additional resources would be needed to serve on the scale that will be required for a national CER effort, and/or a reprioritization of existing CTSA resources, but CTSAs already have such programs operating and should be able to play this role. Additional support via CTSA non disease/organ‐specific T32 and KL2 awards would seem to be the most available resource for supporting involvement in formal training programs, targeted toward programs capable of training the future researchers in CER methods and skills. The relative dearth of training opportunities in CER nationwide makes the capability of the CTSAs to coordinate curricular development and share course materials especially valuable. In addition, new T32 awards could be provided for CER training, preferably from NCRR so that they are not organ system specific. Finally, in expanding such training and career development capacity, efforts should be made to help correct the under‐representation of minorities in research.

Community engagement of the public and of clinical practice communities

A central goal of CTSAs, via their Community Engagement Components and more generally, is community outreach and engagement, to patients and their caregivers, clinicians, coalitions and advocacy groups, and others in the clinical and public health communities. Therefore CTSAs are very well positioned to be an active community engagement arm of the CER enterprise.

A key role highlighted by the recent reports on CER and by others is the need to reach out to the community for its assistance in choosing clinically relevant CER questions and helping do a much better job of recruiting patients and physicians into studies. These are core responsibilities of CTSAs. Further, CTSAs can use health information technology (HIT), knowledge transfer and information exchange, and other approaches to assist in the implementation and dissemination of CER findings, as they emerge. For example, CTSAs are in a good position to use their local EHRs to modify local practice, incorporating the findings of CER. At least as important, CTSAs are well positioned to investigate what types of EHR interventions would be effective in changing clinical practice. These are all consistent with the emphasis in the FCC‐CER report that there be a focus on “real‐world” data and impact.

Further, CTSAs are well positioned to develop community engagement methods that will be needed for well directed and impactful CER. All of the approaches used by the CTSAs to engage the public can and should be used to engage members of the community in CER clinical trials, and also to educate the public so that it can maximally benefit from the fruits of CER. These efforts should also address issues of disparities in health and healthcare among communities, in agenda‐setting and in the conduct and impact of research.

Again, additional resources and/or re‐prioritization of existing CTSA resources would need to be made available for this effort to be maximally effective.

Research methods development for clinical and translational research, including CER

There is a great need for advanced statistical, analytical, epidemiological, and study design methods for CER in a very wide range of domains. Examples might be methods for pragmatic trials, Bayesian and adaptive trials, proxy outcomes measures, the study of subpopulations in trials, and how to incorporate genomic questions/information into clinical trials. These methods and their needs for development are beyond the scope of this report; they are the focus of the CER Strategic Goal Committee Deliverable 3 (Methods) Workgroup and will be addressed in a forthcoming separate report. Nonetheless, the fact that the CTSA infrastructures include a focus on sophisticated analytic methodology and on creating new methods will be important for optimally supporting the rapidly growing field. With the explicit focus on infrastructure and novel methods, with the faculty for graduate programs in clinical and translational research that includes CER, and with expertise in clinical and analytic informatics, CTSAs are well positioned to contribute to the “intellectual infrastructure,” i.e., research methods needed for the expansion and growing impact of CER.

Informatics related to research and clinical practice

One of the most prominent features of CTSA member institutions is the development of informatics infrastructure both within the CTSA institution and across institutions, for connecting researchers and clinicians involved in clinical research, for identifying and matching available resources with needs (including physical resources, computational resources, and expertise), for providing both local and distance learning resources and technologies, and for mining and combining clinical research‐relevant databases of various sizes. Each of these has been and will continue to be a focus of the informatics components of each CTSA and for the CTSA informatics community. In this regard, such a cross‐CTSA community effort will depend in part on active collaborative infrastructures beyond the CTSA Wiki.

CTS As have already found that informatics is a key component of facilitating collaboration essential to innovative and impactful research. One potential CER‐relevant application derives from our survey finding that there is presently substantial variation in CER expertise across CTSA institutions. With the growing need for, and prospect of growing funding for CER, most current and future CTSA institutions will want to build their capacity for CER. Development and maintenance of a thorough and comprehensive catalog of CER resources at CTSAs will likely prove a valuable resource requiring ongoing support. IT approaches could aid this through systematic searches of datasets of funded grants (e.g., CRISP), the published literature, and even faculty CVs to identify evolving CER expertise and resources. These datasets will need to be searchable as investigators require CER expertise. Moreover, because not every institution and investigator with expertise and/or interest in CER is connected to a CTSA, it is reasonable to envision a national database of CER expertise and results beyond CTSAs, maintained by the CTSA Consortium, but broadly accessible. This will be useful in turn in informing the translation of CTS A bioscience discoveries into clinical practice elsewhere, and discoveries elsewhere into practice at the CTSA institutions.

Some CER will involve not just combining published data in systematic reviews, but will likely also require mining both national and multiple and combined datasets that cross institutions and even involve large de‐identified multi‐institutional datasets drawn perhaps from EHRs across real‐world settings, domestically or internationally. The availability of the domestic EHR data will accelerate exponentially with the national emphasis and funding for HIT. The CTSA informatics community is already working on implementing the combining of such data warehouses. It is likely that such data warehouses will be important to national CER efforts; the CTSA informatics community should become a leading force there also. Additionally, they should be helping lead establishing and gathering data from registries and practice‐based research networks. Such databases (both research and clinical) should be designed to facilitate discovery of new relationships among their data elements, especially as the cost of traditional clinical trials research continues to become a limiting factor.

In synergy to the community engagement work of CTSAs, and beyond collecting comparative effectiveness data from the clinical arena, the implementation of the conclusions of CER will require the distribution of (just‐in‐time) guidelines and decision support to the clinical arena. Here too, the CTSA bioinformatics community likely has opportunities to provide national leadership for both the infrastructure and decision support needs to fully leverage CER.

Finally, the CTSA informatics community has been providing infrastructure locally for distance‐based and time‐displaced education. They are also developing shared curricula in bioinformatics. Such efforts will need to expand to education about comparative effectiveness and CER nationally, to jump start institutions that are under‐represented in CER expertise.

Linking and coordination of research efforts

A central tenet of CTSAs is that they are to act as links and gateways to many organizations, disciplines, and communities, all in the service of advancing clinical and translational research and its ultimate impact on the public's health. For reasons of broadening the constituency and involvement in CER, it should be clear that these roles should be arranged in a way that extends these links far beyond the CTSAs themselves, to include other important entities and individuals. To accomplish this, a mechanism that provides a gateway to wide capacity and involvement is needed. To facilitate this, we propose a CTSA CER Gateway that includes membership of all CTSAs, but also allows membership of outside entities of all types, individuals, organizations, governmental agencies and offices, including the healthcare community, healthcare industry, the public, and other entities with an interest in partnering with a CTSA to advance CER. Connecting to other related Federal agencies potentially will be facilitated by the FCC‐CER, which has membership of the agencies involved in CER. The Community Engagement roles of CTSAs provide a natural avenue for engagement of communities, as outlined further above. Partnering with industry may include a variety of collaborations with health plans (e.g., using claims and electronic medical records data), pharmaceutical companies (e.g., on clinical trial research and other types of CER studies), and medical device and IT companies. Beyond research, these partners might also serve as opportunities for training CER researchers. In the context of these linking and collaborating efforts, the CTSA Consortium should examine and assess how these interactions currently take place, how they can be improved, and how all stakeholders can collaborate to improve CER studies, methods, and infrastructure, ultimately to the benefit of the public's health.

In keeping with the fact that CTSAs act as infrastructures for facilitating and accelerating research, we propose that this be done by creating an informatics infrastructure and staff that will support this activity, along with face‐to‐face meetings and other mechanisms of collaboration.

In terms of distribution of work, we note that one way of framing CER and its application to practices is as having two overall types of activities:

• 1CER projects that answer the question: of two or more treatments available, which is best for what patient and under what circumstances?
• 2Effective healthcare strategies for using CER that inform us about what strategies can be used to close the gaps between what we know clinically and what we do in practice.

CTSAs have varying capacity in these two dimensions, and the partnerships will presumably be wide and different for each. For the first of these, the CER studies, it is likely that AHRQ will continue to be the locus for efforts in evidence synthesis, HIT, registries and other retrospective studies, among other work, and the development of new methods related to these. The NIH will likely be the locus for the conduct of comparative effectiveness randomized trials, practical trials, prospective cohort studies, and other types of such studies, and the development of methodology related to these. The second of these objectives will include studies comparing different delivery systems, care models, knowledge transfer and information exchange and related activities. This will likely involve AHRQ, NIH, CDC, HRSA, and perhaps CMS, and other partners in healthcare. However, in all cases, there are major potential roles for CTSAs that would place them as partners in CER infrastructure and networking, and a key resource for training programs.

Conclusion and Recommendations

With a mission to help translate biomedical research into impact on healthcare and health, CTSAs are a natural link between the various disciplines needed for CER and a link among the various research, healthcare, and public organizations needed to have the desired impact on the public's health. Already, each CTSA is very aware of its limited resources to accomplish this important goal. Nonetheless, each now is accomplishing some part of this, as is the National CTSA Consortium.

The related mission of CER is no less ambitious: the translation of understanding of which therapy is best for which patient under which circumstances in real‐life care, and facilitating its implementation in medical practice and communities.

The great overlap conceptually and organizationally in these missions make CTSAs well suited to address CER. They bring their specified components that include research infrastructure and access to data; education, training, and career development programs; community engagement of the public and of clinical practice communities; and informatics related to research and clinical practice. The CTSA Consortium's effirts linking and coordinating research efforts within and across CTSAs, and more broadly across the scientific and healthcare communities, can be a major asset for facilitating the development of this critical area of research and application. Taken together, with sufficient focus and resources, CTSAs should be a key facilitator of CER as the nation seeks to improve both healthcare and its delivery in the coming years. Thus, we conclude that the CTSAs should be leveraged by the creation of CTSA CER programs composed of multiple CTSA components. These programs would serve as the CTSA focus for rapidly providing high‐quality CER in this country, both at their own institutions, and as part of the National CTSA Consortium.

Commensurate with the scale of need for CER, creating multi‐component CTSA CER programs will require substantial attention and resources to have the scale to have real impact. We recommend the following actions to generate these steps for the CTSA Consortium:

• 1Organize and support clinical trial infrastructure capacities to be responsive to the needs of CER, including especially the ability to link these capacities across institutions to support multi‐site CER clinical trials. This should result in more efficient generalizable comparative effectiveness trials, with shorter preparation and conduct, thereby generating useful results more rapidly. If CER studies do not provide results in an acceptable time period, the value of the research will be severely compromised.
• 2Support education, training, and career development in CER by funding NIH T (training), K (career development), and other appropriate awards sufficiently to contribute substantially to the CER workforce needed. This should help develop the workforce of scientists and disciplines of all types needed to conduct CER on the scale required for real support of healthcare improvement.
• 3Develop capacities using CTSA Community Engagement components to incorporate input from the public and the clinical practice communities into generating, prioritizing, and developing CER questions and research protocols; interpreting CER results; and implementing CER results into practice. Also, this should include addressing disparities in health and healthcare in agenda‐setting and conducting CER. This should increase the relevance of CER to community health andhealthcare needs and the implementation of these results for the benefit of the community and nation.
• 4Develop the capacity to generate new research methods for evidence generation (studies and trials), evidence synthesis (systematic reviews and modeling studies), and evidence translation of CER to enhance current research and to provide approaches for the new opportunities and needs of CER. This should result in higher‐quality data analyzed in a more sophisticated and useful manner to better inform clinical practice and policy decisions.
• 5Participate in the implementation of electronic health records (EHR) and health information technology (HIT) to ensure the resulting data can be used for the conduct of CER and to study which types of EHR‐based interventions are successful in changing practice and which are not. The proper leveraging of EHRs and HIT will allow for efficient wide‐scale conduct of CER and the embedding of information from CER to disseminate EHR‐based clinical decision support to improve care.
• 6Organize a CTSA Network for CER that links to agencies and entities outside CTSAs. This will provide direct access to the CTSA resources for CER among CTSA researchers nationally, and also to the other NIH ICs and key federal agencies such as AHRQ, CDC, FDA, other agencies, other entities, and stakeholders in the healthcare community and industry and the public.

Of note, and as illustrated in Table 1 , these recommendations are closely parallel to the recommendations for addressing key CER research needs in the IOM and FCC‐CER reports.

Table 1.

Comparison of CER research recommendations of CTSA white paper, IOM Report, FCC‐CER report, and potential mapping to CTSA components.

Objectives	CTSA white paper	IOM report	FCC‐CER report	Related CTSA components*
Develop priorities and conduct research	(Rec. 3)	(Rec. 5)	(Rec. 1)	Community Engagement and Research
Clinical trial and data and informatics infrastructure	(Rec. 1)	(Rec. 8)	(Rec. 3)	Design, Biostatistics and Clinical Research Ethics; Regulatory Knowledge and Support; Clinical Research Center
Workforce development by education and career development	(Rec. 2)		(Rec. 2)	Research Education, Training, and Career Development
Community engagement, including dissemination of CER findings, and implementation of findings into practice	(Rec. 3)	(Rec. 6)	(Rec. 4)	Community Engagement and Research; CER‐specific components
Develop new innovative CER methods	(Rec. 4)	(Rec. 7)	(Rec. 2)	Novel Methods; Research Education, Training, and Career Development; CER‐specific components
Informatics and data systems to support CER	(Rec. 5)	(Rec. 8)	(Rec. 3)	Biomedical Informatics; Design, Biostatistics, and Clinical Research Ethics
Organize and provide incentives for linkages and cooperation within and between CER stakeholders	(Rec. 6)	(Rec. 8)	(Rec. 4)	Governance; Entire CTSA; CER‐specific components

*This list of examples of types of CTSA components that would lead CTSAs’ CER efforts is not necessarily complete; different components may be engaged at different CTSAs.

The IOM makes four recommendations, numbered 5–8 in its report, that outline what it considers to be the essential priorities for a robust CER. Recommendation 5 describes the need for a mechanism for prioritization and coordination of participating entities, which is outside of the direct functions of CTSAs—although CTSAs could certainly contribute to this work IOM Recommendation 6 focuses on community engagement of the same sort as this White Paper’ s Recommendation 3. The IOM Recommendation 7 emphasizes the importance of CER methods development, as does this White Paper's Recommendation 4. Recommendation 8 of the IOM indicates the importance of developing large‐scale, clinical, and data networks to facilitate CER as reflected in this White Paper's Recommendations 1, 5, and 6.

Analogously, of the four recommendations the FCC‐CER made for a framework for CER, like the IOM Recommendation 5, the first focuses on the need for prioritization and support for CER. The second FCC‐CER Recommendation pertains to the need to grow human (workforce) and scientific (methods) capital for conducting CER, represented by this White Paper's Recommendations 2 (workforce development) and 4 (methods development). Recommendation 3 of the FCC‐CER pertained to the crucial nature of creating a CER data infrastructure, analogous to this White Paper’s Recommendation 5 (informatics). Finally, its Recommendation 4 focuses on the need for dissemination of CER findings into practice, which includes this White Paper's Recommendations 1 (CER infrastructure), 3 (community engagement), and 6 (CER network creation), and also parallels the overall mission of the CTSAs.

This near complete correspondence of the IOM and FCC‐CER recommendations for CER to the recommendations of this White Paper based on the capacities of CTSAs reinforces the idea that CTSAs have a substantial role to play in supporting the needed growth and development of CER.

Implementing this paper's recommendations will represent substantial organizational and financial investments in a time of constraints. Nonetheless, the recent allocation of ARRA funds for CER and CER's anticipated central role in upcoming Healthcare Reform suggest that the Federal Government understands that substantial investments will be needed. Representing extant resources well poised to address the needs for conducting CER, the CTSA Consortium represents potentially cost‐effective and already available resources that would seem to justify such an investment. This will be particularly so if CTSAs support the roles of the other NIH ICs and other agencies and entities in leveraging and growing the national capacity for CER.