External Community Review Committee:

Abstract

Background: Major gaps exist between what we know and what we do in clinical practice and community health programs and narrowing this gap will require substantive partnerships between academic researchers and the communities they serve.

Objectives: We describe a research pilot award program that makes a unique commitment to community engagement through the addition of an External Community Review Committee to the typical research review process that gives external stakeholders decision-making power over research funding.

Methods: Whereas engaging community reviewers in discussion and rating of research proposals is not novel, the ICTR ECRC review process is distinct in that it is subsequent to peer review and uses different criteria and methodology. This method of engagement allows for the community review panel to re-rank scientifically meritorious proposals—such that proposals funded do not necessarily follow the rank order from scientific peer review. The approach taken by UW ICTR differs from those discussed in the literature that present a model of community-academic co-review.

Results: This article provides guidance for others interested in this model of community engagement and reviews insights gained during the evolution of this strategy; including how we addressed conflict, how the committee was able to change the pilot award program over time, and individual roles that were crucial to the success of this approach.

Conclusions: The advantages of this approach include success through traditional academic metrics while achieving an innovative shared-power mechanism for community engagement which we believe is critical for narrowing the gap between knowledge and practice.

Traditionally, the scientific community makes decisions regarding the investment of research dollars. In addition, dissemination of results in peer-reviewed journals and conferences often bypass community stakeholders.¹ As a result, community input into research priorities is negligible. Yet the public and funders expect greater returns on their research investments.² Community-academic partnerships show promise in addressing these issues and achieving health improvements.^3–7

In 2007, the University of Wisconsin–Madison was awarded a Clinical and Translational Science Award from the National Institutes of Health (NIH), requiring us to create pilot award programs to provide researchers funding for conducting translational research. In designing its pilot program, the UW-Madison Institute for Clinical and Translational Research (UW ICTR) made a unique commitment to engagement of community stakeholders in the application process by creating an External Community Review Committee (ECRC) with decision-making power regarding expenditure of the Institute’s research funds for “Type 2” (i.e., practice-based or community-engaged) translational research (in addition to requiring applicants to—at a minimum—involve a representative of their stakeholder community in their research project). This creation of the ECRC marked a departure from typical approaches to engagement by moving beyond community engagement in an advisory role⁸ to one that shares decision-making power. Our search of the literature did not reveal the existence of other processes similar to the ECRC, although the Canadian Institutes of Health Research, the Western Australian Health Department, and the Cancer Council NSW have consumer community members review their research grant applications (members do not have a voting/funding decision role),^8,9 and we did find recommendations within the literature for developing grant application and review processes that incorporate the values of community-based participatory research (CBPR).^6,7 Our goal in creating and engaging this committee comprised of community stakeholders is to bridge the “lack of fit between the dynamics of true community collaborations and the peer-reviewed funding approach to setting research priorities.”¹⁰

Our community-academic partnership is represented fully in the writing of this manuscript through the voices of Dr. Smith (the faculty program director), Ms. Kaufman (an ECRC member and former public health nurse and consultant focused on health issues of importance to minority and low-income women and children), and Ms. Dearlove (a staff member with experience in both academic and non-profit community organizations).

Defining “Community”

With the release of its Community Participation in Research R01 PA-08-077,¹¹ the NIH broadly defined community as “referring to target populations that may be defined by: geography; race; ethnicity; gender; sexual orientation; disability, illness, or other health condition; or to groups that have a common interest or cause, such as health or service agencies and organizations, health care or public health practitioners or providers, policy makers, or lay public groups with public health concerns.” UW ICTR adopted and implemented this inclusive definition of “community” for its pilot awards program—requiring applicants to incorporate community input into their research projects and engaging community health stakeholder input in the research review and funding decision process as members of its ECRC.

The External Community Review Committee— Formative Decisions

UW considered several community-engagement options for its review process, including initial review of proposals by community members to assess the value of the research to the Wisconsin community; incorporation of community members on the scientific review panel; and final review of proposals by community members. The latter approach was selected because it carried the most decision-making power, making the ECRC ultimately responsible for making the final funding recommendations.

Although enthusiastic about engaging community members in the pilot process, institutional leaders initially voiced concern that community members would not adequately understand the scientific merit of the proposed research and would decide to fund proposals with low scientific merit. A second concern was whether the ECRC could identify faculty or projects that would succeed in subsequent applications to NIH and other research funders. A final concern was whether community members would bring institutional biases or focus only on their own priority issues. The faculty lead of the grant program, in an “engagement motivator” role,¹² worked to address each of these concerns and promote the intrinsic value of shared control over decisions. For example, we assured institutional leaders that only proposals with an adequate level of scientific merit would be forwarded to the ECRC for their review, and developed an initial ECRC review process that tied the community member scores to those determined during the scientific merit review process (ECRC members could score within a set positive or negative range from the scientific score). Although these changes to the ECRC process helped institutional leaders be more comfortable with this approach, they were ultimately required to take a “leap of faith” that outcomes of the process would have strong value to both faculty and the community.

The Pilot Program in Practice

The UW ICTR Type 2 Translational Research pilot award process is designed to attract and award scientifically meritorious research; thus, most of the application requirements and review criteria mirror components of NIH and other federal calls for proposals. But UW ICTR also used the application process as a mechanism for shaping how academic investigators think about and conduct research that is relevant to and engages communities most likely to implement the research findings. We addressed this through (1) annual changes to the proposal requirements, pushing investigators farther each year toward the principles of community engagement, (2) the criteria set forth for the reviewers of both scientific and community review, and (3) adding a new grant program in 2009 ($200,000 for 2 years) that required more robust collaboration and shared funding with community partners.

Table 1 shows the original application requirements/process and how the influence of the ECRC (through their recommendations for changes to the pilot process each year) encouraged a new paradigm that set community engagement as a required element of funded pilot research.

Table 1.

Pilot Program: Original RFA Requirements and Changes Over Time

Original RFA Requirements (2008, first round of funding
$50,000 for 12 months
Proposal sections:
Scientific Abstract
Budget
Narrative:
Background & Significance
Preliminary Studies
Research Design & Methods
Translational Nature of Project
Each applicant must address 2 of these special criteria:
Community Collaboration
Interdisciplinary Collaboration
ICTR translational program Collaboration
Marshfield Research Foundation Collaboration
New junior-senior faculty team collaboration
Targeted project:
Pediatrics
Health Disparities
HIV/AIDS
Letters of Support, if applicable
RFA Additions & Changes per ECRC Recommendations
2008 (2nd round of funding)
Every proposal must also include a Community Abstract Letters of support from both academic and community collaborators mandatory and required at both the “letter of intent” and full proposal stages
2009
Collaboration with a community stakeholder no longer considered an optional “special criterion.” Every applicant must indicate who their community collaborator is and what role they will play in the research project Mandatory grant writing workshop added to the process. Taught by a member of the ECRC. Emphasis on how applicants should communicate their research to a lay audience. New grant program added: $200,000 for 24 months; some of budget must go to community collaborator
2012
Additional sections added to application: applicants must include dissemination plan and a paragraph describing what they anticipate the future policy implications of their research findings will be.

Methods

How ECRC Members are Chosen and Introduced to Their Role

Significant effort went towards the composition of the ECRC. Our goals for identifying members included engaging individuals who had a basic understanding of the grant review process, primarily since they had participated in community development/implementation grant reviews themselves, and who would be able to participate in discussions about research proposals without advocating for a specific disease, condition, or approach.

Our goals for building a committee were to engage a group who would represent rural and urban voices, different disciplines, perspectives, races/ethnicities; who would be committed to this endeavor and partnership; and finally, who would feel comfortable expressing concerns and honest opinions about this approach in order to strengthen the committee’s influence over time.

To ensure membership met these goals, we built a “potential member matrix” from recommendations by academic and community members. Input was solicited from faculty who were experienced with community-collaboration in research, as well as representatives of state and local government, existing community advisory board members throughout the state who been engaged in research through CBPR processes, health practitioners, etc. Once populated, this list was shared with a core group of faculty and staff who chose the first 12-member committee; a similar process was used in subsequent years and we maintain the number of members at 8–12 each year.

Since 2007, ECRC members have come from health services agencies (hospitals, federally qualified mental health centers, large and small health systems, advocacy organizations, Wisconsin Medical Society); governmental agencies (State Department of Health, State Division of Public Health, county health departments, the Oneida Nation, former legislator); nonprofit organizations (Hospice, League of Women Voters); and educational organizations (UW Extension, state technical colleges, universities).

Between 2008 and 2012, a total of 32 unduplicated members served on this committee; each year a core number of members (up to 75%) return to provide continuity and historical perspective for the process. We provide honoraria for participating members, as many have hours of travel and time away from work.

The ECRC Training Process

We deliver a training primer prior to ECRC meetings to affirm the goals of the ECRC process and pilot grant program. When a group of prospective members is selected, staff contacts each individual to introduce the mission of the grant program, the intent of the community-driven committee, and ascertain interest and availability. If interest is expressed, the prospective member is sent a copy of the current “request for applications” (RFA) as well as a briefing document defining “type 2” translational research and the UW ICTR’s pilot award review process (see Appendix A).

Because much correspondence is initiated before actual proposals are submitted, ECRC members have approximately four months before they receive proposals. Once proposals go through the first three steps of the review process (technical, peer, and study section review), selected materials from those proposals that have achieved a minimum level of merit during the scientific review part of the process are sent to ECRC members for their review in advance of the ECRC meeting.

The final step of the ECRC training process takes place at the actual meeting, before discussions begin. First, there is time allotted at the beginning of the agenda for the committee members to get acquainted or re-acquainted before the actual proposal discussions begin. The Chair then presents a PowerPoint (see Appendix B) that provides an overview of the CTSA (UW-ICTR), a review of the kinds of research that this award program supports (“type 2 translational” research), conflict of interest/confidentiality, and the tasks before the committee for that day—with ample opportunity for questions and answers. We provide coffee, snacks, and lunches to members during the day’s activities.

The ECRC Meeting

Members review but do not score components of all the scientifically meritorious proposals in advance of the meeting. These materials initially included only the scientific abstracts for review, but each year, ECRC members requested that additional sections be added to the RFA requirements—and that they subsequently receive these sections in their review portfolio. ECRC review portfolios now include a community abstract (added requirement in 2009); letters of collaboration and support from community collaborators (2009); sections of the proposal narrative that address the nature and extent of community collaborations (2009, 2010, 2012); and a section related to dissemination plans and future policy impact of research findings (2012).

While the portfolios initially presented proposals in scientific merit order, as institutional leaders grew more comfortable with the process—and at the request of the ECRC—we discontinued the approach of tying ECRC scoring to scientific merit scores and presented proposals in alphabetical order for the ECRC to rank based solely on their review criteria, analysis, and discussion.

ECRC Discussion and Scoring

After the initial presentation, the Chair presents a summary of each proposal to the group and then opens the discussion. ECRC members use the following criteria to evaluate and score proposals:

• Significance: Does this research area address important problems or critical barriers to progress in advancing/improving clinical practice and/or community health and policy? Does this project have long-term potential to contribute to the advancement of health?

• Priority: Should the aims of this proposal be a research priority for UW ICTR?

• Community: Does this research project effectively and meaningfully incorporate the input of community partners/stakeholders?

Every committee member is asked to review and comment on every proposal (except in the case of a conflict of interest); therefore, there is no “lead presenter” who begins the conversation per se. Members take turns discussing the merits of each proposal based on the review criteria and make specific recommendations to the applicants when warranted (e.g., suggestion of an alternative or additional community partner, advice with regard to the feasibility of a given approach based on a member’s unique perspective of the community involved, etc.). As expected from the literature regarding effective community-academic partnerships, members want to see evidence within the proposals that the research team values community contributions and focus on community-identified issues and opportunities.¹³ Disagreements among committee members are addressed until no further discussion is warranted and then each member assigns their scores on a private score sheet (see Appendix C). PowerPoint slide deck for meeting available at See cpr-7-3-301-supp02.pdf ^{(949.1KB, pdf)}

All comments are captured in the meeting minutes and are shared with applicants as part of their de-identified critiques. For those who are awarded funding, they are required to submit a letter to UW ICTR responding to all review concerns (both scientific and community) before they can receive funding.

A significant change in the ECRC discussion and scoring process occurred in 2012 when the oversight and management of the committee’s proceedings was turned over to a community member. From 2007 to 2012, the faculty lead managed the ECRC meeting and clarified questions related to the mission of the committee—without participating in discussions or scoring. In 2012, the discussion was led by a new “community” chair, with no faculty members present at the ECRC meeting. This change was possible for two reasons. First, the presence of the faculty lead was always intentionally limited to a coordinator, rather than a discussion participant; therefore, the transition seemed timely and certainly justified by the committee’s mission, role, and design. Second, approximately one third to one half of the committee members had participated in every meeting since the first round of funding—and the relationship/trust built among the members, the faculty lead, and the staff allowed for amenability to this change.

Results

Over the first five rounds of the program, ICTR received 141 Type 2 Translational Research applications, and of those received, the ECRC received a total of 63 scientifically-meritorious applications to review. UW ICTR was able to fund 41 awards.

Overall, the 41 funded applicants have shown a return on investment of approximately $20 million in federal/external non-federal grants and 16 publications to date.

The ECRC intervened and changed funding decisions 21% of the time, such that 9 projects were chosen for funding because of ECRC review; in other words, they would not have been funded had it not been for the ECRC. These 9 projects have provided a substantial return on investment—including 6 funded external peer-reviewed grant submissions, for a total of almost $4.5 million, as well as 7 publications in peer-reviewed journals. This translates to the “rescued” projects representing 23% of the subsequent grant funding and 44% of the subsequent publications.

Although the sample size is too small to offer a causal explanation for their success, we know that the research programs of these PIs were highly attuned to the interests of funding agencies and represented programs for which funding was available, such as patient-centered care research and health equity/disparities research. It should also be noted that the ECRC members agreed with scientific reviewers on the remaining 32 projects—they recommended that they be funded as well.

For those academic health centers or clinical and translational science award recipients who may be interested in using this model of community engagement, we would like to share our own insights from this process that may provide value to those considering replicating this model of engagement.

1. Communicate openly and consistently about roles and expectations; and anticipate some conflict

Each year, conflicts arose, resulting from different priorities, perspectives, values, and language. We had extensive discussions about issues crucial to meaningful scoring, such as: how to define “community” (e.g., whether research conducted in collaboration with clinicians at academic health center clinics was a “community” partnership); how to determine “impact/significance” (e.g., by the percentage of state population affected versus severity of disease); and how to assign “priority” (e.g., individual-focused versus community-wide interventions).

We also depended on open communication in order to ensure that the review process was one of genuinely shared power. Each year that the committee asked for changes in the RFA, requested additional pieces of the proposals to make their decisions, or wanted to untether their scores from those assigned during scientific review, we discussed the pros and cons of these changes together, and then for significant changes, the faculty lead made the case to ICTR administration to modify our methods and codify these changes.

Feedback from community reviewers was synthesized and considered after every review cycle. UW ICTR’s willingness to accept changes to the process recommended by the ECRC created opportunities to improve the application and review processes in ways that strengthened both the funding program and the relationship between academia and research in the community and ultimately, we believe, resulted in stronger applications. (See Table 2 for quotes from ECRC members regarding their appreciation of how the pilot process changed to reflect their concerns and feedback.)

Table 2.

ECRC Commentary

What is your understanding of the goals of this committee?
“My understanding is that this committee’s role is to help bring the community perspective to the research activities of the University so that translation from research to implementation is a process that makes more sense, is more practical and is hopefully shorter for those of us who are likely end users—to give us a kick at the can early in the process.”
“It is the intersection between the real world and academia in the largest sense. It allows community input in a traditionally academic process.”
“This process is the only one like it [that I know of] where the community gets the final word. When it gets to us –scientific review has been completed—we get to speak as a community voice.”
“The process offers a balanced and more level playing field in which our comments are incorporated into research for the betterment of the proposal and best use of the funding.” “In the larger context of funding processes, this is radical but hopefully of great interest to funders.”
Do you feel that this committee has had an impact and if so, what evidence do you see of this impact?
“We do score somewhat differently from scientific review—and one would assume that the members of this committee must score somewhat similarly. If the traditional process had been used, different projects would have been funded—that is impact of this committee.”
“We can see that our recommendations are reflected in each year’s new RFA. And we feel that the proposals have gotten better every year.”

2. Expect changes—from both academia and community members—to be incremental and be prepared for the time commitment

Recognizing the evolution of the role of the ECRC over time is important for those considering implementing this model. UW initially expressed concerns about creating a review component that gave funding decision-making authority to a group of non-academic community health stakeholders after scientific review. Community members were at first concerned that the process would not truly share decision-making power, and that decisions “not liked” by institutional leaders would be reversed or ignored. Using an incremental approach over a five-year timeframe proved critical to success, allowing time for academic and community stakeholders to develop trust and comfort with engagement processes and change.

Engaging communities in making funding decisions about research requires commitment to the process and a long-term dedication to keeping eyes on the prize. Having an academic leader or “champion” who was accountable for creating, managing, and consistently advocating for these processes was vital to the program’s success. This individual must have internal respect and be capable of managing the tensions between the hierarchy of an academic research setting and expectations of community member reviewers.

3. Choose the right partners for your mission

Success in this experiment required creating defined roles and engaging people with “the right stuff” to fill them. On the academic side, commitment to community engagement and leadership by the ICTR Principal Investigator (PI) proved to be vital in making wide-sweeping changes over a short time trajectory, especially because change involved determining who received funding. The faculty lead and the ICTR PI were engagement motivators.¹² The faculty lead paid careful attention to process and system issues, communicating the goals and norms of the program clearly, assuring transparency, and exemplifying sharing of power and true collaboration.

Hiring “boundary spanners” (non-faculty staff who ran the day-to-day processes and communicated extensively with the faculty lead, research applicants, grantees, and ECRC) proved critical to our success.¹² This staff listens, interprets thoughtfully, and communicates clearly. Their passion for academic research that makes a difference in community practice clearly reduced friction and led to creative solutions for problems that arose. Both had experience in community and university settings, and could speak the language of both camps, while conveying fairness and shared power.

University assets were balanced by thoughtful and experienced community reviewers. Diversity of backgrounds and experiences in delivering and managing health services and policies were enhanced by choosing reviewers who have applied for community development/implementation grants and managed them. It was crucial that reviewers did not come to the table to advocate on behalf of any organization that they might be associated with—but instead consider the overall needs of the health of Wisconsin residents (this expectation was reiterated at each meeting). Over the five years of these meetings, no one individual overwhelmed the group’s discussion—members naturally take turns presenting their perspectives and actively engage other members in discussion. Finding committee members who are committed to this group process is invaluable.

Discussion

While we recognize that engaging community reviewers in discussion and rating of research proposals is not novel, this method of engagement differs in that it allows for a community review panel to re-rank scientifically meritorious proposals—using community engagement criteria—with the result that proposals funded do not necessarily follow the rank order from the first level of scientific peer review.

Co-review models in the literature present goals of community reviewer programs as providing mechanisms for public accountability, increasing transparency in the peer review process, providing feedback on lay abstract portions of applications, and communicating merits of investments in health research.¹⁴ These programs note, however, that “Community Reviewers do not score nor appraise the scientific quality of the applications.”⁸ NIH has also made contributions to the discussion of community engagement in research; however, these recommendations and practices refer primarily to training researchers how to effectively conduct and participate in community-engaged research and training reviewers how to best evaluate community-engaged research.⁶

While the outcomes of co-review have important implications for community-academic partnerships, they do not tackle the goal of providing shared power to community partners in academic research funding decisions. This recommendation is clear in the CBPR literature, specifically, “the input of the community participants in the review process must be heard and incorporated into the final decision-making processes. For example, community members need to be oriented to how the review process is conducted, their roles need to be clearly defined from the beginning, and how their input is going to be ``weighted” needs to be clarified. Community members’ perspectives and expertise might best be applied to assess specific partnership-related criteria across all applications, rather than taking a lead review role on the entirety of a few applications.”⁷ Each year we ask our External Community Review Committee members if they would be interested in a co-review process in which they would serve alongside scientific reviewers. Our ECRC members have consistently declined this approach, sharing that they feel this would diminish their role and their voice in the review process.

Perhaps not surprising is that the success of this approach may well be related to its design following the same recommendations found in the literature for creating successful community-academic research partnerships: the importance of personal connections, open and bidirectional communication, full participation by all parties, ability to deal with unexpected consequences, mutual benefit, and equal respect.^6,13

Engaging community reviewers as decision-makers for academic translational research grants created a successful program jointly owned by academic and community partners. Although it is still too early to determine long-term impact from grants selected by the ECRC for funding, indicators signal clear promise. The program achieved traditional academic metrics of subsequent grants and manuscripts for the pilot awardees, but also succeeded in engaging a cadre of enthusiastic community members from around the state who remain committed to the process.

The ECRC developed by UW ICTR represents a promising new strategy for engaging community members in research funding decisions. This model could be adopted or adapted by other academic funding programs wishing to increase the engagement of community members in their research processes. The advantages of this approach include success through traditional academic metrics while achieving a high level of community engagement critical for narrowing the gap between knowledge and practice.

Appendix A. ICTR Type 2 Translational Research Briefing Document

The Institute for Clinical & Translational Research (ICTR): Community–Academic Partnerships Core (2009 version)

What is the Institute for Clinical and Translational Research (ICTR)?

The mission of ICTR is to create an environment that facilitates the transformation of research into a continuum extending from investigation through discovery to translation into practice.

Created in 2007 with funding from National Institutes of Health and the UW School of Medicine and Public Health, ICTR includes core resources for education and training, biostatistics, bioinformatics, laboratory research, clinical trials, and community-oriented research.

ICTR Partners.

• School of Medicine & Public Health

• School of Nursing

• School of Pharmacy

• School of Veterinary Medicine

• College of Engineering

• Marshfield Clinic

What is the Community-Academic Partnerships (ICTR-CAP) Core?

As a core component of ICTR, ICTR-CAP seeks to support research partnerships that solve problems in translating new and existing knowledge into improvements in clinical practice and community health. Drawing on resources at the University and the engagement of our statewide community, we have created a collaborative network of organizations and partners with the aim of generating timely, relevant scientific findings that can be applied to improving health. Our efforts have focused on:

• Engaging faculty, enhancing their skills, and providing them with resources to conduct collaborative, multidisciplinary community-engaged Type 2 translational research, and

• Building community infrastructure, skills, and capacity to support successful and effective research partnerships that will improve clinical practice and community health.

How do we define “Community”?

ICTR-CAP has embraced the National Institutes of Health’s inclusive definition of community, which targets engagement of the “end users” of research.

“Community” refers to target populations that may be defined by: geography; race; ethnicity; gender; sexual orientation; disability, illness, or other health condition; or to groups that have a common interest or cause, such as health or service agencies and organizations, health care or public health practitioners or providers, policy makers, or lay public groups with public health concerns.

“Community-based organizations” refer to organizations that may be involved in the research process as members or representatives of the community. Possible community partners include, but are not limited to, Tribal governments and colleges, state or local governments, independent living centers, other educational institutions such as junior colleges, advocacy organizations, health delivery organizations (e.g., clinics, hospitals, and networks), health professional associations, nongovernmental organizations, and Federally-qualified health centers.”

National Institutes of Health 2008 Program Announcement.

PA-08-077, Community Participation in Research (R01)

http://grants.nih.gov/grants/guide/pa-files/PA-08-074.html

What is Type 2 Translational Research?

Translational Research focuses on moving scientific findings closer to real-world applications; this research has been broken down into two “types” according to the National Institutes of Health.

Type 1 Translational Research studies how to move basic science discoveries into clinical trials of new diagnostic and treatment methods.

Type 2 Translational Research includes research to determine whether or why major gaps exist in current practice and then asks questions to discover the best ways to translate new and existing findings into improvements. Type 2 translational research often engages community members, organizations, and clinicians as partners in the research process.

Why is Type 2 Translational Research important?

Examining the GAP Between Knowledge and Practice.

Research questions: Are recommended interventions used or even applicable in typical health delivery settings? Why or why not?

• Identifying the gaps between recommended guidelines for diabetes management and actual clinical practice

• Determining the reasons why individuals do not follow recommended guidelines for exercise

• Identifying subgroups of individuals for whom recommended interventions might have no benefit, or even be harmful.

The focus on Type 2 translational research is based on a growing understanding that while funding for the discovery of new medicines and techniques to improve human health is well-established, funding to ensure that these new treatments and findings actually reach patients and populations they are intended to help—and are implemented correctly and effectively—has not kept pace. Dr. Steven Woolf (2008) explains:

“At a time when experts warn of the fragmented health care system and of a widening “chasm” in access, quality, and disparities, interventions to close these gaps—the work of Type 2—may do more to decrease morbidity and mortality than a new imaging device or class of drugs . . .”

The Community-Academic Partnerships core of the UW-Madison Institute for Clinical and Translational Research (ICTR-CAP) is dedicated to increasing attention to Type 2 research throughout UW-Madison and the State of Wisconsin and to stimulating outstanding new Type 2 research that leads to peer-reviewed funding.

Developing, Evaluating and/or Disseminating interventions to IMPROVE Practice and Health.

Research questions: What interventions are most effective in changing the behavior of individuals, providers, organizations, or systems to improve practice and ultimately health? Examples include:

• Piloting a practice improvement program that targets greater adherence to guidelines, use of proven efficacious treatments, or efficiency in care delivery

• Evaluating an initiative to increase preventive screening behaviors such as screening for breast, cervical or colorectal cancer

• Evaluating health promotion programs targeting individual behaviors such as quitting smoking or increasing exercise in an employer setting

• Developing patient safety initiatives targeting systems redesign in hospitals

Examples of 2008 Pilot Project Topics.

• Diabetes (prevention and assessment tools, 2 pediatric and 1 adult study)

• Improving management systems in ICU and cardiac care units

• Remote assessment of traumatic brain injury

• Screening and treatment for chronic diseases (chronic kidney disease, depression, colon cancer)

• Treating tobacco use, drug addiction and medication adherence in partnership with community agencies

• Obesity prevention program feasibility studies

• Patient-centered and family-centered care assessments

For a complete list of project titles and abstracts, please visit http://www.ictr.wisc.edu/node/263

To see the 2009 Type 2 Translational Research Grant Writing Workshop (including an overview of Type 2 Translational Research), please visit

http://videos.med.wisc.edu/videoInfo.php?videoid=6401

http://videos.med.wisc.edu/videoInfo.php?videoid=8469

http://videos.med.wisc.edu/videoInfo.php?videoid=8470

The ICTR-CAP Type 2 Translational Research Pilot and Community Collaboration Programs

2008 was the first year that ICTR-CAP offered funding for Type 2 translational pilot research. The goal of the funding program is to support the production of excellent, collaborative Type 2 translational pilot research that addresses important health issues and that will lead to further external, peer-reviewed research projects.

In 2008, during two consecutive rounds of funding, we received 60 pilot applications. Pilot grant submissions were scored by content and methodology experts and scientifically meritorious applications were forwarded to an External Community Review Committee for final decisions. Eighteen Type 2 translational pilot research projects were awarded.

This year, ICTR-CAP will expand its funding mechanisms to include both the Type 2 Translational Research Pilot Program and a new Type 2 Translational Research Community Collaboration Program. The new Community Collaboration Program is intended to support larger research projects that solve problems in translating clinical and health-related scientific knowledge into meaningful changes in practice, with the goals of improved health outcomes and/or reduced costs in/for the State of Wisconsin.

Research Proposal Review Process

Applications submitted for the pilot or community collaboration grants will undergo a 5-step review process:

1. Technical review to assess application completeness and ensure issues of non-supplanting funding and conflict of interest have been addressed

2. Scientific review by a group of three experienced researchers

3. Ranking by the full ICTR-CAP Steering Committee

4. Final determinations made by External Community Review Committee based on project summaries and consideration of priorities

Appendix B.

See cpr-7-3-301-supp01.pdf ^{(803.8KB, pdf)}

Appendix C. ICTR Type 2 Translational Research ECRC Scoring Guide & Score Sheet

Institute for Clinical & Translational Research Community-Academic Partnerships Core: 2012 External Community Review Committee Scoring Guide

Please base your scoring of this project’s overall impact on the following criteria:

1. Significance: This research area addresses important problems or critical barriers to progress in advancing/improving clinical practice, community health and/or policy. The project has long-term potential to contribute to the advancement of health.

2. Priority: The aims of this proposal should be a research priority for the UW-Madison and Marshfield.

3. Community: This research project has effectively and meaningfully incorporated the input of community partners and/or end users. The following questions are asked of the PI and research team in their proposal.

   • “Who is the constituency or group that will benefit from your research? Please describe how a representative of that group is involved in your research and what role they will play.”
   • “How and when will your project results be disseminated to/with your community partner and/or other specific clinical practice, community or policy audiences? Developing a dissemination plan is a key part of the collaborative research planning process and can help your team focus the project and identify key audiences. Note: the sophistication of this plan should match the scope of the project and care should be taken for careful interpretation of results.”

Scoring Calibration Guide

The 2012 ECRC Scoring is based on NIH’s scoring system which utilizes a 9-point rating scale (1 = exceptional; 9 = poor). The overall impact score for each discussed application will be determined by calculating the mean score from all the eligible impact scores, and multiplying by 10; the overall impact score will range from 10 (high impact) to 90 (low impact).

Scores	Descriptor	Anchor
1	Exceptional	Exceptionally strong potential for impact in a highly significant area
2	Outstanding	Extremely strong potential for impact in a highly significant area
3	Excellent	Very strong potential for impact in a highly significant area
4	Very Good	Strong potential for impact in a highly significant area
5	Good	Strong potential for impact in a moderately significant area
6	Satisfactory	Some potential for impact in a highly significant area
7	Fair	Some potential for impact in a moderately significant area
8	Marginal	Limited potential for impact in a highly significant area
9	Poor	Limited potential for impact in a moderately significant area

NOTES:

1. Please use all of the information provided to you to assess the degree to which applicants have met these criteria.

2. We encourage you to use the full scoring range as you review each proposal.

3. Each application that has been forwarded for your review has been through two phases of scientific review and has been determined to have achieved scientific merit, meaning that the scientific reviewers were comfortable that the scientific goals can be achieved.

ICTR External Community Review Committee SCORING SHEET

Your Name: ______________________________________________________

FUNDING PROGRAM	PI NAME	TITLE OF PROJECT	YOUR SCORE
Pilot ($50K for 1 year)	A
	B
	C
	D
Collaboration Grant ($200K for 2 years)	E
	F
	G