Putting Practice into Research

For many years, health services researchers have contended that clinicians and others who deliver, manage, purchase, or influence health care should seek out and use the latest and best evidence when making clinical, managerial, and policy decisions. This campaign has brought both good news and a challenge.

The good news is that the message is meeting a receptive audience. This past year, for example, when large employers formed the Leapfrog Group to use their collective purchasing power to improve patient safety, they began with extensive reviews of the research on factors and practices associated with safe outcomes and cited this evidence in their messages (Leapfrog Group 2001).

The challenge is that the onus is now on the research community to show it can carry out its half of the job: producing information that is timely, on point, and in a format decision-makers can use. Meeting this challenge requires researchers do the same thing they ask of the health care system itself: talk with their customers, closely examine their systems and processes to see what works and does not work, and do the reengineering and innovation necessary to make sure that research, like health care itself, is more timely, efficient and customer-centered.

To ensure responsiveness to user needs, the Agency for Healthcare Research and Quality (AHRQ), as part of a broader set of initiatives, started such conversations years ago. The Agency consulted with a wide range of existing and potential users of our research to determine how it could improve what we do and how we do it. One innovation directly resulting from this consultation and redesign process was the decision to support practice-based research networks, in which people in the field delivering health care or managing health care systems are an integral part of the research enterprise. Over the past two years, AHRQ created an Integrated Delivery System Research Network (IDSRN) and also developed new planning and infrastructure grants for primary care practice-based research networks (PBRNs). This AHRQ Update discusses the impetus and rationale for the new programs, briefly describes the IDSRN and PBRN initiatives, and then discusses the broader implications of these models for the research community.

BACKGROUND

To ensure its research is responsive to user needs, AHRQ has held formal and informal meetings not only with the research community, but also with current and potential users of health services research, including clinicians, systems administrators, and policy-makers at the federal, state and local levels. One clear message heard from many decision-makers was their desire to practice evidence-based medicine, evidence-based management, and evidence-based policy-making, and to do so, their need for more timely information, and information relevant to the decisions at hand. In particular, they noted a need for information on how to improve access, quality, efficiency and safety in ambulatory care.

Researchers said they wanted their work to be relevant and useful, but felt encumbered by the fact that much of the relevant ambulatory care data was proprietary, and interpreting the data often required partnerships they did not have with practitioners within these organizations.

The Agency also heard from practice-based researchers; individuals whose primary responsibility is patient care, but who also contribute to research efforts in primary care or managed care settings. Some of these practice-based researchers have access to excellent data about ambulatory care, but often lack the resources or connections to do relevant research that both their organizations and the broader community of decision-makers could use. For example, they told AHRQ that they often design and implement clinical or organizational interventions designed to improve care—interventions that called out for evaluation research and potential replication elsewhere—but by the time they organize and fund an evaluation component through conventional grant mechanisms, the intervention is over and the opportunity to learn from it is systematically lost.

The cumulative recipe for relevant research emerging from these groups includes field input on the most important questions to address and how to address them in clinical settings; a fast-track way to fund and organize the inquiry; a way to get information to users (and check its applicability and adaptability); and a direct and succinct dissemination path for the eventual lessons, tailored to the needs of users. Agency responses to this recommendation included the creation of the IDSRN and new support for PBRNs. As the next sections show, the two initiatives differ significantly in structure and mechanism, but share three features designed to implement these recommendations; features that increasingly are part of the broader research enterprise at AHRQ. These features are: (1) strong links between researchers (clinicians and social scientists) and those clinicians and others who deliver care or manage health care organizations; (2) collaboration and synergies across research projects; and (3) creation of a sustained infrastructure for research.

IDSRN

Directly or indirectly, most health care in the United States is provided through complex health systems such as managed care organizations, hospitals and hospital networks, large physician groups, and nursing homes. As a result, these organizations have become increasingly important as both creators and users of information. Many have considerable research capacity, including sophisticated data systems that follow patients over time and across sites of care; ties between research and operations staff; and strong teams of researchers. (Nelson, Quiter, and Solberg 1998). On the other hand, most delivery systems, even some very large ones, do not have these capacities. In creating the IDSRN, AHRQ sought to expand the capacity for research in and among integrated delivery systems, and to create a mechanism for faster, more collaborative research.

The IDSRN consists of nine consortia which operate under a three-year master task order contract, with an option to renew for another two years. Together, the consortia provide care to over 50 million Americans, including privately insured, uninsured, Medicare, and Medicaid patients. The delivery systems and the populations served are also diverse. Some are group or staff model HMOs, while others are network plans or fee-for-service providers. The sites span the country, serving rural, suburban, small town and inner-city patients through health plans, hospitals, ambulatory care practices, nursing homes, and home health plans, all with strong data systems and in-house or affiliated research capacities. The size of the networks, their diversity, the strength of their data, and their new opportunities for collaboration through the network create a powerful opportunity for research, enabling them to do research on the impact of different organizational or clinical interventions on many subpopulations of interest (see www.ahrq.gov/research/idsrn.htm).

As this article goes to press, the Agency has awarded a total of 26 task orders. Most are short turn-around projects, spanning 12 to 18 months, using health plan data. Most focus on ambulatory care issues, though a few study hospital or nursing home care. A large number of the recent task orders focus on patient safety (in keeping with the Agency's 2001 funding in this area), but other priority areas include quality measurement and improvement, racial and ethnic disparities, evaluations of organizational and clinical interventions, information technology, cultural competency, perinatal health, and bioterrorism. (See Table 1.) Most projects have been funded by AHRQ, but several were sponsored by other Department of Health and Human Services agencies and the volume of external sponsorship is expected to grow.

Table 1.

IDSRN Partners and Their Collaborators (http://www.ahrq.gov/research/idsrn.htm)

The rapid turn-around nature of these contracts permits the Agency to quickly generate findings to priority policy questions. Last year, for example, an Institute of Medicine roundtable meeting examined the policy implications of studies linking high volume of hospital procedures with better outcomes. One of the participants noted that little is known regarding what, within the process of care, causes this association, or what lies behind some of the outliers (small volume hospitals with good outcomes and large volume hospitals with poor outcomes). The Agency decided to use the IDSRN to learn the answer, and the Center for Health Care and Evaluation at UnitedHealth Group is now doing a qualitative study focusing on the processes of care at high- and low-volume hospitals with good and bad outcomes. In another instance, the Center for Medicare and Medicaid Services (CMS, formerly the Health Care Financing Administration, or HCFA) wanted to identify some potential models through which managed care organizations (including group and network models) could improve cultural competence, so the HMO Research Network will create some models under the IDSRN.

New topics will be derived from Agency expert meetings (for example, a new project on linking maternal and child data was the direct result of an expert meeting on Women's Health), and Departmental priorities (projects examining and seeking to expand health plan capacity to gather information about race and ethnicity). Additionally ideas are likely to emerge from the IDSRN itself, and in particular, from the operational leadership of integrated delivery systems. And, of course, the Agency welcomes suggestions from the research community.

In January we expect to hold the first annual meeting of the network. These meetings will provide an opportunity to discuss potential new research questions and funding sources, ways the network can collaborate across sites to enhance the value of the research it produces, and ways the network can work more closely both with users of the research and with the broader research community. The Agency welcomes researcher input on these questions as well.

PRIMARY CARE PRACTICE-BASED RESEARCH NETWORKS (PBRNs)

While most of the IDSRN contractors began with very strong data systems but with less history as “networks,” most of the primary-care based research networks (PBRNs) had less well developed data systems, but more experience conducting network research. A PBRN is a group of ambulatory practices devoted principally to the primary care of patients, affiliated with each other (and often with an academic or professional organization) in order to investigate questions related to community-based practice. Each PBRN is a collaborative network of office-based practices and is capable of rapidly identifying clinically relevant questions in primary care practices and producing rigorous research. PBRNs are characterized by an abiding commitment to research and an organizational structure that transcends a single study.

Since the mid-1970s, when PBRNs first appeared in the U.S., the number and maturity of those networks have increased substantially, particularly over the past decade. About half of the existing PBRNs are regional or local in scope, two are national and most others are statewide or multi-state. Collectively, family physicians represent about 65 percent of the participating clinicians, pediatricians about 25 percent, with general internists, OB/Gyns, advanced practice nurses and physician assistants comprising the other 10 percent.

An important feature of almost all PBRNs is the close collaboration between practicing clinicians and researchers. With traditional research approaches, study questions are typically generated by researchers who determine the research methods and interpret the study findings. Within most PBRNs, participating clinicians help define and frame practice-relevant research questions and take part directly in data collection and interpretation of results with the research team, bringing scientific rigor to the effort. This “bubble-up” approach to research produces results which can “follow a shortened feedback loop into practice” (Nutting, Beasley, and Werner 1999).

For example, a study conducted recently in the Oklahoma Physicians Resource/Research Network focused on optimal methods of managing laboratory test results in practice (Mold, Cacy, and Dalbir 2000). The study was intended to be a cross-over trial of alternative methods of managing results. However, as soon as it became apparent that one method under consideration produced superior results, most participating practices immediately adopted all or part of it. In theory, the process of applying the results of any research (including biomedical research) to the practice of medicine may be less onerous in PBRN settings.

Despite considerable success in producing research, most PBRNs have been challenged over the years to find adequate funding for research endeavors, particularly for the supporting network infrastructure. Almost all have relied heavily on volunteerism for central staff support and the cooperation of participating clinicians. Universities or professional organizations affiliated with the networks often provide a modest amount of support. In addition, PBRNs have received funding for individual research projects. From 1990 to 1999, AHRQ (then known as the Agency for Health Care Policy and Research) spent almost $13 million on studies conducted within primary care-based research networks. Most PBRNs, however, have not matured sufficiently to compete successfully for major federal grants. (A significant limitation has been the lack of adequate information technology to allow networks to collect and aggregate research data from practices. In fact, most networks still rely on paper and pencil methods of recording data.)

To assist PBRNs in carrying out activities that enhance their capacity to conduct research, AHRQ awarded planning grants last fall to 19 networks across the United States1 (see Table 2). Together, these PBRNs provide access to more than 5,000 primary care providers and almost seven million patients who are being followed in a variety of primary care practice settings in 49 states. Each grant supports the development of a PBRN-specific plan to: (1) establish or improve electronic collection and aggregation by the network of data derived from the individual practices; (2) increase the network's capacity to study the health care of racial and ethnic minority and/or underserved populations; (3) improve the ability of network practices to translate research findings into practice; and (4) identify potential sources of ongoing support for the network (see www.ahrq.gov/research/pbrnfact.htm).

Table 2.

Primary Care-Based Research Networks (PBRNs) (http://www.ahrq.gov/about/cpcr/cpcrover.htm#PBRN)

After successful completion of the planning phase, the PBRNs competed for continuation grants. Continuation funds will allow the PBRNs to define more accurately the clinicians, patients and practices involved in each network; information that has been difficult to obtain because of limited data collection capacity. In addition, several of the PBRNs will pilot test various methods of electronic data collection in network settings. Others will study issues related to data privacy and confidentiality in the conduct of primary care research.

IMPLICATIONS FOR RESEARCHERS AND USERS OF RESEARCH

These two new initiatives are quite different from one another, as well as from other programs at the Agency. IDSRN builds on integrated delivery system networks and a mixture of clinical and social science researchers, while PBRNs build on primary care networks and largely clinical researchers. The former works through task orders and the latter through grants. Both are something of a departure from the Agency's past in that they provide infrastructure and support for sustained field-based research programs.

In one fashion, however, these two initiatives are similar: they bring together three principles the Agency has been pursuing to make its research more timely and useful to decision-makers. These trends affect opportunities for all health services researchers, regardless of whether they are involved, or expect to be involved in these particular networks.

Principle #1: Build strong links between researchers and those who deliver care or manage health care organizations

A strong trend in Agency research is linking researchers more closely with clinicians and delivery system leaders. Through projects such as the Consumer Assessment of Health Plans (CAHPS^®), the Child Health Insurance Research Initiative and others, the Agency has found that bringing practitioners into the research enterprise can increase the utility and actual use of research.

First, close practitioner–researcher links help us target research to priority questions: the closer the link between researchers and the users of research, the greater the likelihood that research will seek to address the questions users want answered.

Second, these links can increase access and appropriate use and interpretation of provider data. Much of the data needed for clinical and delivery system research are proprietary, and few researchers outside these organizations have used it. Close practitioner–researcher partnerships make it more likely that the data will be used, used properly, and—where necessary—improved for future use.

Third, provider involvement in teaching and research can have spill-over effects, reinforcing a practice of looking to the relationship between evidence and decision-making.

Fourth, clinicians and other decision-makers are more likely to see the applicability of information coming from people and institutions like their own. In both the IDSRN and PBRNs, a major focus of activity will be ensuring that early findings from individual projects reach the organizational leadership of all network members, and indeed the leadership of those who are outside the network. For example, when the United Health Group identifies organizational and other the factors associated with better outcomes in high volume and low-volume hospitals, these findings will be of potential interest not just to United Health Group but to all health plans and purchasers seeking to improve quality through their hospital contracts. Or, when the Virginia Ambulatory Care Outcomes Research Network (ACORN) completes its AHRQ-supported project describing and classifying ambulatory medical errors, its classification scheme will be of interest and potential use to all health care providers and quality improvement coordinators in ambulatory care settings.

Principle #2: Create collaborations across related projects

A second principle of the Agency is to encourage collaboration and synergies across related research projects. Collaborations can increase the efficiency and impact of research in several ways. First, they permit studies of infrequent events, or focused on small subpopulations such as children or minorities, or people with rare diseases. Second, collaborative studies permit one to reach conclusions about the generalizability of findings across different types of delivery systems, and different financial arrangements. Finally, collaborations enable researchers to synthesize findings across projects, in order to explain any differences among the studies (Durham 1998).

Collaboration is at the core of both the PBRNs and the IDSRN, although they follow somewhat different models. In the PBRN, the Agency has identified and strengthened existing collaborative networks of primary care providers. With the IDSRN, some of the contractors (e.g., the HMO Research Network) already existed as networks, while others came together for the first time under this initiative. Moreover, the networks are encouraged to collaborate among themselves to eventually form a network of networks. In fact some of the earliest projects involved several of the contractors.

Principle #3: Create an infrastructure for sustained research over time

A third principle for the Agency is to build a sustained infrastructure for research in priority areas, so the efficiency of the research enterprise can be enhanced. While practice-based networks have been around in the United States for some time (Nelson, Quiter, and Solberg 1998; Durham 1998), most have had rather tenuous lives, competing for and episodically receiving grants from AHRQ and others, but lacking a sustained, predictable source of funding and staffing. A major goal of the PBRN and IDSRN initiatives was to create a sustained infrastructure of research sites that could conduct families of related projects over time. An investment in this infrastructure would help create a system of practice-based laboratories for the study of everyday health care, following the model of the sustained biomedical research study groups of the National Institutes of Health and others. The kinds of action the Agency is taking to build infrastructure in the practice-based research networks—use of planning grants, use of multi-year grants and contracts, leveraging agency dollars by collaborating with other funders—also represent a broad trend across the Agency.

CONCLUSION

Most researchers and funding agencies have a strong professional interest in seeing that their findings are used to improve practice and policy. For an agency working with public funds, achieving this goal is not a preference, but an imperative. To achieve these goals, the Agency is forging links between researchers and practitioners, encouraging collaboration and synergies across research projects, and bolstering the infrastructure for research. Practice-based research initiatives supporting the IDSRN and PBRNs provide one of the Agency's best vehicles for achieving these goals.