Comparative Effectiveness Research

SUMMARY:

The goal of comparative effectiveness research is to improve health care while dealing with the seemingly ever-rising cost. An understanding of comparative effectiveness research as a core topic is important for neuroradiologists. It can be used in a variety of ways. Its goal is to look at alternative methods of interacting with a clinical condition, ideally, while improving delivery of care. While the Patient-Centered Outcome Research initiative is the most mature US-based foray into comparative effectiveness research, it has been used more robustly in decision-making in other countries for quite some time. The National Institute for Health and Clinical Excellence of the United Kingdom is a noteworthy example of comparative effectiveness research in action.

“Neo-modern” effectiveness research began with James Lind in Scotland in the mid-18th century. Graduates of the Edinburgh Medical School developed a concept of “arithmetical medicine.”^1,2 Dr Lind's historic controlled study of 6 different treatments for scurvy was, in reality, the first true “evidence” of comparative effectiveness research (CER).³ In the early 20th century, Ernest Codman started looking at “outcomes management” in patient care.⁴

The Institute of Medicine provided the current definition of CER as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.”⁵ The purpose of CER is to help all role groups associated with the health care enterprise make informed decisions useful in the management of an individual patient. In this article, we will explore the historical development of CER at the federal level in the United States up to and including the present day. We will reflect on some of the opportunities available through the American Society of Neuroradiology.

Meaningful geographic variation exists in health care. This, at least in part, relates to limited evidence and is one of the issues that augur well for CER differences in use corresponding to large discrepancies in Medicare spending per enrollee in various geographies.^6,7

Geographic variation occurs, in part, when definitive evidence for a treatment does not exist. For neurointerventional specialists, a ready example might be the application of the International Study of Unruptured Intracranial Aneurysms data or application of intra-arterial stroke therapy. This challenge is widely pervasive and not limited to neurointerventional or neuroradiology practice.⁸ It is rational to believe that patients in different locations have varying disease prevalence. It is also equally rational to believe that the variation known to exist and meticulously documented in treatment paradigms might reflect more than just different underlying diseases. Variability in practice can result in waste and inefficiency.^9–11

Selective History of CER with a Focus on Recent US Regulatory and Legislative Actions

“Arithmetical medicine” was first practiced in 18th century Edinburgh, Scotland.¹ As stated above, at the University of Edinburgh, James Lind performed a controlled trial for scurvy by using 6 separate treatments.³ Pierre Louis developed what could be loosely translated as the numeric method in 19th century France. This approach ended the use of phlebotomy in pneumonia.¹ In the US, in the early 1900s, Ernest Codman published A Study in Hospital Efficiency: As Demonstrated by the Case Report of the First Five Years of a Private Hospital.⁴ In that book, Codman described the field that would burgeon into outcomes management by reviewing 337 discharged patients between 1911 and 1916. Codman's work was also noteworthy for recording and publicizing 123 errors.

In 1972, the Office of Technology Assessment was created with a portion of its mandate focused on clinical effectiveness in health care.¹² The Office of Technology Assessment was eliminated in 1995.

In 1978, the National Center for Health Care Technology was created. As the name suggests, the Center was designed to conduct health care technology–related research that provided advice on research priorities. The Center closed 3 years later but, in that time, had made approximately 75 recommendations to the Centers for Medicare and Medicaid Services regarding coverage.⁶

The Agency for Health Care Policy and Research was established in 1989 and created the National Guideline Clearinghouse to guide treatment decisions.¹³ The Patient-Centered Outcomes Research Trust Fund (PCORTF) was reorganized as it is known today into the Agency for Healthcare Research and Quality.

In 2003, the Medicare Modernization Act authorized that as much as US $50 million be spent on CER.¹⁴ This funding (at lower levels) has continued with time, and the Agency for Healthcare Research and Quality established an “effective healthcare” program, which is charged with reviewing and synthesizing existing evidence.¹⁵

Veterans Affairs has long focused on the clinical effectiveness of treatments given to its patients. This has been aided by a forward-looking approach aimed at using electronic medical records. Veterans Affairs sponsors reviews through the Technology Assessment Program, which could also be properly characterized as CER.

The Centers for Medicare and Medicaid Services is critical in paying for health care but has had only a finite role with CER. Bearing that in mind, the Centers for Medicare and Medicaid Services has indeed taken small steps into CER.^6,16

The 2009 American Recovery and Reimbursement Act brought CER into the forefront on the basis of the size of the federal financial commitment (see below).^17,18 The CER was apportioned $1.1 billion, and it gave birth to the Federal Coordinating Council, whose job was to help coordinate federally funded CER activities.¹⁷ There are 15 members of the council.

On June 30, 2009, the council made recommendations for CER funding priorities. They also categorized activity, seeing where there were holes, and identified how these holes would inform their recommendations going forward. The council indicated that the expansion of CER will improve health care decision-making going forward.^18,19

On March 23, 2010, the Patient Protection and the Affordable Care Act (also known as the Affordable Care Act) became law.^20–22 Building on the work that had already occurred during the Obama administration, the Affordable Care Act had critical elements related to CER. The Patient-Centered Outcomes Research Institute (PCORI) is an integral part of the legislation, and its explicit tasks are to evaluate and use CER.^22,23

The Patient-Centered Outcomes Research Institute

The PCORI set about identifying research priorities and developing an agenda that will inform the various participants (health care providers, patients, administrators, payers, and so forth) about different health care choices. Much of the basis of the PCORI had been established by the previously discussed Federal Coordinating Council. PCORI is headed by a 21-member Board of Governors, which, by design, includes a broad section of role groups reflecting the diverse constituents of the PCORI. A 17-member Methodology Committee defines the methodologic standards for research. The Government Accountability Office appoints the Board of Governors and the members of the Methodology Committee. At the time of the founding legislation, it was thought that the PCORI, as an independent body, required funding outside the normal appropriations process. As such, the Patient-Centered Outcomes Research Trust Fund was created.

The PCORTF receives funds from 2 separate sources. In the aggregate, the funding is significant. The initial funding from the US Treasury during 2010–2012 provided $210 million. Beginning in 2013, the Trust Fund started receiving funds from its second source—fees assessed on health insurance plans, including Medicare. In 2013, the PCORTF received $150 million from the US Treasury and an annual fee assessed on different health care plans of approximately $170 million dollars for a total of $320 million. From 2014 through 2019, the PCORTF will continue to receive the same appropriation from the US Treasury and a larger assessment from the various health plans for an estimated $650 million per year. In total, the calculation is that $3.5 billion will be provided by these different funding sources before September 30, 2019. This represents the largest federally funded commitment to comparative effectiveness research in the history of our nation.²³

On March 5, 2012, the Board of Governors approved the following definition of patient-centered outcomes research: “Patient-centered outcomes research helps people and their caregivers communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options.”²³

PCORI also developed methodologic standards in the form of questions (Table 1). The answers to these questions are found in Table 2.

Table 1:

Examples of patient-centered questions addressed in PCORI research^a

Examples
“Given my personal characteristics, conditions and preferences, what should I expect will happen to me?”
“What are my options and what are the potential benefits and harms of those options?”
“What can I do to improve the outcomes that are most important to me?”
“How can clinicians and the care delivery systems they work in help me make the best decisions about my health and health care?”

^aSource: Patient-Centered Outcomes Research Institute (PCORI).²³

Table 2:

PCORI responses to patient-centered questions^a

Responses
Assess the benefits and harms of preventive, diagnostic, therapeutic, palliative, or health-delivery system interventions to inform decision-making, highlighting comparisons and outcomes that matter to people.
Be inclusive of an individual's preferences, autonomy, and needs, focusing on outcomes that people notice and care about such as survival, function, symptoms, and health-related quality of life.
Incorporate a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination.
Investigate (or may investigate) optimizing outcomes, while addressing the burden to individuals, the availability of services, technology, and personnel and other stakeholder perspectives.

^aSource: Patient-Centered Outcomes Research Institute (PCORI).²³

PCORI established the National Priorities for Research and Research Agenda as part of their basic charge. This framework will provide guidance to funding CER with federal funds.²³ The 5 research priorities are the following: 1) assessment of prevention, diagnosis, and treatment options; 2) improving health care systems; 3) communication and dissemination of research; 4) addressing disparities; and 5) accelerating patient-centered outcomes research and methodologic research.

As part of developing this National Priorities and Research Agenda, the PCORI held a 53-day public comment period on the draft plan in 2012.²⁴ The high level of public interest is reflected in the fact that the Institute received 474 comments for the Priorities and Agenda. The result was that 15 major, aggregated themes emerged from the public comments; these were in line with the 5 proposed priorities.

One can obtain funding from the PCORI using 2 fundamental approaches. The first is an investigator-initiated approach, which was launched in May 2012 and, in some ways, is typical of more traditional methodologies. The second approach relies on patients and other stakeholders to initiate. Five topics have been identified for accelerated consideration by using that second initiative.²³

While comparative effectiveness has been celebrated, issues continue to affect the research. A key issue underlying all major clinical research, whether randomized clinical trials or observational, is dissemination and implementation of findings. This issue may be the greatest challenge of CER, because even well-conducted and very prominent randomized controlled trials may not result in changes in physician behavior. Dissemination is included as one of the goals of the PCORI Methodology Committee.²⁵ Even then, given the strong tendency for inertia and much previous effort intended to promote dissemination of research findings and incorporation into practice that did not succeed, it is not clear whether or how this will ultimately translate into action.

Many national medical societies are addressing this growing interest in CER by developing educational materials, providing training courses, and creating funding support. The American Society of Neuroradiology has initiated a new funding mechanism specifically for CER in neuroimaging. Other efforts include developing CER workshops at its annual meeting and jointly sponsoring national CER training programs targeting junior investigators.

Summary

CER is designed to compare research of alternative approaches to prevent, diagnose, treat, and monitor clinical conditions and to support patient-centered care that will produce superior patient outcomes. Comparative effectiveness research has captured the attention of the biomedical community, including physicians, other health care professionals, and clinical researchers; the public, including patients and their advocates; and policy makers, including funding agencies and health care insurers.²⁶ CER should allow us to improve the evidence base and better inform decisions going forward. Ultimately, the authors hope that it will improve the quality of care and help control health care costs.²⁷

ABBREVIATIONS:

CER

comparative effectiveness research

PCORI

Patient-Centered Outcomes Research Institute

PCORTF

Patient-Centered Outcomes Research Trust Fund