Comparative effectiveness research and its utility in In-clinic practice

Abstract

One of the important components of patient-centered healthcare is comparative effectiveness research (CER), which aims at generating evidence from the real-life setting. The primary purpose of CER is to provide comparative information to the healthcare providers, patients, and policy makers about the standard of care available. This involves research on clinical questions unanswered by the explanatory trials during the regulatory approval process. Main methods of CER involve randomized controlled trials and observational methods. The limitations of these two methods have been overcome with the help of new statistical methods. After the evidence generation, it is equally important to communicate the results to all the interested organizations. CER is beginning to have its impact in the clinical practice as its results become part of the clinical practice guidelines. CER will have far-reaching scientific and financial impact. CER will make both the treating physician and the patient equally responsible for the treatment offered.

INTRODUCTION

The unsustainable growth in healthcare costs and lack of high-quality evidence from real world practice has initiated alternative methods of research in healthcare. In the last few years, the use and applicability of clinical research have been questioned world over. New concepts like comparative effectiveness research (CER) are being introduced into healthcare policies to improvise clinical research.[1] Although, CER is being perceived as a new research model in healthcare, it is definitely a revisited concept and not a new idea. Earlier scientists used CER models for research without the tag of “comparative effectiveness research.” This model was actually reborn officially with the American Recovery and Reinvestment Act of 2009 when huge funds were allocated for progress in patient-centered healthcare.[2] Since the inclusion of CER into healthcare policies, it has drawn considerable attention as an upcoming, potential approach for improving healthcare in terms of better delivery, decreased costs, and relevant high-quality research.[3] Although the terminology was officially introduced in the USA in 2009, the road map of CER is still being finalized.[4]

DEFINITION

While introducing the concept, the Institute of Medicine, USA, defined CER as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policymakers to make informed decisions that will improve health care at both the individual and population levels.”[1]

CER aims to generate new scientific evidence, provide answers to questions left unanswered during the regulatory approval process, reduce clinical uncertainty, disseminate the evidence, and guide health care choices.[5] CER involves comparison of two or more agents or interventions, which are considered true therapeutic alternatives in actual clinical practice.[6] Importantly, CER is not only limited to comparison of two pharmacological interventions in real life setting, but, it is also applicable for comparison of medical devices, procedures, health services, or any other competing intervention.[6] CER ultimately intends to involve both the healthcare provider and the patient in the decision making of the treatment.[7]

WHAT LED TO THE RESURFACING OF THE CONCEPT OF COMPARATIVE EFFECTIVENESS RESEARCH?

The limitations of the current research process led to the evolution of CER. First and foremost, it is difficult to implement the results of clinical trials conducted for regulatory processes into the real world settings. These are often termed as explanatory trials, which imply that these are conducted to explore if and how an intervention works.[8] Randomized controlled trials (RCTs) are generally of explanatory type.[9] The advantages of RCTs, like randomization and blinding have helped in pushing the status of RCTs to the top of the ladder of clinical research. RCTs are conducted with stringent inclusion and exclusion criteria at suitable clinical centers. The RCTs are set up like a laboratory experiment with all steps clearly specified in the protocol before the start of the experiment. As RCTs are mainly conducted for regulatory requirements, optimum treatment regimens are used for selected patients. In spite of such characteristics, it is difficult to use the results of RCTs for an average patient seen in the real world setting.[7] Results of such trials cannot be used for patients of the real-world with individual patient variations and preferences.[9] The real-world patients are different from those described in textbooks and those being studied in protected study environments. Patients are often carefully selected, excluding patients who are sicker or older and those who have trouble adhering to treatment plans. Racial and ethnic minorities are generally under-represented. Usually, those regimes and follow-up protocols are used that maximize the benefits and limit the harms of treatments being evaluated. There is a need of pragmatic trials, i.e. those trials where the question is whether the intervention actually works in real life.[8] The explanatory trials usually compare the treatment with placebo and in case the comparison is with another intervention, it is generally of noninferiority or equivalence design.

Next, the high cost and inefficiencies of the clinical trials have increased the economic burden on the healthcare system. Furthermore, the healthcare provider faces the challenge of choosing the best treatment from the available standard and alternative regimes or devices. Another issue is the noninvolvement of the patient in decision making regarding treatment modality or in choosing the best regime. The comparative analysis of the available treatments is known neither to the physician nor to the patients. All these points have collectively led to the inappropriate healthcare system and thus a need for a better system.[2,10,11]

STEPS OF COMPARATIVE EFFECTIVENESS RESEARCH

As CER is an extensive field, the US healthcare guidelines have proposed seven steps for the proper conduct of CER [Table 1].[5] Comprehensively, the first important step is the identification of a clinical problem and the outcome to be addressed, e.g., will lowering serum low-density lipoprotein-cholesterol (LDL-C) levels with statins among patients with low risk of vascular events lead to any benefit? After identifying the issue, it is important to review the current evidence and then understand the inadequacies according to the real life scenarios. Then, generate new evidence with different methods and train new researchers. Identification and generation are followed by dissemination of evidence so as to reach out to all stakeholders of healthcare.

Table 1.

Steps of CER

Although all steps have their significance, the fourth step becomes critical for the conduct of research [Table 1]. This can also be subdivided as evidence generation and/or evidence synthesis.[12] Evidence generation includes experimental methods and observational methods. Evidence synthesis includes systematic reviews, meta-analysis and use of different statistical models for the synthesis of evidence.[12]

METHODS OF COMPARATIVE EFFECTIVENESS RESEARCH

CER will yield clinical relevant results only if statistically significant and high-quality valid methods are applied. The methods to conduct CER are evolving, and improvements are being suggested to the already known methods that is, RCTs and observational studies. CER study design should be feasible, have high validity and should be appropriate for the research question. The conventional efficacy studies for regulatory approvals are different from the CER studies set-up in the real world.

Evidence generation

Experimental methods

Randomized controlled trial

RCT, the gold standard design of research is the benchmark for comparison of all other study designs. RCT design is an integral part of drug regulatory process, but it can be a part of the CER as pragmatic trials with modifications in conduct and analysis. The introduction of Bayesian and analytical adaptive methods can help in overcoming some limitations of RCTs i.e. time taken, sample size, and high cost. The RCTs can also be designed with broader inclusion criteria, and patient-centered outcomes and then patients may be passively followed through routine care with the help of hospital registries.[9,13] RCTs may be head to head trials comparing two standard treatment regimens as part of CER.[14] RCTs are the best source for evidence synthesis in CER.

Observational studies

This trial design is used in routine clinical practice settings so as to have maximum applicability of the results. The advantageous points over the explanatory trial design are the involvement of diverse population, large sample size, real world setting, high external validity, simple design and low cost. Interestingly, the patient population is heterogeneous in observational studies, whereas, homogeneity is integral for explanatory studies.[9,12,15] The study environment is not tightly controlled and participants are followed in the normal routine set-up e.g. an explanatory study of lipid-lowering agent may be providing the patients with calculated and specific dietary intake whereas in an observational study, the patients may be advised an estimate of total calorie intake in a day. Observational studies can evaluate adverse effects associated with short-term and long-term use of treatments. Furthermore, observational studies can compare multiple regimes simultaneously unlike explanatory studies with a limited number of treatment arms.[9] Observational methods are becoming more popular because of easy access to clinical registries, electronic health records, and administrative databases. However, the observational studies may not generate high-quality evidence because of confounders such as performance, selection and detection biases, incompleteness of information and undefined start time (or time-zero) of the intervention. Performance bias occurs when one intervention is associated with better health care facilities in the set-up. Selection bias may occur at the time of enrollment, if patients are divided into treatment arms according to the severity of the disease or prevalent users are compared with new users. Detection bias may occur when the outcomes are more easily detectable in one group as compared to the other.[12]

Despite the above mentioned limitations of the nonrandomized observational studies, these imperfect studies can help in generating scientific evidence if these are well-designed, implemented and then analyzed and reported carefully.[16] Selection bias and the “time-zero” aspect can be tackled by excluding patients who have already been on the treatment being evaluated. Prevalent users should not be compared with new users. This trial design is often referred as “New-user” design for observational studies.[12,15] This excludes the chance of using only those patients who have tolerated the drug out of many who had adverse effects or did not respond to the treatment. Some statistical models can be used to control the confounders. Propensity score analysis involves balancing the factors influencing the treatment choice.[15] The instrumental variable method is another analytical method in observational studies. A characteristic (instrument) is chosen, which is associated with treatment allocation, but not the outcome of interest. The instrument can be a geographical area, distance to a healthcare facility, characteristics of healthcare facility etc. Oncology studies generally choose area-level treatment as an instrument.[12] Multivariable regression analysis, a common statistical method for Oncology clinical trials, also helps in evaluating the outcome of interest while keeping the variable constant. Furthermore, with the incorporation of disease registries, information will be more complete and accurate in future. Use of many more statistical models can help in using the observational study design in research. Although pragmatic trials are integral for CER, these cannot replace the RCTs for drug discovery. Studies like comparison of cancer incidence among patients taking two different types of insulin preparations; incidence of kidney injury among patients taking statins and comparison of multivessel coronary bypass surgery and multivessel percutaneous coronary intervention are examples of CER studies.[17,18,19]

Evidence synthesis

Systematic reviews

It is a qualitative and quantitative analysis of data and involves answering clinically relevant questions, which have been left unanswered during the regulatory approval process for drugs or devices. These questions might have become important because of the lack of information regarding a specific query or because of variable and contradictory results. With reference to the earlier question of lowering LDL-C levels among patients with low risk of vascular event; result of a meta-analysis shows that a decrease of 1 mmol/L of LDL-C level among individuals with low risk of major vascular event is associated with an absolute reduction in major vascular events of about 11/1000 over 5 years.[20]

Dissemination of information of comparative effectiveness research

After the generation and synthesis of evidence, next crucial step is the dissemination of information. The collective well informed decisions can only be taken with easy access to the correct evidence. The Agency for Healthcare Research and Quality which is the nodal organization for CER in the United States, is still planning the ways for the dissemination of CER information.[5] It is proposed that the important messages should be communicated to the clinicians, consumers, caregivers, and policymakers. Furthermore, a publicly available database should be created. This process is complex as both the clinicians’ and patients access different sources for information. The clinicians often look to specialty societies or other guideline writing groups (e.g. Joint Commission Report for management of hypertension; American Diabetes Association guidelines for the management of diabetes mellitus) for help in interpreting evidence. Some dissemination techniques like providing links to health information topics from the hospital websites, availability of consumer guides with clear information in nonscientific language at all healthcare desks in the hospitals can easily be incorporated. The healthcare providers should be well-informed about the recent advancements. Proper dissemination of results can have immediate effects even, e.g., after the publication of Women's Health Initiative Trial showing the adverse cardiovascular effects of hormone replacement therapy (HRT), there was a significant decrease in use of HRT among postmenopausal women.[21,22]

In India, dissemination of information to the patients is a big challenge due to illiteracy and the developing health care system. Rural population can be given access to important information with the help of Anganwadi workers, involvement of Nongovernmental Organizations (NGOs) and spread of information through media, especially radio and television. Furthermore, the consumer guides can be in the regional language. Although, Indian patients are less in involved in decisions regarding treatment as compared to western world, yet, things can change if the patients are fully aware and have the capability to be a part of the decision taking.

CHALLENGES AND LIMITATIONS OF COMPARATIVE EFFECTIVENESS RESEARCH

Consistent with other approaches, even CER also has its limitations. In this method of research, how do we define a “real life setting”; can the clinical setting of Europe be similar to the small hospital setting of India?[8] No, so the evidence of one region or country may not be applicable to another area because of a diverse population and healthcare systems. Second, the clinical practitioners are not trained enough to become investigators for clinical studies. They need to be trained for conducting healthcare research. Currently, most of the researchers are from tertiary institutes, and it may be difficult for them to conduct research in community settings. Third, CER implies increasing the administrative burden of busy clinicians and financial burden of the society.[23] Regular recruitment of patients for large sample size studies can be another area of concern. Furthermore, the patient follow-up may be longer while comparing the safety and efficacy for chronic diseases, which is an important are of CER. New statistical analysis methods have to be incorporated to target the limitations of both RCTs and observational studies.

IMPACT OF COMPARATIVE EFFECTIVENESS RESEARCH OF REAL WORLD CLINICAL PRACTICE

If CER is older than thought, has it shown an impact on the healthcare system or not? As the concept of CER is still evolving, there is limited information about the impact of CER results on clinical practice. The impact of CER will be visible when it will become an integral part of medical research and primary care physicians and the patients get more familiar with CER.[24]

An analysis of the impact of CER on clinical practice depicts an inconclusive effect.[25] As mentioned earlier, there was an immediate response to the publication of the results of the Estrogen Plus Progestin trial (a component of Women's Health Initiative).[21] Similarly, the results of ALLHAT/JNC7 had an impact on the prescribing of thiazide diuretics among hypertensive patients.[25,26] Gibson et al. evaluated the real-world impact of some CER studies. The results of Pravastatin or Atorvastatin Evaluation and Infection Therapy–Thrombolysis in Myocardial Infarction-22, study were published in the year 2004, with a significant finding that intensive lipid lowering therapy provides greater protection from death and major cardiovascular events as compared to the standard therapy.[27] Parallel to the publication of the results, the use of intensive therapy increased, but it did not show any postpublication surge for the coming years. The shift from standard therapy to intensive therapy occurred after three years.[25] The results of another study i.e., the Spine Patient Outcomes Research Trial were incorporated in the clinical practice guidelines for management of low back pain, but the management practice did not change in the coming years.[25,28] Similarly, the mammography with magnetic resonance imaging (MRI) study showed that MRI had the highest sensitivity to detect breast cancer as compared to mammography, ultrasound, and clinical breast examination, but this didn’t increase the utilization of MRI for cancer detection.[25,29] The association between the results of CER and the change in practice is not a singular relation, it is multifactorial like the timing of the release of clinical practice guidelines, dissemination of these results, access to latest guidelines and availability of the substitute therapy, etc., The future of medical practice will definitely be impacted by CER.[30,31]

DISCUSSION

CER or standards of care research has become very meaningful in the current scenario. It is an emerging area of research relevant to many areas of health care, especially pharmacotherapy. The current gap in clinical research and practice can be filled by generating, synthesizing, and disseminating accurate evidence. CER can help in identifying the treatments that may work best in the clinical settings and in special subgroups of patients.

Till now, CER has not become an integral part of our healthcare system. The Indian drug regulatory authorities and the government need to include CER in our system. CER can help in generating highly relevant area-specific evidence from Indian population. CER can also help in a continuous pharmacoeconomic analysis of treatment regimens. Treatments with good efficacy and at an affordable cost are essential for countries like ours. Well informed patients and healthcare professionals with the best knowledge can help in re-building the lost trust between the doctor and patient. CER will help the healthcare providers and patients to take responsibility for their decisions in the typical clinical setting and the policy makers to make recommendations and guidelines from the real-life evidence.

However, comparative research has its own challenges and limitations which need to be addressed. The concerns of insufficient infrastructure, lack of trained clinical researchers need attention. The crucial issue of risk assessment and consent forms are confusing for the research community. As the research will be conducted in the hospital setting, care needs to be taken to avoid reaching harmful or useless conclusions. Over the coming years, CER should be implemented in letter and spirit to improve the healthcare delivery. The limitations and the challenges should not deter both the researcher and the patient from clinically relevant research. CER will make both the healthcare provider and the patient more responsible and accountable.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.