Pragmatic Clinical Trials in CKD: Opportunities and Challenges

Abstract

Randomized controlled trials in CKD lag in number behind those of other chronic diseases, despite the high morbidity and mortality experienced by patients with kidney disease and the exorbitant costs of their health care. Observational studies of CKD frequently yield seemingly paradoxic associations of traditional risk factors with outcomes, making it difficult to extrapolate the results of trials conducted in people with normal kidney function to patients with CKD. However, many completed trials in CKD have been limited by intermediate outcomes of unclear clinical significance or narrow eligibility criteria that limit external validity, and implementation of proven therapies remains a challenge. It is therefore imperative that the nephrology community capitalize on recent interest in novel approaches to trial design, such as pragmatic clinical trials. These trials are meant to promote research within real world settings to yield clinically relevant results with greater applicability than those of traditional trials, while maintaining many advantages, such as controlling for potential sources of bias. We provide a description of pragmatic clinical trials and a discussion of advantages, disadvantages, and practical challenges inherent to this study design, in the context of specific scientific questions relevant to patients with CKD.

The Need for Clinical Trials in CKD

In the second half of the past century, significant progress has been made in understanding principles of causal inference and in statistical techniques underlying the proper planning and analysis of interventional trials.¹ As a result, the randomized controlled trial (RCT) has become the gold standard tool for proving the efficacy and safety of therapeutic interventions.^2,3 After their ubiquitous acceptance as the sine qua non of causal assessment, RCTs evolved from small single-center studies to large, multicenter trials. With this shift, the cost of conducting RCTs increased dramatically, common disease states amenable to drug intervention (such as hypercholesterolemia and hypertension) became a dominant focus of inquiry,⁴ and commercial entities took on enlarging roles in trial design and conduct.^5,6 At the same time, gaps emerged between demonstration of treatment efficacy and implementation of proven therapies in clinical care, and high-quality RCT evidence has remained scarce for many important and practical clinical questions.

Upon this general background, conducting investigations in CKD has faced additional challenges. The number of RCTs conducted in patients with CKD or ESRD is one of the smallest compared with other medical subspecialties,⁷ in spite of the rising prevalence of CKD, the enormously high morbidity and mortality experienced by patients with kidney disease, and the high costs of CKD and ESRD care.^8,9 Published nephrology RCTs have often examined intermediate outcomes that are of unclear significance to patients, providers, and families, with important exceptions. The complex and heterogeneous nature of CKD has often led to restrictive enrollment criteria that limit external validity.⁷ Insufficient implementation of the few proven therapies has contributed to increased risk of CKD and its complications in economically and socially disadvantaged populations.^10,11 Moreover, observational studies of people with CKD and ESRD have frequently observed seemingly paradoxic associations of traditional risk factors with clinical outcomes,¹² making it difficult to extrapolate the results of RCTs conducted in patients with normal kidney function to patients with CKD.

It is therefore imperative that the nephrology community capitalize on the recent wave of interest in novel approaches to clinical trial design, such as pragmatic clinical trials (PCTs). This review article will provide a description of the concept of PCTs, a discussion of the advantages, disadvantages, and practical challenges of PCTs, and examples of PCTs conducted among patients with CKD.

What Are PCTs?

PCTs have been recognized as important tools for evaluating medical interventions since at least 1967.¹³ Over time, the term “pragmatic” has been used to refer to a variety of interrelated elements of trial design.^14–16 Central to all definitions is an emphasis on external validity. PCTs are designed so that their results can be quickly and directly applied to relevant clinical populations. Toward this goal, common features of PCTs include broad eligibility criteria, comparisons of clinically relevant alternatives, integration of research into clinical practice, and evaluation of a broad range of relevant health outcomes.^17,18 The Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico trial of thrombolysis for acute myocardial infarction is often cited as an example of an early and successful PCT.¹⁹ PCTs have been contrasted with explanatory trials, which strive to determine whether an intervention has beneficial effects in humans, often using conditions that maximize the likelihood of demonstrating a treatment effect while narrowing generalizability.^13–16 Some authors have compared PCTs to effectiveness evaluation and explanatory trials to efficacy evaluation.^14,15

In reality, any single trial is neither fully pragmatic nor fully explanatory.¹⁵ The PRagmatic Explanatory Continuum Indicator Summary-2 tool identifies nine domains to quantitate the extent to which a clinical trial is pragmatic.²⁰ The most pragmatic trials apply broad eligibility criteria, recruit from clinical settings, conduct procedures in the context of usual care using available clinical infrastructure, apply interventions with flexible protocols using usual encouragement, assess outcomes relevant to patients using readily obtainable metrics, and apply intention-to-treat analysis (Figure 1). Rarely is a clinical trial pragmatic in all aspects of its design. Often, some design elements are made less pragmatic in order to maintain internal validity or to accommodate practical needs for implementation. Ultimately, the choice of design for any clinical trial depends on the underlying scientific question and the context in which it is addressed.

Figure 1.

Domains of the explanatory-pragmatic clinical trial design spectrum.

PCTs are suited for the evaluation of a wide variety of medical interventions including individual medical treatments and strategies to deliver those treatments to patient populations. Depending on the intervention and setting, PCTs may employ cluster randomization, active comparator groups, and quasi-experimental designs. The electronic health record (EHR) plays a central role in most PCTs. EHR data can efficiently facilitate every step of a clinical trial, from screening to enrollment, collection of baseline data, intervention implementation, safety monitoring, and outcome ascertainment.^18,21

Interest in PCTs has grown quickly over the last decade,¹⁵ and the nephrology community has been an early leader in this surge, as demonstrated by the examples below. PCTs that involve populations with kidney disease have been facilitated and encouraged through specific funding mechanisms from sponsors critical to nephrology research, including the National Institutes of Health, Department of Veterans Affairs, and the Patient Centered Outcomes Research Institute. Of note, interest in PCTs has grown along with interest in another movement that may appear conflicting – the Precision Medicine Initiative.²² Indeed, whereas PCTs focus on applicability to populations, precision medicine focuses on applicability to individuals. However, there is no reason that trials advancing precision medicine cannot incorporate pragmatic design elements, and PCTs can inform precision medicine by building in personalized components (e.g., intervention flexibility) or the evaluation of between-participant heterogeneity in response.

Potential Benefits of PCT Designs in CKD

There are several potential benefits to PCTs in CKD (Table 1). First, PCTs may include a broad range of patients with CKD, enhancing external validity. Patients with CKD have diverse underlying and comorbid conditions and often have complicated medical regimens that may be more acceptable for PCTs than traditional RCTs with strict eligibility criteria. In addition, PCTs may include participants from socially disadvantaged populations that bear a disproportionate burden of disease and experience large translational gaps between research and practice,^10,11,23 who may be excluded from efficacy trials owing to a lack of reliable transportation or limited English language proficiency or may choose not to enroll in efficacy studies because of mistrust in the research process.^24,25 Socially disadvantaged patients may be more willing to participate in the context of their usual site of health care²⁶; recruiting and enrollment data from early PCTs will help address this possibility.

Table 1.

Selected advantages and challenges for PCTs in nephrology

Advantages	Challenges
Enhanced external validity	Informed consent and regulatory oversight
Ability to enroll socially disadvantaged populations	Achieving large separation between comparison groups
Directly applicable to patient care	Selection and ascertainment of outcomes
Relevant patient-centered outcomes	Funding
Reduced cost	Lack of experience and training

Second, findings from PCTs can often be directly applied to the practice environment(s) in which the study was conducted. This helps translate research results to clinical care, lowering well known barriers to the implementation of existing practice guidelines for care of patients with CKD.^27,28 The majority of CKD patients are cared for by primary care providers with or without collaboration with nephrologists,^8,29 and PCTs may be well positioned to affect CKD care in the primary care setting. Examples of studies bridging nephrology and primary care include trials of CKD care coordination with patient navigators³⁰ and strategies to implement guideline-concordant CKD care with electronic checklists,³¹ primary care practice facilitation,³² and pharmacist-led interventions.³³

Third, pragmatic trials offer an opportunity to examine clinical questions that have been identified by diverse stakeholders, including patients, caregivers, and providers, as high-priority CKD research areas (Box 1).^34,35 Outcomes of interest to these stakeholders include traditional clinical events such as hospitalizations, emergency room visits, and mortality as well as patient-centered outcomes of growing interest, such as symptoms, treatment preferences, quality of life, and satisfaction with care. Use of the EHR can effectively ascertain clinical events, although further planning is required to collect patient-reported outcome data, which are not currently available in most EHRs.²¹

Fourth, pragmatic trials for CKD may be less costly than efficacy trials. This is particularly important when considering the length of follow-up necessary to observe some clinical outcomes of CKD, such as progression to ESRD.

Examples of Research Questions Identified by Diverse Stakeholders as High-Priority CKD Research Areas

 What is the optimal screening frequency in populations at high-risk of developing CKD?

How effective are lifestyle programs for preventing deteriorating kidney function in early CKD?

What is the effectiveness of strategies to improve provider awareness and adherence to guidelines on improving outcomes for patients with CKD?

What is the effectiveness of patient-safety interventions in reducing complications among CKD patients?

What is the effectiveness of computer decision support for CKD management in reducing complications?

What strategies will improve family consent for deceased donor kidney donation, taking different cultural groups into account?

Barriers and Obstacles for PCTs in CKD

Informed Consent

Informed consent is a cornerstone of ethical clinical research, but PCTs pose unique challenges that have led to debate regarding optimal approaches and alternatives to the traditional written informed consent process.^36,37 Written, in person, informed consent may alter the nature of the intervention, introduce prohibitive logistic barriers, and lead to selection bias. These risks are particularly apparent for PCTs that employ cluster randomization, i.e., assigning groups of participants rather than individuals to treatment arms. With the increasing use of EHRs, an electronic consenting process can be considered to reduce costs.³⁸ However, such a process may introduce selection bias, as target populations would necessarily be literate and have access to information technology, and recent national surveys have suggested a general preference for an interactive written consent process rather than alternatives, such as informing patients that a clinical site is participating in a PCT.^39,40

Regulatory Issues

Institutional review boards (IRBs) serve to protect the safety of study subjects, and each has different rules and regulations to account for local conditions of risk. Multicenter PCTs require harmonization of these local regulations to ensure participant safety and data integrity. This process may be facilitated through use of central IRBs with shared review,⁴¹ but confusion regarding divisions of responsibility for collaborating central and local IRBs remains a barrier for widespread adoption.¹⁸ Moreover, the concepts and design elements of PCTs are new to many IRBs and data and safety monitoring boards, who may expect, request, or even demand traditional clinical trial design elements that reduce pragmatic elements of PCTs. Contributing to this tension, it may be difficult to define appropriate standards of care for patients with CKD because many guidelines may not offer clear consensus or be based on limited evidence.⁴² PCTs seeking to challenge traditional care practices may have difficulty defining an appropriate control group or gaining IRB or data and safety monitoring board approval for a design that involves more flexible protocols or less extensive research-specific monitoring than traditional explanatory trials.

Outcome Ascertainment

Ideally, PCTs identify outcomes using available clinical data sources, e.g., EHRs, to enhance relevance and promote efficiency. Challenges for use of such data include lack of specificity for coded clinical events, potentially leading to misclassification of study outcomes, and missing or biased outcomes data. For example, laboratory outcomes, such as changes in slope of eGFR, serum phosphorus, or hemoglobin can be strongly influenced by loss to follow-up, and clinical events can be difficult to ascertain if participants change health care systems. Outcomes that are ascertained from national databases, such as initiation of dialysis or transplantation, may not be affected to the same degree. Patient-reported outcomes, such as health-related quality of life measures, medication adherence, and medication side effects, are difficult to track pragmatically, as they are not often ascertained as part of routine clinical care.²¹

Implementation Barriers

As with traditional RCTs, sufficient separation in treatment between comparison groups is needed to detect differences in clinical outcomes.⁴³ For PCTs, large separations between groups may be difficult to achieve in a clinical setting with large commonalities in practice, particularly if study participants have little contact with dedicated research staff. PCTs may also increase burden to practicing providers, requiring effective strategies to increase provider engagement and participation without compromising the integrity and “pragmatism” of the study. In addition, PCTs in nephrology are likely to require partnerships with large, integrated health systems, and it may be difficult to harmonize protocols while maintaining local care priorities when there are substantial differences in workflow, EHR, personnel, and clinic culture. Nephrology experience with PCTs is nascent, so many implementation barriers will be new to investigative teams.

Novel Therapeutics

Novel therapeutic approaches are sorely needed to reduce the incidence and progression of CKD and its complications. The initial evaluation of new drugs is often best accomplished using more traditional RCTs that feature explanatory design elements. For example, extensive eligibility criteria may be used to limit participation to a subset of patients more likely to benefit and less likely to experience adverse events than the broader target population, and extra efforts may be used to enhance adherence and maintain separation of treatment groups. PCTs may be more suitable for subsequent studies in broader populations, for drugs that have known efficacy and are relatively safe, and for studies that evaluate implementation of existing therapies or health care delivery.

Examples of Ongoing Pragmatic Trials in CKD

Screening for CKD

The effectiveness of screening for CKD is currently being evaluated in a high risk primary care population of United States veterans (Table 2). Using cluster-randomization by primary care team, this pilot PCT aims to evaluate two strategies to improve BP control among nondiabetic hypertensive veterans with unrecognized CKD in primary care (NCT02059408). In the usual care arm, no systematic screening for CKD is recommended. The first intervention evaluates the efficacy of a screen-and-educate strategy that utilizes a “triple-marker” CKD screening approach (creatinine, cystatin C, and albuminuria⁴⁴) coupled with provider education, compared with usual care. This strategy takes advantage of the VA EHR to identify eligible persons, order screening tests, and deliver recommendations to primary care providers. The second intervention evaluates whether a screen, educate, and intensify treatment program coled by a primary care provider and a clinical pharmacist can improve BP management among persons with CKD, compared with the screen-and-educate provider strategy and with usual care. The primary outcome of the study is change in BP and secondary outcomes include processes of care and clinician burden. This pragmatic study design can be easily translated to any primary care practice with an EHR that utilizes the chronic care model.

Table 2.

Selected examples of ongoing PCTs in CKD

Study	Intervention	Outcomes(s)	Pragmatic Aspects	Barriers and Obstacles
Screening for CKD	Usual care versus screen-educate versus screen-educate-intensify BP management by clinical pharmacist	BP levels, change in BP: proportion with BP controlled	VA clinical practice	Informed consent
		Processes of care, appropriate use of ACE/ARB	Wide inclusion criteria	Ensuring screening after lab orders placed
			Utilization of EHR for patient identification, delivery of interventions and outcome ascertainment	Database design for study management
			Use of ancillary health personnel embedded in primary care
KARE	Provider intervention: CKD registry versus usual care	Change in systolic BP; proportion of patients with BP control	Set in “real world” primary care practices	Accurate outcome ascertainment
	Patient intervention: language concordant self-management program versus usual care		Utilization of EHR for outcome ascertainment
			Recruitment of study participants that are not well represented in RCTs.
ICD-Pieces	CKD care enhanced by information technology, clinical support and practice facilitator versus usual care	Hospitalizations Cardiovascular events, readmissions, ER visits, deaths	Broad eligibility criteria	Data transmission from multiple sites
			Delivery of the intervention in clinical setting by primary care providers	Implementation of various interventions
			Electronic ascertainment of outcomes	Changes in standard clinical practices
			Intention-to treat analysis based on assigned cluster
ECHO-CKD	Calcitriol plus usual care versus usual care alone	Composite of clinical cardiovascular events	Integration into clinical health care system	Efficient informed consent
			Minimal eligibility criteria	Recruiting
			Utilization of EHR for recruiting and outcome ascertainment	Retention and adherence
			Clinical events outcome	Accurate outcome ascertainment

ACE/ARB, angiotensin converting enzyme inhibitor/angiotensin 2 receptor blocker; VA, Veterans Affairs; ER, emergency room.

The Kidney Awareness Registry and Education Study

The Kidney Awareness Registry and Education (KARE) study (NCT01530958) is a two by two factorial trial that assesses the effectiveness of a multilevel intervention to improve BP control among low-income patients with CKD (Table 2). The study design relies on two levels of randomization: primary care practice teams are randomized to one of two arms (access to an electronic CKD registry or usual care); patients of each practice team are subsequently randomized to participate in a CKD self-management program or usual care. With four study arms, KARE allows assessment of the individual and additive effect of both interventions. Although the KARE study design is complex, its pragmatic elements include: setting and participants, intervention components (electronic CKD registry embedded in the EHR and a patient self-management support program that relies on language-concordant health coaches), outcome ascertainment (change in systolic BP and proportion of patients with BP control), and relevance to practice. By relying on existing infrastructure and recruiting actively empaneled patient participants, KARE results will be directly applicable to populations that receive CKD care in similar health systems.

Improving Chronic Disease Management with Pieces

Improving Chronic Disease Management with Pieces (ICD-Pieces) is a large PCT spanning several large health care systems that will compare EHR-based clinical-decision support to primary care providers and practice facilitators with usual care among patients with coexistent CKD, diabetes, and hypertension (Table 2).⁴⁵ The study uses a novel technology platform, Pieces, which has been successfully used in other chronic conditions, such as heart failure, to reduce hospital readmissions and improve clinical outcomes,^46,47 in a cluster-randomized design. Study outcomes include hospitalizations, readmissions, cardiovascular events, ER visits, and death. Other pragmatic characteristics of ICD-Pieces are broad inclusion criteria, flexibility of protocols, implementation of the interventions within the context of available clinical infrastructures, robust clinical support tools, electronic ascertainment of outcomes, and intention-to-treat analysis based on assigned cluster. The goal of this PCT is to evaluate a promising new model for the care of CKD patients with multimorbidity.

Evaluating Calcitriol Heart Outcomes in CKD

Evaluating Calcitriol Heart Outcomes in CKD (ECHO-CKD) is a PCT, currently in a pilot stage, designed to test the cardiovascular effects of calcitriol, a drug commonly used in CKD for which long-term clinical risks and benefits remain uncertain (Table 2).^48,49 The overall goal of the ECHO-CKD trial is to determine whether calcitriol plus usual care, compared with usual care alone, reduces the risk of clinical cardiovascular events among patients with stage 3–4 CKD. Although this question could be addressed using a more traditional explanatory trial, a pragmatic approach facilitates application of results and reduces cost. Pragmatic features of the planned study include integration into a large comprehensive health care system, utilization of electronic resources to identify participants, the application of minimal eligibility criteria, delivery and monitoring of the study intervention using clinical pharmacy and laboratory services, and the ascertainment of clinical outcomes of importance to patients and providers.

Other PCTs in Nephrology

PCTs may also be useful to improve care in dialysis, glomerular disease, kidney transplantation, hypertension, and other fields within nephrology. As an example, the TiME trial (NCT02019225) evaluates the effects of a minimum hemodialysis session duration of 4.25 hours compared with usual care on mortality, hospitalizations, and health-related quality of life among patients with ESRD initiating treatment with thrice weekly maintenance hemodialysis. This PCT uses a cluster randomization design, with the allocation of outpatient dialysis units to the two intervention arms. Another novel aspect of the TiME trial is that it is conducted through a partnership between academic investigators and two large dialysis provider organizations in approximately 320 dialysis facilities, thereby combining traditional funding mechanisms with existing industry resources and expertise. The TiME trial is planning to enroll approximately 6432 patients, and it is estimated to be completed in September 2017. In glomerular disease, NephCure has brought together patients, families, and researchers to support patient-oriented research well suited to PCTs. In kidney transplantation, many interventions are made using protocols starting from a clear time zero, and alternative protocols may be effectively compared using pragmatic trial designs. Hypertension is a highly prevalent condition with many available treatments for which BP targets, specific antihypertensive agents, and antihypertensive combinations or approaches could be compared using pragmatic designs. The Living Textbook of PCTs, sponsored by the National Institutes of Health Collaboratory, further describes ongoing PCTs within and beyond nephrology.¹⁸

Next Steps for PCTs in CKD

PCTs offer a valuable approach to increase the evidence base for the treatment of patients with CKD with high external validity and applicability and low costs. However, informed consent, regulatory issues, outcome ascertainment, and other implementation barriers provide challenges to designing and implementing PCTs. If barriers to PCTs can be surmounted and benefits realized, then further training, infrastructure, and funding will be needed to facilitate additional studies. Few investigators in nephrology have substantial experience in PCTs, and training will be necessary to advance and implement high-quality PCT methods. Because PCTs are critically dependent on the health care setting in which they are based, investment in EHRs and partnerships of researchers with health care systems will be required to expand bandwidth. For example, within the Veterans Health Administration, a national coordinating center has been developed to help design and launch PCTs. Currently, the Patient-Centered Outcomes Research Institute and several institutes within the National Institutes of Health have offered funding mechanisms specific to PCTs; additional funding mechanisms appropriate for this study design will be required. Ongoing studies, including those described above, will provide valuable information on the feasibility, strengths, limitations, and hurdles of PCTs in CKD, as well as additional next steps required to move forward this promising approach to clinical trials.

Disclosures

I.D.B. has received consultancy fees from Amgen, Bayer (Whippany, NJ), Boehringer-Ingelheim (Ridgefield, CT), and Janssen (Titusville, NJ). S.N. has served on the event adjudication committee of clinical trials sponsored by AbbVie (North Chicago, IL), Bayer, and Boehringer-Ingelheim.