Pragmatic Trials in Maintenance Dialysis: Perspectives from the Kidney Health Initiative

Abstract

Pragmatic clinical trials are conducted under the real-world conditions of clinical care delivery. As a result, these trials yield findings that are highly generalizable to the nonresearch setting, identify interventions that are readily translatable into clinical practice, and cost less than trials that require extensive research infrastructures. Maintenance dialysis is a setting especially well suited for pragmatic trials because of inherently frequent and predictable patient encounters, highly granular and uniform data collection, use of electronic data systems, and delivery of care by a small number of provider organizations to approximately 90% of patients nationally. Recognizing the potential for pragmatic trials to generate much needed evidence to guide the care of patients receiving maintenance dialysis, the Kidney Health Initiative assembled a group of individuals with relevant expertise from academia, industry, and government to provide the nephrology community with information about the design and conduct of such trials, with a specific focus on the dialysis setting. Here, we review this information, and where applicable, use experience from the ongoing Time to Reduce Mortality in End Stage Renal Disease Trial, a large cluster-randomized, pragmatic trial evaluating hemodialysis session duration, to illustrate challenges and solutions to operational, ethical, and regulatory issues.

The nephrology community has expressed concern about the inadequate evidence base for guiding clinical practice and the slow pace of improving outcomes for patients with kidney disease.^1,2 A significant barrier to evidence generation is the inefficiency and expense of clinical trials, the most rigorous method for evaluating treatment approaches. The number of published clinical trials addressing kidney disease is lower than for many other disease areas^3,4 and most nephrology trials, like those in other medical specialties, have small sample sizes.^5,6 Moreover, interventions found to be beneficial in well conducted trials are often not widely implemented in practice.^7–9

In recent years there has been a renewed interest in pragmatic clinical trials, also referred to as “real-world” clinical trials, because of their potential to yield findings that are highly generalizable, identify interventions that are readily translatable into clinical practice, and increase the efficiency of trial conduct.^10–12 In contrast to explanatory trials which evaluate efficacy by testing treatments under idealized conditions and in highly selected patients, pragmatic trials evaluate effectiveness of interventions implemented under the usual conditions of clinical care and for the broad group of patients with the condition of interest. Pragmatic trials usually study treatments that are already in widespread use, with the goal of identifying optimal approaches.

Maintenance dialysis is a setting that is well suited for pragmatic clinical trials. In dialysis, encounters with patients are frequent and predictable, data generated through routine care are highly granular and uniform across patients, electronic data systems are widely utilized, and, in the United States, the delivery of care to approximately 90% of the patients by a small number of provider organizations facilitates centralized trial implementation. Recognizing the potential of pragmatic trials in the dialysis setting, the Kidney Health Initiative – a consortium of >75 organizations founded by the American Society of Nephrology and the US Food and Drug Administration (FDA) – assembled a working group with representation from academia, industry, and government to identify key considerations for conducting pragmatic trials in dialysis.¹³ In this paper, we discuss several of the important scientific, operational, and regulatory issues for such trials. Where applicable, we provide concrete illustrations from experience with the Time to Reduce Mortality in End Stage Renal Disease (TiME) Trial (NCT02019225), an ongoing, large, cluster-randomized pragmatic trial of hemodialysis session duration being conducted through a partnership between academic investigators and two large dialysis organizations as one of the demonstration projects of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory.¹⁴

Pragmatic Trial Design

Characteristics of pragmatic trials and their implications for trial design are provided in Table 1.^15,16 Pragmatic trials often randomize by “cluster” rather than by patient to facilitate implementation of the intervention(s) by clinical personnel within a health care team and to avoid contamination across treatment groups. Sample size requirements are usually larger for pragmatic than explanatory trials in order to overcome cluster effects (if cluster randomization is used) and because of reductions in treatment effects resulting from trial populations that are not highly selected for patients most likely to adhere to or benefit from the intervention. Approaches to implementing the intervention are typically flexible to ensure that the trial experience generalizes to the clinical setting. End points for pragmatic trials are often limited to those ascertainable from routine, clinically acquired data available in electronic medical or administrative records, unless there are sufficient resources to support large-scale data capture specifically for the trial.

Table 1.

Pragmatic trial characteristics and design implications

Component	Characteristics	Implications
Eligibility criteria	Nonrestrictive in order to increase generalizability	Allows for rapid enrollment; longer follow-up is possible if trial duration is fixed
		Necessitates large sample size if nonselectivity reduces effect size
		Allows for EMR-based identification of participants
Intervention	Often already in widespread use	Sometimes allows for waiver or modification of informed consent
	Implemented by clinicians rather than researchers	Reduces required research infrastructure
	Can be implemented with flexibility	Effect size reduced if incomplete adherence and crossover between treatment groups
Control	Alternative to intervention that is also already in widespread use, or “usual care”	With usual care control, secular changes during trial conduct should be anticipated
Randomization	Often by cluster rather than individual participant	If cluster-randomization, sample size determination should incorporate:
	Treatment assignment usually not masked	Intracluster correlation coefficient
		Size of clusters; variability of cluster size
		Number of clusters
		If cluster-randomization, less potential for contamination across treatment groups but greater possibility of baseline imbalances across treatment groups
		If individual participant randomization, smaller sample size is required but sample may be less representative of population with condition
		Lack of masking allows for management by clinicians rather than research personnel
Outcomes	Often ascertained through electronic health records and/or administrative databases	Requires upfront attention to data specifications for outcome ascertainment
	Usually clinically important rather than intermediate or surrogate outcomes
Adverse event monitoring	Adverse events often ascertained through electronic health records and administrative databases	Can necessitate use of targeted rather than comprehensive adverse events

EMR, electronic medical record.

The design of the TiME Trial is summarized in Table 2. In brief, dialysis facilities are randomized to Intervention or Usual Care. Facilities in the Intervention group adopt an approach of prescribing dialysis session durations of at least 4.25 hours for incident patients; facilities in the Usual Care group have no trial-driven approach to session duration. The primary outcome is mortality, and major secondary outcomes are hospitalizations and quality of life. Pragmatic features of the TiME Trial include broad eligibility criteria, the implementation of the intervention by treating nephrologists and dialysis unit personnel rather than by researchers, and reliance on data obtained through routine clinical care. The major trial outcomes (mortality, hospitalization rate, and quality of life) are patient-centered and ascertainable through the electronic data systems of the dialysis providers. To address cluster effects, the sample size determination incorporated an intrafacility correlation coefficient for mortality, the primary outcome of the trial, as well as the number and size distribution of clusters. These factors increased the required number of participating patients by 28%.

Table 2.

TiME Trial design

Design Element	Details
Treatment groups	Intervention: dialysis facility adopts approach of prescribing hemodialysis session duration of ≥4.25 h
	Usual Care: no trial-driven approach to hemodialysis session duration
Randomization	Cluster-randomization by dialysis facility
	Stratification by provider organization, facility race distribution, and facility catheter use
Target sample size	6432 patients
	402 clusters
Eligibility criteria – patients	Age ≥18 yr
	Initiation of maintenance hemodialysis within the past 120 d
	Ability to provide consent for dialysis
	Currently treated with in-center hemodialysis
Eligibility criteria – facilities	Agreement by facility leadership and nephrologists to adopt trial intervention
	Capacity to accommodate treatment session durations of ≥4.25 h for incident patients
Duration of intervention	Median 2.5 yr
Source of data	Dialysis provider organization electronic data systems
Primary outcome	Mortality
Major secondary outcomes	Hospitalizations
	Quality of life
Consent approach	Opt-out
Analysis	Primary: intention to treat, patient-level
	Secondary: incorporation of intervention adherence
Oversight	Single IRB of Record
	External Data and Safety Monitoring Board

Partnerships between Researchers and Health Systems

Because pragmatic trials are intentionally integrated into clinical care delivery, health care providers rather than researchers carry out the research protocol. The incorporation of research into care delivery has been described as a “learning health system,” a term reflecting the goals of systematically learning from clinical care encounters and developing models for implementation that are sustainable from one learning activity (e.g., a clinical trial) to another.^17–19 In a learning health system, partnerships between researchers and the health care delivery organizations are critical, especially if the goal is to accommodate multiple intervention trials concurrently and over time. Thus, whereas the principal stakeholders for explanatory trials are the investigators, sponsors, and participants, for pragmatic trials health system administrators and clinicians are also key stakeholders.²⁰

Harmonizing interests of investigators and providers is essential for the success of pragmatic trials but can be challenging. Priorities for research questions often differ between investigators and health system leaders.²⁰ Leaders of dialysis organizations, like those of other health care systems, are interested in evaluating interventions that add long-term value or address challenges in care delivery, payer targets, or regulatory requirements.²¹ In contrast, academic investigators might be more likely to prioritize studies that inform clinical practice guidelines. Additionally, dialysis providers favor studies that provide answers quickly in order to adopt approaches rapidly on the basis of the results and minimize the effect of the trial on competing initiatives. Quality improvement (QI) initiatives are often appealing to health systems as a way to systematically implement what are thought to be best practices. Incorporating features of pragmatic clinical trial design into QI programs allows the effectiveness of such practices to be evaluated with scientific rigor. For example, the use of facility-level randomization for a hemodialysis central venous catheter care QI initiative allowed a dialysis provider organization to not only implement the clinical care approach but to assess its effect on infection outcomes.²²

For the TiME Trial, the academic investigators and dialysis provider organizations, Fresenius Medical Care and DaVita, partnered early in the planning stages, before obtaining funding, to develop the research question and implementation approach. Members of the research divisions at both dialysis companies are voting members of the TiME Trial Steering Committee, and the work of implementing the trial is shared by the group. Although engagement by the leadership of the dialysis organizations has been present throughout planning and implementation, one of the more challenging aspects for the TiME Trial has been developing approaches for direct interaction between the researchers and the clinicians at individual dialysis units. Engaging “on-the-ground” clinicians, a frequently reported challenge for large pragmatic trials,^23,24should be overcome more readily if clinical trials become routinely incorporated into the delivery of clinical care.^25,26

Selection of Trial Sites

Criteria for site selection in pragmatic clinical trials will vary depending on the study design, particularly when cluster randomization is used. Characteristics of clinical facilities can markedly affect the likelihood of successful participant recruitment, protocol implementation, and delivery of information to staff and patients. In the dialysis setting, although it is important to evaluate dialysis units individually, there are efficiencies in collaborating with provider organizations that comprise large networks of facilities that can serve as research sites. Such collaborations streamline the creation of clinical trial agreements and allow for a uniform approach to trial implementation by utilizing existing standardized practices, policies, and operating procedures.¹²

For individual participating dialysis units, stable structure and function are important for successful implementation of a pragmatic trial that relies on facility personnel for its implementation. Prioritization of study sites can be informed by preliminary assessments of the number of eligible patients, strong leadership from both the medical director and clinical manager, and a track record of compliance with policies and procedures. Review of ongoing clinical trials and QI initiatives within candidate facilities for overlap or conflict with the interventions or effect on the intended outcomes is also important. For the TiME Trial, the initial identification and evaluation of candidate facilities was not performed by the academic investigators but rather by the dialysis organization research teams in collaboration with regional administrators who had the necessary knowledge about the facility dynamics and capacity for trial implementation.

Adherence by Sites and Participants

An important benefit of conducting trials in partnership with health care delivery organizations is the ability to leverage existing systems for trial implementation. Dialysis provider organizations have well developed systems for communicating with large numbers of geographically disparate facilities, launching new initiatives, and providing feedback about performance. These systems can be used to motivate facility adherence to a trial protocol. For the TiME Trial, dialysis units randomized to the Intervention group receive regular facility-specific reports about adherence to the trial intervention. At the individual participant level, approaches used routinely by dialysis personnel to provide feedback about clinical targets (e.g., regular distribution of “patient report cards”) can be leveraged to promote adherence to the trial intervention.

Data Acquisition

Compared with many other health care delivery systems, dialysis has the advantage of highly standardized clinical data collection despite the large numbers of facilities and treating clinicians. Dialysis units maintain information systems with granular patient-specific demographic and clinical data. These systems consolidate the day-to-day flow of laboratory, treatment, dialysis machine, and operational data, most of which are stored as discrete data elements, rather than text fields.²⁷ When using existing data systems for a trial conducted by multiple dialysis provider organizations, it is important to implement data specifications appropriate for the analytic requirements of the trial. Quality measure development and specification information, posted by the Centers for Medicare and Medicaid Services for its purposes, may be applicable for other uses such as data extraction for clinical trials.²⁸

For the TiME Trial, prespecified data elements are transmitted from the data warehouses of DaVita Healthcare Partners and Fresenius Medical Care to the data coordinating center at the University of Pennsylvania. Compared with more conventional trials that rely on study-specific primary data collection, the approach used by the TiME Trial is highly efficient, inexpensive, and less prone to errors associated with manual data entry. Relying exclusively on data obtained by dialysis facilities has important limitations, one of which is the lack of information about events that occur outside of the dialysis facility, but some of these can be reduced through supplementation with other sources such as claims data.^29,30

Informed Consent

Obtaining informed consent for large pragmatic trials can pose challenges if: (1) study implementation is highly centralized without research personnel at the study sites, (2) cluster-wide implementation of the intervention is required for either scientific or operational reasons, or (3) the consent requirement will lead to imbalances in participant characteristics across treatment groups (as might occur if the randomized treatment arm for the cluster is known before enrollment of individual participants). For research that is subject to the Department of Health and Human Services (HHS) regulations, waiving or modifying the informed consent process may be permitted if specific criteria are met, one of which is that the research presents no more than minimal risk of harm to participants.³¹ Although it is intuitively clear that the minimal risk requirement limits the interventions that can be studied when using trial designs that necessitate waiving or modifying consent, it is important to recognize that there is variability across institutional review boards in interpreting minimal risk and active debate among ethicists about how risks should be assessed.³² Additionally, revisions currently under consideration to the federal regulations for human subjects research protection (referred to as the Common Rule), have implications for the concept of minimal risk and how consent is approached in pragmatic trials.³³ The TiME Trial met the criteria for waiving consent and is using an opt-out approach for trial participation in combination with an Institutional Review Board (IRB)-approved waiver of Health Insurance Portability and Accountability Act Authorization.³⁴

Specific additional FDA regulations regarding informed consent apply to clinical investigations conducted under FDA oversight. Trials investigating drugs or devices (even if determined to be exempt from Investigational New Drug [IND] or Investigational Device Exemption [IDE] requirements, respectively), must adhere to the FDA regulations directed at human subjects protections, including informed consent requirements as detailed in the Code of Federal Regulations (CFR) 21 CFR 50, and the requirements for IRBs (21 CFR 56).^35,36 Unlike the Common Rule, no waivers exist for obtaining informed consent for clinical investigations regulated by FDA except when participants are in life-threatening situations in which consent is not feasible.³⁷ However, a provision does exist that allows an IRB to waive the documentation of informed consent for certain types of minimal risk research.³⁸ Although the TiME Trial is not investigating a drug or device, and thus does not fall under FDA purview, this provision would likely allow for the opt-out consent process used in the TiME Trial and has also facilitated the conduct of pragmatic trials investigating FDA-regulated products.³¹

Institutional Review Boards

For all multicenter studies, whether pragmatic or explanatory, oversight by numerous local IRBs can pose challenges.^39–43 Variability across institutions in approaches and interpretation of regulations are important contributors to the slow pace and expense of large trials, and it has been argued that review by multiple IRBs may paradoxically reduce protections to participants.⁴⁴ For pragmatic trials, particularly those that use cluster-wide implementation of an intervention, variability across institutions in the interpretation of regulations, such as the criteria for waiving informed consent, can result not only in delays but also in irresolvable impasses. Moreover, for pragmatic trials in which local clinicians implement the intervention but researchers are not directly affiliated with the sites, reporting and responding to local IRBs is not practical. For the TiME Trial, the University of Pennsylvania IRB is serving as the IRB of record, and authorization agreements were created between each dialysis provider organization and the University of Pennsylvania IRB. Although the use of central IRBs or single IRBs of record is becoming increasingly common, willingness of institutions to utilize these models varies, and concerns have been raised about eliminating local context from the review process.^43,45 However, both a recent draft policy by the NIH and proposed revisions to the Common Rule by HHS and other federal bodies promote the use of a single IRB for multisite studies, suggesting that in the near future this approach may become the norm rather than the exception.^33,46 Of note, the FDA has made clear in guidance documents that it supports the use of a central IRB for drug studies⁴⁷; however, the Federal Food, Drug, and Cosmetic Act requires local IRB review for device studies.⁴⁸

Data and Safety Monitoring Committees and Adverse Event Reporting

Clinical trials, particularly those with safety risks, often have Data and Safety Monitoring Boards (DSMBs) to evaluate safety, efficacy, and quality of trial conduct while the study is underway. Although external monitoring is usually desirable for pragmatic trials, design or implementation features might necessitate an approach by the DSMB that differs from that used for most explanatory trials.⁴⁹ For example, the TiME Trial DSMB does not have individual serious adverse event reports available for review but instead relies on prespecified, clinically acquired laboratory data or trial outcomes to assess safety. Additionally, in the interest of understanding real-world experience, a pragmatic trial DSMB might focus little attention on monitoring adherence to the intervention. On the other hand, for an open-label, cluster-randomized pragmatic trial in which group assignment is known before participant enrollment, a DSMB might be highly interested in comparing participant characteristics across treatment groups to assess selection bias. Although there are not regulatory requirements for DSMBs, efforts currently underway to generate consensus documents about external monitoring will presumably address different types of trials including pragmatic trials.⁵⁰

FDA IND or IDE Provisions

There are perceptions that trials conducted under IND or IDE provisions cannot use highly pragmatic design or implementation approaches because of data collection and safety reporting requirements^51,52; however, this is not necessarily the case. For example, questions have been raised about whether the electronic clinical data systems used by dialysis providers meet the FDA regulations on electronic records described in 21 CFR part 11. FDA guidance that describes how electronic source data should be captured, reviewed, and retained specifically states that the FDA does not intend to assess the “part 11 compliance” of the computer system (e.g., the electronic medical record) used in a clinical investigation.⁵³ Notwithstanding the guidance recommendations, additional clarification from regulatory authorities on proper approaches to handling electronic data from dialysis information systems could encourage sponsors to pursue pragmatic clinical trials of drugs and devices used in dialysis. Another challenge for pragmatic trials conducted under IND or IDE provisions is determining who among the many individuals implementing the trial meets the regulatory definition of “investigator.”⁴² Because this designation carries specific responsibilities for both sponsors and investigators, the extent to which pragmatic trials can efficiently increase the evidence base will depend on whether clinicians operating in a learning health system are subject to such regulatory requirements.⁵⁴ From a safety standpoint, FDA requirements for adverse event reporting might be a barrier to pragmatic trials that rely on electronic medical record data rather than investigators’ reports of individual events. However, both the regulations themselves and guidance documents make it clear that the FDA may waive these requirements if certain criteria are met.^55,56 Thus, for pragmatic trials comparing well established treatments, alternative approaches to monitoring patient safety proposed by the sponsor might be deemed acceptable by the FDA.

Moving Forward

The use of pragmatic clinical trials to fill the large gaps in our knowledge about caring for patients receiving maintenance dialysis is highly appealing and there are several fundamental questions that, at least at first glance, appear to lend themselves well to this approach. Examples include identifying optimal targets for serum phosphorus concentration, strategies for anemia and iron management, and electrolyte composition of dialysis solutions. However, in order for the full potential of pragmatic trials to be realized there are important challenges that require focused attention (Table 3). Some of these are operational or technical – e.g., the lack of efficient systems for engaging and communicating with “on-the-ground” nephrologists at dialysis units, and limited access to clinical information from care delivered outside of the dialysis unit. Others are cultural or philosophical, such as reluctance by some IRBs to cede oversight to another institution, or differing views about acceptable approaches to adverse event monitoring for trials evaluating treatments already in widespread use. Ethical challenges include reconciling tensions between providing benefits to large populations and maintaining protections for individuals. And, because of the lack of distinction between full and modified informed consent, unless the unmodified informed consent process can be operationalized in large trials incorporated into clinical care delivery, regulatory requirements limit the interventions that can be studied to those that pose minimal risk. Despite these challenges, there is tremendous interest, both nationally and globally, in increasing the momentum for pragmatic trials, and funders of research, including industry sponsors, are increasingly embracing this approach, in part to reduce cost, but also to generate findings that are rapidly translatable to practice.^57–64 As more of these trials are conducted in dialysis, in other areas of nephrology, and in medicine more broadly, approaches to overcome operational and technical barriers will continue to be identified, the views of potential research participants on consent approaches and data sharing will be better understood, and perhaps, over a more extended time frame, revisions will be made to current regulations and policies that will expand the scope of questions that can be answered.

Table 3.

Increasing the impact of pragmatic trials

Domain	Areas for Focused Efforts
Operational and technical	Optimizing systems for interacting with clinicians at health care delivery sites, particularly for nonemployee physicians
	Developing approaches to minimize disruptions to clinical work flow by research activities
	Developing strategies for handling complex data extraction, creating systems interoperability, addressing missing data, establishing computable phenotypes, and incorporating documentation tools into electronic medical records
Cultural and philosophical	Elevating interest by health system leaders, clinicians, and patients in incorporating research into the clinical care setting
	Addressing privacy concerns of providers and patients
	Harmonizing varied interpretations of research-related risks
	Addressing resistance by institutions to the use of a single IRB of record
	Refining approaches to data and safety monitoring
Ethical	Recognizing tradeoffs between improving outcomes for populations and protecting rights of individuals
	Identifying appropriate “gatekeepers” for access to participants
	Clarifying the importance of downstream and indirect effects of research on nonparticipants
Regulatory	Addressing the lack of distinction by the Common Rule between modifying and waiving informed consent
	Addressing adherence to regulations regarding vulnerable populations
	Harmonizing relevant Common Rule and FDA regulations
	Clarifying acceptable approaches to use of electronic data for trials under IND and IDE provisions
	Clarifying options for safety reporting for trials under IND and IDE provisions
	Addressing the requirement for research indicator for Medicare claims
	Clarifying distinctions between research and quality improvement
Funding	Increasing collaborations between academic investigators and health systems to identify and maximize opportunities for incorporating trials into health care delivery
	Educating grant reviewers about unique design or implementation considerations for pragmatic trials

Disclosures

E.L.Jr. was an employee of Fresenius Medical Care (Waltham, MA) when this project was initiated. M.K. is employed by and owns stock in DaVita Healthcare Partners.