The California Resuscitation Outcomes Consortium (CAL-ROC): A novel collaboration to facilitate the implementation of randomized clinical trials in the prehospital setting

Abstract

Background

Few large randomized clinical trials (RCTs) have been conducted to inform the prehospital phase of care for out-of-hospital cardiac arrest (OHCA). We describe the development of a consortium to facilitate large-scale prehospital RCTs through a novel research collaboration, leveraging pre-existing prehospital and hospital data capture infrastructure.

Consortium description

We developed a consortium consisting of 173 emergency medical services (EMS) Provider Agencies and four academic “hubs.” The consortium is an innovative collaboration consisting of a diverse set of EMS experts from across California and designed to overcome logistical, cost, and regulatory challenges associated with prehospital research. All participating agencies share data via a state EMS database, California EMS Information System (CEMSIS), and contribute data to the national Cardiac Arrest Registry to Enhance Survival (CARES) database. Data from CEMSIS and CARES will be linked to capture RCT outcomes. We abstracted two years of data from the CARES database to characterize the population served by the consortium and facilitate sample size calculations for future trials. We estimate that the consortium will have the ability to enroll a diverse population of patients with OHCA, at a rate of approximately 19,000 per year across all sites, for a future trial of cardiac arrest therapies.

Conclusion

This collaboration uses pre-existing data infrastructure to capture prehospital and hospital outcome data to facilitate large-scale prehospital RCTs for time-critical emergencies.

Introduction

Despite the significant public health burden associated with out-of-hospital cardiac arrest (OHCA), there is a significant disparity in the number of large randomized clinical trials (RCTs) that inform best practices in care for this disease, as compared to other leading causes of death, such as myocardial infarction, heart failure, or stroke.1, 2, 3, 4 A systematic review of adult cardiac arrest RCTs between 1995 and 2014 identified major gaps in trials evaluating protocol interventions, post-cardiac arrest care, and long-term survival outcomes.⁵ A subsequent review of all cardiac arrest RCTs between 2015 and 2022 documented a continued paucity of published studies.⁶ The relative lack of RCTs for OHCA, and specifically of trials focused on the prehospital and immediate post-resuscitation phases of care, threatens the advancement of evidenced-based cardiac arrest care.

A major barrier to conducting OCHA RCTs is the complexity and discontinuity between prehospital and hospital information technology infrastructure, which leads to challenges in capturing and linking patient data in large, multi-center trials.⁷ While most immediate OHCA care takes place in the prehospital setting, the primary patient-centered outcome measures often can only be collected after the patient arrives at or is discharged from the hospital.⁷ Data collection in two distinct venues followed by record linkage presents a complex logistical challenge resulting in significant financial barriers to conducting research.⁴ In 2011, the Resuscitation Outcome Consortium (ROC) Cardiac Epistry database was established, which prospectively collected prehospital and hospital-based outcome data for patients with OHCA at each of the 11 sites in the United States (US) and Canada, that was then submitted to a central database.⁸ This novel database supported OHCA RCTs, leading to practice changes for OHCA care worldwide.⁵ Version 3 of the ROC Cardiac Epistry concluded data entry in June of 2015.⁹ To date, no other similar US-based consortium has been developed, although international resuscitation research consortia—including the Pan-Asian Resuscitation Outcomes Study (PAROS) clinical research network, the Australian Research Outcomes Consortium (Aus-ROC), and the Japanese Association for Acute Medicine Out-of-Hospital Cardiac Arrest (JAAM-OHCA) Registry—continue to operate.10, 11, 12.

Further barriers to conducting large RCTs of treatments for OHCA include operational and logistical challenges. These include barriers related to the training of emergency medical services (EMS) clinicians to administer investigational treatments, the willingness of patients and EMS clinicians to participate in trials, the implementation of randomization in the prehospital setting, and, lastly, ethical and regulatory considerations related to conducting research under exception from informed consent (EFIC).⁷

We developed the California Resuscitation Outcomes Consortium (CAL-ROC) to establish vital research infrastructure and specifically address the challenges associated with conducting large, multi-site RCTs aimed at testing interventions to treat OHCA, as well as other emergent medical and traumatic conditions. The goal of this manuscript is to describe the development, design, and structure of CAL-ROC. This description may serve as a guide for other investigators seeking to develop similar consortia to conduct prehospital clinical trials.

Consortium description

Setting

The provision of EMS services in California is overseen by the state of California EMS Authority (CA-EMSA), which determines state scope of practice and, per established regulation, further directs medical oversight to the 34 Local EMS Agencies (LEMSAs). A LEMSA is a regionally defined governmental organization authorized by California’s EMS Act to provide regulatory oversight of all emergency medical services within a geographical area (county or group of counties).¹³ Led by an administrative director and a medical director, LEMSAs set all EMS policies and protocols for their region, authorize the EMS Provider Agencies operating in the region to deliver direct patient care services, designate all 9-1-1 receiving hospitals and specialty centers, coordinate disaster response, and oversee regional quality improvement efforts (Fig. 1b).¹⁴ CAL-ROC was formed, over the course of three years, as a collaboration amongst 13 LEMSAs and three individual EMS Provider Agencies. The three individual EMS provider agencies have approval from their associated LEMSAs, although these LEMSAs are not participating as a whole region in CAL-ROC. Overall, 173 EMS Provider Agencies are represented in the consortium, the majority of which provide fire-based EMS services. Together, this unique group represents LEMSAs and EMS Provider Agencies with both high- and low-volume services, varying response paradigms (i.e. BLS or ALS), and air and ground transport capabilities, serving diverse populations from across the rural–urban spectrum with varying disease acuity. Additionally, CAL-ROC encompasses investigators from four affiliated academic institutions within California (Fig. 1a).

Fig. 1.

a. Map of Emergency Medical Services Agencies Participating in the Consortium b. Organizational and Regulatory Structure of California Emergency Medical Services.

Organizational structure and EMS medical direction

The consortium builds upon the existing structure of EMS systems in California such that each LEMSA and EMS Provider Agency is a participant affiliated with one of four academic “hubs”: (1) The Lundquist Institute at Harbor-University of California, Los Angeles (Lundquist/UCLA); (2) University of California, Davis (UCD); (3) University of California, San Diego (UCSD); and (4) University of California, San Francisco (UCSF). Each hub has a research team experienced in all aspects of EMS research including training, implementation, data collection and entry, and system quality improvement. All hubs and many of the LEMSAs have participated collaboratively in one or more randomized clinical trials15, 16 or in The Strategies to Innovate Emergency Care Clinical Trials Network (SIREN).¹⁷ Several prehospital randomized trials, conducted across multiple hubs, have been implemented via collaborative agreements associated with the Pediatric Emergency Care Applied Research Network (PECARN) and the history of participation in these trials will accelerate the development of contracts and data-sharing agreements under CAL-ROC.

The CAL-ROC consortium infrastructure also includes a Clinical Coordinating Center (CCC) and Data Coordinating Center (DCC). The CCC is located within The Lundquist/UCLA hub and provides clinical oversight of the trial through a multiple-principal investigator (PI) leadership model, including a basic scientist and an EMS physician. CAL-ROC is further enhanced by a third PI based at the University of Utah, which serves as the consortium’s centralized DCC, providing regulatory oversight for human subjects’ management through a single institutional review board model, trial data management, and analysis support (Fig. 2).

Fig. 2.

CAL-ROC Organizational Structure; CCC = Clinical Coordinating Center, DCC = Data Coordinating Center, DSMB = Data Safety Monitoring Board, EMS = Emergency Medical Services, FDA = Food and Drug Administration, sIRB = single Institutional Review Board, TLI = The Lundquist Institute, UC = University of California.

Medical oversight and direction are provided by the EMS Medical Directors of the LEMSAs and EMS Provider Agencies participating in CAL-ROC. LEMSA EMS Medical Directors are empowered by the State of California to determine policies and procedures for their agencies and EMS personnel, thus, enabling the development of uniform protocols which can be evaluated over time as a part of a clinical trial.¹⁴ A key component in the formation of CAL-ROC has been a collaborative, iterative, decision-making model between the LEMSA EMS Medical Directors, EMS Provider Agency Medical Directors, CCC and DCC staff, and the academic hubs to establish a shared concept of clinical trial design and medical oversight. This collaborative framework has facilitated the establishment of a large and flexible network of LEMSAs and EMS Provider Agencies which will be vital for the success of CAL-ROC.

The CAL-ROC organizational structure is further enhanced by the presence of two advisory boards and multiple working committees (Fig. 2). The advisory boards consist of the Scientific Advisory Board—composed of national experts in clinical trial design and EMS trial implementation, community engagement experts, and a research ethicist—and the Community Advisory Board, comprising public and civic members to provide the public perspectives from the diverse set of communities served by CAL-ROC. The working committees include EMS Training, EMS Operations, Study Monitoring, and Publications. These committees will develop educational and training modules, implement the training and study protocol, monitor performance at each EMS site, and publish and disseminate study results.

Data collection and analysis

To address the significant challenges related to prehospital and hospital data collection during a prospective trial and accurate linkage of data, CAL-ROC will employ a novel approach to data collection that takes advantage of existing health data streams, established clinical data registries, and data-sharing agreements.

At present, all LEMSAs and EMS Provider Agencies in the consortium have fully implemented electronic patient care report (ePCR) systems for documentation of prehospital care. Data from ePCRs are accessible by EMS Medical Directors and are shared with CA-EMSA. Within CA-EMSA, EMS data from all California EMS 9-1-1 Provider Agencies, including those in CAL-ROC, are aggregated in the California EMS Information System (CEMSIS).¹⁸ CA-EMSA follows the same standardized reporting structure as the National EMS Information System (NEMSIS) registry, which is sponsored by the National Highway Traffic Safety Administration’s Office of EMS, and maintained by the Technical Assistance Center at the University of Utah Data Coordinating Center.¹⁹ In parallel, all LEMSAs and EMS Provider Agencies within CAL-ROC also contribute data to the Cardiac Arrest Registry to Enhance Survival (CARES) database, which collects standardized information about OHCA characteristics, treatments administered, time intervals related to EMS care, and hospital-based outcomes.²⁰

A key infrastructure factor that will enable the success of the CAL-ROC is that CEMSIS and CARES data can be linked by a unique identifier existing in both databases, effectively linking prehospital and hospital data. CAL-ROC trials will use CEMSIS to capture prehospital data and CARES to capture hospital outcome data for the OHCA trials. This negates the need for burdensome study-specific emergency department and hospital data collection procedures, which would be cost-prohibitive to conduct across sites enrolling tens of thousands of patients. Linked data will then be housed in the DCC (Fig. 3).

Fig. 3.

CAL-ROC Data Flow; CARES = Cardiac Arrest to Enhance Survival database, CEMSIS = California Emergency Medical Services Information System, DCC = Data Coordinating Center, EMS = Emergency Medical Services.

While most data will come from CEMSIS and CARES, select data fields not existing in either registry may be collected via “paramedic self-reports” immediately after care delivery. For example, the occurrence of rearrest in the immediate post-resuscitation period after OHCA is not reliably captured in either database; a “paramedic self-report” regarding the presence of rearrest will be entered via a secure link directly into a REDCap database maintained by the DCC or will be embedded within the ePCR.

Exception From Informed Consent (EFIC) and regulatory compliance

It is generally not feasible to obtain informed consent from participants in trials of treatments for acute life-threatening conditions, such as those that will be targeted by CAL-ROC.21, 22, 23, 24 Thus, participants in such RCTs must often be enrolled under exception from informed consent (EFIC).²⁵

The EFIC process requires engagement and consultation with the local community, which the LEMSAs participating in the consortium are uniquely positioned to facilitate. As part of the oversight of their respective regional EMS systems, each LEMSA already routinely engages with EMS community partners and the public through its comprehensive committee structure.²⁶ Local and statewide EMS Commissions meet regularly throughout the year and are well positioned to disseminate information about and receive feedback on proposed CAL-ROC clinical trials. The commissions are composed of representatives from prehospital and hospital-based medicine (i.e., paramedics, nurses, and physicians of various specialties), ambulance companies, local governmental organizations (i.e., Public Safety, City Management, Public Health), the general public, and local hospitals. Public testimony is encouraged as part of this forum, which is essential to meet requirements for community consultation under EFIC. The University of Utah will act as the single IRB for CAL-ROC trials and will supervise the EFIC process for all CAL-ROC studies. The use of the single IRB structure will address lessons learned from previous resuscitation outcomes consortia, in which investigators reported challenges related to coordinating multiple regulatory bodies and locating ethical reviewers with sufficient expertise in prehospital care.²⁷

Additionally, each academic hub will utilize established relationships with the Clinical Translational Science Institutes (CTSI) at their institution to facility community engagement programs and appropriate community consultation under EFIC. Each academic hub will also leverage their university affiliation to gain access to research ethicists who can guide decision-making during the trial design and execution, advise study investigators on the EFIC process, and inform the process of study result dissemination.

Patient population

To understand the potential study population served by CAL-ROC, we abstracted two years of data (2021–2022) from the CARES database regarding patients with OHCA who were treated by agencies participating in the consortium. The EMS Agencies in CAL-ROC serve a combined total population of over 24 million and the annual OHCA volume was estimated to be approximately 19,000 per year. We describe patient volumes, demographic characteristics, and outcomes overall and by academic hub in Table 1.

Table 1.

Characteristics of patients with OHCA served by consortium agencies (N = 38,350).

Variable		UC Davis	UCSF	Lundquist/UCLA	UC San Diego	Overall
Combined Population (millions)		2.4	3.9	11.3	6.6	24.2
Average Annual OHCA (n)		2000	2900	9000	5300	19,200
Age (IQR)		65 (53–76)	67 (54–78)	67 (54–79)	66 (53–77)	66 (54–78)
Sex (% Male)		67	65	63	64	64
Race (%)
	Asian	4	17	5	5	7
	Black	9	22	10	12	12
	Hispanic	12	12	16	28	19
	Pacific Islander	1	2	1	1	1
	White	43	38	15	48	30
	Not Documented*	30	7	53	6	31
Arrest characteristics (%)
	Shockable cardiac rhythm	13	14	10	10	11
	Witnessed Arrest	39	38	36	42	38
	Bystander CPR Performed	44	38	41	44	42
Outcomes (%)
	Sustained ROSC†	22	27	22	23	23
	Survival to Hospital Admission	24	26	19	20	21
	Survival to Hospital Discharge	8	10	7	7	7
	CPC 1–2 at Hospital Discharge	7	8	5	5	6

CPC = cerebral performance category, CPR = cardiopulmonary resuscitation, OHCA = out of hospital cardiac arrest, ROSC = return of spontaneous circulation

^*Race and ethnicity are not captured for patients with termination of resuscitation in the field or who are transported to a medical center that does not participate in CARES. Race and ethnicity are defined according to the US Office of Management and Budget classification, as per the CARES data dictionary.²⁰

^†CARES defines “Sustained ROSC” as an estimated period of 20 consecutive minutes during which chest compression are not required and signs of circulation persist.²⁰

Discussion

CAL-ROC was formed under a shared vision of conducting large-scale RCTs in the out-of-hospital environment and brings together a diverse set of EMS experts from across California. The infrastructure proposed by CAL-ROC specifically addresses major hurdles associated with data capture and accurate linkage between the prehospital and hospital settings. While most large-scale out-of-hospital RCTs have historically used an extensive network of research staff to manually collect and submit prehospital and hospital data,5, 6 CAL-ROC plans to leverage existing prehospital and data registries to capture outcomes. This enables the CAL-ROC team to overcome significant operational barriers associated with data collection, which have often led to high costs for traditional prehospital RCTs. In doing so, we will be able to carry out trials with a far lower per-patient cost than conventional RCTs while maintaining rigorous implementation standards.²⁸

Establishing data infrastructure for prospective prehospital research has historically resulted in high operational costs and has thus represented a significant barrier to conducting prehospital trials. For a recent national, multi-center prehospital trial, the total cost was approximately $10 million with an estimated per patient cost of between $1500 and $3000.²⁹ Assuming a similar per patient cost for a hypothetical future trial enrolling 76,000 patients in OHCA over four years, the total cost to implement the trial with a traditional design could exceed $200 million. However, with our plan to leverage pre-existing data infrastructure, we estimate that such a trial conducted within the CAL-ROC infrastructure will cost approximately $100 per patient to implement. This projection illustrates the potential cost savings through this novel trial mechanism.

While challenges related to data collection and associated costs represent the foremost barrier to conducting prehospital RCTs, there are other operational and logistical barriers that must be overcome. A prior study interviewed paramedics involved in OHCA RCTs to better understand their attitudes toward research and the challenges they face when conducting trial interventions. While the majority of paramedics felt research was important, some expressed ethical concerns about enrolling patients without consent and others described negative attitudes toward trial interventions from non-trial EMS personnel.30, 31 For this reason, previous ROC investigators have emphasized the need to involve paramedics early in the research design process.³² Additional barriers to conducting prehospital trials include operational time-pressure, recruiting and training sufficient EMS Agency participants, the heterogeneity of regional EMS protocols, and complying with regulatory requirements related to EFIC.7, 33, 34, 35, 36, 37, 38, 39, 40, 41 Together, these obstacles further constrain the operationalization of large, multi-site RCTs in the prehospital setting not only for OHCA, but many other time-critical conditions. These challenges represent future targets of research for the CAL-ROC network, including a qualitative evaluation of paramedic perspectives toward staying on scene after ROSC in OHCA.

Conclusion

The formation of CAL-ROC and the development of its inaugural trial is the culmination of an iterative planning process that incorporated input from EMS researchers and study design experts, EMS Provider Agency Medical Directors, frontline paramedics, and community members. Through the use of existing health information technology and established data registries for prehospital and hospital data collection, the CAL-ROC study team aims to overcome many of the barriers associated with traditional out-of-hospital RCTs while maintaining validity of results and minimizing cost. Overall, an innovative collaboration such as CAL-ROC may provide a framework for future researchers to expand implementation of clinical trials in EMS.

CRediT authorship contribution statement

Juliana Tolles: Writing – review & editing, Writing – original draft, Formal analysis, Data curation, Conceptualization. Jake Toy: Writing – review & editing, Writing – original draft, Conceptualization. Nichole Bosson: Writing – review & editing, Methodology, Conceptualization. David G. Dillon: Writing – review & editing, Data curation, Conceptualization. J. Joelle Donofrio-Odmann: Writing – review & editing, Conceptualization. James J. Menegazzi: Writing – review & editing, Data curation, Conceptualization. Juan Carlos C. Montoy: Writing – review & editing, Data curation, Conceptualization. John M. VanBuren: Writing – review & editing, Methodology. Marianne Gausche-Hill: Writing – review & editing, Writing – original draft, Project administration, Methodology, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.