How I Do It: Writing, Reviewing, and Reading Standard and Pragmatic Clinical Trial Protocol Papers

Abstract

Randomized controlled trials provide foundational evidence in clinical medicine. However, the quality of evidence generated by clinical trials depends on the quality of their design, conduct, and analysis. A comprehensive protocol is essential for conducting a rigorous, high-quality trial, and publication of the protocol is an important step in transparent trial reporting. Protocol papers summarize key elements of the protocol for a clinical audience. Such papers afford trialists with an opportunity to explain study design choices and provide details about protocol execution that aid in trial interpretation but do not necessarily fit within the main trial manuscript. Consumers of protocol papers should understand the standards for adequate reporting and use a systematic approach to evaluate the planned trial and identify potential threats to its quality. As pragmatic trials have become increasingly common in critical care, their protocols merit special consideration due to unique trial design features that impact trial interpretation, such as embedding trial operations in clinical practice and using electronic health record data. In this perspective, we review components of protocol papers, discuss current reporting guidelines, and highlight key considerations for writing and reading critical care protocol papers, with a specific focus on pragmatic trials.

Introduction

Randomized controlled trials offer the highest level of evidence in clinical medicine. However, merely randomizing patients to receive an intervention does not guarantee that a trial will produce high-quality results. The quality of evidence generated by a trial depends on its design, conduct, and analysis.

Consider the PARACHUTE trial, published in the British Medical Journal’s satirical Christmas Edition in 2018.¹ In this trial, the authors randomized participants to jump out of an aircraft with versus without a parachute and reported no difference in the primary outcome of death or traumatic injury. The caveat—the jump was from ground height. This represented a “protocol change” that was made during the study when the authors faced low enrollment because they were unable to recruit participants to perform a jump during flight. This alteration of jump height fundamentally changed the research question. While this example is intentionally tongue-in-cheek, it highlights how fatal flaws can be hidden within trial manuscripts and underscores the importance of carefully reviewing the trial protocol to understand how the trial was performed as context for interpreting the results.

Protocol papers — manuscripts published ahead of a trial that summarize the protocol for clinical audiences — illuminate details of trial design and conduct and serve as important tools for evaluating the quality of a trial. In this article, we review the publication of protocol papers, discuss current reporting guidelines, and highlight key considerations for writing and reading critical care protocol papers, with a specific focus on pragmatic trials.

What is a protocol paper?

A protocol is a standard document that describes a trial’s design and outlines operational procedures. A clearly written comprehensive protocol is essential for conducting a rigorous trial (Box 1). Complementing the protocol, the statistical analysis plan (SAP) pre-specifies the approach to data collection and analysis and can be included in the protocol or as a separate document. Most journals require authors to include the protocol and SAP as supplemental documents with the final trial publication. Many researchers also choose to publish protocol papers, manuscripts that provide a high-level summary of the protocol and analysis plan for a clinical audience. These articles, which are often peer-reviewed, report on key elements of the planned trial and should be published prior to trial completion to ensure transparent communication of the a priori trial plans.²

Box 1. Importance of clinical trial protocols for key users.

	For the trial team, the protocol serves as an instruction manual to guide trial operations, including screening, enrollment, randomization, data collection, and analysis.
	For regulators and data monitoring boards, the protocol offers a comprehensive view of the planned trial procedures, which allows regulators and data safety monitoring boards (DSMBs) to ensure appropriate monitoring and protection of human subjects.
	For clinicians, the protocol provides context for how a trial was conducted, which is necessary for interpreting the results. This is particularly important in pragmatic trials, which focus on understanding the effectiveness of interventions in real-world settings. The protocol paper provides a summary of the protocol and a discussion of the trial design for clinical readers.
	For patients in the study, the protocol ensures study procedures and safety monitoring are carried out properly. More broadly, the protocol ensures that trial results are valid and can be applied correctly in practice.

Why publish a protocol paper?

Publishing protocol papers is beneficial for both researchers and clinicians.³ All trials involve trade-offs to balance rigor with efficiency and feasibility. The protocol paper offers researchers the opportunity— often lacking in the main trial manuscript — to describe and justify their design choices. The protocol paper also promotes awareness of the trial among journal editors and other investigators, which helps prevent redundant research, facilitates collaborations, and provides press for the trial. For clinicians, the protocol paper is an accessible way to learn about a planned trial. Reading a protocol paper allows clinicians to evaluate the trial on the merit of the question and design before being biased by the results. While protocol papers are often published for large multicenter trials, sharing a trial protocol and describing trial design choices facilitates interpretation for all trials, regardless of size.

A note about pragmatic trials

Pragmatic trials, which evaluate the effectiveness of interventions in actual clinical practice, are becoming increasingly common in critical care.⁴ This contrasts with explanatory trials which evaluate the efficacy of interventions in often-narrow clinical contexts. Specifically, an explanatory trial asks whether an intervention works in an ideal setting; a pragmatic trial asks whether it works in the “real-world.” Pragmatic trial designs facilitate the enrollment of large, broad populations that reflect patients treated in clinical practice. For example, an explanatory trial might compare norepinephrine versus vasopressin among patients with confirmed septic shock following a strict protocol⁵, whereas a pragmatic trial would compare the same vasopressors among all patients who have a vasopressor ordered using routine clinical protocols for vasopressor administration.⁶ In critical care, pragmatic trials have been used to study antibiotic selection, vasopressor thresholds, oxygen targets, and intubation approaches.^6–12

Like rigorous explanatory trials, rigorous pragmatic trials provide high quality evidence. However, researchers and trial consumers must be aware of the ways that pragmatism affects trial interpretation, for example contamination in delivery of the trial intervention and limitations in the availability of data needed to understand the treatment effect (Table 1). The protocol paper offers an opportunity for researchers to describe and justify pragmatic design choices and discuss their potential impact on trial results. Throughout this review, we highlight specific considerations for writing and reading a pragmatic trial protocol paper.

Table 1.

Components of a clinical trial protocol

Element	Details	Threats to validity in pragmatic trials	Considerations for pragmatic trial protocols
Title	Must include the trial design, population, and interventions		Title should include designation of the trial as pragmatic
Roles and Responsibilities	Lists investigators, contributors, and sponsors
Version tracking	Lists prior protocol versions with dates and notes changes made with each update. Essential to ensure transparency.
Protocol summary	Brief, high-level overview of the trial
Background, aims, and hypotheses	Provides rationale for the study and clearly states the hypotheses that will be tested
Setting and participants
Setting	Describes where the study will be performed, e.g., what types of hospitals (community, academic) and the locations of these hospitals	Performing trials in “real-world” settings increases the risk of contamination, learning effects, and secular trends in co-interventions.	Provide sufficient detail to allow the reader to compare the trial setting to their practice setting.
Study population	Details the inclusion and exclusion criteria	Given that clinicians are often responsible for enrollment, potentially subjective eligibility criteria can introduce selection bias, particularly in cluster-randomized designs where the treatment assignment is known in advance.	Provide enough information to allow the reader to determine if the included patients are representative of typical patients in their practice. Explain how selection bias will be mitigated, particularly for cluster-randomized trials.
Enrollment	Describes the process for identifying eligible patients and enrolling them in the trial	Clinicians may selectively exclude patients based on preconceived biases about a treatment’s effect, which can threaten external validity.	If electronic tools, such as EHR alerts, are used for enrollment: describe the tool, report how it is triggered to fire in the EHR and explain how the clinician will interact with the tool. If the trial is practice-embedded and enrollment is carried out by clinical teams: describe this process, including what training clinical teams will receive and how enrollment will be monitored to avoid introducing bias.
Study procedures
Randomization	Describes the approach to randomization, including sequence generation, allocation concealment, and implementation	If randomization is performed by clinical teams, robust approaches to ensuring allocation concealment are essential to prevent selection bias.	If an electronic tool is used for randomization, describe the tool and its design. If randomization is carried out by clinical teams (e.g., using numbered envelopes), describe who will carry out this task, how they will be trained, and any other steps taken to ensure allocation concealment.
Intervention	Describes the intervention	Pragmatic trials that rely on clinical teams to carry out interventions are at risk for contamination between arms and learning effects.	Describe how the intervention differs from standard care. Is the intervention added to standard care (e.g., telehealth call to promote spontaneous awakening and breathing trials53)? Or does the intervention modify standard care (e.g., order for one antibiotic versus another8 or selection of oxygen targets12)?
Comparator	Describes the comparator	Secular trends in co-interventions may affect trial interpretation.	In pragmatic trials embedded in clinical practice, it is especially important to detail usual clinical care at sites (e.g., the protocol may include copies of clinical care protocols for oxygen monitoring in a trial of oxygen targets29)
Blinding	Describes blinding of trial procedures and provides rationale when blinding is not possible		Blinding treatment assignment is often impossible. However, outcome adjudication should be done blindly.
Data and analysis plan
Data collection	Includes approach to data collection and management	Routinely collected clinical data can be subject to differential reporting of outcomes, particularly in non-blinded trials, and missing data.	If using EHR data, include a detailed variable list with data sources and definitions to ensure reproducibility.
Statistical analysis plan	Details the primary analysis plan, primary and secondary outcomes, and any pre-specified secondary or subgroup analyses		If using EHR data, should address common issues, including how missingness will be handled and how outcomes that may not be available in the EHR, such as 90-day mortality, will be ascertained
Sample size estimates	Reports sample size calculation, including interim analyses and stopping rules	Cluster randomized designs limit power due to similarities among patients within clusters, which need to be addressed in sample size calculations.	The CONSORT statement for pragmatic trials recommends reporting a minimally important difference (smallest clinically relevant difference the trial can detect).19 Protocol papers for cluster randomized trials should account for clustering in power calculations.
Safety and monitoring
Risk assessment and protection of human subjects	Describes risks/benefits of the trial and provides a plan for protection of human subjects		Must include a rationale for waiver or alteration of informed consent, if used
Safety and data monitoring and reporting	Includes the plan for trial monitoring and the role of the DSMB.		If using EHR data, describe how often data queries will be performed to collect data and how these data will be shared with the DSMB
Adverse events	Defines adverse events and reports how these will be monitored and reported.	Using routinely collected clinical data can lead to inconsistently reported adverse events.	If using EHR data, specify how adverse events will be identified, in particular the role of EHR queries versus manual review and the frequency of this monitoring

EHR= electronic health record

Writing and reading a protocol paper

Guidelines and Requirements

Numerous guidelines for clinical trial development, analysis, and reporting provide a roadmap for researchers (Table 2). There are two major guidelines for reporting: the CONSORT guidelines for trial results and the SPIRIT guidelines for trial protocols. Both of these statements were developed by the Enhancing the Quality and Transparency of health research (EQUATOR) Network and were updated in 2025.¹³ Adhering to these guidelines increases the accuracy and transparency of reporting.¹⁴

Table 2.

Major guidelines for clinical trial development and reporting

Guideline/Statement	Year Published	Focus	Comments
Enhancing the Quality and Transparency Of Health Research (EQUATOR) Network Statements
CONSORT Statement 47	2025	Reporting clinical trial results	Reference during trial design. A completed checklist is often required with the publication of trial results.
CONSORT Extensions	Various dates. Extensions were not updated with the 2025 guidelines, but future updates are planned.	Provide specific guidance on reporting clinical trial results with consideration for study design and key topics	Reference extensions relevant to your trial design. A few specific examples are highlighted below. Additional topics include:  - By study design: adaptive (ACE statement), cluster randomized, stepped-wedge cluster randomized, pragmatic, pilot and feasibility, non-inferiority, factorial  - By topic: equity, conference abstract reporting, non-pharmacologic interventions, patient reported outcomes
CONSORT Pragmatic Trials19	2008	Reporting pragmatic trial results	Clarifies CONSORT checklist items that merit special consideration in pragmatic trials
CONSORT-ROUTINE20	2021	Reporting results of trials using routinely collected clinical data	Addresses the unique challenges of using EHR data in clinical trials. Serves as an important reference for EHR-embedded clinical trials
CONSORT-Outcomes48	2022	Reporting trial outcomes	Provides guidance for selecting and measuring outcomes, including topics relevant to pragmatic trials (e.g., handling missingness, multiplicity in analysis)
SPIRIT Statement 13	2025	Reporting a clinical trial protocol	Reference during clinical trial design and protocol development. A completed checklist is often required with the publication of a trial protocol paper.
SPIRIT Extensions	Various dates	Provide specific guidance for reporting protocol elements by study design and topic	Topics include: outcomes, patient-reported outcomes, artificial intelligence interventions, and factorial designs
Template for Intervention Description and Replication (TIDieR)49	2014	Reporting a clinical trial intervention	Checklist that ensures interventions are adequately described to allow reproducibility.
Other relevant policies and guidelines
The National Institutes of Health Pragmatic Trials Collaboratory50	Updated as needed	Reporting pragmatic trial protocols	Website provides useful guidance and a helpful template for reporting pragmatic trial protocols.
Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2)21	2015 (PRECIS-2)21, 2009 (PRECIS)22	Trial classification	Tool used to determine whether a trial should be classified as explanatory or pragmatic. Pragmatic trials should register in the PRECIS-2 database and should make the PRECIS-2 wheel publicly available, e.g., with protocol publication.
International Committee of Medical Journal Editors (ICMJE) policy on clinical trial registration16	2005	Trial registration	Participating journals follow this policy and require the registration of clinical trials in accepted registries, such as ClinicalTrials.gov.
World Health Organization (WHO) Trial Registration data set15	Versions updated as needed	Trial registration	Details information that should be included in a clinical trial registration. Using this data set is required in the SPIRIT checklist (item 1b); the data set should be referenced during protocol development.
Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN)51	2020	Subgroup analysis	Tool to assess the credibility of effect modification in subgroup analyses.
Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement52	2020	Risk-based heterogeneity of treatment effect (HTE) analysis	Provides guidance on performing analyses for risk-based HTE in clinical trials.

The SPIRIT guidelines include a 34-item checklist addressing core protocol elements that should be included in trial protocols and protocol papers (Table 1). The 2025 update provides an additional checklist item focused on patient and public involvement in trial development. Protocol paper authors should provide a completed SPIRIT checklist detailing the location of information for each item in the manuscript. Reviewers and journal editors should ensure the protocol paper adheres to both the letter and intent of the guidelines.

In addition to following reporting guidelines, clinical trials must be publicly registered per both national regulations and international standards.^15,16 Registering a trial before enrollment facilitates harmonization and collaboration between researchers, reduces redundancy across contemporaneous trials, and publicly records the trial plan. Protocol papers should report trial registration information, and reviewers and journal editors should compare the protocol paper to the trial’s registry entry, flagging any discrepancies for explanation and correction.

Beyond guidelines—other key elements of protocol papers

Even with perfect adherence, the SPIRIT checklist provides only a minimum standard for protocol reporting.¹³ Unfortunately, incomplete reporting is common, with studies finding that on average, published protocol papers include only 75% of SPIRIT checklist items.^17,18 Furthermore, SPIRIT and CONSORT guidelines may miss items important for particular trial designs or disciplines. While the CONSORT and SPIRIT “extensions” provide focused guidance on some topics (Table 2), coverage is incomplete. For example, SPIRIT lacks an extension for pragmatic trials, so authors must read the corresponding CONSORT pragmatic trial extension.¹⁹ Reference to other extensions — such as CONSORT-ROUTINE for using routine clinical data in trials — may provide additional help to authors and readers, but formal completion of multiple checklists is burdensome and difficult for journals to monitor.²⁰

Therefore, to assist critical care researchers and clinicians, we highlight key issues relevant to critical care protocol papers and provide a list of questions readers and reviewers should ask when reading a protocol paper (Box 2).

Box 2. Questions to ask when writing, reading, and reviewing a critical care protocol paper.

Area of focus	Writing	Reading	Reviewing
Trial design	• What is the trial design? What advantages does this design offer? What threats to validity does it introduce and how were these mitigated? • What aspects of the trial design prevent the introduction of bias? In particular, consider enrollment, allocation concealment, protocol adherence, and data collection. • If the trial is pragmatic, why was this design chosen? • Are there trial design decisions that require specific justification?	• What is the research question? • If the trial is performed as described, will it answer the proposed research question? • Will the results apply to patients in my practice? • Does the paper address other key elements as indicated based on the research question and trial design?  ○ Do I understand where this trial will be carried out and what patients it will enroll?  ○ Do I understand the procedures for enrollment, randomization, and the intervention? Are adequate protections in place to ensure fidelity to the planned procedures?  ○ Do I understand how the data, particularly outcomes, will be collected and defined? • Do I see any threats to the validity of the trial based on the planned trial design?	• What is the research question? • What is the trial design? Is this the right design for the proposed research question? • Are key study design decisions affecting generalizability and validity adequately justified? • Is the trial setting adequately explained to understand implications related to generalizability? • Are the eligibility criteria, intervention, and outcomes specified with sufficient detail to allow independent replication? • Are pragmatic trial components, or lack thereof, (e.g. effectiveness versus efficacy design) adequately justified?
Adequacy of reporting	• Have I followed SPIRIT13 and other relevant reporting guidelines? • Have I described the protocol in adequate detail that it can be understood by clinical readers and replicated by other researchers? • Have I reported all financial support and conflicts of interest? If there are relevant conflicts of interest, have I described how these were mitigated?	• Does the paper follow SPIRIT13 and other relevant reporting guidelines? • Are the components of the protocol described in enough detail that I could, with the right resources, replicate the trial in a similar setting?	• Does the manuscript adhere to the SPIRIT guidelines13 and applicable extensions (or related CONSORT guidelines/extensions)? • Do authors report funding and conflicts of interests and are these adequately addressed to prevent bias in the design?
Quality	• Has the trial ended? Protocol papers should be published before the trial is complete to ensure full transparency. • Is the trial properly registered through an accepted registration site? • Have I reported all protocol amendments?	• Is the trial properly registered? Does the registration match the protocol paper? • Were any changes made to the protocol after the start of the trial that could introduce new threats to trial validity?	• Is the trial properly registered? Does the registration match the protocol paper? • Were any changes in the protocol explained and sufficiently justified? Is the protocol aligned with its trial registry entry?

Trial design

Explanatory and pragmatic trials exist on a continuum, and opinions vary about how much emphasis protocol papers should place on this component of trial design. In fact, the authors of the 2008 CONSORT extension on pragmatic trials intentionally did not recommend that researchers specify the trial type in the protocol title.¹⁹ However, given a trial’s classification on the continuum of explanatory to pragmatic is relevant to which reporting guidelines apply and what items should be included in the protocol, we suggest that researchers clearly specify the intended design in the protocol paper and justify the design choice as it relates to answering the research question. The PReCIS wheel, which was updated in 2015, now offers an accessible and moderately objective method for placing trials on the continuum of explanatory to pragmatic and can be a useful tool to assist authors in trial classification.^21,22 For pragmatic trials, rationale for the pragmatism (or lack thereof) for each design element is ultimately more important than nomenclature, and protocol papers offer a key opportunity for authors to explain and justify the use of pragmatic elements in their trial.

In critical care, it can be helpful for authors to further specify where a pragmatic trial lies on the spectrum of electronically-embedded to practice-embedded, as these designs face different threats to their validity that must be addressed in the protocol. While there is no formal classification scheme for these designations, pragmatic critical care trials can be classified as electronically versus practice-embedded based on how they perform: 1) screening/enrollment, 2) randomization, 3) intervention, and 4) data collection (Figure 1). Electronically-embedded trials use tools embedded in the electronic health record (EHR), such as best practice advisories and order sets, to conduct the trial. For example, the ACORN trial used the EHR to re-direct clinicians placing an antibiotic order to a study order set that randomized patients to piperacillin-tazobactam versus cefepime.⁸ Embedding a trial in the EHR increases efficiency but risks potentially inaccurate data, requiring rigorous approaches to variable specification and data collection. Practice-embedded trials rely on bedside teams to conduct the trial. For example, in the RSI trial, clinical teams randomized patients to etomidate versus ketamine for intubation in real-time using opaque, numbered envelopes at the bedside.⁷ This allows efficient enrollment of broad patient populations but can risk selection bias and protocol non-adherence if clinical staff are not well-trained.

Figure 1.

Continuum of pragmatic trial designs in critical care.

Pragmatic trials in critical care range from being fully embedded in the electronic health record (EHR) to being embedded in routine practice at the bedside. This figure explains how these different types of trials approach screening, randomization, intervention delivery, and data collection using examples. The colored text represents examples from published critical care trials. Orange text provides an example of an EHR-embedded trial, the Effect of Antibiotic Choice on Renal Outcomes (ACORN).⁸ Blue text provides an example of a practice-embedded trial, the Ketamine or Etomidate for Tracheal Intubation of Critically Ill Adults (RSI).⁷

Trial setting

Patient populations and care systems vary widely between intensive care units (ICUs).^23–26 This can lead to the inclusion of very different patients even within trials focused on the same conditions. For example, the CLASSIC and CLOVERS trials both evaluated sepsis fluid resuscitation strategies and are often presented together when discussing evidence-based fluid management.^27,28 However, they enrolled vastly different patients: patients in CLASSIC were enrolled in the ICU and had a mortality rate of over 40%; patients in CLOVERS were enrolled earlier and had a mortality rate of only 14%. This wide variation highlights the need to include a detailed description of the trial setting in the protocol paper, including hospital type (e.g., academic versus community), unit specialization and patient populations (e.g., medical versus mixed ICUs), and typical clinicians and clinical team structure (e.g., teaching staff, attendings). Understanding the care context in which a trial intervention is deployed is particularly important for pragmatic trials. Protocol papers for these trials may need to provide additional context about routine clinical care, i.e., what defines the “real world” for which the trial is providing evidence. For example, the protocol for the PILOT trial of oxygen targets includes copies of the hospital’s clinical procedures for ventilator management to illustrate how oxygen targets were typically set by respiratory therapy and how this process was modified for the trial.²⁹

Eligibility Criteria

Protocol papers provide a key opportunity for authors to define and justify eligibility criteria so readers can understand the applicability of results to practice. Given the heterogeneity of critical illness, critical care researchers should specify with particular care how enrollment criteria related to syndromes (e.g., sepsis or ARDS) or illness severity will be defined and operationalized.³³ Critical care trials often target rapidly evolving critical illnesses where the time window for enrollment can be narrow and differences in care processes for ICU admission or practice variation in testing and treatment can alter the apparent eligibility for otherwise identical patients. For example, using ICU admission as the criterion for study entry may create a diffuse trial population, given the bar for ICU admission varies widely across hospitals.^23,24 To address this problem, the ARDS, Pneumonia, and Sepsis (APS) Consortium, a prospective cohort study, is enrolling patients based on clinical criteria that are agnostic to patient location, such as receipt of vasopressors or non-invasive respiratory support (NCT06521502). While vasopressor initiation and oxygen practices also vary across hospitals and providers, these criteria help move from a broad trial population towards a more uniform group of patients with true cardiovascular and pulmonary organ dysfunction.

Enrollment and randomization

Given that mental status alteration is a common complication of critical illness and side effect of its treatments, feasibility and generalizability for most critical care trials depends on application of methods to enroll subjects who lack capacity to provide informed consent. Pragmatic critical care trials have evaluated urgent treatments in the acute setting under the U.S. concept of waiver of informed consent.³⁴ In the protocol paper, authors should discuss their approach to enrollment, including the consent process and rationale for any use of waiver of consent, alteration of consent, or exception from informed consent, keeping in mind that regulations regarding consent waivers vary across institutions and countries. This introduces challenges for multicenter international studies, as exemplified in the trial by Casamento et al comparing morphine and fentanyl for analgosedation, which used a waiver in one center but required opt-out in the other.³⁵

The protocol paper should also describe procedures for monitoring enrollment given the systematic exclusion of certain patients during trial enrollment threatens external validity. In explanatory trials, exclusions are often tracked as part of screening. In EHR-embedded trials, exclusions can be tracked electronically. The ACORN trial, for instance, presented clinicians with an electronic alert for every eligible patient and recorded the reason why clinicians opted for antibiotic selection outside of the trial when this occurred, allowing real-time tracking of enrollment.⁸ Exclusions may be more challenging to monitor in practice-embedded pragmatic trials where clinicians are enrolling patients. Protocol papers for such trials should therefore detail specific enrollment monitoring plans. This is particularly important because while having clinical teams enroll and randomize patients greatly increases efficiency, clinical staff may not have the same level of experience with clinical trial operations as research staff, which can also lead to selection bias. In a classic example, in the PHANTAsi trial of pre-hospital antibiotics, selection bias was introduced because, despite training, ambulance personnel were opening multiple randomization envelopes to be able to administer antibiotics.³⁶ Protocols for practice-embedded trials should describe how the trial will ensure allocation concealment, including who will carry out randomization and what training they will be provided about study procedures.

Pragmatic trials also often employ cluster randomization, where treatments are assigned at the unit or hospital, rather than individual patient, level. For example, the Isotonic Solutions and Major Adverse Renal Events Trial (SMART) randomized ICUs to stock balanced crystalloid versus normal saline, alternating by month.¹⁰ Such designs can improve feasibility, potentially prevent contamination by physically or temporally separating treatment assignments (e.g., stocking one fluid at a time in SMART)³⁰, and facilitate large sample sizes. Cluster randomization is also necessary for analyzing patients exposed to interventions targeting clinicians, clinics, units, or hospitals. However, unless addressed carefully in the study design and analysis plan, cluster randomization can introduce selection bias given clinicians know the treatment assignment and thus may consciously or subconsciously enroll, manage, or document differently for patients on the basis of treatment assignment. Therefore, protocol papers for trials using cluster randomization, including stepped-wedge trials, should detail approaches to mitigating selection bias, for example, employing objective eligibility criteria and/or using intention-to-treat analyses that include all patients in the study units regardless of whether they received treatment.^31,32 Substantial within-cluster outcome correlation can also decrease statistical power in cluster randomized trials. Protocol papers for such trials should account for clustering in power calculations and also report observed within-cluster correlation (e.g., intracluster correlation coefficient [ICC]) to facilitate design of future trials.^31,32

Protocol adherence

As in other disciplines, critical care trial protocol papers allow researchers to detail plans for monitoring adherence to protocol-specified treatments and evaluations. This is particularly important for pragmatic trials, where interventions and assessments are performed within routine clinical care, which increases the risk of contamination between arms (two arms are treated similarly) and learning effects due to shared clinical staff.^30,37 Pragmatic trial protocol papers should provide details about the methods used to minimize contamination and should include a discussion of the rationale and approach to real-time protocol adherence monitoring. For example, in EHR-embedded trials like ACORN, adherence can be monitored through dashboards that track clinician interaction with study order sets, allowing real-time feedback and re-education of clinical teams. Research teams can also monitor adherence in practice-embedded trials and use personalized communications with clinicians to minimize cross-over and contamination.

The real-world setting of pragmatic trials introduces additional challenges that should be addressed in the protocol paper. For example, protocol papers should explain the rationale for not blinding when blinding is deemed to be infeasible, which is common in pragmatic trials evaluating interventions that represent routine treatments (e.g., antibiotics).³⁸ Furthermore, given the focus of pragmatic trials on evaluating interventions in real-world settings, such trials often avoid protocolizing co-interventions (e.g., fluid resuscitation in a trial of vasopressors). However, co-interventions evolve over time and are subject to secular trends, so protocol papers need to specify how pertinent co-interventions will be monitored, to contextualize the interpretation of the trial results.

Data, data sources, and outcomes

While details of the SAP are outside the scope of this manuscript, the protocol paper should include a summary of the proposed statistical methods, including data collection, to highlight key analytic plans for the reader. Defining data sources and variables is particularly important for pragmatic trials that rely on EHR data. Traditionally, trial staff abstract data for clinical trials from the medical record using structured case report forms (CRFs), which are developed before the trial and define what data will be collected. In contrast, the EHR contains an immense amount of data that can be collected at any time, which may make planning data collection seem less important than in explanatory trials. However, inaccurate information in EHR datasets is common and can affect the interpretation of trial results. Therefore, researchers should put careful thought into EHR-based data collection plans and mirror the process used for CRF development, including pre-specifying data sources and variable definitions in the protocol paper. Protocol papers should specifically describe how EHR data will be abstracted, for example via automated queries, manual chart review, or a combination, and should outline plans for data validation, for example targeted manual adjudication of key outcomes.^6,10 While blinding of clinicians and patients is often infeasible in pragmatic trials, outcome assessors can be blinded to minimize bias.³⁸

Pragmatic trials that use routine clinical data also present unique challenges for outcome collection and safety monitoring given they depend on clinical teams to record events.^20,37 For example, in the VASSPR trial comparing low versus high thresholds for adding vasopressin in septic shock, vasopressor extravasation is a key safety outcome.⁶ However, clinical teams may differentially document extravasation between arms (e.g., clinicians may have more concern about extravasation in the low-threshold arm and monitor these patients more closely), or fail to document extravasations completely, missing potential adverse events. Researchers should select objective primary outcomes that are less subject to bias and include a detailed plan for safety monitoring.³⁷ Protocol papers should also pre-specify how the analysis will handle missing data, which may be more common when using routine clinical data.²⁰

Outcomes

While all variables collected during a trial should be pre-specified and well-defined, authors of the protocol paper should take particular care to specify and precisely define the trial outcomes. Even for critical care outcomes that appear straightforward and objective, different measurement methods may substantially alter interpretation. Capturing accurate post-hospital mortality, for instance, requires careful patient follow-up or linkage to other resources outside the hospital EHR, such as state and national registries.³⁹ Other common outcomes in critical care trials are even more complex. Composite outcomes such as ventilator-free days have been variably defined in the literature.⁴⁰ Intermediate outcomes like change in severity of illness scores are subject to measurement errors and missing data, requiring rigorous approaches to standardized data collection.^41,42 Furthermore, given critically ill patients experience high post-ICU morbidity and mortality^43,44, trials should justify their follow-up period and ensure that it is sufficient to capture the time period during which treatment effects are most likely to occur.

Pre-specifying outcomes in the protocol paper also helps prevent outcome switching, the change in outcomes from what was prespecified in the protocol. Outcome switching poses a major threat to trial validity by potentially “cherry-picking” outcomes post-hoc and overestimating the benefits of a treatment. Yet, outcome switching remains extremely common. One group found that only 9/67 (13.4%) trials reported outcomes that matched the outcomes specified in the protocol—the rest of the trials both added new outcomes and/or did not report on pre-specified outcomes.⁴⁵ Better outcome specification can help avoid such pitfalls.

Protocol amendments

While strict protocol stability is ideal, authors often have good reasons to amend their protocol after study initiation. For example, protocol amendments to address low enrollment can increase screening volume and improve a trial’s performance.⁴⁶ However, protocol changes also have the potential to introduce bias and cloud interpretation of results. In the satirical PARACHUTE example, the researchers addressed low recruitment by making a protocol amendment to lower the height of the jump to ground level. Clearly, this modification improved recruitment but at the cost of fundamentally changing the trial question. Whenever protocol amendments are made, the change, along with a rationale, should be clearly and transparently reported in the protocol paper (if the change was made before publication) and in the final trial paper.³ Readers should closely scrutinize all protocol amendments, especially in eligibility criteria, randomization, outcomes, and data collection, where there is high risk that changes may impact trial integrity.

Beyond the protocol: Ancillary documents

Many documents are used in trial operations beyond the protocol, including the SAP, CRFs, manual of operations, staff training documents, and DSMB charters. While publication of such ancillary documents is not typically required by guidelines or journals, sharing these materials as supplements to the protocol paper or via public data repositories improves transparency, aids interpretation of the trial, and facilitates secondary uses of collected data.

Conclusions

Protocol papers provide researchers with an opportunity to transparently pre-specify, describe, and justify key design decisions and allow readers to rigorously and prospectively evaluate a trial.