Workflows to automate covariate-adaptive randomization in REDCap via data entry triggers

Abstract

Objective

Covariate-adaptive randomization algorithms (CARAs) can reduce covariate imbalance in randomized controlled trials (RCTs), but a lack of integration into Research Electronic Data Capture (REDCap) has limited their use. We developed a software pipeline to seamlessly integrate CARAs into REDCap as part of the all2GETHER study, a 2-armed RCT concerning HIV prevention.

Materials and Methods

Leveraging REDCap’s Data Entry Trigger and a separate server, we implemented software in PHP and R to automate randomizations for all2GETHER. Randomizations were triggered by saving a specific REDCap form and were automatically communicated to unblinded study personnel.

Results

Study arms were highly comparable, with differences across covariates characterized by Cohen’s d = 0.003 for continuous variables and risk differences <2.4% for categorical/binary variables.

Conclusions

Our pipeline proved effective at reducing covariate imbalance with minimal additional effort for study personnel.

Discussion

This pipeline is reproducible and could be used by other RCTs that collect data via REDCap.

Introduction

Comparability of study arms in randomized controlled trials (RCTs) is crucial for interpreting trial results.^1–4 However, accidental bias, which occurs when study arms markedly differ on important pre-randomization covariates (referred to as covariate imbalance) is a well-documented limitation of simple randomization.^1,5 Covariate-adaptive randomization algorithms (CARAs) reduce the likelihood of accidental bias by considering current covariate imbalance (ie, on already-randomized participants) and new participants’ covariates when randomizing.^6,7 Most modern applications involve randomizing to the study arm that would reduce covariate imbalance according to pre-specified balance criteria with a greater than 50% probability.^8–10 Approaches to randomization, such as the minimization^6,7 or minimal sufficient balance (MSB)⁸ algorithms, can reduce covariate imbalance and increase the power of analyses.^11,12

Despite the advantages, the use of CARAs has lagged. Reviews of published RCTs found that 11%-12% of RCTs used such algorithms, while nearly 70% of trials report using stratified randomization, which randomizes participants within subgroups delineated by pre-selected covariates.¹³ Compared to stratified randomization, CARAs have the capacity to handle a greater number of covariates and potential to induce stronger covariate balance.^8,14,15 Among the barriers to their broader use are their complexity and limited software options.^5,16 Using CARAs requires careful consideration about how covariate data validation and randomization fit into study workflows. Moreover, Research Electronic Data Capture (REDCap),^17,18 which has become the de facto data capture platform for many academic research centers, did not offer modules for CARAs until version 14.7. Researchers using CARAs often must write their own study-specific code files and ensure study personnel execute those scripts with every new randomization.

In this article, we describe automation of a specific CARA, the MSB algorithm, for use with REDCap as part of an RCT, the all2GETHER trial, which recently concluded recruitment.¹⁹ In the following sections, we outline the platform built for all2GETHER, describe interim results, and consider how other trials may leverage this framework.

Methods

all2GETHER is a relationship education and HIV prevention program designed for gay, bisexual, queer, and transgender and non-binary people who have sex with cisgender men.^20,21 In its current iteration, all2GETHER consists of a web app with modules containing videos and interactive activities. For partnered dyads, it also includes dyadic relationship skills coaching sessions covering topics like conflict resolution and relationship agreements.

We designed a 2-armed RCT to compare the all2GETHER intervention with a control condition in which participants are provided with HIV prevention resources from the Centers for Disease Control and Prevention (CDC).¹⁹ The trial aimed to recruit N = 5000 participants to ensure 80% power to detect a 36% risk reduction in HIV incidence (primary endpoint). Participants in the trial were 16-34 years-old and confirmed HIV-negative; they were able to engage in the trial on their own (ie, if they were single or their partner does not wish to participate) or as part of a partnered dyad. Single individuals and dyads were the unit of randomization, so both partners of a dyad were randomized to the same study arm and attended coaching sessions together. Only the prinicpal invesigator (PI) was blinded to random assignment. The protocol described applying MSB to participation status (single, partnered but participating alone, dyad), age, race/ethnicity, gender identity, and PrEP usage at baseline. Test results for rectal and urethral sexually transmitted infections at baseline were added to the algorithm in June 2023 in response to emerging imbalance.

The trial used Northwestern University’s instance of REDCap (v14.5) for data collection and storage, which did not support MSB randomization at the time of study launch (2022). To integrate MSB into study workflows at scale (N = 5000 target recruitment), we constructed an automated process that leverages REDCap’s Data Entry Trigger (DET) and application programming interface (API). This process runs automatically simply by saving a specific “Randomization” form in the REDCap project and returns a random assignment to unblinded study staff in seconds. All relevant code is available in a GitHub repository²² and additional technical details are outlined in Appendix A1.

We map the overall framework in Figure 1, which has the following core elements: (1) REDCap project containing a dedicated form indicating whether a participant should be randomized, (2) DET, (3) secure server on which to execute MSB, (4) software that implements MSB, and (5) diagnostics for quality assurance and failsafe contingencies. Working across these core elements are a series of processes (A-E) described in detail below. Critically, these processes involve verifying that randomization is required, automatically executing MSB if so, and documenting the results.

Figure 1.

Automated Covariate-Adaptive Randomization with REDCap Data Entry Triggers.

Automated workflow components

1. REDCap project with “Randomization” form. Participant covariate information is collected prior to randomization and stored in the all2GETHER REDCap database. In addition, the database includes a form called “Randomization” (Figure 1) containing an item that indicates whether a participant is ready to be randomized: the randomization indicator. Covariate information can be collected in various REDCap forms, but we found that collating covariates in the Randomization form via calculated fields was a useful strategy. This allowed study staff to validate covariate information prior to randomization.

2. Data-Entry Trigger. The DET is a REDCap feature that allows users to specify a server that will be notified whenever data are entered or changed in a REDCap project. When a project record is modified, either by adding new data or modifying existing data, the DET sends an HTTP POST request with key-value pairs as part of the query string (QS) to that server containing information including the project ID, record ID, and instrument modified.

3. Secure server. To enable the processing of HTTP requests, we used a virtual machine running Apache web server (v2.4.37) configured for TLS1.2 and PHP-FPM (PHP FastCGI Process Manager). The server undergoes regular patching to maintain updates and security.

4. MSB software implementation. Our server contains software to listen for HTTP POST requests and parse the QS (Process B), format REDCap data pulled to the server via API (Process C), run the MSB algorithm (Process D), and return relevant information (including the new randomization) to REDCap (Process E). To “listen” for HTTP POST requests, our server contains a PHP script (v8.2.13) that parses the QS from the DET. The remainder of the processes are coded in the R programming language (v4.4.1). Each of these scripts automatically trigger additional steps in the randomization process. For instance, if the QS indicates a randomization is potentially required, the PHP script automatically calls R scripts to do so. Because all steps of MSB execution occur remotely and automatically on a secure server, they can be blinded to relevant study personnel.

5. Diagnostics and contingencies. Randomization success or failure is documented on logs kept on the server, along with reasons for failure (including software or data entry errors). Attributes of randomization, including the covariate imbalances identified by the MSB algorithm, are documented both on the server and in REDCap fields, which can be summarized via REDCap Report Dashboards. To handle server outages or other communication disruptions, a static randomization list was pre-generated as a back-up and is accessible to unblinded study personnel; in the event of an outage, a random allocation can be identified from this list and this allocation will inform future runs of the algorithm when the server is back online.

Automated workflow processes

1. DET sends a Query String as POST request to secure server. When data are saved or modified in REDCap, the DET sends a QS to the designated server. This QS contains information about the REDCap project ID, record ID, and instrument modified.

2. Software on server parses QS. The PHP script identifies if the HTTP POST request comes from the correct REDCap project and if the “Randomization” form was modified, indicating randomization may be needed. In such cases, this software automatically triggers relevant actions to execute MSB (C-E below).

3. Software on server processes REDCap data. If the DET QS indicates a participant is potentially ready for randomization (Process B), this can be verified by assessing extant REDCap data. An R script extracts data to the server (using REDCap API) and identifies if the record identified in the QS has yet to be randomized and the randomization indicator (Component 1) signifies they are ready to be randomized. Because MSB, like all CARAs, requires covariate data on both already-randomized and yet-to-be-randomized participants, the code identifies already-randomized participants as those with existing study arm allocations in REDCap and extracts and formats their covariate information. With these data, the R script then initiates randomization (Process D).

4. MSB software parses REDCap data and generates allocation. The R software pipeline extracts covariate data on participants identified in Processes (B-C) who either are ready to be or have already been randomized. These data are fed into the MSB algorithm, which returns the new study allocation, randomization probability, and balance criteria that informed the randomization.

5. New random allocation passed to REDCap. The output in Process (D), for which there exist fields within REDCap, is sent back into REDCap via the API. REDCap User Rights and Data Access Groups (DAGs) are used to ensure proper blinding. A REDCap Report dashboard is used to track allocations, probabilities, and balance criteria.

Results

Enrollment for the trial closed in March 2025 after randomizing 2086 participants (1088 to all2GETHER, 998 to control). Automated randomization occurred with minimal server interruptions; the back-up randomization list was consulted for 18 allocations. All other 2068 randomizations were run when study staff saved the Randomization form, with each randomization taking only seconds. Statistical theory^1,2 and research guidance^23–26 suggest that threats to validity are minimal when covariate imbalance is no worse than a standardized difference between arms of d = 0.1 or a risk difference of 10%. Standardized differences²⁴ for covariates in all2GETHER are reported in Table 1, with the largest being d = 0.052 for PREP use (absolute risk difference = 2.2%).

Table 1.

Standardized differences for covariates used in the all2GETHER trial.

Covariate	Status	Limited participation	Race/ethnicity	GNC/trans	PrEP use	STIs	Age
Standardized difference	0.005	0.020	0.011	0.019	0.052	0.049	0.003

Abbreviations: GNC, gender nonconforming; PrEP, pre-exposure prophylaxis; STI, sexually transmitted infection.

Discussion

While developing the automated pipeline for all2GETHER required collaborative frontend investment, it has proven robust and effective, eliminating delays and reducing the operational complexity associated with manual algorithm execution. Success hinged on key communication among team members, including about server patching, potential outages, and database updates. Critically, as the pipeline involves pulling and formatting covariate information from REDCap, changes relevant REDCap fields (eg, adding a new racial or gender category) must be reflected in server-side software. However, statistical guidance on CARAs underscores the purposeful selection of covariates that are highly prognostic of outcomes^5,11,16,27 and such variables have seldom been the subject of revision in REDCap for all2GETHER.

We describe the workflow here and provide code and instructions in a related GitHub repository²² with the hope of reducing barriers to using CARAs in RCTs in settings using REDCap. Figure 1 can be considered a general strategy; alternate CARAs can be implemented (eg, minimization) within this framework, and the code can be modified to include participant covariates relevant for any given study and the processes and components described above can be tailored to fit research objectives and day-to-day demands. Because the entire pipeline is designed to occur automatically on a secure server, it is possible to account for various blinding criteria within this framework. Though we attempt to provide suggestions on the structure of these core elements, we note that their actual implementation will almost certainly vary across research efforts. This pipeline is currently in use for the Effectiveness of Two Aspirin Doses for Prevention of Hypertensive Disorders of Pregnancy: ASPIRIN TRIAL (clinicaltrials.gov ID: NCT06468202), while elements of it are underpinning the randomization process of the Liver Cirrhosis Network Rosuvastatin Efficacy and Safety for Cirrhosis in the United States (LCN RESCU) trial (NCT05832229).

Though not available at all institutions (including Northwestern University as of this writing), the latest instance of REDCap includes advanced randomization features that can incorporate CARAs without the need for external servers.²⁸ Although specific implementations, including of MSB, are yet to be completed, we note that this increased flexibility would greatly simplify this framework, potentially eliminating the need for the DET or even a separate secure server. Our overall pipeline remains an option for researchers without access to the newest instance, while our implementation of MSB in the R programming language may help facilitate implementation within the latest randomization module.

Author contributions

Jacob M. Schauer (Conceptualization, Methodology, Software, Validation, Writing—original draft, Writing—review & editing), Marc O. Broxton (Methodology, Project administration, Software, Supervision, Writing—original draft, Writing—review & editing), Luke V. Rasmussen (Investigation, Methodology, Writing—original draft, Writing—review & editing), Gregory Swann (Conceptualization, Formal analysis, Project administration, Validation, Visualization, Writing—original draft), Michael E. Newcomb (Conceptualization, Funding acquisition, Methodology, Resources, Supervision), and Jody D. Ciolino (Conceptualization, Methodology, Supervision, Writing—original draft, Writing—review & editing)

Supplementary material

Supplementary material is available at JAMIA Open online.

Funding

This work was supported by 2 grants from the US National Institutes of Health, including the support for the all2GETHER trial from the National Institute of Allergy and Infectious Diseases (grant no. U01 AI156874-01) and by the National Center for Advancing Translational Sciences (grant no. UM1TR005121).

Conflicts of interest

We do not have any competing interests to disclose.

Data availability

Data are available upon request. Software described in this article are available at the linked GitHub repository.