Bayesian enrollment modeling for several emergency medicine clinical trials

Abstract

Background

Accrual of participants into clinical trials is a fundamental and important aspect for the management of trial progress. Monitoring of trial accrual often provides insight into the potential timing of key study events, such as interim analysis, as well as the feasibility of the overall trial to enroll the projected sample size in the original estimated timeline.

Methods

A Bayesian first order simple dynamic linear model with weakly informative priors is utilized to characterize enrollment rates temporally within pre-defined time windows (quarterly) for the duration of a trial.

Results

Application of the model to three ongoing clinical trials demonstrates the utility of the model to characterize the observed accrual patterns. Additionally, the applications demonstrate the flexibility of the model to react to variable accrual patterns without overreacting to the variability of accrual within a trial due to expected causes, such as seasonal variability in disease incidence, or unexpected causes, such as a global pandemic.

Conclusions

Much statistical literature has been dedicated to predicting when key study events are likely to occur by utilizing current estimated rates of participant accrual; however, study teams, sponsors, and funding agencies have interest in the previous trends in participant accrual. This work presents an addition to the literature which allows parties interested in assessing trial progress to do so by providing a flexible framework for the standardized characterization of trial accrual which is not overly sensitive to the expected variability of trial accrual.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-025-08748-3.

Introduction

For multicenter clinical trials, monitoring of participant accrual is a fundamental and important aspect of trial management in order to help assure that the trial is progressing adequately. Trial accrual is often assumed to be constant. However, this assumption is often invalid due to pre-specified trial start-up procedures or unplanned delays, gradual or rolling site initiations, unanticipated site attrition, enrollment fatigue, or (as recently experienced) a global pandemic [1]. Major trial events, such as interim or final analyses, are often set to occur at enrollment or outcome-based milestones. Incorrect or invalid assumptions regarding trial accrual can be a costly mistake, as this can result in clinical trials being shut-down prematurely or requiring additional unanticipated funding [2]. If a trial ceases enrollment and proceeds to the final analysis with a smaller sample size than intended, both the clinical and statistical significance of the potential findings may suffer [3].

Literature regarding clinical trial accrual rates is abundant, with novel methods focusing on the characterization and prediction of overall and site level accruals [1, 3–10]. Two common methods for characterizing clinical trial accrual are Brownian motion and Poisson processes [1, 4]. These stochastic approaches provide a natural methodologic approach to the modeling of enrollment data, which naturally fluctuates due to the variability associated with trial enrollments. Furthermore, the characterization of current accrual and predictions of future accrual utilizing these methodologies allow for better trial management and decision making when facing a trial which suffers from poor enrollment [1].

While much of the literature is dedicated to the prediction of accrual at a later time, many sponsors and funding agencies are also interested in understanding accrual patterns previously observed. Providing a characterization of the observed accrual rates, one which is not overly sensitive to the random variability, would provide an indication of temporal trends and other potential non-random drivers impacting the enrollment rate.

As a motivation for this work, we focus on the Strategies to Innovate EmeRgENcy care (SIREN) clinical trials network. SIREN is a collaborative effort, funded by two institutes of the National Institutes of Health (NIH), focused on improving outcomes for patients with neurologic, cardiac, respiratory, and hematologic emergencies. Relevant stakeholders, including the funding agencies and the data and safety monitoring board, routinely monitor trial progress. A standardized yet flexible approach for the characterization of trial progression would aid these stakeholders in decision making. In this work, we propose using a first order normal dynamic linear model (NDLM) to obtain smoothed estimates of clinical trial accrual. In the context of accrual rates, the NDLM allows for non-monotonic relationships between the estimated accruals in a predefined time period, with the estimated accrual for a given period depending on the accrual observed during the previous period [11]. In the following sections, we will provide the details of our model and illustrate the application of the proposed model for three ongoing clinical trials in SIREN.

Methods

We propose the utilization of a first order simple dynamic linear model to characterize enrollment rates for pre-defined time periods, e.g., quarters, over the duration of a trial. The proposed approach below would allow for the estimation of smoothed accrual rates under a Bayesian framework as well as the quantification of ad-hoc quantities of interest, such as the likelihood of a meaningful change in the accrual rates, beyond that expected from normal variability.

Without loss of generality, let $y_j$ represent the number of observed enrollments in quarter j for $j=1,...,J$. For an ongoing trial, J represents the current quarter during which enrollment is occurring. We assume that the enrollments in each quarter follow a Poisson distribution; that is,

$$\begin{array}{c}\hfill y_j\sim \text{Poisson}({\lambda }_j\times T_j)\end{array}$$

where ${\lambda }_j\times T_j$ is the mean of the Poisson. The $T_j$ is the observed proportion of the quarter, j, which has elapsed; thus, ${\lambda }_j$ represents the enrollment per quarter. Using this construction, $T_j$ is an observed value for each quarter, and ${\lambda }_j$ requires the specification of a prior distribution. For all time periods which have been completed $T_j=1$; however, for the current time period, $T_J<1$. This observed value is used to appropriately reflect the number of enrollments in that time period.

The prior specified for the first enrollment period, since no previous time points exist to provide information, is ${\lambda }_1\sim \text{Gamma}(1.384,0.0692)$. This prior was chosen as it provides a prior mean for the mean quarterly accrual at 20 participants with a standard deviation of 17 participants and can be modified to match the expected quarterly accrual rate for any given trial.

For all other time periods, the enrollment rate for the current period will rely upon the previous period’s enrollment. Letting $N(\mu ,P)$ represent the normal distribution with parameters mean and precision, we assume that

$$\begin{array}{c}\hfill log({\lambda }_j)\sim N(log({\lambda }_{j-1}),{\tau }^{-2})\end{array}$$

where

$$\begin{array}{c}\hfill \tau \sim Unif(0,3)\end{array}$$

and ${\lambda }_{j-1}$ is the enrollment rate for the previous time period.

Given this prior and likelihood combination, we can express the posterior distributions of all model parameters relying on the accrual data observed. We show this by displaying the complete conditionals. The complete conditional for $j=1$ is:

$$\begin{array}{c}\hfill {\lambda }_1|...\sim \text{Poisson}(y_1|{\lambda }_1)\times \mathrm{\Gamma }({\lambda }_1|1.384,.0992)\times \text{N}(log({\lambda }_2)|log({\lambda }_1),{\tau }^{-2}).\end{array}$$

Furthermore, for $j>1$, the complete conditional is:

$$\begin{array}{c}\hfill {\lambda }_j|...\sim \text{Poisson}(y_j|{\lambda }_j)\times \text{N}(log({\lambda }_j)|log({\lambda }_{j-1}),{\tau }^{-2})\times \text{N}(log({\lambda }_{j+1})|log({\lambda }_j),{\tau }^{-2})\end{array}$$

where $N(\mu ,{\tau }^{-2})$ represents the normal distribution.

In summary, the $y_j$ is quarterly observed data that is assumed to follow a $\text{Poisson}({\lambda }_j\times T_j)$ distribution, where ${\lambda }_j$ represents the enrollment for the full quarter. Through an initial prior for ${\lambda }_1$ at the first quarter and smoothing for $log({\lambda }_j)$ between all subsequent quarters using a log normal dynamic linear model, we provide a posterior distribution of the quarterly rates. The posterior distributions above demonstrate that the estimated accrual for the current period is a function of the data as well as smoothed posteriors from adjacent time periods through the normal dynamic linear model (NDLM). We provide BUGS code in the Appendix.

Case studies

We now apply our model to 3 ongoing clinical trials. For each of these trials, accrual data were calculated by summing the total number of accruals occurring during 3 month intervals, with this calculation beginning at trial initiation. For each trial, the model was provided with 1,000,000 burn-in iterations and an additional 1,000,000 samples drawn, with these samples thinned so that every ${10}^{\mathit{th}}$ sample was retained. This resulted in an analysis chain length of 100,000 samples. Model accrual Bayesian posterior distributions are estimated using the posterior mean and 95% credible interval for each ${\lambda }_j$ for each quarterly accrual rate $j=1,...,J$.

As noted in the the “Methods” section, the enrollment rate is estimated on a quarterly basis; therefore, we present quarterly accrual rates for our motivating examples. For these studies, recruitment milestones are typically generated on a quarterly basis to avoid the variability associated with monthly characterizations of accrual. As such, the model has not been tested with other time periods; however, we expect that increasing/decreasing the periods of time modeled would change the estimated enrollment rates in magnitude and variability. For example, if we were to increase the time period from quarterly to yearly, we would expect the variability in the estimates to decrease. With respect to the model, the decrease in variability would reduce the smoothing provided by the model. If we were to decrease the time period from quarterly to weekly, we would potentially see an increase in variability, with the model likely smoothing the estimates more.

BOOST-3

Brain Oxygen Optimization in Severe Traumatic Brain Injury-3 (BOOST-3, NCT03754114) is a multicenter, randomized, phase III clinical trial comparing both the effectiveness and safety of two alternative goal-directed critical care management strategies for patients with severe traumatic brain injury; one based on both brain tissue oxygen and intracranial pressure monitoring and the other based on intracranial pressure monitoring alone. The study started enrollment on July 23, 2019, and planned to enroll for 4 years. The maximum sample size for the study is 1094 participants with an assumed average quarterly enrollment of 72 participants (24 per month).

Figure 1 shows the observed quarterly enrollment rates, the posterior quarterly enrollment estimates from the proposed model, and the 95% credible intervals for the posterior quarterly enrollment estimates. Table 1 shows the actual accrual rates for the given quarter, estimated accrual rate for the given quarter, 95% credible interval for the estimated accrual rate, and number of sites contributing at least one subject to the total quarterly accrual for the BOOST-3 trial.

Fig. 1.

Estimated versus observed quarterly accrual rates for the BOOST-3 clinical trial. Circles represent the observed accrual rates, the solid line represents the estimated accrual rates, and the shaded region reflects the 95% credible interval around the estimated accrual rates

Table 1.

Number of sites contributing at least one enrollment, the number of subjects accrued, and the estimated subject accrual for the given quarter of interest for BOOST-3

Quarter	Sites contributing	Accrual	Estimated accrual
2019-10-22	2	2	6.6 (3,11.3)
2020-01-21	7	13	11.1 (6.8,16.7)
2020-04-21	6	13	13.7 (8.8,19.9)
2020-07-22	8	17	18.9 (12.6,26.4)
2020-10-21	15	38	34.3 (25,45.5)
2021-01-20	17	37	36.8 (27.3,48)
2021-04-21	18	39	39 (29,50.4)
2021-07-22	18	41	41.3 (31.1,53.1)
2021-10-21	22	48	45.6 (34.7,58.5)
2022-01-20	21	34	35.9 (26.4,46.8)
2022-04-21	21	35	37.3 (27.4,48.5)
2022-07-22	24	59	53.8 (41.5,68.5)
2022-10-21	17	36	37.5 (27.7,48.6)
2023-01-20	16	32	32.4 (23.6,42.7)
2023-04-21	19	30	29.6 (21.2,39.3)
2023-07-22	11	20	24.9 (17,34.1)
2023-10-21	20	47	43.2 (31.8,56.5)

We see that the model does well in characterizing the observed quarterly enrollments, which indicates that external drivers are not likely the cause of the observed variability in the accrual rates. For BOOST-3, we note an initial increasing trend in the accrual rates, due to the site start-up process, after which the accrual rate reaches a steady state. Both the observed and modeled data demonstrate a trend toward increasing accrual over time, but accrual is consistently below the a priori enrollment targets.

Competing effects of multiple non-random sources of variability in accrual may cancel each other and make the effects harder to visualize, such as increasing accrual due to onboarding of new sites at the same time that accrual is being suppressed due to the COVID-19 pandemic. As an example, despite the onset of the COVID-19 pandemic, the graph shows increasing accrual through late 2021, which is near the onset of the Omicron wave of the COVID-19 pandemic; however, the subsequent decrease observed in the accrual rates could also be attributed to the known temporal trends in traumatic brain injury [12, 13].

ICECAP

Influence of Cooling Duration on Efficacy in Cardiac Arrest Patients (ICECAP, NCT04217551) is a multicenter, phase II/III adaptive clinical trial that uses the hypothermia duration-response-curve to determine whether therapeutic cooling is effective and to select an optimal duration of cooling in comatose survivors of out-of-hospital cardiac arrest. Duration of cooling is a component of hypothermia dose and ranges from 6 to 72 h in this trial. This trial will have up to 10 arms, reflecting incremental durations (doses) of cooling. The study started enrollment on May 18, 2020, and planned to enroll for 4 years. The maximum sample size for the study is 1800 participants with an assumed average quarterly enrollment of 125 participants (41.67 per month).

Similar to Figs. 1 and 2 shows the observed quarterly enrollment rates, the posterior quarterly enrollment estimates from the proposed model, and the 95% credible intervals for the posterior quarterly enrollment estimates. Table 2 shows the actual accrual rates for the given quarter, estimated accrual rate for the given quarter, 95% credible interval for the estimated accrual rate, and number of sites contributing at least one subject to the total quarterly accrual for the ICECAP trial.

Fig. 2.

Estimated versus observed quarterly accrual rates for the ICECAP clinical trial. Circles represent the observed accrual rates, the solid line represents the estimated accrual rates, and the shaded region reflects the 95% credible interval around the estimated accrual rates

Table 2.

Number of sites contributing at least one enrollment, the number of subjects accrued, and the estimated subject accrual for the given quarter of interest for ICECAP

Quarter	Sites contributing	Accrual	Estimated accrual
2020-08-17	4	11	17.5 (10.2,26.6)
2020-11-16	17	48	45.1 (34.2,58)
2021-02-15	24	77	73.5 (58.5,90.4)
2021-05-18	32	73	71.7 (57.3,88.1)
2021-08-17	23	54	57.7 (44.6,72.4)
2021-11-16	33	86	82.1 (66.2,100)
2022-02-15	29	63	65.3 (51.6,80.6)
2022-05-18	32	77	74.1 (59.4,90.8)
2022-08-17	26	52	54.4 (42,68.3)
2022-11-16	30	58	57.5 (44.8,71.9)
2023-02-15	30	56	55.6 (43.2,69.8)
2023-05-18	25	49	50.5 (38.6,63.6)
2023-08-17	26	58	57.2 (44.6,71.4)
2023-11-16	28	57	57.1 (43.8,72)

As seen in BOOST-3, both the observed and modeled quarterly data demonstrate a trend toward increasing accrual over time, but again accrual is consistently below the a priori enrollment targets. Unlike BOOST-3, we note fluctuations in enrollment which are indicative of increased variability in the accrual rates, potentially attributed to the COVID-19 pandemic. Variability in accrual rates can be induced by a variety of factors, including internal study factors such as sites opening for enrollment or external factors including the COVID-19 pandemic. In this case, the increases in variability are observed after the study was expected to hit a “steady state”, around the 3rd or 4th quarter, where there would be dramatic decreases in the number of sites opening. The reduction in the number of sites opening for enrollment combined with the timing of the COVID pandemic indicates that a potential explanation for the variability in accrual we see are the corresponding COVID waves.

HOBIT

Hyperbaric Oxygen Brain Injury Treatment Trial (HOBIT, NCT02407028) is a multicenter, randomized, adaptive phase II clinical trial with the primary goal of selecting a dose regimen of hyperbaric oxygen that is most likely to be effective at improving functional outcome in patients with severe traumatic brain injury. The study started enrollment on June 25, 2018, and planned to enroll for 5 years. The maximum sample size for the study is 200 participants with an assumed average quarterly enrollment of 10 participants (3.33 per month).

Figure 3 again shows the observed quarterly enrollment rates, the posterior quarterly enrollment estimates from the proposed model and the 95% credible intervals for the posterior quarterly enrollment estimates. Table 3 shows the actual accrual rates for the given quarter, estimated accrual rate for the given quarter, 95% credible interval for the estimated accrual rate, and number of sites contributing at least one subject to the total quarterly accrual for the HOBIT trial.

Fig. 3.

Estimated versus observed quarterly accrual rates for the HOBIT clinical trial. Circles represent the observed accrual rates, the solid line represents the estimated accrual rates, and the shaded region reflects the 95% credible interval around the estimated accrual rates

Table 3.

Number of sites contributing at least one enrollment, the number of subjects accrued, and the estimated subject accrual for the given quarter of interest for HOBIT

Quarter	Sites contributing	Accrual	Estimated accrual
2018-09-24	2	4	5.6 (3.6,8.7)
2018-12-24	5	8	5.8 (4,8.9)
2019-03-25	4	5	5.6 (3.9,8.3)
2019-06-25	4	5	5.7 (4,8.4)
2019-09-24	5	13	6 (4.1,10.1)
2019-12-24	3	3	5.1 (3.3,7.1)
2020-03-24	3	3	4.7 (2.6,6.5)
2020-06-24	1	1	4.5 (2.3,6.3)
2020-09-23	3	6	4.7 (2.9,6.5)
2020-12-23	2	4	4.7 (2.9,6.5)
2021-03-24	2	5	4.8 (3,6.6)
2021-06-24	3	3	4.8 (3,6.6)
2021-09-23	2	6	5 (3.4,7.3)
2021-12-23	4	8	5.2 (3.6,8)
2022-03-24	3	4	4.9 (3.3,7)
2022-06-24	3	5	4.9 (3.2,6.9)
2022-09-23	3	6	4.8 (3.1,6.8)
2022-12-23	3	4	4.5 (2.7,6.3)
2023-03-24	1	1	4.3 (2.3,6)
2023-06-24	2	4	4.5 (2.6,6.4)
2023-09-23	2	6	4.7 (2.7,7.2)

Unlike in the previous examples, for HOBIT, there are more discrepancies between the observed quarterly enrollment rates and the accrual characterized by the model. The results for HOBIT demonstrate the ability of the model to provide smoothed accrual estimates which do not overreact to the variability in the observed accrual data. The noise caused by the small quarterly accrual makes the smoothed model results more useful. For example, there is not an overreaction to the early increasing trend in enrollment. The results show that, since recovering from the initial COVID-19 wave, the estimated accrual rates for HOBIT demonstrate a relatively consistent trend since the trial began enrollment.

Discussion

Participant accrual in clinical trials for pre-defined time periods can be characterized using a NDLM. The utilization of a NDLM provides smoothed estimates of the accrual rates within a flexible yet standardized approach. We applied this approach to three ongoing clinical trials within the SIREN Network: BOOST-3, ICECAP, and HOBIT. The results of these applications demonstrate the flexibility of the model with respect to varied accrual rates and the ability of the model to provide smoothed estimates of accrual. This work was in response to discussions with funding agencies and other stakeholders seeking to better understand the progression of trial accrual over the duration of the study, and to try to distinguish random fluctuations in accrual from those that may be causally related to identifiable factors.

Conclusions

In this work, we developed an approach for the standardized characterization of trial accrual which is not overly sensitive to the variability often observed in accrual rates and applied our approach to three ongoing clinical trials. The current work has some potential limitations. The approach described here does not account for the number of sites open to enrollment during each quarter. This assumption is likely violated during the initial stages of the study when clinical sites are being opened for enrollment. Similarly, the estimated accrual rates are assumed to be constant within each quarter; however, as with the prior assumption, this assumption is likely violated early in the study when clinical sites are being opened for enrollment. As the number of sites open to enrollment are increased, the global accrual rate is expected to increase. Adapting the presented model to allow for the inclusion of the site initiation process would remedy these violated assumptions. The model also does not account for seasonal variation in the frequency of traumatic injuries, which can affect rates of accrual in trials of patients with TBI [12, 13].

In addition to modifying the model to account for the known contributions of these to accrual variability, it may also be possible to incorporate a classification model to cluster sites by accrual to help understand site contributions to variability.

Authors' contributions

JB, BG, and SDY drafted the manuscript. All authors read and approved the final version of the manuscript.

Funding

This work was supported by the National Institute of Neurological Disorders and Stroke and the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Numbers U24NS100659, U24NS100655, U01NS099046, UH3HL145269, U24HL145272, U01NS095926, and U01NS095814. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Data availability

The data presented are derived from ongoing clinical trials, and as such they are not available.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.