A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analyses

Ravi Garg; Gabriel Torrealba-Acosta; Steffen Mickenautsch; Vance W Berger

doi:10.1371/journal.pone.0315342

. 2025 Mar 19;20(3):e0315342. doi: 10.1371/journal.pone.0315342

A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analyses

Ravi Garg ^1,^*,^#, Gabriel Torrealba-Acosta ^2,^#, Steffen Mickenautsch ^3,^#, Vance W Berger ^4,^#

Editor: Cem Bilgin,⁵

PMCID: PMC11922233 PMID: 40106441

Abstract

Objectives

Little is known about the integrity of randomization for randomized controlled trials (RCT) included in alteplase for stroke meta-analyses. If the RCTs were not properly randomized, the results could not be accepted at face value. The objective was to assess the integrity of randomization in individual patient data (IPD) meta-analyses supporting alteplase for acute ischemic stroke.

Methods

We assessed randomization reporting, performed qualitative risk of bias assessments arising from the randomization process, and performed fixed effects meta-analyses of baseline variables for which zero heterogeneity is expected if all included RCTs have unbiased randomization. Fixed-effects meta-analyses of baseline age, weight, and National Institute of Health Stroke Scale (NIHSS) score were performed. If heterogeneity was present (I² > 0%), trials were systematically removed, starting with the RCT with the largest t-statistic until the I² value was 0%.

Results

The NINDS rt-PA Stroke Study had a high risk of bias, the ECASS-3 RCT had some concerns, and all other trials were graded as low risk according to the Cochrane Risk of Bias (ROB-2) tool. The NINDS rt-PA Stroke Study contributed to heterogeneity in age and weight meta-analyses, and the ECASS-3 RCT contributed to heterogeneity in the NIHSS score meta-analysis. Removal of suspect trials resulted in the expected I² value of 0%.

Conclusions:

The NINDS rt-PA Stroke Study and ECASS-3 trials contributed to heterogeneity in fixed effects meta-analyses of baseline variables while there should have been none. These RCTs are likely a source of selection bias in IPD meta-analyses due to suspect randomization.

Introduction

Meta-analyses of randomized controlled trials (RCTs) are considered the highest level of evidence for clinical practice guideline (CPG) recommendations [1]. Meta-analyses are observational by nature and retain all the biases of component RCTs [2]. Component RCTs rely on randomization to control for selection bias and ensure the validity of statistical testing. Selection bias may be difficult to assess after randomization and depends on methodological reporting. Baseline imbalances following randomization may be a symptom of selection bias or a chance finding. The Cochrane Risk of Bias 2 (RoB 2) tool recommends assessing for baseline imbalances but emphasizes that “baseline differences that are compatible with chance do not lead to a risk of bias” [3]. A chance finding can only be assured, however, if both the random generation of the allocation sequence and the actual randomization procedure were unbiased [4]. For RCTs that employ highly restrictive randomization procedures, such as permuted block randomization, or that have poor methods of allocation sequence concealment, the assumption of unbiased randomization and baseline differences due to chance is a common misconception [4, 5]. Unlike missing outcome data, for which sensitivity analyses can be performed on summary data to ensure study results are robust, selection bias is assessed primarily by a qualitative audit of the randomization process. Adequate self-reporting of the randomization process, however, does not necessarily equate to effective randomization [4].

Hicks and colleagues have popularized an agnostic analysis to assess for bias introduced by component RCTs included in meta-analyses by inadequate allocation sequence concealment and other sources of biased randomization [6–12]. This method relies on the principle that there should be zero heterogeneity in a fixed-effects meta-analysis of baseline continuous variables from component RCTs.

A recent acute ischemic stroke CPG recommends alteplase therapy within 4.5 hours of symptom onset based on meta-analyses of RCTs [1]. Of the eight component RCTs used in recent individual patient data (IPD) meta-analyses, two have suffered from substantial baseline imbalances in important prognostic factors following randomization: the NINDS rt-PA Stroke Study and ECASS-3 [13, 14]. In the NINDS rt-PA Stroke Study, there were imbalances in baseline aspirin use; baseline stroke severity measured by the National Institute of Health Stroke Scale (NIHSS) score; stroke sub-type; and baseline computed tomography (CT) all favoring the alteplase arm [14]. In the ECASS-3 RCT, there was similarly an imbalance in the baseline NIHSS score and additionally, an imbalance in prior stroke status both favoring the alteplase arm [13]. Although multiple reanalyses have been published focusing on covariate adjustments, latent covariate imbalances cannot be accounted for if the randomization is not valid [15].

The possibility that component RCTs of alteplase treatment for acute ischemic stroke introduced bias arising from the randomization process into meta-analyses has not been assessed. Therefore, the purpose of this study was to assess for bias arising from the randomization process in component RCTs included in alteplase for ischemic stroke IPD meta-analyses.

Methods

The publicly available NINDS rt-PA Stroke Study and IST-3 data sets were used for these analyses. The data was first accessed for research purposes on July 26^th, 2022. The authors did not have access to information that could identify individual participants during or after data collection.

Study design and population

Institutional review board or ethics approval was not obtained (exempt determination) based on guidance at the authors institutions. Randomized controlled trials used in IPD meta-analyses by the Stroke Thrombolysis Trialists’ (STT) Collaborative Group were chosen for this analysis [16–21]. The STT Collaborative Group pooled IPD from nine RCTs for meta-analyses: NINDS A (N = 291); NINDS B (N = 333); ECASS I (N = 620); ECASS II (N = 800); ATLANTIS A (N = 142); ATLANTIS B (N = 613); ECASS III (N = 821); EPITHET (N = 101); and IST-3 (N = 3035) for a total of 6756 participants [22–29]. For this analysis, NINDS-A and NINDS-B were considered one trial as has been previously done given continuous, shared randomization between the two parts [15]. We excluded the ECASS 1 RCT, as has been previously done, to mimic treatment estimates used to support recommendations from clinical practice guidelines labeled “patients who would have met a 4.5 hour revised US label” by IPD meta-analysis authors [21]. The primary outcome of the meta-analysis was a modified Rankin Scale score of 0-1 at 3-6 months post-stroke. The Oxford Handicap Scale score (OHS) was treated as an equivalent to the mRS score.

Data analysis

Data was abstracted from the published RCTs independently by two researchers (R.G. and G.T.A.). Data from the NINDS rt-PA Stroke Study and IST-3 RCT were abstracted from the publicly available datasets. Key components for randomization methods were assessed for all component RCTs. We assessed the risk of bias of RCTs included in the meta-analysis using the RoB 2 tool [3]. We compared our risk of bias assessments to those in the included meta-analyses.

The technique proposed by Hicks et al. was used for the primary analysis [6]. Three continuous variables were abstracted from component RCTs using a standardized form: age, NIHSS score, and weight. Age and NIHSS score were chosen due to their status as being the most important prognostic variables for ischemic stroke outcomes for which any mean difference between groups, irrespective of the p-value from null hypothesis significance testing (NHST), would be considered relevant in the assessment of post-randomization confounding [30]. Weight was chosen as a validation variable to ensure the integrity of the analysis due to consistent reporting in alteplase for stroke RCTs.

The t-statistic was calculated from an independent two-sample t-test for the difference in baseline continuous variables, and the studies were ranked, in descending order, by the absolute value of their t-statistic for each variable. Trials expressing baseline variables as medians were excluded. A fixed-effects meta-analysis for each variable was performed to measure heterogeneity by I² and trials were removed with subsequent iterations until the meta-analysis revealed no heterogeneity (I² = 0%). To assess the assumptions of the procedure, it was repeated performing a random-effects meta-analysis (DerSimonian and Laird method) starting with the trial with the two smallest t-statistics and adding RCTs based on ascending order of their t-statistics until heterogeneity was observed. The point estimates and 95% confidence intervals for I² and τ² were assessed at each iteration. The confidence intervals for I² and τ² were determined using non-parametric bootstrapping and unequal tail probabilities [31]. All analyses were performed in R Studio version 2023.06.1 + 524. R code for the analysis can be provided for reasonable requests.

Measures

To quantify the degree of selection bias, a pooled risk difference was determined with and without suspect trials. An mRS score of 0-2 was chosen to increase the precision of estimates.

Results

Randomization methods reporting and risk-of-bias arising from the randomization process

The randomization methods and risk-of-bias for the included RCTs were reported in Table 1.

Table 1. Randomization reporting and risk of bias for included randomized trials.

Trial	Randomization Method	Allocation Sequence Concealment	Risk of Bias Arising from the Randomization Process
NINDS rt-PA Stroke Study	Stratified block randomization. Block sizes varied.	Non-numerically ordered drug pre-packs. Patient ID matched to randomization schedules kept at site of randomization.	High
ECASS-2	Stratified block randomization. Fixed block sizes of four.	Unreported, but noted to use “sequential patient numbers.”	Low
ATLANTIS-A	Stratified block randomization. Block size unreported.	Interactive voice system	Low
ATLANTIS-B	Stratified block randomization. Block size unreported.	Interactive voice system	Low
ECASS-3	Permuted block randomization. Fixed blocks sizes of four.	Interactive voice system	Some Concerns
EPITHET	Permuted block randomization. Fixed block sizes of four.	Sequentially numbered drug pre-packs.	Low
IST-3	Minimisation.	Web-based or telephone system.	Low

Open in a new tab

Based on ROB-2, the NINDS rt-PA Stroke Study was rated as having a high risk of bias arising from the randomization process (S1 Table); while the ECASS-3 was rated as having some concerns (S4 Table). The remaining five RCTs [ATLANTIS (S2 Table); ECASS-2 (S3 Table); EPITHET (S5 Table); and IST-3 (S6 Table)] were rated as having a low risk of bias arising from the randomization process. In addition to being the only trials that were not graded as a low risk of bias, the NINDS rt-PA Stroke Study and ECASS-3 RCT were the only RCTs with baseline imbalances in important prognostic variables favoring the treatment arm (S1 Table; S4 Table). The risk of bias arising from the randomization process was not reported in the five IPD meta-analyses published.

Heterogeneity in baseline variables

The differences in independent means of age, weight, and NIHSS score; and their associated t-statistics were reported in S7 Table. The EPITHET RCT only reported age as a mean ± SD while the NIHSS was reported as a median and weight was unreported [22]. The ECASS-2 RCT did not report continuous baseline characteristics as means ± standard deviations or report interquartile ranges associated with medians and was excluded from the analysis [25]. The NINDS rt-PA Stroke Study had the highest t-statistic for age (2.19) and weight (2.69). The ECASS-3 RCT had the highest t-statistic for NIHSS (2.24) (S7 Table).

A fixed-effects meta-analysis for age revealed the NINDS rt-PA Stroke Study was the only study for which the 95% confidence interval did not include the null. Heterogeneity was present (I² = 15%, 95% CI 0-50%) [Fig 1].

Fig 1 — The NINDS rt-PA Stroke Study (NINDS) is the only included study for which the 95% confidence interval did not include the null.

Exclusion of the NINDS rt-PA Stroke Study was required to achieve no heterogeneity (I² = 0%, 95% CI 0-0%).

A fixed-effects meta-analysis for weight revealed the NINDS rt-PA Stroke Study was the only study for which the 95% confidence interval did not include the null. Heterogeneity was present (I² = 45%, 95% CI 0-68%) [Fig 2]. Exclusion of the NINDS rt-PA Stroke Study was required to achieve no baseline heterogeneity (I² = 0%, 95% CI 0-0%).

Fig 2 — The NINDS rt-PA Stroke Study was the only included study for which the 95% confidence interval did not include the null.

A fixed-effects meta-analysis for NIHSS revealed the ECASS 3 RCT was the only study for which the 95% confidence interval did not include the null. Heterogeneity was present (I² = 31%, 95% CI 0-55%) [Fig 3]. Exclusion of the ECASS 3 RCT was required to achieve no baseline heterogeneity (I² = 0%, 95% CI 0-0%). The procedure for all three variables was validated in reverse (S1–S3 Figs). The results were robust to changes in sample size and were identical using τ².

Fig 3 — The ECASS-3 RCT was the only included study for which the 95% confidence interval did not include the null.

Quantification of selection bias

The pooled absolute risk difference, using the DerSimonian-Laird method, for all included trials was 3% (95% CI, -1% - 8%) (Fig 4). After the exclusion of the NINDS rt-PA Stroke Study and ECASS-3, the pooled absolute risk difference was reduced to 1% (95% CI, -4% - 6%) (Fig 5). The I² estimate including all studies was 58% compared to 50% after removal of the NINDS rt-PA Stroke Study and ECASS-3.

Discussion

Qualitative risk of bias assessments arising from the randomization process, based on ROB 2, graded the NINDS rt-PA Stroke Study as a high risk of bias while the ECASS-3 RCT had some concerns. Both trials inadequately reported random sequence generation and randomization implementation, and both trials contributed to heterogeneity in the fixed-effects meta-analyses of baseline variables. Taken together, these trials are a source of systematic error in meta-analyses. We estimated that 2% of the average treatment effect (ATE) can be attributed to selection bias.

Our results found congruence between the qualitative risk of bias assessments and the evaluation of heterogeneity in the fixed-effects meta-analyses of baseline variables such that the two trials that were not graded a low-risk of bias (NINDS rt-PA Stroke Study and ECASS-3 RCT), also contributed to heterogeneity in the meta-analyses. Other authors, however, have reported discrepant results between the qualitative risk of bias assessments and the evaluation of heterogeneity in fixed-effects meta-analyses of baseline variables. For example, in a meta-analysis of RCTs of transversus abdominis plane block efficacy in hysterectomy, six RCTs were graded with a low risk of bias arising from the randomization process, and one was rated as having some concerns. Despite the favorable qualitative risk of bias assessments, meta-analyses of two baseline variables identified heterogeneity, while there should have been none if randomization was unbiased [6].

There are other limitations to full reliance on the qualitative risk of bias assessments. The ROB 2 tool is dependent on the self-reporting of RCTs. For instance, in a meta-analysis on the effect of vitamin K supplementation on bone mineral density and fractures, most trials had an unclear risk of bias in the random sequence generation or allocation sequence concealment domains [10]. A fixed-effects meta-analysis of baseline age revealed no heterogeneity providing reassurance about the integrity of randomization in these individual studies. These examples support previous recommendations on more detailed reporting in RCTs to control for selection bias (Table 2) [32].

Table 2. Recommendations for improved randomization reporting in randomized controlled trials. Adapted from Berger and Cristophi [32].

Concern	Reporting Recommendation
Allocation Discretion	Planned allocation proportions and numbers of screened/randomized patients by assigned group.
Deferred Enrollment	Indicate if any participants were screened multiple times or confirm that no duplicates occurred.
Allocation Concealment	Explain how future group assignments were kept secret.
Predicted Allocations	Provide details on randomization methods, including any restrictions like block sizes, and describe how group assignments were masked and any instances where assignments were revealed.
Baseline Imbalances	Assess and report whether participant characteristics, especially when prognostic, were evenly distributed between groups.
Selection Bias	Plot key variables and outcomes against the likelihood of being assigned to each group (reverse propensity score) and note any errors in stratification.

Open in a new tab

The NINDS rt-PA Stroke Study and ECASS-3 RCT have similarities that are worth noting. Both RCTs had baseline imbalances in important prognostic variables (S1 and S4 Tables) and were also the only trials that contributed to heterogeneity in meta-analyses of baseline variables. The use of NHST to assess for differences at baseline is controversial [33]. A major argument against the routine use of NHST is that unbiased randomization produces an expected distribution of p values that does not necessarily apply to a single hypothesis test. Additionally, this distribution may be affected by the correlation between baseline characteristics and the number of hypothesis tests performed. Conversely, the premise of these arguments is that randomization, and not just random sequence generation, is unbiased which may be a foregone conclusion. Our results support the use of NHST at baseline and are consistent with other RCTs for which substantial baseline imbalances, coupled with flawed randomization methods, raised suspicion of selection bias [4].

Both RCTs employed permuted block randomization for which the potential for third-order selection bias has been thoroughly described [5]. Permuted blocks facilitate predictability in future allocations and may lead to more favorable patient selection in the experimental arm, and this has been referred to as the convergent strategy of guessing without certainty [34]. In the NINDS rt-PA Stroke Study, varying block sizes were used. Contrary to popular belief, varying the block size does not lessen the chance of randomization subversion [34, 35]. Rather, this strategy may be as problematic as using fixed block sizes. In a review of 179 open-label RCTs employing the same methodology chosen for the current report, the authors found baseline heterogeneity for age in all trials implementing a block randomization scheme, trials implementing a fixed block size of 4, and trials implementing varied block sizes including a block size of 2 [36]. In the ECASS-3 RCT, a fixed block size of four was used.

The principal issue with varied block sizes, which are smaller on average, and a fixed block size of four is the same: high predictability of future allocations. To illustrate, if the first allocation is identified as alteplase, the second allocation is twice as likely to be placebo as alteplase using a block size of 4. If a record of allocations is kept the last allocation is 100% predictable in 1/3^rd of block sequence permutations [36]. Even if the allocation sequence is strictly concealed, the potential for accidental unblinding in alteplase trials is high by intracerebral hemorrhage, systemic bleeding, or angioedema. Additionally, neither the NINDS rt-PA Stroke Study nor the ECASS-3 RCT reported how the frothing reaction that occurs with alteplase reconstitution was mimicked in the placebo arm as is explicitly reported in more modern alteplase RCTs [37].

The EPITHET and ATLANTIS RCTs also employed blocked randomization. Still, they did not contribute to baseline heterogeneity in the age meta-analysis suggesting other elements of susceptible randomization may be required to precipitate randomization subversion. For example, randomization was decentralized in the NINDS rt-PA Stroke Study. Other elements that precipitate randomization subversion, such as the motivation of investigators or conflicts of interest, are unlikely to be fully substantiated post hoc.

Although permuted blocks facilitate the convergent strategy of guessing and may introduce third-order selection bias, the ability to directly subvert randomization is also possible; and this has been suspected in the NINDS rt-PA Stroke Study [14]. In this trial, sealed envelopes with the treatment assignments were attached to study pre-packs for purposes of emergent unblinding for adverse events. At the end of the trial, sixteen envelopes were opened for unblinding of which eight did not have listed safety reasons. Comparatively, only five envelopes were opened for unblinding in the ECASS-2 RCT which had a larger sample size. The NINDS rt-PA Stroke trial data revealed deviations from the expected 1:1 allocation ratio in most randomization strata; violation of a pre-specified randomization rule at 3 centers; and multiple baseline imbalances in prognostic variables favoring the alteplase arm when evaluated on exclusion criteria (S8 Table) [38]. A similarly detailed audit of the randomization process of other included trials, including the ECASS-3 RCT, cannot be performed due to a lack of publicly available trial data, study protocols, or product licensing applications.

Our results have important implications for meta-analyses and CPG recommendations regarding alteplase for acute ischemic stroke. First, we found that meta-analyses, including the NINDS rt-PA Stroke Study and ECASS-3, overestimated the ATE by 2%. Importantly, covariate adjustments do not adequately control this bias, as flawed randomization might lead to latent covariate imbalances. To perform a meta-analysis without removing suspect trials, the reverse propensity score (RPS) could be used to correct for selection bias [39]. The RPS is the conditional probability of a patient being assigned a specific treatment given prior allocations under block randomization. The randomization schedules are required to calculate the RPS, however, and are not publicly available.

As meta-analyses are widely considered the pinnacle of the evidence-based medicine pyramid, our findings have important implications for CPG recommendations. Currently, CPGs do not acknowledge the presence of selection bias in the NINDS rt-PA Stroke Study and ECASS-3 [1,40]. Aligned with the best methodological practices for CPG development, we believe the strength of the recommendation, currently “strong”, should be reconsidered based on limitations in the quality of the evidence [41].

Despite the inherent risk of selection bias, permuted block randomization remains a common randomization procedure in stroke trials and may be a more pervasive problem than appreciated. For example, a recent thrombolytic trial employing stratified block randomization resulted in unequal group allocations by the stratifying factor and ten baseline factors favoring the thrombolytic arm concerning for selection bias [42, 43]. Therefore, methods that are superior to permuted block randomization, such as the asymptotic maximal procedure, simple randomization, and minimization, should be used. Bayesian adaptive randomization designs have also recently been successfully employed in stroke clinical trials [44]. Alternatively, if permuted block randomization is performed, the integrity of the process should be verified using the RPS.

Limitations

The current report has limitations that are worth noting. The IPD meta-analyses include the ECASS-2 RCT which was excluded in the current report due to a lack of reporting of the chosen continuous variables. In ECASS-2, the median age and NIHSS were reported without their associated interquartile ranges. We believe this deviates from the best practices of baseline reporting. As previously noted, only trials that were not graded with a low risk of bias contributed to heterogeneity while ECASS-2 was graded with a low risk of bias. Given the heterogeneity contributions of the NINDS rt-PA Stroke Study and ECASS-3 RCT, it is unlikely that the exclusion of ECASS-2 meaningfully changes the conclusions as we also validated the procedure in reverse.

We found discrepancies in the trials that contributed to heterogeneity in meta-analyses: the NINDS rt-PA Stroke Study contributed to heterogeneity in age and weight meta-analyses while the ECASS-3 RCT only contributed to heterogeneity in the NIHSS score meta-analysis. It is notable in the meta-analysis of baseline NIHSS, that the 95% confidence interval begins to increase only after the addition of the NINDS rt-PA Stroke Study when the procedure was conducted in reverse, although the point estimate remained 0 (S3 Fig). Importantly, the chosen variables may not reflect the baseline characteristic that randomization was subverted on as this may be an unknown or categorical covariate. Rather, there may be an association between the variable for which heterogeneity was detected and the baseline characteristic(s) that randomization was subverted on. Alternatively, the discrepancies in heterogeneity may simply be a chance finding and this has been previously described using the current methodology in a systematic review of spinal manipulative therapy for chronic low back pain [7]. We agree with the original authors of the current method that any observed baseline heterogeneity is a cause for concern. As such, the reported results are best served in congruence, as opposed to isolation, with trial-level data. In the case of the NINDS rt-PA Stroke Study, the reported results are consistent with a risk of selection bias assessment using trial-level data [14]. A similar trial-level assessment of the ECASS-3 RCT could confirm or deny these findings.

We were unable to replicate multiple subgroup analyses published in IPD meta-analyses. Any subgroup, however, would retain systematic error found in the pooled estimate of all trials. The current report also only addressed bias arising from the randomization process. Other limitations of the IPD meta-analyses such as unblinding in the IST-3 RCT; handling of missing outcomes; and inclusion of RCTs with small sample sizes were not addressed in the current report and should also be considered [45–47].

Due to the lack of diversity in randomization methods used in the included trials, we could not substantiate if certain randomization procedures inherently impact the detection of baseline heterogeneity in the current method. Five out of six included trials used permuted block randomization, and one used minimization. Due to the predictability of future allocations, we believe this randomization method has a specifically higher risk of selection bias than others such as minimization with a random factor as used in the IST-3 RCT.

Finally, as previously noted, we did not have access to the randomization schedules and were unable to calculate an RPS to definitively detect and correct for selection bias; or assess for over-stratification.

A strength of the current report is the use of agnostic, validated analysis using summary data to assess randomization. The procedure has been reported in meta-analyses of tranexamic acid for post-partum hemorrhage; vitamin K supplementation on bone mineral density and fractures; transversus abdominis plane block efficacy in hysterectomy; interventions for the management of primary frozen shoulder; and atypical antipsychotics in dementia suggesting strong external validation [8–12]. The procedure was robust to assumptions, including changes in sample size, consistent with a recent simulation [48]. An important future area of meta-epidemiological research is determining, using the current method, whether certain randomization procedures portend a higher risk of selection bias than others.

Conclusions

Among RCTs included in IPD meta-analyses of alteplase for acute ischemic stroke, the NINDS rt-PA Stroke Study had a high risk of bias arising from the randomization process while the ECASS-3 RCT had some concerns. Both trials inadequately reported randomization and contributed to heterogeneity in fixed effects meta-analyses of baseline variables suggesting biased randomization. Results from these RCTs, and meta-analyses that include these RCTS, should be accepted cautiously.

Supporting Information

S1 Table. Risk of Bias Arising from the Randomization Process in the NINDS rt-PA Stroke Study.

(DOCX)

pone.0315342.s001.docx^{(22.7KB, docx)}

S2 Table. Risk of Bias Arising from the Randomization Process in the ATLANTIS-A and ATLANTIS-B Randomized Clinical Trials.

(DOCX)

pone.0315342.s002.docx^{(14.5KB, docx)}

S3 Table. Risk of Bias Arising from the Randomization Process in the ECASS-2 Randomized Clinical Trial.

(DOCX)

pone.0315342.s003.docx^{(14.8KB, docx)}

S4 Table. Risk of Bias Arising from the Randomization Process in the ECASS-3 trial.

Abbreviations: NIHSS, National Institute of Health Stroke Scale.

(DOCX)

pone.0315342.s004.docx^{(14.7KB, docx)}

S5 Table. Risk of Bias Arising from the Randomization Process in the EPITHET trial.

(DOCX)

pone.0315342.s005.docx^{(14.6KB, docx)}

S6 Table. Risk of Bias Arising from the Randomization Process in the IST-3 trial.

(DOCX)

pone.0315342.s006.docx^{(14.5KB, docx)}

S7 Table. Mean differences in age, weight, and National Institute of Health Stroke Scale scores; and the associated t-statistic for all included trials.

(DOCX)

pone.0315342.s007.docx^{(14.9KB, docx)}

S8 Table. Overview of selection bias in the NINDS rt-PA Stroke Study.

(DOCX)

pone.0315342.s008.docx^{(16KB, docx)}

S1 Fig. The I² point estimate (orange dot) and associated 95% confidence intervals for the age meta-analysis.

(TIF)

pone.0315342.s009.tif^{(334.4KB, tif)}

S2 Fig. The I² point estimate (orange dot) and associated 95% confidence intervals for the weight meta-analysis.

(TIF)

pone.0315342.s010.tif^{(256.2KB, tif)}

S3 Fig. The I² point estimate (orange dot) and associated 95% confidence intervals for the NIHSS meta-analysis.

(TIF)

pone.0315342.s011.tif^{(305.6KB, tif)}

Acknowledgments

We thank the patients who participated in the included stroke trials.

Data Availability

Data Availability: The IST-3 RCT data set can be accessed here: https://github.com/ravigarg415/IST_3 while the NINDS rt-PA Stroke Study data set can be requested from the National Institute of Neurological Disorders and Stroke here: https://www.ninds.nih.gov/current-research/research-funded-ninds/clinical-research/archived-clinical-research-datasets.

Funding Statement

The author(s) received no specific funding for this work.

References

1.Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2019;50(12):e344–418. [DOI] [PubMed] [Google Scholar]
2.Borzak S, Ridker PM. Discordance between meta-analyses and large-scale randomized, controlled trials: examples from the management of acute myocardial infarction. American College of Physicians. 1995, p. 873–7. [DOI] [PubMed] [Google Scholar]
3.Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomized trials. BMJ. 2019;366. [DOI] [PubMed] [Google Scholar]
4.Berger VW, Weinstein S. Ensuring the comparability of comparison groups: is randomization enough?. Control Clin Trials. 2004;25(5):515–24. doi: 10.1016/j.cct.2004.04.001 [DOI] [PubMed] [Google Scholar]
5.Berger V. Selection bias and covariate imbalances in randomized clinical trials. John Wiley & Sons; 2007. [DOI] [PubMed] [Google Scholar]
6.Hicks A, Fairhurst C, Torgerson DJ. A simple technique investigating baseline heterogeneity helped to eliminate potential bias in meta-analyses. J Clin Epidemiol. 2018;9555–62. doi: 10.1016/j.jclinepi.2017.10.001 [DOI] [PubMed] [Google Scholar]
7.Clark L, Fairhurst C, Cook E, Torgerson DJ. Important outcome predictors showed greater baseline heterogeneity than age in two systematic reviews. J Clin Epidemiol. 2015;68(2):175–81. doi: 10.1016/j.jclinepi.2014.09.023 [DOI] [PubMed] [Google Scholar]
8.López-Ruiz C, Orjuela JC, Rojas-Gualdrón DF, Jimenez-Arango M, Ríos JF de L, Vásquez-Trespalacios EM, et al. Efficacy of transversus abdominis plane block in the reduction of pain and opioid requirement in laparoscopic and robot-assisted hysterectomy: a systematic review and meta-analysis. Rev Bras Ginecol Obstet. 2022;44(1):55–66. doi: 10.1055/s-0041-1740595 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Ker K, Shakur H, Roberts I. Does tranexamic acid prevent postpartum haemorrhage? A systematic review of randomised controlled trials. BJOG. 2016;123(11):1745–52. doi: 10.1111/1471-0528.14267 [DOI] [PubMed] [Google Scholar]
10.Mott A, Bradley T, Wright K, Cockayne ES, Shearer MJ, Adamson J, et al. Effect of vitamin K on bone mineral density and fractures in adults: an updated systematic review and meta-analysis of randomised controlled trials. Osteoporos Int. 2019;30(8):1543–59. doi: 10.1007/s00198-019-04949-0 [DOI] [PubMed] [Google Scholar]
11.Rex SS, Kottam L, McDaid C, Brealey S, Dias J, Hewitt CE, et al. Effectiveness of interventions for the management of primary frozen shoulder : a systematic review of randomized trials. Bone Jt Open. 2021;2(9):773–84. doi: 10.1302/2633-1462.29.BJO-2021-0060.R1 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Hulshof T, Zuidema S, van Meer P, Gispen-de Wied C, Luijendijk H. Baseline imbalances and clinical outcomes of atypical antipsychotics in dementia: A meta-epidemiological study of randomized trials. Int J Methods Psychiatr Res. 2019;28(1):e1757. [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Alper BS, Foster G, Thabane L, Rae-Grant A, Malone-Moses M, Manheimer E. Thrombolysis with alteplase 3-4.5 hours after acute ischaemic stroke: trial reanalysis adjusted for baseline imbalances. BMJ Evid Based Med. 2020;25(5):168–71. doi: 10.1136/bmjebm-2020-111386 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Garg R, Mickenautsch S. Risk of selection bias assessment in the NINDS rt-PA stroke study. BMC Med Res Methodol. 2022;22(1):172. doi: 10.1186/s12874-022-01651-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Ingall TJ, O’Fallon WM, Asplund K, Goldfrank LR, Hertzberg VS, Louis TA, et al. Findings from the reanalysis of the NINDS tissue plasminogen activator for acute ischemic stroke treatment trial. Stroke. 2004;35(10):2418–24. doi: 10.1161/01.STR.0000140891.70547.56 [DOI] [PubMed] [Google Scholar]
16.Stroke Thrombolysis Trialists’ Collaborative Group. Details of a prospective protocol for a collaborative meta-analysis of individual participant data from all randomized trials of intravenous rt-PA vs. control: statistical analysis plan for the Stroke Thrombolysis Trialists’ Collaborative meta-analysis. Int J Stroke. 2013;8(4):278–83. doi: 10.1111/ijs.12040 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Whiteley WN, Emberson J, Lees KR, Blackwell L, Albers G, Bluhmki E, et al. Risk of intracerebral haemorrhage with alteplase after acute ischaemic stroke: a secondary analysis of an individual patient data meta-analysis. Lancet Neurol. 2016;15(9):925–33. doi: 10.1016/S1474-4422(16)30076-X [DOI] [PubMed] [Google Scholar]
18.Emberson J, Lees KR, Lyden P, Blackwell L, Albers G, Bluhmki E, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet. 2014;384(9958):1929–35. doi: 10.1016/S0140-6736(14)60584-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Lees KR, Emberson J, Blackwell L, Bluhmki E, Davis SM, Donnan GA, et al. Effects of Alteplase for Acute Stroke on the Distribution of Functional Outcomes: A Pooled Analysis of 9 Trials. Stroke. 2016;47(9):2373–9. doi: 10.1161/STROKEAHA.116.013644 [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Bluhmki E, Danays T, Biegert G, Hacke W, Lees KR. Alteplase for acute ischemic stroke in patients aged >80 years: pooled analyses of individual patient data. Stroke. 2020;51(8):2322–31. doi: 10.1161/STROKEAHA.119.028396 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Hacke W, Lyden P, Emberson J, Baigent C, Blackwell L, Albers G, et al. Effects of alteplase for acute stroke according to criteria defining the European Union and United States marketing authorizations: Individual-patient-data meta-analysis of randomized trials. Int J Stroke. 2018;13(2):175–89. doi: 10.1177/1747493017744464 [DOI] [PubMed] [Google Scholar]
22.Davis SM, Donnan GA, Parsons MW, Levi C, Butcher KS, Peeters A, et al. Effects of alteplase beyond 3 h after stroke in the Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET): a placebo-controlled randomised trial. Lancet Neurol. 2008;7(4):299–309. doi: 10.1016/S1474-4422(08)70044-9 [DOI] [PubMed] [Google Scholar]
23.Clark WM, Wissman S, Albers GW, Jhamandas JH, Madden KP, Hamilton S. Recombinant tissue-type plasminogen activator (Alteplase) for ischemic stroke 3 to 5 hours after symptom onset. The ATLANTIS Study: a randomized controlled trial. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. JAMA. 1999;282(21):2019–26. doi: 10.1001/jama.282.21.2019 [DOI] [PubMed] [Google Scholar]
24.Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g) : results of a double-blind, placebo-controlled, multicenter study. Thromblytic therapy in acute ischemic stroke study investigators. Stroke. 2000;31(4):811–6. doi: 10.1161/01.str.31.4.811 [DOI] [PubMed] [Google Scholar]
25.Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, et al. Randomised double-blind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet. 1998;352(9136):1245–51. doi: 10.1016/s0140-6736(98)08020-9 [DOI] [PubMed] [Google Scholar]
26.Hacke W, Kaste M, Bluhmki E, Brozman M, Dávalos A, Guidetti D, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med. 2008;359(13):1317–29. doi: 10.1056/NEJMoa0804656 [DOI] [PubMed] [Google Scholar]
27.IST-3 collaborative group, Sandercock P, Wardlaw JM, Lindley RI, Dennis M, Cohen G, et al. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet. 2012;379(9834):2352–63. doi: 10.1016/S0140-6736(12)60768-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Group S-P, NIoN D. Tissue plasminogen activator for acute ischemic stroke. New Eng J Med. 1995;333(24):1581–8. [DOI] [PubMed] [Google Scholar]
29.Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA. 1995;274(13):1017–25. doi: 10.1001/jama.1995.03530130023023 [DOI] [PubMed] [Google Scholar]
30.Mandava P, Kalkonde YV, Rochat RH, Kent TA. A matching algorithm to address imbalances in study populations: application to the National Institute of Neurological Diseases and Stroke Recombinant Tissue Plasminogen Activator acute stroke trial. Stroke. 2010;41(4):765–70. doi: 10.1161/STROKEAHA.109.574103 [DOI] [PubMed] [Google Scholar]
31.Jackson D, Bowden J. Confidence intervals for the between-study variance in random-effects meta-analysis using generalised heterogeneity statistics: should we use unequal tails? BMC Med Res Methodol. 2016;16(1):118. doi: 10.1186/s12874-016-0219-y [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Berger VW, Christophi CA. Randomization technique, allocation concealment, masking, and susceptibility of trials to selection bias. J Mod App Stat Meth. 2003;2(1):80–6. doi: 10.22237/jmasm/1051747680 [DOI] [Google Scholar]
33.Stang A , Baethge C. Imbalance p values for baseline covariates in randomized controlled trials: a last resort for the use of p values? A pro and contra debate. Clinical Epidemiology. 2018:531-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Berger VW, He X. Concealing Block Sizes Is Not Sufficient. Clin Orthop Surg. 2015;7(3):422–3. doi: 10.4055/cios.2015.7.3.422 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Berger VW. Varying the block size does not conceal the allocation. J Crit Care. 2006;21(2):229; author reply 229-30. doi: 10.1016/j.jcrc.2006.01.002 [DOI] [PubMed] [Google Scholar]
36.Clark L, Burke L, Margaret Carr R, Coleman E, Roberts G, Torgerson DJ. A review found small variable blocking schemes may not protect against selection bias in randomized controlled trials. J Clin Epidemiol. 2022;14190–8. doi: 10.1016/j.jclinepi.2021.09.009 [DOI] [PubMed] [Google Scholar]
37.Khatri P, Kleindorfer DO, Devlin T, Sawyer RN Jr, Starr M, Mejilla J, et al. Effect of alteplase vs aspirin on functional outcome for patients with acute ischemic stroke and minor nondisabling neurologic deficits: The PRISMS Randomized Clinical Trial. JAMA. 2018;320(2):156–66. doi: 10.1001/jama.2018.8496 [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Garg R, Mickenautsch S. Risk of selection bias assessment in the NINDS-rtPA stroke study: a reply to raised concerns. Research Gate2023 Available from: https://www.researchgate.net/publication/375960388_Risk_of_selection_bias_assessment_in_the_NINDS-rtPA_stroke_study_a_reply_to_raised_concerns [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Berger VW. The reverse propensity score to detect selection bias and correct for baseline imbalances. Stat Med. 2005;24(18):2777–87. doi: 10.1002/sim.2141 [DOI] [PubMed] [Google Scholar]
40.Cantrill SV, Brown MD, Brecher D, Diercks DB, Gemme SR, Gerardo CJ, et al. Clinical policy: use of intravenous tissue plasminogen activator for the management of acute ischemic stroke in the emergency department. Ann Emerg Med. 2015;66(3):322-333.e31. doi: 10.1016/j.annemergmed.2015.06.031 [DOI] [PubMed] [Google Scholar]
41.Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. doi: 10.1016/j.jclinepi.2010.04.026 [DOI] [PubMed] [Google Scholar]
42.Xiong Y, Campbell BCV, Schwamm LH, Meng X, Jin A, Parsons MW, et al. Tenecteplase for Ischemic Stroke at 4.5 to 24 Hours without Thrombectomy. N Engl J Med. 2024;391(3):203–12. doi: 10.1056/NEJMoa2402980 [DOI] [PubMed] [Google Scholar]
43.Garg R. Tenecteplase for ischemic stroke at 4.5 to 24 hours without thrombectomy. PubPeer2024 Available from: https://pubpeer.com/publications/30D56099E89EF0E310B6D48C2679EE# [DOI] [PubMed]
44.Lee JJ, Xuemin Gu, Suyu Liu. Bayesian adaptive randomization designs for targeted agent development. Clin Trials. 2010;7(5):584–96. doi: 10.1177/1740774510373120 [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ. 2013;346:f2304. doi: 10.1136/bmj.f2304 [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care. 2013;17(1):R2. doi: 10.1186/cc11919 [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Garg R. Methodological survey of missing outcome data in an alteplase for ischemic stroke meta-analysis. Acta Neurol Scand. 2022;146(3):252–7. doi: 10.1111/ane.13656 [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Mickenautsch S, Yengopal V. Trial number and sample size do not affect the accuracy of the I2-point estimate for testing selection bias risk in meta-analyses. Cureus. 2024;16(4).author billing [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. 2025 Mar 19;20(3):e0315342. doi: 10.1371/journal.pone.0315342.r001

Author response to Decision Letter 0

5 Jun 2024

PLoS One. doi: 10.1371/journal.pone.0315342.r002

Decision Letter 0

Cem Bilgin

9 Dec 2024

PONE-D-24-20124A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analysesPLOS ONE

Dear Dr. Torrealba-Acosta,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jan 23 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Cem Bilgin

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that there are restrictions to data sharing for this study. For studies involving human research participant data or other sensitive data, we encourage authors to share de-identified or anonymized data. However, when data cannot be publicly shared for ethical reasons, we allow authors to make their data sets available upon request. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

Before we proceed with your manuscript, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible.

Please update your Data Availability statement in the submission form accordingly.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This manuscript provides a valuable critical assessment of randomization integrity in pivotal alteplase trials for ischemic stroke, identifying concerning baseline imbalances in two key studies. While the methodology is sound, further clarity on the broader clinical implications and potential methodological enhancements for future trials would strengthen the study's impact. By addressing these limitations, the authors could contribute to more robust guidelines and better-supported conclusions in stroke treatment research.

1. The manuscript highlights significant baseline imbalances in the NINDS and ECASS-3 studies, suggesting potential selection bias. Could you discuss if these imbalances could be mitigated or accounted for in future meta-analyses, perhaps by employing statistical adjustment techniques? How might this affect the robustness of the meta-analytic findings?

2. Your findings raise questions about the integrity of widely cited alteplase trials. Could you elaborate on how your results might impact current clinical guidelines for ischemic stroke treatment? Are there implications for ongoing or future clinical trials on stroke treatment?

3. The study depended on publicly available datasets and could not perform a detailed review of the randomization protocols for all included trials. Could you clarify if future research with access to detailed randomization logs and protocols might yield different results? Additionally, are there specific data points that, if available, would strengthen the conclusions drawn?

4. Given the variability in randomization approaches across the RCTs analyzed, could you elaborate on how these differences might inherently impact baseline heterogeneity? It would be beneficial to explain if specific randomization methods were more prone to bias, as this could guide future research designs.

Reviewer #2: 1- Detailed analyses are made and supported by graphs.

2- The number of references is sufficient.

3- Although there are sometimes complex narratives, the sentences are made meaningful and understandable by collecting the sentences afterwards.

4- Although the study investigates the methods used in alteplase meta-analyses for stroke, it provides seminal information and recommendations for the evaluation of many studies.

5- A very good meta-analysis study with statistical data. It is a study that will be taken as an example with its many perspectives and applications and will contribute to the literature. Thank you to the authors. I recommend it to be accepted for publication in your journal.

Reviewer #3: This is a well written methodologic overview of methodological assessment of randomization integrity.

The authors note that based on ROB-2, the NINDS rt-PA Stroke Study was rated as having a high risk of bias arising from the randomization process (S1 Table ); while the ECASS-3 was rated as having some concerns (S4 Table). What exactly was the signal to point to actual risk? Was it insufficient stratification factors up front leading to imbalanced overall? The last column of Table S1 and S4 are after the fact to show imbalance perhaps. Is there any other indicator available?

Although the block sizes on the Figures in the supplement are an indication of the relative study size, it would be helpful to have more descriptive information about the actual individual sample sizes or other relevant information.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: Yes: Mehmet Fatih INECIKLI

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

PLoS One. 2025 Mar 19;20(3):e0315342. doi: 10.1371/journal.pone.0315342.r003

Author response to Decision Letter 1

17 Jan 2025

Manuscript Title: A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analyses

Journal: PLOS ONE

Manuscript ID: PONE-D-24-20124

Date: 12/13/2024

Dear Dr. Cem Bilgin,

We appreciate the constructive feedback and insightful comments provided by the reviewers. We have carefully considered each point and made the necessary revisions to enhance the clarity and scientific rigor of our manuscript. There were no direct responses to Reviewer 2 based on their comments. The “Track Changes” option was employed in Microsoft Word and all new text was highlighted in yellow.

Below, we provide detailed responses to each comment, including specific line numbers where changes have been implemented in the revised manuscript. For clarity, we have structured our responses in a tabular format.

We trust that these revisions address the reviewers' concerns. We are grateful for the opportunity to improve our manuscript and look forward to your feedback. Please do not hesitate to contact us if further clarification is needed.

Sincerely,

Ravi Garg, MD

Gabriel Torrealba-Acosta, MD

Steffen Mickenautsch,BDS, Ph.D

Vance W. Berger, Ph.D

Reviewer 1 Comment Response

The manuscript highlights significant baseline imbalances in the NINDS and ECASS-3 studies, suggesting potential selection bias. Could you discuss if these imbalances could be mitigated or accounted for in future meta-analyses, perhaps by employing statistical adjustment techniques? How might this affect the robustness of the meta-analytic findings?

We appreciate your comments which improved the quality of the manuscript. Please see the added text in lines 770 and 795; 832-840; and 856-858.

While known covariate imbalances can be adjusted for, latent or unknown covariates imbalances cannot be adjusted for using common methods. Rather, a reverse propensity score can be used to determine and correct for selection bias.

We estimated the average treatment effect for the meta-analyses that include the suspect trials will be overestimated by 2%.

Your findings raise questions about the integrity of widely cited alteplase trials. Could you elaborate on how your results might impact current clinical guidelines for ischemic stroke treatment? Are there implications for ongoing or future clinical trials on stroke treatment?

Currently, the most cited clinical practice guidelines do not acknowledge the presence of selection bias in the NINDS rt-PA Stroke Study or ECASS-3. Best methodological practices for guideline development recommend downgrading recommendations for evidentiary quality concerns. Therefore, we advocate for guidelines to reassess their recommendations.

Despite the inherent risk of selection bias under a permuted block randomization procedure, this remains one of the most common procedures overall. We have provided evidence of selection bias from a very recently published thrombolytic trial to support our concerns. Other randomization procedures such as simple randomization, urn randomization, the asymptotic maximal procedure or minimization should be used.

The study depended on publicly available datasets and could not perform a detailed review of the randomization protocols for all included trials. Could you clarify if future research with access to detailed randomization logs and protocols might yield different results? Additionally, are there specific data points that, if available, would strengthen the conclusions drawn?

We have added a table (Table 2) that outlines necessary reporting to mitigate concerns for selection bias. The most necessary, single piece of data necessary to assess for selection bias, under permuted block randomization, is the randomization schedule.

Given the variability in randomization approaches across the RCTs analyzed, could you elaborate on how these differences might inherently impact baseline heterogeneity? It would be beneficial to explain if specific randomization methods were more prone to bias, as this could guide future research designs.

We would like to clarify that only one trial of the selected randomized trials did not use permuted block randomization while all other did. As such, assessing if different randomization procedures have a greater chance of contributing to baseline heterogeneity was not possible in our included studies. However, we agree that a larger meta-epidemiological study is worthwhile given the frequency that this method has been employed.

Reviewer 3 Comments Response

What exactly was the signal to point to actual risk?

Was it insufficient stratification factors up front leading to imbalanced overall? The last column of Table S1 and S4 are after the fact to show imbalance perhaps. Is there any other indicator available?

We appreciate your comments which improved the quality of our manuscript. Please see the new supplementary table 8 (S8 Table). Please also see the revised S1 and S2 Figures.

Historically, covariate imbalances is what raised concerns of selection bias in the NINDS rt-PA Stroke Study and ECASS-3. To fully answer your question, please see the newly added STable 8 and citation 14. There were both risk factors, a plausible mechanism and evidence from trial data that made selection bias manifest. We cannot, in the same way, substantiate this claim for ECASS-3 due to lack of publicly available study protocols, trial data and regulatory documentation. You are correct that imbalances are indeed after the fact (outcome of selection bias), although they are often the first clue to the presence of selection bias.

While intuitively stratification might reduce covariate imbalances, stratification factors cannot avert selection bias which is systematic error in group allocation. In certain cases, such as stratification by site of recruitment, predictability in next group allocation may be higher and worsen the chance of selection bias.

See revised S1-S3 figures.

Attachment

Submitted filename: response_to_reviewers.docx

pone.0315342.s013.docx^{(20.9KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0315342.r004

Decision Letter 1

Cem Bilgin

4 Feb 2025

A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analyses

PONE-D-24-20124R1

Dear Dr. Torrealba-Acosta,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Cem Bilgin

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0315342.r005

Acceptance letter

Cem Bilgin

PONE-D-24-20124R1

PLOS ONE

Dear Dr. Torrealba-Acosta,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Cem Bilgin

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Table. Risk of Bias Arising from the Randomization Process in the NINDS rt-PA Stroke Study.

(DOCX)

pone.0315342.s001.docx^{(22.7KB, docx)}

S2 Table. Risk of Bias Arising from the Randomization Process in the ATLANTIS-A and ATLANTIS-B Randomized Clinical Trials.

(DOCX)

pone.0315342.s002.docx^{(14.5KB, docx)}

S3 Table. Risk of Bias Arising from the Randomization Process in the ECASS-2 Randomized Clinical Trial.

(DOCX)

pone.0315342.s003.docx^{(14.8KB, docx)}

S4 Table. Risk of Bias Arising from the Randomization Process in the ECASS-3 trial.

Abbreviations: NIHSS, National Institute of Health Stroke Scale.

(DOCX)

pone.0315342.s004.docx^{(14.7KB, docx)}

S5 Table. Risk of Bias Arising from the Randomization Process in the EPITHET trial.

(DOCX)

pone.0315342.s005.docx^{(14.6KB, docx)}

S6 Table. Risk of Bias Arising from the Randomization Process in the IST-3 trial.

(DOCX)

pone.0315342.s006.docx^{(14.5KB, docx)}

S7 Table. Mean differences in age, weight, and National Institute of Health Stroke Scale scores; and the associated t-statistic for all included trials.

(DOCX)

pone.0315342.s007.docx^{(14.9KB, docx)}

S8 Table. Overview of selection bias in the NINDS rt-PA Stroke Study.

(DOCX)

pone.0315342.s008.docx^{(16KB, docx)}

S1 Fig. The I² point estimate (orange dot) and associated 95% confidence intervals for the age meta-analysis.

(TIF)

pone.0315342.s009.tif^{(334.4KB, tif)}

S2 Fig. The I² point estimate (orange dot) and associated 95% confidence intervals for the weight meta-analysis.

(TIF)

pone.0315342.s010.tif^{(256.2KB, tif)}

S3 Fig. The I² point estimate (orange dot) and associated 95% confidence intervals for the NIHSS meta-analysis.

(TIF)

pone.0315342.s011.tif^{(305.6KB, tif)}

Attachment

Submitted filename: response_to_reviewers.docx

pone.0315342.s013.docx^{(20.9KB, docx)}

Data Availability Statement

[pone.0315342.ref001] 1.Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2019;50(12):e344–418. [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref002] 2.Borzak S, Ridker PM. Discordance between meta-analyses and large-scale randomized, controlled trials: examples from the management of acute myocardial infarction. American College of Physicians. 1995, p. 873–7. [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref003] 3.Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomized trials. BMJ. 2019;366. [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref004] 4.Berger VW, Weinstein S. Ensuring the comparability of comparison groups: is randomization enough?. Control Clin Trials. 2004;25(5):515–24. doi: 10.1016/j.cct.2004.04.001 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref005] 5.Berger V. Selection bias and covariate imbalances in randomized clinical trials. John Wiley & Sons; 2007. [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref006] 6.Hicks A, Fairhurst C, Torgerson DJ. A simple technique investigating baseline heterogeneity helped to eliminate potential bias in meta-analyses. J Clin Epidemiol. 2018;9555–62. doi: 10.1016/j.jclinepi.2017.10.001 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref007] 7.Clark L, Fairhurst C, Cook E, Torgerson DJ. Important outcome predictors showed greater baseline heterogeneity than age in two systematic reviews. J Clin Epidemiol. 2015;68(2):175–81. doi: 10.1016/j.jclinepi.2014.09.023 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref008] 8.López-Ruiz C, Orjuela JC, Rojas-Gualdrón DF, Jimenez-Arango M, Ríos JF de L, Vásquez-Trespalacios EM, et al. Efficacy of transversus abdominis plane block in the reduction of pain and opioid requirement in laparoscopic and robot-assisted hysterectomy: a systematic review and meta-analysis. Rev Bras Ginecol Obstet. 2022;44(1):55–66. doi: 10.1055/s-0041-1740595 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref009] 9.Ker K, Shakur H, Roberts I. Does tranexamic acid prevent postpartum haemorrhage? A systematic review of randomised controlled trials. BJOG. 2016;123(11):1745–52. doi: 10.1111/1471-0528.14267 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref010] 10.Mott A, Bradley T, Wright K, Cockayne ES, Shearer MJ, Adamson J, et al. Effect of vitamin K on bone mineral density and fractures in adults: an updated systematic review and meta-analysis of randomised controlled trials. Osteoporos Int. 2019;30(8):1543–59. doi: 10.1007/s00198-019-04949-0 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref011] 11.Rex SS, Kottam L, McDaid C, Brealey S, Dias J, Hewitt CE, et al. Effectiveness of interventions for the management of primary frozen shoulder : a systematic review of randomized trials. Bone Jt Open. 2021;2(9):773–84. doi: 10.1302/2633-1462.29.BJO-2021-0060.R1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref012] 12.Hulshof T, Zuidema S, van Meer P, Gispen-de Wied C, Luijendijk H. Baseline imbalances and clinical outcomes of atypical antipsychotics in dementia: A meta-epidemiological study of randomized trials. Int J Methods Psychiatr Res. 2019;28(1):e1757. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref013] 13.Alper BS, Foster G, Thabane L, Rae-Grant A, Malone-Moses M, Manheimer E. Thrombolysis with alteplase 3-4.5 hours after acute ischaemic stroke: trial reanalysis adjusted for baseline imbalances. BMJ Evid Based Med. 2020;25(5):168–71. doi: 10.1136/bmjebm-2020-111386 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref014] 14.Garg R, Mickenautsch S. Risk of selection bias assessment in the NINDS rt-PA stroke study. BMC Med Res Methodol. 2022;22(1):172. doi: 10.1186/s12874-022-01651-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref015] 15.Ingall TJ, O’Fallon WM, Asplund K, Goldfrank LR, Hertzberg VS, Louis TA, et al. Findings from the reanalysis of the NINDS tissue plasminogen activator for acute ischemic stroke treatment trial. Stroke. 2004;35(10):2418–24. doi: 10.1161/01.STR.0000140891.70547.56 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref016] 16.Stroke Thrombolysis Trialists’ Collaborative Group. Details of a prospective protocol for a collaborative meta-analysis of individual participant data from all randomized trials of intravenous rt-PA vs. control: statistical analysis plan for the Stroke Thrombolysis Trialists’ Collaborative meta-analysis. Int J Stroke. 2013;8(4):278–83. doi: 10.1111/ijs.12040 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref017] 17.Whiteley WN, Emberson J, Lees KR, Blackwell L, Albers G, Bluhmki E, et al. Risk of intracerebral haemorrhage with alteplase after acute ischaemic stroke: a secondary analysis of an individual patient data meta-analysis. Lancet Neurol. 2016;15(9):925–33. doi: 10.1016/S1474-4422(16)30076-X [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref018] 18.Emberson J, Lees KR, Lyden P, Blackwell L, Albers G, Bluhmki E, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet. 2014;384(9958):1929–35. doi: 10.1016/S0140-6736(14)60584-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref019] 19.Lees KR, Emberson J, Blackwell L, Bluhmki E, Davis SM, Donnan GA, et al. Effects of Alteplase for Acute Stroke on the Distribution of Functional Outcomes: A Pooled Analysis of 9 Trials. Stroke. 2016;47(9):2373–9. doi: 10.1161/STROKEAHA.116.013644 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref020] 20.Bluhmki E, Danays T, Biegert G, Hacke W, Lees KR. Alteplase for acute ischemic stroke in patients aged >80 years: pooled analyses of individual patient data. Stroke. 2020;51(8):2322–31. doi: 10.1161/STROKEAHA.119.028396 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref021] 21.Hacke W, Lyden P, Emberson J, Baigent C, Blackwell L, Albers G, et al. Effects of alteplase for acute stroke according to criteria defining the European Union and United States marketing authorizations: Individual-patient-data meta-analysis of randomized trials. Int J Stroke. 2018;13(2):175–89. doi: 10.1177/1747493017744464 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref022] 22.Davis SM, Donnan GA, Parsons MW, Levi C, Butcher KS, Peeters A, et al. Effects of alteplase beyond 3 h after stroke in the Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET): a placebo-controlled randomised trial. Lancet Neurol. 2008;7(4):299–309. doi: 10.1016/S1474-4422(08)70044-9 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref023] 23.Clark WM, Wissman S, Albers GW, Jhamandas JH, Madden KP, Hamilton S. Recombinant tissue-type plasminogen activator (Alteplase) for ischemic stroke 3 to 5 hours after symptom onset. The ATLANTIS Study: a randomized controlled trial. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. JAMA. 1999;282(21):2019–26. doi: 10.1001/jama.282.21.2019 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref024] 24.Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g) : results of a double-blind, placebo-controlled, multicenter study. Thromblytic therapy in acute ischemic stroke study investigators. Stroke. 2000;31(4):811–6. doi: 10.1161/01.str.31.4.811 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref025] 25.Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, et al. Randomised double-blind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet. 1998;352(9136):1245–51. doi: 10.1016/s0140-6736(98)08020-9 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref026] 26.Hacke W, Kaste M, Bluhmki E, Brozman M, Dávalos A, Guidetti D, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med. 2008;359(13):1317–29. doi: 10.1056/NEJMoa0804656 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref027] 27.IST-3 collaborative group, Sandercock P, Wardlaw JM, Lindley RI, Dennis M, Cohen G, et al. The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet. 2012;379(9834):2352–63. doi: 10.1016/S0140-6736(12)60768-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref028] 28.Group S-P, NIoN D. Tissue plasminogen activator for acute ischemic stroke. New Eng J Med. 1995;333(24):1581–8. [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref029] 29.Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA. 1995;274(13):1017–25. doi: 10.1001/jama.1995.03530130023023 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref030] 30.Mandava P, Kalkonde YV, Rochat RH, Kent TA. A matching algorithm to address imbalances in study populations: application to the National Institute of Neurological Diseases and Stroke Recombinant Tissue Plasminogen Activator acute stroke trial. Stroke. 2010;41(4):765–70. doi: 10.1161/STROKEAHA.109.574103 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref031] 31.Jackson D, Bowden J. Confidence intervals for the between-study variance in random-effects meta-analysis using generalised heterogeneity statistics: should we use unequal tails? BMC Med Res Methodol. 2016;16(1):118. doi: 10.1186/s12874-016-0219-y [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref032] 32.Berger VW, Christophi CA. Randomization technique, allocation concealment, masking, and susceptibility of trials to selection bias. J Mod App Stat Meth. 2003;2(1):80–6. doi: 10.22237/jmasm/1051747680 [DOI] [Google Scholar]

[pone.0315342.ref033] 33.Stang A , Baethge C. Imbalance p values for baseline covariates in randomized controlled trials: a last resort for the use of p values? A pro and contra debate. Clinical Epidemiology. 2018:531-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref034] 34.Berger VW, He X. Concealing Block Sizes Is Not Sufficient. Clin Orthop Surg. 2015;7(3):422–3. doi: 10.4055/cios.2015.7.3.422 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref035] 35.Berger VW. Varying the block size does not conceal the allocation. J Crit Care. 2006;21(2):229; author reply 229-30. doi: 10.1016/j.jcrc.2006.01.002 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref036] 36.Clark L, Burke L, Margaret Carr R, Coleman E, Roberts G, Torgerson DJ. A review found small variable blocking schemes may not protect against selection bias in randomized controlled trials. J Clin Epidemiol. 2022;14190–8. doi: 10.1016/j.jclinepi.2021.09.009 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref037] 37.Khatri P, Kleindorfer DO, Devlin T, Sawyer RN Jr, Starr M, Mejilla J, et al. Effect of alteplase vs aspirin on functional outcome for patients with acute ischemic stroke and minor nondisabling neurologic deficits: The PRISMS Randomized Clinical Trial. JAMA. 2018;320(2):156–66. doi: 10.1001/jama.2018.8496 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref038] 38.Garg R, Mickenautsch S. Risk of selection bias assessment in the NINDS-rtPA stroke study: a reply to raised concerns. Research Gate2023 Available from: https://www.researchgate.net/publication/375960388_Risk_of_selection_bias_assessment_in_the_NINDS-rtPA_stroke_study_a_reply_to_raised_concerns [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref039] 39.Berger VW. The reverse propensity score to detect selection bias and correct for baseline imbalances. Stat Med. 2005;24(18):2777–87. doi: 10.1002/sim.2141 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref040] 40.Cantrill SV, Brown MD, Brecher D, Diercks DB, Gemme SR, Gerardo CJ, et al. Clinical policy: use of intravenous tissue plasminogen activator for the management of acute ischemic stroke in the emergency department. Ann Emerg Med. 2015;66(3):322-333.e31. doi: 10.1016/j.annemergmed.2015.06.031 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref041] 41.Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. doi: 10.1016/j.jclinepi.2010.04.026 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref042] 42.Xiong Y, Campbell BCV, Schwamm LH, Meng X, Jin A, Parsons MW, et al. Tenecteplase for Ischemic Stroke at 4.5 to 24 Hours without Thrombectomy. N Engl J Med. 2024;391(3):203–12. doi: 10.1056/NEJMoa2402980 [DOI] [PubMed] [Google Scholar]

[pone.0315342.ref043] 43.Garg R. Tenecteplase for ischemic stroke at 4.5 to 24 hours without thrombectomy. PubPeer2024 Available from: https://pubpeer.com/publications/30D56099E89EF0E310B6D48C2679EE# [DOI] [PubMed]

[pone.0315342.ref044] 44.Lee JJ, Xuemin Gu, Suyu Liu. Bayesian adaptive randomization designs for targeted agent development. Clin Trials. 2010;7(5):584–96. doi: 10.1177/1740774510373120 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref045] 45.Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ. 2013;346:f2304. doi: 10.1136/bmj.f2304 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref046] 46.Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care. 2013;17(1):R2. doi: 10.1186/cc11919 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref047] 47.Garg R. Methodological survey of missing outcome data in an alteplase for ischemic stroke meta-analysis. Acta Neurol Scand. 2022;146(3):252–7. doi: 10.1111/ane.13656 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0315342.ref048] 48.Mickenautsch S, Yengopal V. Trial number and sample size do not affect the accuracy of the I2-point estimate for testing selection bias risk in meta-analyses. Cureus. 2024;16(4).author billing [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

A methodological assessment of randomization integrity in alteplase for acute ischemic stroke individual patient data meta-analyses

Ravi Garg

Gabriel Torrealba-Acosta

Steffen Mickenautsch

Vance W Berger

Roles

Abstract

Objectives

Methods

Results

Conclusions:

Introduction

Methods

Study design and population

Data analysis

Measures

Results

Randomization methods reporting and risk-of-bias arising from the randomization process

Table 1. Randomization reporting and risk of bias for included randomized trials.

Heterogeneity in baseline variables

Fig 1. Forest plot of mean differences for baseline age for included trials.

Fig 2. Forest plot of mean differences for baseline weight for included trials.

Fig 3. Forest plot of mean difference for baseline NIHSS for included trials.

Quantification of selection bias

Fig 4. Pooled absolute risk difference for all included trials.

Fig 5. The pooled effect of treatment with and without the NINDS rt-PA Stroke Study and ECASS-3 RCT.

Discussion

Table 2. Recommendations for improved randomization reporting in randomized controlled trials. Adapted from Berger and Cristophi [32].

Limitations

Conclusions

Supporting Information

Acknowledgments

Data Availability

Funding Statement

References

Author response to Decision Letter 0

Decision Letter 0

Cem Bilgin

Roles

Author response to Decision Letter 1

Decision Letter 1

Cem Bilgin

Roles

Acceptance letter

Cem Bilgin

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases