Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2024 Dec 12;19(12):e0303315. doi: 10.1371/journal.pone.0303315

Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationale

Phillip Staibano 1,2,*, Tyler McKechnie 2,3, Alex Thabane 2, Daniel Olteanu 4, Keean Nanji 2,5, Han Zhang 1, Carole Lunny 6, Michael Au 1, Michael K Gupta 1, Jesse D Pasternak 7, Sameer Parpia 2, JEM (Ted) Young 1, Mohit Bhandari 2,8
Editor: Sathish Muthu9
PMCID: PMC11637403  PMID: 39666716

Abstract

Background

Randomized-clinical trials (RCTs) are the gold-standard for comparing health care interventions, but can be limited by early termination, feasibility issues, and prolonged time to trial reporting. Adaptive clinical trials (ACTs), which are defined by pre-planned modifications and analyses that occur after starting patient recruitment, are gaining popularity as they can streamline trial design and time to reporting. As adaptive methodologies continue to be adopted by researchers, it will be critical to develop a risk of bias tool that evaluates the unique methodological features of ACTs so that their quality can be improved and standardized for the future. In our proposed methodological review, we will develop a list of risk of bias items and concepts, so that a risk of bias tool specific to ACTs can be developed.

Methods and analysis

We will perform a systematic database search to capture studies that have proposed or reviewed items pertaining to methodological risk, bias, and/or quality in ACTs. We will perform a comprehensive search of citation databases, such as Ovid MEDLINE, EMBASE, CENTRAL, the Cochrane library, and Web of Science, in addition to multiple grey literature sources to capture published and unpublished literature related to studies evaluating the methodological quality of ACTs. We will also search methodological registries for any risk of bias tools for ACTs. All screening and review stages will be performed in duplicate with a third senior author serving as arbitrator for any discrepancies. For all studies of methodological quality and risk of bias, we will extract all pertinent bias items, concepts, and/or tools. We will combine conceptually similar items in a descriptive manner and classify them as referring to bias or to other aspects of methodological quality, such as reporting. We will plan to generate pertinent risk of bias items to generate a candidate tool that will undergo further refinement, testing, and validation in future development stages.

Ethics and dissemination

This review does not require ethics approval as human subjects are not involved. As mentioned previously, this study is the first step in developing a tool to evaluate the risk of bias and methodological quality of ACTs. The findings of this review will inform a Delphi study and the development of a risk of bias tool for ACTs. We plan on publishing this review in a peer-reviewed journal and to present these findings at international scientific conferences.

Introduction

Evidence-based medicine has revolutionized the development of clinical practice guidelines and decision making in healthcare [1]. Randomized-controlled trials (RCTs) are the gold-standard for comparing the effectiveness and safety of novel healthcare interventions [2]. Conventional RCTs, however, can be burdened by high costs, early termination due to feasibility issues, and an overly rigid design that does not permit adjustments for unforeseen challenges [3]. These issues are amplified in surgical trials and as such, the annual number of published surgical trials remains stagnant [4, 5]. As a response to these challenges, researchers have begun using adaptive trial designs, which allow for dynamic protocol changes after beginning patient recruitment. Adaptive clinical trials (ACTs) utilize at least one pre-planned interim analysis to modify the protocol of an ongoing trial while maintaining integrity and validity of the data collected [6]. Trial adaptations performed following an interim analysis include sample size re-calculation, editing the number of treatment arms, amending allocation ratios, and/or terminating a trial early for success or lack of efficacy. Adaptive designs can improve the trial running process by optimizing patient recruitment, combining clinical trial stages, minimizing sample size, and accelerating time to trial analysis and reporting [6]. For instance, the TAILoR trial of telmisartan in HIV employed an interim analysis at half maximal patient recruitment and dropped the most ineffective medication dosage group based on a pre-specified efficacy threshold [7]. Moreover, adaptive designs streamlined the clinical trial process during COVID-19 by optimizing the number of therapies evaluated and minimizing the number of patients enrolled in each trial [8]. Adopting ACTs when prolonged RCTs are impractical may also reduce necessary funding, thereby overcoming barriers to conducting trials in developing nations [9]. Stakeholders, however, report that adaptive trial designs remain nebulous with practical barriers, including high bias potential, ethical concerns, and a lack of knowledge dissemination amongst trialists [10]. Other challenges include the need for a robust network of researchers and biostatisticians to ensure that the, often complex, trial protocol is well-planned and has undergone rigorous statistical stimulation prior to beginning patient recruitment [11]. In addition, statistical software required for adaptive methods is limited in its accessibility and expensive. Clinicians, researchers, and funding agencies are not well-versed in adaptive design terminology and practices, nor is there proper standardization in adaptive trial reporting [12].

In 2020, CONSORT published an extension for adaptive trials to guide ACT reporting [13]. These guidelines ACT-specific methodological components such as pre-planned interim analyses and sample size estimation (and re-estimation) descriptions [13]. In conjunction with CONSORT reporting guidelines, a validated risk of bias tool developed in a similar manner to the Cochrane risk-of-bias 2.0 tool [14], may improve the design of ACTs and the quality of future meta-analyses combing ACTs. Risk of bias tools are designed for specific study designs (e.g. RCTs) and help to promote methodological transparency and reproducibility while minimizing bias, so results can be accurately interpreted and soundly applied to patient care. For conventional RCTs, there exist several tools and checklists to guide reporting or evaluate quality and risk of bias (Table 1) [14–27]. There is no existing risk of bias tool to evaluate the methodological limitations of ACTs, which is of particular importance due to the potential for ACTs to be impacted by bias if not soundly designed [28]. It is for this reason that we have decided to embark upon creating a novel risk of bias tool to improve the quality of future ACTs and meta-analyses of ACTs.

Table 1. Tools and checklists to aid in randomized controlled trial methodological quality.

Adapted from Lunny et al. (2021) [31].

Tool purpose Examples of tools or checklists Description of the example tool or checklist Available tool for ACTs
Assess the quality of published RCTs CASP-RCT [24], CEBM-RCT [16], Jadad scale [15], JBI checklist [25], LEGEND [27], TRACT [17], SIGN [26] These tools to assist in conducting and reviewing published RCTs that consist of multiple domains addressing trial design and internal validity, result reporting, and interpretability. None
Assess the risk of bias of published RCTs Cochrane RoB tool (2.0) [14] Cochrane RoB 2.0 is a five-domain domain tool that evaluates bias arising from randomization, protocol deviations, missing outcome data, outcome measurement, and result reporting. Studies are given an overall bias risk score of low, high, or some concerns. None
Assess the external validity of RCTs None [20] None None
Guidance for the complete reporting of RCTs CONSORT [21] CONSORT is a 25-item checklist that assists with transparent reporting of RCT design, analysis, and interpretation. CONSORT-ACE [13]
Guidance checklist for clinical readers of RCTs Godin et al. (2011) [23], Nimavat et al. (2020) [22] These resources provide a short checklist of simple questions addressing methodology, reporting, and interpretation for clinicians to keep in mind when appraising RCTs. Ferreira et al. (2020) [18], Park et al. (2020) [19]
Open in a new tab

CASP-RCT, critical appraisal skills program–randomized controlled trials; CEBM-RCT, Centre for Evidence-Based Medicine–randomized controlled trials; CONSORT, consolidated standards of reporting trials; CONSORT-ACE, consolidated standards of reporting trials adaptive designs extension; JBI, Joanne Briggs Institute; LEGEND, Let Evidence Guide Every New Decision; RCT, randomized controlled trial; RoB, risk of bias; SIGN, Scottish Intercollegiate Guidelines, Network; TRACT, Trustworthiness in randomised controlled trials

Our proposed methodological review has two main objectives: (1) to identify and describe any current risk of bias items, tools, or checklists specific for ACTs and (2) to compile a list of risk of bias items and concepts that can be used to develop a risk of bias tool for ACTs. We will develop our risk of bias tool for ACTs in accordance with the framework described by Whiting et al. (2017) [29].

Materials and methods

Study design

We present a protocol to describe the rationale for performing a methodological review of ACTs and generate a list of risk of bias items and concepts related to ACTs. We will follow the methodological framework proposed by Whiting et al. (2017) and Sanderson et al. (2007) [29, 30]. This protocol was written with guidance from a methodology review protocol published by Lunny and colleagues, who set out to create a novel risk of bias tool for network meta-analyses [31]. As described by Lunny and colleagues, subsequent steps in creating a risk of bias tool will include (1) a Delphi survey and panel to select, refine, and compile bias items into a single candidate tool; (2) a pilot test to further refine the proposed tool; and (3) a knowledge translation strategy to disseminate the final risk of bias tool [31]. With regards to the Delphi study, we will plan to first distribute a knowledge survey to methodologists and content experts to gather their opinion on an ACT risk of bias tool and how it should be utilized and disseminated. Next, a pre-selected steering panel will generate a candidate list of risk of bias items that will be distributed in multiple rounds to a Delphi panel that will rate the utility of including each item and/or concept in the candidate tool. Pilot testing will include the evaluation of tool useability, efficiency, and comprehension among content experts in adaptive methodologies and trial design. Our final knowledge translation strategy will include publication and presentation of the final tool, housing the tool in an accessible website, and providing training sessions and webinars to future tool users [29]. These steps will be further addressed in future studies as we progress through this framework in developing this proposed ACT risk of bias tool. We did perform this methodological review protocol in accordance with the PRISMA-P checklist [32].

Eligibility

There will be two types of studies included in this methodological review (Table 2). Study type 1 will be studies that describe items and/or concepts related to bias, reporting, or methodological quality of ACTs. We will retain all items related to methodological bias and/or reporting as they may be able to be translated into a risk-of-bias tool. Study type 2 will be studies that assess the methodological quality, or risk of bias, of ACTs using criteria that focus on methodological features specific to ACTs. Both study types will be analyzed with the goal of collating bias items and/or concepts. We will also review and gather related items/concepts from any published risk of bias tools or reporting quality tools used for conventional RCTs (Table 1). We will include all articles with any publication status and written in any language. We will focus on methodological studies of ACTs and so, we will not be evaluating published ACTs or studies based on disease type, clinical populations, or tested interventions. In cases where the co-authors are not fluent or review authors are unable to understand the study text, we will utilize Google Translate (Mountain View, CA, USA).

Table 2. Eligible study types for methodological review.

Study type 1: Studies that describe items and concepts relating to bias, reporting, and/or methodological quality in adaptive clinical trials
Inclusion criteria Exclusion criteria

  • Any publication type (e.g., primary study, published abstract, thesis dissertation, pre-print manuscript)

  • Published in any language.

  • Any study design (i.e., randomized trial, meta-analysis, scoping review, systematic review, narrative review, author editorial, letter to the editor).

  • Mention, describe, evaluate at least one risk of bias, reporting, or methodological quality item, concept, tool, or checklist pertaining to ACTs.

  • Reported methodological item or tool not mentioned or described in the context of adaptive trial designs.
Study type 2: Studies that assess the methodological quality or risk of bias of adaptive clinical trials
Inclusion criteria Exclusion criteria

  • Any publication type (e.g., primary study, published abstract, thesis dissertation, pre-print manuscript)

  • Published in any language.

  • Any study design (i.e., randomized trial, meta-analysis, scoping review, systematic review, narrative review, author editorial, letter to the editor).

  • Evaluate or describe the methodological quality or risk of bias of at least one ACT

  • Reported methodological item or tool not mentioned or described in the context of adaptive trial designs.
Open in a new tab

ACT, adaptive clinical trials

Search strategy

All databases to be used in this review were selected with guidance from Lunny et al. (2021) [31]. We will search all databases with no language or publication type limits. We will search the following databases: MEDLINE (Ovid), CINAHL, EMBASE (Ovid), the Cochrane library, the Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science, BIOSIS, Derwent Innovations Database, and KCI. We will also search clinical trials registries including clinicaltrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP). We will search the following grey literature databases and resources: the EQUATOR network, dissertation abstracts, websites of evidence synthesis organizations (e.g., Campbell Collaboration, Cochrane Multiple Treatments Group, CADTH, NICE-DSU, Health Technology Assessment International (HTAi), Pharmaceutical Benefits Advisory Committee, Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, European Network for Health Technology Assessment, Guidelines International Network, ISPOR, International Network of Agencies for Health Technology Assessment, and JBI), and methods collections (e.g., Cochrane Methodology Register, AHRQ Effective Healthcare Programme). We will also search LIGHTS and LATITUDES (https://www.latitudes-network.org/), which are two methodological registries that capture guidance and validity assessment tools, respectively [33]. All online registries will be searched using the following terms “adaptive clinical trial”, “bias”, and/or risk of bias”. The words found within the titles, abstracts, and MeSH terms of relevant articles were used to develop focused search strategies for each database. Reference lists of studies found will also be searched for additional papers to be included. The MEDLINE search will be validated for 10 studies identified by senior authors prior to screening. Eligibility screening will only begin after these 10 trials are identified from the search strategy. All database search strategies are described in S1 File.

The search strategy will be generated by two authors (P.S. and D.O.) alongside a librarian specialist. It will be generated and reviewed in accordance with PRESS (Peer Review Electronic Search Strategies) guidelines [34]. Any concerns with search strategy generation will be raised with a senior methodologist (M.B.). The database search will be conducted without limitations to publication type, status, language, or date to identify existing tools or articles.

Screening and data extraction

First, we will pilot eligibility criteria in Microsoft Excel (Redmond, WA, USA) by evaluating a sample of 25 citations amongst two independent reviewers. If high agreement is achieved (≥70%), then we will continue to abstract screening with two reviewers. If less than high agreement is achieved, then the eligibility criteria will be re-examined, and additional teaching sessions will be provided to reviewers. All screening and full-text review will be conducted using the web-based application, Covidence (http://www.covidence.org; Melbourne, Australia). Study titles and abstracts will then be assessed for relevance and eligibility. All screening and full-text review will be performed in accordance with PRISMA guidelines (S2 File). Any disagreements identified during these screening and review stages will be resolved via discussion until consensus is reached. A third senior reviewer will arbitrate if screening or full-text review disagreements cannot be resolved.

A data extraction form will be generated using Microsoft Excel and piloted by reviewers on five included studies. Two authors will extract data from all included studies. We will first categorize all sources based on our eligibility criteria and we will extract author details, publication year, and study type. For all studies that identified bias items, tools, or quantified methodological bias in ACTs, we will generate the following list of headings: type of tool (e.g., tool, scale, checklist, or domain-based tool); scope of the tool; number of items within the tool; domains within the tool; whether the item relates to reporting or methodological quality; ratings of items and domains within the tool; methods used to develop the tool and the availability of an “explanation and elaboration”. These fields were all derived from Lunny et al. (2021) and Page et al. (2018) [31, 35]. Data will be extracted on items that are relevant to ACTs and all items will initially be extracted verbatim.

Data analysis and reporting

All studies evaluating methodological quality, and/or proposing bias tools, items, concepts, checklists will be collated. These studies will undergo descriptive analyses based upon previously extracted fields. All bias items will all be mapped to corresponding domains within the CONSORT-ACE guidelines, as this is the only known quality tool specific to ACT features. If no risk-of-bias tools or relevant items are identified in our review, then we will plan to hypothesize items and formulate a candidate tool. We will develop a candidate tool that will undergo refinement by the authors, and in subsequent steps will be further evaluated and refined using a Delphi consensus method prior to undergoing validation and testing. Currently, our plan is to create a standalone risk of bias tool for ACTs that will gather inspiration from the domains of the Cochrane RoB 2.0 tool. But, as we proceed to the knowledge user survey, Delphi process, and candidate tool development phases, we will further evaluate the feasibility and advantages of instead developing an extension to the Cochrane RoB 2.0. All statistical analyses will be performed using R software (version 4.3.2, Vienna, Austria).

Public and public involvement

Patient or the public were not involved in the design of this research protocol.

Discussion

A comprehensive risk of bias tool is needed to improve the reproducibility and transparency of future ACTs and ACT meta-analyses. A recent scoping review demonstrated that ACTs adhere poorly to the reporting recommendations published in the CONSORT-ACE statement [12]. We are, therefore, going to use the framework developed by Whiting and colleagues to develop a new risk of bias tool to improve the quality of ACTs [29]. A risk of bias tool for ACTs is needed since these designs often demonstrate key methodological differences from conventional RCTs, such as interim analyses and adaptive decisions made after the trial has begun patient recruitment [6]. The current repertoire of risk of bias tools used for conventional RCTs do not possess domains that address these unique adaptive design features. Moreover, adaptive decisions made during the running of a clinical trial can increase methodological bias in the final trial analysis, thus emphasizing the need for a risk of bias tool to be used when evaluating ACTs. Potential limitations include missing any published methodology studies or ROB tools of ACTs, but we will counteract this via a broad search strategy of peer-reviewed literature databases, grey literature databases, and methodological tool repositories. As computational technologies continue to improve, their role in generating adaptive trial paradigms and performing statistical simulations may revolutionize the future of trial design and medical innovation [36]. We, must, therefore ensure that methodological tools are developed at a similar pace, so that these novel trial designs are standardized, transparent, reproducible, and interpretable for the future.

Supporting information

S1 File. Search strategies.

(DOCX)

pone.0303315.s001.docx (16.1KB, docx)
S2 File. PRISMA flow diagram for prospective article screening and full-text review.

(DOCX)

pone.0303315.s002.docx (56.6KB, docx)
S3 File. Completed PRISMA-P checklist.

(DOC)

pone.0303315.s003.doc (82.5KB, doc)

Data Availability

No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

Funding Statement

The author(s) received no specific funding for this work.

References

  • 1.Masic I, Miokovic M, Muhamedagic B. Evidence based medicine—new approaches and challenges. Acta Inform Med. 2008;16(4):219–25. doi: 10.5455/aim.2008.16.219-225 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Sibbald B, Roland M. Understanding controlled trials. Why are randomised controlled trials important? BMJ. 1998;316(7126):201. doi: 10.1136/bmj.316.7126.201 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Nichol AD, Bailey M, Cooper DJ, Polar, Investigators EPO. Challenging issues in randomised controlled trials. Injury. 2010;41 Suppl 1:S20–3. Epub 20100422. doi: 10.1016/j.injury.2010.03.033 . [DOI] [PubMed] [Google Scholar]
  • 4.Pronk AJM, Roelofs A, Flum DR, Bonjer HJ, Abu Hilal M, Dijkgraaf MGW, et al. Two decades of surgical randomized controlled trials: worldwide trends in volume and methodological quality. Br J Surg. 2023;110(10):1300–8. doi: 10.1093/bjs/znad160 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, Bhangu A. Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ. 2014;349:g6870. Epub 20141209. doi: 10.1136/bmj.g6870 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Pallmann P, Bedding AW, Choodari-Oskooei B, Dimairo M, Flight L, Hampson LV, et al. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018;16(1):29. Epub 20180228. doi: 10.1186/s12916-018-1017-7 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Pushpakom SP, Taylor C, Kolamunnage-Dona R, Spowart C, Vora J, Garcia-Finana M, et al. Telmisartan and Insulin Resistance in HIV (TAILoR): protocol for a dose-ranging phase II randomised open-labelled trial of telmisartan as a strategy for the reduction of insulin resistance in HIV-positive individuals on combination antiretroviral therapy. BMJ Open. 2015;5(10):e009566. Epub 20151015. doi: 10.1136/bmjopen-2015-009566 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Stallard N, Hampson L, Benda N, Brannath W, Burnett T, Friede T, et al. Efficient Adaptive Designs for Clinical Trials of Interventions for COVID-19. Stat Biopharm Res. 2020;12(4):483–97. Epub 20200729. doi: 10.1080/19466315.2020.1790415 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Alemayehu C, Mitchell G, Nikles J. Barriers for conducting clinical trials in developing countries- a systematic review. Int J Equity Health. 2018;17(1):37. Epub 20180322. doi: 10.1186/s12939-018-0748-6 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Madani Kia T, Marshall JC, Murthy S. Stakeholder perspectives on adaptive clinical trials: a scoping review. Trials. 2020;21(1):539. Epub 20200617. doi: 10.1186/s13063-020-04466-0 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Zhu H, Wong WK. An Overview of Adaptive Designs and Some of Their Challenges, Benefits, and Innovative Applications. J Med Internet Res. 2023;25:e44171. Epub 20231016. doi: 10.2196/44171 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Purja S, Park S, Oh S, Kim M, Kim E. Reporting quality was suboptimal in a systematic review of randomized controlled trials with adaptive designs. J Clin Epidemiol. 2023;154:85–96. Epub 20221215. doi: 10.1016/j.jclinepi.2022.12.010 . [DOI] [PubMed] [Google Scholar]
  • 13.Dimairo M, Pallmann P, Wason J, Todd S, Jaki T, Julious SA, et al. The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design. Trials. 2020;21(1):528. Epub 20200617. doi: 10.1186/s13063-020-04334-x . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Sterne JAC, Savovic J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. Epub 20190828. doi: 10.1136/bmj.l4898 . [DOI] [PubMed] [Google Scholar]
  • 15.Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12. doi: 10.1016/0197-2456(95)00134-4 . [DOI] [PubMed] [Google Scholar]
  • 16.Haile ZT. Critical Appraisal Tools and Reporting Guidelines. J Hum Lact. 2022;38(1):21–7. Epub 20211118. doi: 10.1177/08903344211058374 . [DOI] [PubMed] [Google Scholar]
  • 17.Mol BW, Lai S, Rahim A, Bordewijk EM, Wang R, van Eekelen R, et al. Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot. Res Integr Peer Rev. 2023;8(1):6. Epub 20230620. doi: 10.1186/s41073-023-00130-8 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Ferreira D, Barthoulot M, Pottecher J, Torp KD, Diemunsch P, Meyer N. A consensus checklist to help clinicians interpret clinical trial results analysed by Bayesian methods. Br J Anaesth. 2020;125(2):208–15. Epub 20200620. doi: 10.1016/j.bja.2020.04.093 . [DOI] [PubMed] [Google Scholar]
  • 19.Park JJH, Harari O, Dron L, Lester RT, Thorlund K, Mills EJ. An overview of platform trials with a checklist for clinical readers. J Clin Epidemiol. 2020;125:1–8. Epub 20200513. doi: 10.1016/j.jclinepi.2020.04.025 . [DOI] [PubMed] [Google Scholar]
  • 20.Jung A, Balzer J, Braun T, Luedtke K. Identification of tools used to assess the external validity of randomized controlled trials in reviews: a systematic review of measurement properties. BMC Med Res Methodol. 2022;22(1):100. Epub 20220406. doi: 10.1186/s12874-022-01561-5 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Schulz KF, Altman DG, Moher D, Group C. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152(11):726–32. Epub 20100324. doi: 10.7326/0003-4819-152-11-201006010-00232 . [DOI] [PubMed] [Google Scholar]
  • 22.Nimavat BD, Zirpe KG, Gurav SK. Critical Analysis of a Randomized Controlled Trial. Indian J Crit Care Med. 2020;24(Suppl 4):S215–S22. doi: 10.5005/jp-journals-10071-23638 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Godin K, Dhillon M, Bhandari M. The three-minute appraisal of a randomized trial. Indian J Orthop. 2011;45(3):194–6. doi: 10.4103/0019-5413.80036 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.(2023) CASP. Randomized-controlled trial CASP tool [online] [January 12, 2024]. https://casp-uk.net/.
  • 25.Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evid Synth. 2020;18(10):2127–33. doi: 10.11124/JBISRIR-D-19-00099 . [DOI] [PubMed] [Google Scholar]
  • 26.(SIGN) SIGN. Randomized-controlled trial tool [online] [January 12, 2024]. https://www.sign.ac.uk/.
  • 27.Clark E, Burkett K, Stanko-Lopp D. Let Evidence Guide Every New Decision (LEGEND): an evidence evaluation system for point-of-care clinicians and guideline development teams. J Eval Clin Pract. 2009;15(6):1054–60. doi: 10.1111/j.1365-2753.2009.01314.x . [DOI] [PubMed] [Google Scholar]
  • 28.Buyse M. Limitations of adaptive clinical trials. Am Soc Clin Oncol Educ Book. 2012:133–7. doi: 10.14694/EdBook_AM.2012.32.13 . [DOI] [PubMed] [Google Scholar]
  • 29.Whiting P, Wolff R, Mallett S, Simera I, Savovic J. A proposed framework for developing quality assessment tools. Syst Rev. 2017;6(1):204. Epub 20171017. doi: 10.1186/s13643-017-0604-6 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36(3):666–76. Epub 20070430. doi: 10.1093/ije/dym018 . [DOI] [PubMed] [Google Scholar]
  • 31.Lunny C, Tricco AC, Veroniki AA, Dias S, Hutton B, Salanti G, et al. Methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: protocol and rationale. BMJ Open. 2021;11(6):e045987. Epub 20210624. doi: 10.1136/bmjopen-2020-045987 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. Epub 20150102. doi: 10.1136/bmj.g7647 . [DOI] [PubMed] [Google Scholar]
  • 33.Hirt J, Schonenberger CM, Ewald H, Lawson DO, Papola D, Rohner R, et al. Introducing the Library of Guidance for Health Scientists (LIGHTS): A Living Database for Methods Guidance. JAMA Netw Open. 2023;6(2):e2253198. Epub 20230201. doi: 10.1001/jamanetworkopen.2022.53198 . [DOI] [PubMed] [Google Scholar]
  • 34.McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6. Epub 20160319. doi: 10.1016/j.jclinepi.2016.01.021 . [DOI] [PubMed] [Google Scholar]
  • 35.Page MJ, McKenzie JE, Higgins JPT. Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review. BMJ Open. 2018;8(3):e019703. Epub 20180314. doi: 10.1136/bmjopen-2017-019703 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Cascini F, Beccia F, Causio FA, Melnyk A, Zaino A, Ricciardi W. Scoping review of the current landscape of AI-based applications in clinical trials. Front Public Health. 2022;10:949377. Epub 20220812. doi: 10.3389/fpubh.2022.949377 . [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

Sathish Muthu

4 Jul 2024

PONE-D-24-13528Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationalePLOS ONE

Dear Dr. Staibano,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 17 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sathish Muthu

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

I appreciate the effort put forth in this work and the importance of developing a risk-of-bias tool for adaptive clinical trials (ACTs). This is a crucial step towards improving the quality and transparency of ACTs, which have the potential to streamline clinical research. I would the authors to provide a clear view of the Adaptive clinical trials for the benefit of the authors in the introduction with examples and their advantages and limitations. Further in the methodological aspect does the authors consider the tool as a standalone tool or as a additive tool to RoB2 is not clearly made out. Analysis need to be detailed on the type of regression to be utilized.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

********** 

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

********** 

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

********** 

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

********** 

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

********** 

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review your manuscript entitled "Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationale." I appreciate the effort put forth in this work and the importance of developing a risk-of-bias tool for adaptive clinical trials (ACTs). This is a crucial step towards improving the quality and transparency of ACTs, which have the potential to streamline clinical research. I hope the following comments and suggestions will improve the quality of the manuscript.

Abstract

1. The authors mentioned that "we will perform regression analysis to identify factors associated with poor reporting quality and high risk of bias," but this was not discussed in the methods section in the main text. Please specify the type of regression analysis to be used (e.g., Poisson, linear, logistic) and provide details on the variables to be included in the regression model in the main text.

2. It would be helpful to provide more details on the planned dissemination strategy (e.g., publication, conference presentation, or any other strategies) for the developed risk-of-bias tool, beyond mentioning that this is the first step in its development.

Introduction

3. The description of Adaptive clinical trials (ACTs) does not provide a clear picture of the concept. Please consider explaining the concept in this section to facilitate better understanding of the later sections.

4. Adaptive clinical trials (ACTs) is mentioned twice in one sentence: "Adaptive clinical trials (ACTs) Adaptive clinical trial (ACTs)." Please revise for better clarity.

5. The authors may consider briefly mentioning the potential challenges or limitations of ACTs, in addition to the advantages, to further highlight the importance of a risk-of-bias tool.

Methods

6. Please consider providing a brief overview or example of the Delphi process and subsequent steps planned for the development and validation of the risk-of-bias tool.

7. It is unclear how prospective/retrospective cohort, cross-sectional, and case series studies could provide information related to methodological bias, reporting, or quality in ACTs. Please clarify or revise the inclusion criteria under: "Study type 2: Studies that describe items relating to methodological bias, reporting, or quality in ACTs."

8. The authors mentioned "Eligibility: There will be three types of studies included in this scoping review." Did the authors mean methodological review instead of scoping review?

9. Please clarify whether the authors plan to include studies that assess the methodological quality or risk of bias of ACTs using tools designed for traditional RCTs (e.g., Cochrane RoB 2.0, Jadad scale), or if the focus is solely on tools specifically designed for ACTs.

Discussion

10. Please expand on the potential implications and impact of developing a risk-of-bias tool for ACTs, both for researchers and clinicians.

********** 

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2024 Dec 12;19(12):e0303315. doi: 10.1371/journal.pone.0303315.r002

Author response to Decision Letter 0

4 Jul 2024

PONE-D-24-13528

Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationale

PLOS ONE

Dear Dr. Staibano,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 17 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sathish Muthu

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

I appreciate the effort put forth in this work and the importance of developing a risk-of-bias tool for adaptive clinical trials (ACTs). This is a crucial step towards improving the quality and transparency of ACTs, which have the potential to streamline clinical research. I would the authors to provide a clear view of the Adaptive clinical trials for the benefit of the authors in the introduction with examples and their advantages and limitations. Further in the methodological aspect does the authors consider the tool as a standalone tool or as a additive tool to RoB2 is not clearly made out. Analysis need to be detailed on the type of regression to be utilized.

We have removed the regression analysis, as we do not feel as though it contributes to development of a risk of bias tool for ACTs. We stated in the methods section that we will plan to create a standalone tool, but as the process enters the Delphi and candidate tool development stages, we will determine the role of instead creating an extension to the Cochrane RoB 2.0 tool.

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review your manuscript entitled "Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationale." I appreciate the effort put forth in this work and the importance of developing a risk-of-bias tool for adaptive clinical trials (ACTs). This is a crucial step towards improving the quality and transparency of ACTs, which have the potential to streamline clinical research. I hope the following comments and suggestions will improve the quality of the manuscript.

Abstract

1. The authors mentioned that "we will perform regression analysis to identify factors associated with poor reporting quality and high risk of bias," but this was not discussed in the methods section in the main text. Please specify the type of regression analysis to be used (e.g., Poisson, linear, logistic) and provide details on the variables to be included in the regression model in the main text.

Thank you. This was an oversight and was not removed from a previous version of the draft. We do not believe that a regression analysis will contribute to the development of a risk of bias tool.

2. It would be helpful to provide more details on the planned dissemination strategy (e.g., publication, conference presentation, or any other strategies) for the developed risk-of-bias tool, beyond mentioning that this is the first step in its development.

We have added to this section.

Introduction

3. The description of Adaptive clinical trials (ACTs) does not provide a clear picture of the concept. Please consider explaining the concept in this section to facilitate better understanding of the later sections.

Thank you. We further explained this concept.

4. Adaptive clinical trials (ACTs) is mentioned twice in one sentence: "Adaptive clinical trials (ACTs) Adaptive clinical trial (ACTs)." Please revise for better clarity.

This has been amended. Thank you.

5. The authors may consider briefly mentioning the potential challenges or limitations of ACTs, in addition to the advantages, to further highlight the importance of a risk-of-bias tool.

We have further added to the limitations of ACTs. Thank you.

Methods

6. Please consider providing a brief overview or example of the Delphi process and subsequent steps planned for the development and validation of the risk-of-bias tool.

This has now been included. Thanks.

7. It is unclear how prospective/retrospective cohort, cross-sectional, and case series studies could provide information related to methodological bias, reporting, or quality in ACTs. Please clarify or revise the inclusion criteria under: "Study type 2: Studies that describe items relating to methodological bias, reporting, or quality in ACTs."

We agree. Those study types will realistically not evaluate methodological quality in ACTs. We have thus changed this to only include study types that are likely to include this information. Further, we have amended this to only include two study types (1) studies that have described or suggested any risk of bias or methodological quality items for ACTs and (2) any existing studies that have evaluated methodological bias/quality in ACTs. We have removed those study types. We will also not be looking for any published ACTs since this will not contribute to the identification of risk of bias items.

8. The authors mentioned "Eligibility: There will be three types of studies included in this scoping review." Did the authors mean methodological review instead of scoping review?

Yes, this was changed. Thank you.

9. Please clarify whether the authors plan to include studies that assess the methodological quality or risk of bias of ACTs using tools designed for traditional RCTs (e.g., Cochrane RoB 2.0, Jadad scale), or if the focus is solely on tools specifically designed for ACTs.

We will include those tools as well. We have stated this more clearly.

Discussion

10. Please expand on the potential implications and impact of developing a risk-of-bias tool for ACTs, both for researchers and clinicians.

We have done this. Thank you.

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Attachment

Submitted filename: PONE-D-24-13528 - Response to reviewers.docx

pone.0303315.s004.docx (22.1KB, docx)

Decision Letter 1

Sathish Muthu

23 Aug 2024

PONE-D-24-13528R1Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationalePLOS ONE

Dear Dr. Staibano,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 07 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sathish Muthu

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Kindly address the concerns and comments of the reviewers as noted below

The manuscript demonstrates a high level of quality and possesses considerable research significance; however, there remain several issues that require resolution, specifically as outlined below:

1. Abbreviations in the tables should be completed.

2. Since the screening literature does not limit the study content, the included trials will vary greatly in terms of study purpose, clinical stage, disease type, etc., and there may be large heterogeneity in the study results. How to solve this problem? Also, please add limitations of this study to the discussion.

3. Please supplement the flow diagram for the selection process of articles.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have thoroughly addressed my feedback in this version, and I appreciate the effort put into the revisions. It looks well-polished and complete. Thank you for your work on this.

Reviewer #2: I have been able to read the revised version of the article. The authors have made a significant effort to improve the manuscript. I congratulate the authors for this novel proposal.

Reviewer #3: The manuscript demonstrates a high level of quality and possesses considerable research significance; however, there remain several issues that require resolution, specifically as outlined below:

1. Abbreviations in the tables should be completed.

2. Since the screening literature does not limit the study content, the included trials will vary greatly in terms of study purpose, clinical stage, disease type, etc., and there may be large heterogeneity in the study results. How to solve this problem? Also, please add limitations of this study to the discussion.

3. Please supplement the flow diagram for the selection process of articles.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: M. Pardo Rios

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2024 Dec 12;19(12):e0303315. doi: 10.1371/journal.pone.0303315.r004

Author response to Decision Letter 1

23 Aug 2024

PONE-D-24-13528R1

Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationale

PLOS ONE

Dear Dr. Staibano,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 07 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sathish Muthu

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Kindly address the concerns and comments of the reviewers as noted below

The manuscript demonstrates a high level of quality and possesses considerable research significance; however, there remain several issues that require resolution, specifically as outlined below:

1. Abbreviations in the tables should be completed.

2. Since the screening literature does not limit the study content, the included trials will vary greatly in terms of study purpose, clinical stage, disease type, etc., and there may be large heterogeneity in the study results. How to solve this problem? Also, please add limitations of this study to the discussion.

3. Please supplement the flow diagram for the selection process of articles.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have thoroughly addressed my feedback in this version, and I appreciate the effort put into the revisions. It looks well-polished and complete. Thank you for your work on this.

Reviewer #2: I have been able to read the revised version of the article. The authors have made a significant effort to improve the manuscript. I congratulate the authors for this novel proposal.

Reviewer #3: The manuscript demonstrates a high level of quality and possesses considerable research significance; however, there remain several issues that require resolution, specifically as outlined below:

1. Abbreviations in the tables should be completed.

Response: We have now defined these abbreviations.

2. Since the screening literature does not limit the study content, the included trials will vary greatly in terms of study purpose, clinical stage, disease type, etc., and there may be large heterogeneity in the study results. How to solve this problem? Also, please add limitations of this study to the discussion.

Response: This review is focused on methodological studies of ACTs to identify methodological bias items specific to ACTs. We are not including studies based on disease populations and we are not searching for published interventional ACTs. We have further clarified in the methods. We added potential limitations in the discussion.

3. Please supplement the flow diagram for the selection process of articles.

Response: We have added the PRISMA flow chart as a model for how we will select articles and search peer-reviewed databases and grey literature databases.

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: M. Pardo Rios

Reviewer #3: No

Attachment

Submitted filename: Response_Reviewers PONE-D-24-13528R1.docx

pone.0303315.s005.docx (18.7KB, docx)

Decision Letter 2

Sathish Muthu

12 Sep 2024

Methodological review to develop a list of bias items for adaptive clinical trials: Protocol and rationale

PONE-D-24-13528R2

Dear Dr. Staibano,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sathish Muthu

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Congratulations to the authors for addressing the comments sufficiently to be considered for publication.

Reviewers' comments:

Acceptance letter

Sathish Muthu

24 Sep 2024

PONE-D-24-13528R2

PLOS ONE

Dear Dr. Staibano,

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team.

At this stage, our production department will prepare your paper for publication. This includes ensuring the following:

* All references, tables, and figures are properly cited

* All relevant supporting information is included in the manuscript submission,

* There are no issues that prevent the paper from being properly typeset

If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps.

Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sathish Muthu

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File. Search strategies.

    (DOCX)

    pone.0303315.s001.docx (16.1KB, docx)
    S2 File. PRISMA flow diagram for prospective article screening and full-text review.

    (DOCX)

    pone.0303315.s002.docx (56.6KB, docx)
    S3 File. Completed PRISMA-P checklist.

    (DOC)

    pone.0303315.s003.doc (82.5KB, doc)
    Attachment

    Submitted filename: PONE-D-24-13528 - Response to reviewers.docx

    pone.0303315.s004.docx (22.1KB, docx)
    Attachment

    Submitted filename: Response_Reviewers PONE-D-24-13528R1.docx

    pone.0303315.s005.docx (18.7KB, docx)

    Data Availability Statement

    No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES