Development of a core outcome set for clinical trials in inflammatory bowel disease: study protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey

Abstract

Introduction

Crohn’s disease (CD) and ulcerative colitis (UC), the main forms of inflammatory bowel disease (IBD), are chronic, progressive and disabling disorders of the gastrointestinal tract. Although data from randomised controlled trials (RCTs) provide the foundation of evidence that validates medical therapy for IBD, considerable heterogeneity exists in the measured outcomes used in these studies. Furthermore, in recent years, there has been a paradigm shift in IBD treatment targets, moving from symptom-based scoring to improvement or normalisation of objective measures of inflammation such as endoscopic appearance, inflammatory biomarkers and histological and radiographic end points. The abundance of new treatment options and evolving end points poses opportunities and challenges for all stakeholders involved in drug development. Accordingly, there exists a need to harmonise measures used in clinical trials through the development of a core outcome set (COS).

Methods and analysis

The development of an IBD-specific COS includes four steps. First, a systematic literature review is performed to identify outcomes previously used in IBD RCTs. Second, semistructured qualitative interviews are conducted with key stakeholders, including patients, clinicians, researchers, pharmaceutical industry representatives, healthcare payers and regulators to identify additional outcomes of importance. Using the outcomes generated from literature review and stakeholder interviews, an international two-round Delphi survey is conducted to prioritise outcomes for inclusion in the COS. Finally, a consensus meeting is held to ratify the COS and disseminate findings for application in future IBD trials.

Ethics and dissemination

Given that over 30 novel therapeutic compounds are in development for IBD treatment, the design of robust clinical trials measuring relevant and standardised outcomes is crucial. Standardising outcomes through a COS will reduce heterogeneity in trial reporting, facilitate valid comparisons of new therapies and improve clinical trial quality.

Strengths and limitations.

• This protocol outlines the first international consensus effort to develop a core outcome set (COS) for use in inflammatory bowel disease (IBD) clinical trials. With over 30 novel therapeutic compounds in development for IBD treatment and rapidly evolving treatment targets, the need to harmonise clinical trial efficacy and safety outcomes in a COS is exigent.

• The multistep process to develop the COS is rigorous and involves a detailed systematic literature review, semistructured interviews with key stakeholder groups, two-round Delphi survey to prioritise key outcomes and a consensus meeting to ratify the COS.

• To develop the COS, we will seek input from multiple stakeholders, including patients, clinicians, researchers, pharmaceutical industry representatives, healthcare payers and regulators. This will generate diverse viewpoints reflecting clinical practices from around the world.

• Although the scope of this COS will be focused towards use in prospective clinical trials in IBD, the selected outcomes may not be relevant for open-label or retrospective studies of IBD treatment.

Introduction

The inflammatory bowel diseases (IBD), Crohn’s disease (CD) and ulcerative colitis (UC), are chronic, progressive and often disabling disorders of the gastrointestinal tract with no cure. Worldwide, the incidence of IBD is increasing with the highest incidence in North America and Europe; however, rapidly rising rates of disease in Asia¹ have recently been observed. Typical symptoms of these diseases, which include diarrhoea, gastrointestinal bleeding and abdominal pain, cause impaired quality of life, reduced work capacity and social stigmatisation.² Although the aetiology of IBD is unknown, existing evidence implicates the development of a dysregulated immune response in genetically susceptible individuals consequent to complex interactions between the intestinal microbiome and environmental exposures.³ Both CD and UC are lifelong diseases without a cure that typically require continued medical therapy and surgery in a large proportion of patients. In addition, the direct and indirect costs associated with IBD are estimated to exceed US$30 billion annually in the USA alone.^{4 5}

Treatment of CD and UC is focused on controlling inflammation with anti-inflammatory and immunosuppressive agents, with goals of induction and maintenance of remission. In particular, the adoption of biological therapies over the past two decades has revolutionised IBD management, making sustained remission an achievable therapeutic target.⁶ Approval of these new agents has relied on data from robust randomised controlled trials (RCTs)^7–14 that in recent years have increased in size and sophistication. Advances in this field continue at an increasingly rapid pace with multiple classes of agents in late phase development.^{15 16} In parallel, a paradigm shift in treatment targets for IBD has occurred, with a move away from symptom-based scoring^17–19 to normalisation of more objective measures of inflammation, such as endoscopic appearance, inflammatory biomarkers and histological and radiographic end points.

Furthermore, recognising the need to accurately measure the patient experience with IBD, the US Food and Drug Administration (FDA) has advocated for measurement of patient-reported outcomes (PROs) in clinical trials.²⁰ The utilisation of PROs as a treatment end point in IBD trials poses unique challenges: importantly, symptom scoring is likely to remain a central component of IBD PROs, despite poor sensitivity and specificity for predicting mucosal inflammation.²¹ Symptom scoring may also be confounded by psychological comorbidity and perceived stress,²² resulting in disparities between PROs and objectively assessed endoscopic, radiographic and histological disease activity, especially in Crohn’s disease. Thus, the adoption of PROs as a primary therapeutic target in clinical trials would require careful evaluation.

In addition to the shift in efficacy outcomes measured in IBD trials, the assessment of safety outcomes has also changed with the introduction of biological and immunomodulator therapies, which are often used in combination. As novel treatments are developed to target different components of the immune response, short-term and long-term safety evaluations are essential. These include the risks of bacterial infections (including tuberculosis), viral infections (including hepatitis B or herpes zoster virus reactivation), malignancy, lymphoma, infusion and injection reactions and development of antidrug antibodies.²³

These shifts in the research environment have led investigators and regulatory authorities to re-evaluate the key efficacy and safety outcomes measured in IBD clinical trials. The selection of appropriate outcomes is critical for several reasons. First, their operating properties determine trial efficiency and ultimately drive both our ability to accurately identify effective new therapies and the cost of drug development programmes. Second, choice of outcomes can shape clinical practice if the selected end points are perceived to be relevant to both patients and healthcare professionals. Third, identification of standardised outcomes has potential to facilitate and improve the quality of systematic reviews and meta-analyses. Finally, outcome measures are critical components of the analyses used by payers to determine the safety and relative cost-effectiveness of competing treatments and significantly influence regulatory and formulary policy.²⁴

It is apparent that insufficient attention has been paid to the standardised assessment of outcome measures for IBD trials. Notably, no formalised consensus exists regarding what to measure, how to measure and when to measure selected efficacy and safety outcomes in IBD trials.²⁵ Given the evolving landscape of IBD treatment end points and the rapid development of new therapies, an international consensus agreement on core outcomes for use in future IBD trials is of critical importance.

A core outcome set (COS) is a consensus-derived minimum set of outcomes that should be measured and reported in all clinical trials of a given disease.²⁴ The expectation is that core outcomes will always be collected and reported, but the COS is not restrictive such that investigators are still encouraged to explore other outcomes in addition to the COS. COS have been developed and used effectively in several specialties, most prominently in rheumatology through the Outcome Measures in Rheumatology initiative.²⁶ Protocols have been proposed for COS development in other areas of health research^27–33 and to facilitate this activity the Core Outcome Measures in Effectiveness Trials (COMET) initiative has begun.³⁴ Implementation of a successful COS should reduce heterogeneity in outcome reporting, enhance the quality of evidence synthesis and systematic reviews and increase the relevance of clinical research for multiple stakeholders.³⁵

This protocol establishes the context and scope for COS development in IBD, outlines the methods to be adopted for each step of COS development and increases awareness of this effort to encourage IBD researchers and other stakeholders from around the world to participate.

Methods and analysis

Our interest in developing this COS has been listed in the non-database list of the COMET initiative (www.comet-initiative.org). This project will use published recommendations²⁴ for the development of an international consensus IBD-specific COS in a multistep process. Detailed methodology for each step of the process is provided in the relevant sections below.

1. Completion of a systematic review to identify efficacy and safety outcomes currently reported in IBD RCTs.

2. Identification of additional outcomes important to key stakeholders, including patients with IBD and patient advocacy groups, clinicians, researchers, pharmaceutical industry representatives, healthcare payers, regulators and policy-makers through semistructured stakeholder interviews.

3. Prioritisation of outcomes and generation of a consensus outcomes list using a two-round Delphi survey.³⁶

4. Ratification of the COS in a consensus meeting of global experts.

Scope of the core outcome set

This COS is intended as the international standard for clinical trials examining the efficacy of treatments in adult patients (≥18 years) with IBD. Patients included within the scope of this COS include those with

1. CD—including both luminal and perianal fistulising disease and

2. UC—including patients with pouchitis after colectomy

Health interventions included within the scope of this COS include trials of therapeutic compounds and treatment algorithms. Effectiveness of surgical interventions will not be evaluated in this COS.

Identifying existing knowledge

To our knowledge, two existing initiatives have potential conceptual overlaps with the development of a COS. However, both projects have differing aims and neither of these identified projects have the same scope as the COS:

1. The International Consortium for Health Outcomes Measurement (ICHOM) is developing a standardised outcome set for IBD.³⁷ The ICHOM initiative is centred on devising patient-based and value-based healthcare outcomes, which is most relevant as a quality metric for healthcare payers, with a broader scope on healthcare provision rather than a specific focus on core outcomes for assessment in clinical trials.

2. The Selecting Therapeutic Targets in Inflammatory Bowel Disease programme was initiated by the International Organization for the Study of IBD.⁶ Their recommendations for clinical, endoscopic, histological, imaging, biomarker and patient-reported targets in CD and UC aim to guide clinical practice rather than drive end-point selection for clinical trials and drug development.

Step 1: systematic literature review

A literature review will be conducted to identify and compare outcomes reported in existing studies of interventions for adult patients with IBD. No sources of financial support will be used for the systematic review.

Types of studies, participants and interventions

RCTs and systematic reviews of RCTs (with or without meta-analysis) will be included. Studies not describing IBD treatment outcomes, conference proceedings/abstracts without complete trial description or studies for which full text is not available in English will be excluded. Trial participants will include all adult patients with IBD (≥18 years), including specific subgroups of patients with perianal fistulising CD and UC patients developing pouchitis after restorative proctocolectomy. Interventions will include trials of therapeutic compounds (including systemic and topical corticosteroids, anti-inflammatories and mesalamine compounds, immune modulating agents, prebiotic and probiotic therapies, biologicaal and biosimilar therapies, faecal microbiota transplantation and small molecule therapy) and trials of management algorithms applied to patients with IBD. Both effectiveness and safety outcomes will be assessed. Surgical interventions will be excluded.

Search methods for identification of studies and study eligibility

Full terms of a comprehensive, electronic search strategy developed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines are detailed in online supplementary files 1 and 2.³⁸ The search strategy will be applied to Medline, PubMed, Embase and the Cochrane Central Register of Controlled Trials. ClinicalTrials.gov will be searched for relevant projects currently underway, and we will also screen abstracts from the American College of Gastroenterology Annual Scientific Meeting, Digestive Disease Week, United European Gastroenterology Week and European Crohn’s and Colitis Organisation conference proceedings published from January 2007 through June 2016. The reference lists of relevant studies will be searched for additional studies not identified from the electronic database search. No language restrictions will be applied to the initial search strategy but studies without English-language full text will be excluded from the selection of relevant articles. Given the substantial changes in IBD trial design over the past two decades, we will restrict the search to studies published after 1998 to ensure selection of more contemporary and relevant outcomes. Two review authors (CM and CEP) will independently screen the abstracts returned from the search strategy and any studies not meeting inclusion criteria will be excluded. In cases of dispute, a third review author (VJ) will be consulted.

Assessment of methodological quality

As the primary focus of the systematic review will be to generate a list of potential outcome measures, the methodological quality of the reported outcomes in included studies will be assessed using four questions³⁹:

1. Is the primary outcome clearly stated?

2. Is the primary outcome clearly defined so that another researcher would be able to reproduce its measurement (eg, measurement tools, measurement timing)?

3. Are secondary outcomes clearly stated?

4. Are secondary outcomes clearly defined?

As the primary scope of this project evaluates outcome reporting, the overall methodological quality of the included studies from systematic reviews will not be evaluated.

Data extraction, analysis and presentation

Independent data extraction will be performed by two review authors (CM and CEP) using a standardised extraction form for the following: author details and affiliation, year and journal of publication, study design, study population (CD, UC, perianal fistulising CD and pouchitis), intervention(s) under review, primary and secondary effectiveness and safety outcome(s) reported, outcome definition(s) and outcome measurement tool(s). Disagreement will be resolved through discussion and if resolution is not possible, a third reviewer (VJ) will be consulted. Original study authors will be contacted if there is unclear/unavailable data. The data will be synthesised and presented in a descriptive table, with all reported outcome measures and the quality of outcome reporting. Efficacy outcomes will be stratified by category: clinical, endoscopic, histological, radiologic, laboratory, patient-reported and composite scales of multiple outcome measures. Safety outcomes will be stratified by adverse event type (eg, infections, cardiac adverse events, malignancies, lymphoma, infusion/injection reactions, immunologic adverse events) and by severity (hospitalisation, intervention discontinuation, death). These outcomes will then be condensed into a preliminary list for consideration in semistructured interviews and the Delphi survey.

Records will be managed in EndNote X8 reference software (Clarivate Analytics, Boston, MA).

Step 2: Stakeholder involvement

Outcomes measured in clinical trials must be meaningful to patients, healthcare providers and healthcare systems who receive, deliver and pay for care, respectively. Therefore, the input of multiple stakeholders affected by a COS for IBD trials will be sought. Semi-structured interviews will be conducted with the following aims:

1. Preliminary prioritisation of the importance of efficacy and safety outcome measures generated through the systematic review

2. Augmentation of this list with additional items considered important to stakeholders but not captured in the literature

Stakeholder interview participants and recruitment

We will engage and conduct interviews with the following stakeholder groups: (1) patients with IBD; (2) specialists caring for patients with IBD, including gastroenterologists, surgeons and specialist nurses; (3) representatives from patient advocacy groups; (4) representatives from the pharmaceutical industry; and (5) representatives from regulatory agencies (eg, FDA, European Medicines Agency, Health Canada). Participants will be purposively sampled to obtain a comprehensive representation in demographics, patient clinical characteristics, treatment experiences and professional expertise. Sample size will be estimated pragmatically to achieve saturation of views represented in the qualitative data. An initial sample size of 30 interviews is estimated or at theme saturation.

Data collection and analysis

Qualitative semistructured interviews will be conducted, allowing all participants to raise issues considered of greatest importance. A topic guide will be provided to ensure all interviews address critical topics pertaining to COS development, including: (1) patient experiences of living with IBD and the benefits and harms of IBD-related treatment; (2) outcomes believed to be relevant and important to include in IBD trials and why; (3) measurement tools for use in IBD clinical trials that are effective, reliable and practical and (4) relative importance of outcomes identified from the systematic review. Face-to-face or telephone interviews lasting 30–60 min will be conducted by experts in qualitative methods and all interviews will be recorded and transcribed verbatim. Recordings will be imported into qualitative analysis software and narrative data will then be indexed and mapped to a thematic framework, providing a summary of participants’ key points and priorities.⁴⁰

Step 3: Delphi survey

An international Delphi survey, informed by literature review and semistructured stakeholder interviews, will then be performed to achieve consensus on the outcomes for inclusion in the COS. The Delphi method allows panel members to anonymously derive consensus through multiple rounds of sequential questionnaires. After each round, the group responses are provided to panellists who can then reconsider their position in light of other viewpoints. The anonymity of the Delphi method avoids the opinions of prominent personalities from dominating the consensus and also facilitates wide international participation.³⁶ The Delphi process will consist of two rounds of electronic-based questionnaire, response and feedback. All electronic questionnaires will be pilot tested prior to distribution to ensure clarity.

Selection of panel members

For this study, the Delphi panel will include a minimum target sample size of 50 respondents. We aim to recruit a diverse participant pool, with involvement from each major stakeholder group, including patients, clinicians, researchers and representatives from patient advocacy groups, industry and research funding organisations. Selected participants will reflect a broad range of clinical experiences and geographical expertise, with representation from Canada, the USA, the UK, continental Europe and the Asia-Pacific.

Researchers with extensive experience in IBD will be sought for the Delphi survey. During the systematic review, a list of authors with at least 25 publications in the field of IBD over the past 10 years (2006–2016), including at least two clinical trials or one systematic review of clinical trials on IBD will be compiled and invited to participate. The lead and corresponding authors of clinical trials or systematic reviews will be preferentially invited to participate. Clinicians experienced in managing IBD will be recruited through convenience sampling. Specifically, clinical medical and surgical leads of dedicated IBD centres from North America, Europe and the Asia-Pacific will be identified and recruited; this recruitment strategy has been previously used by other COS developers.^{28 29}

Patients will be eligible for inclusion in the Delphi survey if they have a confirmed history of CD or UC, attendance of healthcare for IBD and fluent understanding of written English. Patients will be identified through national and international patient advocacy groups and authors’ connections. Strong collaborative partnerships between the authorship team and IBD centres in Europe and the Asia-Pacific will aim to incorporate multinational patient representation. Representatives from the pharmaceutical industry will also be invited to participate; this group will comprise approximately 10% of Delphi survey participants.

All potential participants will be emailed an invitation letter outlining the aims and details of the study and the rationale and importance of completing the entire Delphi process. Respondents who agree to take part will be assigned a unique identification number. For each round of the process, participants will have 3 weeks to complete the survey with generic email reminders sent at the one and 2-week marks. All data will be stored against the unique identifier only; participants will be blinded to the other respondents in the study. Only the lead author (CM) and primary investigator (VJ) will have access to the complete list of Delphi survey panellists. For each round of the Delphi survey, response and attrition rates will be calculated.

Delphi round one

In the first round, participants will be asked to identify the stakeholder group to which they belong, and complete questions about their professional background and experience with clinical research relevant to IBD. They will then be presented with the complete list of efficacy and safety outcomes generated from the literature review and stakeholder interviews. Outcome order will be randomly assigned to mitigate the influence of display order on scoring. Participants will be asked to rank each outcome on a scale from 1 to 9, based on the Grading of Recommendations Assessment, Development and Evaluation working group definitions.⁴¹ Scores of 1–3 indicate an outcome that is not important for inclusion, scores of 4–6 indicate an outcome important but not critical for inclusion and scores of 7–9 indicate an outcome felt critical for inclusion in the COS. An option to select ‘Unsure of significance’ will also be available. Participants will be asked to focus on ranking the most important outcomes for inclusion highly and excluding outcomes felt to be of lesser importance; regardless of score, all outcomes will be carried to the second round. Finally, through free-text entry, participants will have the option to clarify compelling arguments for and against inclusion of outcomes and to identify additional outcomes not included in the first round questionnaire.

Responses from round one will be analysed and collated into a feedback report. Descriptive statistics will be used to summarise the number of participants scoring each outcome and the distribution of scores. Responses to open-ended questions will be reviewed by the authorship team to evaluate for substantial arguments and additional suggestions will be reviewed for uncaptured outcomes in the first round questionnaire. Subgroup analysis will be conducted, stratifying scores by stakeholder group to evaluate for differences from other panellist responses. Panellists who do not complete the first round survey will not be invited to participate in round two.

Delphi round two

In round two, each participant will be provided with the number of respondents and distribution of scores for each efficacy and safety outcome from the first round, stratified by stakeholder group. They will then be shown their own score from round one and asked to rescore each outcome, with consideration based on insights from the group. Each outcome will be rescored on a scale from 1 to 9 as previously described, and participants will be specifically asked whether each outcome should be included in the COS. Changes in score from round-to-round will be documented.

Responses from round two will be analysed with descriptive statistics. Outcomes for which ≥70% of panellists scored it 7–9 and fewer than 15% of panellists scored it 1–3 will be decided a priori to have met consensus for inclusion.²⁴ Conversely, outcomes for which ≥70% of panellists scored it 1–3, and fewer than 15% of panellists scored it 7–9 will be defined to have met consensus for exclusion. Outcomes not meeting these definitions will be classified as lack of consensus. While these definitions are subjective, they have been recommended by previous COS authors²⁴ and avoid post hoc definitions of consensus that may bias the results.

Step 4: consensus meeting

A face-to-face consensus meeting with key stakeholders will be held after completion of the Delphi process. The meeting will be chaired by an independent facilitator with the objective of finalising the outcomes for inclusion in the COS. Participants will be purposively sampled from panellists completing both rounds of the Delphi study; approximately 30 participants from diverse stakeholder groups will be invited to participate. The results from each round of the Delphi survey will be reviewed and participants will ratify the efficacy and safety outcomes that meet consensus criteria for inclusion and exclusion. Participants will then discuss the outcomes for which there was lack of agreement; based on the discussion, participants will then anonymously vote for each outcome for inclusion and exclusion in the finalised COS using a format similar to that of the Delphi survey.

Ethics and dissemination

Ethical considerations

As with previous COS development projects, this project is considered a service evaluation not directly influencing patient care or safety.^{27 42} All participants involved will be asked for their consent before participating in either stakeholder interviews or the Delphi survey, and all procedures will be conducted according to the Declaration of Helsinki.

Dissemination

With over 30 novel therapeutic compounds in various stages of clinical development,⁴³ the adoption of an international consensus COS will be critical in ensuring future clinical trials report valid, meaningful and standardised efficacy outcomes. This need is particularly exigent, commensurate with the transition from traditional symptom-based outcomes such as the Crohn’s Disease Activity Index and Mayo Clinic score, to a diverse array of endoscopic, histological, radiographic and patient-reported end points. In addition, with the increasing adoption of biological therapies for IBD management, it is essential for clinical trials to identify unique safety considerations associated with novel therapies. Reporting of treatment-specific safety outcomes such as infectious, malignant, immune, surgical and drug-related adverse events may promote the development of future preventative strategies for optimising short-term and long-term patient safety. Through this COS, we intend to reduce outcome reporting bias, reduce reporting heterogeneity, improve clinical trial quality in IBD and facilitate more robust data synthesis of treatment interventions.

A finalised COS reporting guideline and explanatory document will be drafted, including all efficacy and safety outcomes and measurements as determined by the Delphi rounds and consensus meeting. These documents will be disseminated by high-impact publication.