Establishing key performance indicators for inflammatory bowel disease in the UK

Abstract

Background and aims

Healthcare quality improvement (QI) is the systematic process to continuously improve the quality of care and outcomes for patients. The landmark Inflammatory Bowel Disease (IBD) UK National Audits provided a means to measure the variation in care, highlighting the need to define the standards of excellence in IBD care. Through a consensus approach, we aimed to establish key performance indicators (KPIs), providing reliable benchmarks for IBD care delivery in UK.

Methods

KPIs that measure critical aspects of a patient journey within an IBD service were identified though stakeholder meetings. A two-stage Delphi consensus was then conducted. The first involved a multidisciplinary team of IBD clinicians and patients to refine definitions and methodology. The second stage assessed feasibility and utility of the proposed QI process by surveying gastroenterology services across UK.

Results

First, the four proposed KPIs were refined and included time from primary care referral to diagnosis in secondary care, time to treatment recommendation following a diagnosis, appropriate use of steroids and advanced therapies prescreening and assessment. Second, the Delphi consensus reported >85% agreement on the feasibility of local adoption of the QI process and >75% agreement on the utility of benchmarking of the KPIs.

Conclusions

Through a structured approach, we propose quantifiable KPIs for benchmarking to improve and reduce the individual variation in IBD care across the UK.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Several quality improvement (QI) initiatives in inflammatory bowel disease (IBD) nationally have previously led to improvement in patient care and service delivery.

WHAT THIS STUDY ADDS

• There is now a clear need to reassess which quality metrics can now provide dynamic benchmarking of important contemporary challenges by means of a minimalistic but robust data collection methodology.

• Through a two-stage Delphi consensus with key stakeholders and the IBD clinical/patient community, we have established four key performance indicators (KPIs) along with relevant methodology for implementation nationally.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• This British Society of Gastroenterology IBD QI initiative will focus on the performance of IBD services against defined KPIs and will complement the IBD UK benchmarking tool which assess performance against defined IBD standards via patient surveys and service self-assessments.

Introduction

A number of national IBD (inflammatory bowel disease) quality improvement (QI) initiatives have been undertaken in the UK over the last 15 years.¹ The UK IBD Audit, established in 2004, undertook five rounds of national audit between 2005 and 2016.² This transitioned to the UK IBD Registry—a national registry that collects and reports patient level data to assist IBD teams in providing better care to their patients. The IBD Registry facilitated longitudinal collection and reporting of metrics around screening prior to biologics initiation and monitoring.³ In 2019, the British Society of Gastroenterology (BSG) guidelines and the IBD UK Standards defined the process and outcome measures that represent a high-quality clinical IBD service. Subsequently, the IBD UK patient survey and clinical service self-assessment in 2019/2020 allowed services and patients to feedback on care against the IBD Standards.⁴ The national report that followed ‘Crohn’s and Colitis Care in the UK: The Hidden Cost and a Vision for Change’ highlighted key areas that needed addressing, including delays in diagnosis, the need for quicker access to specialist advice and treatment and for more personalised and holistic care.⁵ To date, no service in the UK currently meets all criteria set and there remains significant variation in care.^{4 6 7} The ability to monitor and benchmark services can help stream pathways towards patient-centred healthcare, as well as guide teams towards meaningful QI targets. This process drives change towards improvements in clinical outcomes and IBD patient experience.^{8 9} Key performance indicators (KPIs) are meaningful and manageable quality metrics that aim to measure performance to identify quality of service, allow benchmarking (to provide comparability) and facilitate recognition of areas for improvement.¹⁰

With a growing population of patients with IBD within the UK, access to newer advanced therapies, evolution of treatment targets, a shift towards patient empowerment and introduction of national programmes such as Getting It Right First Time, there is now a need to revisit quality indicators.¹¹ Furthermore, the UK has seen the rapid introduction of major, and possibly long-lasting adaptations in provision of IBD services during the COVID-19 pandemic.¹² There is now a clear need to reassess which quality metrics can provide dynamic benchmarking of important contemporary challenges that will help facilitate a positive change for patients and services.

Through key stakeholder meetings and Delphi consensus surveys, we aimed to define clinically relevant KPIs with a strong emphasis for patient care and deliverability for IBD services across the UK.

Methodology

Following stakeholder meetings with the BSG IBD section, IBD Registry, patient charity Crohn’s and Colitis UK, BSG Clinical Services and Standards Committee and the Royal College of Physicians, four candidate KPIs were identified by informal consensus:

KPI 1—Time from primary care referral to diagnosis in secondary care.

KPI 2—Time to treatment recommendation following a diagnosis.

KPI 3—Appropriate use of steroids.

KPI 4—Advanced therapies prescreening and assessment.

As summarised in table 1, we undertook a two-stage Delphi consensus to further discuss the relevance and feasibility of the four identified KPIs along with proposed methodology for data collection, standards to assess against and benchmarking.¹³ An initial proposal on the KPI definitions and the QI process were developed and agreed on by stakeholder members prior to round 1 of Delphi consensus survey. In round 1, a proposed description of the data collection process, metrics and outcome was outlined along with statements across several domains supporting each candidate KPIs (as shown in online supplemental document 1). Panellists were asked to independently rank the statements for each KPI, using a 5-point Likert scale along with a free-text option.

Table 1.

Summary of methodological approach for establishing key performance indicators (KPIs) in inflammatory bowel disease (IBD)

Initial stakeholder meeting to propose and develop KPIs	Four KPIs were proposed through meetings with key stake holders and a preliminary methodological approach for the QI process outlined: Time from primary care referral to diagnosis in secondary care Time to treatment recommendation following a diagnosis Appropriate use of steroids Advanced therapies prescreening and assessment
Delphi survey round 1	KPIs with methodology of the QI process presented to a clinical IBD expert panel and patients Statements across the following themes for each of the candidate KPIs presented to the panel for ranking using a 5-point Likert scale (’strongly agree’, ‘agree’, ‘neither agree nor disagree’, ‘disagree’, ‘strongly disagree' Is the KPI being measured is a relevant and critical part of a patient’s experience? Does the methodological process appropriately represent that journey? What standards would be acceptable to help understand performance? Is the QI process achievable nationally? Is it an important clinical priority in the current era? Does engaging in this QI initiative have the potential to lead to a favourable change?
Round 1 report and stakeholder meeting	Report following round 1 generated for review. Stakeholder meetings to update the KPI and QI methodology based on feedback from round 1
Delphi survey round 2	KPIs with updated methodology of the QI process presented to the wider BSG membership Statements presented to survey opinions and challenges on local relevance (utility) and feasibility for participation in this QI programme using a 5-point Likert scale
Round 1 report and stakeholder meeting	Report following round 2 generated for review Stakeholder meetings to update the KPI and QI methodology based on feedback from round 2

BSG, British Society of Gastroenterology; QI, quality improvement.

KPI definitions and methodology were updated for the next round of the Delphi consensus survey (online supplemental document 2). Round 2 aimed to outline opinions and challenges on local relevance and feasibility for participation in this QI programme as well as clarifications on contentious aspects. A similar 5-point Likert scale was used. The wider BSG membership (not limited to the IBD section/practitioners) was then invited to take part in round 2. The Delphi surveys were conducted using Research Electronic Data Capture electronic data capture tools hosted by the IBD Registry team.^{14 15}

Results

Round 1 of Delphi consensus survey

Sixty participants completed the Delphi survey. A minimum number of patients or a fixed time period for data collection was no longer mandated. Data items being collection were reduced further and benchmarking against national median performance with percentile rank reporting was proposed for adoption for KPI 1 and 2 and defined national standards refined for KPI 3 and 4. A full demographic of panellists and overview of results for round 1 is shown in online supplemental document 1.

Round 2 of Delphi consensus survey

Round 2 of the survey was conducted between April 2022 and May 2022. A total of 72 complete responses across 53 NHS sites across UK; 44 based in England, 5 in Scotland and 4 in Wales. There were no respondents from hospitals based in Northern Ireland.

Of 58, 25 (43.1%) sites reported an estimated IBD population base of >4000 patients while 11/58 (22.34%) reported an IBD population base of <2000 patients. Of 71, 34 (47%) of the respondent sites were already enrolled with the IBD Registry for the biologics therapies audit. The complete results from round 2 are shown in online supplemental document 1.

There was greater than 85% agreement among survey participants on the feasibility of local delivery based on the proposed methodology for each individual KPIs. There was greater than 75% agreement on the utility to the IBD service of benchmarking for each individual KPIs. Of 71, 60 (84.4%) IBD services expressed a preference for either a minimalistic approach or use a simple web-based tool for data collection rather than using the IBD Registry’s current more comprehensive tools. However, comparatively, a greater proportion of sites participating in IBD Registry’s biologics audit using the existing tools were keen to continue using it for data collection. Thirty-six per cent of respondents were happy to submit patient identifiers in patients who have not explicitly consented to the IBD Registry (allowed under S251 regulation/approved exemption). Thirty-nine per cent of respondents did not agree while 25% were not sure.

Following round 2, there were further meetings with stakeholder to address any concerns and finalise the KPI definitions and QI methodology to take forward. The steroid use benchmark was revised and the panel instead opted to benchmark against national performance rather than predefined standards.

Key performances indicators

Following these rounds of consensus-building surveys and stakeholder meetings the definitions, outcome measures, data collection, benchmarking and reporting methodologies for each of the KPIs have progressed through several iterations. These final agreed versions are presented as below.

KPI 1: time from primary care referral to diagnosis in secondary care

Outcome measure and definitions

The time to diagnosis KPI will measure the local performance for time to a documented diagnosis of IBD in secondary care following a primary care referral. Time to diagnosis is defined as days between date of an appropriate referral from primary care for suspected IBD to a documented diagnosis of IBD in clinical records in secondary care. Documented diagnosis is defined as a formal documentation of a confirmed diagnosis of IBD in the patient’s records. The diagnosis of IBD would be based on the clinician’s judgement, supported by a combination of relevant investigation. A highly likely suspected diagnosis of IBD (such as ileitis or segmental colitis) that warrant treatment or monitoring will be included in this definition.

Proposed data collection methodology

Data will be collected prospectively from all newly diagnosed patients over a period of a year. This may be done at any time point in the patient’s initial journey following a diagnosis, that is, first outpatient or inpatient clinical review when the diagnosis is confirmed or treatment commenced. The aim is to collect data on as many patients as feasible with no defined minimum number of patients. A minimum threshold may, however, be set to allow benchmarking following preliminary statistical analysis. IBD services that find prospective data collection challenging may consider collecting data retrospectively.

Data items required for each patient enrolled

• Date of referral on the referral letter from primary care.

• Date of formal documentation of a confirmed diagnosis of IBD in the clinical records.

• Diagnosed as an inpatient following a following acute (non-elective) hospital admission (yes/no).

Setting and reporting standards for benchmarking

Benchmarking of individual sites will be performed against the national median performance and performance defined as percentile/rank in relation to national median. At present, a national standard/target for time to diagnosis cannot be defined; however, an exploratory standard may be used for statistical analysis. The local percentile rank, local median time and national median time to treatment recommendation following a diagnosis will non-publicly reported to the individual IBD services. Diagnoses made following hospitalisation in patients with prior primary care referrals will be reported separately but not as part of the KPI. It is envisaged that the initial round/s of QI will help determine national medians to help set a target waiting time for future rounds.

KPI 2: time to treatment recommendation following a diagnosis

Outcome measure and definitions

The time to treatment recommendation following a diagnosis KPI measures the local performance for time to recommendation of treatment for IBD following a diagnosis. Treatment is defined as oral or rectal mesalazine, thiopurines, biological therapies, small molecule drugs oral or rectal steroids, IBD-specific surgery, disease-modifying nutritional therapies (such as exclusive enteral nutrition) and therapies pertaining to IBD specific clinical trials. An active documented decision to watch and wait for mild disease will be considered as ‘treatment’ (eg, in patients with mild terminal ileitis). Advice/guidance given around management of Crohn’s including advice given on smoking cessation or dietary change is excluded from the definition. For treatments commenced in secondary care and recommendations made to primary care—the date when the documented treatment recommendation was made will be recorded. Patients declining treatment would be included with recording of date of treatment recommended. Date treatment was commenced will be collected as a non-mandatory data point. Treatment commenced as an inpatient following hospitalisation (including those diagnosed on that admission) will be reported as separately but not part of the overall KPI.

Proposed data collection methodology

This will be a prospective data collection of all newly diagnosed patients over a period of a year. Patients in KPI2 should be linked to KPI1 with congruency in date of formal documentation of a confirmed diagnosis. Data items for KPI1 and KPI2 may, therefore, be collected together. This may be done at any time point of the patient’s initial journey following a diagnosis, that is, first outpatient or inpatient clinical review following commencement of treatment. The aim is to collect data on as many patients as feasible with no defined fixed number of patients. A minimum threshold may, however, be set to allow benchmarking. IBD services that find prospective data collection challenging may consider collecting data retrospectively.

Data items required for each patient enrolled

• Date of formal documentation of a confirmed diagnosis of IBD in the clinical records.

• Date treatment recommended.

• Date treatment commenced (non-mandatory data item).

• First treatment received following a diagnosis as an inpatient following an acute (non-elective) hospital admission (yes/no).

Setting and reporting standards for benchmarking

As with KPI1, benchmarking of individual sites will be performed against the national median performance and performance defined as percentile/rank in relation to national median. At present, a national standard/target for time to treatment following diagnosis cannot be defined; however, an exploratory standard may be used for statistical analysis. The local percentile rank, local median time and national median time to treatment recommendation following a diagnosis will non-publicly reported to the individual IBD services. Treatment recommendation following diagnosis as an inpatient will be reported as a sub-KPI. It is envisaged that the initial round/s of QI will help determine national medians to help set a target waiting time for future rounds.

KPI 3: oral steroid use

Outcome measure and definitions

The oral steroid KPI measures the proportion of patients exposed to ≥2 courses and proportion of patients exposed ≥3 courses of oral steroids in a year in an unselected cohort of patients with IBD. A course of corticosteroids will be defined as a minimum of at least 5 days of consecutive use. Steroids would include any class of oral corticosteroids including budesonide. Enemas and suppositories will be excluded. Steroid use should measure those obtained through secondary care and primary care prescriptions as well as home supplies. Steroid use would include any given indication rather than IBD alone (the two multicentre national audits found only 3% of steroid excess was from non-IBD indications). Steroid excess is defined the prescription of 2 or more steroid courses over 12 months or >3 months over a 12-month period by the BSG IBD and European Crohn’s and Colitis Organisation (ECCO) guidelines. The currently envisaged denominator for this KPI includes steroids exposed and unexposed patients, but a secondary set of analyses with the denominator of only steroids exposed patients will be undertaken to allow validation and confirmation of this approach

Proposed data collection methodology

A consecutive unselected cohort of patients with IBD (regardless of prior steroid exposure) attending outpatient clinics will be invited to take part. A snapshot of steroid use over the prior 12 months will be assessed as per the definitions of a steroid course and metrics proposed. IBD services will be encouraged to capture data from a diverse range of clinical setting (that include flare and routine appointments) in order to reduce the risk of a selection bias. Data will be collected via dedicated online tools. Patients enrolled may potentially be invited to participate in a linked prospective patient reported steroid use QI process through the IBD Registry. The aim is to capture as many patients as feasible with no defined fixed number of patients. A minimum threshold may, however, be set to allow representative benchmarking. The eventual aim is to move towards a consecutive prospective clinician reported or patient reported steroid exposure data for this KPI.

Data items required for each patient enrolled

• Total number of courses of steroids in the last 12 months (≥0).

• Total duration (in weeks) of steroid use in the last 12 months (≥0).

Setting and reporting standards for benchmarking

For the initial round of QI, no standards are set for what would be considered to be appropriate steroid use for benchmarking. Benchmarking of individual sites will be performed against the national average proportion of patients exposed to ≥2 courses steroid courses in 1 year, average proportion of patients exposed to ≥3 courses of oral steroids in 1 year and median total duration (in weeks) of steroid use over 12 months. Individual site performance will be defined as percentile/percentile rank in relation to national median with reporting of metrics for each of the defined benchmarks. Outcomes from the initial round/s of QI may be used to formally develop a national standard for future rounds.

KPI 4: advanced therapies prescreening and assessment

Outcome measure and definitions

The advanced therapies prescreening and assessment KPI measures the proportion of patients meeting standards for pretreatment screening prior to initiation of advanced therapies and assessment of efficacy and safety after induction of therapy and at 1 year. Advanced therapies include biologics and small molecules that are used for treatment of IBD. Thiopurines and methotrexate are excluded. Pretreatment screening for infections prior to commencement of biologics is defined as per BSG guidance and includes HBV, HCV and HIV (and may include VZV if no history of chickenpox, shingles or varicella vaccination) and tuberculosis screen. This may have been performed at any time point in patient’s immunosuppression history. The interval prior to repeating these tests would be based on the clinical team’s discretion. For Janus kinase inhibitors pretreatment screening should include lipid profiles. Assessment of efficacy and safety following induction can be any documented review of patients between week 8 and week 20 after commencement of advanced therapies. Assessment of efficacy and safety at 1 year can be any documented review of patients between month 10 to month 14 after commencement of advanced therapies (if the respective treatment is still ongoing). This in-person or remote review at both these time points may be conducted by any competent member of the IBD service. The review should consider both safety and clinical parameters (including a patient-reported outcome measure), and an objective assessment of disease activity.

Proposed data collection methodology

The process is similar to the current IBD Registry biologics audit; however, with fewer data collection metrics. Data may be collected by IBD services both prospectively and retrospectively (case note reviews) and should include patients having commenced advanced therapies from January 2021. Data will be entered following the commencement of each new advanced therapy for an individual patient. A patient may, therefore, have multiple entries following sequential changes to their advanced therapy. A mid-treatment switch to a biosimilar, dose optimisation or a change in the mode of administration of the same advanced therapy (such as intravenous to subcutaneous) would not restart a data collection episode for that patient. A minimum number of patients is not defined but a minimum threshold will be set for representative benchmarking.

Data items collected for each patient enrolled

• Was the patient screened for infections before starting on an advanced therapy (split by individual screening parameters)? (Yes/No)

• Was there a documented assessment of efficacy and safety between week 8 and week 20 after commencement of advanced therapy in patients with ongoing use? (Yes/No/No longer on the treatment)

• Was there a documented assessment of efficacy and safety between month 10 and month 14 after commencement of advanced therapy in patients with ongoing use? (Yes/No/No longer on the treatment)

Setting and reporting standards for benchmarking

The standard for minimum expected proportion of patient’s being prescreened prior to initiation of advanced is set at 95%. The standard for minimum expected proportion of patient’s being assessment following induction and at 1 year after commencement of advanced therapies are both set at 90%. The advanced therapy screening and assessment KPI will be reported to individual sites as three separate sub-KPIs each covering different aspects:

1. Screened prior to advanced therapy use (further split by individual parameters).

2. Documented assessment following induction of advanced therapy.

3. Documented assessment at 1 year following commencement of advanced therapy.

Reports will be provided to individual sites on the proportion of patients that met screening and assessment criteria as well as performance against national average.

Discussion

Quality indicators for the medical management of IBD have been explored by other groups outside UK—for example, by Crohn’s and Colitis Foundation of America in 2013, Canadian quality initiative Promoting Access and Care through Centres of Excellence in 2019 and more recently the Spanish Working Group on Crohn’s Disease and Ulcerative Colitis in 2022.^{9 16 17} KPIs from these international initiatives cover very similar themes such as structured clinical pathways or processes for the diagnosis, monitoring and treatment of IBD and improved access to helplines. Furthermore, quality measures specifically based patient-centred outcomes for IBD (International Consortium for Health Outcomes Measurement (ICHOM) Standard Set for IBD) have also been developed for use.¹⁸ The goal of this QI initiative is to establish KPIs for IBD that could enable IBD services to make a measurable difference to patients by improving the safety, effectiveness, quality and experience of care being delivered. Establishing KPIs in IBD does not just rely on its importance in measuring a critical and modifiable aspect of a patient’s journey. An equally strong focus is needed when it comes to the feasibility and adoption of the proposed QI methodology that represented this journey. The Delphi consensus survey focused on these issues and consequently lead to refinement of KPI definitions and the proposed methodology with consensus agreement among the UK IBD community.

The importance of the four KPIs as a significant clinical priority was highlighted by both IBD services and patients in the Delphi consensus survey. Delay in diagnosis and treatment of IBD have significant short-term and long-term implications to the patient and the IBD service. The POP-IBD study reported that less than half of IBD patients with a delayed diagnosis received specialist review within 18 months following initial primary care presentation.¹⁹ Delayed diagnosis has been shown to be associated with higher IBD-related complications including hospitalisation, emergency surgery, corticosteroid use and strictures.^20–22 A key finding demonstrated in the IBD UK National Report was that over a quarter of patients waited over a year for a diagnosis with 41% visiting A&E at least once prior to their diagnosis.⁴ Furthermore, it is well recognised that both UC and Crohn’s are progressive chronic diseases and an early treat to target approach is associated with favourable outcomes.^{23 24} Streamlining secondary care pathways to facilitate early diagnosis and treatment would consequently lead to improved patient outcomes and quality of life measures.

The evidence on the detrimental impact of excessive steroid use on quality of life and long-term outcomes in IBD is clear and reduction of steroid use is universally advocated by IBD societies.^{6 25 26} There was a strong consensus agreement on the appropriateness of the steroid use KPI especially considering it had been validated via the national multicentre audits.^{27 28} These had identified avoidable or potentially avoidable steroid excess in 50% of patients who met the definition for steroid excess. This steroid KPI aims to facilitate evaluation of steroid use rather than steroid excess within individual IBD services. Repeated courses of steroids without institution of an appropriate maintenance regime are associated with poor care.²⁹ Understanding individual steroid use would allow IBD services to consider initiatives such as rapid access flare clinics, proactive disease control, patient empowerment and early institution and optimisation of maintenance therapies. The advanced therapies KPI is similar to the current IBD Registry’s biologics therapies audit and QI programme and aims to measure the efficacy, safety and appropriate use of biological therapies.³ The four KPIs represent an appropriate mixture of process and outcome measures to reflect the performance of IBD services.

The methodological approach proposed caters for under resourced sites with data items for each KPI kept to an absolute minimum without significantly compromising its integrity. This was particularly important for those services which continue to receive paper referrals from primary care or do not have robust and accessible electronic health records (eHRs). The survey suggested that IBD services currently not participating with the IBD Registry’s biologics audit expressed a preference for a bespoke web-based data entry tool. While this is an appealing methodology for retrospective audit data collection, multiple and different fully integrated data collection systems are already in clinical use for the sites engaged with the IBD Registry that allow prospective data capture during routine care. It would be desirable that most IBD services will eventually transition to using these integrated IBD Registry data collection systems that facilitate rapid cycles of analysis and near-real time feedback. Consistently, there was a relatively strong consensus to move towards a rolling prospective QI methodology as this would facilitate an iterative process that will build on successes and goalposts identified. This need for continual reassessment is an important aspirational goal, but currently unrealistic especially in absence of integrated national eHRs similar to Joint Advisory Group endoscopy KPI reporting.^{30 31} Nearly half of the respondents felt the QI initiative will be representative of their IBD population, while the other half did not believe it would be or were not sure. The adequacy representation will need to be explored further in future with formal feedback from sites following in the initial round of the QI initiative.

The Delphi surveys highlighted several pitfalls in the QI strategy which consequently led to adaptation or clarification in definitions and methodology. It was clear that the proposed KPIs could potentially be influenced by factors outside the control and scope of the secondary care IBD team (eg, primary care referrals to 2-week wait colorectal clinics, patient-related factors and access to diagnostic imaging). Conversely, it is anticipated that identification of these factors would in fact allow streamlining of pathways direct to ‘suspected IBD’ clinics and establishment of local policies for rapid management of flares.

The process towards the development of formal consensus derived KPIs in IBD aspires to establish a clear shared understanding with IBD services nationally of what QI in critical aspects of a patient’s journey could potentially achieve. Through these KPIs, IBD services will be able to make an inference about the quality of care provided and indicate areas that require more detailed investigation. Our next step is to now progress towards running this QI initiative as a pilot across a few IBD services nationally.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

Ethics statements

Patient consent for publication

Not applicable.