Protocol for a prospective multicentre cohort study to develop and validate two new outcome measures for patients with inflammatory bowel disease

Abstract

Introduction

Most of the health-related quality of life (HRQoL) measures for patients with inflammatory bowel disease (IBD) were designed to be used in outpatient settings and are therefore not suitable for use in acute inpatient settings. None of the currently used clinical severity indices for patients with IBD have been properly validated. The aim of this study was to describe the development of a new HRQoL questionnaire and a clinical severity index for patients with ulcerative colitis or Crohn's disease that were short, valid and suitable at any stage of their disease. The new HRQoL and disease severity index will be easily used at the point of care, and invaluable monitoring tools for clinical care, audit and research.

Methods and analysis

This is a prospective multisite validation study of two new outcome measures, the Crohn's and Colitis quality of life (CCQ) questionnaire and the Clinical IBD severity score (CISS). We plan to recruit patients with ulcerative colitis or Crohn's disease. The questionnaire items will be selected through extensive literature review and a focus group involving patients, methodologists, statisticians and IBD specialists. The CCQ questionnaire will be completed by patients attending IBD clinics, having endoscopy procedures or when admitted to hospital. CISS will be completed by clinicians while assessing patients with IBD. Psychometric analysis will be carried out to test the validity and reliability of the questionnaires and to determine the potential to produce shorter versions of CISS and CCQ. The construct validity of CCQ will be tested against short form-12 and the European Quality of Life Five Dimensions. The construct validity of CISS will be tested against biochemical markers, clinical and endoscopic indices to assess severity.

Ethics

This study was approved by the South East Wales Research Ethics Committee (Ref 11/WA/0239).

Article summary.

Article focus

• None of the currently used clinical severity indices for patients with inflammatory bowel disease (IBD) have been properly validated.

• Most of the health-related quality of life (HRQoL) measures for patients with IBD have been designed for use in outpatient settings and are therefore not suitable for use in acute inpatient settings.

• This article describes the protocol for a prospective multisite validation study of two new outcome measures, the Crohn's and colitis HRQoL (CCQ) questionnaire and the Clinical IBD severity score (CISS).

Key messages

• The main focus of the article is the development and validation of two outcome measures to assess the quality of life and disease severity of patients with IBD.

• This article provides an insight into the methods used to develop and validate new outcome measures which can be applied to any disease.

Strengths and limitations of this study

• CISS will be the first fully validated clinical severity index for patients with IBD.

• The CCQ will be the first HRQoL tool that will be applicable to all types and presentations of IBD.

• It may be difficult to recruit adequate numbers of patients with less common presentations of IBD like perianal Crohn's disease or patients with extra-intestinal manifestations.

Introduction

Inflammatory bowel disease (IBD) affects approximately one person in every 250 in the UK population.¹ The cost of IBD to the National Health Service (NHS) has been estimated at about £720 million per annum, based on the prevalence and an average cost of £3000/year/patient.² The lifetime medical costs associated with the care of IBD are comparable with major chronic diseases such as diabetes mellitus or cancer.² Antitumour necrosis factor α drugs are new and effective biological treatments for both ulcerative colitis (UC)³ and Crohn's disease (CD),⁴ but the long-term outcomes are still unknown and there are a number of safety issues.⁵ Therefore, the National Institute for Health and Clinical Excellence has recommended the establishment of a Registry for patients with IBD treated with biological therapy.⁶ Assessment of response to treatment will require the measurement of both health-related quality of life (HRQoL) and disease severity.

HRQoL questionnaires are often employed as measures of health status and are important outcome measures in clinical trials. They should form an integral part of any outcome monitoring efforts but are often omitted from large-scale registries because of constraints on the amount of data these registries can collect. To facilitate the adoption of HRQoL measures in large IBD registries, it is important to develop instruments which are short and easy to complete, yet valid, reliable and applicable to all IBD patients.

IBD has a great impact on the quality of life as it is not curable and follows an unpredictable, relapsing and remitting course with significant variation in the pattern and complexity of symptoms that may affect each patient, sometimes leading to hospital admission, surgery and bowel resection, which may leave the patient with a stoma. There are several specific HRQoL measurement tools for patients with IBD.^7–9 However, all of them have been designed for use in the outpatient setting with stable patients and there is no HRQoL instrument that is validated for use both in the community and by patients who are acutely ill or have a stoma or perianal disease.

In clinical practice, assessing disease severity is an important part of IBD management and a standardised and quantitative evaluation of the severity of IBD is needed for the registry. Owing to the varied presentations of IBD, a number of clinical indices have been put forward using different parameters which are based on different principles.^{10 11} Currently available disease severity indices measure disease severity at a single point of time rather than over a longer time period, which is an important outcome to assess quality of care. Therefore, a new index that assesses the chronic severity of IBD over a long period of time is needed. A recently reported disability measurement tool for patients with IBD has good correlation with other quality of life and disease severity tools, but has not been validated on acutely unwell patients, only those seen in outpatients.¹² An instrument has been developed to assess the cumulative bowel damage in Crohn's disease, but this index is specific for Crohn's disease and requires imaging reports for completion.¹³ In order to be widely utilised and generalisable, the index should include as few items as possible which are easily obtainable in any clinical setting and applicable to the majority of patients. An index should also possess the required psychometric properties such as validity and reliability.^{14 15} In IBD, there are several different disease severity indices available.^{10 11} However, none of them have been properly validated. Therefore, for a successful IBD registry, there is a need for a short yet reliable and valid severity score index to assess response to treatment and to detect early relapse for all types and presentations of IBD.

Aim

The aim of the study was to develop valid and reliable HRQoL and disease severity measurement tools that are suitable for use with patients at any stage of their IBD. These tools will be easily recorded at the point of care to support their use for a registry of patients with IBD.

Objectives of the study

To validate an HRQoL questionnaire (Crohn's and Colitis Quality of Life (CCQ) questionnaire) and derive a short form (Short CCQ) that is suitable for acute and stable patients with IBD and covers patients with a stoma or fistula.

To validate a single clinical severity index (Clinical IBD severity index (CISS)) and derive a short form that is suitable for all types and presentations of IBD.

Methods and analysis

Developing the outcome measures

Crohn's and colitis HRQoL questionnaire

The initial version of the Crohn's and Colitis health-related quality of life questionnaire will be based on the UK Inflammatory Bowel Disease Questionnaire⁹ that was developed and validated in 2000. An extensive literature review will be carried out to identify supplementary items to reflect the wide range and frequency of symptoms of IBD when patients are seen as outpatients or inpatients with perianal disease and with a stoma. The first draft of CCQ will be as inclusive as possible to cover all patients with IBD. We will validate CCQ with a wider group of patients to include patients with both CD and UC as well as patients with a stoma and Crohn's perianal disease. The questionnaire will be examined by IBD specialists, methodologists and statisticians to ensure good face and content validity of the items. To ensure that the resulting questionnaire is clear to patients, we will conduct a feasibility study, asking a small sample of patients to complete the questionnaires. Quality of life will thus be presented as a simple score that will be derived from items completed by patients with different IBD phenotypes in a broad spectrum of settings. The score will enable monitoring over time and comparative assessment across different UK locations.

Clinical IBD severity score

The development of Clinical IBD Severity Score (CISS) will follow a clinimetric approach.^{14 15} Items will be selected through a review of 17 existing clinical severity indices commonly used in studies for UC and CD (box 1). To ensure that the selected items are applicable to patients with IBD, a focus group of at least six IBD specialists from different UK hospitals, statisticians and methodologists will review these items and ensure good face and content validity. CISS will assess the acute and chronic severity of IBD. It will have supplementary questions for perianal disease (to be used when applicable). CISS will be the first clinical severity index to include all presentations of patients with IBD in one single index.

Box 1. Commonly used clinical severity indices in the literature.

1. Ulcerative colitis

   • Truelove and Witts severity index²⁶
   • Powell tuck index²⁷
   • Simple clinical colitis severity index²¹
   • Lichtiger score²⁸
   • Clinical activity index (CAI)²⁴
   • Physician global assessment²⁹
   • Improvement based on individual symptom score³⁰
   • Ulcerative colitis clinical severity score³¹
   • Seo score³²
   • Mayo clinic activity score²³
2. Crohn's disease

   • Crohn's disease activity index³³
   • Harvey Bradshaw index²⁰
   • Van Hees index³⁴
   • The Cape Town index³⁵
   • Fistula Drainage assessment³⁶
   • Perianal Disease Activity Index (PDAI)³⁷
   • Perianal Crohn's disease Activity Index (PCDAI)²²

Recruitment

This is a prospective multicentre study which will be carried out over a 3-year period. Sites will be invited and patients will be recruited over a 24-month period. Data analysis and production of the final version of the questionnaires will be carried out in the following 12 months. We will recruit patients with UC or CD. Invitations will be sent to teaching and general district hospitals across the UK. We will aim to recruit at least four different UK sites. Patients’ medical records will be reviewed to confirm their eligibility as below:

Inclusion criteria

• Confirmed diagnosis of UC, CD or indeterminate colitis.

• Age 18 years and above.

• Not in a vulnerable group (such as people with mental illness or memory problems, learning difficulties or physical disabilities).

• Able to consent.

Exclusion criteria

• Patients without a definite diagnosis of UC, CD or indeterminate colitis.

• Age less than 18 years.

• Patients within a vulnerable group (such as people with mental illness or memory problems, learning difficulties or physical disabilities).

• Unable to consent.

If patients meet the inclusion criteria, they will be invited to participate in the study when they attend outpatients or are admitted to hospital. Patients will be asked to give written consent following an oral and written explanation of the study.

The long version of the CCQ questionnaire will be completed by patients while they are in hospital, in outpatients or at home. Patients will also complete the generic short form 12 (SF12)¹⁶ and European Quality of Life Five Dimensions (EQ5D)¹⁷ questionnaires. They will be asked to complete the same questionnaires within 6 weeks after the initial completion to check test–retest reliability and responsiveness.

CISS will be recorded by the healthcare professionals when reviewing patients in clinic or on the ward. CISS will be recorded again in 2–6 weeks after the initial completion in order to check CISS test–retest reliability and responsiveness.

Sample size

When validating a questionnaire, it is important that the sample used is representative of the population in which the instrument is to be used. There are no general criteria for the required sample size in a validation study,¹⁵ which is typically based on the assumption that the number of respondents should exceed the number of items in the questionnaire by at least a factor of three.¹⁸ Some authors suggest that rather than the overall sample size, it is the ratio of participants to items that is important and recommend a 10 to 1 ratio for each item.¹⁹ We anticipate that CISS will have 17 different items and CCQ will have 32 different items. We will therefore aim to recruit a minimum of 170 patients and 320 patients for the validation study of initial versions of CISS and CCQ, respectively.

Psychometric analysis

Data will be analysed using the Statistical Package for Social Sciences (SPSS) V.19 licensed to Swansea University. The main components of psychometric analysis will be:

Internal consistency

Internal consistency is a measure of reliability. It measures the degree of correlation between different items in the scale. Internal consistency will be assessed by item–total correlations and Cronbach α. Items with item–total correlation below 0.2 or more than 0.8 will be rejected^{14 15} as they add little information to the scale. Items will also be considered for rejection if more than 80% or less than 20% of patients gave the same response because they would not be able to differentiate different levels of severity. Items that are ambiguous or found difficult to answer will be considered for removal or rewording. Cronbach α of the resulting scale should be >0.7.¹⁵ Item discrimination power, which is the ability of items to discriminate between patients with different level of severity, should be >0.4¹⁵; otherwise, it will be considered for rejection. We will carry out stepwise regression of the total scores on the individual items and select the items that represent 95% of the variation in the scores to produce short versions of the CCQ and CISS.¹⁵

Validity

Construct validity refers to the correlation of the scale with other instruments that are believed to assess the same attribute with high degrees of validity and reliability.¹³ It is commonly measured using the Pearson correlation coefficient (r). Construct validity will be accepted if Pearson correlation is 0.4–0.8.^{14 15} Lower correlations mean either one of the tests has low validity or the two tests are measuring different phenomena, while high correlations mean the two tests are very similar and the new scale is not needed. Construct validity of CCQ will be assessed using SF12¹⁶ and EQ5D.¹⁷ Construct validity of CISS will be checked using biochemical markers: C reactive protein, white cell count, haemoglobin, albumin and clinical indices: Harvey Bradshaw index²⁰ (for CD), Simple clinical colitis activity index²¹ (for UC) and perianal disease activity index²² (for perianal CD). These clinical indices will be selected because they are easy to use and widely cited^{10 11} in the literature. When patients undergo endoscopy, the following endoscopic indices will be recorded: Mayo clinic score,²³ Rachmilewitz scores²⁴ in UC and simple endoscopic score²⁵ in CD.

Test–retest reliability or reproducibility

Reproducibility is used to assess reliability of the test applied on two occasions. To measure reproducibility, CCQ and CISS will be administered to 20% of patients on two occasions. There should be no overall change in their clinical condition between these two intervals. For CCQ, patients will be asked if their health has changed since they last filled the questionnaires. For CISS, we will use physician's global assessment to assess if patients’ conditions have changed or not. Patients with no change will be included in the reproducibility analysis using the intraclass correlation coefficient. For practical reasons, we will allow a period of 2–6 weeks after the first assessment or since the first questionnaire was completed. Previous studies have illustrated that a period of less than 2 weeks is not reliable as patients might remember their answers and select them again.^{14 15} Therefore, we expect to include patients with a quiescent to moderate IBD for the reproducibility analysis because patients with severe IBD will be more likely to have their disease changed or have surgery within 2 weeks. A value of intraclass correlation of 0.75^{14 15} will be accepted.

Responsiveness

Responsiveness is the ability to detect change. This will be computed by applying CCQ and CISS to 20% of patients on two occasions. For CCQ, patients will be asked if their health has changed since they last filled the questionnaires. For CISS, we will use physician's global assessment to assess if patients’ conditions have changed or not. Patients whose clinical conditions have changed will be included in the responsiveness analysis using the responsiveness ratio. Responsiveness ratio will be calculated by dividing the mean change in scores for patients who reported a change with the SD of the scores of those who remained stable.¹⁵ A ratio more than 0.5 will be accepted.^{14 15} For practical reasons, we will allow a maximum of 6 weeks after the first assessment or since the first questionnaire was completed.

Interobserver reliability

This is a measure of reliability to assess the degree of consistency between different observers. We will check the interobserver reliability of CISS on 20% of patients using interclass correlation. Two observers will use CISS to assess the same group of patients. We will divide the patients into two groups. The first group will be assessed by a physician and a specialist nurse, while the second group will be assessed by two physicians. A correlation of >0.75 will be acceptable.¹⁵

Ethics and dissemination

This study has been approved by the South East Wales Research Ethics Committee (Reference 11/WA/0239). NHS code of confidentiality and Data protection will be adhered to. Once patients are identified, all personal details will be anonymised. Only study-related staff will have access to the data. Access will be permitted to appropriately qualified personnel and in accordance with the Data Protection Act 1998. All data acquisition, storage and transmission will comply with the Data Protection Act.

We are committed to publishing our results as widely as possible in peer-reviewed journals and to ensuring that appropriate recognition is given to everyone who works on the study.