Development and validation of a rapid, generic measure of disease control from the patient's perspective: the IBD-Control questionnaire

Abstract

Introduction

The use of patient reported outcome measures to support routine inflammatory bowel disease (IBD) care is not widespread and suggests that existing questionnaires lack relevance to day-to-day decisions or are too cumbersome to administer. We developed a simple, generic tool for capturing disease control from the patient's perspective to address these barriers.

Methods

Development based on literature review, patient focus groups/interviews and a steering group, defining a limited set of generic questions. The ‘IBD-Control’ questionnaire comprises 13 items plus a visual analogue scale (VAS) (0–100). Prospective validation involved baseline completion of IBD-Control, quality of life (QoL) questionnaire (UK-IBD-Q), EuroQol (EQ-5D), Hospital Anxiety and Depression Score; and clinician assessment (blinded to questionnaire; recording Harvey-Bradshaw Index or Simple Clinical Colitis Activity Index; Global Clinician Rating; treatment outcome).

Results

299 patients returned baseline surveys (Crohn's disease, n=160; ulcerative colitis, n=139) and 138 attended for repeat visits. Completion time (mean; SD): 1 min 15 s; 25 s; Internal consistency: Cronbach's α for all 13 items (0.85); for subgroup of eight questions (‘IBD-Control-8’; 0.86). Strong correlation between IBD-Control-8 and IBD-Control-VAS (r=0.81). Test-retest reliability (2 week repeat): intra-class correlation=0.97 for IBD-Control-8 and 0.96 for IBD-Control-VAS. Construct validity: Moderate-to-strong correlations between IBD-Control-8 and IBD-Control-VAS versus activity indices, UK-IBD-Q and EQ-5D (utility) with r values 0.52–0.86. Discriminant validity (mean instrument scores for remission, mild, moderate or severe): p<0.001 (analysis of variance (ANOVA)). Sensitivity to change: Effect sizes: 0.76–1.44.

Conclusions

The IBD-Control is a rapid, reliable, valid and sensitive instrument for measuring overall disease control from the patient's perspective. Unlike existing patient reported outcome measures, its simplicity, ease-of-use and generic applicability make it a candidate for supporting routine care.

Significance of this study.

What is already known about this subject?

• The routine use of patient-reported outcome measures (PROMs) in clinical practice is increasingly advocated as a means of supporting patient-centred care, informing decisions and driving service quality.

• A range of PROMs has been developed to measure quality of life in inflammatory bowel disease but none has found widespread practical use in routine settings.

• Barriers to adoption of PROMs include patient and administrator burden and lack of tangible relevance to day-to-day decision-making or quality assurance of services.

What are the new findings?

• The IBD-Control questionnaire is the first PROM to capture disease control from the patient's perspective using a simple set of generic items applicable to all patients with IBD.

• Summary scores from the instrument show strong validity versus more complex quality of life questionnaires, generic utility measures, disease activity scores and global physician assessment.

• As a patient-completed screening tool, the IBD-Control provides a rapid and reliable means of identifying those in a quiescent state and the individual items and summary scores have potential to support efficient models of disease follow-up.

How might it impact on clinical practice in the foreseeable future?

• In routine practice, IBD-Control has potential to provide patients and healthcare teams with a practical means of capturing simple but reliable outcome data in a standardised and easily interpretable form.

• The strong measurement properties suggest potential for use in clinical studies, particularly in long-term or large-scale trials, surveys or registries where its minimal user and administrator burden will favour serial capture of PROM data in routine settings.

Introduction

The goal of therapy for inflammatory bowel diseases (IBD) is to achieve and maintain disease control and thereby optimise quality of life (QoL). Hence, assessment of disease control is a core component of care and underpins management decisions. Surprisingly, no survey instrument has been developed for routine clinical practice with the specific aim of capturing disease control from the patient's perspective.

The routine use of patient reported outcome measures (PROMs) in healthcare is gaining increasing political and professional support as a means of informing day-to-day decisions and driving service quality.^{1 2} PROMs are standardised, validated questionnaires intended for completion by patients in order to measure their perceptions of their own functional status and well-being. PROMs have begun to find a role in national audits and registers and there is rapidly growing interest in their potential to inform individual care.¹

No single PROM instrument has gained widespread popularity in the UK or internationally for routine use in IBD. When formal measurement of health status is undertaken (eg, for clinical trials) the traditional emphasis has been on clinician-reported indices assigning scores for selected symptoms, vital signs and laboratory or endoscopic parameters.^3–7 However, it has long been recognised that clinician-reported indices are not entirely objective ⁸ and fail to capture the impact of disease from the patient's perspective.

The last decade has seen a wealth of qualitative research employing focus groups, interviews, expert panels and factor analysis to identify key issues for patients with IBD.^9–17 This has resulted in a number of multidomain PROMs being developed as measures of disease-specific QoL, typically combining symptom questions with more generic questions that relate to overall well-being, energy/vitality, bodily pain and impacts of disease on physical, social and emotional function.^9–17 Only a limited number of these instruments has found use in clinical trials ^{10 16} but none has established a significant place in routine practice. This may reflect, in part, the patient and administrator burden inherent in using lengthy multidomain questionnaires. Shortened forms may have more potential for routine clinical applications ^{18 19} but are not used widely.

If PROMs are to find a place in routine care, instruments need to be acceptable to patients and healthcare teams, demonstrating added value to normal practice and with simple and clinically-relevant interpretation. This means combining a high level of user-friendliness (ie, short and rapidly completed tools with low burden and easy interpretation) with strong and explicitly stated measurement properties to support decisions at the level of the individual patient. The objective of this study was to develop and validate a novel questionnaire intended to rapidly capture disease control from the patient's perspective (‘IBD-Control’). Our design criteria mandated that the tool should be short, simple and generic in content in order to maximise its potential for routine use across the full spectrum of patients with IBD. As there is no gold standard measure of disease control in IBD, we used a wide array of external measures to validate the questionnaire, including disease activity indices, generic and disease-specific QoL questionnaires and pragmatic clinical end points (global physician assessment and treatment outcome).

Methods

Development phase

Design criteria: Based on the intended practical clinical application for the IBD-Control questionnaire, we formulated a number of guiding principles for its development. These design criteria were used as a filter to select items generated from literature review and the qualitative patient study. Essential attributes for the choice of individual question items were specified as follows: (A) Include items that capture a patient's global self-assessment of overall disease control; (B) Items must reflect generic areas of concern, function or impact that are important to patients with both main forms of IBD; (C) Items should not focus on individual gastrointestinal symptoms since their relevance and relative importance will vary depending on disease type and extent.

Criteria for the overall design of the instrument were: (A) Minimal user burden (ie, a rapid completion time with a limited total number of question items and simple response categories); (B) The tool should generate overall summary scores with clearly defined interpretation at the individual patient level; and (C) The development and validation process should provide evidence for its acceptability, reliability, validity and responsiveness based on modern psychometric standards.^20–22 The project was guided by a steering group of medical and nursing professionals with expertise in the management of IBD.

Literature review of existing PROMS used in IBD: A detailed review of the English-language literature was undertaken using the PubMed electronic database supplemented with manual searching of bibliographies and conference proceedings within relevant journals. We used combinations of the search terms ‘inflammatory bowel disease’, ‘ulcerative colitis’, ‘Crohn's disease’, ‘questionnaire’, ‘validation’, ‘quality of life’, ‘patient-reported outcome’ and ‘control’. We excluded clinician-reported measures (ie, disease activity indices) and those developed specifically for paediatric patients. Self-administered PROMS reported for adult patients with patients with IBD where categorised as either generic (instruments developed originally for general use across different diseases) or disease-specific (tools developed specifically for patients with IBD). From the literature review we generated a list of core domains (figure 1) to serve as a framework for analysing outputs of the qualitative work with patients.

Figure 1.

Flow chart summarising development and validation of IBD-Control. IBD, inflammatory bowel disease.

Qualitative study: We undertook two focus groups (six patients), one for each condition (ulcerative colitis (UC) or Crohn's disease (CD)) plus 13 one-to-one interviews with informed consent. A semistructured approach was taken to encourage patients to talk about the general concept of ‘control’ of IBD and the things they associated with ‘good control’ or ‘poor control’. Verbatim contributions were recorded and summarised as field notes by the lead interviewer (CO). These items or themes were grouped and mapped onto a framework based on the review of existing IBD questionnaires. At the end of each session, the patients were shown a selection of questionnaires and asked to contribute ideas or preferences about question format, layout, response categories, item number and completion time. ‘Saturation’ was reached when patient interviews ceased to add new items or themes to the existing framework.

Selection of items: First, we selected a set of generic questions to represent each of the core domains (figure 1), identifying where possible individual question items that have broad, general coverage rather than specific wording to conform to our design criteria. For example, the item selected for social impact would ask about missing ‘planned activities’ (rather than a more specific question about missing ‘work’, ‘study’ or ‘a social event’). To qualify for potential inclusion in the IBD-Control, each item needed to be identified as relevant to the construct of disease control in the patient contributions and validated in piloting.

Piloting and modification: To test for acceptability, lack of ambiguity and content validity we pretested the draft instrument in a pilot group of 30 patients who were asked to complete the IBD-Control and provide verbal or written feedback or annotations regarding content and format. This resulted in changes to item wording or layout.

Prospective validation phase (psychometric testing)

The prospective validation study recruited patients at the investigating centre, a university hospital serving a population of approximately 330 000 people with an established secondary care IBD service providing outpatient and inpatient medical and surgical care.²³ Patients were recruited during routine visits to the outpatient clinics prior to consultation with the doctor or specialist nurse, during other treatment-related visits (eg, azathioprine monitoring clinic or infusion visit for biologics) or at the time of admission for inpatient care. Inclusion criteria specified a confirmed diagnosis of IBD on the basis of clinical, endoscopic, radiological and/or histological criteria with disease duration of at least 6 months. Exclusion criteria specified non-English speaking subjects, cognitive impairment or serious active psychiatric disease.

After informed consent, the patients completed the IBD-Control questionnaire and were then asked to complete a questionnaire pack comprising a disease-specific QoL questionnaire (the UK-IBD-QoL),^{16 17} a generic health status instrument (EuroQol, EQ-5D-3L),²⁴ and the Hospital Anxiety and Depression Scale.²⁵ The research team undertook a simultaneous assessment of current disease activity, using the Harvey-Bradshaw Index (HBI) for CD ⁵ and the Simple Clinical Colitis Activity Index (SCCAI) for UC.⁶ The research team and clinicians were blinded to the results of patient-completed questionnaires.

Where the study visit was taking place at a scheduled clinical review (eg, outpatient attendance), the treating clinician or specialist nurse was asked to complete a questionnaire at the end of the consultation to indicate the current state of IBD (Global Physician Assessment) using a categorical scale (remission, mild, moderate or severe), blinded to patient surveys. All treatment decisions were recorded, capturing whether new therapies were started, existing drug doses changed, therapies discontinued or surgery recommended. The research team reviewed the hospital case records and clinical information systems to extract background clinical information regarding diagnosis, duration of disease, previous hospitalisation and surgery, disease extent, presence of stoma or perianal fistulae, major comorbid illness and current therapy for their IBD.

Assessment of psychometric (measurement) properties of IBD-Control

Internal consistency was assessed for categorical questions and any subscales derived from them using Cronbach's α.²⁶ In addition, we used Spearman's correlation to study the association between individual categorical questions and the rating from the instrument's visual analogue scale (IBD-Control-VAS).

Reproducibility was assessed under two sets of conditions using serial observations and the Intraclass Correlation Coefficient. First, a group of 20 patients underwent a repeat assessment at 2 weeks from baseline, completing the IBD-Control and UK-IBD-QoL on both occasions (test-retest group). In this group, we defined a stable health state based on the patient's response to the IBD-Control transition question item (Q2, response=‘No change’) plus ≤10-point change on the UK-IBD-Q total score. Second, we compared IBD-Control scores obtained from a group of patients who completed assessments at baseline and then at a subsequent routinely scheduled visit (routine care group). In this group, we defined a stable state on the basis of there being no change in the physician global assessment between visits.

Construct validity was assessed by comparing the scores derived from the questionnaire with several independent external measures of patient health state. These measures included generic QoL (EQ-5D utility index and EQ-5D visual analogue score), disease specific QoL (UK-IBD-QoL), physician global assessment and clinical disease activity indices. Bivariate correlations were expressed as Spearman's correlation coefficients and between-group mean scores were compared by t test or analysis of variance (ANOVA). Multiple variable linear regression was used to explore the relative strength and independence of individual question items as predictors of the various external measures of health state and to select items to include in subscores.

Responsiveness was assessed in those patients who completed baseline and follow-up assessments and experienced a change in their health state. There is no true external gold standard to define a change in disease control and so we performed responsiveness analyses using two alternative external criteria to define a change in health state between visits. First, we defined improvement or deterioration on the basis of self-reported UK-IBD-QoL total score, with a change of >10 points taken as a cut-off. As an alternative, we used the physician global rating to identify a change in clinical state (any change from baseline category, eg, mild to moderate). A range of responsiveness statistics were calculated including correlations in change scores, effect size, standardised response mean (SRM) and modified SRM (MSRM).

Definition and evaluation of cut-off values for detecting the ‘quiescent’ state: A key potential application for IBD-Control was to serve as a screening test to rapidly and reliably identify patients with optimal self-reported disease control. Such patients are suitable for alternative forms of chronic disease follow-up (eg, self-care, telephone or virtual clinics) rather than traditional, costly and inconvenient office or clinic attendance. We defined a state of ‘quiescent’ disease using a combination of parameters, as follows:

1. Disease-specific measures consistent with clinical remission as defined by disease activity indices within reported remission ranges (SCCAI<4; HBI<5), UK-IBD-QoL total score ≥90 points and Physician Global Assessment rated as ‘remission’. To allow for missing or incomplete data, patients were required to have sufficient data to satisfy any two of the criteria.

2. Not currently receiving oral corticosteroid therapy, a course of antibiotics, therapeutic (polymeric) diet, induction with biologics or awaiting surgical treatment.

3. IBD-Control transition question (Q2) indicates condition was unchanged or improved in the last 2 weeks (ie, excluding patients who responded that bowel symptoms were ‘worse’).

4. No escalation of therapy occurred as a consequence of the clinical consultation.

Individuals satisfying all four of the above criteria would be regarded as well-controlled. Patients who did not meet these criteria where designated as ‘not quiescent’. We used receiver-operating characteristics analysis to assess the performance of the IBD-Control as a tool for identifying this subgroup of quiescent patients. Given the intended clinical application, we defined optimum cut-off scores with an emphasis on achieving high specificity (≥85%) to minimise the risk of false positives (identifying a non-quiescent patient as quiescent). The cut-off value with the highest sensitivity×specificity product was selected, as described.²⁷

Results

Development phase

Literature review: A PubMed search identified 7244 references with a combination of search terms: [‘Patient reported outcome measure’ OR ‘PROM’ OR ‘quality of life’ OR ‘questionnaire’ OR ‘outcome measure’] AND [‘Inflammatory bowel disease’ OR ‘ulcerative colitis’ OR ‘Crohn's disease’ OR ‘colitis’ OR ‘IBD’]. Of these, 682 articles included the additional terms: [‘development’ or ‘validation’ OR ‘psychometric’]. We identified 71 relevant articles relating either to IBD-specific questionnaires (23 instruments for adults including variants or short forms; 4 for paediatrics were excluded) or to the validation of existing generic PROMS in patients with IBD (3 instruments). Of the disease-specific PROMS for adults with IBD, only the original McMaster IBD-Q (32 items) ¹⁰ and a similar instrument developed in the UK ¹⁷ were found to have undergone detailed psychometric evaluation in accordance with modern guidelines ^{20 21} and subsequent use as outcome measures in clinical trials. Hence, we chose the UK IBD-Q as our measure of QoL in the validation study. Of the generic instruments, the EuroQoL EQ-5D ²⁴ had the most publications and so was our choice of utility measure. No instrument was identified that measured IBD disease control from the patient perspective but we reviewed questionnaires for other chronic diseases.

Focus groups and one-to-one interviews: Saturation was reached after completing two focus groups and 13 one-to-one interviews. Consistently, the idea of disease control was expressed by contributors as their overall sense that disease-related symptoms were minimised or absent (ie, a global sense of control). As expected, the control of a wide variety of specific gastrointestinal symptoms was identified as important by individual patients but these did not meet our criteria for generic item selection. However, a common theme across all participants was the idea that good control implied that their particular array of bowel symptoms was minimised and not impacting on day-to-day physical functioning (overall energy and vitality; and sleep), social functioning (daily activities, work and recreational) or emotional functioning (feelings of anxiety or depression). In addition, the control of symptoms of discomfort or pain was a distinct factor. Loss of disease control was expressed as either a worsening of overall bowel symptoms or the development of an unfamiliar new symptom.

Control of IBD was linked also with ideas about the perceived effectiveness or acceptability of current treatments. This is not a core theme that appears in the established model for QoL in IBD.^9–19 Good control would imply feelings that treatment was ‘right’ or ‘was working’ whereas poor control would invoke the sense that treatment was ineffective. For some patients, the ability to control the condition themselves (ie, by making adjustments to medication according to symptoms) was seen as part of the concept of disease control. Additional treatment-related themes included perceived difficulties with using certain drugs or side effect. Good control would imply a lack of such treatment concerns.

Description and specification of the IBD-Control instrument

The IBD-Control instrument is shown in online supplementary appendix 1. The paper version comprises a single-sided A4 document that includes five sections numbered 1 through 5. The first four sections contain a series of 13 categorical questions, each of which has three response options. Of the categorical questions, 12 items have response options of ‘Yes’, ‘No’ or ‘Not sure’ and the remaining question is a transition question for overall bowel symptoms with options of ‘Better’, ‘No change’ or ‘Worse’. The fifth section contains the horizontal VAS, anchored between zero (worst possible control) and 100 (best possible control). The time horizon for disease control assessment is the ‘past 2 weeks’.

Scoring: Each response to the 13 individual items is scored as follows: zero points for least favourable reply; one point for intermediate or indeterminate reply; two points for most favourable reply. The IBD-Control-8 subscore is calculated by summing scores for Q1a, Q1b, Q3a, Q3b, Q3c, Q3d, Q3e and Q3f resulting in a range of 0–16 (0=worst control). The rationale for selecting this subgroup of eight questions is explained later in the results section. The IBD-Control-VAS scores are in the range 0–100 (0=worst control).

Validation phase

Patient sample

The characteristics of the patient sample at baseline are summarised in table 1. There were 299 patients who completed the baseline assessment, of whom 160 (53.5%) had a diagnosis of CD and 139 (46.5%) were suffering from UC. The spectrum of disease severity was very similar between the two forms of IBD (CD vs UC), with no significant difference in mean scores for generic QoL (EQ-5D utility or VAS), disease-specific QoL (UK-IBD-QoL total score) or proportion of patients in remission (physician global assessment). Of the baseline cohort, 138 patients were recruited to undertake a repeat assessment at a scheduled follow-up visit (returning patients), summarised in table 2. Patients were recruited between March 2011 and June 2012.

Table 1.

Characteristics of 299 patients with inflammatory bowel disease (IBD) at baseline assessment

Characteristic	All patients	Crohn's disease	Ulcerative colitis
Number of cases	299	160	139
Age, years	43 [16]	41 [15]	46 [16]
Gender
Male	130 (43.5%)	68 (57.5%)	77 (55.4%)
Female	169 (56.5%)	92 (42.5%)	62 (44.6%)
Disease duration, years	10 [10]	10 [10]	9 [10]
Previous surgery
Yes	83 (27.8%)	76 (47.5%)	7 (5.0%)
No	216 (72.2%)	84 (52.5%)	132 (95.0%)
Perianal disease (fistula)
Yes	27 (9.0%)	27 (16.9%)	0 (0.0%)
No	272 (91.0%)	133 (83.1%)	139 (100.0%)
Stoma present
Yes	10 (3.3%)	8 (5.0%)	2 (1.4%)
No	289 (96.7%)	152 (95.0%)	137 (98.6%)
Medication
Topical 5-ASA	36 (12.0%)	2 (1.3%)	34 (24.5%)
Topical steroid	9 (3.0%)	2 (1.3%)	7 (5.0%)
Oral 5-ASA	177 (59.2%)	68 (42.5%)	109 (78.4%)
Oral corticosteroid	26 (8.9%)	10 (6.3%)	16 (11.5%)
Standard immunosuppressants	104 (34.8%)	71 (44.4%)	33 (23.7%)
Biological agent	38 (12.7%)	30 (18.8%)	8 (5.8%)
Dietary therapy (polymeric diet)	8 (2.7%)	8 (5.0%)	0 (0.0%)
Disease activity indices
Harvey-Bradshaw Index	n/a	5 [5]	n/a
Simple Clinical Colitis Activity index	n/a	n/a	4 [3]
Quality of life questionnaires
EQ-5D utility score	0.68 [0.30]	0.65 [0.30]	0.70 [0.29]
EQ-5D visual analogue scale	65 [23]	65 [22]	65 [24]
UK-IBD-Q	86 [20]	85 [18]	88 [21]
Physician global assessment
Remission	161 (61.9%)	80 (60.6%)	81 (63.3%)
Mild	58 (22.3%)	37 (28.0%)	21 (16.4%)
Moderate	30 (11.5%)	11 (8.3%)	19 (14.8%)
Severe	11 (4.2%)	4 (3.0%)	7 (5.5%)

Continuous variables expressed as mean [SD] and categorical variables as number (%) where appropriate.

Table 2.

Characteristics of 138 patients with inflammatory bowel disease (IBD) at second visit (returning patients)

Characteristic	All patients	Crohn's disease	Ulcerative colitis
Number of cases	138	82	56
Age, years	41 [15]	38 [13]	45 [16]
Gender
Male	65 (47.1%)	36 (43.9%)	29 (51.8%)
Female	73 (52.9%)	46 (56.1%)	27 (48.2%)
Disease duration, years	8 [9]	9 [8]	8 [9]
Previous surgery
Yes	45 (32.6%)	43 (52.4%)	2 (3.6%)
No	93 (67.4%)	39 (47.6%)	54 (96.4%)
Perianal disease (fistula)
Yes	20 (14.5%)	20 (24.4%)	0 (0.0%)
No	118 (85.5%)	62 (75.6%)	56 (100.0%)
Stoma present
Yes	5 (3.6%)	4 (4.9%)	1 (1.8%)
No	133 (96.4%)	78 (95.1%)	55 (98.2%)
Medication
Topical 5-ASA	11 (8.0%)	0 (0.0%)	11 (19.6%)
Topical steroid	5 (3.6%)	0 (0.0%)	5 (8.9%)
Oral 5-ASA	82 (59.4%)	34 (41.5%)	48 (85.7%)
Oral corticosteroid	16 (11.6%)	6 (7.3%)	10 (17.9%)
Standard immunosuppressants	61 (44.2%)	44 (53.7%)	17 (30.4%)
Biological agent	29 (21.0%)	24 (29.3%)	5 (8.9%)
Dietary therapy (polymeric diet)	5 (3.6%)	5 (6.1%)	0 (0.0%)
Disease activity indices
Harvey-Bradshaw index	n/a	5 [5]	n/a
Simple Clinical Colitis Activity index	n/a	n/a	5 [4]
Quality of life questionnaires
EQ-5D utility score	0.61 [0.34]	0.60 [0.35]	0.62 [0.33]
EQ-5D visual analogue scale	61 [23]	63 [22]	56 [25]
UK-IBD-Q	83 [19]	84 [18]	82 [22]
Physician global assessment
Remission	49 (44.1%)	27 (44.3%)	22 (44.0%)
Mild	38 (34.2%)	26 (42.6%)	12 (24.0%)
Moderate	18 (16.2%)	6 (9.8%)	12 (24.0%)
Severe	6 (5.4%)	2 (3.3%)	4 (8.0%)

Continuous variables expressed as mean [SD] and categorical variables as number (%) where appropriate).

Acceptability and feasibility

Completion time was measured in a sample of 10 patients (aged between 25 years and 64 years), with a mean completion time for the IBD-Control of 1 min and 15 s (SD: 25 s). The range was between 42 s and 2 min 1 s. Completion rates for the 13 individual questions ranged between 93.3% (Q2) and 99% (Q3a and Q3d). Of the 299 patients who completed the IBD-Control at baseline visit, 259 (86.6%) provided responses to all 13 question items and 272 (91%) completed the VAS question. We prespecified that any question returning the same response from 80% or more of the patients would be considered for exclusion from the questionnaire. Such questions are unlikely to be sensitive to different levels of severity. However, none of the question items yielded the same response from 80% or more of the patients.

Internal consistency

Scored responses to each of the 13 individual questions showed significant positive correlations with the IBD-Control-VAS score, with Spearman's ρ values ranging from 0.24 to 0.70 (table 3). These findings suggest that the individual items within the questionnaire are measuring aspects of the same construct (‘disease control’) and that there is internal consistency within the questionnaire. A linear regression model retaining eight of the question items (IBD-Control-8) accounted for 70% of the variance in the IBD-Control-VAS score (R-Square 0.704). Bivariate correlation coefficients for each of the eight items within the IBD-Control-8 subscore ranged from 0.39 to 0.70.

Table 3.

Correlation between individual question items within the IBD-Control instrument and visual analogue scale for disease control (IBD-Control-VAS) and disease-specific quality of life total score measured using the UK-IBD-QoL questionnaire

Item	Question	IBD-Control-VAS	UK-IBD-QoL
Q1a	… your IBD has been well controlled in the past 2 weeks*	0.70	0.63
Q1b	… your current treatment is useful in controlling your IBD*	0.39	0.38
Q2	… have your bowel symptoms been getting better, getting worse or not changed	0.36	0.24
Q3a	… miss any planned activities because of IBD*	0.63	0.67
Q3b	… wake up at night because of symptoms of IBD*	0.61	0.67
Q3c	… suffer from significant pain or discomfort*	0.64	0.66
Q3d	… often feel lacking in energy (fatigued)*	0.51	0.63
Q3e	… feel anxious or depressed because of your IBD*	0.58	0.70
Q3f	... think you need a change to your treatment*	0.60	0.50
Q4a	… would you like to discuss alternative types of drug	0.42	0.26
Q4b	… would you like to discuss ways to adjust your own treatment	0.33	0.31
Q4c	… would you like to discuss side effects or difficulties with your medicines	0.24	0.29
Q4d	… would you like to discuss new symptoms that have developed	0.30	0.29

Item responses were scored as 0 (worst option), 1 (intermediate, eg, ‘Not sure’) or 2 (best option), so a higher score should indicate better disease control. Correlation coefficients are expressed as Spearman's ρ values (p<0.01 in all cases).

*Question items included in the IBD-Control-8 subscore.

IBD, inflammatory bowel disease; QoL, quality of life.

We assessed degree of agreement between the 13 categorical question items by Cronbach's α statistic.²⁶ The higher the α value, the higher the internal consistency between the individual questions. A score above 0.70 is regarded as desirable but values in excess of 0.90 are suggestive of excessive overlap or redundancy between individual items. In the present study, Cronbach's α was 0.85 overall for all 13 question items and 0.86 for the subscore of eight questions (IBD-Control-8). This provides further evidence for internal consistency for the ‘disease control’ construct.

The IBD-Control-8 subscore and the IBD-Control-VAS score are intended to represent a summary measure of the same construct (ie, an overall rating of disease control from the patient's perspective). Hence, the two summary scores should show a strong positive correlation with each other. This was confirmed for patients with IBD overall (r=0.811; p<0.001), and for CD (r=0.78; p<0.001) and UC (r=0.83; p<0.001) separately.

Reliability

This was tested under two sets of conditions. First, we evaluated reliability in a test-retest group of patients who completed evaluations 2 weeks apart. Of the 20 subjects, 13 reported no change in their disease control over the past 2 weeks (IBD-Control transition question, Q2) and had stable UK-IBD-QoL scores (±10-points from baseline). There was no significant difference in mean IBD-Control-8 or IBD-Control-VAS scores between visits and the intraclass correlation coefficient was 0.97 for the IBD-Control-8 subscore and 0.96 for IBD-Control-VAS (table 4A; p<0.01).

Table 4.

Reproducibility of IBD-Control summary scores for stable patients

Instrument score (scale)	Mean difference (Visit 2—Visit 1)	SD of difference	Intraclass correlation (95% CI)
(A) Patients returning at 2 weeks (test-retest group)*
IBD-Control-8 subscore (0–16)	−0.45	1.81	0.97 (0.90 to 0.99)
IBD-Control-VAS (0–100)	+2.25	9.79	0.96 (0.88 to 0.99)
(B) Patients returning for a scheduled clinic visit (routine care group)†
IBD-Control-8 subscore (0–16)	0.00	3.72	0.87 (0.71 to 0.94)
IBD-Control-VAS (0–100)	−0.56	23.00	0.81 (0.58 to 0.91)

*Data for 13 patients responding ‘not changed’ to the IBD-Control transition question (‘Over the past 2 weeks have your bowel symptoms been getting worse, getting better or not changed’) and with stable UK-IBD-Qol total scores (<10-point change between two visits).

†Data for 32 patients with a stable (unchanged) physician global assessment rating between two visits (mean interval between visits 131 days).

Two-way fixed effects model for consistency, p<0.001.

IBD, inflammatory bowel disease; QoL, quality of life; VAS, visual analogue scale.

Second, reliability was assessed in 32 patients who completed questionnaires at consecutive clinical visits and were categorised as having no change in their physician global assessment between the two visits. The mean interval between the two visits was 131 days. Again, there was no significant difference in mean summary scores on the IBD-Control questionnaire between visits. For these patients, intraclass correlation coefficient was 0.87 for the IBD-Control-8 subscore and 0.81 for IBD-Control-VAS (table 4A; p<0.01).

Construct validity

There is no true gold standard for measuring disease control from the patient's perspective. However, it is possible to test a number of hypotheses concerning the logical relationships that should exist between the item scores or summary scores obtained with the IBD-Control instrument and a range of other external criteria, measures or patient characteristics.

Validity of individual question items: Good disease control would be expected to be associated with better QoL. Consistent with this, each individual question item within the IBD-Control demonstrated a significant positive correlation with the total score for the UK-IBD-QoL (table 3). Hence, ‘positive’ responses to questions within the IBD-Control are correlated with better overall disease-specific QoL. The correlation coefficients for each of the IBD-Control-8 subscore items and total UK-IBD-QoL score ranged from 0.38 to 0.70. These findings suggest the individual items within the instrument are valid.

We used a single question item to capture feelings of anxiety or depression (‘… felt anxious or depressed because of your IBD’). Responses to this question (No, Not Sure, Yes) showed moderate-to-strong correlation with the summary scores for the multi-item Hospital Anxiety and Depression Scale questionnaire ²⁵for anxiety and depression subscores (r=0.57 and r=0.62, respectively, p<0.001).

Validity of IBD-Control summary scores: The summary scores generated by the disease control measure are predicted to correlate positively with disease-specific (UK-IBD-QoL) and generic (EQ-5D utility and VAS) QoL scores. Consistent with this, we found moderate-to-strong positive correlations between the IBD-Control summary scores (IBD-Control-8 and IBD-Control-VAS) and each of the QoL scales (table 5 and figure 2). A linear regression model containing the eight items from the IBD-Control-8 subscore accounted for 75% of the variance in UK-IBD-QoL total score (R Square 0.747).

Table 5.

Construct validity: Correlations between the IBD-Control summary scores and external measures of health status in patients with inflammatory bowel disease (IBD) at baseline assessment

External measure	IBD-Control-8			IBD-Control-VAS
	All	CD	UC	All	CD	UC
UK-IBD-QoL score	0.86	0.81	0.90	0.76	0.65	0.85
EQ-5D utility score	0.70	0.68	0.67	0.52	0.50	0.58
EQ-5D VAS	0.68	0.65	0.71	0.70	0.69	0.71
Physician global assessment	−0.58	−0.45	−0.67	−0.58	−0.47	−0.65
Harvey-Bradshaw Index	n/a	−0.68	n/a	n/a	−0.60	n/a
Simple Clinical Colitis Activity Index	n/a	n/a	−0.72	n/a	n/a	−0.75

Correlation expressed as Spearman's ρ values (p<0.01 in all cases).

CD, Crohn's disease; QoL, quality of life; UC, ulcerative colitis, VAS, visual analogue scale.

Figure 2.

Validity of IBD-Control summary scores in relation to UK-IBD-QoL Questionnaire. IBD, inflammatory bowel disease; QoL, quality of life.

Conversely, negative correlations would be expected to exist between IBD-Control summary scores and traditional clinical measures of disease activity or severity. In the present study, we obtained clinician-reported outcomes in the form of traditional symptom-based activity indices (HBI or SCCAI) and also as a global physician rating of current disease activity. We found highly significant moderate-to-strong correlations between the IBD-Control scores and the disease severity measures in the expected direction (ie, negative coefficients) (table 5).

These associations provide strong evidence to support the construct validity of the IBD-Control instrument. In addition, a valid measure of disease control would be expected to differentiate between mutually exclusive groups of patients categorised using an accepted external clinical criterion. Hence, mean scores for disease control would be expected to be significantly different between patients ‘in remission’ and those with ‘active disease’. Comparison of mean IBD-Control-8 and IBD-Control-VAS scores across the four physician global assessment categories (remission, mild, moderate or severe) confirmed good discriminant validity across the spectrum of disease (figure 3).

Figure 3.

Validity of IBD-Control summary scores in relation to physician global assessment.

Multiple linear regression models examined whether additional patient factors (age, gender, disease type, previous surgery) were independently associated with IBD-Control scores after taking account of current disease severity. All models confirmed the strong association between IBD-Control summary scores and disease activity indices, generic and disease-specific QoL and global assessment. However, no consistent independent associations were observed for the other predictor variables.

Mapping of IBD-Control-8 to EQ-5D utilities: Ease of use makes the IBD-Control a candidate outcome measure for large scale observational clinical studies, trials, surveys or registries. Hence, the ability to convert IBD-Control summary scores to an estimate of utility could support health economic evaluations. Figure 4 illustrates the association between IBD-Control-8 subscore and the EQ-5D utility index.

Figure 4.

Validity of IBD-Control-8 subscore in relation to EuroQol (EQ-5D) utility index. IBD, inflammatory bowel disease; QoL, quality of life.

Items not included in the IBD-Control-8 subscore: Five questions were excluded from the IBD-Control-8 summary subscore. The transition question (Q2) was excluded since this asks about change in status over 2 weeks rather than measuring current health status. The remaining questions (grouped together as items Q4a to 4d in the final questionnaire) had significant but weaker correlations with IBD-Control-VAS and/or UK-IBD-QoL and did not add significantly to performance of linear regression models predicting these measures. However, the development of new symptoms (Q4d) was regarded as an important ‘stand-alone’ item to retain in the IBD-Control instrument since a new symptom should serve as an automatic trigger for traditional clinical evaluation.

The three remaining questions (Q4a to 4c) are related to treatment concerns. These items were found to be predictive of the clinical outcome of the consultation in terms of likelihood of treatment escalation (note that clinicians were blinded to questionnaire responses). Of 67 patients reporting no treatment concerns on the three items, only 5 (7.5%) had their treatment escalated, whereas 59 of 192 (30.7%) patients with indeterminate or positive responses to at least one of the three questions had treatment escalated (p<0.001, χ² test; OR: 4.1 (95% CI 1.7 to 9.8)).

Responsiveness

A vital measurement property for any PROM is that the instrument is sensitive to gains or losses in the measurement concept of interest. We evaluated the responsiveness of the instrument by comparing serial data collected in the patients who returned for a second visit.

Change scores: A crude method for assessing responsiveness is to calculate ‘change scores’ for the instrument (ie, subtract score at visit 1 from the score at visit 2) and to examine the correlation between this value and a change score derived from another relevant clinical measure. For change scores in IBD-Control-8 subscore and IBD-Control-VAS, there were significant correlations with change scores for each of the key external outcome measures (r values ranging from 0.25 to 0.78). The strongest correlation for change in IBD-Control scores were versus the change in UK-IBD-QoL total score (IBD-Control-8, r=0.72; IBD-Control-VAS, r=0.55; p<0.001) and the weakest for HBI (IBD-Control-8, r=−0.25; IBD-Control-VAS, r=−0.32; p<0.001).

Responsiveness statistics: Several methods have been proposed to provide more robust quantitative expressions of the magnitude and meaning of changes in instrument scores. Effect size (ES) is calculated as the size of the change in scores (ie, the difference between mean scores at the baseline and repeat assessments) divided by the SD of baseline scores. Benchmarks for assessing the relative size of change have been proposed whereby an ES of 0.2 is considered small, 0.5 as medium and ≥0.8 as large. The IBD-Control-8 and IBD-Control-VAS showed moderate-to-strong ES with values ≥0.76 (table 6) indicating good responsiveness.

Table 6.

Responsiveness statistics for the IBD-Control summary scores in subjects who improved or deteriorated. Data for stable patients (no change) are included for reference

Criteria for change in state	n	Mean difference (SD)	Effect size*	Standardised response mean†	Modified standardised response mean‡
(A) IBD-Control-8 sub-score
UK-IBD-QoL
Improved by >10 points	19	+5.50 (4.69)	1.44	1.17	2.72
No change (±10 points)	63	+0.13 (2.02)	0.02	0.06	0.06
Deteriorated by >10 points	26	−4.92 (3.88)	0.99	1.27	2.44
Physician global assessment
Improved by ≥1 point	27	+4.52 (5.56)	1.08	0.81	1.22
No change	32	0.00 (3.72)	0.00	0.00	0.00
Deteriorated by ≥1 point	27	−3.32 (4.81)	0.76	0.69	0.89
(B) IBD-Control-VAS
UK-IBD-QoL
Improved by >10 points	19	+28.21 (31.33)	1.09	0.90	1.84
No change (±10 points)	63	+0.84 (15.34)	0.03	0.05	0.05
Deteriorated by >10 points	26	−26.85 (28.20)	0.90	0.95	1.75
Physician global assessment
Improved by ≥1 point	27	+29.55 (31.78)	1.14	0.93	1.28
No change	32	−0.56 (23.00)	0.02	0.02	0.02
Deteriorated by ≥1 point	27	−19.71 (30.53)	0.76	0.65	0.86

*Effect size (ES)=Mean difference divided by the SD of the baseline absolute scores for the group.

†Standardised response mean (SRM)=Mean difference divided by SD of the difference for the group.

‡Modified SRM (MSRM)=Mean difference for the group divided by SD of the difference for unchanged patients.

IBD, inflammatory bowel disease; QoL, quality of life.

The SRM differs from ES in that the denominator is the SD of change scores (rather than of baseline scores). This takes account of variability in change per se rather than variability in absolute baseline scores. Responsiveness of the IBD-Control summary scores measured by SRM was strong (table 6).

Finally, in the modified SRM (MSRM) the numerator remains the mean change in absolute scores for the group but for this statistic the denominator is the SD of change scores taken specifically from individuals who are identified by other means as clinically stable. In the present study, this was taken from a subgroup of patients who showed no change in physician global rating between assessments and had a stable disease-specific QoL score (UK-IBD-Q total score changed by no more than 10 points between visits). The IBD-Control-8 and the IBD-Control-VAS scores have MSRM values ≥0.86 (table 6).

Sensitivity and specificity of IBD-control in identifying ‘quiescent’ patients

Of 217 patients at baseline who had formal clinical assessment and all relevant data, 80 individuals could be categorised as quiescent at the time of their routinely scheduled clinic visit. None required treatment escalation (including induction with biologics or referral for surgery). Receiver-operating characteristics analysis for IBD-Control-8 subscore and IBD-Control-VAS indicated strong performance could be achieved with either measure as a test to identify quiescent patients from the remaining cases (figure 5). For IBD-Control-8, a cut-off of ≥13 points identified patients with quiescent IBD with 67.5% sensitivity and 90.6% specificity, whereas for IBD-Control-VAS a cut-off of ≥85 points achieved 64.3% sensitivity and 90% specificity. These cut-off scores would mean that only 1 in 10 patients failing to meet our stringent clinical criteria for quiescent IBD would be falsely categorised as quiescent but none of these ‘false positives’ required any treatment change at the time of consultation and all were considered to be either in remission or only mildly active by the attending clinician. These data confirm the potential clinical application of IBD-Control as a rapid triage tool to categorise low-risk (quiescent) patients suitable for alternative forms of follow-up.

Figure 5.

Performance of IBD-Control as a screening test to identify ‘quiescent’ patients: Receiver operating characteristic (ROC) curves for IBD-Control-8 subscore (solid line; area under curve: 0.90; p<0.001) and IBD-Control-VAS (broken line; area under curve: 0.86; p<0.001). For IBD-Control-8, a cut-off of ≥13 points identified patients with quiescent IBD with 67.5% sensitivity and 90.6% specificity, whereas for IBD-Control-VAS a cut-off of ≥85 points achieved 64.3% sensitivity and 90% specificity. The quiescent state was defined using strict composite criteria (see text). IBD, inflammatory bowel disease.

Discussion

The IBD-Control questionnaire is a novel PROM designed to provide a rapid, user-friendly assessment of disease control from a patient's perspective. It was developed with patient involvement in defining the measurement construct (ie, what is ‘disease control’?) and in generating patient-centred items. Key requirements were that IBD-Control should be applicable to routine clinical care and that it would contain items relevant to both main forms of IBD. This resulted in the generation of a short and deliberately generic set of questions.

As expected, there was considerable consensus and overlap between items or domains identified by patient discussions of disease control and the themes reported in the extensive literature on QoL in IBD.^{9–19 28–30} Hence, good control implies the absence of disease impacts on energy/vitality, ability to perform daily activities, sleep, mood and freedom from pain or discomfort. Question items on these topics feature prominently in short forms of existing disease-specific QoL questionnaires and/or in the generic instruments that have been applied to IBD. In addition, our qualitative study identified a discrete domain of ‘treatment concerns’, encompassing perceptions of effectiveness of treatment, satisfaction with choice of therapy, ability to adjust own treatment and concern about side effects and we incorporated these items into the IBD-Control.

This study shows that IBD-Control is acceptable to patients based on its rapid completion time and high completion rates. Acceptability is a vital characteristic for any PROM if the intended application is to support routine care—a property that may be lacking with the current multi-item, multidomain QoL measures. Despite its simplicity, the IBD-Control exhibits strong measurement properties.

Strengths of the present study include our emphasis on patient involvement in item generation and questionnaire design to complement literature review and clinical opinion. Furthermore, we adopted a comprehensive approach to assessing the full range of psychometric (measurement properties) of the questionnaire in order to conform to current international standards.^20–22 This required the simultaneous collection of a wide array of patient-reported and clinician-reported outcome measures in a large cohort of patients. A key strength of the IBD-Control tool is that it has been developed primarily for clinical application and to support patient-centred care. Hence, our validation has not rested simply on showing correlations with traditional disease measures across the patient population but has considered the performance of summary scores at individual level and for a specific clinical purpose.

We were particularly interested in generating a tool that would reliably identify a patient in a ‘quiescent’ state, defined stringently using a composite of criteria including physician rating and treatment outcome. This has application to models of care that seek to reduce the need for traditional face-to-face consultation (eg, ‘virtual’ clinics). The instrument produces scores relevant to the planning of day-to-day care, serving as a potential trigger or triage tool to guide the necessity, timing and nature of more formal clinical assessment. The inclusion of items relating to concern about treatment or new symptoms further distinguishes the IBD-Control from traditional QoL tools. Hence, it is intended to provide information to support patient-focused consultations and summary scores to support traditional clinical assessment and decision-making.

Potential limitations of our approach to PROM development was the adoption of rigid, prespecified design criteria that mandated the IBD-Control should contain only a small number of ‘general’ items with simple response categories to optimise acceptability. This meant a pragmatic approach was taken to item selection to favour broad, generically worded questions covering core domains with active exclusion of items about individual gastrointestinal symptoms. We did not undertake a process of generating a very large number of items followed by factor analysis and item reduction. We believe our approach was justified by the wealth of existing published information identifying key areas of concern for patients with IBD. We used qualitative data from patients to confirm that each item was relevant to the concept of disease control. Nevertheless, there was a risk that producing an instrument to this strict specification could have resulted in suboptimal measurement properties. However, the findings of our prospective evaluation suggest quite the opposite—the IBD-Control questionnaire has very strong properties when compared with a wide array of external measures.

There is inevitable selection bias inherent in recruiting patients willing to complete multiple questionnaires for a validation study. However, our patient sample size was large and contained individuals with a broad spectrum of demographic and clinical attributes that are similar to our local denominator IBD population.²³ Although the proportion of cases with moderate-to-severe disease was only 15–20%, the linear trends and consistent confidence limits for IBD-Control scores observed across the disease spectrum suggests sufficient cases in the study. By recruiting patients at the time of ‘real-life’ clinical visits and asking for IBD-Control completion before other assessments we sought to reproduce the intended application for the tool. Clearly, further validation in other settings and in specific patient subgroups is desirable.

Current guidelines for IBD do not endorse the use of any specific PROM to guide the organisation or monitoring of care nor to support patient-level decision-making. Hence, currently available PROMs serve largely as research tools. This contrasts with some other chronic diseases where the use of specific patient-centred outcome measures has been incorporated into guidelines and care pathways.¹ Recently, quality improvement programmes for IBD in a range of healthcare settings have highlighted the potential for routine capture of PROMs to help drive better patient-centred services.^{31 32} We suggest that the simplicity of the IBD-Control in combination with its strong measurement properties and relevance to both forms of IBD make it ideally suited to support such initiatives, in addition to having a potential role as an outcome measure in pragmatic clinical trials. As an addition to routine clinical care, the instrument offers the realistic possibility of capturing serial PROM data with minimal user burden and its content is eminently suited to adaptation to electronic capture via web-based systems, portals or mobile devices.