Estimating Minimally Important Differences for the Bladder Cancer Index Using Distribution and Anchor Based Approaches

Abstract

Purpose:

The BCI (Bladder Cancer Index) is a validated, condition specific health questionnaire assessing urinary, bowel and sexual function and quality of life among patients with bladder cancer. We aimed to establish minimally important difference score thresholds that signal clinical importance.

Materials and Methods:

For 1 year after surgery we followed a prospective cohort of 150 patients who underwent radical cystectomy between 2013 and 2016. Usable data on 138 patients were analyzed. The BCI and the Medical Outcomes Study SF-36 (36-Item Short Form Health Survey) questionnaires were completed prior to cystectomy, and 3, 6 and 12 months postoperatively. Distribution based, minimally important differences were estimated at ⅓ and ½ SD for each index domain across time points. Changes in index domain scores anchored to changes in a SF-36 overall health assessment question were used to estimate anchor based, minimally important differences. Pooled averages are reported between time points and methods.

Results:

The distribution based, minimally important difference of ⅓ SDs for urinary, bowel and sexual domains ranged between 5.3 and 7.3, 4.6 and 5.6, and 6.0 and 8.9 points, respectively. Ranges of ½ SDs were 8.8 and 10.9, 6.8 and 8.4, and 8.9 and 13.5 points, respectively. The anchor based approach resulted in minimally important difference estimates of 6.2, 7.3 and 6.8 points, respectively. Aggregated results across the 2 approaches resulted in minimally important differences of 6 to 9, 5 to 8 and 7 to 11 points for urinary, bowel and sexual domains, respectively.

Conclusions:

Using 2 independent approaches to our knowledge we established the first minimally important difference estimates for the BCI. Defining patient reported outcome thresholds is important to interpret changes or differences in BCI scores.

Patient reported outcomes are defined as any report of the status of a patient health condition which comes directly from the patient without interpretation by a clinician or anyone else.¹ Outcomes include symptoms, certain aspects of functioning, general QOL and HRQOL. As key research measures used in clinical trials and patient centered studies, PROs enable researchers and clinicians to comprehensively assess the effects of illness and medical treatments by capturing information which only patients can report.^2,3 Some patient reported outcome measures are also used in clinical care to assess patient symptoms. As they become more widespread in research and clinical arenas, consumers of PROs will need reference points to help interpret scores.

While reference ranges and score thresholds are available for some measures, they do not exist for most patient reported outcome measures. In such cases changes or differences in PRO scores which are clinically important or meaningful can be determined by methods that estimate MIDs.⁴ A MID is the smallest change in an outcome which reflects a change in clinical condition or intervention and which a patient would identify as important.⁵ MIDs have been developed for disease specific HRQOL instruments such as the EPIC (Expanded Prostate Cancer Index Composite) and general HRQOL measures such as the SF-36 and the HAQDI (Health Assessment Questionnaire-Disability Index).^6,7 Although a number of bladder cancer specific PRO measures exist, such as the BCI, the FACT-Bl (Functional Assessment of Cancer Therapy-Bladder Cancer) and the VCI (Vanderbilt Cystectomy Index), to our knowledge MIDs have not been developed for any of them.

Because patients with bladder cancer treated with cystectomy face significant functional and QOL deficits, QOL outcomes are important for patient counseling before surgery and for patient evaluation after cystectomy. However, the unavailability of reference ranges to help clinicians and researchers interpret PRO scores hinders use and usefulness.^8–10 Accordingly developing and validating MIDs should be a priority to enable the interpretation of PRO scores and advance PRO oriented research. The objective of this study was to develop cystectomy specific MIDs for the BCI.

METHODS

Study Population

Patients with bladder cancer treated with cystectomy at 1 of our 2 centers were enrolled in a prospective, longitudinal QOL and complication study between 2013 and 2016. Inclusion criteria included men and women 18 to 89 years old who were diagnosed with nonmetastatic (cT14aN0M0) bladder cancer and scheduled for cystectomy. Study participants completed a battery of health questionnaires before and after surgery.

Of the 150 patients enrolled in the study 132 underwent cystectomy and completed preoperative and postoperative questionnaires. A total of 77 patients who completed all surveys at every time point were included in the final analysis. Institutional review board approval was obtained and all patients provided written informed consent.

Study Measures and Data Collection

The primary study objective was to estimate MIDs of each BCI domain based on changes in BCI scores across study time points. The BCI is a reliable, disease specific HRQOL instrument containing 36 items which assess 3 domains (urinary, bowel and sexual) with function and bother subdomains.^11,12 It is responsive to diversion specific symptoms and functional changes after cystectomy,¹³ and has served as a primary outcome measure in several bladder cancer outcome studies.^9,14 Scores range from 0 to 100 with higher scores corresponding to better functioning and HRQOL. To our knowledge MID thresholds indicating clinically significant changes or differences have not previously been estimated.

The SF-36 is a general health measure that is widely used across disease and health conditions. It consists of 36 items which ask respondents about general and mental health as well as physical, emotional and social functioning.¹⁵ A single general health item was used as an anchor in this analysis.

The BCI and the SF-36 were administered preoperatively, and at 3, 6 and 12-month followup intervals after cystectomy and urinary diversion. Responses from returned surveys were double entered into a secured online survey database (VisionTree, San Diego, California) and scored according to standard scoring algorithms for each instrument. Using standardized study forms we collected sociodemographic factors, the Charlson comorbidity index, disease characteristics (clinical and pathological stages, histology and prior management strategies), the treatment received (neoadjuvant chemotherapy, surgical approach and diversion type) and followup clinical data (laboratory data and complications). These data were entered into REDCap™, a HIPAA (Health Insurance Portability and Accountability Act) compliant study data registry.

Distribution Based and Anchor Based Minimally Important Difference Estimations

Distribution based and anchor based approaches were used to estimate MIDs.^4,16,17 The distribution based approach relies on the statistical distribution of PRO scores and a measure of variability such as the SD. A ½ SD approximates a medium effect size and a ⅓ approximates a small effect size.^4,18 Although the distribution based method does not directly associate a specific clinical change with the magnitude differences in the PRO, it is a simple and well established method to express a standardized metric of observed change.^{7,16,17,19,20} We calculated BCI score means and SDs across BCI domains at each time point. The SD of each BCI domain across time points was divided by 2 and 3 to establish ½ SD and ⅓ SD estimates, respectively. They were recorded as distribution based MID thresholds.

The anchor based approach compares changes in a validated external indicator to the change in the score of the PRO instrument under evaluation, thereby anchoring changes in the test PRO instrument to an external criterion. A global assessment tool in which patients report overall health as better, unchanged or worse typically serves as an external criterion to distinguish meaningful changes in scores.^4,16,19 As our anchor we selected item 1 from the SF-36 questionnaire, “In general, would you say your health is...” with corresponding Likert scale choices of 1) poor, 2) fair, 3) good, 4) very good and 5) excellent.¹⁵ This criterion item had satisfactory correlation with the total item SF-36 questionnaire (0.66) and the SF-36 general health perceptions domain (0.83) averaged across all time points, supporting its use as an anchor as described previously.⁷

Anchor based estimates were derived by 2 previously described approaches,^4,16 first by anchoring BCI changes to any change in the SF-36 criterion item between time points and second by anchoring BCI changes only in case of a 1-point change in the SF-36 item between time points. For each interval (eg pretreatment to 3, 3 to 6 and 6 to 12 months) patients were grouped by changes in the response to the SF-36 anchor question compared to the preceding time point. Cases in which SF-36 responses were the same for both time points delimiting an interval were recorded as no change and not included in analysis. For the any change approach patients were categorized by an increase or decrease in the SF-36 anchor score between time points regardless of the magnitude of the change. In this approach response changes from fair to good (a 1-point change) and from fair to very good (a 2-point change), for example, were grouped together. Concurrent changes in BCI scores were then calculated and recorded across all patients in these groups as defined by the change in the SF-36 criterion item.

Next, patients were regrouped using a 1-point change threshold. In this approach we considered only patients with an adjacent ±1 point change in the SF-36 anchor response. For example, a change from excellent to very good (a -1-point change) was included but a change from excellent to fair (a —3-point change) was not included. Although a 1-point change grouping may conceptually approximate a minimal difference more directly, both approaches are routinely used.¹⁶ Concurrent changes in BCI scores were then calculated and recorded across all patients in the criterion defined groups. Figure 1 shows a conceptual framework for the anchor based approach.

Figure 1.

Anchor based method to identify MID in BCI scores between time points by grouping patients according to response change in SF-36 item 1 anchor question, including BCI score distribution at time 1 (gray curve), and in patients with 1-point or greater increase (green curve) and 1-point or greater decrease (pink curve) at time 2.

To ensure that a robust, anchor based approach was used we averaged changes in BCI scores during the study intervals and across the 2 anchor based methods. Changes in BCI scores which were discordant with the direction of the SF-36 anchor item were excluded from study. This means that if the SF-36 anchor item response improved and the BCI domain response worsened, the BCI score change was excluded from the cumulative averaged result. Distribution and anchor based BCI MID results were then combined to form aggregated MID ranges for each domain.

All analyses were performed with standard SAS®, version 4.2 statistical software and testing was 2-sided.

RESULTS

A total of 132 patients with bladder cancer, including 99 men and 33 women, with a median age of 70 years (IQR 63—76) participated in the study and returned surveys. The supplementary table (https://www.jurology.com/) shows demographic information. Across time points the BCI urinary domain SD ranged from 16 to 21.8 points and the bowel domain SD ranged from 13.7 to 16.8 points. The sexual domain had the highest variation with SDs ranging from 18 to 27 points. Table 1 lists mean BCI scores with the corresponding distribution based MIDs at ⅓ and ½ SD. When averaged over the study intervals, the distribution based ⅓ and ½ SD MIDs were 6.1 and 9.1 points for the urinary domain, 5.3 and 8.0 points for the bowel domain, and 7.2 and 10.8 points for the sexual domain, respectively. Mean BCI scores for the urinary and bowel domains were similar across study assessment points but sexual domain scores initially decreased and then improved slightly at 12 months (table 1).

Table 1.

Distribution based BCI scores and corresponding MIDs with time based on ½ and ⅓ SDs across BCI domains

			MID
BCI domain	No. Pts	Mean BCI	⅓ SD	½ SD
Pretreatment:
Urinary	73	78.6	7.26	10.89
Bowel	75	79.7	5.62	8.43
Sexual	64	48.5	8.99	13.49
3 Mos:
Urinary	72	75.6	6	9.01
Bowel	77	75.3	5.61	8.41
Sexual	65	32.1	6	8.99
6 Mos:
Urinary	74	78.3	5.86	8.8
Bowel	74	80.9	4.57	6.86
Sexual	65	31.3	6.59	9.89
12 Mos:
Urinary	77	81.6	5.33	7.99
Bowel	73	80.8	5.41	8.12
Sexual	68	36.3	7.1	10.65

Table 2 shows the distribution of SF-36 anchor responses at study assessment times and changes in scores during the intervals. Of the patients 74% to 79% evaluated their general health as good or better across the study. Table 2 also lists the counts, proportions and direction (ie increase or decrease) of patients with any change in SF-36 anchor question over each time interval. Approximately half of the cohort reported no change in the SF-36 anchor question at each interval.

Table 2.

SF-36 anchor distribution and response changes at study assessment intervals

	No. Assessment Interval (%)
	Baseline	3 Mos	6 Mos	12 Mos
Excellent	12 (9.0)	8 (7.8)	11 (11.6)	9 (10.2)
Very good	38 (28.6)	31 (30.1)	31 (32.6)	25 (28.4)
Good	52 (39.1)	37 (35.9)	28 (29.5)	36 (40.9)
Fair	26 (19.5)	25 (24.3)	18 (18.9)	16 (18.2)
Poor	5 (3.8)	2 (1.9)	7 (7.4)	2 (2.3)
Response change since previous assessment:	–
Decrease		20 (27.0)	13 (17.6)	17 (23.0)
Increase		21 (28.4)	19 (25.7)	15 (20.2)
No change		33 (44.6)	42 (56.7)	42 (56.8)

Table 3 shows BCI domain score changes at each interval stratified by anchor based approaches as a 1-point change and as any change in the SF-36 anchor groups. The average pooled MID across all time points and variations of the anchor based approaches for the directionally concordant SF-36 item 1 anchor and the BCI domain changes were 6.15, 7.31 and 6.78 points for urinary, bowel and sexual BCI domains, respectively.

Table 3.

Difference in BCI scores between time points using any method and 1-point change anchor based method on SF-36 anchor item 1, “In general, would you say your health is...”

	BCI Score Difference (mos)
	3	6	12	Av Across Methods*
Urinary:				6.15
1-Point decrease	8.81*	9.42	6.08
1-Point increase	11.24*	1.77*	0.93*
Any decrease	11.02*	8.1	5.48
Any increase	9.42*	4.96*	1.01*
Bowel:				7.31
1-Point decrease	6.81*	10.56	2.99*
1-Point increase	4.22	12.38*	4.42
Any decrease	7.57	10.07	3.29*
Any increase	0.91	10.8*	3.75
Sexual:				6.78
1-Point decrease	13.06*	2.25*	0.2*
1-Point increase	9.36	1.49*	12.71*
Any decrease	17.34*	2.86*	0.9*
Any increase	11.67	5.18*	11.8*

^*Concordance in direction of SF-36 and BCI change.

Figure 2 shows MID ranges estimated by distribution and anchor based approaches. Aggregate results based on pooled averages of the 2 approaches suggested clinically meaningful MIDs of 6 to 9 points for the urinary domain, 5 to 8 points for the bowel domain and 7 to 11 points for the sexual domain (table 4).

Figure 2.

Average BCI MIDs using distribution and anchor based (green bars) methods across all domains. Blue bars indicate ⅓ SD. Red bars indicate ½ SD.

Table 4.

Aggregated results of BCI MIDs by domain

Domain	Mean BCI MID/Median (range)
Urinary	7/6 (6–9)
Bowel	7/7 (5–8)
Sexual	8/7 (7–11)

DISCUSSION

In the absence of reliable reference ranges or thresholds it is critical to distinguish changes or differences in scores which are and are not clinically meaningful to evaluate and interpret PROs. Therefore, reference points which signal clinical importance are needed to guide the interpretation of PROs used in clinical trials and comparative effectiveness research studies, and to inform clinical assessments of patient care.

Among patients with bladder cancer the BCI is one of the few psychometrically tested and validated, condition specific health questionnaires.^{10–12,21,22} In addition to its growing application in research, a recent comparative study suggested that the BCI may better assess function and QOL after cystectomy compared to another commonly used cystectomy specific questionnaire.¹³ Using 2 independent but complementary approaches to determine the MIDs of BCI domains in patients with bladder cancer who underwent radical cystectomy, we estimated initial MID score ranges of 6 to 9 points for the urinary domain, 5 to 8 points for the bowel domain and 7 to 11 points for the sexual domain.

Two recent QOL outcome studies among patients who underwent cystectomy underscore important differences in how results can be interpreted based on criteria indicating clinical or statistical significance. In a study of 31 patients who received an ileal conduit and 33 treated with an orthotopic neobladder Moncrief et al reported a statistically significant 20-point difference in urinary function scores favoring the ileal conduit (p = 0.002) but they failed to find a statistically significant difference in other BCI domains.¹³ The difference in the urinary domain score was statistically significant and would also be considered clinically significant according to our MID estimates of 6 to 9 points. However, sexual domain scores, which were 11 points higher in patients with a neobladder than in those with an ileal conduit, did not statistically differ (p = 0.61), although the observed difference would be considered clinically meaningful according to our MID criteria (estimated sexual domain MID of 7 to 11 points). In another 128 patients with bladder cancer treated with cystectomy who were evaluated by the BCI 5 years after surgery Gellhaus et al noted that statistically significant differences between patients with an ileal conduit, a neobladder or a catheterizable pouch were consistent with clinically meaningful differences as determined by the MID estimates of urinary and sexual domain scores.⁹ These 2 examples point to the importance of the magnitude of score changes and differences, and the usefulness of MIDs to judge clinical significance, particularly given the limitations of relying on statistical significance alone.

In our study we identified ranges of clinically meaningful differences in patient reported BCI scores. In addition to being the first study to our knowledge to develop MIDs for a bladder cancer specific QOL instrument, the strengths of our study include prospective, longitudinal PRO assessment, which avoided recall bias, and the use of multiple estimation approaches to triangulate MIDs.

Notwithstanding these strengths, several limitations should be discussed. 1) Some changes in BCI scores and SF-36 anchors were in different directions, meaning that an increase in the BCI score was accompanied by a decrease in the SF-36 item or vice versa. These cases were not included in our primary analyses, further limiting our sample from which to estimate MIDs. Opposing directions in score changes may arise for a number of reasons. For example, patients may experience problems detected by 1 instrument while there is improvement in the domains captured by the other instrument. A general 1-item anchor may also be less sensitive or responsive to changes in health states or too dissimilar to expect consistent directional agreement between the anchor and the PRO.

As described, when estimating MIDs, the magnitude of change is more important than the direction of change. Therefore, score changes in patients whose condition is becoming worse vs better can be examined separately or pooled.⁴ To account for this we calculated MID averages across all intervals dependent and independent of directional agreement between the BCI score change and the SF-36 anchor, and noted similar MID estimates for both approaches.

2) Because we estimated MID ranges among patients with bladder cancer treated with cystectomy, the MIDs should be considered cystectomy specific, although the BCI was designed to evaluate patients with bladder cancer with or without cystectomy. Therefore, MID estimates may differ in noncystectomy cases.

3) We used a single global health anchor to estimate MIDs in the anchor based approach. Additional approaches to triangulate MID estimates include using multiple global health anchors and responder estimations, which involves retrospective self-reports and is often subject to recall bias. Nevertheless, our distribution and anchor based approaches are well established, accepted methods to estimate MIDs.^4,7,16’¹⁷

CONCLUSIONS

The findings of this study provide an initial range of MIDs across BCI domains which estimate clinically meaningful differences and/or changes in scores. Although these results should be validated in subsequent studies, they provide a basis to estimate clinically important differences and changes in BCI scores after radical cystectomy.

Abbreviations and Acronyms

BCI

Bladder Cancer Index

HRQOL

health related quality of life

MID

minimally important difference

PRO

patient reported outcome

QOL

quality of life

SF-36

36-Item Short Form Health Survey