Responsiveness to Change and Minimally Important Differences of the Patient-Reported Outcomes Measurement Information System Gastrointestinal Symptoms Scales

Abstract

Background

The NIH-sponsored Patient-Reported Outcomes Measurement Information System (PROMIS) Gastrointestinal (GI) Symptoms scales were developed to assess patients’ GI symptoms in clinical settings.

Aims

To assess responsiveness to change and provide minimally important difference (MID) estimates for the PROMIS GI Symptoms scales.

Methods

A sample of 256 GI outpatients self-administered the eight PROMIS GI Symptoms scales (gastroesophageal reflux, disrupted swallowing, diarrhea, bowel incontinence/soilage, nausea and vomiting, constipation, belly pain, and gas/bloating/flatulence) at two visits. Patient self-reported and physician-reported assessments of the subjects’ overall GI condition were employed as change anchors. In addition, we prospectively assessed change at both visits using a GI-symptom anchor, the Gastrointestinal Symptom Rating Scale (GSRS). Responsiveness to change was assessed using F-statistics. The minimally changed group was those somewhat better or somewhat worse on the retrospective anchors and changing by one category on the modified GSRS (e.g., from slight to mild discomfort to moderate to moderately severe discomfort).

Results

Responsiveness to change was statistically significant for 6 of 8 PROMIS scales using the self-report GI anchor, 3 of 8 scales using the physician-reported anchor, and 5 of 5 scales using the corresponding GSRS scales as anchors. The MID estimates for scales for improvement and worsening were about 0.5–0.6 SD using the GSRS anchor and generally larger in magnitude than the change for the “about the same” group.

Conclusions

The responsiveness and MID estimates provided here for the PROMIS GI Symptoms scales can aid in scale score interpretation in clinical trials and observational studies.

Introduction

Chronic gastrointestinal (GI) disorders have a high prevalence, are rising in incidence, generate large direct and indirect costs of care, and are associated with work productivity decrements and impairments in other aspects of health-related quality of life (HRQOL) [1–4]. Given the significant burden of GI disorders, it is important to assess patient-reported outcomes (PRO) in clinical care and research [5].

We developed the GI Symptoms scales as part of the National Institutes of Health (NIH)-funded Patient-Reported Outcomes Measurement Information System (PROMIS^®) project. The PROMIS GI Symptoms instrument is a generic measure that is applicable in the general population and different GI disorders [6]. This study assesses the responsiveness to change and estimates minimally important differences (MIDs) for the PROMIS GI scales.

Methods

Data Sources and Measure

Participants

The GI Symptoms scales were developed using the standard PROMIS qualitative and quantitative methodology [6–8]. The items were administered to 865 patients with GI disorders including gastroesophageal reflux disease (GERD), inflammatory bowel disease (IBD), irritable bowel syndrome (IBS), systemic sclerosis (SSc), and other common GI disorders at 4 centers in United States: University of California Los Angeles Medical Center, Cedars-Sinai Medical Center, VA West Los Angeles Medical Center, and University of Michigan Hospital; and to 1177 individuals selected to be representative of the 2010 US general population.

For the present study, we planned to recruit 300 patients from physicians’ offices to yield 80% power (α = 0.05, 2-tailed test) to detect an effect size of 0.16 for within group change. These patients were approached during their routine visits and provided an IRB-approved pamphlet inviting them to participate in the study. To maximize the possibility of detecting change in the GI symptoms, we targeted patients who were given a new treatment intervention or had a change in their GI management (increase or decrease in pharmacologic or non-pharmacologic therapies).

Instruments

The PROMIS GI Symptoms instrument is a 60-item questionnaire that assesses 8 domains: gastroesophageal reflux (13 items), disrupted swallowing (7 items), diarrhea (5 items), bowel incontinence/soilage (4 items), nausea and vomiting (4 items), constipation (9 items), belly pain (6 items), and gas/bloat/flatulence (12 items). There is no single global score to assess GI Symptoms. All items are administered using a 5-point categorical response scale. For each scale, all scales are calibrated using an item response theory graded response model [9] and scored on a T-score metric with a mean of 50 and SD of 10 in the US general population. A higher score denotes more GI symptoms. Subjects without symptoms on a scale are scored at the lowest possible score for that particular scale. For example, for reflux scale, subjects without symptoms received a score of 34 (minimum score for the reflux scale). We also assessed the overall severity of the underlying GI illness at baseline and at the follow-up visit using a single global item [“In the past 7 days, how would you rate your gastrointestinal condition?” (excellent, very good, good, fair, or poor)]. This item was included during the validation of the UCLA GIT 2.0 questionnaire in SSc [10].

The Gastrointestinal Symptom Rating Scale (GSRS) is a 15-item questionnaire that evaluates the five common symptom clusters of Gl disorders: abdominal pain, reflux, indigestion, constipation and diarrhea [7, 11]. Items ask about the past week using a 7-point categorical response scale from no discomfort to very severe discomfort. The self-administered version of the GSRS utilized in this study was modified for use with the general population and shown to have acceptable reliability, validity, and responsiveness to change in patients with different GI disorders [11, 12]. In this study, the response categories were combined to form a 4-point scale as follows: no discomfort, slight to mild discomfort, moderate to moderately severe discomfort, and severe to very severe discomfort.

We also administered the ten PROMIS global health items [13] and the EQ-5D preference-based HRQOL measure [14]. The PROMIS global health items yield a global physical health scale (four items on overall physical health, physical function, pain, and fatigue) and a global mental health scale (four items on quality of life, mental health, satisfaction with social activities, and emotional problems).

MIDs were estimated using an anchor-based approach [15]. An “anchor” is an external indicator of change of clinical relevance used to evaluate change in a PRO measure. We used three different anchors. At the second of two visits, we administered two retrospective recall anchors (one reported by the patient and another by the physician): “Compared to last visit, how is your/your patient’s overall GI condition at this time?” (completely better, considerably better, somewhat better, about the same, somewhat worse, considerably worse, or completely worse) [10]. In addition, we prospectively assessed change in GSRS at two time points (i.e., both clinic visits).

Statistical Analysis

Descriptive statistics are presented as percentages for categorical variables and means (standard deviations) otherwise. Responsiveness to change is estimated using ANOVA F-statistics with the GI scales as the dependent variables and the anchors as the independent variables. MIDs are estimated by examining change in scores of different GI scales (time₂ – time₁) in patients who reported being somewhat better or somewhat worse at time₂ compared to time₁. Similarly, a change of one point in the modified GSRS scales was used as a basis for estimating the MIDs in corresponding PROMIS GI scales.

To assess the usefulness of an anchor, previous research has recommended reporting the correlation between the anchor and the change score; a correlation of at least 0.30 has been suggested as the threshold for an acceptable association between the anchor and the PRO measure [15, 16]. We assessed the associations between the anchors and the change scores for the GI scales using Spearman rank-order correlations. We reported MID estimates only for those anchors that satisfied this threshold of usefulness (i.e., a correlation or at least 0.30).

Results

We recruited 256 patients who completed both baseline and follow-up visits at a median of 88 days (range 4–257 days) apart. The mean (SD) age was 53 (15) years, 55% were male, and 85% had some college education (Table 1). Physician-reported diagnoses included GERD (33%), IBD (24%), IBS (23%), SSc (14%), chronic constipation (13%), and other disorders (39%); some patients had more than 1 GI condition. Patients with GI disorders had baseline PROMIS GI scales scores that were 0.2–0.8 SD (52.0 for reflux scale and 58.0 for belly pain and gas/bloat/flatulence scales; a higher score indicates more symptoms) worse than the US general population (where the mean score is 50; Table 2).

Table 1.

Baseline demographics of subjects

Variable	Subjects (n = 256)
Age, mean (SD)	53 (15)
Female, %	45
Race/ethnicity, %
Hispanic	15
White	55
Black	22
Asian	4
Other	4
Education, %
Some high school or less	2
High school graduate	12
Some college	39
College degree	46
Marital status, %
Married	40
Divorced/widowed/separated	32
Not ever married	38
Physician diagnosed conditions, n (%)
Irritable bowel syndrome	54 (23)
Gastro esophageal reflux disease	81 (33)
Inflammatory bowel disease	57 (24)
Systemic sclerosis	33 (14)
Chronic constipation	30 (13)
Other conditionsa	93 (39)

^aCommon conditions in this category included cirrhosis, gastroparesis, and functional abdominal pain syndrome; more than 1 GI condition was diagnosed for some participants

Table 2.

Descriptive statistics for PROMIS GI scales, EQ-5D, and PROMIS Global health scales at baseline and follow-up

Patient-reported outcomes	Baseline mean scores, mean (SD)	Follow-up mean scores, mean (SD)
PROMIS GI scales
Reflux	52 (10)	51 (10)
Swallowing	53 (11)	52 (10)
Diarrhea	56 (11)	56 (12)
Incontinence	54 (12)	54 (12)
Nausea/vomiting	54 (11)	53 (11)
Constipation	55 (10)	54 (10)
Belly pain	58 (12)	57 (12)
Gas/bloat/flatulence	58 (11)	56 (11)
EQ-5D	0.62 (0.27)	0.62 (0.29)
PROMIS global physical	43 (10)	43 (10)
PROMIS global mental	45 (10)	44 (11)

PROMIS GI scales are calibrated with a mean of 50 and SD of 10 in the US general population

The percentage of patients having the minimum possible score on the PROMIS scales ranged from 0.4% (for reflux and gas/bloat/flatulence scales) to 39% (for fecal incontinence scale) while 2% or less of patients had the maximum possible score on the PROMIS scales (Table 3). Cronbach’s coefficient α was >0.70 for all scales. Self-reported GI severity revealed 12% reporting no symptoms, 26% very mild-to-mild symptoms, 32% moderate symptoms, and 30% severe-to-very severe symptoms.

Table 3.

Minimum and maximum scores and Cronbach’s coefficient a for the PROMIS GI symptoms scales at baseline

PROMIS GI scales	Min scorea	Max scorea	% with min score	% with max score	Cronbach’s α
Reflux	33	82	0.4	0.4	0.83
Swallowing	41	85	28	0.4	0.91
Diarrhea	40	82	1	2	0.89
Incontinence	44	91	39	0.4	0.90
Nausea/vomiting	41	86	18	0.4	0.73
Constipation	37	77	4	0.4	0.88
Belly pain	40	81	1	0.4	0.88
Gas/bloat/flatulence	34	83	0.4	0.4	0.94

^aCalculated in the longitudinal data

Responsiveness to change using the patient retrospective recall as the anchor was statistically significant for 6 of 8 PROMIS GI scales; 3 of 8 PROMIS GI scales were statistically significant using the physician-reported retrospective assessment as the anchor; and 5 of 5 PROMIS GI scales with corresponding GSRS scales were statistically significant (Table 4).

Table 4.

Responsiveness to change for the PROMIS GI scales using three anchors

PROMIS GI scales	Self-reported patient anchor		Physician-reported anchor		GSRS anchor
	F	P level	F	P level	F	P level
Reflux	4.99	0.0007	3.36	0.0034	2.73	0.03
Swallowing	3.83	0.0049	3.16	0.0053	NA	NA
Diarrhea	3.98	0.0038	1.3	0.2577	11.29	<0.001
Incontinence	2.04	0.0892	1.99	0.0678	NA	NA
Nausea/vomiting	2.42	0.0487	2.62	0.0176	NA	NA
Constipation	1.30	0.2724	2.09	0.0547	25.2	<0.001
Belly pain	3.94	0.0041	0.291	0.4855	12.58	<0.001
Gas/bloat/flatulence	2.86	0.0042	1.31	0.2538	29.56	<0.001

Rank-order correlations between retrospective patient and physician anchors was 0.61 and between retrospective reports of change in GI symptoms versus prospective change in the PROMIS GI scales ranged from 0.11 for bowel incontinence to 0.25 for belly pain (patient anchor) and from 0.02 for bowel incontinence to 0.17 for reflux (physician anchor) (Table 5). Change in GSRS scales were more strongly related to change in the PROMIS GI scales and exceeded the stated threshold of >0.30 for a useful anchor, with rank-order correlations ranging from 0.40 to 0.52. Therefore, for calculation of MID estimates, we used only the GSRS scales.

Table 5.

Spearman correlations of change in the PROMIS GI scores

PROMIS GI scales	Patient-reported change anchor	Physician-reported change anchor	GSRS change anchor
Reflux	0.18*	0.17**	0.40¶ (reflux scale)
Swallowing	0.13	0.14*	NA
Diarrhea	0.16*	0.03	0.57¶ (diarrhea scale)
Incontinence	0.11	0.02	NA
Nausea/vomiting	0.14*	0.10	NA
Constipation	0.11*	0.15*	0.54¶ (constipation scale)
Belly pain	0.25¶	0.09	0.48¶ (abdominal pain scale)
Gas/bloat/flatulence	0.15*	0.12	0.51¶ (indigestion scale)

GSRS anchors are reported for corresponding PROMIS scales

^*P < 0.05;

^**P < 0.01;

^¶P < 0.001

Most patients reported being somewhat better, about the same, or somewhat worse (Table 6). Based on the change in GSRS scales anchors, MID estimates for improvement ranged from −5 to −6 (0.5–0.6 SD) and 1–6 (0.1–0.6 SD) for worsening and were generally larger than change for the about the same group (Table 6).

Table 6.

Minimally important differences for the PROMIS GI scales using GSRS anchor

PROMIS GI scales	Mean (N)
	Much better	Somewhat better	About the same	Somewhat worse	Much worse
Reflux	−7 (29)	−5 (29)	0 (142)	1 (32)	5 (19)
Swallowing	NA	NA	NA	NA	NA
Diarrhea	−8 (31)	−5 (30)	1 (131)	6 (42)	10 (16)
Incontinence	NA	NA	NA	NA	NA
Nausea/vomiting	NA	NA	NA	NA	NA
Constipation	−11 (17)	−5 (28)	0 (165)	6 (30)	7 (10)
Belly pain	−13 (7)	−6 (36)	0 (177)	6 (26)	9 (5)
Gas/bloat/flatulence	−17 (9)	−6 (42)	−1 (168)	6 (27)	10 (5)

PROMIS GI Reflux scale versus GSRS Reflux scale; PROMIS GI Diarrhea scale versus GSRS Diarrhea scale; PROMIS GI Constipation scale versus GSRS Constipation scale; PROMIS GI Belly pain scale versus GSRS Abdominal pain scale; and PROMIS GI Gas/bloat/flatulence scale versus GSRS Indigestion scale

GSRS Gastrointestinal Symptom Rating Scale; Negative score denotes improvement

Discussion

The ability of HRQOL instruments to detect clinically important changes is crucial to their usefulness in evaluating the effectiveness of different therapies [17]. PROMIS instruments have been found to have as good or better precision than existing measures studies [7, 18].

We evaluated responsiveness to change and estimated MIDs for the eight PROMIS GI Symptoms scales in a longitudinal observational cohort. Six of the eight scales were responsive to change (except for bowel incontinence and constipation scales) using a self-reported GI anchor and 5 of 5 scales using the corresponding GSRS scales as an anchor. The lack of responsiveness of the bowel incontinence scale may be at least in part due to the high proportion of the sample with a minimum score at baseline and a relatively low proportion of patients who were actually treated for this underlying disorder.

MID estimates help researchers and clinicians understand whether PRO-score differences are large enough to matter (i.e., whether differences are meaningful either between two treatment groups or within one group over time) [10, 15, 19]. For example, an average change of two units may be statistically significant in a study due to large sample size, but it may not be perceived as beneficial by the subjects. In addition, since MID estimates may differ for worsening versus improvement groups [10, 20], we present MID estimates for improvement and worsening. MID estimates ranged from 0.5 to 0.6 SD (or 5–6 units), except for worsening for the Reflux scale (0.1 SD or 1 unit). In a clinical study, an improvement of ≥5 units in the diarrhea scale (MID for improvement) within one group over time or a difference of ≥5 units between two groups should be considered as clinically important improvement. MID estimates are not applicable at an individual level, and there are different statistical tests to determine if a change within an individual is beyond measurement error [21]. Also, the MID estimates were larger than the change observed for the “about the same” group. In another study in patients with cancer where 6 different PROMIS instruments were administered at two different time points, the MID estimates for improvement were similar and ranged from 0.25 to 0.60 SD [19]. The MID estimates need to be interpreted individually for each scale rather than average for the 8 scales.

The correlation coefficients for global anchors versus change on the PROMIS GI scales were <0.30 whereas coefficients for change on GSRS scales versus change on the PROMIS GI scales were >0.30 (Table 5). As there is inherent uncertainty in interpretation of MID estimates [15], we a priori included three anchors to estimate MID (as done by previous researchers [10, 19]). However, we could only use GSRS scales as the anchor and have provided MID estimates for 5 of 8 GI Symptoms scales where we had the corresponding GSRS scales.

Our study has several strengths. Our MID estimates are based on a large sample size of patients with different GI disorders seen in clinical and academic centers. Second, we prospectively incorporated anchors in order to estimate the MIDs, and our sample successfully recruited patients with self-reported severity that was uniformly distributed from very mild to very severe.

Our study also has limitations. First, we were unable to use objective tests (such as endoscopy or manometry) as change anchors. In light of the breadth of GI conditions included in our study, we could not identify a standardized test that was applicable across all the conditions. Future studies should corroborate our estimates using objective measures within defined GI populations. Second, as previously reported, most patients considered themselves about the same between the two time points [10]. This is despite the fact that we enriched our patients with those who had a change in GI management and reveals challenges in assessing MID estimates in observational cohorts. Due to a majority of patients that considered themselves about the same, we were unable to assess MID estimates by different GI disorders since the number of patients in each subgroup was small. Third, as discussed above, correlation coefficients between patient and physician global anchors versus change on PROMIS GI scales were less than 0.30, a cut off to evaluate the strength of an anchor. This study highlights the difficulty in choosing anchors a priori and careful considerations should be given in future studies. Lastly, we were only able to provide MID estimates for 5 of 8 GI Symptoms scales.

In conclusion, we provide MID estimates for the PROMIS GI Symptoms scales. This information can aid in interpreting scale scores in future trials and observational studies. These data should be considered preliminary and confirmed with larger cohorts and/or clinical trials.