Reliability and Validity of Pain and Urinary Symptom Severity Assessment in Urologic Chronic Pelvic Pain; A MAPP Network Analysis

Abstract

OBJECTIVE:

To assess reliability and validity of an efficient severity assessment for pelvic pain and urinary symptoms in urologic chronic pelvic pain syndrome (UCPPS), which consists of interstitial cystitis/bladder pain syndrome (IC/BPS) and chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS).

METHODS:

578 patients were assessed using brief, empirically derived self-report scales for Pelvic Pain Severity (PPS) and Urinary Symptoms Severity (USS) four times during a one-month period and baseline clinic visit that included urologic, pain and illness-impact measures. Mild, moderate and severe categories on each dimension were examined for measurement stability and construct validity.

RESULTS:

PPS and USS severity categories had adequate reliability and both discriminant validity (differential relationships with specific clinical and self-report measures) and convergent validity (common association with non-urological somatic symptoms). For example, increasing PPS was associated with pelvic tenderness and widespread pelvic pain, whereas USS was associated with urgency during a bladder filling test and increased sensory sensitivity. PPS and USS categories were independently associated with non-urological pain and emotional distress. A descriptive analysis identified higher likelihood characteristics associated with having moderate to severe PPS or USS or both. Lack of sex interactions indicated that the measures are comparable in IC/BPS and CP/CPPS.

CONCLUSIONS:

Women and men with UCPPS can be reliably subgrouped using brief self-report measures of mild, moderate or severe pelvic pain and urinary symptoms. Comparisons with a broad range of clinical variables demonstrate the validity and potential clinical utility of these classifications, including use in clinical trials, health services and biological research.

Introduction

Interstitial cystitis/bladder pain syndrome (IC/BPS) and chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS) are collectively referred to as urologic chronic pelvic pain syndrome (UCPPS)¹. The Multi-Disciplinary Approach to the Study of Chronic Pelvic Pain (MAPP) Research Network (see Appendix A) established by the National Institute of Diabetes and Digestive and Kidney Diseases is a collaborative, multidisciplinary effort to identify key phenotypic and etiologic factors of these costly and difficult-to-treat disorders (https://www.mappnetwork.org).¹ UCPPS can present with a broad range of symptoms including pelvic pain that may or may not be related to urination, increased urinary urgency and frequency, and comorbid symptoms of non-pelvic pain, sleep disruption and negative mood. Although a variety of self-report scales have been developed, a lack of agreement on how to best organize and quantify UCPPS symptoms presents significant problems for both clinical management and design of randomized controlled trials. A recent MAPP study utilized items from the Genitourinary Pain Index (GUPI) and Interstitial Cystitis Symptom Index (ICSI)² and empirically identified two components of primary UCPPS symptoms. One, labeled Pelvic Pain Severity (PPS), included items related to pelvic pain, the other, Urinary Symptom Severity (USS), included items related to urinary frequency. Although moderately correlated, it was hypothesized that differential severity on these dimensions may prove useful to identify important UCPPS sub-groups or phenotypes for mechanistic research and clinical management. The PSS and USS measures have been successfully used in several MAPP studies to quantify clinical outcomes, examine predictors of symptom change and assess relationships between UCPPS symptoms and physiological outcomes.^3–7

Although empirically derived, this two-dimensional UCPPS symptom organization has not been adequately examined for construct validity. Construct validity refers to a process whereby a measure is evaluated by examination of associations with hypothesized related and unrelated characteristics with the goal to demonstrate that the measurement tool behaves (or does not behave) in accord with the conceptual understanding of the underlying construct. This process is a critical step in development of empirically derived measures for which there is no specific gold standard to assess validity. It also enhances the utility of the measurement by pointing to how well the measured construct relates to markers of underlying mechanisms and established phenotypes. Specifically, for the USS and PPS measures, it is important to demonstrate their differential associations with other key UCPPS symptoms, and to document that they provide a parsimonious approach to quantify key severity variables for individual patients.

Many syndromes have “mild”, “moderate”, and “severe” specifiers that serve to efficiently communicate the degree of severity. The current analysis tests the construct validity of the PPS and USS assessment scheme and the reliability and utility of using mild, moderate and severe categories. Key hypotheses include: 1) Tertile subgrouping of PPS (PPS-gp) and USS (USS-gp) are stable (reliable) over one month, 2) Tertile based subgroups of PSS and USS show discriminant validity with relevant clinical variables; i.e., USS-gp, but not PPS-gp are associated with other self-report measures of urinary frequency and the opposite for association of PPS-gp with measures of pelvic pain severity, 3) Both PPS-gp and USS-gp show independent associations with measures of illness impact such as quality of life (QOL) and negative affect, indicating their individual contributions to overall UCPPS severity, 4) PPS-gp and USS-gp show differential associations with relevant functional assessments such as sensitivity during a pelvic exam and response to bladder filling suggestive of differential underlying pathophysiology. To facilitate clinical use of the PPS-gp and USS-gp measures, data will also be presented on characteristics for participants with mild or moderate/severe symptoms across the dimensions.

Materials and Methods

The MAPP Research Network, multisite, longitudinal Trans-MAPP Symptom Patterns Study (SPS), enrolled 620 men and women with UCPPS and included comprehensive cross-modality assessments of clinical, biological, brain imaging and pain sensitivity measure with follow up to 36 months.⁸ The current analysis focuses on variables collected during a one month run-in period and the baseline assessment of the SPS.

Participants:

Recruitment, and inclusion and exclusion criteria for SPS have been published.⁸ Participants had a diagnosis of IC/BPS or CP/CPPS, pain severity of at least 1 on a 0–10 numerical rating scale (NRS), were 18 years or older, and had UCPPS symptoms present the majority of the time during three of the previous six months. The sample was largely White and from urban or suburban areas. Recruitment was from both clinics and community advertisement, and income levels were diverse. Current treatments varied widely (see Appendix D, Table 5).

Design:

Following informed consent and enrollment, participants completed the GUPI and ICSI weekly via the internet during a four-week run-in period. Measures for the main analyses were completed during a baseline visit. At the same visit (or in a few cases within 2 weeks) participants underwent pelvic examinations to assess the numbers of levator ani muscle groups that were tender on palpation, and a MRI brain imaging procedure that included a standardized bladder filling protocol.⁹

Measures:

The primary independent variables of urinary symptoms and pelvic pain were characterized on two distinct dimensions as previously described. ² PPS varied from 0 to 28 and was the sum of the pain subscore of the Genitourinary Pain Index (GUPI)¹⁰ and Item 4 of the Interstitial Cystitis Symptom Index (ICSI)¹¹. USS varied from 0 to 25 and was the sum of the GUPI urinary subscore and ICSI Items 1–3. The items used and scoring are included in Appendix B. To allow for easier identification of mild, moderate, and severe symptom levels, cutoffs were set to yield roughly equal tertiles of participants (Figure 1). Classifying continuous data into similar sample size groups avoids arbitrary cutpoints and is not a statistical approach to subgroup, but rather a clinical one to help patients and clinicians identify relative severity outcomes. Similar to the original data, there was a correlation between the ordinal scoring (0,1,2) of the two severity measures (r=.53).²

Figure 1:

Scatterplot of PPS and USS indices. Size of dots indicates number of individuals at each point. Dotted lines indicate demarcation of tertile based subgroups and colored rectangles correspond to the moderate/severe and mild grouping shown in Figure 4.

Dependent variables are shown in Table 1 and have been previously described.^{12 6, 8} Participants were asked to identify their single most bothersome symptom over the past two weeks (Most Bothersome Symptom); classified into either urinary frequency or pelvic pain. Single item, 0–10 NRS’s with higher scores indicating worse outcomes were used to assess overall pelvic region pain or discomfort severity (Pelvic Pain Discomfort NRS), non-pelvic pain severity (Non-Pelvic Pain NRS), and mood (Mood NRS), over the past two weeks. The RICE criteria (0–2) was scored as no symptoms, painful bladder filling, painful urgency, or both.¹³ Neuropathic type pelvic pain was determined from the total score of the Pain Detect scale referenced to pelvic pain.¹⁴ Presence and distribution of systemic pain, genital pain, and pelvic floor tenderness were assessed using a CHOIR whole-body map, ¹⁵ gender-specific genital map, and standardized pelvic floor examination. Urge on Bladder Filling was the difference in urge ratings from empty to partially filled bladder following consuming 350 cc of water.⁹ Current Flare from, ‘Are you currently experiencing a flare of your urologic or pelvic pain symptoms?’ (Y/N). Nociplastic, or ‘centralized,’ pain as total score from the Fibromyalgia Severity Scale which includes widespread pain, cognitive and somatic symptoms.^{16, 17} Somatic Awareness and Sensory Sensitivity are subscales of the Complex Medical Symptom Index assessing unpleasant bodily sensations and sensitivity to exteroceptive (e.g., sound, light) stimuli respectively.¹⁸ Physical and mental well-being were assessed using the SF-12,¹⁹ Sleep disturbance and fatigue using the PROMIS short form scales,²⁰ and anxiety and depression using the Hospital Anxiety and Depression scales.²¹ Current medication usage (Medication Type) was classified hierarchically as none, peripheral medications only, some centrally-acting (e.g., antidepressants), or opiates.

Table 1:

Variable Descriptive Statistics

Variable	Variable Label	Mean	Median	Std Dev	N Miss	N
	Pelvic and Urinary Pain Severity
Pelvic pain severity (PPS)		14.27	14.61	5.64	0	578
Urinary symptom severity (USS)		11.62	11.00	6.22	0	578
	Bladder and Urologic Variables
Most bothersome symptom	SYMQ Q8 Most Bothersome symptom (%pain)	392 (71%)			33	545
Currently Experiencing Flare	SYMQ Q12 Experiencing a Flare now (0–1)?	114 (20%)			11	567
Pelvic Pain/Discomfort NRS	SYMQ Q5 Urologic/Pain Symptom Severity (0–10)	4.67	5.00	2.37	9	569
RICE criteria	None, pain during urination or filling, both (0–2)	1.34	2.00	0.74	9	569
Neuropathic type pelvic pain	Pain Detect Score (0–38)	9.29	8.00	6.56	9	569
Widespread pelvic pain	Genital Body Map Index4 (0–4)	1.40	1.00	1.14	10	568
Pelvic floor tenderness	Pelvic Floor Tenderness Index6 (0–6)	3.52	4.00	2.06	25	553
Urge on bladder filling	Urge with 350 cc water ingestion (0–10)	2.46	2.00	2.33	100	478
	Common Comorbid Conditions
Non-pelvic pain NRS	SYMQ Q6 Non-Uro/Pelvic Pain Symptoms (0–10)	3.46	3.00	2.63	11	567
Mood NRS	SYMQ Q7 Mood positive to negative (0–10)	3.97	4.00	2.47	11	567
Widespread body pain	Widespreadness of Pain Index12 (0–12)	2.39	2.00	2.49	19	559
Nociplastic pain	Fibromyalgia Score (0–31)	8.31	7.00	5.27	19	559
Somatic awareness	CMSI Somatic Awareness (SA) Sum (0–18)	4.89	4.00	3.58	9	569
Sensory sensitivity	CMSI Sensory Sensitivity (SS) Sum (0–4)	1.15	0.00	1.47	9	569
	QoL and Illness Impact
Physical well-being	SF-12 QoL Physical (0–100)	45.55	46.50	10.02	13	565
Mental well-being	SF-12 QoL Mental (0–100)	43.32	43.98	10.73	14	564
Sleep disturbance	PROMIS Sleep T-Score (0–100)	54.70	54.30	9.48	17	561
Fatigue	PROMIS Fatigue T-Score (0–100)	55.04	55.10	9.90	18	560
Depression	HADS Depression Score (0–21)	5.76	5.00	4.53	19	559
Anxiety	HADS Anxiety Score (0–21)	7.34	7.00	4.78	21	557
Medication Type	Currently taking medications for any pain?	1.88	2.00	0.89	9	569

Statistical Analyses

Category-specific Intraclass Correlation Coefficients (ICC) were estimated within a one-way components of variance model for categorical data, in which participant is incorporated as a random effect. An ICC for repeated measures of the same outcome above 0.6 is considered an indicator of substantial reliability, and conversely, an indicator of low within-participant variability.²²

Generalized linear models were used to assess construct validity of the PPS and USS severity subgroups across the outcome measures. Within each outcome model, PPS-gp and USS-gp were included as ordinal predictors (coded 0,1,2) to test for a linear trend between the categories and the outcome. These effects were considered significant at p<0.01 (Appendix C for details). Both PPS-gp and USS-gp effects were adjusted for age, and sex by diagnosis which were included as main effects. To aid in comparing the magnitude of PPS-gp and USS-gp across outcomes, analyses performed on standardized outcomes are displayed in forest plots of effect size estimates and 99% Confidence Intervals (CIs). Unstandardized models are tabled in Appendix D. Initial models including PPS-gp and USS-gp interactions with age and sex by diagnosis were examined but these did not show significant interaction effects so the terms were dropped (Appendix D, Table 2).

To illustrate the characteristics of individuals who are higher or lower on the two severity dimensions, GLM analyses comparing those in the moderate/severe PPS-gp, the USS-gp or both, to those with mild PPS and USS were performed, again adjusting for age and sex by diagnosis. Similarly, as in the models testing for linear trend between the tertile subgroups, standardized models were plotted and unstandardized tabled.

RESULTS

Stability of PPS and USS:

ICC values for the tertile subgrouping (PPS-gp = 0.65 (99% CI .61-.70); USS-gp = 0.76 (99% CI .72-.79)) demonstrated substantial reliability for how participants were classified into subgoups across the four-week run-in period, although they were, as expected, lower than those for the continuous measurements (PPS = 0.76 (99% CI .72-.79), USS = 0.86 (99% CI .84-.88)).

Construct Validity:

Figures 2a, 2b, and 2c show the relationship between the PPS and USS subgroups and each of the outcome measures as well the main effects for age and sex by diagnosis. Standardized values for point estimates and 99% CIs are shown to facilitate by-group interpretation (see Appendix D for unstandarized tables). Both PPS-gp and USS-gp were highly and similarly related to reporting pain or urinary symptoms, respectively, as their most bothersome symptom, and showed independent relationships with the likelihood of meeting the RICE criteria for painful bladder filling and/or painful urgency. Both the PPS-gp and USS-gp were associated with measures of neuropathic pelvic pain, non-pelvic pain NRS and nociplastic pain. Additionally both PPS and USS tertiles were associated with quality of life and illness impact measures, such as physical well being, sleep disturbance, fatigue and depression. Some measures were only associated with the PPS tertiles and not the USS tertiles, including widespread pelvic pain, pelvic floor tenderness, mood, mental well being, anxiety, more centralized medication use, and being in a flare. Conversely, measures only associated with USS tertiles were urge on bladder filling, somatic awareness and sensory sensitivity. Sleep disturbance was related to both dimensions but somewhat more with USS.

Figure 2a-c:

Effect sizes for a one-category increase in PPS-gp and USS-gp for the standardized clinical outcomes. PPS-gp and USS-gp effects were adjusted for sex and diagnosis, which was included as a main effect in the model (and shown in the left panel), and for the other group effect. The effect estimates and corresponding 99% confidence intervals are displayed. Effect sizes for which the 99% CI does not cross over the dotted vertical X=0 line are statistically significant at the Type I error rate of .01.

Characteristics of Individuals with Moderate or Severe Symptoms:

Individuals with moderate/severe PPS, USS, or both, were compared to participants with mild USS and PPS on selected clinical outcomes (Figures 3a–c), and summarized in Figure 4 which illustrates those characteristics most associated with moderate or greater severity on each of the dimensions or both. Participants with moderate/severe PPS have greater widespread genital/pelvic pain and neuropathic type pain and likelihood to be in a flare. Moderate/severe USS is associated with urgency on bladder filling, poorer physical well being, sleep disturbance and sensory sensitivity. General psychological issues, including poorer mental well being and mood, depression and anxiety are more likely when moderate/severe on both PPS and USS. Similarly, greater pelvic floor tenderness, non-pelvic pain severity, nociplastic pain, and fatigue tend to occur with both moderate/severe PPS and USS.

Figure 3a-c:

Effect sizes for the difference in standardized outcomes between the moderate/severe PPS, USS and Both subgroups and the mild subgroup. Group effects were adjusted for sex and diagnosis, which was included as a main effect in the model (and shown in the left panel), and for the other group effects. The effect estimates and corresponding 99% confidence intervals are displayed. Effect sizes for which the 99% CI does not cross over the dotted vertical X=0 line are statistically significant at the Type I error rate of .01.

Figure 4:

Summary of clinical measures elevated in those with moderate/severe PPS and/or USS symptoms. Measures placed in one of the three moderate/severe quadrants are those in which that subgroup’s values were significantly different from the mild symptom group as shown in Figure 3a–c. In the cases where a characteristic was significantly different from the mild group for two subgroups (e.g., USS mod/severe and both mod/severe), and the size of the associations was similar based on overlap of confidence intervals, it was placed in the more specific category (e.g. USS mod/severe). If the Both mod/severe subgroup was substantially higher than the other significant subgroup it was placed on the dividing line between categories.

DISCUSSION

The wide range of UCPPS symptoms complicates assessment of illness severity for both the researcher and clinician. This study provides new evidence demonstrating the reliability and validity of a two-dimensional approach to severity assessment based on a small number of items from two commonly used self-report instruments. Measures of PPS and USS have been used as outcomes in several studies following the initial paper describing their empirical derivation, but limited data on their psychometric properties have been available. Our analysis shows that women and men with UCPPS can be reliably subgrouped based on mild, moderate or severe symptoms of pelvic pain and/or urinary symptoms. This approach is further supported by our comparisons with a broad range of clinical variables demonstrating the validity and potential utility of these classifications.

Although continuous measures, such as total scale scores, are appealing as outcome variables, it is often useful to identify individuals within severity subgroups for participant selection stratification and sample description in clinical trials, and describing treatment response for personalized medicine. A measurement system should provide stable results over repeated testing when symptoms remain stable (test-retest reliability). The tertile subgroups for PPS and USS showed adequate stability over four weekly assessments, especially given the probability of some real symptom changes over this time-period. Further supporting the usefulness of this severity assessment approach, the multivariable data showed robust and differential relationships between the PPS-gp and USS-gp classifications and relevant clinical variables. As hypothesized, PPS-gp was most associated with both self-report and exam-based measures of pelvic pain, including pain as the most bothersome symptom, a pain/discomfort severity scale, greater number of pelvic/genital pain locations, and pelvic tenderness on exam. Similarly, USS-gp was associated with urinary frequency as the most bothersome symptom, and ratings of urge following bladder filling. Interestingly, pelvic pain seems in some ways a predominant symptom based on about 70% reporting it as their most bothersome symptom and its close association with flare status. Differentiation of the two domains was also evident in other clinical variables; PPS-gp was associated with negative mood, overall poorer mental well-being and anxiety symptoms and USS-gp more associated with sensory sensitivity, somatic awareness, and poorer overall physical well-being. The PPS-gp findings regarding impact of pain on psychological and mood variables, especially anxiety, are consistent with data from other chronic pain conditions.^{23, 24} It might also be expected that USS severity would be related to sleep disturbance based on interruptions from nighttime frequency and urge, but the finding of USS-gp being most associated with increased sensory sensitivity and decreased physical well-being suggest a broader sensory-alteration phenotype may be underlying increased urinary frequency and urgency. PPS and USS both showed significant and independent associations with severity of non-pelvic pain, fatigue and depression but not number of non-pelvic pain locations. Increased use of centrally acting medications was found in those with moderate/severe PPS and USS. Overall, these results indicate the two UCPPS severity dimensions show discriminant validity, in that they are robust markers for relevant pain and urinary symptoms, and convergent validity in their common association with greater non-urological symptomology. Importantly, although women show greater severity on some measures, the reported relationships were largely independent of sex and diagnosis. This finding is important because it supports the generality of the findings and utility of the measures for assessing severity across the wide age range of UCPPS as well as men and women with IC/BPS and men with CP/CPPS. Limitations for our approach include the use of retrospective reports rather than the more burdensome voiding diaries, although self-reports do show adequate correspondence with diaries.²⁵ In addition, the simple tertile subgrouping based on the current sample used here requires cross-validation in future studies to confirm its clinical utility.

The descriptive analysis in Figure 4 highlights characteristics that are more likely in patients that show moderate to severe PPS or USS or both. This may be useful information for clinicians by illuminating potential comorbid problems for assessment and treatment. These data together with the primary results suggest the PPS-gp and USS-gp assessment can reliably classify individuals with UCPPS into clinical groups, which may in future help to predict differential response to various treatments (i.e., severity-based prognostication). This is important for clinical trial design and subgroup analysis, health services research, and studies aiming to identify the pathophysiology underlying the wide range of UCPPS symptoms. Further research using this two dimensional model can be directed at associating how physiologically identified phenotypes correspond to these two symptom-based domains and exploring the hypothesis of altered general sensory processing associated with USS.