The International Prostate Symptom Score (IPSS) Is an Inadequate Tool to Screen for Urethral Stricture Recurrence After Anterior Urethroplasty

Abstract

OBJECTIVE

To validate the use of the International Prostate Symptom Score (IPSS) as a stand-alone tool to detect urethral stricture recurrence following urethroplasty.

MATERIALS AND METHODS

This study included 393 men who had undergone anterior urethroplasty and were enrolled in a multi-institutional outcomes study. Data analyzed included pre- and post-operative answers to the IPSS in addition to findings from a same- day cystoscopy. IPSS from men found to have cystoscopic recurrence were then compared to scores from those with successful repairs, and receiver operating characteristic curves were plotted to illustrate the predictive ability of these questions to screen for cystoscopic recurrence.

RESULTS

Mean postoperative scores were lower (fewer symptoms) in successful repairs; IPSS improved from preoperative values regardless of recurrence. Successful repairs had significantly better degree of improvement in question #5 (assessing weak stream) compared to recurrences. Receiver operating characteristic curves demonstrated the highest area under the curve for the IPSS quality of life question (0.66) that alone outperformed the complete IPSS questionnaire (0.56).

CONCLUSION

The IPSS had inadequate sensitivity and specificity to be used as a stand-alone screening tool for stricture recurrence in this large cohort of men, highlighting the need to continue development of a disease-specific, validated patient-reported outcome measure.

Urethral strictures may lead to significant loss of quality of life (QOL).¹ Urethroplasty is widely considered the gold standard for treatment, with men achieving a long-term success rate of 85%–95%.² Surveillance for stricture recurrence after urethroplasty includes regular follow-up with clinical history, questionnaires, or uroflowmetry; however, confirmation of a patent urethra typically requires invasive modalities such as cystoscopy or retrograde urethrography. Currently there is no standardized postoperative screening approach.³

Patient-reported outcome measures (PROMs) are increasingly being utilized along with traditional invasive measures to both diagnose urethral strictures and to monitor patients with strictures after management. As urethral strictures traditionally presents with obstructive lower urinary tract symptoms (LUTS), it has been postulated that PROMs may be effective in screening for recurrence after urethroplasty.⁴ PROMs, when used alone as a screening tool, have not been rigorously validated vs an invasive measure such as cystoscopy. According to a systematic review of follow-up protocols, the International Prostate Symptom Score (IPSS) is the most common PROM used to monitor for recurrence and has been shown in studies to be useful for urethral strictures.^4,5

The purpose of this study is to validate the use of the IPSS as stand-alone screening tool for stricture recurrence following anterior urethroplasty. Given the widespread and established usage of the IPSS in assessing stricture outcomes, we are interested in how it performs relative to newer PROMs. We hypothesize that when compared to the gold-standard cystoscopy, utilization of both total scores and scores from individual questions that characterize obstructive symptoms on the IPSS will have adequate sensitivity and specificity to be used as a standalone screening tool, thus minimizing the need for routine invasive testing.

MATERIALS AND METHODS

Subjects

From 2009 to 2014, men from 7 institutions involved in the Trauma and Urologic Reconstruction Network of Surgeons (TURNS) were enrolled in an Institutional Review Board-approved (site specific) study that prospectively collected urethroplasty outcomes data. The specifics of the study have been described elsewhere.⁶ For this study, the Filemaker database created for the TURNS project was retrospectively queried for men who underwent routine postoperative cystoscopy to assess the integrity of the repair (generally at 3–6 and then 12 months) and a same-day IPSS questionnaire.

Urethroplasty Failure Definition

Recurrence of urethral stricture was defined as the inability to advance a 17 French cystoscope past the previously reconstructed portion of the urethral lumen. In this study, we were unable to distinguish stricture recurrence at the site of repair from “new” urethral strictures at a differing location. Recurrent LUTS or need for subsequent intervention after urethroplasty was not considered stricture recurrence for the purposes of this study.

PROMs

All patients included in this study completed a standard IPSS questionnaire during follow-up visit per TURNS protocol. The majority of these patients also completed an IPSS questionnaire preoperatively as part of routine workup. Of interest for the study were the total IPSS and the scores from individual voiding questions (incomplete emptying, frequency, intermittency, urgency, weak stream, straining, nocturia), including the QOL question, obtained at the time of cystoscopy. In addition, for men with both pre- and postoperative IPSS, we assessed the change (ΔIPSS) in total and individual IPSS questions.

Statistical Analysis

We used descriptive statistics to characterize demographic information including patient age, follow-up time, stricture length, stricture location, and repair type. Univariate analysis was used to compare IPSS data of men with successful repairs to those with cystoscopic recurrences. Receiver operating characteristic analyses were performed to determine which IPSS data were best at predicting cystoscopic recurrence by utilization of the area under the curve (AUC) values. Sensitivity, specificity, positive predictive value, and negative predictive value were then calculated for the IPSS data points, with the highest AUCs using various thresholds that are commonly used in clinical practice and cited in historical research studies.⁷ All statistics were computed with SAS 9.3 (Cary, NC) with statistical significance set at P < .05.

RESULTS

Demographics

There were 393 men in the TURNS database who met study criteria. The main reason for ineligibility was incomplete same-day cystoscopy or IPSS data (n = 436). uMean age of patients was 45.1 ± 15.7 years with a mean cystoscopy follow-up time of 11.14 ± 12.80 months. Cystoscopic recurrence was detected in 54 patients (13.7%).

Mean intraoperative stricture length was 3.21 ± 2.44 cm; men who went on to have recurrences had longer strictures than those successfully repaired (4.24 cm vs 3.04 cm, P = .0138). Most common stricture location was the bulbar urethra (n = 269) followed by the penile urethra (n = 69). Excision and primary anastomosis was the most common repair type (n = 166) followed by varying substitution techniques with buccal mucosal grafting (n = 163).

Preoperative

Preoperative IPSS data were available in 263 patients (66.9%). Mean preoperative IPSS total score was 18.30 ± 8.50 with a QOL score of 4.1 ± 1.5. The highest mean individual question score was question #5 (weak stream) at 3.8 ± 1.7 followed by question #2 (frequency) at 2.8 ± 1.6. There was no significant difference in preoperative IPSS individual question, QOL, or total scores between the successful repair and recurrence groups.

Postoperative

Mean postoperative IPSS individual question scores for incomplete emptying, intermittency, weak stream, straining, and nocturia were significantly higher in the recurrence group compared to those with successful repair (Table 1). Men with recurrences reported worse QOL (1.9 vs 1.2, P = .0009) and a higher total IPSS score (8.9 vs 5.4, P < .0031).

Table 1.

Comparison of postoperative and change (Δ) in IPSS scores between the successful repair and cystoscopic recurrence group

	Postoperative			Change (Δ) From Preoperative
	Success Mean	Recurrence Mean	P Value	Success Mean	Recurrence Mean	P Value
Question 1 – Incomplete Emptying	0.65	1.12	.0301	−1.90	−1.61	.4481
Question 2 – Frequency	1.15	1.44	.1137	−1.72	−1.46	.4618
Question 3 – Intermittency	0.55	1.08	.0182	−1.72	−1.44	.4760
Question 4 – Urgency	0.78	1.07	.1201	−1.41	−1.71	.4317
Question 5 – Weak Stream	0.63	1.44	.0016	−3.15	−2.21	.0205
Question 6 – Straining	0.40	1.07	.0023	−2.34	−1.79	.1593
Question 7 – Nocturia	1.18	1.65	.0174	−1.00	−1.04	.9312
Quality of Life	1.17	1.93	.0009	−2.90	−2.09	.0769
Total IPSS Score	5.35	8.87	.0031	−13.20	−11.29	.2944

Bold data represent statistically significant values (P < .05).

IPSS improved (decreased) following surgery for all individual questions regardless of cystoscopic recurrence. Significant difference in degree of improvement was seen in question #5 (weak stream), where successful repairs had a mean decrease of 3.2 compared to a decrease of 2.2 in the recurrence group (P = .0205). The degree of improvement was not significantly different for the other questions or total IPSS score.

Receiver operating characteristic curves predicting cystoscopic recurrence demonstrated that the QOL question had the highest AUC at 0.66 followed by question #5 (weak stream) with an AUC 0.60. Total IPSS score had an AUC of 0.56 (Fig. 1).

Figure 1.

Receiver operating characteristic curve of International Prostate Symptom Score questions and total score as predictors of cystoscopic recurrence. (Color version available online.)

Sensitivity Analysis

The reporting of any degree of “weak stream” (question #5 score ≥ 1) had a sensitivity of 52 and a specificity of 66 in detecting cystoscopic recurrence (Table 2). A QOL score of ≥1 had a sensitivity of 78 and a specificity of 41. Presence of any LUTS (IPSS total score ≥ 1) had a sensitivity of 96 with a specificity of 12 in detection of recurrence. Negative predictive values for question #5, QOL, and total score ranged from 87% to 95%.

Table 2.

Sensitivity, specificity, PPV, and NPV of question #5 (weak stream) vs total IPSS score at various thresholds

Question #5 (Weak Stream)	Sensitivity	Specificity	PPV	NPV
≥ 1	52	66	19	90
≥ 2	39	86	31	90
≥ 3	28	92	36	89
≥ 4	17	95	36	88
≥ 5	9	98	42	87

Quality of Life	Sensitivity	Specificity	PPV	NPV
≥ 1	78	41	17	93
≥ 2	52	71	21	91
≥ 3	33	84	23	89
≥ 4	20	91	25	88
≥ 5	9	96	25	88
≥ 6	2	99	20	87

IPSS Total Score	Sensitivity	Specificity	PPV	NPV
≥ 1	96	12	15	95
≥ 2	80	25	15	89
≥ 3	73	40	16	91
≥ 4	67	48	17	90
≥ 5	65	58	20	91
≥ 6	61	64	20	92
≥ 8	56	75	25	92
≥ 11	38	84	26	90
≥ 16	23	93	33	89
≥ 20	15	96	39	88
≥ 25	4	98	29	87
≥ 30	2	99	33	87

IPSS, International Prostate Symptom Score; NPV, negative predictive value; PPV, positive predictive value.

DISCUSSION

In this study, we have shown that the IPSS as a standalone method to detect urethral stricture recurrence following urethroplasty has limited sensitivity and specificity relative to the cystoscopic gold standard. For example, using the established IPSS LUTS severity criteria of >7 (moderate) and >19 (severe), only 56% and 15% sensitivity was achieved, respectively.⁸ The best discriminator was the QOL question, which had an AUC of 0.66 and achieved a sensitivity of 78 and a specificity of 41 when a cut point of ≥1 was used. Obstructive voiding symptoms that are typical of men presenting with urethral strictures had minimal utility in discriminating between a patent and strictured urethra.⁹

Heyns and Marais specifically focused on the cut points of an American Urological Association (AUA) symptom index total score of >10 or >15 combined with uroflowmetry to identify men with strictured urethras.⁷ In our screening cohort using cystoscopy as the gold standard, IPSS total scores of >10 and >15 would have identified only 38% and 23% of strictures, respectively. Morey et al demonstrated the clinical relevance of the AUA symptom index in urethral strictures, noting that those who failed urethroplasty had postoperative scores that remained largely unchanged from the preoperative state.⁴ Importantly, they also correlated AUA symptom index scores with objective findings from retrograde urethrograms and uroflowmetry studies. The relatively small sample size of the study (n = 50, 9 recurrences) and inclusion of men who had previously failed surgery may not make these findings generalizable to the overall population. In addition, these men were presenting with recurrence and voiding symptoms, whereas in this study, we were utilizing the IPSS for screening purposes. Furthermore, their definition of failure included both endoscopic and radiographic findings without specifying exact caliber. To our knowledge, we are the first group to validate the IPSS in a large group of men (n = 393, 54 recurrences) using objective cystoscopic criteria (<17 F caliber urethral lumen) obtained on the same follow-up visit.

Uroflowmetry has become a useful study in the diagnosis and management of LUTS related to benign prostatic hyperplasia (BPH) and urethral strictures.^10–12 Prior studies have established its utility specifically in detecting stricture recurrence postoperatively, where Q_max < 10 mL/s, Q_max < 15 mL/s, and ΔQ_max (change from preoperative) < 10 mL/s have shown promise as viable screening tools.^7,13–15 We recently demonstrated that using a novel parameter Q_max-Qaverage may provide a way to quantify a flattened voiding curve. Our attempt to validate uroflowmetry as a screening tool revealed a strong correlation in men under the age of 40 (AUC 0.93 for Q_m-Q_a in detecting cystoscopic recurrence), but was of limited use in the older population given the wide distribution of Q_max likely from the effects of prostatic obstruction.¹⁶

Whereas the IPSS focuses solely on classic voiding parameters, it is likely that a broader approach to symptoms will be required. Nuss et al found that in a group of men presenting with urethral strictures, 21% did not have symptoms captured by the AUA symptom index and 10% were asymptomatic in their evaluation.⁹ Important factors not typically included in a voiding PROM include urethral and bladder pain, which have a higher incidence in men presenting with urethral strictures than previously thought.¹⁷ The superior performance of the “QOL from urinary symptoms” question of the IPSS in detecting cystoscopic recurrence relative to the individual questions may stem from its ability to represent these often unmeasured factors.

The recent development of a validated PROM for urethral strictures by Mundy and colleagues has broadened the approach to assessing LUTS following urethroplasty.¹⁸ Through their structured approach of clinician consensus and patient interviewing, they derived a questionnaire that deviated from the IPSS by elimination of storage symptoms, addition of a question on postmicturition dribble, and addition of Peeling’s voiding picture.¹⁹ Our study corroborates the lack of specificity of storage symptoms, as frequency and urgency symptoms were similar between successful repairs and recurrences. Furthermore, in our study an individual with a successfully repaired stricture would only expect to see significant degree of improvement in strength of stream (question #5) compared to those with objective recurrence. The studies by Jackson et al represent an advancement from a traditional approach to stricture symptoms, and underscore the importance of combining measurements of LUTS, QOL, and overall health status.^18–20

Despite these advances in PROM development, the relatively low prevalence of recurrence has impeded validation and adoption of these urethral strictures questionnaires for postoperative purposes.^18,21 Usage of a PROM on its own specifically for detection of recurrent strictures will likely always present a challenge. We have observed a number of men with “asymptomatic” recurrences in this study and previous work that did report a “relative” slowing of their urinary stream since their urethroplasty, although these types of directed questions have yet to be developed into questionnaire form.⁶ Furthermore, as was shown in this study, the vast majority of men presenting with recurrence reported voiding that was much improved relative to their preoperative voiding, thus being likely to report “improved” QOL and no bothersome symptoms. Clearly, more work must be done in fine-tuning the content and consistency of a PROM specific to urethral strictures before applying it as a stand-alone screening tool.²¹

Our study has several limitations. First, we looked only at anatomical recurrences without considering the functional outcome; a number of our cystoscopic recurrence patients were therefore asymptomatic. It remains unknown whether early detection of these men has a clinical impact. Second, a number of patients enrolled in TURNS studies did not meet our strict study criteria, which required the IPSS and cystoscopy to be performed on the same day. Third, we did not utilize objective voiding data, such as uroflowmetry in our analysis because we were specifically validating the IPSS. However, combining subjective and objective measures would likely have improved test sensitivity and specificity. Finally, although the IPSS remains a popular questionnaire for reconstructive urologists, its reliability has been questioned even in the general LUTS and BPH population, so the results may not be surprising to many readers.^22–24 We believe that future studies on non-invasive testing for recurrence should focus more on patient-specific voiding parameters (ie, nongeneric) that can be followed longitudinally.

CONCLUSION

The IPSS is neither adequately sensitive nor specific enough as a stand-alone tool to monitor for stricture recurrence after urethroplasty. Although, urethroplasty-specific questionnaires have been developed and are currently in development, these data highlight the challenges of capturing stricture symptoms in a generic voiding questionnaire. Patient-specific subjective and objective measures will be necessary to decrease the need for invasive testing.