Validity Testing of the Stopwatch Urine Stream Interruption Test in Radical Prostatectomy Patients

Abstract

Objective

To assess convergent validity of the stopwatch urine stream interruption test (UST). Specific aims were to describe relationships among stopwatch UST scores and four common clinical indices of pelvic floor muscle strength: 24-hour urine leakage, confidence in performing pelvic muscle exercise, 24-hour pad count, and daily pelvic muscle exercise count.

Design

Secondary analysis; instrumentation study.

Methods

The final sample consisted of baseline stopwatch UST scores and measurements of comparison variables from 47 participants in a randomized clinical trial of three approaches to pelvic floor training for patients with urinary incontinence following radical prostatectomy. The sample size provided 80% power to detect correlations of moderate strength or higher. The stopwatch UST was conducted in an examination room at the study site by trained study personnel. Measurements of comparison variables were obtained from three instruments: 24-hour pad test, Broome Pelvic Muscle Self-Efficacy Scale, and 3-day bladder diary. Relationships among study variables were evaluated with Pearson’s correlation coefficients.

Results

Stopwatch UST scores were moderately correlated with 24-hour urine leakage on the 24-hour pad test (r = .35, p < .05), the most robust comparison measure. Correlations between stopwatch UST scores and all other comparison measures were in the appropriate direction, although weak and did not reach statistical significance

Conclusion

Findings suggest the stopwatch UST may be a valid index of pelvic floor muscle strength in men following radical prostatectomy. With further testing, the stopwatch UST could become a valuable clinical tool for assessing pelvic floor muscle strength in radical prostatectomy patients with urinary incontinence.

Background

Prostate cancer is highly prevalent worldwide¹ and the most commonly diagnosed cancer in American men.² Today, more than 90% of prostate cancers in the United States are diagnosed in the local and regional stages³ and radical prostatectomy is the treatment chosen by the majority of men with localized disease.⁴ However, radical prostatectomy carries a significant risk of urinary incontinence (UI) due to post-operative urethral sphincter incompetence (stress UI), detrusor overactivity (urge UI), or a combination of these factors (mixed UI).⁵

An estimated 70,000 radical prostatectomies are performed annually in the United States, mostly in men between the ages of 50 and 70 years. Dramatic growth in the use of minimally invasive approaches to radical prostatectomy during the past decade has benefited patients in terms of reduced blood loss, shorter hospitalizations, and improved post-operative mobility. For many, return to work and usual activity occurs within 2 to 4 weeks.⁶ Nevertheless, most patients still experience some degree of UI for several weeks or months following surgery. Moreover, UI persists in up to 20% of patients, requiring ongoing use of containment products.⁷ Regardless of its time-limited course in most radical prostatectomy patients, incontinence is often demoralizing and impairs quality of life even in the context of successful cancer treatment.^8–11

Systematic pelvic floor training is the mainstay of behavioral interventions for UI following radical prostatectomy.⁷ Pelvic floor muscle training involves building awareness, strength, and endurance of the levator ani and accessory muscles.¹² Pelvic floor training typically consists of: 1) identification of the pelvic floor muscles, 2) isolation of the muscles to the extent possible; and 3) development of muscle strength sufficient to sustain a contraction of 6 – 10 seconds followed by a relaxation period of 5 –10 seconds. Clinicians tend to prescribe a daily exercise protocol of 36 – 50 contraction-relaxation repetitions, performed in sitting, standing, and reclining positions. Optimum benefit may require 6 to 12 months of adherence to a pelvic floor exercise regimen⁵, but benefits may last for as long as 5 years.⁷

In clinical practice, the efficacy of pelvic floor training is often evaluated through the use of bladder diaries that capture self-reported data such as UI frequency, UI volume, and daily pad count. This approach lacks objectivity, can be burdensome for patients, and fails to address the fundamental focus of pelvic floor training, improvement in pelvic floor muscle strength. Objective measures of pelvic floor muscle strength include uroflowmetry and the digital test developed by Brink and colleagues.¹³ However, uroflowmetry is costly and typically available only in urology practices. The digital test of pelvic floor muscle strength is invasive and regular use is unacceptable to many radical prostatectomy patients.

The stopwatch UST¹⁴ is a simple, inexpensive, objective alternative for measurement of pelvic floor muscle strength. The test involves use of a digital stopwatch to quantify the time taken to interrupt urine flow by contracting the pelvic floor muscles. Reliability and validity of the stopwatch UST as a measure of pelvic floor muscle strength have been supported in women with stress urinary incontinence (UI). Sampselle¹⁴ found that stopwatch UST scores demonstrated: test-retest reliability or repeatability (r = 0.69, p<.001); convergence with measurements from the uroflowmeter, a computerized pressure-sensitive device that records the volume, speed, and duration of urination (r = 0.90, p < .001); congruence with digital assessment of pelvic floor muscle strength (r = −.49, p < .001); and the ability to discriminate between patients with and without self-reported stress UI (t = −4.83, p < .001). Similarly, in a prospective simple-blind study of 101 women with and without symptoms of stress UI (n = 51 and 50 respectively), Amaro and colleagues¹⁵ found that only 25% of women with stress UI were able to perform the UST, compared to 80% of continent women (p < .05). Congruent with these findings, visual presence of perineal contraction and digitally palpated pelvic floor muscle strength were significantly greater among women in the continent group.

In contrast, Onyeka and Ogah¹⁶ questioned the validity of the UST as an index of pelvic floor muscle function based on findings from their sample of 104 women with urodynamically confirmed stress UI. Eighty-seven percent of these women were able to stop their urine stream within 2 to 5 seconds for up to two seconds during urodynamic testing. No justification was provided for the selected criterion of UST success in this study, nor was there an explanation for the decision to measure UST performance with categorical rather than continuous data. Evidence also refutes the validity of the UST as a measure of detrusor function in both older women¹⁷ and men.^18,19 The trustworthiness of the stopwatch UST as a measure of pelvic floor muscle strength, however, has not been studied in men.

Study Purpose

The purpose of this study was to assess convergent validity of the stopwatch UST in men who underwent radical prostatectomy. Convergent validity is an appraisal of how closely an instrument is related to different methods of measuring the same phenomenon.²⁰ Thus, specific aims were to describe relationships among stopwatch UST scores and four common clinical indices of pelvic floor muscle strength: 24-hour urine leakage, confidence in performing pelvic muscle exercise (PME), 24-hour pad count, and daily PME count.

Methods

This instrumentation study was part of a randomized pilot clinical trial that evaluated three approaches to pelvic floor training for patients with UI following radical prostatectomy.²¹ Participants from the parent study were: English-speaking; 50 years of age or older; recovering from radical prostatectomy performed at the study site less than 1 month prior to enrollment; incontinent of urine (any amount) following indwelling catheter removal; and without a pre-operative history of UI. Data collection occurred prior to the intervention at baseline (post-operative month 1) and again at the end of postoperative months 3, 6, and 9.

Comparison measures for this study were limited to those used in the parent study. Since the parent study did not employ the gold standard uroflowmeter or digital test of pelvic floor muscle strength, validity testing was limited to evaluation of the stopwatch UST’s convergent validity (i.e., correlation of the stopwatch UST with other credible measures of pelvic floor muscle strength).

The sample consisted of baseline stopwatch UST scores and measurements of comparison variables from 48 of the 54 participants in the parent study. Stopwatch UST scores from the remaining 6 participants were not obtained due to either the participant’s withdrawal from the study prior to baseline data collection (n=3) or inability to produce a urine stream at the time of data collection (n=3). A sample size of 48 provided 80% power to detect correlations of approximately 0.40²² assuming a type I error rate of .05.

Measures

Measurements of comparison variables (24-hour urine leakage, confidence in performing PME, 24-hour pad count, and daily PME count) were obtained from three instruments: 24-hour pad test, Broome Pelvic Muscle Self-Efficacy Scale, and 3-day bladder diary. The 24-hour pad test is the gold standard objective indicator of the amount or severity of involuntary urine loss. The test has been shown to yield results that are reproducible, correspond to urodynamic findings of continence or incontinence, and correlate highly with pad testing carried out over a longer period.^23–26

To perform the pad test, participants wore a set of 12 pre-weighed absorbent pads for 2 hours each over a 24-hour period. Used pads were returned within 72 hours of test completion to prevent measurement errors related to evaporative loss.^{25, 26} Pads were then weighed on a digital gram scale to detect differences between dry and wet weights, which correspond to the amount of urine loss in milliliters. Differences between dry and wet weights were summed to obtain 24-hour urine leakage in milliliters.

The Broome Pelvic Muscle Self-Efficacy Scale is an established measure of perceived self-efficacy and outcome expectations for performing pelvic floor muscle exercises to prevent urine loss.²⁷ The scale is written at an 8^th grade reading level and consists of 23 items divided into two subscales: efficacy expectations and outcome expectations. The efficacy expectations subscale asks about confidence in performing pelvic floor muscle exercises in various situations. The outcome expectations subscale asks about confidence that pelvic floor muscle contraction will prevent urine loss in a variety of circumstances. Participants completed both the 14-item efficacy and 9-item outcome expectations subscales, which required rating each item on an 11-point scale ranging from 0 (no confidence) to 100 (complete confidence). Scores for each subscale are derived from the average of scores for individual subscale items. The total scale score is derived from the average of total scores for each subscale. Higher scores indicate greater perceived efficacy and outcome expectations concerning pelvic floor muscle exercises. Total scale scores can range from 0 – 100, which are interpreted as follows: 0 – 32 (low), 33 – 66 (moderate), and 67 – 100 (high).

In testing with samples of community-dwelling older women, both subscales and the total scale demonstrated acceptable test-retest and internal consistency reliability; content, construct and predictive validity were also supported.^{27, 28} In the parent study²¹, reliability coefficients for the Broome Scale were high (alphas=.92–.95) for radical prostatectomy patients at each of four data collection points over the first 9 post-operative months. Mean scores ranged from 70.5 at baseline, prior to pelvic floor training, to 84.4 at the final data collection point, when most participants were experienced in pelvic floor muscle exercises and UI was no longer an issue. Moreover, improvement in exercise self-efficacy over time was statistically significant (p<.01) within all groups, supporting the sensitivity of the Broome Scale in this population.

The 3-day bladder diary is a well-accepted self-reported record of voiding frequency, episodes of urine leakage, fluid intake, 24-hour pad count and daily pelvic floor muscle exercise count. Bladder diaries have long been used in clinical trials of UI interventions for radical prostatectomy patients.^29–33

Several studies support use of the 3-day bladder diary to obtain reliable and valid measurement of voiding frequency and episodes of urine leakage in community-dwelling men and women.^23,34,35 The superiority of the 3-day bladder diary over both the 2-day and 7-day diary relative to patient compliance and burden has also been documented in both groups.^34,36 In the parent study²¹, UI volume measurements from the gold standard 24-hour pad test were moderately correlated with both number of large UI episodes (r=.33, p<.05) and 24-hour pad counts (r=.45, p<.01) from the 3-day bladder diary, supporting criterion validity of the 3-day bladder diary as a measure of UI volume in radical prostatectomy patients.

Participants completed the 3-day bladder diary independently at home prior to conducting the 24-hour pad test. The 24-hour pad count and daily PME count were obtained by averaging the counts recorded for these variables on each day of the diary.

Study Procedures

The stopwatch UST was conducted in an examination room at the study site by the principal investigator (JPR) or a trained research assistant. Prior to data collection, interrater reliability of ≥ .90 was achieved among all study personnel who administered the stopwatch UST. The four research assistants who administered the test received one hour of training by the principal investigator to learn the protocol and acquire the coordination and psychomotor skills needed to conduct the stopwatch UST. Following this initial training, each research assistant was paired with the principal investigator for assessment of interrater reliability. Research assistants required between 10 and 50 simulated stopwatch UST administrations to achieve the desirable interrater reliability.

The stopwatch UST was performed at the participant’s first reported urge to void following intake of 720 ml of water. The participant was screened from view, given a large styrofoam container, and instructed to void when prompted by the tester, who stood adjacent to the participant on the opposite side of the screen. A signal to interrupt urine flow was given by the tester five seconds after voiding was initiated, at which point the stopwatch was activated by the tester. The stopwatch was deactivated by the tester when s/he could no longer hear the urine stream. The time elapsed between activation and deactivation of the stopwatch was recorded as the UST score, which represents the time taken by the participant to interrupt his urine flow by contracting his pelvic floor muscles.

Data Analysis

Frequencies and measures of central tendency were reviewed for all variables and data cleaning procedures were then performed. One missing value for 24-hour pad count constituted the only missing value for comparison measures in the data set and was replaced with the mean 24-hour pad count value for the sample. One stopwatch UST outlier was excluded from the data set to correct for significant skewness and kurtosis in the distribution. The final sample of stopwatch UST scores and comparison values from 47 participants in the parent study was distributed normally (skewness +1.14; kurtosis +1.82).³⁷ The strength and statistical significance of relationships among study variables were then evaluated with Pearson’s correlation coefficients. An a priori level of significance of p < .05 was assumed for the study.

Results

Descriptive characteristics of the sample are summarized in Table 1. Participants (N=47) were 50 to 72 years old (59.3 ± 6.4 years, mean ± SD). The sample was predominantly white (78.7%), married (80.9%), employed (72.3%), college educated (61.7%), and relatively affluent (59.6%). The majority had Gleason scores of 7 or greater (57.4%), robotic radical prostatectomies (70.2%), preservation of both neurovascular bundles (61.7%), and negative surgical margins (66.0%). Body mass index ranged from 20.5 to 38.7 (M = 28.5 ± 4.2) and 24-hour urine leakage of 9.4 to 1212.3 ml (317.1 ±331.5) was observed. Participants had stopwatch UST scores that ranged from 0.40 to 2.03 seconds (1.00 ± 0.33), wore an average of 4.1 (± 2.8) pads in a 24-hour period, had a moderate to high degree of confidence in their ability to perform and use pelvic floor muscle exercises to prevent UI, and reported an average of 27.6 (± 16.1) contractions per day.

Table 1.

Descriptive Characteristics of Sample^a

Variable	M ± SD or n(%)	Range
Age	59.3 ± 6.4	50 – 72
Race: White	37 (78.7)
Marital status: Married	38 (80.9)
Employment: Working	34 (72.3)
Education: 16+ years	29 (61.7)	10 – 19
Annual household income: $90K+	28 (59.6)
Surgery: Robotic	33 (70.2)
Gleason score: 7+	27 (57.4)	6 – 9
NV bundles lost: 0b	29 (61.7)
Body Mass Indexc	28.5 ± 4.2	20.5 – 38.7
Surgical margins: Negatived	31 (66.0)
24-hour urine leakage (ml)	317.1 ± 331.5	9.4 – 1212.3
Stopwatch UST (seconds)	1.00 ± 0.33	0.40 – 2.03
Broome Scalee	70.6 ± 14.6	42.94 – 98.21
24-hour pad count	4.1 ± 2.8	0 – 17.7
Daily PME count	27.6 ± 16.1	7.5 – 75

Notes.

^aN = 47.

^bN = 42.

^cN= 43.

^dN=46

^eTotal score = 0 – 100.

Stopwatch UST scores were moderately correlated with 24-hour urine leakage on the pad test (r = .35, p < .05). Correlations between stopwatch UST scores and all other comparison measures were in the appropriate direction, although weak and did not reach statistical significance (see Table 2). Incidental findings included moderate correlations between 24-hour pad count and both 24-hour urine leakage (r = .44, p < .01) and pelvic floor muscle exercises self-efficacy on the Broome Scale (r = −.33, p < .05).

Table 2.

Correlations between Stopwatch UST and Comparison Measures

Measure	1	2	3	4	5
1. Stopwatch UST	-	.346*	.114	.050	−.087
2. 24-hour urine leakage		-	−.230	.440**	−.166
3. Broome Scale			-	.331*	.051
4. 24-hour pad count				-	−.049
5. Daily PME count					-

^*p < .05, two tailed

^**p < .01, two tailed

Discussion

Convergent validity of the stopwatch UST in radical prostatectomy patients is supported by the relationship between stopwatch UST scores and 24-hour urine leakage on the pad test, the most robust comparison measure. The weak relationships observed between the stopwatch UST and other comparison measures are overshadowed by the finding that all were in the appropriate direction and occurred in a study that relied on a convenience sample that was insufficient in number to detect statistically significant correlations of less than moderate strength. For these reasons, we believe that the stopwatch UST deserves further study as an index of pelvic floor muscle strength in radical prostatectomy patients. A logical next step would be replication of Sampselle’s seminal study²² in order to examine criterion validity (i.e., correlation with gold standard measures of pelvic floor muscle strength) in the radical prostatectomy patient population.

The incidental finding of a link between 24-hour urine leakage on the pad test and 24-hour pad count from the bladder diary differs from findings by Dylewski and colleagues³⁸ and supports continued exploration of self-reported pad count as a legitimate alternative to the painstaking 24-hour pad test for appraisal of pelvic floor muscle strength in clinical practice. The association between PME self-efficacy and 24-hour pad count was in the expected direction and supports Self-Efficacy Theory.³⁹ Hence, greater confidence in the ability to perform PME and achieve desired outcomes may improve adherence to an exercise protocol and reduce UI.

Anecdotes and reflections recorded by the principal investigator in a procedural log over the course of the parent study indicate that the stopwatch UST is easy to administer in the office setting. Patients even looked forward to the test as a concrete benchmark of their progress in building pelvic floor muscle strength and regaining bladder control. Interrater reliability of ≥ .90 was achieved among all study personnel following a training session of approximately one hour and between 10 and 50 simulated test administrations.

Implications for Nursing Practice

Clinical measures are important to providers, patients, and payers for benchmarking progress toward desired outcomes. However, clinical measures are worthless unless they are both accurate and appropriate. Accuracy is determined by evaluating evidence of a measure’s reliability, or the degree to which it renders consistent and dependable measurements of the outcome it was designed to measure. Appropriateness is determined by evaluating evidence of a measure’s validity, or the degree to which it actually measures the outcome that it was designed to measure. Evaluation of convergent validity, the degree to which different measures of the same outcome yield similar results, is one of the most practical and objective approaches to validity testing. This study yielded evidence of convergent validity of the stopwatch UST, a measure of pelvic floor muscle strength, in radical prostatectomy patients.

In nursing practice, clinical measures of nursing-sensitive outcomes are needed to evaluate outcomes of nursing interventions. Pelvic floor muscle training is an independent nursing intervention that has shown promise for improving UI in radical prostatectomy patients. Van Kampen and colleagues⁴⁰ have shown that UI duration after radical prostatectomy can be estimated reliably from urine loss on day 1 after catheter removal. However, this does not mitigate the need for a simple, non-invasive, objective measure that can be used by radical prostatectomy patients and their providers to appraise individual gains in pelvic floor muscle strength, the fundamental focus of pelvic floor training and precursor to continence. The stopwatch UST may ultimately serve this purpose.

For providers, the stopwatch UST yields objective data that can be used to determine a patient’s pelvic floor muscle strength at baseline and monitor changes that occur with pelvic floor training. With further testing, stopwatch UST scores may also be used to supplement self-reports of adherence to prescribed pelvic floor training regimens. Ultimately, stopwatch UST scores could also be normed to provide an additional predictor of UI duration in radical prostatectomy patients.

For patients, our experience suggests that the stopwatch UST may sustain motivation to adhere to the prescribed pelvic floor training regimen. The radical prostatectomy patients in our study were action-oriented in their recovery from UI.⁴¹ They valued stopwatch UST scores as numerical benchmarks of progress and were driven to “improve their numbers.”

For payers, the stopwatch UST may ultimately serve as a mechanism for gauging progress toward recovery of bladder control, particularly if norming of scores is possible. Normed stopwatch UST scores could stimulate the development of evidence-based critical pathways for recovery of bladder control in radical prostatectomy patients and contribute to cost-effective management.

As a clinical measure, the stopwatch UST has the added values of practicality and cost-effectiveness. The test can be administered in any setting, including a patient’s home, outpatient clinic, medical office, and long-term, sub-acute, or acute healthcare facility. Costs associated with the stopwatch UST are minimal; the only items needed are a digital stopwatch and a receptacle for urine. Providers do require training to administer the test, including learning to use the stopwatch, implementing the protocol, and interpreting results for patients. In our study, total training time for research assistants was less than three hours, which includes the time needed to obtain adequate interrater reliability.

Conclusion

Study findings suggest that the stopwatch UST shows promise as a practical and cost-effective benchmark of pelvic floor muscle strength. Reliability and validity of the stopwatch UST have already been demonstrated in women with stress UI. Our findings support convergent validity of the stopwatch UST in the population of radical prostatectomy patients with UI. With further testing, the stopwatch UST may ultimately be useful in clinical research and nursing practice with this population.

Key Points.

• Urinary incontinence (UI) following a radical prostatectomy can be demoralizing and adversely affect quality of life despite successful cancer treatment.

• Systematic pelvic floor training is the mainstay of behavioral interventions for UI following a radical prostatectomy.

• The stopwatch urine stream interruption test (UST) is a simple, inexpensive, objective measure of pelvic floor muscle strength that is well accepted by patients.

• Findings suggest that the stopwatch UST may be a valid index of pelvic floor muscle strength in radical prostatectomy patients.

• With further testing, the stopwatch UST could become a valuable clinical tool for assessing progress toward recovery of bladder control in radical prostatectomy patients.