Validity, Accuracy, and Predictive Value of Urinary Tract Infection Signs and Symptoms in Individuals With Spinal Cord Injury on Intermittent Catheterization

Abstract

Background/Objective:

To determine the validity, accuracy, and predictive value of the signs and symptoms of urinary tract infection (UTI) for individuals with spinal cord injury (SCI) using intermittent catheterization (IC) and the accuracy of individuals with SCI on IC at predicting their own UTI.

Design:

Prospective cohort based on data from the first 3 months of a 1-year randomized controlled trial to evaluate UTI prevention effectiveness of hydrophilic and standard catheters.

Participants:

Fifty-six community-based individuals on IC.

Main Outcome Measures:

Presence of UTI as defined as bacteriuria with a colony count of at least 10⁵ colony-forming units/mL and at least 1 sign or symptom of UTI.

Methods:

Analysis of monthly urine culture and urinalysis data combined with analysis of monthly data collected using a questionnaire that asked subjects to self-report on UTI signs and symptoms and whether or not they felt they had a UTI.

Results:

Overall, “cloudy urine” had the highest accuracy (83.1%), and “leukocytes in the urine” had the highest sensitivity (82.8%). The highest specificity was for “fever” (99.0%); however, it had a very low sensitivity (6.9%). Subjects were able to predict their own UTI with an accuracy of 66.2%, and the negative predictive value (82.8%) was substantially higher than the positive predictive value (32.6%).

Conclusions:

The UTI signs and symptoms can predict a UTI more accurately than individual subjects can by using subjective impressions of their own signs and symptoms. Subjects were better at predicting when they did not have a UTI than when they did have a UTI.

INTRODUCTION

The clinical diagnosis of urinary tract infection (UTI) in the general population is typically established by assessing for key symptoms, such as dysuria and urinary frequency, followed by finding evidence of the presence of bacteria or inflammation in the urinary tract using dipstick and urinary analysis. Once this is accomplished, most often empirical treatment with antibiotics is started while results of urine culture are pending. Although this approach is useful for the general population, it is not applicable to individuals with spinal cord injury (SCI) because they usually do not complain of dysuria because of loss of sensation, nor do they complain of increased urinary frequency because they most commonly have abnormal voiding that requires alternative bladder emptying methods. They typically manage their bladders using indwelling urethral or suprapubic catheters, clean intermittent catheterization (IC), or in men, condom catheters.

IC has become a mainstay in the care of patients with SCI who have adequate hand function or a willing health care provider. It is favored mainly because of evidence that it decreases the incidence of UTI and urologic complications (1,2). In addition to standard uncoated polyethylene catheters, the market today provides hydrophilic-coated catheters. It is hypothesized that the hydrophilic catheters may cause fewer UTIs than standard catheters because there is evidence that the former is associated with less urethral irritation and the suggestion that this may lead to decreased bacteriuria (3). However, there is a lack of randomized controlled trials (RCTs) that compare the performance of hydrophilic catheters to standard catheters.

There are no standard criteria for the exact level of bacteriuria or on the exact set of signs and symptoms that constitute a symptomatic UTI in individuals with SCI on IC. A frequently used UTI criterion for these individuals is the presence of significant bacteriuria (≥10⁵ CFU/mL) and at least 1 sign or symptom of UTI. The National Institute on Disability and Rehabilitation Research (NIDRR) consensus statement on the prevention and management of UTI among people with SCI lists the following set of signs and symptoms as suggestive of UTI: leukocytes in the urine, fever, discomfort or pain over kidney or bladder or during urination, urinary incontinence, increased spasticity, autonomic dysreflexia, cloudy urine with increased odor, malaise, lethargy, and sense of unease (4).

Signs and symptoms of UTI in individuals with SCI on IC are generally considered to have poor sensitivity and specificity. However, there is a scarcity of data on validity, accuracy, and predictive value of UTI signs and symptoms in the medical research literature. The goal of this study was to determine the sensitivity, specificity, and predictive value of UTI signs and symptoms for individuals with SCI on IC. Additionally, it evaluated the accuracy with which individuals with SCI on IC can predict their UTI based on subjective perception of their own signs and symptoms.

METHODS

This study pursued its goals by analyzing a subset of the data that was collected during a 1-year RCT designed to evaluate the effectiveness of hydrophilic catheters in prevention of UTI in individuals with chronic SCI. The RCT collected data on laboratory results (urinalysis and urine culture), UTI signs/symptoms, resource utilization, and social-demographic characteristics. The study reported here used laboratory and UTI signs/symptoms data, collected during the first 3 months of the 1-year RCT, to address questions pertaining to the validity, accuracy, and predictive power of UTI signs and symptoms.

The RTC was approved by the University of Washington Institutional Review Board and was conducted with volunteer community-dwelling persons with SCI from 2001 to 2005. All study participants signed an informed consent, which included consent to provide the urine samples and UTI signs and symptoms data that were used as the basis for this study.

Individuals were included in the RCT if they met the following criteria: a SCI for more than 6 months; self-reported history of 2 or more UTIs during the past year; previously performing IC with a noncoated catheter; not planning to change the method of bladder drainage during the study period; naïve to hydrophilic catheters; and at least 18 years old. Individuals were excluded if they had evidence of upper tract abnormalities or renal or bladder calculi in a screening renal ultrasound. Before beginning data collection, subjects were screened for UTI using urinary analysis with culture and sensitivity. Subjects who had a UTI were treated with oral antibiotics before assignment to 1 of the 2 types of catheters. A total of 56 individuals were enrolled in the RCT and randomly assigned to either the hydrophilic catheter group or the standard catheter group.

The main outcome measure was the incidence of UTI, defined as bacteriuria of at least 10⁵ colony-forming units (CFU)/mL plus the presence of at least 1 UTI sign or symptom. The UTI signs and symptoms were taken from the NIDRR consensus statement on prevention and management of UTI among people with SCI (4). They were leukocytes in the urine, discomfort or pain over kidney or bladder, incontinence, increased frequency of catheterization, fever, increased spasticity, autonomic dysreflexia, cloudy urine, foul smell in the urine, feeling sick, feeling tired, and sense of unease. All participants were asked to use their regular health care provider as they would normally for the management of any UTI that they developed.

Study participants completed a UTI signs/symptoms questionnaire and provided urine samples once per month for 3 months. The main questions in the questionnaire were “how certain are you that you have a UTI?” and “please tell us about any increase in specific symptoms that you are experiencing that may be related to a UTI.” The participants had 5 options for the answer to the first question: “definitely do have a UTI,” “do have a UTI, “not sure,” “do not have a UTI,” and “definitely do not have a UTI.” The idea behind the graded answers was to facilitate the answering process for participants who would either not be sure or would not feel comfortable giving a strong “yes” or “no” answer to a question like “do you think you have a UTI today?” For the signs and symptoms of UTI, which were easier for the individual to determine if present or not, “yes” or “no” responses were provided for all that were being studied, except for the presence of leukocytes in the urine, which was determined by urinalysis. The questionnaire, created specifically for the RCT, was not previously validated or used in other studies.

Participants either delivered their urine specimen or a study investigator picked up the specimen from their homes. As mentioned, urine samples were obtained on a monthly basis. This was thought to be more convenient and less time consuming than asking subjects to provide samples on a bi-weekly or weekly basis. The subjects provided a catheterized specimen taken after discarding 40 to 50 mL of urine and using the clean technique. The specimens were collected in a 4-oz sterile urine cup and transported on ice to the laboratory within 2 hours.

For the purposes of this study, validity is the extent to which a test measures what it was designed to measure. It has 2 components: sensitivity and specificity. Sensitivity is the ability to correctly identify individuals who have a specific disease, and specificity is the ability to correctly identify individuals that do not have a specific disease. Accuracy is the ability of the test to produce true values for the measurement and is defined as the proportion of true results among all test results. Predictive value measures the true presence and absence of a disease. Positive predictive value is the proportion of true positives among all positives, and negative predictive value is the proportion of true negatives among all negatives.

RESULTS

Of the 56 participants enrolled in the RCT, 3 were dropped because of incomplete records, and 2 were dropped because of absent symptom information, resulting in a total of 51 study participants contributing data. Of the participants in this study, 26 were assigned to a standard polyethylene catheter and 25 to a hydrophilic catheter. Of the 153 possible patient visits (51 participants with monthly visits for 3 months), 130 were available for analysis because of missing data related to participants being unable to attend their scheduled visits or because of participants providing urine samples but not recording their signs and symptoms. There were data on 63 patient visits for the standard polyethylene catheter group and on 67 patient visits for the hydrophilic catheter group.

For the standard catheter group, the distribution of SCI neurologic levels was 13 cervical, 8 thoracic, and 5 lumbar. For the hydrophilic catheter group, the distribution was 6 cervical, 18 thoracic, and 1 sacral. Of all 51 participants, 26 (51%) were at risk for autonomic dysreflexia because they all had SCI neurologic level equal to T6 or above. For the standard catheter group, the distribution of ASIA Impairment Scale (AIS) grade was 14 AIS A, 5 AIS B, 3 AIS C, and 4 AIS D. For the hydrophilic catheter group, the distribution was 17 AIS A, 4 AIS B, 1 AIS C, and 3 AIS D. All study participants, except 1 with C2 SCI, were able to perform intermittent catheterization on their own.

During the first 3 months of the RCT, there were a total of 12 UTIs in the standard catheter group and a total of 17 UTIs in the hydrophilic catheter group. A chi-square analysis was performed to determine whether the type of catheter had an effect on the incidence of UTIs. The results of the analysis, using a significance criterion of P < 0.05, suggested that the type of catheter had no significant effect on the incidence of UTI (P  =  0.39). Additionally, a series of chi-square analyses were performed to examine whether the type of catheter had an effect on any of the signs and symptoms. Again, the results of the analysis, using the same significance criterion of P < 0.05, suggested no significant effect; the P values ranged from 0.14 to 0.96, they were all greater than 0.05, and we could not refute the null hypothesis of no effect. After verifying that the type of catheter had no significant effect on UTI, or on any of its sign and symptoms, we proceeded to combine the data from the standard and hydrophilic catheter groups for all subsequent analyses. This allowed for a larger sample size (130) and greater ability to possibly identify small effects.

The standard catheter group contained 12 women and 14 men, whereas the hydrophilic catheter group contained 5 women and 20 men, a composition that was quite different in terms of male to female ratio. Because of this, a chi-square analysis was performed to examine if sex had an effected on the incidence of UTI or to the UTI prediction accuracy. The P value for incidence of UTI was 0.11 and the P value for UTI prediction accuracy was 0.45. These results suggested that sex had no significant effect on UTI or UTI prediction accuracy.

Table 1 shows the sensitivity, specificity, predictive value, and accuracy of UTI signs and symptoms of subjects on IC. “Cloudy urine” had the highest accuracy (83.1%) and second highest positive predictive value (61.3%) and sensitivity (65.5%). “Leukocytes in urine” had the highest sensitivity (82.8%) and highest negative predictive value (93.8%). “Fever” and “autonomic dysreflexia” had the highest specificities (99.0%) but very low sensitivities because of the low (or absent) number of true positive tests. “Foul smell in urine” had the second highest accuracy (79.2%) and the third best sensitivity (48.3%).

Table 1.

Validity, Accuracy, and Predictive Value of UTI Signs and Symptoms

Table 2 shows the study participant's ability to self-predict whether or not they have a UTI. Subjects were asked “how certain are you that you do or do not have a UTI?” The prediction accuracy was calculated after grouping the 5 possible graded responses into only 2 categories: negative or positive. The grouping that led to the highest UTI prediction accuracy categorized the test as negative if the participant answered “no, definitively” or “do not have a UTI,” and categorized the test as positive if the participant answered “yes, definitively” or “do have a UTI” or “not sure.” The overall accuracy was 66.2%. The negative predictive value (82.8%) was substantially higher than the positive predictive value (32.6%). In other words, participants were much better at predicting when they did not have a UTI than when they did have a UTI.

Table 2.

Patient Self-Prediction of UTI

The top 3 bacterial pathogens present with colony count of at least 10⁵ CFU/mL in urines of subjects diagnosed with UTI were Enterococcus (24%), Escherichia coli (18%), and Klebsiella pneumoniae (15%).

DISCUSSION

Although an expert panel has identified a set of signs and symptoms that is clinically useful for the diagnosis of UTI in individuals with SCI (4), there is a paucity of published data on how these signs and symptoms compare with each other in terms of validity, accuracy, and predictive value. This study suggests that “leukocytes in the urine” has the highest sensitivity (82.8%) and that “cloudy urine” and “foul smell in the urine” have the highest accuracy (83.1% and 79.2%, respectively) combined with above average sensitivities (65.5% and 48.3%, respectively). The highest specificities occurred for “fever” and “autonomic dysreflexia” (99.0%); however, both of these also had the highest false-negative rates and the lowest number of true-positives, and therefore, have low utility in UTI screening for this population. The low incidence of autonomic dysreflexia may in part be because of the fact that only 51% of the subjects had SCI neurologic level equal to T6 or above. It is interesting to notice that all signs and symptoms listed in Table 1 except “increased spasticity” are more accurate predictors of UTI than asking patients how certain they are whether or not they have a UTI.

These results suggest increased utility of assessing the appearance and smell of the urine in the management UTI for individuals with SCI on IC. This may be especially helpful in the inpatient setting, where urine is collected several times per day. Urine can be hazy or cloudy because of the presence of amorphous phosphates and urates; after these, the most common causes of hazy, cloudy, or turbid urine are white cells, red cells, epithelial cells, and bacteria. The normal ammonia odor of urine is the result of the breakdown of urea; bacterial infections may produce a strong unpleasant smell (5).

The medical research literature is scarce on formal studies that evaluate the sensitivity and specificity of UTI signs and symptoms in individuals with SCI. Deresinski and Perkash (6,7) studied the value of symptom evaluation in predicting the presence of bladder bacteriuria in males with SCI who were catheter-free but using condoms. They concluded that symptoms were not particularly useful in predicting urine culture results because of their nonspecific nature and their frequent absence in the presence of bacteriuria. The authors stated that many of the symptoms reported were vague and nonspecific, such as unexplained sweating, which may be suggestive of autonomic dysreflexia, which in turn can be triggered by any number of stimuli other than UTI. In our study, autonomic dysreflexia was found to have no positive predictive value, and despite its negative predictive value of 77.5%, its false-negative rate was too high (100%) for it to be a useful negative predictor of UTI.

Linsenmeyer and Oakley (8) conducted a 9-month prospective case review study that evaluated 147 persons with SCI who presented to an outpatient urologic clinic with symptoms that they attributed to a UTI. The diagnosis of a UTI was determined by clinical signs and symptoms, bacterial colony count greater than 10⁴ CFU/mL, and evidence of localized bladder wall tissue invasion manifested by increased white blood cells (WBCs ≥ 10/HPF) in the urine. Unlike the study reported here, which included only intermittent catheterization, the study of Linsenmeyer and Oakley included various methods of bladder management: indwelling urethral catheter, IC, suprapubic catheter, reflex voiding, voiding with control, and diaper use. Of the 147 patients, 61% were accurate in predicting the presence of UTI based on their symptoms. There were a total of 46 subjects with SCI on IC; the accuracy for this group was 50%.

The study by Linsenmeyer and Oakley (8) was based on outpatients who presented to the urologic clinic with symptoms they attributed to a UTI. This study was based on community-based individuals who volunteered for a RCT and were scheduled to provide urine sample and sign/symptom data regardless of whether they believed they had a UTI or not. This resulted in a higher percentage of clinic visits with no UTI, compared with the study by Linsenmeyer et al (8), where they went to the clinic because they thought they had a UTI. This higher percentage of clinic visits with no UTI may have led to the higher accuracy observed in this study (66.2%) compared with the study by Linsenmeyer and Oakley (50%). As shown in Table 2, the negative predictive value of subject self-prediction of UTI (82.8%) is much higher than the positive predictive value (32.6%). Regardless of the differences in the final accuracy results, both studies suggested that individuals with SCI on IC are frequently not accurate in self-predicting UTI.

As mentioned previously regarding the signs and symptoms questionnaire, the most accurate UTI self-prediction occurred when we grouped the “not sure” response with the responses that were clearly positive (“yes, definitively” and “do have a UTI”). This may have happened because UTI self-prediction had a negative predictive value that was substantially higher than its positive predictive value, meaning that subjects were more “sure” about not having a UTI, so they tended to answer “not sure” if they thought there was a small chance that they did in fact have a UTI.

The analysis of group assignment suggested no difference, in the incidence of UTI, between the hydrophilic and the standard catheter group. This is consistent with a recent Cochrane review study (9) on bladder management by intermittent catheterization, which compared sterile vs clean catheterization, coated vs uncoated catheters, and other strategies designed to reduce UTIs. The Cochrane study reviewed 4 parallel group trials designed to compare coated catheters with uncoated catheters. One trial, reported by De Ridder et al (10), showed fewer UTIs in the hydrophilic-coated catheter group, but was only marginally statistically significant. All of the other 3 trials suggested no difference. Based on the current evidence, it is not possible to state that 1 catheter type is better than the other.

It is important to point out that, despite the fact that the preliminary analysis shown in the results section indicated that the type of catheter (standard vs hydrophilic) had no significant effect on UTI or on the UTI sign and symptoms, the goal of the study was not to compare standard and hydrophilic catheters. It does not evaluate the possible complications of IC, such as hematuria from trauma to the urethral wall, nor does it address the effects that the different catheters may have on risk for conditions such as epididymitis, fistula, calculi, or carcinoma. The study used a preliminary comparative analysis to ensure the appropriateness of combining UTI data to from both types of catheters. This allowed for a larger sample size for the measurement of sensitivity, specificity, accuracy, and predictive value of UTI signs and symptoms for the SCI individuals on intermittent catheterization.

The study did not rule out other concurrent medical conditions, such as pneumonia or bacteremia, which may have been the cause of the recorded signs and symptoms. This was a limitation, but of the total of 130 subject visits, there were only 3 cases of fever for which there may have been clinical justification for further workup, and of those 3, only 1 was not a UTI. Another limitation was that the study used a signs and symptoms questionnaire that had not been previously validated. However, the questions came directly from the signs and symptoms listed in the UTI criteria of the NIDRR consensus statement (4) and were similar to what a physician would ask to inquire about a possible UTI. In this sense, the validity of the questionnaire mirrors the validity to the UTI criteria itself, with inherent challenges that derive from the subjective nature of the symptoms.

Escherichia coli is the most common cause of uncomplicated UTI and accounts for approximately 75% to 95% of all infections (11). In this study, E. coli accounted for only 18% of all infections. This pattern of organism distribution suggests that individuals with SCI on IC have complicated UTIs.

CONCLUSION

In this study, the most accurate signs of UTI in SCI individuals on IC were “cloudy urine” and “foul smell in the urine,” whereas the most sensitive sign was “leukocytes in the urine.” This study also showed that the UTI signs and symptoms predicted a UTI more accurately than did individuals using subjective impressions of their own signs and symptoms and that the subjects were better at predicting when they did not have a UTI than when they did have a UTI.