Support for the Use of a New Cutoff to Define a Positive Urine Culture in Young Children

Abstract

Background:

Conventional urine culture selects for a narrow range of organisms that grow well in aerobic conditions. In contrast, examination of bacterial gene sequences in the urine provides a more complete and relatively unbiased evaluation of the organisms present. Thus, by using 16S rRNA gene amplicon sequencing as the reference standard, we now have the ability to assess the accuracy of urine culture in diagnosing urinary tract infection (UTI).

Methods:

We enrolled febrile children 1 month to 3 years of age with normal urinary tract anatomy that underwent bladder catheterization for suspected UTI at the Children’s Hospital of Pittsburgh. Using 16S rRNA gene amplicon sequencing as the reference standard, we calculated the accuracy of urine culture at various cutoffs [10,000, 50,000, and 100,000 colony forming units per milliliter (CFU/mL)]. Children with ≥80% relative abundance of any organism on 16S rRNA gene amplicon sequencing with elevated urinary markers of inflammation were defined as having a UTI.

Results:

When using a cutoff of 10,000 CFU/mL, the sensitivity and specificity of urine culture were 98% (95% CI: 93–100%) and 99% (95% CI: 97–100%), respectively. Using a cutoff of 50,000 CFU/mL decreased sensitivity n to 80% (95% CI: 68–93%) without changing the specificity. Using a cutoff of 100,000 further decreased sensitivity to 70% (95% CI: 55 – 84%).

Conclusions:

Conventional culture remains an accurate method of diagnosing UTIs in young children; however, these data suggest that a cutoff of 10,000 CFU/mL provides the optimal balance between sensitivity and specificity for children undergoing bladder catheterization.

Article Summary:

Evidence supports the use of a cutoff of 10,000 colony forming units per milliliter to define a positive urine culture.

INTRODUCTION

The definition of a positive urine culture has been the subject of controversy for decades. Using a cutoff of 100,000 colony forming units per milliliter (CFU/mL) to define urinary tract infection (UTI) in adult patients was largely based on a small case-control study reported by Kass¹ in 1956 in which he compared urine culture results of women with clinically-diagnosed pyelonephritis and asymptomatic controls; most women with pyelonephritis had colony counts greater than 100,000 CFU/mL and most asymptomatic women had colony counts less than 10,000 CFU/mL. Almost 30 years later, in a cross-sectional study of young children who underwent bladder catheterization to rule out UTI, Hoberman et al. compared characteristics of children who had growth of 10–49,000 CFU/ml and 50–99,000 CFU/mL.² Among the 35 samples with growth between 10,000 and 99,000 CFU/mL, mixed growth and/or Gram positive cocci were more frequently observed among children with colony counts of 10–49,000 CFU/mL compared to children with colony counts of 50–99,000 CFU/mL. The cutoff of 50,000 has since been the accepted cutoff for the interpretation of culture results of samples collected using catheterization in children younger than 2 years of age.³ However, because a culture-independent reference standard was not used in either of these two studies, they can only be considered as explanatory,⁴ and can provide no more than rough approximations of a cutoff that might prove useful in clinical practice. Indeed, there have been reports of problems with the currently accepted pediatric cutoff of 50,000 CFU/mL. A notable example comes from a study by Swerkersson at al.⁵ in which a sizeable proportion of young children with radiologically confirmed pyelonephritis had colony counts below the currently accepted cutoff of 50,000 CFU/mL.

To investigate tradeoffs between sensitivity and specificity at various cutoffs, a cross sectional study is needed in which both a urine culture (the index test) and a culture-independent reference standard is performed on unselected samples from subjects in whom it is clinically sensible to suspect UTI. Recent advances in 16S sequencing, which uses the exact sequence of the highly-conserved 16S ribosomal RNA (rRNA) gene to identify the bacteria present in samples, now provides us with a sensitive and relatively unbiased reference standard for identification of organisms in the urine. In a previous study, we found a high concordance between 16S rRNA gene amplicon sequencing (hereinafter referred to as 16S sequencing) conventional urine culture in a cohort of young children (no overlap with current cohort) being evaluated for UTI.⁶

In this cohort of young, febrile children undergoing bladder catheterization to rule out UTI, using 16S sequencing as the reference standard, we calculated the accuracy of conventional culture at different cutoffs to identify the one that provides the optimal balance of sensitivity and specificity.

METHODS

Between June 2019 and May 2020, we enrolled consecutive children presenting to the Emergency Department at the Children’s Hospital of Pittsburgh who had remaining urine after all clinical tests had been completed. The study was approved by the Institutional Review Board of the University of Pittsburgh. Children were included if they were between 1 month to through 2 years and 11 months of age, had fever (documented temperature of ≥38° C in the emergency department or by parental report) within 24 hours of presentation, and had a urine sample collected via catheter to rule out UTI. We excluded neonates because this study is part of a larger study examining biomarkers for pyelonephritis. The diagnosis of pyelonephritis is accomplished by performing a renal scan, which is difficult to perform in neonates.

Children were excluded if they had received systemic antibiotics or corticosteroids within 3 days prior to enrollment, had other concurrent systemic bacterial infections, were immunodeficient, had a neurogenic bladder, or had major genitourinary abnormalities (e.g., spina bifida, dysplastic kidneys, Grade IV or IV vesicoureteral reflux). Conventional urine culture was performed at the Hospital’s clinical microbiological laboratory using standard microbiological methods and the number of colony forming units (CFU)/mL were reported as <10,000, 10–49,000, 50–99,000, and ≥100,000.

Urine sample processing for 16S sequencing

An aliquot of residual urine was used for 16S sequencing. Aliquoting generally occurred within one hour of collection; however, if delays were anticipated, samples were kept refrigerated. The aliquot was frozen in a cryovial at −80 °C without preservatives. Before shipping, we added 70 μLs of urine conditioning buffer (Zymo, D3061-1-8) to every 1 mL of frozen urine sample. Samples were shipped overnight in a cold pack to Pangea Laboratory, Tustin, CA, USA, for 16S sequencing analysis using the PrecisionBIOME^™ NGS Microbial Test.

DNA was extracted from the urine specimen using the ZymoBIOMICS DNA Miniprep Kit according to the manufacturer’s instructions (Zymo Research Corporation, Irvine, CA). The extracted DNA was prepared for microbiome analysis following the PrecisionBIOME^™ workflow, which included library preparation using the Quick-16S^™ NGS library prep kit (V1–3 regions, Zymo Research Corporation, Irvine, CA), sequencing of barcoded amplicons with the MiSeq sequencing platform (Illumina, San Diego, CA), and bioinformatics analysis using a the PrecisionBIOME^™ bioinformatics pipeline capable of producing species-level resolution of bacterial and fungal sequences (data on fungi present and cytokines will be reported separately). Negative controls (transport medium alone and unused swabs) were included. To control for contamination, we also included cell and DNA mock communities as positive controls (ZymoBIOMICS microbial community standard, catalog Nos. D6300 and D6305; Zymo Research Corp). Potential sequencing errors and chimeric sequences were removed with the DADA2 pipeline.

16S sequencing Data Processing

We used Uclust to perform taxonomic classifications using a PrecisionBIOME^™ custom database. We computed phylotypes as percent proportions based on the total number of sequences in each sample. Species level resolution of this sequencing approach was previously confirmed by shotgun sequencing.⁷ We excluded samples with <1000 sequences per sample.

Target condition being diagnosed

In a previous study conducted between 2011 and 2017,⁶ we found a high concordance rate between conventional urine culture and 16S sequencing in the identification of bacteria in the urine. This established, we felt it was now appropriate, in the context of this exploratory study, to substitute 16S sequencing instead of urine culture in diagnosis of UTI (the target condition). In clinical practice, addition to presence of bacteriuria on culture, we require elevation of marker(s) of inflammation to diagnose UTI.³ Accordingly, to diagnose UTI in the context of this proof-of-concept study, in addition to requiring that ≥80% of sequences belonged to a single taxon (i.e., relative abundance of any taxa ≥80%), we also required elevation of urinary markers of inflammation. In constructing our reference standard, we chose a cutoff of 80% (for our primary analysis) because we felt that, as clinicians we would be compelled to diagnose UTI in a febrile child with urinary markers of inflammation whose urine had this level of bacteriuria. The remainder of the children were categorized as having “No UTI”. As a sensitivity analysis, we also examined results using relative abundance cutoffs of 50% and 90%.

Urinary markers of inflammation

Each child had a microscopic urinalysis in which white blood cells (WBCs) were enumerated (per cubic millimeter or per high powered field) and a dipstick test in which the leukocyte esterase test was reported semiquantitatively (none, trace, 1+, 2+, 3+). Because the sensitivity of both the WBC and leukocyte esterase tests are generally low,⁸ we also measured, using methods used in our previous studies,⁹ Neutrophil gelatinase-associated lipocalin (NGAL) in an aliquot of residual urine. NGAL is an inflammatory marker released by neutrophils or by intercalated cells in the kidney in response to UTI.^{9, 10} While use of urinary NGAL for the diagnosis of UTI is relatively recent development, given the robust evidence supporting its use to diagnose UTI,¹⁰ we decided, a priori, to categorize a child as having evidence of inflammation if any of the following was present: ≥10 WBC/mm³, ≥5 WBC/hpf, ≥trace leukocyte esterase, or NGAL level above 39.9 ng/mL.¹⁰ We conducted a sensitivity analysis examining results had we not considered NGAL as a marker of inflammation.

Index test being evaluated

We defined the results of conventional culture as positive if the urine culture exhibited growth of at least one organism with a count of at least 10,000 CFU/mL and at least one of the urinary markers of inflammation was elevated. Other cutoffs evaluated were 50,000 CFU/mL and 100,000 CFU/mL.

Statistical analyses

For the primary analysis, we calculated sensitivity and specificity of urine culture at detecting UTI (using a relative abundance cutoff of 90% in a child with urinary markers of inflammation), along with corresponding 95% Wald confidence intervals. We then repeated the analyses using different definitions for UTI (i.e., using relative abundance cutoffs of 50% and 80% instead of 90%). We summarized and analyzed the data using SAS version 9.4 (SAS Institute Inc).

RESULTS

Overall, 341 urine samples were included in the study (Figure 1). Table 1 describes the demographic characteristics of the 341 children included. Most children enrolled were female (74%), white (67%), and most (64%) had a documented temperature of 39° C or more. The mean age of children at diagnosis was 12.5 months and the mean temperature at presentation was 39.3° C. In children with urinary markers of inflammation but without at least one organism with a count of 10,000 CFU/mL or more on culture, the median relative abundance of the predominant identifiable organism was 15% (IQR: 7%−32%).

Figure 1.

Flowchart showing the number of children included and excluded

Table 1.

Demographic and Clinical Characteristics of 341 Children with Suspected Urinary Tract Infection

Characteristic	Total (N=341)
	Number (%)
Age (months)
<2	37 (11)
2–11	136 (40)
12–23	120 (35)
24–35	48 (14)
Sex/circumcision
Female	252 (74)
Male	89 (26)
Male, uncircumcised	20 (6)
Male, circumcised	50 (15)
Male, circumcision status unknown	19 (5)
Race
White	228 (67)
Black	75 (22)
Asian	14 (4)
Multiracial/Other	4 (1)
Not reported	20 (6)
Maximum reported temperature
≥ 39° C	219 (64)
38–39° C	122 (36)
Any urinary marker of inflammationa
No	268 (79)
Yes	73 (21)

^aUrinary white blood cells count of ≥5/high powered field, white blood cells count of ≥10/mm³, leukocyte esterase of ≥trace, or urinary neutrophil gelatinase lipocalin of ≥39.9 ng/mL

Using a relative abundance cutoff of 80%, 46 children in our sample had a UTI. Of these, 41 (89%) had E. coli as the predominant pathogen (Figure 2). When using a cutoff of ≥10,000 to define a positive urine culture, among 46 children with UTI, 45 were correctly identified by conventional urine culture (sensitivity of 98%, CI: 93–100%, Table 2). The one child that was missed was a 4-month-old with 3+ leukocyte esterase on dipstick, a WBC count of 73/hpf, an elevated NGAL of 319, in whom >99% sequences identified by 16S sequencing mapped to Klebsiella species. On culture, this child had <10,000 CFU/mL of Gram-negative rods and <10,000 CFUs of gram positive cocci; neither organism was further identified because of the low colony counts. Of the 295 children without UTI, 291 were correctly identified as such by conventional culture (specificity of 99%, CI: 97–100%).

Figure 2.

Genus-level taxonomic assignment of the top amplicon sequence variants recovered from urine samples of children with and without urinary tract infection. UTI was defined by presence of at least 80% relative abundance of any organism in a child with elevated urinary markers of inflammation. White bars indicate that no taxonomic assignment was determined. Some bars do not add up to 100% because of sequences that could not be mapped to known taxa.

Table 2.

Accuracy of Conventional Urine Culture at Different Cutoffs Compared with 16S rRNA Gene Amplicon Sequencing

	Number correctly identified as having UTI	Number correctly identified as not having UTI	Sensitivity (95% confidence interval)	Specificity (95% confidence interval)
Primary analysis a
Cutoff of 10,000	45/46	291/295	98% (93% – 100%)	99% (97% – 100%)
Cutoff of 50,000	37/46	293/295	80% (68% – 93%)	99% (98% – 100%)
Cutoff of 100,000	32/46	293/295	70% (55% – 84%)	99% (98% – 100%)
Sensitivity analysis using more stringent definition of UTI b
Cutoff of 10,000	43/44	289/297	98% (92% – 100%)	97% (95% – 99%)
Cutoff of 50,000	37/44	294/297	84% (72% – 96%)	99% (98% – 100%)
Cutoff of 100,000	32/44	294/297	73% (58% – 87%)	99% (98% – 100%)
Sensitivity analysis using less stringent definition of UTI c
Cutoff of 10,000	49/50	289/291	98% (93% – 100%)	99% (98% – 100%)
Cutoff of 50,000	39/50	290/291	78% (66% – 90%)	100% (99% – 100%)
Cutoff of 100,000	34/50	290/291	68% (54% – 82%)	100% (99% – 100%)

UTI, Urinary tract infection

^aUTI defined by presence of any taxa at ≥80% relative abundance in a child with urinary markers of inflammation

^bUTI defined by presence of any taxa at ≥90% relative abundance in a child with urinary markers of inflammation

^cUTI defined by presence of any taxa at ≥50% relative abundance in a child with urinary markers of inflammation

Using a cutoff of ≥50,000 CFU/mL decreased the sensitivity of urine culture to 80% (95% CI: 68–93%, Table 2). Change in cutoff to ≥50,000 had a negligible effect on specificity (i.e., specificity remained at 99%, CI:98–100%, Table 2). Characteristics of the 8 extra children with UTI that would have been missed if a cutoff of 50,000 had been used are listed in Table 3; all children with missed UTIs had organisms currently considered as uropathogens, and, by definition, all were symptomatic and had elevated urinary markers of inflammation.

Table 3.

Characteristics of Children in Whom Urinary Tract Infection Would have been Missed when Using a Cutoff of 50,000 CFU/mL (instead of 10,000 CFU/mL).

Age (months)	Elevated WBC count or leukocyte esterasea	Elevated NGALb	Most abundant organism on 16S (relative abundance)	Conventional culture result: Organism(s) and colony count per mL
12	No	Yes	Klebsiella (0.86)	Klebsiella, 10–49,000
27	Yes	Yes	Escherichia coli (0.91)	E. coli, 10–49,000
16	Yes	No	Escherichia coli (0.92)	E. coli, 10–49,000
28	Yes	Yes	Escherichia coli (>0.99)	E. coli, 10–49,000
13	Yes	Yes	Escherichia coli (0.98)	E. coli, 10–49,000
9	No	Yes	Escherichia coli (0.83)	E. coli, 10–49,000
7	Yes	Yes	Escherichia coli (>0.99)	E. coli, 10–49,000
10	Yes	Yes	Escherichia coli (0.99)	E. coli, 10–49,000

^a≥10 WBC/cubic millimeter, ≥5 WBC per high power field, or ≥trace leukocyte esterase

^bNGAL level 39.9 ng/mL and above

Use of a cutoff of 100,000 CFU/mL would have reduced sensitivity to 70% (95% CI: 55 – 84%, Table 2). Changes in the definition of the reference standard had little effect on estimates of accuracy (Table 2).

As can be seen in Table 4 in the supplement, sensitivity and specificity estimates would have been similar had we not included NGAL as a marker of inflammation.

DISCUSSION

Previous studies have attempted to understand urine culture results by comparing colony counts in extreme states (i.e., pyelonephritis vs. asymptomatic)¹ or by comparing characteristics of subjects above and below certain colony count cutoffs.² Such study designs, although necessary during the early exploratory phases of research on diagnostic testing, cannot provide a true assessment of the accuracy of a test in clinical practice, nor can they be used to identify cutoffs that optimize sensitivity and specificity.⁴ Here, using 16S sequencing as the reference standard for the diagnosis of UTI, we were able to assess the diagnostic accuracy of urine culture at various colony count cutoffs. We found that, in febrile children under 3 years of age undergoing bladder catheterization to rule out UTI, using a colony count of 10,000 CFU/mL would have led to fewer missed cases of UTI than using a colony count of 50,000 CFU/mL (missed 20% of cases) or 100,000 CFU/mL (missed 30% of cases). Per study design, all children with missed UTI had elevated urinary markers of inflammation and were febrile. Accordingly, our data suggest that a cutoff of 10,000 CFU/mL best differentiates young children with and without a bona fide UTI.

There are many reasons why 16S sequencing, especially when paired with urinary markers of inflammation, is a suitable reference standard for the diagnosis of UTI. First, unlike urine culture, which is optimized to detect E. coli,¹¹ 16S sequencing analysis provides a largely unbiased assessment of bacteria present in the urine. Indeed, a growing body of evidence^{12, 13} suggests that the results of 16S sequencing analysis align well with results of Expanded Quantitative Urine Culture -- a more sensitive variation of the urine culture that uses additional culture media, larger volumes of the urine used to inoculate culture plates, longer incubation times, and a variety of atmospheric conditions.^{14, 15} However, while the data on the higher sensitivity of 16S sequencing compared to urine culture seems undisputed, its increased sensitivity could theoretically come at the cost of lower specificity, especially because of the required amplification step. Thus, one might imagine a scenario in which many non-clinically relevant organisms could be identified by 16S sequencing, leading to a large number of inappropriately diagnosed UTIs. In our study, however, use of 16S sequencing (paired with markers of inflammation) uncovered only 1 child, in which UTI was missed. This suggests that 16S sequencing analysis (at a relative abundance threshold of 80%) paired with inflammatory markers of inflammation was a suitable reference standard to use for this study.

When we started this study, we were concerned about using a relative measure of abundance as our reference standard. However, our results (in this study and in the previous one) show that, for the diagnosis of UTI, relative abundance from 16S sequencing and absolute culture results were surprisingly similar. We hypothesize that for a pathogen like E. coli to account for 80% or more of all sample reads on 16S sequencing, the number of copies of its gene have to exceed the combined counts¹⁶ of all the other bacteria normally present as part of the urobiome by 4-fold. Thus, the 80% relative abundance threshold will only be met when the absolute abundance of a uropathogen is quite high.

Our results support the continued use of urine culture (at a cutoff of 10,000 CFU/mL) in the population studied. However, because of continuing technological advances in the accuracy, speed, and cost of 16S sequencing, it may soon become available in clinical centers. If this occurs, identification of organisms could be accomplished in a matter of hours instead of days. Thus, it is important that comparative studies are conducted to establish the pros and cons of using this technology in the clinical setting.

Several limitations of this study need to be considered. We used a definition of UTI that required the presence of symptoms, elevated urinary markers of inflammation, and a high relative abundance of organisms on 16S sequencing analysis. Using a less stringent definition (e.g., not requiring urinary markers of inflammation) might have led to a higher rate of missed UTIs, but we would have been less certain that all these UTIs represented bona fide UTIs. Our choice of 80% for the relative abundance threshold required to define UTI, although based on data from our previous study, was nevertheless, to an extent, arbitrary. We acknowledge that less abundant organisms may also be capable of causing significant disease. Ultimately, as borne out in the sensitivity analysis, the cutoff chosen for the primary analysis had little influence on our conclusions. This is likely because, in most cases, samples were either dominated by sequences from a single known uropathogen (Figure 2) or had very low abundance of a variety of organisms that had, in earlier studies, been detected in urine samples from asymptomatic individuals.¹⁴ Our findings only directly apply to children who have bladder catheterization performed to rule out UTI; a higher cutoff for urine culture may be needed if more contamination, for example related to the method of collection, is expected. We did not use preservatives before freezing the samples; however, samples were never left at room temperature, and our results do not suggest bias due to the approach we used. Our clinical laboratory did not report exact colony counts for the conventional urine cultures; this might have been helpful in pinpointing the colony count threshold less than 50,000 CFU/mL that best optimized the sensitivity and specificity of the urine culture. Most UTIs in our sample were caused by E. coli. Larger studies are needed to further examine the accuracy of colony count thresholds for less common uropathogens. Finally, 16S sequencing, albeit less biased than urine culture, can itself be subject to certain biases, especially when the biomass in samples is relatively low;¹⁷ however, these biases are unlikely to change the pattern observed from one in which many organisms are present at relatively low abundances to one in which the sample is dominated by one organism. Strengths of the study included consecutive enrollment of symptomatic children with suspected UTI, use of a reference standard based prior data, performance of the index test and reference standard on all included children, the use of a priori clinically based definitions, use of positive and negative controls during extraction and sequencing, and the use of validated sequencing approach.

Inevitably, lowering the cutoff for the diagnosis of UTI, from 50,000 CFU/mL to 10,000 CFU/mL, will increase the number of children falsely labeled as having UTI. Nevertheless, this is appropriate because (1) the number children falsely labeled as having a UTI will be less than the number of children with missed UTIs that will be uncovered (see Table 2), and (2) the negative consequences of missing a febrile UTI usually outweigh that of prescribing an additional course antimicrobials.

In conclusion, using 16S sequencing as the reference standard, we found preliminary empirical evidence to support use of a cutoff of 10,000 CFU/mL for the diagnosis of UTI in young febrile children.

What’s known on this subject:

To establish the cutoff with the highest accuracy on the urine culture test requires comparison with a reference standard. No studies to date have compared the urine culture with a culture-independent reference standard.

What this study adds:

Data from this diagnostic accuracy study provides support for the use of a cutoff of 10,000 colony forming units per mL to diagnose urinary tract infection in febrile children undergoing bladder catheterization.

Abbreviations

UTI

Urinary Tract Infection

CFU

Colony Forming Unit

rRNA

ribosomal ribonucleic acid

WBC

white blood cell

NGAL

Neutrophil gelatinase-associated lipocalin

hpf

high power field