Usefulness of UF-5000 automatic screening system in UTI diagnosis

Abstract

We have evaluated the Sysmex UF-5000 cytometer use in microbiology for the screening of negative urines, looking for cut-off points to detect bacteria and leukocytes. The number of processed urines was 3569, the highest to date in these studies. The best general cut-off point has been 100 bact/μl, giving an area under the ROC curve of 0.868, a sensitivity of 96%, a specificity of 50%, 1.17% of false negatives, and saving 40% of cultures. The PPV and NPV have been 35.5 and 95.4 respectively. The leukocyte count has not been useful. Finally, we have evaluated urine screening usefulness, concluding that in laboratories such as ours (284 urines/working day) or smaller, it is not cost-effective.

Introduction

Urine samples are the most abundant in clinical microbiology laboratories, the majority being negative in culture. Diagnosis and treatment based only on clinical criteria can give 33% false positives and unnecessary or erroneous treatments [1]. For over a decade, urine screening methods using automatic systems have been implemented with the intention of lowering the number of cultured samples, reducing the workload, and anticipating the report of negative results [2]. These systems come from the field of clinical biochemistry and use bioluminescence, nephelometry, flow cytometry, and urinary sediment analysis techniques through automated microscopy images, providing dozens of parameters, of which, only the bacterial count and the leukocyte count have been shown to be of any use for microbiological screening [1–5]. Nevertheless, the reference standard, nowadays, is still urine culture plate [6].

Following the criteria of the European guidelines for urine analysis [7], the sensitivity of the screening systems should always remain above 90%. The critical point to carry out an effective screening is the establishment of cut-off points for bacteria and/or leukocytes that allow differentiating the urines with pathological microbial counts from those that present contaminating microbiota or are negative. A very restrictive cut-off point could lead to a high percentage of false negatives, while a more relaxed one could lead to so many false positives that the technique ceased to be cost-effective. An example of the latter is the Korean study by Song et al. [8] with Sysmex UF-5000, in which only 10% of the samples analyzed were validated as negative, forcing 90% of the urine samples to be seeded. Thus, as the cut-off points are modified, the sensitivity, specificity, positive predictive value, and negative predictive value of the technique change. It must be remembered that the values of sensitivity and specificity obtained in any study are always referred to the reference standard used, which contrary to expectations, is not so standardized: many laboratories use lower standards than the ones defined by Kass (positive if >10⁵ CFU/ml) for pediatric patients, immunosuppressed patients, or kidney transplant recipients, among others [6, 9] (Table 1).

Table 1.

Cut-off points proposed in different studies for urine screening with Sysmex UF-5000 and their relationship with the criteria established as the reference standard for culture in agar

Authors	Samples	Reference standard	Cut-off Sysmex	% false negatives	Sensitivity/% culture reduction
Alenkauer et al. [1] (2021)	1119	>105 CFU/ml	50 CFU/μl	0.2%	99%/30%
Enko et al. [10] (2021)	344	>105 CFU/ml	135 CFU/μl	–	92.1%/–
Haugum et al. [3] (2021)	3468	>104 CFU/ml	30 CFU/μl	2.7% or 0.77%	95.2%/30%
Algarra et al. [11] (2012)	1730	>104 CFU/ml	50 CFU/μl	3.3%	91.3%/46.5%
Broeren et al. [12] (2011)	1577	Not growth >104 CFU/ml >105 CFU/ml	26 CFU/μl 39 CFU/μl 230 CFU/μl	14% – 0.3%	95%/20% 95%/28% 95%/52%
De Rosa et al. [13] (2018)	2719	>105 CFU/ml	58 CFU/μl	0.6%	99%/55%
This work	3569	>104 CFU/ml >105 CFU/ml	100 CFU/μl	1.17%	96.1%/40%

The objective of our study was to find the best cut-off point for bacteria and/or leukocytes for screening and to evaluate the usefulness of these devices in the clinical microbiology laboratory.

Material and methods

The study was conducted in the Microbiology Unit of the Virgen del Rocío University Hospital in Seville (HUVR), where 71,157 urines from 44,580 patients were processed in 2021 (284 urines per working day). The samples were collected by spontaneous urination, taking exclusively the middle stream. Urine transport tubes carried boric acid as a preservative, which has recently been shown not to affect growth on agar [3]. A total of 3569 randomly selected samples taken between September 2021 and April 2022 were processed. All of them were processed by the Sysmex UF-5000 analyzers and subsequently 10μl were seeded with a calibrated loop on BD Chromagar Orientation agar. Incubation was carried out for 24h at 35°C.

The Sysmex UF-5000 uses blue semiconductor laser fluorescent flow cytometry (ë = 488 nm) and hydrodynamic focusing on two analysis channels: Core Channel, detecting nucleic acid components; and Surface Channel, detecting non-nucleic acid components. Fluorescent flow cytometry counts and classifies particles in urine by analyzing forward scattered light (FSC) that indicates the size and length of the particles and side-scattered light (SSC) that provides information on the cellular content of the particles. The UF-5000 uses 0.8 ml of sample in manual mode and 1.2 ml in automatic mode. It can process up to 100 samples/hour. Once the sample is introduced into the equipment, it is dyed using two separate analytical channels, one for bacteria and one for other particles. Next, the sample is transferred to a flow cell, where the light scattered by each particle is detected in two different directions: one the same as the incident beam and the other forming an angle of 90° with that beam. With this information, Sysmex UF-5000 classifies and quantifies bacteria, yeasts, and leukocytes, apart from other corpuscles such as cells, cylinders, sperm, crystals, and mucus. As calibration standards, it uses microspheres that incorporate a fluorescent marker and compares the signal of these standards with that of the sample. The fluorochrome used stains both live and dead bacteria. The microbial count ranges from 10 to 1×10⁵ CFU/ml [1, 14, 15].

Culture interpretation criteria: >10⁴ CFU/ml was considered positive if the germ was in pure culture, and >10⁵ CFU/ml if two different colonies were isolated (Table 1). If three or more kinds of colonies, the culture was considered contaminated, although they were considered positive for the screening technique. Urine from pediatric patients by suprapubic puncture or catheterization and nephrostomies were not included in the study as far as these urines require different interpretation criteria than those obtained by spontaneous urination [6, 15]. Urine from transplant patients was also not included to avoid false negatives in this group.

The results obtained were analyzed by using the interactive shiny web application FlowUTI [16] (https://covidiario.shinyapps.io/flowuti/), created to determine optimal cut-off values for different UTI markers obtained by flow cytometry, such as bacteria or leukocytes. Sensitivity, specificity and positive and negative predictive values were determined by using this app.

Results

A total of 3569 urines were cultured; 2544 from women and 1025 from men; 2866 came from primary care and 704 from hospitalized patients, although positive cultures were distributed almost 50/50. Seven hundred sixty-one specimens (21.3%) were culture positive, 238 (6.6%) contaminated, and 2570 (72%) negative. Among the positive ones, 85% had gram-negative isolates and 12% gram-positive isolates. Three percent had yeasts. The most frequent germs were Escherichia coli (50.6%), Klebsiella pneumoniae (16.5%), Enterococcus faecalis (6.7%), Proteus mirabilis (4.1%), Pseudomonas aeruginosa (3.26%), Enterobacter cloacae (2.8), Klebsiella aerogenes (1.4%), and Candida albicans (1.3%).

The best general cut-off point turned out to be 301 CFU/μl, a value with the largest area under the ROC curve and the best Youden index, giving a sensitivity of 90%. However, in order to reduce the number of false negatives to a minimum, we imposed a higher sensitivity (>95%) as a condition, which we achieved with a cut-off of 100 CFU/μl (Fig. 1). Also, trying to increase specificity, we analyzed male and female subpopulations separately, obtaining optimized cut-off points of 50 CFU/μl for men (Fig. 2), and 200 CFU/μl for women (Fig. 3), keeping >95% sensitivity. In the same way, and considering the specific epidemiology presented by the elderly population compared to adults and young people, the cut-off points for the population aged 55 years or older, and those younger than that age were analyzed. In both cases, the optimal cut-off point was 100 CFU/μl (Table 2).

Fig. 1.

Total 100 CFU/μl. Sensitivity 96.1%

Fig. 2.

Men 50 CFU/μl. Sensitivity 96%

Fig. 3.

Women 200 CFU/μl. Sensitivity 95.2%

Table 2.

UF-5000 results with various cut-off points for bacteria and leukocytes/μl, Youden’s index, and AUC. Minimum cut-off values for each group for which the sensitivity is at least 95% are shown

	SENS%	SPEC%	VPN%	VPP%	FP%	FN%	Youden’s index	ROC
Total 100 CFU/μl	96.1	32.2	95.4	35.5	48.9	1.1	0.28	0.87
Women 200 CFU/μl	95.2	32.7	94.5	35.5	48.4	1.3	0.28	0.85
Men 50 CFU/μl	96	43	96.8	46.8	40.9	1.1	0.39	0.91
>=55 years 100 CFU/μl	96.6	43	95.3	51.3	35	1.3	0.39	0.92
<55 years 100 CFU/μl	95	25	95.6	23	81.6	0.93	0.23	0.82
Total 21 WBC/μl	91.7	27.8	89.7	33	52	2.3	0.21	0.77

Leukocytes were also considered as a possible criterion, finding the optimal cut-off point at 21 WBC/μl. The area under the ROC curve was 0.77 (Fig. 4). The sensitivity was 91%, while the specificity remained at 28%. The use of both values combined (cut-off point of >100 CFU/μl or >21 WBC/μl) barely raises sensitivity to 97%, while the specificity is reduced to 15%.

Fig. 4.

Total 21 WBC/μl. Sensitivity: 91.7%

Finally, the percentage of contaminated crops was 6.6%. These cultures have been considered positive for screening purposes in order to calculate the effectiveness of cytometry.

Discussion

Several studies have been carried out about the cut-off points and efficacy of the Sysmex autoanalyzer. In the reviewed literature, the proposed cut-off points show high variability, which is probably related to the different populations and pathologies that each center treats and to the way in which each researcher defines his or her own reference standard as seen in Table 1.

In this work, the best cut-off point for all patients was 100 CFU/μl. This value keeps sensitivity above 95%, reducing the number of seeded samples by 40%. However, to maintain this high sensitivity, the specificity is reduced to 32.2%, resulting in a high number of false positives. Regarding the division of the sample into subpopulations either by sex or age, and as we see in Table 2, sensitivity or specificity did not significantly improve, so it would not be useful.

A drawback of lowering the cut-off point to avoid false negatives is that it can facilitate contamination from one urine to the next (carryover), as Haugum et al. [3] have recently shown. To avoid this, the author recommends making an aliquot for culturing from a tube not previously inserted into the UF-5000. We have not observed this problem.

The leukocyte count is one of the points of debate on this topic. Our optimal cut-off point is 21 WBC/μl. The area under the ROC curve is, in this case, 0.77 (Fig. 4), so, the criterion, by itself, is less useful to discriminate positives than the use of the bacteria count. The sensitivity is 91%, while the specificity remains at 27.8%. The use of both values combined (cut-off point of >100 CFU/μl or >21 WBC/μl) slightly improves sensitivity, placing it at 97%, but in exchange, it sensibly reduces specificity, leaving it at 15%. In some of the studies reviewed, leukocytes are useful [13, 17–19] while in others they are not [1, 12, 20]. But as far as our data is concerned, the leukocyte count barely provides sensitivity to the result and, on the contrary, notably increases the number of false positives, so we do not consider it a useful parameter (Table 2 and Fig. 4).

We agree with other works that as we increase the number of bacteria that are considered to give a positive culture (reference standard), the sensitivity of the analyzer increases [1, 12, 21]. As we consider some urines positive depending on the clinic, even with 10⁴ CFU/ml, we have less sensitivity than other publications [12, 14, 17]. Marshall et al. [14], taking a plate count of >10² CFU/ml as a UTI criterion and using a very low cut-off in screening that should provide a minimum of false negatives, obtained a sensitivity of 80.9% and a specificity of 78.0%. Instead, Manoni et al. [17], taking a plate count >10⁵ CFU/ml as a criterion and with a screening cut-off of 125 CFU/μl, obtained a sensitivity of 99% and a specificity of 77%. In our opinion, this shows that is not possible to establish a standardized cut-off point for urine screening, and that, each laboratory, depending on the profile of patients and according to the specific criteria for culture interpretation, sets its own optimized cut-off.

Regarding the reduction of cultures, with a cut-off point of 100 CFU/μl, we avoid culturing 40% of the urines. It is important to acknowledge that since we do not pass through the analyzer the urine of small children and transplant recipients, nor those received outside working hours tomorrow, the total percentage of urines whose seeding we avoid is lower. Regarding this, there is also great variability in the different published studies and the percentage of culture reduction ranges between 10 and 55% (Table 1).

The percentage of false negatives (FN) with a cut-off point of 100 CFU/μl was 1.1%, like that obtained by other authors [18, 19] (Table 1). Regarding the microorganisms isolated in these FN, 24% were P. mirabilis and 19% E. faecalis, and in a smaller proportion, we found E. coli, Serratia marcescens, Streptococcus agalactiae, staphylococci, etc. In general, 38% were gram-positive, which, added to P. mirabilis, reached 62%. Our results contrast with De Frutos-Serna et al. [21] who report E. coli as the main agent of false negatives ahead of Proteus spp. and gram positives.

It should be noted that gram-positive cocci are prone to forming bacterial aggregates that can interfere with cytometer readings [12]; this could be an explanation for this predominance of gram positives in FN. Thus, Wang et al. [18] isolated an Enterococcus spp. and a Staphylococcus spp. in their two false-negative samples, and Manoni et al. [17] noted that half of the isolates obtained from false-negative urines corresponded to gram-positive cocci. This agrees with our data.

The percentage of contaminated cultures was 6.6%, like other published works [13, 19, 22] although there are higher ones: Algarra et al. [11] report 20% of contaminated cultures. When evaluating the technique, we decided to consider these contaminants equivalent to a positive, since we agree with Haugum et al. [3] that no parameter of the UF-5000 is useful to predict the contaminants, especially that of desquamative cells. For this reason, those urines labelled as contaminated should be cultured for their correct evaluation.

Screening usefulness

The reduction of the workload, the main point that is usually put forward in favor of screening systems, must be subdued to a task comparison. In our laboratory, the time used by the technical staff to introduce urine in the analyzer and to culture the positive ones is about 5 h a day. This also includes an hour and a half to perform daily controls, besides a general check-up on Mondays, settle barcode errors, ensure that urine is not cloudy, with crystals or with less than 1.5 ml, solve incidents (7–8 a day), and check the connections with the Laboratory Information System (LIS). The samples labelled as “positive” and “contaminated” must be cultured later, and it must be done manually since the samples must be uncapped to introduce them into the UF-5000, and our automatic instrument (WASP Bio-Merieux) requires specimens to keep the cap. There are capping machines although this would add another step of complication and cost to the process.

An alternative would be to avoid screening and directly culture all the urines using an automatic specimen processor. Given that in our laboratory we already have technical personnel attending to this machine, we have calculated that the extra work time would be between one and 2 h only, considerably less time than that involved in attending to the cytometer and processing the urine samples manually. The workload for the specialist would increase by about 20 to 30 min a day (100–120 plates), although he would not have to solve the daily problems of result transmission failures between the UF-5000 and the LIS (about 10 min daily).

It is also argued that screening saves up to 24 h of antibiotic treatment in negative urine [22], and that the new scattergrams shown by the UF-5000 can differentiate between gram-positive and gram-negative germs 24 h before the culture result (only if monomicrobial) with a concordance with the culture of 62%, similar to the urine gram stain according to Enko et al. [10]. In this work, they even propose UF-5000 as an alternative to gram staining, since it presents a better area under the ROC curve. However, in our laboratory, this would only be useful for samples from hospitalized patients who arrive during the morning hours. On the other hand, most of the urine samples we receive come from primary care. To avoid these blind treatments, a 24/7 communication system would be necessary with the responsible doctors and for them to locate the patients by telephone, which is currently unfeasible. And in the case of hospitalized patients, they would benefit if the UF-5000 were permanently in operation 24/7 with its corresponding technicians and doctors, which is not feasible in our hospital either.

The limited reduction in the number of urine cultures in some published works is surprising: sometimes only 10% or 30%. Nevertheless, the authors consider it advantageous in hospitals where 20,000–30,000 urines are processed per year, resulting in a small daily number of plates saved in absolute numbers [8, 21–23], in our opinion.

In terms of cost, using the UF-5000 costs us 0.90 €/sample (Broeren et al. [12] and March-Roselló et al. [24] estimate 1.3 and 2 €/sample), and if we perform an agar culture the cost is 0.70 €/sample. But actually, more than half of the samples are plated anyway after leaving the analyzer because they are positive in the screening, raising the cost to 1.56 €/sample. Alenkaer et al. [1] calculate that the cost of UF-5000 reagents and plates is the same, although they use three agar plates for each urine sample.

Finally, the main advantage of plating every urine sample is that the sensibility and specificity would be 100%.

Conclusion

Our optimal cut-off point, following the criterion of minimizing false negatives, would be 100 CFU/μl without considering subpopulations. And regarding the usefulness of screening, taking into account the costs, handling times, and results of the UF-5000, and according to some publications [12, 21], the analyzer does not seem useful in a laboratory with a similar number of samples as ours or less. Based on our patient profiles and interpretation criteria, it appears that the Sysmex UF-5000 may not be useful in this context.

Author contribution

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by H. Toledo, S. G. Punzón, and J. A. Pérez. The first draft of the manuscript was written by H. Toledo and J. A. Pérez and supervised by J. A. Lepe. G. Martin-Gutierrez designed the FlowUTI software used for data analysis. All authors read and approved the final manuscript.

Declarations

Conflict of interest

The authors declare no competing interests.