Can We Reduce Negative Blood Cultures With Clinical Scores and Blood Markers? Results From an Observational Cohort Study

Abstract

Only a small proportion of blood cultures routinely performed in emergency department (ED) patients is positive. Multiple clinical scores and biomarkers have previously been examined for their ability to predict bacteremia. Conclusive clinical validation of these scores and biomarkers is essential.

This observational cohort study included patients with suspected infection who had blood culture sampling at ED admission. We assessed 5 clinical scores and admission concentrations of procalcitonin (PCT), C-reactive protein (CRP), lymphocyte and white blood cell counts, the neutrophil-lymphocyte count ratio (NLCR), and the red blood cell distribution width (RDW). Two independent physicians assessed true blood culture positivity. We used logistic regression models with area under the curve (AUC) analysis.

Of 1083 patients, 104 (9.6%) had positive blood cultures. Of the clinical scores, the Shapiro score performed best (AUC 0.729). The best biomarkers were PCT (AUC 0.803) and NLCR (AUC 0.700). Combining the Shapiro score with PCT levels significantly increased the AUC to 0.827. Limiting blood cultures only to patients with either a Shapiro score of ≥4 or PCT > 0.1 μg/L would reduce negative sampling by 20.2% while still identifying 100% of positive cultures. Similarly, a Shapiro score ≥3 or PCT >0.25 μg/L would reduce cultures by 41.7% and still identify 96.1% of positive blood cultures.

Combination of the Shapiro score with admission levels of PCT can help reduce unnecessary blood cultures with minimal false negative rates.

The study was registered on January 9, 2013 at the ‘ClinicalTrials.gov’ registration web site (NCT01768494).

INTRODUCTION

Although blood cultures are routinely collected in patients with suspected infection presenting to the emergency department (ED), their sensitivity for bacteremias is low, with <10% of cultures showing growth of bacteria.¹ Moreover, contamination limits their specificity.²

Multiple studies have evaluated clinical scores for their utility in the prediction of bacteremia with the aim to improve the (pre-test) probability of positive culture results. A study conducted by Shapiro and colleagues enrolled 3730 ED patients with suspected infections and found 13 clinical parameters integrated into a single clinical score to be able to predict positive cultures with high accuracy.³ This score, which incorporated major and minor criteria, was also externally validated and proved to be a sensitive but not specific predictor of bacteremia.⁴ Another bacteremia prediction model proposed by Lee and colleagues found 7 clinical variables to accurately predict bacteremia in a total of 2422 patients with community-acquired pneumonia (CAP).⁵ Jones and colleagues studied 270 patients and found systemic inflammatory response syndrome (SIRS) criteria, the basis of the sepsis definition, to be predictive of bacteremia.⁶ Metersky and colleagues studied 13,043 patients with CAP and found the absence of recent antibiotic treatment, liver disease, 3 vital signs, and 3 laboratory abnormalities to be relatively accurate predictors of bacteremia.⁷ Finally, Tokuda and colleagues studied 526 patients with acute febrile illness and generated 3 different risk groups for bacteremia with 2 prediction algorithms (Tokuda scores I and II).⁸ The 5 clinical scores described above are summarized in full detail in Appendix 1.

In addition to the clinical scores discussed above, biomarkers that correlate with the probability of bacteremia have also been described. Several studies have found procalcitonin (PCT) levels to predict blood culture results in patients with pneumonia,^9–13 urinary tract infections,¹⁴ sepsis,¹⁵ and acute febrile illness.¹⁶ Similar data are available for C-reactive protein (CRP),^13,17 neutrophil-lymphocyte count ratio (NLCR),¹⁸ and lymphocytopenia,^18,19 with significant differences in levels of these biomarkers between bacteremic patients and patients with negative blood cultures. Finally, red blood cell distribution width (RDW) has been proposed as a mortality marker for bacteremia.²⁰

Most of these clinical scores have only been evaluated in patients with CAP,^5,7 but not in a more heterogeneous, clinically challenging medical patient population presenting to the ED with suspected infection. We, therefore, aimed to validate the prognostic potential of these clinical scores alone and in combination with novel biomarkers in an ED patient population with suspected infection.

METHODS

Study Design and Setting

This is an observational cohort study. We prospectively included all consecutive medical patients with suspected infection presenting to the emergency department of a Swiss tertiary care hospital with additional regional primary and secondary care functions between February 2013 and October 2013 who had initial blood culture samples drawn. Blood cultures were drawn at the discretion of the treating physician. All patients were participants in the TRIAGE project, a prospective, observational study that aimed to devise an algorithm to optimize triage of adult patients with medical emergencies.^21,22

The aim of this study was to compare 5 different clinical scores and 6 biomarkers for their ability to predict blood culture positivity. The primary endpoint was true blood culture positivity as assessed by 2 independent physicians and an infectious disease specialist according to Centers for Disease Control and Prevention (CDC) criteria (http://www.cdc.gov/getsmart/healthcare/implementation/clinicianguide.html).

Given that this was an observational quality control study, the Institutional Review Board (IRB) of the Canton of Aargau approved the study and waived the need for informed consent (approval number EK 2012/059). The study was registered at the “ClinicalTrials.gov” registration web site (NCT01768494).

Participants and Definitions

Infections were classified on the basis of the main organ involved into the following categories: upper respiratory tract infections, lower respiratory tract infections, urinary tract infections, intra-abdominal infections, skin and soft tissue infections, central nervous system infections, endocarditis, foreign-material associated infections, and “other” infections. Excluded were patients who directly presented to the surgical ward and pediatric patients < 18 years of age.

In all patients, 2 pairs of blood culture samples for both aerobic and anaerobic cultures (equalling 40 mL of blood altogether) were collected before initiation of antibiotic therapy. Blood cultures were processed using an automated colorimetric detection system (BacT/ALERT, bioMérieux, Durham, NC).²³ If blood culture bottles indicated bacterial growth, samples were Gram stained and subcultured. Identification of the pathogen was performed according to routine laboratory procedures. A blood culture was considered truly positive when it yielded a pathogen typical for the infection site. The evaluation was done by 2 independent physicians. In case of uncertainty, the case was discussed with an infectious disease specialist. The following species were usually considered to be contaminants: coagulase-negative Staphylococci, Corynebacterium species, and Propionibacterium species, unless an association with intravascular catheters/devices was suspected. In 1 case of a central line-associated infection and 1 case with clinical suspicion of endocarditis, infections with coagulase-negative Staphylococci were considered to be true infections (Appendix 2).

Clinical Examination and Laboratory Data

In all patients, we recorded pertinent initial vital signs (eg, blood pressure and heart rate) and clinical parameters (eg, chills, vomiting, and comorbidities). Clinical information including socio-demographics, and patient outcomes were assessed prospectively until hospital discharge using the routinely gathered information from the hospital electronic medical system used for coding of Diagnosis-Related Group (DRG) codes.

Samples for later measurement of biomarkers were collected upon ED admission. The following markers were measured as part of routine care: CRP (normal range < 3.0 mg/L), albumin (normal range: 34–50 g/L), WBC (normal range: 4–10 × 10⁹/L), urea (normal range: 2.0–7.0 mmol/L, equals blood urea nitrogen [BUN] in mg/dL divided by 2.8), creatinine (normal range 80–115 μmol/L, divide by 88 for mg/dL), neutrophil percentage/proportion (normal range: 40–85%), neutrophil bands (normal range 0–10%), platelets (normal range 140–400 × 10⁹/L),^12,24 plasma sodium (normal range: 136–146 mmol/L), and red blood cell distribution width (RDW) (normal range: < 15%). The neutrophil-lymphocyte count ratio (NLCR) was calculated by dividing the absolute neutrophil count by the absolute lymphocyte count.

In addition, we measured PCT levels post hoc with an automated rapid sensitive assay (KRYPTOR PCT; Thermo Scientific Biomarkers [formerly BRAHMS AG], Hennigsdorf, Germany; lower limit of detection: 0.02 μg/L).²⁵

Clinical Scores and Biomarkers

The clinical bacteremia scores including the Shapiro score,^3,4 the Lee score,^5,6 the SIRS criteria,^4,6 the Metersky score,⁹ and the Tokuda score I and II^8,22 are summarized in full detail in Appendix 1.

In addition, we also focused on several biomarkers that have been found to predict positive cultures. Biomarkers were used as continuous variables and at predefined cut-offs. First, we measured PCT using Kryptor technology and cut-off values were defined as 0.1 μg/L, 0.25 μg/L, 0.5 μg/L, and 1.0 μg/L based on previous studies.^{9,12,23,24,26} We used the published cut-offs of ≥ 10 and ≥ 12 for NLCR¹⁸ and < 1 × 10⁹ g/L for absolute lymphocytopenia.²⁷ A RDW cut-off of >15 % was used as previously described.²⁰

Statistical Analysis

This report adheres to the STROBE guidelines for reporting observational studies.²⁸ Discrete variables are expressed as counts (percentage) and continuous variables as medians and interquartile ranges (IQR). Frequency comparison was done by the chi square test. The 2-group comparison Mann–Whitney U test was used. To assess the prognostic performance of different parameters in predicting blood culture positivity, logistic regression analysis was used. We used biomarker levels as continuous variables and at predefined cut-offs as defined above. Logarithmic transformation of biomarker levels was used to obtain normal distribution for skewed variables. Receiver operating characteristics (ROC) were calculated, with the area under the curve being a measure of discrimination. The area under the ROC curve (AUC) is thus a summary measure over criteria and cut-point choices. The AUC summary equals the probability that the underlying classifier will score a randomly drawn positive sample higher than a randomly drawn negative sample. To test whether the biomarker levels improve clinical scores, we compared the nested logistic regression model with clinical scores and biomarkers with a model limited to the clinical scores alone. We also performed subgroup analyses to assess the performance of the different scores and markers within different types of infections and in Gram-positive and Gram-negative infections. We used STATA 12.1 (Stata Corp, College Station, TX). All testing was 2-tailed, with P <0.05 considered as indicating statistical significance.

RESULTS

Baseline Parameters

The median age of all included 1083 patients was 67 years (IQR 53–78) and 57.6% were males. True bacteremia was detected in 104 patients (9.6%). A detailed list of detected pathogens is presented in Appendix 2. A total of 28 patients (2.6% of all patients, 21.2% of those with positive blood cultures [28/132]) had contaminated blood cultures.

Table 1 shows patient characteristics on admission overall and separated according to blood culture results. Patients with positive cultures had less frequent antibiotic pretreatment and a lower diastolic blood pressure, whereas the core body temperature was significantly higher (38.5 °C vs 38.0 °C, P < 0.001). Laboratory analysis showed that CRP, NLCR, albumin, and urea were significantly higher in patients with positive blood cultures, whereas the lymphocyte count, RDW, and sodium were significantly lower. In addition, patients with positive blood cultures had almost 8.5-fold higher PCT levels compared to patients with negative cultures (1.69 vs 0.20 μg/L; P < 0.001).

TABLE 1.

Baseline Characteristics of Patients Overall and Separated by Blood Culture Positivity and Negative or Contaminated Blood Cultures

As for in-hospital outcomes, patients with positive blood cultures were more frequently transferred to the ICU (12.5 % vs 6.1 %; P = 0.01), had an increased length of stay (8 days vs 6 days; P = 0.01), and had a significantly higher 30-day mortality rate (15.4% vs 7.7%; P = 0.01).

Clinical Scores and Biomarkers to Predict Positive Blood Cultures

Table 2  displays the performance of the different clinical scores in predicting culture positivity from logistic regression models and discrimination (AUC). Of the clinical scores, the Shapiro score performed best with an AUC 0.729, followed by the Tokuda score II (AUC 0.665). The other clinical scores performed only moderately: Lee score (AUC 0.623), Metersky score (AUC 0.610), SIRS criteria (AUC 0.546), and Tokuda score I (AUC 0.566). Antibiotic pretreatment was a modest negative predictive factor for positivity of blood cultures, with an AUC of 0.552. The best biomarkers were PCT (AUC 0.803), NLCR (AUC 0.700), and lymphocyte counts (AUC 0.675). On the other hand, CRP, RDW, and WBC did not show significant results (AUC 0.645, 0.610, and 0.544, respectively).

TABLE 2.

Results of Univariate Regression Analysis and AUC

Combination of Clinical Scores and Biomarkers

Combining the Shapiro score and PCT showed the best results, with the AUC of the combined model being 0.827 (Table 3). The NLCR, CRP, or lymphocyte count could not significantly improve the predictive ability of the Shapiro score when combined individually with the latter. Combining the biomarkers PCT, NLCR, CRP, and lymphocyte count together with the Shapiro score resulted in an AUC of 0.817, which was not better than the combination of the Shapiro score and PCT alone.

TABLE 2 (Continued).

Results of Univariate Regression Analysis and AUC

Subgroup Analyses

Subgroup analyses (Table 4) showed the best performance of the Shapiro score to be for skin and soft tissue infections (AUC 0.756), urinary tract infections (AUC 0.694), and infections with Gram-negative bacteria (AUC 0.737). The Metersky score had the best performance for skin and soft tissue infections (AUC 0.737). On the other hand, PCT showed the highest AUC for lower respiratory tract infections and infections with Gram-negative bacteria, with AUC values of 0.876 and 0.837, respectively.

TABLE 3.

Combined Models

Diagnostic Measures

Table 5 shows the sensitivity, specificity, and positive and negative likelihood ratios for clinical scores, biomarkers, and promising combinations of the two. At a cut-off of ≥2 points as previously described by Shapiro and colleagues,³ the sensitivity of the Shapiro score for prediction of positive blood cultures was 95.2%. At a PCT cut-off of 0.1 μg/L, the sensitivity for prediction of positive blood cultures was 99.0%.

TABLE 4.

Subgroup Analyses (AUC)

TABLE 5.

Diagnostic Performance for Prediction of Bacteremia by the Shapiro Score and Biomarkers

We also calculated diagnostic performance measures in a combined model using Shapiro's score and PCT. Limiting blood cultures to patients with either Shapiro scores ≥4 points or PCT levels > 0.1 μg/L would reduce negative sampling by 20.2% while still identifying 100% of positive cultures. Using Shapiro scores of ≥3 points or PCT levels > 0.25 μg/L would reduce negative sampling by 41.7% while still identifying 96.1% of positive cultures. (Appendix 3 provides additional details on the 4 patients [3.9%] missed with this algorithm). Finally, using a Shapiro score of ≥5 points or PCT > 1 μg/L would permit reduction of blood cultures by 77.8% and enable the identification of 73.1% of positive cultures.

DISCUSSION

In this study that included consecutive medical patients presenting to the ED with suspected infection, we validated 5 clinical scores and biomarkers as potential predictors of bacteremia. The Shapiro score was the only score with good performance characteristics (AUC 0.729) which is in line with findings of a previous validation study.⁴ This score, when used in our patient population, would have helped reduce the number of blood cultures by 29.6% with minimal loss of sensitivity for true positive blood cultures, which is consistent with potential reduction of blood cultures by 27% in the original publication.²³ Given that there were 1083 blood cultures drawn, a 29.6% reduction would result in ∼320 fewer cultures, with a potential cost saving of 46,400 US dollars [USD] (at an estimated cost of 145 USD for 2 sets of blood cultures per patient based on institutional data available at the University Hospital Basel). Additionally, this would reduce the number of false-positive cultures with subsequent investigations.

As for the biomarkers evaluated in this study, PCT proved to be the most reliable predictor of blood culture positivity, which is in line with previous research.¹¹ Depending on the cut-off applied, PCT levels of < 0.1 μg/L identified patients with a low enough risk for bacteremia that unnecessary blood culture sampling could be avoided in these patients, resulting in significant financial benefits. With a PCT cut-off of > 0.1 μg/L, only 1 patient (1.0%) with positive blood cultures was missed whereas 24.4% of false negative blood cultures could have been avoided. Limiting blood cultures only to patients with increased PCT levels (ie, > 0.1 μg/L or > 0.25 μg/L) would likely result in significant cost benefits even when considering assay costs of ∼25 USD per PCT measurement.

Our study evaluating PCT in a broad and unselected ED patient population is novel since most other studies evaluated the accuracy of PCT to predict bacteremia in particular patient populations with specified focal infections such as pneumonia, urinary tract infections, or in patients with disseminated infections such as sepsis.^9–16 To our knowledge, only 1 previous study has evaluated PCT in such a broad ED population with suspected infection.²⁹ This study, which included an unselected patient population with suspected bloodstream infections, revealed that PCT levels were significantly elevated in patients with positive blood cultures and that PCT levels were significantly correlated with survival in patients with bacteremia. In our analysis, PCT was found to improve the predictive capabilities of the Shapiro score. To our knowledge, this is the first study to investigate the prognostic capacity of PCT in combination with the Shapiro score.

Our results are also in line with a previous ED-based study in Japan looking at the predictive value of different inflammation markers (PCT, CRP, WBC, platelets) to predict culture positivity.³⁰ In this study using primary component analysis, PCT and platelets were found to be more helpful compared to CRP and WBC. Also, another recent study comparing the prognostic accuracy of PCT, CRP, and WBC in 513 patients presenting to the ED with signs/symptoms of local infections or sepsis found PCT to be the most accurate biomarker for diagnosis of sepsis and for mortality prediction.³¹

There exist no clear guidelines as to when blood cultures should be drawn.¹ Because of the low yield of true positive results, routine sampling of blood cultures has been questioned.^1,3 On the other hand, increasing antibiotic resistance calls for rapid bacterial identification and resistance testing to optimize treatment.³² There is thus an unmet need for accurate predictors of bacteremia. A higher pretest probability for true positive results would enable us to define a specific group of patients who would clearly benefit from blood cultures being drawn. Multiple clinical scores and biomarkers have been proposed to address this need. In this context, PCT and the NLCR have been shown to be the most promising ones.^18,33

Although the vast majority of patients with positive blood cultures had PCT levels > 0.25 μg/L, 1 patient with a PCT level < 0.1 μg/L and 16 other patients with a PCT level < 0.25 μg/L also had positive, noncontaminated blood cultures. Appendix 3 provides additional details on the 17 patients with positive blood cultures but low PCT levels. Out of these, 5 patients [29.4%] suffered from endocarditis. Remarkably, 5 of 9 [55.6%] endocarditis cases with true positive blood cultures had relatively low PCT levels (0.15 μg/L to 0.24 μg/L). This finding that PCT is not routinely required to rule in or rule out infective endocarditis is in line with results of other studies.^34–36 Another 8 of the 17 patients (47.1%) presented to the ED within < 24 h of symptom onset, and it is likely that PCT levels may not have reached their peak at this point in time. This hypothesis, however, needs to be evaluated in another prospective study.

In routine clinical practice, blood cultures are often drawn in response to fever.¹ In our study population, however, 30 of the 104 patients with true positive blood cultures (28.8%) had temperatures < 38.0°C and 49 patients (47.1%) had temperatures < 38.5°C. This finding illustrates the limitations with regard to sensitivity and specificity of a single clinical parameter when used in isolation for decision making.

Our study has some limitations. First, the study was conducted at a single center, and the findings may not be readily applicable to other patient groups with different demographic characteristics. Second, due to the lack of clear guidelines concerning indications for obtaining blood cultures, some patients with fever and infections may not have had blood culture sampling on admission to the ED and are therefore not included in our study population. The decision to obtain the culture was left to the ED physicians’ clinical judgment or could even be made by the nursing staff with or without consultation with the clinician. Third, a potential limitation for widespread implementation of the Shapiro score combined with PCT levels > 0.1 μg/L is the large number of predictive factors included in the Shapiro score, which makes it complex and difficult to remember. However, the predictors themselves are routinely measured and available soon after gathering the medical history and physical examination in the ED setting. Determining PCT levels takes longer, however, with the need for point-of-care tests. Despite these limitations, the results of our study can be applied to a broad spectrum of internal medicine patients as the study was conducted in a relatively large number of patients.

Our study warrants future work in this area. A multicenter study to include a greater number of patients may help corroborate the findings of this study and reveal interesting new insights. Replication of this study in other facilities with patient populations having different demographic characteristics may also be informative. A study focusing on the application of our methodology to infections of specific body sites may be worthwhile, given that urinary tract infections and intra-abdominal infections were seen to have a higher risk for bacteremia (Table 1).

CONCLUSIONS

In conclusion, although the Shapiro score is a useful clinical score on its own, combination of the Shapiro score with admission levels of PCT allows clinicians to abstain from ordering a significant number of potentially useless blood cultures, resulting in significant reductions in costs and false-positive results.

Based on the results of this study, a rational approach to blood culture collection may be as follows: for high-risk patients (eg, suspicion of endocarditis, immunosuppressed patients) blood cultures should be collected when ≥4 Shapiro criteria are fulfilled or when PCT levels are > 0.1 μg/L because the risk of false negative is minimal (0% in our study); for all other patients blood cultures should be collected when ≥3 Shapiro criteria are fulfilled or when PCT levels are > 0.25 μg/L as this cut-off reduces blood cultures by >40% with still a low risk of missing a positive culture (3.9%). In accordance with Shapiro and colleagues, we emphasize that careful clinical judgment must be used when applying a general clinical score to an individual patient and that the score should be overridden in instances in which specific circumstances and complex clinical conditions are present.³