Metabolomics and ROC Analysis: A Promising Approach for Sepsis Diagnosis

Sepsis is a major cause of death in the United States and worldwide. Each year approximately 1,000,000 Americans suffer sepsis, with mortality rates of 4-10% in pediatric patients and 28-50% in adults (1,2). Timely diagnosis and initiation of treatments may improve survival of these patients; conversely, each hour delay in treatment of severe sepsis imposes marked increases in mortality (3), and administration of inappropriate antimicrobial treatment lowers survival of septic shock patients severalfold (3,4). Unfortunately, the clinical features of sepsis closely resemble those of non-infectious systemic inflammatory response syndrome (SIRS). Conventional blood and urine culture sepsis tests require 12 hours or more, and the sensitivity of these tests may be poor, resulting in high false negative rates (5). Traditional biomarkers, e.g. procalcitonin and C-reactive protein, do not sufficiently discriminate between sepsis and SIRS (6). Thus, the identification of more specific, sensitive, reliable and rapidly measured biomarkers to differentiate sepsis from SIRS and monitor disease progression and treatment efficacy is a matter of intense interest (2,7). Recent efforts have used metabolomic analyses of various classes of blood metabolites in search of unique biomarkers that can reliably – and in a timely manner – distinguish with acceptable accuracy between sepsis and SIRS and/or non-infectious, non-inflammatory cases.

Neugebauer et al. (8) examined the use of rapidly obtainable serum lipid metabolite profiles for differential diagnosis of non-infectious SIRS vs. sepsis. A panel of more than 180 serum metabolites were measured by high-throughput mass spectrometry techniques which yielded data within minutes, obviously a major improvement over blood and urine cultures. Statistically significant differences for 26 of these metabolites were identified in the sera of sepsis vs. SIRS patients. Receiver operating characteristic (ROC) analyses were applied to assess whether these metabolite concentrations could (a) differentiate SIRS from sepsis, (b) distinguish between different etiologies of sepsis, and (c) provide prognostic information on the clinical course of these diseases. This commentary limits its focus on issues concerning technical aspects, results, and potential clinical utility of ROC analyses, combined with mass spectrometry, in attempts to provide acceptable ROC estimates to differentiate sepsis from SIRS.

A computer-generated ROC diagram plots the true positive rate (sensitivity) of a test on the y-axis against the false positive rate (100-specificity) on the x-axis, yielding the ROC area under the curve (AUC). Of interest in the clinical context are the AUCs as well as the estimated false positive (100-specificity) and estimated false negative (100-sensitivity) rates (9,10). An AUC is a measure of the accuracy of a diagnostic test: a value of 1.0 indicates a perfect test (due to absence of overlap of the test data from the control and diseased states), whereas an AUC value of 0.5 shows the test is no better than random chance, and therefore has no diagnostic or prognostic value. Another precaution is that AUCs from small samples, because they are inherently noisy, should be interpreted with extreme caution (11). While the ROC analysis offers the unique possibility of adjusting the cut-off value for optimizing diagnostic strategies, it has specific limitations (10-12) like other statistic modeling techniques, and is probably best be used as an adjunct analysis instrument from the arsenal of tools available to the practicing clinician.

From the data in Figure 2D (8), the authors applied ROC/AUC analysis to two selected lipid metabolites, the glycerolipid lysoPCaC24:0 and the sphingolipid SMC22:3, revealed by logistic regression after Bonferroni correction to be significantly different in SIRS vs. sepsis patients. Both of these metabolites were present at higher concentrations in the sera of SIRS vs. sepsis patients. Neugebauer et al. report (8) that the combination of these two metabolites yielded a substantially higher AUC (≈0.90) than the conventional (7) biomarkers procalcitonin (≈0.62), C-reactive protein (≈0.74) or interleukin-6 (≈0.58), and even the combination of all 3 conventional biomarkers (≈0.75). Thus, the combination of the 2 selected lipid metabolites revealed more accurate diagnostic and better predictive estimates than the conventional biomarkers (7). It therefore appears that the high-throughput, rapid analyses of serum lipid metabolites afforded a relatively robust differentiation of sepsis from non-infectious SIRS.

A few concerns point toward additional investigation to refine the metabolomic approach. The selected combination of lysoPCaC24:0 + SMC22:3 yielded a notable sensitivity of ≈84% and specificity of ≈86%. Yet these results indicate, in terms of accuracy of the test (10), an estimated false positive rate (100-specificity; SIRS patients misidentified as septic) of ≈14% and an estimated false negative rate (100-sensitivity; sepsis patients misidentified as SIRS) of ≈16%. In the clinical setting the test incorrectly rejects, at the relatively high rate of ≈16%, sepsis in patients who are instead regarded as SIRS cases; similarly, the relatively high false positive rate of 14% means that the test incorrectly diagnoses, at an estimated rate of ≈14%, SIRS patients with sepsis.

Other recent studies (13,14) report on single lipid mediators of inflammation for bacteremia based on the chemical classes of sphingolipids, acylcarnitines, glycero-phosphocholines and fatty acid esters; a number of compounds from these classes were able to distinguish bacteremia patients from non-bacteremia controls yielding ROC-estimated AUCs between ≈0.82 (for fatty acid esters) to ≈0.96 (for acylcarnitines). ROC curve sensitivities ranged from ≈76% to ≈84% and specificities from ≈75% to ≈98%, again implying appreciable estimated false positive and estimated false negative rates by some of these lipid biomarkers. On the other hand, the fact that both Neugebauer et al. (8) and To et al. (14) report relatively high ROC AUCs between ≈0.8 and > ≈0.9 for certain serum lipid metabolites encourages further research in the search for the most definitive and unique lipid biomarkers for bacteremia and sepsis diagnostics.

Another concern is that Neugebauer et al. (8) collected the serum samples over a nearly ten-year span, between September 2002 and January 2012, but the period during which the metabolite analyses were performed is not indicated and presumably is more recent than the sampling. This study design raises questions regarding the chemical stabilities under those conditions of the many analytes. Were all of the analytes stable while frozen, and uniformly so? Did analytes degrade upon thawing the samples for analysis? Were the sampling techniques and compositions of samples collected in 2002-03 comparable to those collected in 2011-12 among patients with the same diagnosis? Also, do the metabolomic analyses provide insights regarding the underlying pathology?

It seems conceivable that serum concentration ratios of metabolites that change in opposite directions in sepsis vs. SIRS or in bacteremia patients vs. non-bacteremia cases might be more powerful prognostic and diagnostic instruments than combining metabolite concentrations that shift in the same direction. It is well recognized that metabolite ratios are more informative mechanistically than single metabolite concentrations: for example HDL-cholesterol status by [cholesterol]/[HDL], cellular energetics by [phosphocreatine]/[creatine], redox state in serum by [lactate]/[pyruvate] and cellular antioxidant status by [GSH]/[GSSG] ratios. Could concentration ratios of certain serum lipid metabolites actually prove more powerful diagnostic and prognostic estimators than total single or combined metabolite concentrations? Furthermore, would ROC analyses of test results for sepsis or bacteremia, when based on metabolite ratios, significantly reduce the clinically important estimates of false positive rates or false negative rates? In this regard, Figure 2D shows the amino acids phenylalanine and serine are elevated in the serum of sepsis vs. non-infectious SIRS patients, and it might be possible that [lysoPCaC24:0 + SMC22:3]/[phenylalanine + serine] ratios would differ more significantly in SIRS vs. sepsis patients than [lysoPCaC24:0 + SMC22:3] alone, and, thereby, afford more robust estimates for differentiation of SIRS from sepsis.