Biomarker discovery and development in pediatric critical care medicine

Abstract

Objective

To frame the general process of biomarker discovery and development, and to describe a proposal for the development of a multi-biomarker based risk model for pediatric septic shock.

Data Source

Narrative literature review and author generated data.

Main Results

Biomarkers can be grouped into four broad classes, based on the intended function: diagnostic, monitoring, surrogate, and stratification. Biomarker discovery and development requires a rigorous process, which is frequently not well followed in the critical care medicine literature. Very few biomarkers have successfully transitioned from the candidate stage to the true biomarker stage. There is great interest in developing diagnostic and stratification biomarkers for sepsis. Procalcitonin is currently the most promising diagnostic biomarker for sepsis. Recent evidence suggests that interleukin-8 can be used to stratify children with septic shock having a high likelihood of survival with standard care. Currently, there is a multi-institutional effort to develop a multi-biomarker based sepsis risk model intended to predict outcome and illness severity for individual children with septic shock.

Conclusions

Biomarker discovery and development is an important portion of the pediatric critical care medicine translational research agenda. This effort will require collaboration across multiple institutions and investigators. Rigorous conduct of biomarker-focused research holds the promise of transforming our ability to care for individual patients and our ability to conduct clinical trials in a more effective manner.

INTRODUCTION

In 2001 the National Institutes of Health convened a panel to develop standard definitions and a conceptual framework for biomarker discovery and development [1]. Based on this consensus conference a biomarker is broadly defined as a characteristic that can be objectively measured and evaluated as an indicator of normal biological processes, pathological processes, or pharmacological responses to a therapeutic intervention. Further, biomarkers can be subdivided into “Type 0” and “Type 1” biomarkers. A Type 0 biomarker provides a signal of the natural history of a disease that correlates longitudinally with known clinical indices and outcomes. A Type 1 biomarker captures the effects of a therapeutic intervention in accordance with its mechanism of action.

Based on this broad definition it is evident that we use biomarkers daily in the pediatric intensive care unit (PICU) without acknowledging them as such. For example, we routinely measure serum creatinine as an indicator of kidney function or evolving kidney injury (a Type 0 biomarker). We also routinely measure serum glucose levels to monitor the biological effects of insulin therapy (a Type 1 biomarker). Other examples of everyday biomarkers used in the PICU include white blood cell counts in cerebral spinal fluid, urinary protein, and intravascular pressure measurements. However, the typical intensivist thinks of a “biomarker” as a novel “blood test” that provides us with patient information that is not readily obtainable using our current diagnostic and monitoring tools. This relatively narrow, but nonetheless highly relevant concept of biomarkers will provide the overall framework for this review. We will first provide a framework for the general discovery and development of biomarkers. Subsequently, we will illustrate several of the key points relating to biomarker discovery and development by describing a recent proposal for the derivation a multi-biomarker based risk model for pediatric septic shock.

BIOMARKER APPLICATION

Table 1 lists four broad functional classes of biomarkers as proposed in recent literature [2, 3]. Diagnostic biomarkers serve to establish the presence or absence of a disease or other clinical conditions. The majority of diagnostic biomarkers are of most value within a specific clinical context and with the addition of adjunctive diagnostic tests. For example, myocardial enzymes (i.e. troponin, creatine phosphokinase) are most useful for accurately diagnosing myocardial infarction when there is well-founded clinical suspicion of myocardial infarction (context) and with the addition of other tests (e.g. electrocardiogram). Some diagnostic biomarkers are of most value for screening purposes. A screening biomarker, which is sub-type of a diagnostic biomarker, is intended to provide a simple, low cost, and low risk approach to check for the possibility of a disease in a population (e.g. heel sticks for metabolic disease screening in newborns, or stool guiac testing for colon cancer screening in adults), rather than to firmly establish a given diagnosis. Ideal screening biomarkers have high sensitivity, without the need for high specificity. Therefore, a positive result derived from a screening biomarker typically requires follow-up with more definitive (i.e. specific) and often more complex diagnostic approaches. Conversely, ideal diagnostic biomarkers should have a high specificity, while the sensitivity becomes relatively less important. Sensitivity, specificity, and related concepts will be discussed further in a subsequent section.

Table 1.

Classes of biomarkers.

Biomarker Class	Potential Clinical Use
Diagnostic	To establish the presence or absence of a disease or clinical condition. Screening biomarkers can be considered as a diagnostic biomarker subtype.
Monitoring	To provide a readout that informs the clinician of the effectiveness of a given therapy for the purpose of titration.
Surrogate	To provide a readout that informs the clinician of the effectiveness of a given therapy for the purpose predicting clinical outcome.
Stratification	To sub-classify or stage diseases in terms of severity and/or outcome.

Another important concept regarding diagnostic biomarkers, particularly in the field of critical care medicine, involves disease severity and the therapeutic options available for a given disease. The clinical significance of a diagnostic biomarker is directly proportional to disease severity and the therapeutic options for that disease. Thus, the most clinically important biomarkers in critical care medicine, and consequently what should be the focus of our translational research agenda, are those that diagnose or indicate risk for a severe illness or outcome for which there is an effective therapy.

Monitoring biomarkers serve to provide a “readout” that informs the clinician of the effectiveness of a given therapy for the purpose of titration. Robust monitoring biomarkers must therefore be functionally or biologically linked to the mechanism of action of a given therapeutic intervention. As previously mentioned, glucose monitoring in the context of insulin therapy is a relevant example of a monitoring biomarker routinely used in the PICU. Unfortunately, most of the monitoring biomarkers that we use in the PICU only provide indirect readouts of therapeutic efficacy, and consequently our titration capabilities are limited for many important therapies. For example, blood tacrolimus (FK-506) levels are routinely followed in solid organ transplant patients admitted to the PICU. The purpose of tacrolimus monitoring is two-fold: to minimize toxicity and to assess the adequacy of immune suppression. The latter purpose, which is singularly important in the management of these patients, is only achieved indirectly. We infer “adequacy of immune suppression” based on a range of blood tacrolimus levels, without actually knowing the true state of immune activity/suppression. Thus, the field of pediatric critical care medicine needs further development of monitoring biomarkers that efficiently and feasibly measure bioactivity directly linked to therapeutic interventions.

Surrogate biomarkers serve a similar general function to that of monitoring biomarkers (i.e. they provide a readout of efficacy), but are intended to predict clinical outcome, rather than for titration of therapy [2, 3]. As the name implies, surrogate biomarkers serve as proxy or substitute endpoints for patient-centered outcomes such as mortality or serious morbidities. The use of surrogate biomarkers has the theoretical advantage of lowering the number of subjects required to conduct a trial having sufficient power to detect efficacy. Robust surrogate biomarkers need to be directly affected by a given therapy, reliably predict the outcome of interest, and be mechanistically linked to the biological processes that affect the outcome of interest. The cardiology field has effectively used surrogate biomarkers (e.g. lipid profiles, c-reactive protein) in the conduct of clinical trials [4–6], but the development of surrogate biomarkers in the field of critical care medicine is a daunting challenge, which may not be completely feasible. Nonetheless, the development of effective surrogate biomarkers is required in the field of critical care medicine in order to better conduct Phase 1 and 2 clinical trials.

Stratification biomarkers (a.k.a. staging biomarkers) serve to sub-classify or stage diseases based on severity and/or outcome. Biomarker-based stratification can be applied to individual patients or in the context of a clinical trial. In the case of an individual patient, an effective stratification biomarker can identify higher risk patients that would be eligible for more toxic, more aggressive, or higher risk therapies (e.g. ECMO). Alternatively, an effective stratification biomarker can indentify patients that are likely to have good outcomes with standard therapies, and thereby can objectively exclude patients from being exposed to unnecessary therapies. In either case, the general goal is to optimize the risk to benefit ratio of a given therapy. The oncology field is replete with excellent examples of effective therapies being predicated on data garnered from robust stratification biomarkers [6–9].

The field of critical care medicine would benefit tremendously from the development of effective biomarker-based stratification strategies. As intensivists we generally deal with syndromes rather than true “diseases” (e.g. septic shock and acute respiratory distress syndrome). Accordingly, the “diseases” that we encounter daily in the PICU are highly heterogeneous from both a clinical and biological standpoint. Relative to this high level of heterogeneity, however, our therapeutic approaches are highly homogenous. We generally apply the same therapeutic approaches to heterogeneous syndromes such as septic shock. We also conduct interventional clinical trials in the PICU without directly addressing the heterogeneous nature of the study cohort. Not surprisingly, the critical care medicine literature is replete with interventional clinical trials that have failed to demonstrate efficacy despite being based on sound preclinical data [10]. A major reason for these multiple failures is the inability to effectively manage and account for the profound heterogeneity of our patient populations, rather than fundamentally flawed biological principles. Thus, the development of robust stratification strategies is another important component of the critical care medicine research agenda, and biomarker-based strategies have great potential to directly address this agenda.

CANDIDATE BIOMARKER DISCOVERY

The biomarker field has become a highly competitive endeavor in the biomedical and pharmaceutical industries. The first step in biomarker development is discovery of “candidate” biomarkers [11]. As the name implies, a candidate biomarker has the potential to serve one of the aforementioned functions in the context of a given disease/syndrome, but remains to be formally tested and validated before being legitimately classified as a biomarker. The path from candidate biomarker status to actual biomarker status is very challenging and very few candidate biomarkers achieve actual biomarker status.

There are two broad approaches to candidate biomarker discovery: knowledge-based approaches and unbiased approaches. The knowledge-based approach to biomarker discovery embodies the traditional scientific method (i.e. hypothesis-driven) and has a long-standing history in all medical fields, including critical care medicine. In this approach, the investigator has a strong working knowledge of the pathophysiology of a given disease and related biological data. Based on this knowledge the investigator derives a hypothesis linking the candidate biomarker and the disease of interest, and subsequently designs a study to test the hypothesis. The critical care medicine literature is replete with studies in which “protein/molecule/metabolite X” (e.g. a cytokine) is demonstrated to be increased (or decreased) in the blood (or other biological fluid) of patients with “disease Y” (e.g. sepsis) [12–17]. While these studies are of value for initial identification of candidate biomarkers, the majority fall short of establishing a more definitive, clinically useful link between the candidate biomarker and the disease of interest. Simply reporting a correlation between a candidate biomarker level and a disease or disease outcome, is merely the first step in biomarker discovery.

The advantages of the knowledge-based approach are that it is grounded in the traditional scientific method and tends to be focused. This type of focused, hypothesis-driven approach is generally favorably regarded by review panels of funding agencies. The main disadvantage of the approach is that it is potentially biased, and is limited by the current state of knowledge in the field.

Unbiased approaches take a large-scale approach to candidate biomarker discovery without making few, if any, a priori decisions or assumptions regarding the potential candidates. This type of approach leverages the high throughput technologies related to the fields of proteomics, transcriptomics, and metabolomics, which allow for the simultaneous, semi-quantitative measurements of thousands of proteins, gene transcripts, or metabolites, respectively, in biological samples. Accordingly, these platforms provide unprecedented opportunities to compare differential expression of proteins, gene transcripts, or metabolites between biological samples representing disease states, gradations of disease states, and normal states. Human blood, a primary target for biomarker discovery and development, has been described as a highly comprehensive and readily accessible proteome potentially providing a representation of all body tissues during health and disease [18].

As the name implies, the primary advantage of these large-scale approaches is that they are relatively unbiased in that no particular protein, transcript, or metabolite becomes the focus of investigation. In other words, all proteins, genes, or metabolites “have a chance” to be discovered as candidate biomarkers through the development of comprehensive bio-molecular signatures of disease states. This type of approach is particularly well-suited for the discovery phase of biomarker development, rather than the latter stages of derivation and validation. The main disadvantages of these approaches are that they are costly, can have limited detection capabilities, and that the large data sets present unprecedented analytical challenges requiring complex bioinformatics prone to generation of false positives and false negatives. In addition, some funding agency review panels view these approaches as non-hypothesis driven “fishing expeditions.”

Another important component of biomarker discovery is the biological material one chooses to use for analysis. Since the biomarker field is, in large part, focused on the development of “blood tests,” blood is a logical source for the discovery phase of candidate biomarkers. However, other biological materials (“proximal fluids”) may be more appropriate in the discovery phase for several reasons including, proximity to the diseased tissue of interest, enrichment for disease-specific candidate biomarkers, and less complex backgrounds [11]. Examples of human proximal fluids include urine, cerebral spinal fluid, ascites, and bronchoalveolar lavage fluid. Proximal fluids can also be derived from appropriate animal models and cell culture models during the initial discovery phase.

Neutrophil gelatinase-associated lipocalin (NGAL) is now recognized as a robust biomarker for acute kidney injury (AKI) in critically ill patients, including children [19–21]. The discovery of NGAL as a candidate biomarker for AKI well illustrates the above concepts of analyzing a proximal fluid (urine) derived from an animal model, and using an unbiased approach (transcriptomics and proteomics). NGAL was initially identified as candidate AKI biomarker in rodent models of kidney ischemia [22, 23]. Analyses of the kidney parenchymal transcriptome and the urine proteome demonstrated that NGAL was one of the most abundant genes expressed in rodents subjected to experimental renal ischemia. The use of kidney tissue and urine as the biological materials was a key component of the discovery phase in that they directly represent or are in close proximity to the tissue of interest (i.e. the kidney), are therefore likely to be enriched for kidney-specific candidate biomarkers, and the urine proteome is several orders of magnitude less complex than the blood proteome. Thus, an unbiased approach based on biological samples from an experimental animal model allowed the discovery of a candidate biomarker (i.e. NGAL) that may have not been readily evident using a knowledge-based approach.

BIOMARKER DERIVATION AND VALIDATION

After the candidate biomarker discovery phase, the next phase of biomarker development involves derivation and validation. The derivation phase involves the establishment of “cut-off” values or ranges at which the biomarker will provide clinically useful information. For example, in the case of a diagnostic biomarker a cut-off value allows one to determine whether or not the disease of interest is present (i.e. sepsis). In the case of a stratification biomarker, a cut-off value allows one to predict a particular outcome (i.e. mortality).

The derivation phase should typically involve a relatively large cohort of study subjects with and without the disease or outcome of interest. Other commonly used terms for derivation cohorts are “training” cohorts and “learning” cohorts. After biomarker cut-off values/ranges are established in the derivation cohort, the performance of these cut-off values/ranges needs to be formally tested in a validation cohort of patients. The ideal validation cohort is also large and completely independent of the original derivation cohort. Unfortunately, a substantial proportion of the critical care medicine literature involving candidate biomarkers does not effectively transition from initial derivation to subsequent formal validation.

The derivation and validation phases of biomarker development serve to determine the validity of the biomarker in providing the correct answer of interest. To determine validity there needs to be an established method or readout to indicate the correct answer (i.e. a gold standard). Some diseases, but not all, have a diagnostic gold standard. For example, isolation of a known pathogen in cerebral spinal fluid is the diagnostic gold standard for meningitis. Many studies in sepsis use a positive blood culture as the gold standard, however false negative culture results may occur from sampling error and false positive results may occur from contamination. For a particular disease the gold standard may change over time based on development of newer diagnostic methods. For example, to diagnose a pulmonary embolism the gold standard test historically was pulmonary angiography. This test has now been replaced by helical computed tomography pulmonary angiography. Therefore tests of validity, including sensitivity and specificity, must be interpreted in the context of the clinical situation and the available gold standards.

In the following paragraphs we will outline common approaches and concepts for determining biomarker validity, also known as test performance characteristics. The approaches described primarily apply to single biomarker strategies. More complex approaches, such as principle component analysis and multivariable logistic regression, are required for multi-biomarker-based approaches.

Sensitivity and Specificity

Sensitivity and specificity calculations provide information regarding intrinsic test validity and are not dependent on the prevalence of a disease or condition. As such, specificity and sensitivity are said to be “fixed” for a given test. In order to calculate sensitivity and specificity data must be presented as dichotomous outcomes. For analysis of survival data this is straightforward (dead vs. not dead). For diagnostic testing, however, a cutoff value is necessary to distinguish the two groups (e.g. glucose levels ≥ 200 mg/dl vs. glucose levels < 200 mg/dl), and decisions involving cut-off values directly affect sensitivity and specificity calculations. Sensitivity and specificity testing involve administering the test to two groups: patients with the disease and another group without the disease. Once the testing is performed the results can be placed in a 2×2 contingency table and measures of test accuracy can be performed (Figure 1). Sensitivity is defined as the proportion of subjects with the disease in whom the test gives a positive result (e.g., patients with bacteremia who have a positive blood culture), whereas specificity is the proportion of subjects without the disease in whom the test gives a negative result (e.g., patients without bacteremia who have a negative blood culture). A highly sensitive test has a low false-negative rate and a highly specific test has a low false-positive rate. Therefore, tests that have a high sensitivity are often used as screening tests because they are helpful for ruling out a diagnosis in patients when the test is negative. Tests with high specificity are used to rule in a disease where a positive test indicates a high probability that the patient has the disease.

Figure 1.

Two-by-two contingency table illustrating calculations for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Positive and negative predictive values

Sensitivity and specificity define the operating characteristics of a given test. Most of the time, however, a clinician’s interest lies in knowing what to do with a test result. Herein lays the utility of positive predictive values (PPV) and negative predictive values (NPV). Predictive values require an index of suspicion of the disease and are affected by the prevalence of the disease in the population. A PPV assesses the validity of a positive test, and puts test specificity in the context of disease prevalence. A high disease prevalence usually results in high PPV, whereas low disease prevalence usually results in low PPV. NPV assesses the validity of a negative test, and puts test sensitivity in the context of disease prevalence. Low disease prevalence usually results in high NPV.

PPV and NPV can also be calculated using a 2×2 table (Figure 1). PPV reflects the proportion of time that a patient with a positive diagnostic test result has the disease (e.g. the probability of bacteremia, given a positive blood culture). NPV reflects the proportion of time that a patient with a negative diagnostic test result does not have the disease (e.g. the probability of not having bacteremia, given a negative blood culture).

Likelihood Ratios

The likelihood ratio (LR) is based on odds rather than probabilities, simultaneously takes sensitivity and specificity into account, and is not dependent on prevalence. A positive LR (+LR) expresses the odds that a positive test result will be observed in patients with the disease versus the odds that the same result will be observed in patients without the disease: sensitivity/(1-specificity). A negative LR (−LR) expresses the odds that a negative test result will be observed in patients with the disease versus the odds that the same result will be observed in patients without the disease: (1-sensitivity)/specificity. An LR of 1 indicates that the test result is just as likely to occur in patients with the disease as in patients without the disease. Clinically useful +LRs are typically >10, whereas clinically useful −LRs are typically <0.1.

Receiver Operating Characteristic Curves

When tests are measured on a continuum, sensitivity and specificity levels depend on where the cutoff is set between positive and negative results. The relationship between sensitivity and specificity can be graphically displayed using a receiver operating characteristic (ROC) curve (Figure 2). Cut-off points are selected and the sensitivity and specificity are determined at each point. The resulting graph depicts the Y-axis as the true-positive rate (sensitivity) as a function of the false-positive rate (1-specificity) on the x-axis. The proximity of the curve to the upper left-hand corner of the graph corresponds to increasing accuracy (i.e. the true positive rate is closer to 1 and the false-positive rate is closer to 0). The overall accuracy of a test can be demonstrated as the area under the ROC curve (AUC). An AUC of 0.5 indicates that the test is positive or negative by chance alone, whereas the “perfect” test would yield an AUC of 1.

Figure 2.

Hypothetical receiver operating characteristic (ROC) curve. The straight diagonal line represents an ROC curve having an area under the curve (AUC) of 0.5, in which case the test is positive or negative by chance alone. In contrast, the hypothetical ROC curve has an AUC of 0.857.

Confidence Intervals

Confidence intervals refer to a range of values about the sample mean or proportion. Confidence intervals reflect the range of values in which the means of different samples drawn from a given population are likely to fall. It is a measure of precision; therefore a narrow confidence interval is more precise than a larger confidence interval. Commonly reported confidence intervals are 95% and 99%. A 95% confidence interval can be interpreted to indicate that 95% of the sample means drawn from a population will fall within the reported interval. Therefore, narrow confidence intervals provide data with a higher level of validity. Large patient cohorts are more likely to yield narrow confidence intervals, whereas small patient cohorts are more likely to yield wide confidence intervals. Confidence intervals should ideally be reported for all of the indices discussed above, including LRs. For ROC curves, interpretation of an AUC with a confidence interval including 0.5 would suggest that the test could be positive or negative by random chance.

SEPSIS-RELATED BIOMARKERS

Since sepsis and septic shock are such important conditions to the daily clinical practice of critical care medicine, there has been a great deal of interest in developing sepsis-related biomarkers. All four classes of biomarkers (diagnostic, monitoring, surrogate, and stratification) have potential applications in sepsis. For example, a monitoring biomarker (e.g. lactate) could assist in titrating cardiovascular support. Alternatively, the development of surrogate biomarkers for sepsis could potentially be a major advancement in the conduct of clinical trials, which currently require very large sample sizes to detect a survival benefit. The most well developed areas thus far, however, relate to diagnostic and stratification biomarkers for sepsis. Accordingly, in the following section we will discuss diagnostic and stratification biomarkers for sepsis in greater detail. The discussion will also help to highlight many of the key concepts described in previous sections.

DIAGNOSTIC BIOMARKERS FOR SEPSIS

Procalcitonin

A daily conundrum in the ICU is reliably distinguishing which patients meeting criteria for systemic inflammatory response syndrome (SIRS) are infected and which are not infected. Accordingly, there have been extensive efforts to develop diagnostic biomarkers for sepsis/infection in critically ill patients. Procalcitonin (PCT) has emerged as the most promising of these diagnostic biomarkers.

PCT is a 116-amino-acid peptide and a precursor of calcitonin [24, 25]. Low levels of PCT circulate in the serum, but are substantially increased during systemic infections [26]. PCT is normally secreted from the thyroid parafollicular or clear (C) cells, however in sepsis PCT can originate from extra-thyroidal sites including liver, spleen, and adipose tissue [27, 28]. PCT can be detected 2–3 hours after an injection of endotoxin in normal human volunteers [29].

In critically ill patients PCT has consistently demonstrated greater accuracy than C-reactive protein (CRP) in determining sepsis, but demonstrates a wide range of sensitivity and specificity across individual studies [30–34]. Trauma patients who developed sepsis had higher PCT levels on admission compared with trauma patients who did not develop sepsis [35]. Using a cutoff value of 1.09 ng/ml and positive blood cultures as the gold standard, PCT had a high sensitivity (100%) and negative predictive value (100%) for diagnosing sepsis, but low specificity and positive predictive value, and no mortality was associated with “missed” infections. In a meta-analysis focused on adults admitted to intensive care units after surgery or trauma, PCT levels in infected patients were about 16 times higher than in patients without infection [31]. However, a separate meta-analysis including a more generalized population of patients in intensive care units, emergency departments (ED), or hospital wards concluded that PCT cannot accurately distinguish sepsis from SIRS [36].

PCT may have value in diagnosing and staging sepsis in the ED. In ED patients, fever, chills, and PCT levels were the strongest predictors for the diagnosis of bacterial infection [37]. Using a rapid semi-quantitative method to determine PCT levels Oh et al. demonstrated that in patients with suspected infection, who presented to the ED and met SIRS criteria, a cutoff value of >2 ng/ml was useful for distinguishing between SIRS/sepsis and severe sepsis/septic shock [37].

In the pediatric population PCT also proves to be a better diagnostic marker for sepsis than CRP [33, 38, 39]. In ROC curve analysis, PCT was better than CRP or the immature to total neutrophil ratio in diagnosing sepsis [32]. In infants < 3 months of age with fever of unknown origin, PCT has a higher diagnostic value than CRP or leukocyte count in predicting serious bacterial infections [40]. Additionally, Han et al. demonstrated that PCT levels were significantly increased in patients with bacterial sepsis compared to controls, but PCT levels were not significantly increased in patients with viral, fungal, or culture negative sepsis compared with controls [41].

The most important clinical applicability of PCT as a biomarker may be in determining the initiation (diagnostic biomarker) and duration (monitoring biomarker) of antibiotic use. In a randomized control trial patients were divided into a PCT-guided treatment plan or a standard antibiotic treatment plan [42]. In the PCT group, antibiotics were discontinued if clinical signs and symptoms of the infection had improved and if the PCT level decreased. The duration of antibiotic treatment in the PCT-guided group was 5.9 ± 1.7 days shorter than in the control group (p < 0.001). Further support of the use of PCT in guiding antibiotic therapy was demonstrated in a large, multicenter, randomized trial in patients with lower respiratory tract infections [43]. The use of a PCT-guided treatment algorithm resulted in lower rates of antibiotic exposure and antibiotic-associated adverse effects compared with standard guidelines. Similar findings were recently published in another randomized, PCT-guided algorithm trial [44].

Interleukin-18 (IL-18)

IL-18 is also gaining attention as a potential diagnostic biomarker for sepsis, but the overall data are currently not as robust as that for PCT. IL-18 is a pro-inflammatory cytokine and a member of the IL-1 super family that activates NF-κB and MAP kinases [45]. IL-18 is produced from a precursor protein and is cleaved to a mature form by caspase-1. Increased IL-18 levels are found in many human inflammatory conditions including rheumatoid arthritis, neonatal infections, and sepsis [46–48].

Plasma IL-18 levels were significantly increased in patients with sepsis (588 ± 83 pg/ml) compared to trauma patients (182 ± 13 pg/ml) on day of admission or at time of sepsis diagnosis [49]. In this same cohort, patients with septic shock had higher IL-18 plasma levels compared to patients without shock. IL-18 plasma levels were also elevated in patients who developed sepsis following major visceral surgery compared to patients who did not develop sepsis [50]. Furthermore, non-survivors with sepsis had significantly higher IL-18 levels compared to survivors from sepsis and IL-18 levels continuously increased during sepsis in non-survivors. Other data suggest that IL-18 may allow for distinction between gram positive and gram negative bacterial infections in critically ill patients [45, 51].

In summary, the role of IL-18 as a diagnostic biomarker for sepsis remains to be further developed, and limited data exist in the pediatric population (thus providing a potential area for translational research). In contrast, there is considerably better developed data indicating that IL-18 may be a robust diagnostic biomarker for AKI [52–54].

STRATIFICATION BIOMARKERS FOR SEPTIC SHOCK

Rationale

Septic shock in humans, as opposed to animal models, is a heterogeneous syndrome having highly variable physiological and biological manifestations in a given patient population [10]. This heterogeneity continues to pose a major challenge to the rational conduct of clinical trials focused on septic shock. Indeed, the vast majority of clinical trials targeted at improving the outcome of septic shock have failed despite being founded on quality pre-clinical data. One major reason for these multiple failures appears to be the inability to manage the inherent heterogeneity of clinical septic shock in the context of a clinical trial. Accordingly, it has been proposed by Marshall that a key challenge in the field is to reduce and manage this heterogeneity by more effectively stratifying patients for the purpose of more effective clinical research [10].

Interleukin-8 (IL-8) as a stratification biomarker in pediatric septic shock

We recently reported that IL-8 can reliably predict survival in critically ill children with septic shock receiving standard care [55]. The identification, derivation, and validation of IL-8 as a stratification biomarker illustrates many of the concepts described in the previous sections.

The initial discovery of IL-8 as a candidate stratification biomarker in pediatric septic shock involved a hybrid approach, which was partially unbiased and partially knowledge-based. The unbiased approach consisted of genome-wide analysis of gene expression (transcriptomics) using whole blood-derived RNA obtained within 24 hours of admission to the PICU. These microarray-based studies interrogated over 30,000 genes simultaneously, and revealed that IL-8 was one of the most highly expressed genes in non-survivors of pediatric septic shock compared to survivors [56]. Because IL-8 is a well known chemo-attractant and activator of neutrophils, we hypothesized that serum IL-8 protein levels could be used as a stratification biomarker in pediatric septic shock (knowledge-based approach).

We then measured serum IL-8 protein levels in the same cohort of patients used for the microarray studies (n = 42, training data set) to construct an ROC curve for 28-day mortality, which yielded an AUC of 0.857. From these procedures we derived an IL-8 cut-off level of >220 pg/ml as having a 75% sensitivity and specificity for predicting 28-day mortality in the training data set. We next applied this cut-off value to a validation data set of 139 children with septic shock and constructed a contingency table to calculate the sensitivity, specificity, positive predictive value, and negative predictive value of this IL-8 cut-off for predicting 28-day mortality. The calculated sensitivity, specificity, and positive predictive value were not sufficiently robust for clinical application. However, the negative predictive value for mortality was 95%, with narrow 95% confidence intervals (87 to 98%), and a negative likelihood ratio for mortality of 0.3. In other words, these data suggested that a serum IL-8 level ≤ 220 pg/ml, measured within 24 hours of admission to the PICU, may have the ability to predict survival in children with septic shock with 95% probability. Subsequently, we tested the veracity of this negative predictive value in a second validation data set, completely independent of our own database: children enrolled in the Phase 3 trial of activated protein C (RESOLVE Trial) [57]. Application of the IL-8 cutoff to the RESOLVE database demonstrated an equally robust negative predictive value and negative likelihood ratio for 28-day mortality.

Based on these data, we have proposed that serum IL-8 measurements, conducted within the first 24 hours of admission to the PICU, can be used to identify children with a high probability of survival with standard care (i.e. stratification) [55]. These low risk patients could then be excluded from interventional clinical trials. By excluding patients in this manner, one could improve the risk to benefit ratio of an experimental intervention by generating a study cohort that would be more likely to realize a benefit from the experimental intervention, particularly if the intervention carries more than minimal risk for serious adverse events. Using a similar approach as that described above, we also recently demonstrated that chemokine (C-C motif) ligand 4 (CCL4, a.k.a. MIP-1β) has a 98% negative predictive value for mortality in children with septic shock [58].

A multi-biomarker-based risk model for pediatric septic shock stratification

The strengths of the individual IL-8- and CCL4-based stratification strategies described above are simplicity and very high negative predictive values for mortality. However, both stratification biomarkers have positive predictive values, sensitivities, and specificities that are too low to develop a comprehensive pediatric septic shock stratification tool meeting a wide variety of clinical and research needs [55, 58]. Given the complexity of septic shock, it is possible that a multi-biomarker based stratification system could fulfill all of these needs. Herein we describe our plan to derive the pediatric sepsis biomarker risk model (heretofore called PERSEVERE: PEdiatRic SEpsis biomarkEr Risk modEl) using 15 candidate stratification biomarkers.

The candidate biomarkers for derivation of PERSEVERE were first identified using an unbiased approach that leveraged the discovery potential of a well-annotated microarray data set (transcriptomics) consisting of 98 children with septic shock and representing the first 24 hours of admission to the PICU [59]. We employed two complementary but distinct unbiased approaches to the discovery of candidate biomarker genes. In the first approach we used normal controls, septic shock survivors, and septic shock non-survivors as the comparison groups, and performed a 3 group ANOVA using all gene probes on the array (87,933 gene probes). The ANOVA consisted of a Welch test with correction for multiple comparisons via a Benjamini-Hochberg False Discovery Rate of 5%. This test yielded > 20,000 differentially regulated gene probes between the 3 comparison groups. We then performed a post-hoc Student-Newman-Keuls test to determine the specific inter-group differences in gene expression. This post-hoc test yielded 137 gene probes differentially regulated between septic shock survivors and non-survivors. This 137 candidate biomarker gene list will heretofore be called “gene list A” for ease of reading.

In the second unbiased approach to candidate biomarker discovery, we applied Support Vector Machines-based dichotomous class prediction modeling to identify candidate biomarker genes [60]. Starting with all genes on the array (87,933) we attempted to predict “survivor” and “non-survivor” classes via leave-one-out cross validation procedures. This class prediction modeling approach was able to correctly predict survival or non-survival in 84 of the 98 patients (86% correct class prediction). More specifically, the model correctly predicted 15 of the 17 non-survivors (88%) and 69 of the 81 survivors (85%). Further, we used the Fisher test for gene selection, and extracted the top 5% class predictor gene probes out of the starting 87,933 gene probes (4,397 gene probes). This 4,397 candidate biomarker gene list will heretofore be called “gene list B” for ease of reading.

Having derived candidate biomarker gene lists A and B by two distinct methods, we next conducted Venn analysis to determine which genes are common to the two gene lists. As shown in Figure 3, we found 117 gene probes common to gene lists A and B. To derive a final list of candidate biomarker genes we examined the above 117 gene list for genes meeting 2 simultaneous, a priori criteria (i.e. a knowledge-based approach): 1) the gene has a reasonable level of biological plausibility with regard to the pathobiology of pediatric septic shock, the host response to infection, and/or the host inflammatory response; and 2) the gene product (protein) can be feasibly measured in the blood compartment. Based on these 2 criteria we derived a final working list of 15 candidate biomarker genes as shown in Table 2.

Figure 3.

Venn analysis comparing gene lists A and B. See text for gene list derivations.

Table 2.

Candidate biomarker gene list for derivation of PERSEVERE.

Gene Symbol	Description
CCL3	C-C chemokine ligand 3; a.k.a. MIP-1α
LCN2	Lipocalin 2; a.k.a. NGAL
MMP8	Matrix metallopeptidase 8
RETN	Resistin
THBS	Thrombospondin 1
GZMB	Granzyme B
HSPA1B	Heat shock protein 70kDa 1B
ORM1	Orosomucoid 1, acute phase protein with unknown function
LTF	Lactotransferrin
ELA2	Neutrophil elastase 1
IL1A	Interleukin 1α
SULF2	Sulfatase 2; extracellular modulator of heparan sulfate proteoglycans
FGL2	Fibrinogen-like 2; acute phase protein similar to fibrinogen

We propose that the candidate biomarker list shown in Table 2 provides a robust foundation for derivation of PERSEVERE. Several features of the biomarker list support the assertion of robustness. First, the patient cohort from which the genes were derived represents 11 major pediatric centers in the United States (see acknowledgment section), thus taking into account any potential variability in standard care and thereby speaking to the potential generalizability of the candidate biomarkers. Second, the initial working gene lists were generated in a systematic, objective, and rigorous manner, based on a combined statistical approach and a class prediction approach. Third, the selection process has limited, if any, bias. The selection process began with the entire probe set on the array. Thus, all genes “had a chance” to be selected. The final step in selecting the candidate biomarkers (i.e. a knowledge-based approach) is the only potential source of bias. Finally, the microarray data from which the genes were selected represent the first 24 hours of presentation to the PICU. This early time point is ideal and clinically relevant for the goals of PERSEVERE.

Robust illness severity scores are currently available for critically ill pediatric populations, but it is clear that these scores are not appropriate tools to stratify individual patients for the purposes of clinical trials, nor for making individual patient decisions [61]. PERSEVERE is intended to predict outcome and illness severity for individual children with septic shock. The 15 candidate biomarkers will be quantified in serum samples using a multi-plex platform. PERSEVERE will be derived in a formal derivation cohort of patients using statistical modeling approaches, which integrate the candidate biomarker measurements with principal component analysis and multivariable logistic regression. The final output will be an equation into which an individual patient’s biomarker measurements can be input to derive an individual risk assessment. Subsequently, PERSEVERE will be validated prospectively in a separate validation cohort. If PERSEVERE comes to fruition, we expect that it will provide an unprecedented decision and stratification tool for the care of individual children with septic shock and for the conduct of interventional clinical trials.

CONCLUDING PERSPECTIVES

Biomarkers for diagnosing, monitoring, and stratifying various forms of critical illness hold the promise of dramatically changing how we practice critical care medicine on a daily basis, and how we conduct clinical trials in the future [2]. Realizing this promise, however, requires rigorous methodologies, validation procedures, and thoughtful application of high throughput technologies. In addition, realizing this promise will require collaboration across many centers and investigators. In our opinion, it is unlikely that biomarker discovery and development specific to the field of pediatric critical care medicine will be a priority for the bio-technology and pharmaceutical industries. Accordingly, clinician-scientists based in pediatric critical care medicine will need to lead biomarker discovery and development specific to the field.