Validation of a host response test to distinguish bacterial and viral respiratory infection

Abstract

Background

Distinguishing bacterial and viral respiratory infections is challenging. Novel diagnostics based on differential host gene expression patterns are promising but have not been translated to a clinical platform nor extensively tested. Here, we validate a microarray-derived host response signature and explore performance in microbiology-negative and coinfection cases.

Methods

Subjects with acute respiratory illness were enrolled in participating emergency departments. Reference standard was an adjudicated diagnosis of bacterial infection, viral infection, both, or neither. An 87-transcript signature for distinguishing bacterial, viral, and noninfectious illness was measured from peripheral blood using RT-PCR. Performance characteristics were evaluated in subjects with confirmed bacterial, viral, or noninfectious illness. Subjects with bacterial-viral coinfection and microbiologically-negative suspected bacterial infection were also evaluated. Performance was compared to procalcitonin.

Findings

151 subjects with microbiologically confirmed, single-etiology illness were tested, yielding AUROCs 0•85–0•89 for bacterial, viral, and noninfectious illness. Accuracy was similar to procalcitonin (88% vs 83%, p = 0•23) for bacterial vs. non-bacterial infection. Whereas procalcitonin cannot distinguish viral from non-infectious illness, the RT-PCR test had 81% accuracy in making this determination. Bacterial-viral coinfection was subdivided. Among 19 subjects with bacterial superinfection, the RT-PCR test identified 95% as bacterial, compared to 68% with procalcitonin (p = 0•13). Among 12 subjects with bacterial infection superimposed on chronic viral infection, the RT-PCR test identified 83% as bacterial, identical to procalcitonin. 39 subjects had suspected bacterial infection; the RT-PCR test identified bacterial infection more frequently than procalcitonin (82% vs 64%, p = 0•02).

Interpretation

The RT-PCR test offered similar diagnostic performance to procalcitonin in some subgroups but offered better discrimination in others such as viral vs. non-infectious illness and bacterial/viral coinfection. Gene expression-based tests could impact decision-making for acute respiratory illness as well as a growing number of other infectious and non-infectious diseases.

Research in context

Evidence before this study: A number of studies have turned to the human host response as an alternative diagnostic strategy given the limitations of traditional pathogen-based testing. This growing body of work includes broad classifiers for identifying the etiology of respiratory infection, pediatric febrile illness, and sepsis, as well as pathogen-specific classifiers for tuberculosis, Lyme disease, and Ebola, to name a few. While these classifiers are in various stages of development, very few have undergone extensive testing or been further developed into a clinically available diagnostic test.

Added value of this study: We previously published gene expression signatures for distinguishing bacterial, viral, and noninfectious causes of respiratory illness using high dimensional ‘omics-based techniques that had an overall accuracy of 87%. In this study, we have implemented this signature onto a real-time PCR test and demonstrated robust performance with AUROCs of 0•85–0•89. Additionally, the host response test showed promise in characterizing more complex phenotypes, including bacterial-viral coinfection and suspected but not microbiologically confirmed infection. This study is distinctive in its translation of our signatures to a standardized, clinic-ready platform and its application to phenotypes that have previously been excluded from testing.

Implications of all available evidence: The ability of a clinical test to rapidly identify the presence or absence of an infection and guide appropriate antibiotic use would improve individual patient care and mitigate the development of antibiotic resistance. More generally, host response diagnostic signatures like the one presented here represent a means by which diagnostics can enable personalized medicine.

1. Introduction

Difficulty in differentiating bacterial, viral, and noninfectious etiologies of respiratory illness contributes to antibiotic overuse. In the U.S., 73% of clinic visits and 61% of emergency department visits for suspected respiratory tract infection led to a prescription for antibiotics, despite most having a viral etiology [[1],[2]]. Driven by excess antibiotic use, antibiotic resistance is emerging at an alarming rate, outpacing novel antibiotic development and contributing to rising healthcare costs [[3],[4]]. Diagnostic tests that discriminate these etiologies of illness could individualize care and mitigate inappropriate antibiotic use. However, traditional pathogen-based diagnostics have limited sensitivity, long time-to-result (as with culture), require a priori suspicion of the pathogen (as with molecular tests), and cannot differentiate infection from colonization.

Measuring the host response offers an alternative diagnostic strategy. Procalcitonin, preferentially rising in bacterial infections, has demonstrated clinical utility in safely decreasing antibiotic use, though that finding was not reproduced in a recent, large, U.S.-based study [[5],[6]]. With respect to its ability to distinguish bacterial and viral etiologies, procalcitonin has shown only modest performance [7], [8], [9]. Biomarker panels that combine multiple analytes may impart greater sensitivity and specificity. Several studies have successfully defined signatures that discriminate bacterial and viral infection using high-dimensional ‘omics-based techniques [10], [11], [12], [13], [14], [15], [16]. However, most signatures have not undergone independent in vitro validation and typically excluded patients with complex phenotypes, such as the immunocompromised, coinfected, chronically infected, or clinically ambiguous. Understanding performance in these heterogeneous populations is vital for the development of this new generation of tests.

We previously published a microarray-derived host gene expression classifier that accurately distinguished bacterial, viral, and noninfectious causes of acute respiratory illness [[17],[18]]. We subsequently translated these signatures onto a real-time PCR test, a reproducible and standardized diagnostic platform. Here, we validate this test's ability to discriminate causes of acute respiratory illness and explore the ability of the host response to characterize coinfected, chronically infected, and clinically equivocal cases as compared to procalcitonin.

2. Materials and methods

2.1. Subject enrollment

Subjects with acute respiratory illness were prospectively enrolled in emergency departments at Duke University, Durham VA Health Care System, Henry Ford Hospital, and University of North Carolina as part of the CAPSOD (Community-Acquired Pneumonia and Sepsis Outcome Diagnostics, ClinicalTrials.govNCT00258869), CAPSS (Community-Acquired Pneumonia and Sepsis Study), or RADICAL (Rapid Diagnostics in Categorizing Acute Lung Infection) studies. All studies were approved by the respective IRBs in accordance with institutional and federal regulations regarding the protection of human subjects. Written informed consent was obtained from all subjects or legally authorized representatives.

2.2. Clinical adjudication and subject selection

All subjects enrolled in CAPSOD, CAPSS, and RADICAL underwent clinical adjudication. This adjudication served as the reference standard for the study. Adjudications were conducted by emergency medicine, hospital medicine, pulmonary medicine, or infectious disease physicians after enrollment but prior to gene expression or procalcitonin measurements, as previously described [[17],[19]]. Information supporting adjudication included history, physical examination, clinical laboratory testing, and radiography. Supplemental viral PCR testing was performed for all subjects using the ResPlex 2•0 viral PCR multiplex assay (Qiagen), xTAG RVP FAST 2 (Luminex), or NxTAG Respiratory Pathogen Panel (Luminex).

Subjects were retrospectively selected for inclusion from the larger pool of study subjects if they fell into one of several adjudicated categories. A “confirmed” bacterial or viral infection required the subject to have a compatible clinical syndrome and identified pathogen. In the absence of supporting microbiological evidence, adjudicators could still make a classification of “suspected bacterial” or “suspected viral” infection if the clinical presentation was consistent with this etiology. Adjudicators could identify multiple infectious etiologies within one subject if multiple pathogens were identified or if the clinical presentation was consistent with coinfection. Adjudication of noninfectious illness was made only when microbiological testing was negative and an alternative, noninfectious diagnosis was established. Additional details regarding the clinical adjudication and subject selection processes are available in the Supplemental Methods section.

2.3. Host gene expression measurement

Peripheral whole blood was collected from each subject at enrollment. Total RNA was extracted using PAXgene Blood miRNA kit (Qiagen). RNA quantity and quality were assessed by NanoDrop Spectrophotometer (Thermo Fisher Scientific) and Aligent 2100 Bioanalyzer with RNA 6000 Nano kit, respectively. A cDNA library was generated from total RNA using SuperScript VILO MasterMix (Thermo Fisher Scientific). Semi-quantitative, real-time PCR was performed on custom TaqMan Low Density Arrays (TLDA) [[20],[21]]. TLDA cards were customized to quantify 87 RNA transcripts (Table S1). Targets were selected from prior microarray-based studies in an iterative process, substituting poorly performing assays with different probes for the same transcript or with other transcripts that were highly correlated with the original [[17],[18]]. Additional details can be found in the Supplementary Methods.

2.4. Model calibration and validation

Due to technical differences between microarray measurements and RT-PCR, we could not apply the microarray-based model to this new data. We therefore trained a new model on RT-PCR data. Specifically, RT-PCR gene expression data were average normalized against two reference transcripts with stable expression across phenotypes (DECR1 and TRAP1). Data were generated in two distinct experiments. 19 technical replicates were utilized to assess model robustness as well as identify potential batch differences, which were corrected using an empirical Bayesian frameworks model [22]. Correlation was high for these technical replicates (R² = 0•96) (Figure S1). Normalized, batch-corrected data from subjects with confirmed bacterial, viral, or noninfectious illness were used to fit a logistic regression model. Scripts were written in R using the Glmnet toolbox [23]. Specifically, we used Least Absolute Shrinkage and Selection Operator (LASSO) for regularization and performed nested cross-validation to select parameters. This resulted in three independent binary classifiers (bacterial versus non-bacterial, viral versus non-viral, and noninfectious versus infectious), of which the largest probability determined class. Performance metrics included positive percent agreement (PPA), negative percent agreement (NPA), and area under the receiver operating characteristic curve (AUROC). The fixed-weight model was then applied to subjects with coinfection and suspected bacterial infection. To allow for coinfection, we defined probability thresholds for the bacterial and viral classifiers allowing us to identify four scenarios: bacterial infection, viral infection, coinfection, and no infection.

2.5. Procalcitonin comparison

Procalcitonin testing was not obtained as part of clinical care and was therefore available as an independent comparator without risking incorporation bias. Procalcitonin was measured using serum or plasma, when available. Serum samples were measured on the Roche Elecsys 2010 analyzer (Roche Diagnostics) or miniVIDAS immunoassay (bioMérieux). Plasma samples were measured using B•R•A•H•M•S PCT sensitive KRYPTOR (Thermo Fisher Scientific). Measurements were treated equivalently regardless of platform. Values >0•25 µg/liter defined bacterial infection and values ≤0•25 µg/liter defined non-bacterial [24]. We compared procalcitonin and gene expression using McNemar's test.

3. Results

3.1. Bacterial, viral, and noninfectious classifiers

In the absence of a reliable gold standard to define infection class, expert clinical adjudication served as the reference standard. Thus, 151 subjects with adjudicated and microbiologically confirmed phenotypes (48 bacterial, 54 viral, 49 noninfectious illness) were identified to evaluate the RT-PCR test's performance (Fig. 1). Instead of healthy individuals, noninfectious illness was selected as a control group since it represents a clinically relevant population that would potentially undergo diagnostic testing. Use of this control population imparts greater specificity to the RT-PCR test. Demographically, the cohort was heterogeneous and encompassed a racially diverse group across a wide age range (Table S2). Groups were well-balanced with respect to gender and race, though the viral cohort was younger (mean 42 years vs. 54 for bacterial and 58 for noninfectious) and less ill, as inferred by the rate of hospitalization (30%, 96%, and 86%, respectively). Table S3 presents the bacterial, viral, and noninfectious illness etiologies.

Fig. 1.

Experimental flow. Coinfection cases included both superinfections (acute bacterial infection following an acute viral infection) and acute-on-chronic coinfections (acute bacterial infection and chronic viral infection). Suspected bacterial cases were those without microbiological evidence but clinically adjudicated as bacterial. RT-PCR: Real Time Polymerase Chain Reaction; AoC: acute-on-chronic.

The gene expression signature was first identified in microarray data but validated using RT-PCR. Due to these technical differences, the models originally generated on microarray data cannot be applied to RT-PCR data. Instead, retraining classifier parameters is required. Therefore, RT-PCR gene expression data from these 151 subjects was used to calibrate the classifiers (Table S1). Although 87 gene targets were included in the RT-PCR test, not all were selected and utilized by the model. Specifically, the LASSO methodology performed regularization and nested cross-validation to select model parameters with model weights presented in Table S1. Only those with non-zero weights were considered informative and retained. This included a total of 41 transcripts: 34 for the bacterial model, 15 for the viral model, and 8 for the non-infectious illness model. Some transcripts were utilized for more than one model explaining why the total parameters for all three models exceeded 41.

Each classifier is binary, and class membership is determined by the highest of the three probabilities (bacterial, viral, or noninfectious). Using this methodology, leave-one-out cross-validation revealed accurate discrimination between groups, with AUROC of 0•85 for bacterial, 0•89 for viral, and 0•88 for noninfectious illness (Fig. 2). When considering all three classes simultaneously, the overall accuracy was 77% (116/151 concordant with adjudicated phenotype). The host gene expression test identified bacterial infection with 75% (36/48) PPA and 92% (95/103) NPA (accuracy 88%). This performance corresponds to a positive likelihood ratio of 3.68 for ruling in bacterial infection and a negative likelihood ratio of 0.27 for ruling out bacterial infection. Viral infection was identified with 78% (42/54) PPA and 86% (83/97) NPA (accuracy 82%). With the noninfectious classifier, infection was correctly excluded in 78% (38/49) of cases (84% accuracy).

Fig. 2.

RT-PCR test performance compared to procalcitonin for microbiologically confirmed, single etiology cases. Upper panels demonstrate AUROC curves for the bacterial, viral, and noninfectious classifiers. Lower panels show the bacterial, viral, and non-infectious probabilities for each subject, organized by the clinically adjudicated phenotype. Procalcitonin comparison is shown on the right side of the panel (values are in ng/mL). A maximum procalcitonin value of 10 ng/mL was used to improve data visualization. RT-PCR: Real time polymerase chain reaction; AUROC: area under receiver operator characteristic; NI: non-infectious illness.

Procalcitonin concentrations were obtained for 137 subjects with samples available for testing. Procalcitonin correctly classified 114 of 137 (83%) as either bacterial or non-bacterial, compared to 121 of 137 (88%) using the host response classifiers (p = 0•23) (Fig. 2). Notably, the performance of the two tests differed based on the classification task. If excluding the non-infectious illness group, accuracy was identical for the two tests in distinguishing bacterial and viral cases (86% vs 86%, p = 1). However, the host response classifiers correctly discriminated bacterial and noninfectious illness more frequently than procalcitonin, though the difference was not statistically significant (86% vs 77%, p = 0•17). Procalcitonin is unable to discriminate viral from non-infectious etiologies. However, the host gene expression test correctly discriminated these two groups in 81% of cases.

3.2. Performance in complex phenotypes

Having evaluated performance of the RT-PCR test in these subjects with a single, known etiology, we next evaluated a more clinically challenging series of phenotypes: coinfection and suspected bacterial infection. Since the true state of these subjects was uncertain, we could not assess performance metrics. Instead, we used the host gene expression RT-PCR test to characterize their underlying biological state (Fig. 1).

The validation described above utilized the highest predicted probability as the test result. However, this does not allow for the identification of coinfection. We therefore defined probability thresholds for the bacterial classifier and the viral classifier. This scheme allows for the identification of bacterial infection, viral infection, both, or neither. This approach does not explicitly use the noninfectious classifier, but the noninfectious subjects are still utilized in training the bacterial and viral classifiers to increase model specificity. A threshold of 0•45 was set for viral infection and 0•20 for bacterial infection, yielding ≥80% PPA for both (Figure S2).

The coinfection cohort included 65 subjects (Table S4). Of this group, 31 had positive testing for both bacterial and viral pathogens. The remaining 34 had positive microbiology for one pathogen and a clinical suspicion for the other pathogen class, or a clinical syndrome consistent with bacterial-viral coinfection. Since the timeline of coinfection can vary, we created subcategories. “Superinfection” (acute bacterial infection following recent acute viral infection) included 53 subjects (19 microbiologically confirmed and 34 suspected cases). Of the 19 subjects with microbiologically confirmed superinfections, the RT-PCR test identified 18 as having a bacterial infection (16 as bacterial alone, two as coinfection) with one subject classified as noninfectious (Fig. 3A). Procalcitonin only identified 68% (13/19) as bacterial (p = 0.13). In contrast, for the 34 subjects with clinically suspected superinfections without confirmatory microbiology, the RT-PCR test identified an equal number as bacterial or viral (12 each, 35%), six (18%) as coinfection, and four (12%) as noninfectious (Fig. 3B). Procalcitonin was positive in only 13 subjects (53% vs 38%, p = 0•27, for identifying bacterial infection; Table S5).

Fig. 3.

Signature application in cases of superinfection. “Superinfection” describes subjects with an acute bacterial infection temporally following an acute viral infection. The red and black lines (left and right, respectively) depict the thresholds for bacterial infection and viral infection, respectively. The dashed lines divide the subjects into their model-predicted classes based on thresholding: bacterial infection, viral infection, coinfection, and no infection. 3A, Model application in microbiologically confirmed superinfections (n = 19). 3B, Model application in clinically adjudicated superinfections without microbiological confirmation. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

The “acute-on-chronic” coinfection cohort included 12 subjects with chronic viral infections (HIV, hepatitis B/C, or CMV) and superimposed acute bacterial infections, all of which were microbiologically confirmed. Six subjects had detectable viral loads, two had unknown viral loads, and four had suppressed viral loads. Those with detectable or unknown viral loads had a mixed host response: five (63%) were classified as bacterial, one (13%) as coinfection, one (13%) as noninfectious, and one (13%) as viral (Fig. 4A). In contrast, all four individuals with suppressed viral loads had a bacterial host response (Fig. 4B). Procalcitonin aligned with the RT-PCR test for each of the 11 subjects in this subcategory that had available procalcitonin resultsFig. 5.

Fig. 4.

Signature application in cases of acute-on-chronic coinfections. “Acute-on-chronic” coinfection describes subjects with chronic viral infection and acute bacterial infection. All subjects had microbiologically confirmed acute bacterial infections. The red and black lines (left and right, respectively) show the thresholds for bacterial infection and viral infection, respectively. The dashed lines divide the subjects into their model-predicted classes based on thresholding: bacterial infection, viral infection, coinfection, and no infection. 4A, Model application in chronically infected subjects with detectable or unknown viral load (n = 8). 3B, Model application in chronically infected subjects with a suppressed viral load (n = 4). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 5.

Signature application in cases of suspected bacterial infections. “Suspected bacterial” describes subjects clinically adjudicated as bacterial infection but without microbiological confirmation (n = 39). The red and black lines (left and right, respectively) show the thresholds for bacterial infection and viral infection, respectively. The dashed lines divide the subjects into their model-predicted classes based on thresholding: bacterial infection, viral infection, coinfection, and no infection. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

We also applied the RT-PCR test to 39 subjects with a suspected bacterial infection (Fig. 5). These were subjects who were clinically adjudicated as having a bacterial infection on clinical grounds (e.g. radiographic infiltrate, neutrophilic leukocytosis, hypoxia) but no identified pathogen. Of these, the RT-PCR test identified 29 (74%) as bacterial, three (8%) as coinfection, five (13%) as noninfectious, and two (5%) as viral. Procalcitonin identified a bacterial infection in 64% of cases compared to 82% for host gene expression (p = 0•02).

4. Discussion

We previously published microarray-based host response signatures that successfully discriminated bacterial, viral, and noninfectious causes of respiratory illness with an overall accuracy of 87% [17]. Here, we translated those signatures to a RT-PCR platform and demonstrated robust performance when validated in an independent cohort of microbiologically confirmed bacterial, viral, or noninfectious cases. With a targeted RT-PCR test, the bacterial, viral, and noninfectious classifiers had overall accuracies of 88%, 82%, and 84%, respectively (AUROCs 0•85–0•89) compared to an imperfect reference standard. The corresponding positive and negative likelihood ratios for bacterial vs. non-bacterial infection were 3.68 and 0.27, respectively. These values in and of themselves are unlikely to rule in or rule out bacterial infection. However, when used in conjunction with other clinical data, such a test would be a valuable adjunct. Furthermore, the likelihood ratios could be tuned by choosing different statistical cutoffs that incorporate the clinical significance of false positive and false negative errors. One statistic that accomplishes this is the average weighted accuracy [25]. The other validation described in this study was in cases of bacterial-viral coinfection, successfully identifying a bacterial infection in nearly all microbiologically confirmed cases even in the presence of a concurrent viral illness. These encouraging results suggest that such a host gene expression-based diagnostic test has the potential to individualize treatment and mitigate inappropriate antibiotic prescribing.

Other studies have also investigated gene expression-based classifiers for diagnosing various infections. This growing body of work includes classifiers for acute respiratory illness [[15],[16]], sepsis [[26],[27]], and pediatric febrile illness [[14],[28],[29]], as well as pathogen-specific classifiers for tuberculosis [[30],[31]], Lyme disease [32], and Ebola [33]. In contrast to most of these studies, which used microarray or sequencing for signature discovery and validation, we validated signature performance using a standardized RT-PCR platform. This is particularly notable because RT-PCR can be readily translated to rapid, clinically relevant platforms [[34],[35]]. Furthermore, our study is distinctive in its exploration of complex phenotypes, including coinfection and indeterminate infection. The translation of a bacterial/viral host response signature to a clinically-relevant platform, as demonstrated in this study, has significance beyond this application. The advances presented here illustrate how this approach can be applied to a multitude of infectious diseases (e.g., tuberculosis, sepsis, undifferentiated fever, rickettsial disease) and non-infectious diseases (e.g., coronary artery disease, oncology, rheumatoid arthritis, systemic lupus erythematosus).

The exact incidence of bacterial-viral coinfection in respiratory tract infection is unclear, with estimates ranging from 7% to 66% [36], [37], [38]. As a result, concerns about coinfection drive a significant amount of inappropriate antibiotic use even when clinical testing identifies a viral etiology [39]. Understanding signature performance in this population is a critical step toward the adoption of gene expression-based testing. Most gene expression-focused studies either exclude this phenotype, have a very small sample size, or do not explore the multiple different categories of coinfection. Therefore, we applied our host response test, which offers independent probabilities of bacterial and viral infection, to multiple types of coinfection. Among this group, the bacterial host response was most often the dominant signature. This may be due to the temporal nature of the infections. With common superinfection scenarios (viral infections followed by a superimposed bacterial infection) and acute-on-chronic infections (chronic viral infection with an acute bacterial infection), the bacterial infection is the more proximate stress on the immune system. Fortunately, this is clinically desirable: the ability to successfully identify a bacterial or coinfection signature would promote the appropriate use of antibacterials in this population.

Within the acute-on-chronic subgroup, a notable pattern emerged based on the level of viremia. The four subjects with suppressed viral loads all had a strong bacterial host response, while the eight subjects with detectable viral loads had mixed results with some subjects revealing a prominent viral host response. This division is supported by earlier work that explored gene expression in active and suppressed HIV [40], [41], [42]. One study performed gene expression analysis on individuals before and after initiation of anti-retrovirals and identified several thousand genes with differential expression. The genes with the largest fold change in expression included several interferon-related genes, such as IFI27, which is a component of the RT-PCR test [40]. Although this subgroup is small within our study, these results underscore the importance of evaluating future host response classifiers in subjects with chronic infections like HIV.

Individuals with suspected but unconfirmed respiratory infection represent a challenge from both an antibiotic stewardship and a diagnostic development standpoint. This situation is very common; the Etiology of Pneumonia in the Community (EPIC) study employed extensive microbiological testing in adults hospitalized with community acquired pneumonia and did not identify a pathogen in 62% of cases [43]. With current diagnostics, it is impossible to know what proportion of this group can be attributed to poor pathogen detection sensitivity versus an underlying noninfectious process. Here, we applied the signatures to 39 subjects with etiology-negative suspected bacterial cases and found that 82% had a bacterial host response with the rest distributed across the other diagnostic possibilities. While it is difficult to gauge the accuracy of the host response signature in any individual subject, the summary statistics provide insights for these indeterminate patients in ways that current pathogen-based methods cannot.

In comparison to procalcitonin, the RT-PCR test offered similar performance in distinguishing bacterial versus non-bacterial infections (accuracy 88% vs. 83%, p = 0•23). Prior studies have shown the superiority of multi-analyte panels compared to procalcitonin including this signature measured using microarrays; we observed a similar trend though our study was likely underpowered to achieve statistical significance [[16],[17],[44]]. The difference between host gene expression and procalcitonin was most pronounced in two scenarios. The first was based on the inability of procalcitonin to distinguish viral from non-infectious illness. In contrast, the host gene expression test correctly classified 81% of these cases. The second major improvement over procalcitonin was in cases of bacterial-viral coinfection. This could be explained by prior experiments showing that in human cell lines cultured with both bacterial and viral pathogens, viral-induced interferon signaling dominated, resulting in procalcitonin inhibition [45]. These findings represents an important yet underappreciated limitation of procalcitonin for which the host gene expression test significantly advances clinical diagnostics. For example, a recent meta-analysis evaluating the ability of procalcitonin to distinguish viral from bacterial pneumonia revealed only 55% sensitivity and 76% specificity using established thresholds [9]. It is also important to highlight that procalcitonin addresses only one diagnostic question: bacterial or non-bacterial infection. It does not discriminate viral infection from non-infectious illness, nor can it reliably identify bacterial/viral co-infection. In contrast, our approach with independent bacterial and viral classifiers allows four possible diagnoses: bacterial infection, viral infection, coinfection, or no infection. Despite similar performance for bacterial vs. non-bacterial infection, the host gene expression test accurately discriminated viral from non-infectious etiologies and was significantly better at identifying bacterial/viral co-infection compared to procalcitonin. Focusing exclusively on bacterial vs. non-bacterial classification oversimplifies clinical practice as highlighted by the inability of procalcitonin to impact antibiotic utilization in a large, U.S.-based randomized clinical trial . The more comprehensive diagnostic information offered by host gene expression could therefore provide significant improvements over current methods to identify the cause of acute respiratory illness.

A host response approach should also be differentiated from a pathogen-detection strategy. A combination of the two represents an ideal strategy but is often impractical due to cost and time considerations. Pathogen-detection tests encompass many technologies including culture, rapid antigen detection tests, and nucleic acid amplification tests. Advantages and disadvantages of these approaches have been reviewed elsewhere [46]. However, one of the limitations common to all is the inability to distinguish infection from colonization (asymptomatic shedding), which has been observed for all the pathogens included on these various test panels [[47],[48]]. A negative result does not exclude the presence of that pathogen (due to sampling bias or poor test sensitivity) nor does it exclude the presence of other pathogens. This concern also extends to situations where a pathogen detection test is positive. Specifically, a positive result (even if due to infection and not colonization) does not exclude the presence of other, undetected pathogens. All of these limitations are addressed by using a host gene expression approach that independently provides information about both bacterial and viral infection. The host gene expression test described here identifies the presence of a bacterial or viral infection when positive and also excludes such an infection when negative.

Perhaps the most significant limitation of the study is the lack of a gold standard to diagnose bacterial or viral infection. Consequently, discordant classifications could represent errors in adjudication or in the test. The validation presented here was performed at two levels. The first included an independent validation of the signature (the specific combination of gene expression targets) in subjects with bacterial, viral, or non-infectious illness. The second validation was of the model (logistic regression model assigning weights to each mRNA in the signature) in a cohort of subjects with complex phenotypes. However, we did not validate the model in an independent cohort of subjects with only a bacterial, viral, or non-infectious etiology. Doing so would further improve external validity. Some categories of coinfection, particularly the chronic viral infection group, were too small to draw definitive conclusions. Relatedly, we did not have the opportunity to explore rarer types of coinfection, including acute viral/chronic bacterial infections and coinfections occurring at distinct anatomic sites (e.g. concurrent respiratory viral infection and urinary tract infection). While the cohorts included individuals as young as 14 years, these results should be validated in younger children. However, the gene expression signatures themselves were validated in silico in a large cohort of pediatric cases [17]. Finally, it is important to acknowledge that the RT-PCR test can only determine pathogen class. In most patients, this is adequate for guiding antibiotic selection. However, in sicker, hospitalized patients, knowing the pathogen type and susceptibility profile can be important for clinical care. Therefore, host response-based testing should be viewed as part of a comprehensive diagnostic strategy, rather than a replacement for conventional pathogen-based testing.

Looking forward, the next challenge will be shortening the turnaround time of the RT-PCR test. The platform described in this study requires several hours of hands-on processing time, which is acceptable for most inpatient settings. However, a test that is simpler to perform and provides more rapid results could be transformative in primary care or the emergency department. Further translation of this host response signature is currently underway, with an estimated time-to-result of approximately 45 min [[34],[35]].

5. Author contributions

Conceptualization: ECL, TWB, SWG, VGF, EBQ, CBC, SFK, AKJ, EPR, RL, EP, ERK, MTM, GSG, CWW, ELT. Data curation: ECL, TWB, MA, BPN, SWG, VGF, EBQ, CBC, SFK, AKJ, EPR, RL, EP, ERK, MTM, GSG, CWW, ELT. Formal analysis: ECL, RH, ERK, MTM, GSG, CWW, ELT. Writing-original draft: ECL, CWW, ELT. Writing-review and editing: ECL and ELT.

Funding sources

This study was supported in part by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under Award Number U01AI066569 and UM1AI104681. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. It was also supported in part by the U.S. DARPAthrough contract N66001-09-C-2082. EL was supported by the Eugene A. Stead Scholarship from Duke University School of Medicine and the Infectious Diseases Society of America Medical Scholars Program. These funding sources had no role in the writing of the manuscript or the decision to submit it for publication. No author was paid to write this article by a pharmaceutical company or other agency. The corresponding authors had full access to all the data in the study and bear final responsibility for the decision to submit for publication.

Funding

NIAID (U01AI066569, UM1AI104681). U.S. DARPA (N66001-09-C-2082).

Declaration of Competing Interest

TWB reports other from Predigen Inc, outside the submitted work; In addition, TWB has a patent “METHODS TO DIAGNOSE AND TREAT ACUTE RESPIRATORY INFECTIONS” (USPTO Pub. No.: US 2018/0245154 A1) pending. CBC reports grants from National Institutes of Health, grants from DARPA, during the conduct of the study; personal fees from National Institutes of Health, outside the submitted work. VGF served as Chair of the V710 Scientific Advisory Committee (Merck); has received grant support from Cerexa/Actavis/Allergan, Pfizer, Advanced Liquid Logics, NIH, MedImmune, Basilea Pharmaceutica, Karius, ContraFect, Regeneron Pharmaceuticals, and Genentech; has NIH STTR/SBIR grants pending with Affinergy, Locus, and Medical Surface, Inc; has been a consultant for Achaogen, AmpliPhi Biosciences, Astellas Pharma, Arsanis, Affinergy, Basilea Pharmaceutica, Bayer, Cerexa Inc., ContraFect, Cubist, Debiopharm, Durata Therapeutics, Grifols, Genentech, MedImmune, Merck, The Medicines Company, Pfizer, Novartis, NovaDigm Therapeutics Inc., Theravance Biopharma, Inc., XBiotech, and has received honoraria from Theravance Biopharma, Inc., and Green Cross, and has a patent pending in sepsis diagnostics. GSG reports other from Predigen outside the submitted work; In addition, GSG has a patent “Biomarkers for the molecular classification of bacterial infection” US 14/1880,668 pending, and a patent “Methods of Identifying Infectious Disease and Assays for Identifying Infectious Disease” US patent US 8,821,876 issued. RH reports grants from ARLG/NIH during the conduct of the study; In addition, RH has a patent “Methods to Diagnose and Treat Acute Respiratory Infections” (Application #PCT/US2016/040437) pending. SFK reports grants from National Institutes of Health during the conduct of the study. MTM reports consultant fees from UpToDate, outside the submitted work; In addition, MTM has a patent Patent pending on Genomic Diagnostics for Respiratory Infections. EBQ reports grants from NIAID, grants from DARPA during the conduct of the study. ELT reports personal fees from Duke University, personal fees from Durham VA Health Care System, grants from DARPA, grants from NIH/ARLG, other from Predigen, Inc., grants from NIH/VTEU, personal fees from bioMerieux during the conduct of the study. In addition, DELT has a patent “Biomarkers for the molecular classification of bacterial infection” pending, and a patent “Methods to diagnose and treat acute respiratory infections” pending. CWW reports personal fees from Duke University, personal fees from Durham VA Health Care System, grants from DARPA, grants from NIH/ARLG, other from Predigen, Inc., grants from NIH/VTEU, personal fees from bioMerieux, personal fees from IDbyDNA, personal fees from Giner, grants from BioFire, grants from Janus, grants from BioMeme, grants from RTI, personal fees from Roche Molecular Sciences, during the conduct of the study; personal fees from Becton Dickinson, grants from Pfizer, grants from Openbiome, grants from MRI Global, personal fees from Sanofi, outside the submitted work; In addition, CWW has a patent “Biomarkers for the molecular classification of bacterial infection “pending, and a patent “ Methods to diagnose and treat acute respiratory infections pending”. Other authors not explicitly listed above have nothing to disclose.