A rapid host–protein test for differentiating bacterial from viral infection: Apollo diagnostic accuracy study

Abstract

Objectives

To determine the diagnostic accuracy of a rapid host‐protein test for differentiating bacterial from viral infections in patients who presented to the emergency department (ED) or urgent care center (UCC).

Methods

This was a prospective multicenter, blinded study. MeMed BV (MMBV), a test based on tumor necrosis factor‐related apoptosis‐inducing ligand (TRAIL), interferon gamma‐inducible protein‐10 (IP‐10), and C‐reactive protein (CRP), was measured using a rapid measurement platform. Patients were enrolled from 9 EDs and 3 UCCs in the United States and Israel. Patients >3 months of age presenting with fever and clinical suspicion of acute infection were considered eligible. MMBV results were not provided to the treating clinician. MMBV results (bacterial/viral/equivocal) were compared against a reference standard method for classification of infection etiology determined by expert panel adjudication. Experts were blinded to MMBV results. They were provided with comprehensive patient data, including laboratory, microbiological, radiological and follow‐up.

Results

Of 563 adults and children enrolled, 476 comprised the study population (314 adults, 162 children). The predominant clinical syndrome was respiratory tract infection (60.5% upper, 11.3% lower). MMBV demonstrated sensitivity of 90.0% (95% confidence interval [CI]: 80.3–99.7), specificity of 92.8% (90.0%–95.5%), and negative predictive value of 98.8% (96.8%–99.6%) for bacterial infections. Only 7.2% of cases yielded equivocal MMBV scores. Area under the curve for MMBV was 0.95 (0.90–0.99).

Conclusions

MMBV had a high sensitivity and specificity relative to reference standard for differentiating bacterial from viral infections. Future implementation of MMBV for patients with suspected acute infections could potentially aid with appropriate antibiotic decision‐making.

1. INTRODUCTION

1.1. Background

Infectious disease‐related complaints are a common reason both pediatric and adult patients present to emergency departments (EDs) and urgent care centers (UCCs). ¹ , ² Differences exist regarding the approach to work‐up and treatment decisions in children and adults based on a variety of factors including age, prior immunization history, and clinical appearance of the patient. Accordingly, significant practice variability and widespread empiric antibiotics are commonplace. ³ , ⁴ , ⁵ Acute care clinicians could benefit from adjunctive accurate diagnostic tools, which complement the clinical workflow of the ED and UCC and can aid in determining whether the etiology of an acute febrile illness is more likely to be bacterial or viral.

1.2. Importance

Despite advances in pathogen‐based diagnostics (eg, syndromic polymerase chain reaction [PCR] tests), tests that directly detect the pathogen are intrinsically limited, given a variety of factors including that detection may be due to colonization and not indicate the disease‐causing agent. ⁶ , ⁷ , ⁸ , ⁹ Furthermore, despite use of multiple pathogen detection panels, microorganisms often are not detected. ¹⁰ , ¹¹ , ¹² Two host biomarkers that have been evaluated for use in clinical practice to help differentiate bacterial from viral infection are C‐reactive protein (CRP) and procalcitonin (PCT). However, as single, predominantly bacterial‐induced proteins, each has performance limitations, including variable diagnostic accuracy across different patient subgroups ¹³ and across pathogens. ¹⁴ Efforts are ongoing to develop novel diagnostic assays that computationally integrate multiple host biomarkers into a host–immune score, leveraging advantages of the host response while overcoming limitations of a single biomarker. ¹⁰ , ¹⁵ , ¹⁶ , ¹⁷

One well‐studied host‐protein test called MeMed BV (MMBV) computationally integrates the circulating levels of two viral‐induced proteins—tumor necrosis factor‐related apoptosis‐inducing ligand (TRAIL) and interferon gamma‐inducible protein‐10 (IP‐10)—together with CRP into a bacterial versus viral likelihood score. ¹⁰ , ¹³ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² In a previous study in children aged 3 months to 18 years with clinical suspicion of respiratory tract infection (RTI) or fever without source (FWS), MMBV demonstrated sensitivity of 93.7% (95% confidence interval [CI]: 88.7–98.7), specificity of 94.2% (95% CI: 92.2–96.1), and NPV of 98.9% (95% CI: 98.0–99.8), with only 9.8% of cases yielding MMBV equivocal results. ¹¹ Notably, the MMBV result indicates a bacterial immune response when a bacterial and viral co‐infection occurs, ¹¹ , ¹⁴ which supports appropriate antibiotic use.

1.3. Goals of this investigation

To date, the majority of diagnostic accuracy studies of MMBV have focused on pediatric patients in inpatient units and EDs outside of the United States with a first‐generation ELISA‐based measurement platform. ¹⁰ , ¹⁸ , ¹⁹ , ²¹ , ²² , ²³ The aim of this study was to establish the diagnostic accuracy of a rapid user‐friendly version of the MMBV platform in both children and adults in EDs and UCCs in the United States and Israel. In addition, we estimated the potential impact of MMBV use on antibiotic use (study design and results are summarized in Video S1).

The Bottom Line

This prospective multi‐center, blinded study examined the diagnostic accuracy of using MeMed BV (MMBV), a novel blood test based on tumor necrosis factor‐related apoptosis‐inducing ligand (TRAIL), for differentiating between bacterial versus viral infections in 563 participants (314 adults, 162 children). MMBV had a high sensitivity 90.0% (95%CI: 80.3–99.7) and specificity 92.8% (90.0–95.5) and negative predictive value of 98.8% (96.8–99.6) for bacterial infections, and thus potentially lending to its future clinical use.

2. METHODS

2.1. Study design and setting

The Apollo study (NCT04690569) was a prospective, diagnostic accuracy study of MMBV for differentiating bacterial from viral infections in children and adults for whom the treating clinician suspected an acute infection. There were 9 ED and 3 UCC participating sites (see list in the Supporting Information). Recruitment was from May 2019 through August 2020. A flow chart of the study design is detailed in Figure S1. The study design, objectives, and statistical framework were aligned with the FDA for supporting regulatory clearance of MeMed BV (test cartridge, MMBV) and the rapid MeMed Key (analyzer).

2.2. Selection of participants

Study enrollment occurred during the working hours of the study coordinators at each site. Every potentially eligible patient was approached and assessed for inclusion in the study. The study investigator had no direct involvement in determining which patient was approached. Enrollment was conducted as follows: Trained study coordinators reviewed the electronic medical records of patients after the patient had been triaged at the participating ED/UCC to determine if the patient was potentially eligible (based on clinical documentation of suspicion of infection and broad eligibility criteria). For patients who were deemed potentially eligible, the coordinator approached the attending clinician and asked if the clinician suspected an infection; for those that indicated yes, the coordinator reviewed all the inclusion/exclusion criteria. Those patients who met all inclusion criteria and had no exclusion criteria were considered eligible and approached for informed consent. Those who consented were enrolled and blood was drawn. Factors that impacted which patients were enrolled (versus not) included working hours of the coordinators, research burden of the study coordinators (eg, some sites were running multiple studies with overlapping target populations), and frequency that blood was drawn as part of clinical care.

Inclusion criteria were age over 90 days with treating clinician suspicion of an acute bacterial or viral infection, duration of current symptoms less than or equal to 7 days, and temperature greater than or equal to 37.8°C (100°F) and/or self‐reported fever within the last 7 days (ie, not required at time of presentation). The principal exclusion criteria were unrelated episodes of febrile illness within the prior 2 weeks or being immunocompromised; full exclusion criteria are detailed in the Supporting Information. For all patients, informed consent was obtained from the subject or his/her legal guardian. Depending on local requirements, assent was obtained from children under 18 years old. Institutional Review Board or Ethics committee approval was obtained in each participating medical center. IRB numbers are provided in the Supporting Information.

2.3. Exposures

Following informed consent, two study‐specific specimens were collected (blood sample and nasopharyngeal sample) and processed to ensure that mandatory data were available for each subject for the purpose of the adjudicated reference standard. In addition, the blood sample was processed to generate the MMBV result. These study‐specific data were not available to the attending physician nor included in the patient's medical record.

2.4. Measurements

2.4.1. Study procedures

The following were mandatory measurements: (1) CRP and PCT test results (processed centrally for all study participants); (2) complete blood count (CBC) with differential; (3) BioFire respiratory panel (adenovirus; coronavirus 229E, HKU1, NL63, OC43; enterovirus; human rhinovirus; human metapneumovirus; influenza A [Flu A; subtypes H1, H1‐2009, and H3]; influenza B [Flu B]; parainfluenza virus 1, 2, 3, 4; respiratory syncytial virus; Bordetella pertussis; Chlamydia pneumoniae; Mycoplasma pneumoniae) (processed centrally for all study participants); (4) MMBV result (processed locally for fresh samples and centrally for frozen samples). Non‐mandatory tests are described in the Supporting Information.

Follow‐up calls were performed by trained study coordinators to collect post ED/UCC discharge information (at least 1 week after the initial visit) as a component of the data provided to the adjudicators for establishing the reference standard infection etiology. Phone calls were made beginning at 7 days and up to 35 days. The follow‐up call was structured, and the detailed template is provided as Supporting Information Appendix 1. Patients were not excluded if unable to be contacted; 113 of 476 (23.7%) did not have follow‐up data.

For each patient identified as meeting eligibility criteria and who gave informed consent, a structured electronic case report form (eCRF) was completed by study coordinators. The details of the data included in the eCRF are given in the Supporting Information. A dedicated software was created to extract the data automatically from an eCRF and create a “medical record” for the purpose of adjudication; a sample medical record is provided as Supporting Information Appendix 2.

2.4.2. Reference standard infection etiology and adjudication process

The reference standard infection etiology was defined by expert adjudication ¹² , ²⁴ given the lack of a widely accepted gold standard, ie, for diagnosing bacterial infection in the absence of positive cultures. Every case was independently reviewed by three expert adjudicators from a pool of 21 international clinician experts, with a minimum of 7 years of relevant experience (see Acknowledgments section). Experts were provided with the “medical record” created for adjudication purposes (see Section 2.4.1). The adjudicators were blinded to MMBV results. Each expert was required to assign the patient one of the following classifications: (i) bacterial, including bacterial‐viral co‐infection (confidence > 90%) or (ii) bacterial, including bacterial‐viral co‐infection (confidence 70%–90%) or (iii) indeterminate or (iv) viral (confidence 70%–90%) or (v) viral (confidence > 90%) or (vi) non‐infectious. Confidence was assigned based on the adjudicator's clinical judgement. An adjudicated bacterial or viral reference standard etiology required that two of three experts assign the same label with greater than 90% confidence, and/or that all three experts assign the same label with greater than 70% confidence. The remaining cases were assigned as having an “indeterminate” reference standard infection etiology. Further details on the adjudication process are provided in the Supporting Information.

2.4.3. Index test

The index test was MeMed BV (MMBV), which was run for this study on the MeMed Key rapid platform. The MMBV test cartridges are single‐use, multiwell containers that receive 100 µL of the patient's serum sample, contain all the reagents and disposables necessary to conduct immunoassays, and are the reservoirs for the waste. Upon insertion of the MMBV test cartridge into the MeMed Key analyzer, three independent immunoassays are conducted in parallel to measure the three host biomarkers (TRAIL, IP‐10, and CRP). Analytical validation of the system is described in Hainrichson et al. ²⁵

MMBV computationally integrates TRAIL, IP‐10, and CRP measurements into a score ranging from 0 to 100 using an algorithm derived previously ¹⁰ and employed in all previous studies. ¹¹ , ¹⁸ , ¹⁹ , ²² The thresholds of five score bins for result interpretation were defined previously: ¹¹ , ²⁵ 0 ≤ score ≤ 10, high likelihood of viral infection (or other non‐bacterial etiology); 10 < score < 35, moderate likelihood of viral infection (or other non‐bacterial etiology); 35 ≤ score ≤ 65, equivocal; 65 < score < 90, moderate likelihood of bacterial infection (including coinfection); and 90 ≤ score ≤ 100, high likelihood of bacterial infection (including coinfection) (Figure 1).

FIGURE 1.

MeMed BV (MMBV) interpretation.

2.4.4. Blinding

Study team members with access to reference standard and clinical data, for example, clinical research coordinators and principal investigators, did not have access to the index test result. Conversely, study team members with access to the index test result, for example, laboratorians or members of the company (MeMed) service team, did not have access to the reference standard and clinical data. In addition to this blinding, none of the expert adjudicators had access to the MMBV result.

2.5. Outcomes

MMBV performance for differentiating between bacterial and viral infection was assessed by comparing the MMBV result to the reference standard infection etiology, with bacterial (or co‐infection) considered positive. Reference standard indeterminate cases were removed from the calculation. Of note, reference standard non‐infectious cases were grouped together with reference standard viral infection etiology and considered negative. The potential impact of MMBV on antibiotic prescription was evaluated.

2.6. Data analyses

The following statistical frameworks were employed:

1. Sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), positive likelihood ratio (LR+), and negative likelihood ratio (LR−) were calculated based on two predefined score thresholds. ¹¹ , ¹⁹ , ²² Cases with MMBV > 65 were classified as a bacterial infection (including co‐infection) and cases with MMBV < 35 were classified as a viral infection (or other non‐bacterial etiology). Cases with scores ranging between 35 and 65 were classified as equivocal, removed from these calculations and are given as a rate.

2. Area under the receiver operating characteristic curve (ROC‐AUC) analysis was conducted using all possible thresholds, and in the case of MMBV, without excluding equivocal patients. Two‐sided statistical significance of the difference between MMBV and PCT AUCs was calculated using the method of Hanley and McNeil. ²⁶

3. The probability of bacterial infection was established as an increasing function of MMBV. For this purpose, patients were assigned to five pre‐determined score bins (see Section 2.4.3) according to their MMBV score and within the bin according to their reference standard infection etiology: bacterial versus viral/non‐infectious. A pass of two statistical tests was required to demonstrate study success and was the basis of the sample size calculation (see Supporting Information for the sample size calculation). Test #1: the Cochran–Armitage (CA) test for trend with a two‐sided 5% level of significance was used to reject the null hypothesis that there is no trend of increasing probability of bacterial infection with higher MMBV score. Test #2: the 95% CI of the interval LR should exclude the value 1 for some of the bins, as inclusion of 1 indicates that there is no significant enrichment of either bacterial or viral cases in that bin (preferably only the CI of the LR of the middle bin would include 1). For each bin, interval LR was defined as the ratio between the bacterial prevalence versus the viral prevalence.

Since MMBV scores were not provided to the treating clinician, the test's impact on actual antibiotic prescription decisions could not be directly calculated. To evaluate the potential impact of MMBV on ant, we examined and compared: (1) the concordance between the antibiotic prescription in the medical record and the reference standard infection etiology; and (2) the concordance between the MMBV result and the reference standard infection etiology. MMBV's conjectured impact on antibiotic use was calculated based on the assumption that the clinician would have changed their prescription practice to align with MMBV (eg, if MMBV result was bacterial then the clinician would have prescribed antibiotics and in cases with equivocal MMBV results, their practice would not change).

3. RESULTS

3.1. Study population characteristics

A total of 563 potentially eligible patients with suspicion of an acute infection were prospectively enrolled at EDs and UCCs in the United States and Israel (Figure 2), of which 87 patients were subsequently excluded (exclusions are listed in Table S1), leaving 476 patients who comprised the final study population. Of those, 372 were assigned a viral reference standard infection etiology, 44 were assigned a bacterial reference standard infection etiology, and the remaining 60 were assigned as indeterminate. The reference standard was based on the etiological labels assigned by three experts, and in 91.7% (341/372) and 93.2% (41/44) of cases, all three unanimously assigned a viral or bacterial label, respectively.

FIGURE 2.

Patient enrollment flow: Note that “viral” includes non‐infectious cases and “bacterial” includes bacterial and viral co‐infections. MMBV, MeMed BV.

Age range for the study population was from 5 months to 92 years, with 34% under 18 years (Table 1); the study population was balanced for gender (52.3% females and 47.7% males). At the time of presentation, 51.5% of patients presented after more than 2 days of symptoms and 2.1% had received antibiotics prior to enrollment. Approximately one‐quarter (25.6%) of the patients reported having at least one comorbidity, with hypertension being the most prevalent (12.0%). The most common ED/UCC discharge diagnosis was RTI, with 60.5% of the patients discharged with a diagnosis of upper RTI and 11.3% of the patients discharged with lower RTI. The majority of patients were discharged home while 13.2% were hospitalized. A microbiological finding was detected as part of routine care in 32.8% of the patients (Table S2A); all detections are listed in Table S2B.

TABLE 1.

Characteristics of eligible study cohort (n = 476).

Characteristic	Frequency
Demographics	Female, n (%)	249 (52.3)
	Adults, n (%)	314 (66.0)
	Age (years), mean (SD)	27.9 (19.4)
	Hispanic/Latino, n (%)	59 (12.4)
Race	Asian, n (%)	20 (5.4)
	Black or African American, n (%)	82 (22.0)
	Other a , n (%)	36 (9.7)
	White, n (%)	234 (62.9)
Current illness	Temperature (°C), mean (SD)	38.2 (1.1)
	Pre‐enrollment antibiotics, n (%)	10 (2.1)
	Days from symptom onset, median (IQR)	3.0 (2.0)
	Symptom duration > 2 days, n (%)	245 (51.5)
	Hospital admission, n (%)	63 (13.2)
	Days hospitalized, median (IQR)	4.0 (1.0)
Comorbidities	Chronic obstructive pulmonary disease, n (%)	5 (1.1)
	Diabetes, n (%)	27 (5.7)
	Hyperlipidemia, n (%)	27 (5.7)
	Hypertension, n (%)	57 (12.0)
	Ischemic heart disease, n (%)	6 (1.3)
Clinical syndrome b	URTI c , n (%)	288 (60.5)
	LRTI d , n (%)	54 (11.3)
	Other e , n (%)	80 (16.8)

Abbreviations: IQR, interquartile range; SD, standard deviation.

^a Each patient can only be included in one race. “Other” includes patients labeled as having more than one race.

^b The clinical syndrome relates to the discharge diagnosis. Patients can be included in more than one clinical syndrome.

^c URTI = upper respiratory tract infection; encompassed acute otitis media; croup; tonsillitis; acute upper respiratory infections; peritonsillar abscess; pharyngitis; pharyngitis due to infectious mononucleosis; acute sinusitis; aphthous stomatitis; bacterial sinusitis; coronavirus (Covid‐19) infection; Covid‐19. Scarlet fever; flu/influenza; herpangina; laryngitis; Rsv; respiratory viral syndrome; strep pharyngitis; tracheitis; viral uri; viral induced wheezing; wheezing‐associated respiratory infection; and viral rhinopharyngitis.

^d LRTI = lower respiratory tract infection; encompassed acute bronchiolitis; acute bronchitis; bronchopneumonia; COPD (chronic obstructive pulmonary disease) exacerbation; community acquired pneumonia; lobar pneumonia; multifocal pneumonia; occult or aytpical pneumonia; bacterial pneumonia; viral pneumonia; viral bronchitis; acute exacerbation of chronic bronchitis; and asthmatic bronchitis.

^e “Other” includes the following clinical syndromes: abdominal pain; abscess; appendicitis; asthma; cellulitis; febrile convulsions; fever; gastroenteritis; headache; and unspecified viral infection.

Patient characteristics stratified according to reference standard are provided in Table S3 and stratified according to children versus adults in Table S4. Hospital admission was significantly more frequent in children than adults (22.8% vs 8.3%).

3.2. MMBV performance

The AUC for MMBV was 0.95 (95% CI: 0.90–0.99), with a sensitivity and specificity of 90.0% (95% CI: 76.4–96.6) and 92.8% (95% CI: 89.5–95.1), respectively (Table 2); the impact of bacterial prevalence on PPV and NPV is modeled in Figure S2. Higher MMBV scores were statistically correlated with a higher likelihood of bacterial infection (Cochrane–Armitage p‐value < 0.0001; Table 3). Notably, only four of the 44 patients assigned a reference standard bacterial infection etiology received viral MMBV results (false negatives; Table S5). Conversely, 25 out of the 372 assigned a reference standard viral infection etiology had an MMBV result indicative of bacterial infection; 19 of 25 were in the score bin 65 < score < 90. MMBV's performance in adults was comparable to the entire cohort (see Table S6; Cochrane–Armitage p‐value < 0.0001). Only eight children were assigned a reference standard bacterial infection etiology; the performance of MMBV in children is given in Table S7 (Cochrane–Armitage p‐value < 0.0001).

TABLE 2.

MeMed BV (MMBV) performance for identification of bacterial versus viral infection, n = 416.

Sensitivity % (95% CI)	90.0 (76.4–96.6)
Specificity % (95% CI)	92.8 (89.5–95.1)
PPV % (95% CI)	59.0 (46.5–70.5)
NPV % (95% CI)	98.8 (96.8–99.6)
LR+ (95% CI)	12.5 (8.4–18.4)
LR− (95% CI)	0.1 (0.0–0.3)
Equivocal %	7.2
ROC‐AUC (95% CI)	0.95 (0.90–0.99)

Abbreviations: CI, confidence interval; LR+, positive likelihood ratio. LR−, negative likelihood ratio; NPV, negative predictive value; PPV, positive predictive value; ROC‐AUC, area under the receiver operating characteristic curve.

TABLE 3.

Interval bacterial likelihood ratios per MeMed BV (MMBV) score bin, n=416.

	No. of patients, n			% of cohort			% of bin
Score (s) bin	All	Bacterial reference standard infection etiology	Viral reference standard infection etiology	All	Bacterial reference standard infection etiology	Viral reference standard infection etiology	Bacterial reference standard infection etiology	Viral reference standard infection etiology	Bacterial likelihood ratio (95% CI)	MMBV result interpretation
90 ≤ s ≤100	30	24	6	7.2	54.5	1.6	80.0	20.0	33.82 (14.62–78.20)	High likelihood of bacterial infection (or co‐infection)
65 < s < 90	31	12	19	7.5	27.3	5.1	38.7	61.3	5.34 (2.78–10.25)	Moderate likelihood of bacterial infection (or co‐infection)
35 ≤ s ≤ 65	30	4	26	7.2	9.1	7.0	13.3	86.7	1.30 (0.48–3.55)	Equivocal
10 < s < 35	66	1	65	15.9	2.3	17.5	1.5	98.5	0.13 (0.02–0.91)	Moderate likelihood of viral infection (or other non‐bacterial etiology)
0 ≤ s ≤10	259	3	256	62.3	6.8	68.8	1.2	98.8	0.10 (0.03–0.30)	High likelihood of viral infection (or other non‐bacterial etiology)
Total	416	44	372	100	100	100

Abbreviation: CI, confidence interval.

The sensitivity, specificity, NPV, and PPV were similar when the reference standard infection etiology was adjudicated by experts blinded to CRP and PCT in addition to MMBV (see Supporting Information for details on CRP/PCT‐blinded adjudication and Table S8 for performance results). A comparable AUC for MMBV was demonstrated irrespective of sex, age, ethnicity, race, time from symptom onset, prior antibiotics, and hospitalization (Table S9). Also, a comparable AUC for MMBV was observed across patients recruited at the UCC versus ED settings (0.95 [95% CI: 0.85–1.00] vs. 0.94 [95% CI: 0.89–1.00]) and across URTI versus LRTI (0.94 [95% CI: 0.87–1.00] vs. 0.97 [95% CI: 0.90–1.00]).

3.3. MMBV distribution in patients with indeterminate reference standard infection etiology

As noted above, among the 476 patients in the study population, there were 60 patients for whom the expert adjudication panel was uncertain regarding whether the underlying etiology was bacterial or viral, despite the availability of comprehensive data including follow‐up; these cases were operationally labeled as reference standard infection etiology “indeterminate” and were not included in the performance calculations. The distribution of MMBV results for these 60 indeterminate cases with expert adjudication uncertainty is shown in Figure 3, of which 73% (44/60) received an MMBV result indicating either bacterial or viral infection, and 27% (16/60) were equivocal. Of note, 41.7% (25/60) had an MMBV result falling into the score bin 90–100 (interpreted as a high likelihood for a bacterial infection) or 0–10 (interpreted as high likelihood for a viral infection).

FIGURE 3.

MeMed BV (MMBV) distribution according to reference standard. Each dot represents a patient in the study population (n = 476). Red line corresponds to group median and red dot corresponds to group average.

3.4. MMBV comparison to PCT and current practice

MMBV significantly outperformed PCT (Figure 4) with an AUC of 0.95 (95% CI: 0.90–0.99) as compared to 0.70 (95% CI: 0.61–0.79). Applying a standard threshold of 0.25 ng/mL for PCT, MMBV also demonstrated superior performance in reclassification analysis (Table S10). Considering the 372 patients with a viral reference standard infection etiology, 74 were given antibiotics, of whom 60 got a MMBV result indicative of a viral infection. To conjecture regarding the overall potential impact on antibiotic prescription for viral reference standard infection etiology patients, cases where MMBV indicated bacterial infection were taken into account, and also cases with an equivocal result, for which clinical practice was assumed to be unchanged. Applying this analysis, use of MMBV could potentially have reduced antibiotic prescription in patients with a viral reference standard infection etiology by 2.2‐fold, from 19.9% (74/372; 95% CI: 16.1–24.3) to 8.9% (33/372; 95% CI: 6.4–12.2).

FIGURE 4.

Performance of MeMed BV (MMBV) versus procalcitonin (PCT) in differentiating between bacterial and viral infection, n = 416. MMBV outperformed PCT (p < 0.0001). AUC, area under the receiver operating characteristic curve; CI, confidence interval.

4. LIMITATIONS

First, enrollment was constrained by the work hours of the site research coordinator, which may introduce a bias in the included population. Second, the reference standard expert panel was provided with CRP and PCT values during adjudication, potentially introducing incorporation bias. However, use of an independent adjudication process with experts blinded to CRP and PCT generated an independent set of reference standard infection etiologies and yielded comparable diagnostic accuracy findings, as shown in Table S8. Third, the reference standard infection etiology was adjudicated without follow‐up data for 23.7% of patients. Fourth, the study population was made up principally of patients with low and/or moderate severity illness, and therefore, findings may not extend to patients with higher severity of illness. Fifth, we did not include immunosuppressed patients or infants under the age of 3 months, for whom an adjunctive test could be helpful. Sufficiently powered studies dedicated to determining assay performance in these populations are in planning. Finally, estimates for improved antibiotic decision‐making are based on a conjectured analysis that has intrinsic limitations, assuming idealized use of this test. Since the test is intended as an adjunctive data point and not intended to replace clinical judgement, future studies are needed to determine how the MMBV result impacts clinicians’ real‐world practice.

5. DISCUSSION

Bacterial and viral infection often present with similar symptoms and cannot be distinguished in a timely manner using tests available today. ¹³ This critical diagnostic uncertainty contributes to challenges in clinician decision‐making, suboptimal patient care and outcomes, as well as rising rates of antimicrobial resistance. The rapid diagnostic test MMBV derives from integration of several biomarkers, including not only bacterial (CRP) but also viral (TRAIL and IP‐10) host response proteins, which exhibit differential expression dynamics in response to acute infection due to their distinct biological pathways. ¹⁰ , ²⁷ , ²⁸ , ²⁹ The present study establishes that MMBV has high diagnostic accuracy for differentiating between bacterial and viral infections in a broad population of relatively low severity, febrile patients presenting to ED and UCC settings (13.2% admitted to hospital). Strengths of the study include a rigorous blinded design and collection of comprehensive patient data to support reference standard infection etiology determination. Another strength is the breadth of the study population, which included children and adults, a range of times from symptom onset, different comorbidities, multiple pathogens, as well as enrollment across different acute care settings, which supports the generalizability of the findings.

In this study population, MMBV attained sensitivity 90.0% (95% CI: 76.4–96.6) and specificity 92.8% (95% CI: 89.5–95.1), comparable to that reported in previous studies in children (sensitivity 93.7% [95% CI: 88.7–98.7], specificity 94.2 [95% CI: 92.2–96.1]) ¹¹ and adults (sensitivity 98.1 [95% CI: 95.4–100.0], specificity 88.4 [95% CI: 83.7–93.1]), ³⁰ supporting its accuracy. There were only four false negative and 25 false positive results, where 19 of 25 false positive results had scores in the moderate (rather than high) likelihood of bacterial infection (65 < score < 90). Previous studies similarly show that a small minority of false positives fall in the bin interpreted as high likelihood of bacterial infection (90 ≤ score ≤ 100). ¹¹ , ³⁰ It is notable that MMBV provided a distinct bacterial (score > 65) or viral (score < 35) result for over 90% of the patients in this diverse study population with reference standard infection etiologies. The remaining ∼10% of cases yielded “equivocal” MMBV results (35 ≤ score ≤ 65). An equivocal MMBV result represents a valid test result (i.e., it is not a failed test) that does not provide added diagnostic information to help determine etiology. In addition to the reclassification analysis, area under the curve analyses indicate that MMBV exhibits higher diagnostic accuracy than PCT in this population. Of note, the latter analysis includes patients with equivocal scores.

Regarding the adjudicated reference standard for determining infection etiology, despite having results of routine tests as well as study‐specific respiratory panel data and (in most cases) follow‐up data, the expert panel adjudication left a small proportion of cases (60/476, 13%) as indeterminant regarding bacterial versus viral infection etiology. This value aligns with findings from a study directly assessing the impact of expert panel size on indeterminant rates. ¹² Of note, an MMBV score ≥ 90 (interpreted as high likelihood of bacterial infection) or ≤10 (interpreted as high likelihood of viral infection) was attained for over 40% of these reference standard indeterminate cases, underscoring the potential of MMBV to assist clinicians even in difficult‐to‐diagnose cases.

Since MMBV results were not provided to the treating clinician, the test's impact on actual antibiotic prescription decisions cannot be evaluated by the present study. However, we can estimate MMBV's potential to impact clinical practice by comparing the test result to actual antibiotic use as documented in the medical record. We find that MMBV could potentially have reduced antibiotic prescription in patients with a viral reference standard infection etiology 2.2‐fold. Notably, this likely overestimates impact given that MMBV is not intended for use as a standalone test.

To conclude, the present study offers an additional step toward a new potential paradigm in which the evaluation of acute care patients with suspected infections includes use of an additional diagnostic adjunctive, which could help achieve the overarching goal of reducing diagnostic uncertainty, with improved antibiotic stewardship and patient outcomes.

AUTHOR CONTRIBUTIONS

Richard G. Bachur, Sheldon L. Kaplan, and Richard E. Rothman participated in discussions with the U.S. Food and Drug Administration clearance and were involved in study design, patient recruitment, data collection and analysis, and drafting and revising the manuscript. Cesar A. Arias, Natasha Ballard, Karen C. Carroll, Andrea T. Cruz, Richard Gordon, Salim Halabi, Jeffrey D. Harris, Kristina G. Hulten, Theresa Jacob, Mark D. Kellogg, Adi Klein, Pninit Shaked Mishan, Sergey M. Motov, Octavia M. Peck‐Palmer, Leticia M. Ryan, Ma'anit Shapira, George S. Suits, Henry E. Wang, and Alexandra Weissman were involved in patient recruitment, data collection, and drafting and revising the manuscript. All authors confirm that they had full access to all the data in the study, reviewed and approved the final version of the manuscript, and accept responsibility to submit for publication.

CONFLICT OF INTEREST STATEMENT

Cesar A. Arias, Natasha Ballard, Karen C. Carroll, Andrea T. Cruz, Richard Gordon, Salim Halabi, Jeffrey D. Harris, Kristina G. Hulten, Theresa Jacob, Mark D. Kellogg, Adi Klein, Pninit Shaked Mishan, Sergey M. Motov, Octavia M. Peck‐Palmer, Leticia M. Ryan, Ma'anit Shapira, George S. Suits, Henry E. Wang, Alexandra Weissman, and Richard E. Rothman have no relevant conflict of interests to declare. Richard G. Bachur and Sheldon L. Kaplan participated in a scientific advisory board on health economic modeling for MeMed and were compensated for their time. Richard E. Rothman participated in a scientific board on health care economic modeling (without compensation). Richard G. Bachur, Sheldon L. Kaplan, and Richard E. Rothman participated in discussions with the U.S. Food and Drug Administration clearance and were involved in study design, patient recruitment, data collection and analysis, and drafting and revising the manuscript.

FUNDING INFORMATION

MeMed

Supporting information

Supporting information

Supporting information

Supporting information

Supporting information

Biography

Richard G. Bachur, MD, is a Professor of Pediatrics and Emergency Medicine at Harvard Medical School in Boston, MA. He is the Chief in the Division of Emergency Medicine and Senior Physician in Pediatrics, President of the Physicians Organization, and Associate Chief Medical Officer and Medical Director of Access at Boston Children's Hospital.

Bachur RG, Kaplan SL, Arias CA, et al. A rapid host–protein test for differentiating bacterial from viral infection: Apollo diagnostic accuracy study. JACEP Open. 2024;5:e13167. 10.1002/emp2.13167