Metagenomic Sequencing as a Pathogen-Agnostic Clinical Diagnostic Tool for Infectious Diseases: a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies

ABSTRACT

Metagenomic sequencing is frequently claimed to have the potential to revolutionize microbiology through rapid species identification and antimicrobial resistance (AMR) prediction. We assess the progress toward these developments. We perform a systematic review and meta-analysis of all published literature on culture-independent metagenomic sequencing for pathogen-agnostic infectious disease diagnostics up to 12 August 2020. Methodologic bias and applicability were assessed using the tool Quadas-2. (Prospero CRD42020163777). A total of 2,023 clinical samples from 13/21 eligible diagnostic test accuracy studies were included in the meta-analysis. Reference standards were culture, molecular testing, clinical decision, or a composite measure. Sensitivity and specificity in the most widely investigated sample types were 90% (95% confidence interval [CI], 78% to 96%) and 86% (45% to 98%) for blood, 75% (54% to 89%) and 96% (72% to 100%) for cerebrospinal fluid (CSF), and 84% (79% to 88%) and 67% (38% to 87%) for orthopedic samples, respectively. We identified a limited use of controls, especially negative controls which were used in only 62% (13/21) of studies. AMR prediction and comparison to phenotypic results were undertaken in four studies; categorical agreement was 88%(80% to 97%), and very major and major error rates were 24% (8% to 40%) and 5% (0% to 12%), respectively. Better human DNA depletion methods are required; a median 91% (interquartile range [IQR], 82% to 98%; range, 76% to 98%) of sequences was classified as human. The median (IQR; range) time from sample to result was 29 hours (24 to 94; 4 to 144 hours). The reported consumable cost per sample ranged from $130 to $685. There is scope for improving the quality of reporting in clinical metagenomic studies. Although our results are limited by the heterogeneity displayed, our results reflect a promising outlook for clinical metagenomics. Methodological improvements and convergence around protocols and best practices may improve performance in the future.

INTRODUCTION

The term metagenomics first appeared in 1998 which referenced the idea that a collection of genes could be sequenced in a single sample without the need for isolation or lab cultivation of a specific species (1). In infectious diseases, metagenomics has the potential to be a disruptive technology that revolutionizes diagnostics, in place of traditional culture-based microbiology which has remained fundamentally unchanged over decades. The complete metagenomic pipeline from sample collection to report is shown in Fig. 1. Traditionally, most diagnostic microbiology depends on culture to identify the causative organism in an infection and to define antimicrobial susceptibilities. However, this process can take several days or, in the case of Mycobacteria sp. and other slow-growing organisms, several weeks. While waiting for results, patients are treated empirically, which may result in unnecessarily broad-spectrum treatment in some and ineffective treatment in others. Additionally, culture is imperfectly sensitive, e.g., many patients with serious infection have negative blood cultures, and culture may be impaired by prior antimicrobial exposure.

FIG 1.

Metagenomic flow diagram. Patient samples undergo nucleic acid extraction. Laboratory methods may be used to reduce human DNA present to improve pathogen sequencing efficacy. Reverse transcription may be utilized to convert RNA to cDNA for sequencing. Optional microbial enrichment is performed to further increase pathogen concentration. Random or specific PCR may be performed if a higher DNA concentration is required. Sequencer specific library preparation kits are used, which may include inherent PCR steps. Purification of the sample is often performed using bead cleaning to remove inhibitors. Samples are then loaded onto the sequencer for sequencing.

Molecular diagnostics based on the detection of antigens, antibodies, or nucleic acid are increasingly used to complement culture-based diagnosis. For example, multiplex PCR syndromic panels are able to identify common pathogens and antimicrobial resistance determinants within a few hours. However, these panels can identify only a restricted number of pathogens and resistance determinants and are therefore typically bespoke for a given clinical syndrome.

In contrast, emerging real-time sequencing technologies potentially offer a pathogen and clinical syndrome agnostic platform for identifying within hours any pathogens and known resistance determinants present. Here, we evaluate progress toward this ambition. Studies have demonstrated that metagenomic pipelines can provide faster diagnoses than conventional culture-based methods which may be crucial for critically unwell patients (2). Additionally, the detection of pathogens from culture-negative samples may have clinical management implications in several other settings, including prosthetic joint infections, chronic wound infections, and endocarditis (3–5). Where sufficient knowledge bases exist, prediction of antimicrobial resistance is potentially possible and detection of mixed infections is also possible, where a subpopulation of resistant strains can be detected (6).

In this review, our objectives were to assess all current literature published on culture-independent metagenomic sequencing for pathogen-agnostic infectious disease diagnostics in participants with potential infection and to describe the specific challenges and progress in the field.

METHODS

Search strategy and selection criteria.

We conducted a systematic review on all available literature up to 13 August 2020 in accordance with the Preferred Reporting System for Systematic Reviews and Meta‐Analyses for Diagnostic Test Accuracy (PRISMA-DTA) guidelines (7). We searched PubMed and Google Scholar using a combination of relevant medical subject headings (MeSH) terms (see Table S1 in the supplemental material). Abstracts of studies identified were screened using the inclusion and exclusion criteria specified in Table S2 in the supplemental material. We focused on pathogen-agnostic sequencing studies and excluded pilot studies, namely, those with <10 samples, and those focused on diagnosis of a specific pathogen or group of pathogens. The reference lists of included manuscripts were also manually searched for relevant studies not identified during the literature search. We assessed methodologic quality and risk of bias using the Quadas-2 tool for diagnostic accuracy studies and excluded any study with a high overall risk of bias (8). Study inclusion, extraction of data, and quality evaluation were assessed by two independent reviewers, and any differences were resolved through discussion or by a third reviewer. Peer-reviewed articles that utilized metagenomic sequencing as a tool for clinical diagnostics in infectious diseases were appraised, and relevant data were extracted where possible. Principle diagnostic accuracy measures were sensitivity and specificity per clinical sample result. Antimicrobial resistance prediction was assessed using the resistance prediction outcome over each combination of antibiotic and sample tested. This study has been registered with the Prospero prospective register of systematic reviews (reference CRD42020163777).

Meta-analysis.

The index test was defined as the metagenomic sequencing test method, while the reference standard defining true positives and true negatives was based on a pragmatic approach of each study’s best measure of the presence or absence of infection with a specific species using either culture, molecular testing, clinical decision, or a combination of these factors. Additional species identified as plausible by studies including polymicrobial infections were considered additional true pathogens and contributed to the numerator and denominator in the sensitivity analysis. Specificity was determined by considering the proportion of samples deemed negative by the reference standard that were identified as negative by metagenomic sequencing. Although it is plausible to consider that any sample can contain a false-positive result, including culture-positive samples, studies differed considerably in reporting this result. Therefore, we included only reference standard negative samples in specificity calculations. Studies where sensitivity could be determined but not specificity were included in the meta-analysis; however, further analyses excluding these studies were also performed.

We used the R meta package for the statistical analysis (9). Outcomes are reported for each study with 95% confidence intervals (CIs), and pooled estimates are provided by using fixed or random-effects models, depending on the absence or presence of heterogeneity, respectively, using the I² test. An I² value of >50% was considered heterogenous.

RESULTS

A total of 551 manuscripts were screened and 530 were excluded (see Fig. S1 in the supplemental material), leaving 21 eligible for inclusion. Specificity could not be determined in 8 studies, leaving 13 studies to be included in the meta-analysis (Table 1).

TABLE 1.

Characteristics of studies included in this systematic review

Study	Type of sample	Sequencing technique	Pathogen(s) found	Reference standard	No. of positive samples	No. of negative samples	Sensitivity (95% CI)	Specificity (95% CI)
Hong et al. (26)	Cerebrospinal fluid	Illumina MiSeq (CA, USA)	Viruses	PCR	19	47	0.74 (0.49–0.91)	0.66 (0.51-0.79)
Miller et al. (24)	Cerebrospinal fluid	Illumina HiSeq	Bacteria, fungi, parasites, DNA viruses, RNA viruses	Conventional laboratory results and additional molecular testing	56	118	0.89 (0.78–0.96)	0.99 (0.95–1)
Wilson al. (25)	Cerebrospinal fluid	Illumina HiSeq	Bacteria, fungi, parasites, DNA viruses, RNA viruses	Conventional laboratory results and additional molecular testing	58	149	0.55 (0.42–0.68)	0.98 (0.94–1)
Blauwkamp et al. (14)	Plasma (cfDNAa)	Illumina NextSeq 500	Bacteria, DNA viruses, fungi, eukaryotic, parasites	Conventional laboratory results and additional molecular testing	182	166	0.93 (0.88–0.96)	0.63 (0.55–0.70)
Parize et al. (10)	Plasma	Ion Proton (Life Technologies, USA)	Bacteria, viruses	Culture, serological diagnosis and PCR	19	82	0.63 (0.38–0.84)	0.71 (0.60–0.80)
Somasekar et al. (13)	Serum	Illumina HiSeq	Viruses	PCR	26	246	0.96 (0.80–1)	1 (0.98–1)
Rossoff et al. (12)	Plasma	Illumina NextSeq 500	Bacteria, DNA viruses, fungi, eukaryotic parasites	Clinical review	61	39	0.92 (0.82–0.97)	0.64 (0.47–0.79)
Schlaberg et al. (19)	Respiratory samples	Illumina HiSeq 2500	Viruses	Clinical review	60	59	0.90 (0.79–0.96)	0.64 (0.51–0.76)
Street et al. (20)	Sonication/synovial fluid	Illumina MiSeq	Bacteria	Culture	69	97	0.88 (0.78–0.95)	0.88 (0.79–0.93)
Thoendel et al. (21)	Sonication/synovial fluid	Illumina HiSeq 2500	Bacteria	Culture	146	67	0.83 (0.76–0.89)	0.69 (0.56–0.79)
Ivy et al. (3)	Sonication/synovial fluid	Illumina HiSeq 2500	Bacteria, fungi	Culture	82	60	0.82 (0.72–0.89)	0.35 (0.23–0.48)
Doan et al. (27)	Intraocular fluid samples	Illumina HiSeq 4000	Bacteria, fungi, parasites, DNA viruses	PCR	31	36	0.87 (0.70–0.96)	0.78 (0.61–0.90)
Cheng et al. (5)	Heart valves	Illumina HiSeq	Bacteria	Clinical review	41	7	0.98 (0.87–1)	0.86 (0.42–1)

^acfDNA, circulating free DNA.

Clinical samples and range of pathogens.

The range of clinical samples included blood (10–14), positive blood cultures (2), urine (15), respiratory samples (16–19), orthopedic device sonication fluid and other bone and joint samples (3, 20–22), cerebrospinal fluid (23–26), intraocular fluid (27), and heart valve tissue (5). The range of pathogens found are detailed in Table 1. The median number of samples sequenced per study was 67 (IQR, 44 to 166; range, 12 to 348).

Methodology. (i) Nucleic acid extraction and sequencing library preparation.

Methods to deplete human cells prior to DNA extraction were used in 8/21 (38%) of studies (see Table S3 in the supplemental material), including differential centrifugation (2, 15, 22) or filtration with a 5-μm membrane (2, 11, 20) to remove human cells, based on their larger mass or size respectively, and the MolYsis Basic5 kit (Molzym, Bremen, Germany) (3, 21) which is reported to differentially lyse human cells.

Nucleic acid extraction kits and techniques were heterogeneous across studies (Table S3). Most studies utilized commercial kits, primarily based on membrane/column methods, while noncommercial methods relied mostly on ethanol precipitation or lysis followed by bead-cleaning methods. A total of 3/21 (14%) studies performed a postextraction enrichment step to facilitate enrichment of microbial DNA from clinical samples, and 7/21 (33%) studies used a DNA amplification method prior to library preparation to achieve the required input DNA concentrations (Table S3). DNA cleanup with magnetic beads was used in 9/21 (43%) studies to purify extracted DNA prior to sequencing.

(ii) Controls.

Studies were assessed for three types of controls as follows, (i) an internal or spiked control, (ii) a negative control, and (iii) a positive control. Only 4/21 (19%) studies reported internal control use and included bacteriophages and synthetic DNA (12, 14, 24, 25). A total of 13/21 (62%) studies reported use of a negative control. The median (IQR; range) of negative-control samples per every 10 samples processed was 1.5 (1.0 to 1.5; 0.5 to 1.5). Only 6/21 (29%) studies reported use of a positive control, including using a mixture of 7 representative organisms and a single Corynebacterium glutamicum ATCC 13032 positive control (3, 13, 14, 17, 24, 25).

Sequencing technologies.

Illumina technology (CA, USA) was used exclusively in 16/21 (76%) studies (see Table S4 in the supplemental material). One study exclusively used MinION (Oxford Nanopore Technologies [ONT], UK) (16), and two studies exclusively used Ion Proton (Life Technologies, USA) (10, 15). One study compared the use of Illumina sequencing versus ONT MinION (2). The BGISEQ-500/50 (BGI, Tianjin, China) platform was used in one study (23).

Bioinformatic approaches.

Figure S2 in the supplemental material outlines the bioinformatic workflows implemented. The most commonly used classification software packages for species identification were alignment based (n = 11) followed by k-mer based (n = 8) and marker-gene based (n = 1) (see Table S4 for details). The most commonly used species databases originate from the NCBI and included the NCBI RefSeq database (n = 9), unspecified NCBI databases (n = 7), MetaPhlAn2 database (n = 3), NCBI GenBank (n = 1), and Karius proprietary database (n = 1).

Assessing true species presence versus contamination.

Various strategies were employed to identify the true presence of an organism versus contamination. The most commonly used methods include an empirical approach setting absolute thresholds based on sequencing known negative and positive samples (n = 4) (2, 3, 24, 25) while other studies (n = 3) used a parametric approach based on the observed distribution of contaminating reads in control and other samples (14, 17, 21). Other strategies are described in Fig. S3. By use of negative controls and other approaches, 16/21 (76%) studies report contamination of some degree, while 5/21 (24%) had no details of contamination available. No study reported no evidence of contamination.

Performance. (i) Species identification.

Methodological quality was assessed using the Quadas-2 tool (see Fig. S4 in the supplemental material) (8). Risk of bias was most commonly seen in the patient domain with 2/21 (10%) studies having high risk and 3/21 (14%) having some concern; all were related to the selection of specific samples for study, rather than use of a random or consecutive sample. This assessment was followed by the index test domain where 4/21 (19%) studies had some concern relating to the reporting of bioinformatic thresholds and methods to assign a result as positive or negative (see Fig. S6 in the supplemental material).

A meta-analysis of species identification performance was conducted by sample type on 13/21 studies where both test sensitivity and specificity could be determined. Sensitivity in the most widely investigated sample types of blood (n = 288), cerebrospinal fluid (CSF) (n = 133), and orthopedic fluid (n = 297) was 90% (78% to 96%), 75% (54% to 89%), and 84% (79% to 88%), respectively (Fig. 2). This varying performance may reflect the number of organisms present in each sample type and the relative abundance of human cells and any other bacteria present. Among CSF studies, Wilson et al. reported lower sensitivity, possibly reflecting a greater effort to obtain a diagnosis in the reference standard used for comparison, including culture and multiplex molecular diagnostics, and to obtain a variety of samples, including tissue biopsy specimen and abscess fluid (25). Furthermore, conservative preestablished bioinformatic thresholds were used which resulted in negative results even though species-specific reads might have been used to identify the appropriate organism at lower thresholds. Among studies on blood, those with low sample sizes generally performed worse, potentially due to limited experience accrued while conducting these studies (10).

FIG 2.

Pooled sensitivity of studies grouped by sample type. This analysis of sensitivity included 13 studies. A random effects model was used due to heterogeneity between studies.

Specificity in blood (n = 533), CSF (n = 314), and orthopedic samples (n = 224) was 86% (45% to 98%), 96% (72% to 100%), and 67% (38% to 87%), respectively (Fig. 3). CSF studies performed better than blood and orthopedic studies in part because preestablished conservative thresholds were used, trading off sensitivity for higher specificity (24, 25). Specificity performance varied substantially across orthopedic studies due to nonstandardized thresholds for determining a negative result in the presence of high contamination and low DNA extraction yields, resulting in many organisms of unknown clinical significance (Fig. 3). A total of 8/21 (38%) studies did not report specificity and were excluded from the meta-analysis. A total of 20/21 (95%) studies found a median of 7 (IQR, 4 to 15; range, 1 to 6) additional pathogens with subsequent clinical interpretation varying throughout. In this analysis, positive and negative predictive values grouped by sample type range between 72% and 98% and 75% to 90% respectively, and diagnostic accuracy between 77% and 95% (see Table S7 in the supplemental material).

FIG 3.

Pooled specificity of studies grouped by sample type. A random effects model was used due to heterogeneity between studies.

Using reported per-study means, a median 91% (IQR, 82% to 98%; range, 76% to 98%) of sequence data was classified as human DNA. Blood samples had the highest median reported of 91% (91 to 98; 84 to 98), and sonication and bone samples had a median of 87% (82 to 93; 76 to 98).

(ii) Antimicrobial resistance prediction.

Nine studies attempted genotypic drug susceptibility prediction and compared results to either drug susceptibility phenotype (n = 4) or had no comparison (n = 5). Four studies that compared results to drug susceptibility phenotypes were included in a meta-analysis (see Table S5 in the supplemental material). Performance depends on the prevalence of resistance and the specific pathogen-antimicrobial combinations tested. In the 4 eligible studies, the median (IQR; range) number of clinical samples per study was 29 (22 to 35; 17 to 39). A median of 196 (45 to 350; 35 to 369) antibiotic predictions was performed per study. Beta-lactams, quinolones, and tetracyclines were some of the most commonly tested classes of antibiotics, and 19% (153/795) of pathogen-antimicrobial combinations were phenotypically resistant (Table S5). All studies evaluated prediction of the correct antimicrobial susceptibility as a categorical value, i.e., sensitive or resistant. Categorical agreement rates were 88% (95% CI, 80% to 97%) across studies using a random effects model. The pooled estimate of very major error rates, defined as the number of samples with phenotypic resistance not predicted by genotype over all samples with phenotypic resistance, was 24% (95% CI, 8% to 40%). The pooled major error rate, defined as predicted resistance by genotype but sensitive phenotype over all phenotypically sensitive samples, was 5% (95% CI, 0% to 12%) (Fig. 4). There was marked heterogeneity in all three metrics (I², 0.86 to 0.92; all P < 0.01).

FIG 4.

Pooled antimicrobial resistance prediction performance. This analysis included 4 studies that focused on species-agnostic samples and compared this to drug susceptibility phenotype. Proportions were combined using a random intercept logistic regression model. A random effects model was used due to heterogeneity between studies.

Time to complete process.

A total of 9/21 (43%) studies reported sample processing times (see Fig. S7 in the supplemental material). Based on the mean time provided from each study, sample extraction/preparation took a median of 3 (IQR, 3 to 3; range, 3 to 3) hours and library preparation a median of 4 (4 to 4; 4 to 4) hours. Reported sequencing using Illumina took 16 (16 to 16; 16 to 16) hours. Where sequence analysis was undertaken in real time using the Nanopore platform, first species prediction took 1 (1 to 1; 1 to 1) hour. The total time from sample extraction to species identification and AMR prediction was 29 (24 to 94; 4 to 144) hours.

Cost.

Only one study reported an average cost estimation per sample which varied from $130 when samples are multiplexed up to $685 for processing a single sample (16).

DISCUSSION

In this review, we describe all studies found to date assessing the performance of metagenomic sequencing as a tool for pathogen-agnostic diagnosis of clinical infection on a variety of clinical samples, including blood, urine, respiratory, cerebrospinal fluid, orthopedic sonication and synovial fluid, intraocular fluid, and cardiac valvular tissue samples. Sensitivity in the most widely investigated sample types of blood (n = 288), CSF (n = 133), and orthopedic fluid (n = 297) was 90% (78% to 96%), 75% (54% to 89%), and 84% (79% to 88%) respectively (Fig. 2). Specificity in blood (n = 533), CSF (n = 314), and orthopedic samples (n = 224) was 86% (45% to 98%), 96% (72% to 100%), and 67% (38% to 87%), respectively (Fig. 3). Analysis by sample type subgroups demonstrate development of workflows for CSF, blood, and orthopedic fluid with reproducible performance (Fig. 2 and 3).

While these results reflect impressive progress in the field, the level of accuracy required is still being broadly assessed, as metagenomics is unlike any other patient diagnostic with specific advantages and disadvantages reviewed elsewhere (28, 29). However, it should be remembered that the current gold standard of culture-based microbiology is imperfect; for example, two standard blood cultures will result in missing at least 10% to 18% of episodes with potentially culturable organisms (30). Additionally, as metagenomics may detect potential pathogens not found by culture, reported specificity may be reduced because of the use of an imperfect reference standard. For example, a median of 7 (IQR, 4 to 15; range, 1 to 62) additional pathogens was found per study. Besides culture, metagenomics can be benchmarked against multiplex syndromic panels; however, ultimately, expert clinical review is required to evaluate the plausibility of additional potential pathogens detected.

While species identification may change therapy in some settings, e.g., as a result of intrinsic resistance or local epidemiology, rapid and precise antimicrobial resistance prediction linked to specific pathogens would greatly improve the time to administration of targeted antimicrobials. Although the categorical agreement rate with phenotyping was 88% (95% CI, 80% to 97%), major and very major error rates were 5% (95% CI, 0% to 12%) and 24% (95% CI, 8% to 40%) respectively, which are well above the current regulatory thresholds. the FDA requires major errors of <3% and the 95% CI on the very major error rate to be ≤7.5% at the upper limit and ≤1.5% at the lower limit (31). It is clear that improvements are needed for AMR prediction to reach acceptable performance, including better understanding and cataloging of the mechanisms underlying antimicrobial resistance and optimized algorithms to detect them within metagenomic data and link them to specific pathogens. This is particularly challenging for plasmid-mediated resistance which may be difficult to assign to a specific species from metagenomic data. Increased yield of pathogen DNA will also likely improve performance. There is a need to establish optimal extraction methods; only one study was found that compared different methods (2). Furthermore, a median of 91% (IQR, 82% to 98%; range, 76% to 98%) of sequence reads was classified as human even after applying known human depletion laboratory techniques, necessitating the need for improved human DNA depletion methods. Detailed information on library preparation methods can be found in a review by Head et al. (32).

Quality control is imperative for reliable metagenomic results and will be essential for regulatory approval for clinical applications. Only 19% and 29% of studies used internal and positive controls, respectively. Similarly, only 62% of studies used negative controls. We recommend all metagenomic studies consider carefully using all three forms of controls to detect and reduce errors.

The ability of metagenomics to detect microbes agnostically and at very low concentrations is a major advantage; however, this also comes with the ability to detect contaminants much more easily which poses a major challenge, especially after DNA amplification. Meticulous laboratory practice is required to minimize contamination risks but some residual contamination is likely, as 16/21 (76%) studies reported contamination to some degree. Studies used various approaches to classify contamination described in Fig. S3; however, there is no convergence on how to exclude contamination and bioinformatically confirm the presence of infection.

It is yet unclear which sequencing platform will emerge as the ideal device for metagenomic sequencing. Successful platforms will need to be scalable and flexible for different batch sizes and offer stable and predictable performance. For detailed information on sequencing platforms, a review is available by Goodwin et al. (33). Current real-time sequencing devices such the Oxford Nanopore technology yield rapid data, which may be a significant development for infection diagnostics but are limited by high per-read error rates which can be problematic especially if only low coverage depth is achieved of the pathogen genome.

As the quantity of microbial genetic data increases, bioinformatic approaches need to be optimized to handle large data sets of complex metagenomic data while minimizing computing power and memory usage. However, accredited workflows that can be reliably used by nonexpert users will be needed for metagenomics to be adopted widely. This will also involve adapting and developing laboratory information management systems to handle different workflows and classes of data.

In severe infection, it has been demonstrated that a time interval of >1 day before initiation of appropriate antimicrobial therapy is associated with a 2-fold increase in the risk of mortality (34). Clinical metagenomics besides identifying fastidious organisms not identified by culture may provide a species and antimicrobial resistance results within 4 to 7 hours in some cases (Fig. S7), instead of the conventional 24 to 72 hours with culture. However, practically, this may be limited by the number of times that a lab is able to perform sequencing runs within given time intervals.

Koch’s postulates have played an important role in establishing causal relationships in microbiology and yet have major limitations, including the fact that we now know of many pathogens that cannot be conventionally cultured. A reconsideration of these postulates which is relevant and aligns to metagenomics is proposed by Fredericks and Relman (35). If metagenomics can be shown to be diagnostically accurate, it will then become important to understand the clinical correlation and implication of positive results when all other assays are negative. This will necessitate large-scale interventional and longitudinal studies to establish the implications for patient outcomes. Additionally, economic evaluations will be needed. The average consumable cost per sample for clinical metagenomics is reported to range from $130 when samples are multiplexed up to $685 for processing a single sample at once. In comparison, the average running cost for blood culture is estimated at <$50 per sample (36). However, costs associated with changes in patient therapy and management from metagenomics and any impact on outcomes will also need to be considered.

Clinical laboratories are highly regulated, with requirements that vary by jurisdiction; for example, in the United States, FDA clearance is required, while in the European Union, a Conformitè Europëenne (CE) mark is required for diagnostic platforms. The FDA intends to regulate infectious disease metagenomic high-throughput sequencing devices as systems, including all the components necessary to generate a result. Detailed information including quality control, validation, and continuous monitoring may be found in the draft guidance by the FDA (37).

Our study had several limitations. First, the meta-analysis provided does not overcome problems inherent in the design and execution of the primary studies. Furthermore, it does not correct bias as a result of selective publication. A total of 8/21 (38%) of studies did not include negative test results, limiting the ability to ascertain false-positive results or contamination within these studies; therefore, they were excluded from the meta-analysis. Using the Quadas-2 tool, the main risk of bias arose from selection of specific samples for study, rather than consecutive samples or a random subset which are recommended. Similarly, to avoid index test bias, we recommend studies provide further clarity on bioinformatic procedures when determining a positive or negative result. Although studies were grouped by sample type, any observed difference may be confounded by other factors, such as differing clinical application, laboratory, and bioinformatic methods or the fact that reference standards differed among culture, molecular testing, clinical decision, or a composite measure (Table S3 to S5). Due to the heterogeneity between studies, we are not able to provide specific recommendations on the ideal metagenomics workflow. But, we anticipate that by bringing studies to date together, this review will provide a mechanism for improving standards and designing better studies in the future. We suggest future research conform to requirements set by the STROBE-metagenomics guideline, STARD, or other equivalent guidelines to improve the quality of study design and reporting (38, 39).

CONCLUSION

With further advances in extraction methods, sequencing technology, and bioinformatic processes, the proficiency of clinical metagenomics is likely to improve. Culture-based diagnostics have remained the backbone of the microbiology lab for over a century (40); however, clinical metagenomics has the potential to be the next frontier of clinical microbiology by not only enhancing our understanding of infectious diseases but also detecting polymicrobial infections and pathogens that are conventionally unculturable. These improvements may lead to more effective and narrow-spectrum treatment options reducing antimicrobial resistance and leaving the microbiome unchanged. It is likely that clinical metagenomics will be a part of the clinician’s armamentarium in the future for identifying and guiding therapy against complex infectious diseases. However, convergence around protocols and further metagenomic studies of improved quality and with higher sample sizes are required for the field to progress further.