Impact of direct from clinical sample sequencing assays for infectious diseases diagnostics: A single-centre retrospective cohort study

Abstract

Background:

The analytical performance of bacterial targeted sequencing (BTS), fungal targeted/panfungal sequencing (FTS), and metagenomic next-generation sequencing (mNGS) assays has been previously evaluated and their clinical use is increasing. Limited evidence is available on their true clinical impact on infectious disease diagnosis and treatment.

Methods:

We conducted a 3-year retrospective cohort study including all patients for whom broad-range sequencing assays were performed directly from clinical samples for the detection of bacterial and fungal pathogens. The operational characteristics, diagnostic and therapeutic impacts of the assays were assessed by reviewing patient clinical files and laboratory information system charts.

Results:

A total of 279 samples from 185 patients were included. The positivity rates for BTS, FTS, and mNGS were respectively 20.5% (47/229), 20% (9/45), and 20% (1/5). Of these 279 samples, 40 (14.3%) had an impact on patient management. The test results helped to establish a diagnosis in 26 (9.3%) cases and led to treatment modifications in 14 (5%). FTS achieved higher impact rates (26.7%) than both BTS (12.2%) and mNGS (0%). Short turnaround times increase impact rates, and the most impactful tests were those performed on bone and intervertebral disc samples, or in patients with negative culture results due to prior antibiotic administration.

Conclusions:

In this study, the overall diagnostic impact of BTS and FTS was high. Both the diagnostic and treatment impact of those assays can be increased if prescribed in well-selected clinical syndromes and performed on well-selected clinical samples.

Introduction

Rapid and accurate pathogen detection is central to infectious diseases (ID) clinical practice and antimicrobial stewardship. Timely diagnosis is paramount in reducing morbidity and mortality, avoiding unnecessary antimicrobial use, and limiting antimicrobial resistance (1–4). Direct from clinical sample DNA/RNA sequencing is increasingly used by clinicians as an agnostic tool to complement conventional microbiology testing, such as culture and pathogen-specific nucleic acid amplification assays. Targeted sequencing assays such as 16s rRNA sequencing (bacterial targeted sequencing [BTS]) and ITS/D1D2 sequencing (targeted fungal/panfungal sequencing [FTS]) can be performed on a variety of samples, whereas metagenomic next-generation sequencing (mNGS) can be ordered on selected sample types. Studies have demonstrated their accuracy and analytical sensitivity in comparison with alternative diagnostic methods (5,6). Flurin et al demonstrated on 2,146 various tissue and body fluid samples that BTS and FTS were more sensitive than culture (53% versus 42%, P < 0.001) (7). In a cohort of 350 patients with sepsis, Blaukamp et al reported that microbial cell-free DNA sequencing agreed with conventional culture in 93.7% of cases and identified an etiologic agent more frequently than all other diagnostic methods combined (169 versus 132 pathogens) (5). In a collection of 95 cerebrospinal fluid samples, Miller et al. reported that metagenomic NGS had 73% sensitivity and 99% specificity for pathogen identification compared with conventional microbiology testing (6).

These assays enable pathogen identification in hypothesis-free investigations, in cases of false-negative cultures due to prior antimicrobials use, or when no other pathogen-specific assay is available for a suspected infection (8–12). These assays may also accelerate microbial diagnosis by avoiding the need for culture amplification. Direct-from-sample sequencing has been successfully used in clinical settings for the diagnosis of respiratory, central nervous system, cardiovascular, and osteoarticular infections, and is now recommended in infectious endocarditis diagnostic guidelines, highlighting integration in routine clinical practice (5,6,10,12–17). Despite growing availability and enthusiasm, data on clinical impact remain limited, with most studies involving few patients or sample types (7,8,11,15,18–23). In this single-3-year retrospective cohort study, we evaluated the diagnostic and therapeutic impacts of direct sequencing assays on clinical sample as well as their operational characteristics in a tertiary care hospital.

Methods

Direct sequencing methods

Three distinct direct-from-sample sequencing assays were included—BTS and FTS, which are send-out assays performed at the Quebec provincial public health laboratory (Laboratoire de Santé Publique du Québec), and RNA/DNA mNGS on cerebrospinal fluid, which is a send-out assay performed at University of California San Francisco. The clinical samples were processed, transported, and analyzed according to reference laboratories guidance and previously published protocols (Supplementary materials 1 & 2) (6,7).

Study design

This single-, centre retrospective cohort study was conducted at the Centre Hospitalier de l'Université de Montréal (CHUM), a 772-bed tertiary care adult referral hospital. All direct-from-sample sequencing tests performed over a 3-year period were included. Data were extracted from hospital electronic medical records (EMRs) and laboratory information systems. Extracted variables included demographic, clinical, laboratory, and pharmacological information. Clinical charts were reviewed for each test to assess how results were interpreted and what actions clinicians took based on the reported information. Tests were classified by indication and timing of request. A first-line test was defined as one ordered before the final conventional culture results, which could be negative or positive. Second-line tests were ordered after final negative culture results and further classified as culture-negative with antimicrobial at the time of sampling or culture-negative without antimicrobial at the time of sampling. A diagnostic impact was defined as a result that established or confirmed the diagnosis without altering therapy. A negative test result interpreted by attending physicians as ruling out infection also qualified. A treatment impact was defined as any change in antimicrobial therapy (cessation, modification, or resumption) directly resulting from the test result. Tests with treatment impact also had a diagnostic impact by definition but were not computed in the diagnostic category to simplify the descriptive analyses. If the clinical team did not document an intervention resulting from the test result, the test was considered as having no impact. Impacts were initially assessed by one investigator (HL) and reviewed by one other (PB or SGL). This process was performed according to pre-established and strict criteria. This process was not blinded.

Statistical analysis

Standard descriptive analyses were performed with R (R-project v. 4.3.1). Sample size calculations were not performed given the descriptive nature of this study. Clinical impact rates were compared across different variables, including sequencing assay, test results, sample types, ordering physician specialty, and turnaround times. Given the small sample sizes within subgroups and the heterogeneous nature of the assays and clinical contexts, descriptive analyses were performed without statistical significance testing.

Ethical considerations

This study was approved by the institutional review board of the Centre de Recherche du Centre Hospitalier de l'Université de Montréal.

Results

Study population and direct sequencing assays

A total of 279 samples from 185 distinct patients were tested by BTS (229/279, 82.1%), FTS (45/279, 15.1%), and mNGS (5/279, 1.8%) (Table 1). Among all patients, 84% were hospitalized and 41% were immunocompromised. Tests were most frequently prescribed to investigate central nervous system (51/279, 18.3%), bone (42/279, 15.1%), and cardiovascular (40/279, 14.3%) infections (Figure 1).

Table 1:

Study cohort characteristics

	Patients; n = 185	Sequencing tests; n = 279
Male, no. (%)	106 (57.3)	157 (56.3)
Mean (SD) age, y	57 (17.2)	55 (17.2)
Inpatient, no. (%)	155 (83.8)	232 (83.2)
Immunosuppressive conditions*, no. (%)	76 (41.1)	129 (46.2)
Test replication† (same assay on same sample), no. (%)
No replication	162 (87.6)	N/A
2 tests	14 (7.6)	N/A
≥3 tests	9 (4.9)	N/A
Multiple testing‡ (distinct assays on same sample), no. (%)
No multiple testing	166 (89.7)	N/A
2 types of tests	16 (8.6)	N/A
3 types of tests	3 (1.6)	N/A

^*Immunosuppressive conditions include, but are not limited to, corticosteroid therapy equivalent to minimally prednisone 5 mg for more than 2 weeks, solid and hematologic malignancy, solid organ transplantation, immunomodulatory monoclonal antibody therapy

^†Some patients had a single type of clinical sample analyzed multiple times with the same direct sequencing assay

^‡Some patients had a single type of clinical sample analyzed multiples times with different types of direct sequencing assay (targeted bacterial sequencing, targeted fungal sequencing, or mNGS)

N/A = Not applicable

Figure 1: Impact of direct sequencing assays in different sample types.

Note: Other samples include but are not limited to ascites, sinus mucosal biopsy, subdural pus, and endovascular prosthesis

BTS = Bacterial targeted sequencing; FTS = Fungal targeted/panfungal sequencing; mNGS = Metagenomic next-generation sequencing

Clinical impact

Of all sequencing tests, 40/279 (14.3%) had a clinical impact in 30/185 patients (16.2%). Among the 279 tests, 26 (9.3%) tests impacted diagnosis alone and 14 (5%) also led to treatment modifications (Tables 2 and 3, Supplementary materials 3). Among the 30 patients, 20 (10.8%) were impacted at the diagnostic level only, while 10 (5.4%) had their management modified based on the sequencing test result. Figure 1 presents the sample types where sequencing had a clinical impact.

Table 2:

Direct sequencing analyses with treatment impact

Patient ID	Test ID	Clinical syndrome	Assay	Test indication	Sample type	Turnaround time (days)	Test result	Final clinical diagnosis	Clinical impact
1	1	Infected sclerosing peritonitis	Bacterial	NCWA	Ascites	Unknown	Haemophilus influenzae	Infected sclerosing peritonitis	TM (levofloxacin introduction)
2	2	Sinusitis	Fungal	First-line test	Sinus mucosal biopsy	Unknown	Negative	Bacterial sinusitis	TC (isavuconazole cessation)
	3	Sinusitis	Fungal	First-line test	Bone sample	Unknown	Negative	Bacterial sinusitis	TC (isavuconazole cessation)
	4	Sinusitis	Fungal	First-line test	Bone sample	Unknown	Negative	Bacterial sinusitis	TC (isavuconazole cessation)
3	5	Infective myositis	Fungal	Unknown	Soft tissue	Unknown	Negative	Cicatricial changes	TC (fluconazole cessation)
4	6	Meningoencephalitis	Bacterial	NCWA	Cerebrospinal fluid	Unknown	Technical limitations	Non-infectious condition	TC (Meropenem cessation)
5	7	Spondylodiscitis	Bacterial	NCWA	Intervertebral disc sample	17	Haemophilus parainfluenzae	Spondylodiscitis	TM (ceftriaxone, vancomycin and metronidazole changed for ceftriaxone)
6	8	Osteomyelitis	Bacterial	First-line test	Bone sample	Unknown	Negative	Non-infectious condition	TC (cefazolin cessation)
7	9	Encephalitis	Bacterial	First-line test	Cerebral biopsy	17	Streptococcus intermedius	Encephalitis	TM (meropenem and vancomycin changed for penicillin)
	10	Cerebral abscess	Bacterial	First-line test	Cerebral collection	17	Streptococcus intermedius	Cerebral abscess	TM (meropenem and vancomycin changed for penicillin)
8	11	Endovascular prosthesis infection	Bacterial	NCWA	Endovascular prosthesis	Unknown	Winkia neuil	Endovascular prostheses infection	TM (ertapenem changed for penicillin)
	12	Endovascular prosthesis infection	Bacterial	NCWA	Perivascular collection	Unknown	Winkia neuil	Endovascular prostheses infection	TM (ertapenem changed for penicillin)
9	13	Cerebral abscess	Bacterial	NCWoA	Cerebral collection	32	Nocardia sp	Cerebral abscess	TM (meropenem changed for linezolid and moxifloxacin)
10	14	Multisystemic disease	Fungal	NCWA	Whole blood	25	Saccharomyces cerevisiae	Fungal multisystemic disease	TM (voriconazole and caspofungin changed for voriconazole and amphotericin B)

NCWA = Negative culture with antimicrobial at the time of sampling; NCWoA = Negative culture without antimicrobial at the time of sampling; TC = Treatment cessation; TM = Treatment modification

Table 3:

Direct sequencing analyses with diagnosis impact

Patient ID	Test ID	Clinical syndrome	Assay	Test indication	Sample type	Turnaround time (days)	Test result	Final clinical diagnosis	Clinical impact
1	15	Infected sclerosing peritonitis	Fungal	First-line test	Ascites	Unknown	Negative	Sclerosing peritonitis	Infection R/O
	16	Infected sclerosing peritonitis	Bacterial	First-line test	Ascites	Unknown	Technical limitations	Sclerosing peritonitis	Infection R/O
	17	Infected sclerosing peritonitis	Bacterial	First-line test	Ascites	Unknown	Technical limitations	Sclerosing peritonitis	Infection R/O
2	18	Sinusitis	Bacterial	NCWA	Sinus mucosal biopsy	Unknown	Streptococcus intermedius	Bacterial sinusitis	Infection R/I
	19	Sinusitis	Bacterial	NCWA	Orbital collection	Unknown	Streptococcus anginosus group	Bacterial sinusitis	Infection R/I
	20	Sinusitis	Bacterial	First-line test	Sinus mucosal biopsy	Unknown	Streptococcus sp	Bacterial sinusitis	Infection R/I
11	21	Ventriculitis	Fungal	NCWA	Cerebrospinal fluid	Unknown	Negative	Non-fungal ventriculitis	Infection R/O
12	22	Fungal pneumonia	Fungal	Unknown	Lung biopsy	Unknown	Negative	Organizing pneumonia	Infection R/O
13	23	Infectious pancreatitis	Fungal	NCWoA	Pancreas biopsy	Unknown	Negative	Non-infectious pancreatitis	Infection R/O
14	24	Septic arthritis	Bacterial	First-line test	Soft tissue	Unknown	Technical limitations	Inflammatory arthritis	Infection R/O
15	25	Discitis	Fungal	Unknown	Intervertebral disc sample	Unknown	Negative	Non-fungal discitis	Infection R/O
16	26	Ventriculitis	Bacterial	NCWA	Cerebrospinal fluid	14	Pseudomonas aeruginosa	Ventriculitis	Infection R/I
17	27	Spondylodiscitis	Bacterial	NCWA	Bone sample	Unknown	Staphylococcus aureus	Spondylodiscitis	Infection R/I
18	28	Liver abscess	Bacterial	NCWA	Liver collection	17	Klebsiella sp	Liver abscess	Infection R/I
19	29	Empyema	Bacterial	NCWA	Pleural fluid	13	Streptococcus sp	Empyema	Infection R/I
20	30	Subdural empyema	Bacterial	NCWA	Subdural pus	12	Fusobacterium necrophorum	Subdural empyema	Infection R/I
21	31	Liver abscess	Bacterial	NCWA	Liver collection	48	Fusobacterium sp	Liver abscess	Infection R/I
22	32	Bacteremia	Bacterial	To confirm a previous positive result is a contaminant	Whole blood	Unknown	Negative	No infection	Infection R/O
23	33	Meningitis	Bacterial	NCWA	Cerebrospinal fluid	10	Klebsiella oxytoca	Meningitis	Infection R/I
24	34	Soft tissue abscess	Bacterial	NCWA	Soft tissue collection	Unknown	Streptococcus pyogenes	Soft tissue abscess	Infection R/I
25	35	Epidural abscess	Bacterial	Unknown	Epidural collection	Unknown	Staphylococcus aureus	Epidural abscess	Infection R/I
26	36	Osteomyelitis	Bacterial	Unknown	Bone sample	23	Technical limitations	Degenerative changes	Infection R/O
27	37	Spondylodiscitis	Bacterial	Unknown	Intervertebral disc sample	Unknown	Staphylococcus epidermidis	Spondylodiscitis	Infection R/I
28	38	Spondylodiscitis	Bacterial	NCWA	Bone sample	16	Streptococcus sp	Spondylodiscitis	Infection R/I
29	39	Fever	Fungal	First-line test	Whole blood	11	Negative	No fungemia	Infection R/O
30	40	Empyema	Fungal	Unknown	Pleural fluid	22	Negative	Bacterial pneumonia	Infection R/O

NCWA = Negative culture with antimicrobial at the time of sampling; NCWoA = Negative culture without antimicrobial at the time of sampling; R/I = Ruled in; R/O = Ruled out

Among the 26 tests with diagnosis impact alone, 14 ruled in and 12 ruled out an infection, respectively (Table 3). Twelve tests with a negative or an indeterminate result due to technical limitation were interpreted by the medical team as confirmation that the clinical syndrome was not infectious. Technical limitation results correspond to nonspecific and hence uninterpretable mixed pathogen sequences. Although those assays have not been previously studied as a “test of cure,” three tests were used by prescribing clinicians to monitor complex bacterial sinusitis treatment response while associated cultures were negative. DNA from fastidious bacteria not recoverable by culture was not detected in our samples.

Among all BTS (229), 28 (12.2%) had clinical impact (Supplementary materials 4). Of these, 19 (8.3%) contributed solely to diagnosis and 9 (3.9%) led to treatment changes. From those 28 positive samples, none (0%) were culture positive for the same pathogen and one (3.5%) was culture positive for a different pathogen. When reviewing the comprehensive laboratory results of the 28 patients, 14 (50.0%) had no positive cultures at all. Two (7.1%) had a prior identical sample, which was positive for the same pathogen, and the subsequent negative BTS result led clinicians to discontinue therapy. Four (14.3%) patients had a prior identical sample that was positive for the same pathogen but within a mixed flora. Three (10.7%) and five (17.9%) patients, respectively, had a superficial culture or blood culture that was positive for the same pathogen that was identified by BTS on a distinct deep or normally sterile sample. For example, BTS on an intervertebral disk confirmed a case of spondylodiscitis caused by Staphylococcus epidermidis in a patient with a prior positive blood culture for the same pathogen, which had initially left clinicians doubtful of its causality. In all BTS assays, the most frequent result was technical limitation (165, no., 72.1%), followed by positive (47, no., 20.5%) and negative tests (14, no., 6.1%). Positive tests had a higher impact rate (21/47, 44.7%) than negative tests (2/14, 14.3%). Tests were most frequently ordered on bone and intervertebral disc (34/229, 14,8%), synovial fluid (32/229, 14.0%), whole blood (30/229, 13.1%), and cerebrospinal fluid (27/229, 11.8%), but were found to have the most impact when performed on cerebral (3/10, 30%), liver (2/8, 25%), and bone and intervertebral disc samples (6/34, 17.6%).

The clinical impact rate for FTS was 26.7% (12/45), with 15.6% (7/45) affecting diagnosis and 11.1% (5/45) affecting treatment (Supplementary materials 5). Among the 12 patients for whom FTS impacted the diagnosis or clinical management, two (16.7%) had no associated cultures and 10 (83.3%) had associated fungal culture that came back negative. Most tests were negative (36 tests, 80%) while 9 (20%) were positive. Negative tests had a higher clinical impact (11/36, 30.6%) than positive ones (1/9, 11.1%). Tests were most frequently ordered on lung biopsies (11 samples, 24.4%), cerebrospinal fluid (7 samples, 15.6%), and whole blood (5 samples, 11.1%). Bone and intervertebral disc samples (3/4, 75.0%) and pleural fluid (1/1, 100.0%) were found to frequently have an impact on clinical management, although only one pleural fluid was tested.

All mNGS (5 tests) were performed on cerebrospinal fluid. Only one was positive for the presence of HIV RNA and EBV DNA in a known patient living with HIV, and hence none of the assays had a clinical impact (Supplementary materials 6).

Direct sequencing assays are not restricted access tests in our institution. Although chart review did not allow to identify the prescribing physician in many cases, at least 169/229 (70.7%) BTS were ordered by ID physicians. BTS was mainly used following negative cultures, which were frequently suspected to be sterilized by ongoing antimicrobials (75 cultures, 32.8%). BTS was also frequently prescribed alone or with other microbiological investigations but as a first-line test, meaning it was requested upfront without waiting for final conventional testing results (62 prescriptions, 27.1%). Tests were more frequently impactful for the first indication (16/75, 21.3%) than the later (7/62, 11.3%) (Supplementary materials 4). For FTS, ID physicians were also the main prescribers with at least 28/45 (62.2%), followed by clinical pathologists (7/45, 15.6%). Tests ordered by ID physicians were more frequently impactful (9/28, 32.1%). FTS was mainly prescribed as a first-line test (17/45, 37.8%), which represented 5/12 (41.7%) of all clinically impactful FTS (Supplementary materials 5).

The median turnaround time between test prescribing and report digitization in the EMR could be calculated with 108 samples, for which endpoints were available, and was 19 days. When breaking down turnaround time in pre-analytical, analytical, and post-analytical standard steps, delays in shipment to the external reference laboratory (median 5 days, n = 171) and report digitization following result reception (median 6 days, n = 15) were the most important modifiable drivers of delays (Figure 2). Among all BTS and FTS for whom a complete turnaround time analysis could be performed, those with a turnaround time of 14 days or less (4/24, 16.7%, and 1/3, 33.3%, respectively) were more frequently impactful than those with a turnaround time greater than 14 days (1/70, 11.4%, and 2/7, 28.6%, respectively) (Supplementary materials 4 and 5).

Figure 2: Turnaround times.

Pre-, per-, and post-analytical steps between test prescription and report digitization. The time between two steps is referred to as the turnaround time (y-axis). For each type of turnaround time, it was possible to find the value of that delay for only a portion of the tests (n). For a given turnaround time, all available values among the tests were gathered in that figure into a box separated into four quartiles (x-axis).

CHUM = Centre Hospitalier de l'Université de Montréal

On 60 occasions, patients underwent more than one direct sequencing test during the same clinical episode. Sometimes, the additional test was associated with a clinical impact while the initial test was not. A replicate was defined as a sequencing test repeated on the same patient, targeting the same gene and using the same type of sample during a single episode (eg, two BTS on whole blood). Replicates were used in 23 patients and led to diagnostic or therapeutic impact in five cases (21.7%). A duplicate was defined as the same test type performed on a different sample from the same patient (eg, BTS on a vertebral biopsy and a paravertebral collection). Duplicates were used in 18 patients and had clinical impact in six cases (27.8%). A multiple test referred to different sequencing test types performed on the same sample during the same episode (eg, BTS and FTS on a cerebral biopsy). Multi-tests were used for 19 patients and had clinical impact in nine cases (42.1%). All replicate, duplicate, and multiple testing results are summarized in Supplementary materials 8.

Among the 279 tests, some results were not documented in physicians’ medical notes, preventing assessment of a clinical impact. In other instances, results were seen and documented by clinicians but did not lead to any clinical interventions, although according to the experts' opinion, specific interventions should have been implemented. All positive tests without a documented clinical impact (n = 35) were reviewed by an ID physician who assessed whether they would have managed the case differently based on the result. Twenty-six (11.4% of all the BTS) were BTS, eight (17,8%) were FTS, and one (20%) was an mNGS test. Among the 35 positive tests, three patients died before the result became available. The results of 13 tests had been seen by the medical team but had no impact, while 2 of those should have had an impact according to the ID specialist review. The results of 19 tests were not seen by the team, while 16 could have had an impact. Thus, among all 35 tests, 18 (51.4%) should have had a clinical impact if appropriately reviewed and acted upon (Supplementary materials 7).

Discussion

This study reports the clinical impact of direct-from-sample sequencing assays. We found that 40/279 tests (14.3%) had a clinical impact and 14 of them (5%) modified patients’ treatment. These results align with comparable studies by Wilson et al and Flurin et al, which respectively reported the treatment impacts of 3.9% and 8% (7,8). Other studies have reported clinical impact rates ranging from 2.2% to 47% (11,15,18–23); however, comparisons are limited by differences in sample size, sample types, sequencing methods, clinical indications, and definitions of clinical impact. In all those studies, impact rates represent the aggregate of heterogeneous scenarios where concomitant culture testing results on either the same or distinct samples complexifies external comparability. This reinforces the relevance of detailed and comprehensive clinical case reviews such as those presented in the Supplementary materials.

FTS achieved a higher impact rate (26.7%) than BTS (12.2%) and mNGS tests (0%). mNGS tests were so rarely used in our study that the associated results cannot be generalized. Other studies found bacterial assays more impactful than fungal ones (15,22,23). One reason might be that those studies systematically performed both assays on all samples, while our FTS was reserved for selected cases and performed upon request. Additionally, fungal infections often require prolonged treatments, so we hypothesize that FTS impact is less influenced by longer turnaround times. In contrast, suspected bacterial infections are usually treated empirically, making BTS less likely to change therapy as patients are often already on appropriate antimicrobials. Moreover, most BTS results were reported as “technical limitations” (165/229, 72.1%) compared with none for FTS. As such results should not affect clinical management, the greater impact of FTS is not surprising. Both negative and positive FTS led to clinical impact (30.6% and 11.1%, respectively).

Our study suggests that different approaches could be used to increase the clinical impact of direct-from-sample sequencing assays. However, this study is retrospective and the statistical significance of such approaches could not be assessed. First, sample type greatly affects clinical impact. Our data show particularly high impact rates for bone and intervertebral disc samples. For BTS, cerebral and liver biopsies or collections yielded good rates while synovial fluid, whole blood, pleural fluid, paravertebral fluid or collections, and lung biopsies showed low impact rates. Second, clinical context affects the likelihood of clinical impact. BTS performed in cases of negative cultures due to prior antimicrobial therapy had the greatest impact, as described in other studies (11,15). Our results suggest avoiding these assays as first-line assays prior to conventional cultures. Third, tests with the shortest turnaround times had the highest impact rate, as infectious episodes were often already resolved when results were obtained late. The median turnaround time of 19 days was longer than the 24 hours to 7 days for other similar studies (9,15,20,23–25). Reducing this turnaround time could translate into even greater clinical impact; for example, by increasing the frequency of sample submissions to external laboratories or by performing the tests in-house, and by ensuring immediate integration of reference laboratories’ results in local EMR. Last, although this was not evaluated in our study, it is reported that the sequencing method strongly influences BTS and FTS yields and positivity rates. The BTS method used in this study often led to indeterminate results as it is based on traditional Sanger sequencing technology, greatly limiting the assays’ performance for very low bacterial inoculum and polymicrobial samples, which yield uninterpretable Sanger sequencing chromatograms. The methods used by Marbjerg et al (22), who employed two different sequencing primers within the 16S rRNA gene, or by Flurin et al (7), who employed next-generation sequencing, have been shown to increase the assay's analytical sensitivity, especially in polymicrobial samples.

One challenge of our study is the sometimes inappropriate interpretation of test results by clinicians that led to diagnostic conclusions or therapeutic decisions. The use of negative tests results to rule out infection or shorten antimicrobial treatment is not accepted as standard practice, although it was previously reported in some clinical studies (9,19). For example, in a prospective study on the use of mNGS assays of various samples, up to 41% of tests were performed to exclude an infection (26). The risk of false-negative results is significant with sequencing assays, as highlighted in several studies, and negative tests should hence not be used to rule out infectious conditions (8,10,27). For example, patient 22 and 29 in our study had negative direct sequencing results on blood samples, which were inappropriately interpreted by clinicians as ruling out infections as bacteremia and fungemia typically have organism levels far below the level of detection of PCR sequencing, especially while patients are on treatment. Similarly for patient 2, fungal sequencing was repeatedly negative and bacterial sequencing repeatedly yielded streptococci species, which led the practitioner to inappropriately rule out the diagnosis of fungal sinusitis, discontinue anti-fungal therapy, and prolong antibacterial treatment. Many clinicians interpreted “technical limitations” results as being negative, although they are defined as nonspecific and uninterpretable given the presence of mixed microbial sequences. Whether those sequences reflect the presence of a pathogen—which can explain clinical manifestations—always remains unclear, but the presence of such sequences clearly do not rule out an infection. Those scenarios highlight the need for appropriate training and companion laboratory stewardship programs to support clinicians in the appropriate use of those relatively novel assays as described in other studies (28). This retrospective, single-centre study has other limitations. For many samples, some data were unavailable, limiting the conclusions we could draw. For instance, the identity of prescribers was often unavailable, preventing a detailed comparative analysis of the impact rates between specialties. Also, to assess a clinical impact during chart review, the impact needed to be documented in the medical chart, which was not always clearly the case. We may have underestimated the impacts, as we showed that independent review of positive results yielded higher theoretical impact rates. Teoh et al observed the same phenomenon regarding their BTS and FTS, and suggest an interesting approach involving an antibiotic stewardship team in monitoring the test results (23).

Several strengths distinguish this study. Each clinical episode was reviewed individually and thoroughly. This approach enabled not only the identification of changes in antibiotic therapy, but also an assessment of diagnostic impact, an aspect often overlooked in the literature (7,11,22). Furthermore, we included a notably wide range of clinical sample types compared with many previous studies (8,15,20,21). This diversity allowed us not only to evaluate the overall impact of direct sequencing assays, but also to identify the contexts in which they are most useful. Part of the existing literature has focused exclusively on the clinical impact of positive tests (11,18), overlooking the potential clinical relevance of negative results, which remains to be clarified given the well-known low negative-predictive value of direct sequencing assays. FTS remains underexplored compared with BTS and mNGS (15,22,23), and is often evaluated in studies with very small sample sizes. Our results suggest that FTS can play a significant clinical role, although, in our cohort, the impact mostly resulted from negative tests being interpreted by clinicians as ruling out infections. This reinforces the need to deploy novel and complex microbiology assays with appropriate companion laboratory stewardship programs and training. Finally, our analysis of repeated testing within a single clinical episode addresses a niche area that has received very little attention to date.

Our study suggests the use of several approaches, which, value added, should be formally measured in better controlled diagnostic studies. The use of multiple direct sequencing assays for the same clinical episode appears to increase diagnostic yield, as this method was associated with good impact rates in our hospital, higher than those (10%) reported by Hogan et al (21). Direct sequencing assays may also serve as valuable tools in antibiotic stewardship. Among our 14 tests with therapeutic impact, 6 (42.9%) led to antibiotic cessation and 7 (50%) led to treatment modification for optimal antibiotic therapy, most through a reduction in the antimicrobial spectrum. Others came to the same conclusion (9,22,25). Our study also revealed that some clinicians were using those assays as follow-up testing despite the risk for prolonged detection of non-viable organisms. Fung et al noted that serum mNGS became positive before conventional methods during recurrent infection (24). Whether direct sequencing assays have a role in the assessment of clinical response or cure was not previously demonstrated and should be investigated.

This single-centre, retrospective study was not designed to assess the pure cost and cost–effectiveness of direct sequencing assays. Costs may vary between laboratories due to factors including local expertise, testing volumes, and contracts. Based on ministry of health reimbursement data and out of province reference lab billing data, cost per analysis were estimated to be CAD $100, $100, and $3,125 for BTS, FTS, and mNGS, respectively. With 229 BTS assays performed at an impact rate of 12.2%, the estimated number needed to impact clinical management was 8.17 at a cost of CAD $2,794 per impacted patient. For FTS, the estimated number needed to impact management was lower at 3.75 for a cost of CAD $1,202 per impacted patient.

Direct-from-sample broad-spectrum sequencing assays are meaningfully transforming diagnostic practices in ID. They combine some advantages of targeted PCRs over culture without the disadvantages of being pathogen-specific. However, their clinical impact is higher in specific contexts, highlighting the need for good diagnostic stewardship practice and clinician training.

Funding Statement

This study was not supported by specific finding agencies and was performed as a quality improvement and laboratory stewardship project.

Contributors

Conceptualization, H Leprohon, M-C Domingo, P Morency-Potvin, P Benoit, S Grandjean Lapierre; Methodology, H Leprohon, P Morency-Potvin, P Benoit, S Grandjean Lapierre; Software, M-C Domingo, P Dufresne; Validation, S Grandjean Lapierre; Formal analysis, H Leprohon, M-C Domingo, P Dufresne, S Grandjean Lapierre; Investigation, H Leprohon, B Tannir, G Jolicoeur, P Dufresne, P Benoit; Resources, S Grandjean Lapierre; Data curation, H Leprohon, B Tannir, G Jolicoeur, P Benoit; Writing – Original Draft, H Leprohon; Writing – Review & Editing, B Tannir, M-C Domingo, P Morency-Potvin, P Benoit, S Grandjean Lapierre; Visualization, H Leprohon; Supervision, P Benoit, S Grandjean Lapierre; Project Administration, S Grandjean Lapierre.

All authors qualify for authorship as per ICMJE recommendations and were therefore either involved in study conception (HL, PMP, PB, SGL), study design (HL, PB, SGL), primary microbiology analyses (MCD, PD), data acquisition and chart review (HL, BT, GJ), data analysis (HL, SGL), data interpretation (HL, SGL), drafting the initial manuscript (HL, SGL), final review and approval of the manuscript prior to peer review (HL, BT, GJ, MCD, PD, PMP, PB, SGL), agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved (HL, BT, GJ, MCD, PD, PMP, PB, SGL).

Ethics approval

Ethics certificate information is available upon request.

Informed Consent

N/A

Registry and the Registration No. of the Study/Trial

N/A

Data Accessibility

All the laboratory protocol and detailed clinical impact data is available on Open Science Framework under DOI https://doi.org/10.17605/OSF.IO/HTZ5P. Primary pathogen identification sequencing data are not deposited on public repository as those were generated in the context of individual patient care clinical management. Contact the corresponding author for access to additional information.

Funding

This study was not supported by specific finding agencies and was performed as a quality improvement and laboratory stewardship project.

Disclosures

The authors disclose no financial or personal relationships with other people or organizations that could inappropriately influence this work.

Peer Review

This article has been peer reviewed.

Animal Studies

N/A