Performance of commercial tests for molecular detection of Shiga toxin-producing Escherichia coli (STEC): a systematic review and meta-analysis protocol

Abstract

Introduction

Rapid detection of Shiga toxin-producing Escherichia coli (STEC) enables appropriate treatment. Numerous commercially available molecular tests exist, but they vary in clinical performance. This systematic review aims to synthesise available evidence to compare the clinical performance of enzyme immunoassay (EIA) and nucleic acid amplification tests (NAATs) for the detection of STEC.

Methods and analysis

The following databases will be searched employing a standardised search strategy: Medline, Embase, Cochrane CENTRAL Register of Controlled Trials, Cochrane Database of Systematic Reviews, PubMed, Scopus and Web of Science. Grey literature will be searched under advice from a medical librarian. Independent reviewers will screen titles, abstracts and full texts of retrieved studies for relevant studies. Data will be extracted independently by two reviewers, using a piloted template. Quality Assessment of Diagnostic Accuracy Studies-2 will be employed to assess the risk of bias of individual studies, and the quality of evidence will be assessed with the Grading of Recommendations Assessment, Development and Evaluation approach. A bivariate random-effects model will be used to meta-analyse the sensitivity and specificity of commercial STEC diagnostic tests, and a hierarchical summary receiver operator characteristic curve will be constructed. Studies of single test accuracy of EIA and NAATs and studies of comparative accuracy will be analysed separately.

Ethics and dissemination

Ethics approval was not required for this systematic review and meta-analysis. Findings will be disseminated in conferences, through a peer-reviewed journal and via personal interactions with relevant stakeholders.

PROSPERO registration number

CRD42018099119.

Strengths and limitations of this study.

• There is little evidence reviewing the relative clinical performance of commercially available tests for Shiga toxin-producing Escherichia coli (STEC).

• A key strength of this study is the comprehensive comparison of enzyme immunoassays and nucleic acid amplification tests to inform clinical practice.

• A limitation is the lack of a common gold standard for STEC identification, which may introduce heterogeneity into our analysis.

• Another limitation is that the finding of a Shiga toxin (Stx) 1 producing STEC that does not also produce Stx2, especially in the absence of bloody diarrhoea, is of unclear clinical and epidemiological value.

Introduction

Shiga toxin-producing Escherichia coli (STEC) cause significant disease. Although prototypical E. coli O157:H7 is the leading cause of haemolytic uraemic syndrome (HUS), other STEC serotypes have been associated with severe disease and large outbreaks.^1–4 Multiple serotypes have now been linked to disease. Unlike the O157 serotype, detection of non-O157 serotypes has increased significantly in the past decade, though likely because of dissemination of technology to detect these organisms.⁵ Patients infected with STEC often seek care through emergency departments (EDs), especially if they have bloody diarrhoea. Strong evidence suggests that antibiotics may increase the risk of developing HUS if administered to people infected with STEC,^6–8 and a recent meta-analysis demonstrated that the early administration of fluids is associated with improved outcomes.⁹ Therefore, it is important that healthcare providers have a means of detecting STEC that is both rapid and applicable to any serotype.

Historically, STEC testing has focused on the O157 serogroup using culture on sorbitol-MacConkey agar, leveraging its inability to ferment sorbitol.¹⁰ This attribute is not shared by other STEC serogroups, so they are overlooked if sorbitol-MacConkey agar culture is the only detection method employed. Further, culture can take days to yield results, delaying informed management.¹¹ In light of the limitations of culture, enzyme immunoassay (EIA) and nucleic acid amplification tests (NAATs) have been developed to detect STEC irrespective of serogroup. Reflecting their popularity, the US Council of State and Territorial Epidemiologists has recently revised the probable STEC case definition to include laboratory evidence from EIA and NAAT.¹²

Numerous tests to detect STEC are commercially available.^{13 14} The EIAs detect Shiga toxin (Stx), and most NAATs detect the Stx genes Stx1 and Stx2, and some additionally seek a locus that is specific to the O157 serogroup. For NAAT, STEC is often one of several enteropathogens detected by the assay. EIA has suboptimal sensitivity, particularly if a time-consuming enrichment step is not conducted.^15–18 Commercial NAATs appear to be more sensitive, but results vary by study and test.^19–21 NAATs are more costly than traditional microbiological techniques owing to the equipment and consumables required to perform them. However, the higher cost may be compensated by increased ascertainment²¹ and/or improved patient outcomes.²² As laboratories consider NAATs, it is crucial to identify the best testing strategy to support time-sensitive, cost-effective treatment decisions. Thus, we will conduct a systematic review of commercial EIA and NAAT for STEC detection to determine if and how their performance differs in terms of diagnostic test accuracy (DTA).

Methods and analysis

This systematic review and meta-analysis will be conducted in accordance with reporting requirements for Preferred Reporting Items for Systematic Reviews and Meta-analyses statement (PRISMA). This protocol was prepared according to PRISMA-Protocol and PRISMA-DTA guidelines.^{23 24}

Research question

What is the accuracy of commercially available EIA and NAAT for the detection of STEC and how do they differ?

Eligibility criteria

• Participants: study participants with acute diarrhoea, who provide a stool specimen or rectal swab for diagnostic testing; any age or subpopulation.

• Setting: healthcare systems or medical facilities, including outpatient clinics, EDs, hospitals, long-term care centres and similar, without geographical limitation.

• Index tests: any commercially available EIA or NAAT for the detection of Stx, or Stx1 and Stx2; NAAT for the identification of the O157 serogroup, if available. Included studies may assess the accuracy of commercially available EIA, NAAT or both, including comparative accuracy studies.

• Reference standard: at least one of the following: enhanced protocols, real-time PCR, sequencing and/or other NAAT.

• Target condition: acute diarrhoea associated with STEC infection.

• Study designs: cross-sectional diagnostic accuracy studies, encompassing all studies with both index and reference tests conducted on stool samples/swabs collected at a single point of time during the acute diarrhoea illness, including both single test and comparative accuracy studies.

• Report characteristics: years 2005 to present (2015 to present for conference abstracts), published or unpublished, in any language.

Literature searches

The following databases will be searched from 2005: MEDLINE, Cochrane CENTRAL Register of Controlled Trials, Cochrane Database of Systematic Reviews, EMBASE, PubMed, SCOPUS and Web of Science. Clinical trial databases (ClinicalTrials.gov), Food and Drug Administration applications, package inserts for commercial assays, company product websites and literature, government/non-governmental organization reports and conference abstracts will also be searched under the advice of STEC subject experts and a medical librarian. The reference lists of included studies will be scanned to identify additional studies of relevance to this review. The specific search strategy can be found in online supplementary appendix I.

Study records

Data management

Records retrieved will be uploaded into EndNote V.8 (Philadelphia, Pennsylvania, USA), and deduplicated using EndNote V.8 and Rayyan for Systematic Reviews (Qatar, 2018).

Selection process

Two reviewers (GAMT, CYL) will independently screen all titles and abstracts in duplicate, and a third reviewer (SBF) will adjudicate any disagreements. Studies will be included if the title and abstract indicate that the manuscript may contain data related to the evaluation of EIA and/or NAAT for the detection of STEC. The full text of all potentially relevant citations will then be obtained and reviewed by two independent reviewers (GAMT, CYL) using the predefined eligibility criteria outlined above, with the involvement of a third reviewer (SBF) in case consensus cannot be reached. Reasons for inclusion and exclusion will be documented. A tool to document the selection process will be developed, piloted with the first 25 search results and modified as necessary.

Data extraction

Two reviewers will extract data independently and in duplicate using a structured form. The form will be piloted on the first five included studies and modified as necessary. Discordances will be resolved through discussions involving the reviewers and subject matter experts. First and last study authors will be contacted if data necessary to calculate sensitivity or specificity are absent from the manuscript. Study characteristics and study outcomes (table 1) will be extracted from included studies.

Table 1.

Data to be extracted from each included study

Item	Rationale
Study characteristics
Data source	Peer-reviewed studies will be distinguished from non-peer-reviewed data for potential subgroup analysis
Funding source	Studies funded by diagnostic test companies may be subject to additional bias; potential subgroup analysis
Study design	Cross-sectional studies are expected; other study designs will be noted for potential subgroup analysis
Population	Population restrictions within the study (eg, by age, HUS status, etc) will be noted for potential subgroup analysis
Setting	Country or region; potential subgroup analysis
Clinical data
Location of care	Primary care versus ED versus hospital, and potentially other; potential subgroup analysis
Diarrhoea definition	Study definition for diarrhoea (eg, ≥3 episodes in 24 hours) will facilitate comparability assessment and interpretation
Diarrhoea duration	Mean/median or restrictions on illness duration at the time of sampling; facilitate comparability assessment and interpretation
Specimen type	Stool specimen or rectal swab; potential subgroup analysis
Bloody diarrhoea	Frequency of bloody diarrhoea; potential subgroup analysis
Test
Brand name	Ease of reference
Type	EIA or NAAT for main comparison
Enrichment	For EIA tests; potential subgroup analysis
Targets	Toxin versus DNA, STEC-only versus multianalyte; interpretation and potential subgroup analysis
Cycle threshold	Cycle cut-off for positivity; facilitate comparability assessment and interpretation
Comparator/reference standard	Composite standard with component tests, discrepant analysis with confirmatory tests; interpretation and potential source of bias
Specimen comparability	Specimens tested by index and comparator from the same point in time, of the same type; potential source of bias
Outcomes
Outcome type	For STEC generally, Shiga toxin 1 vs 2 or O157 vs non-O157; distinguish primary and secondary outcomes
No tested	Outcome calculation and interpretation
No confirmatory tested	Outcome calculation and interpretation
No of true positives	Outcome calculation
No of false positives	Outcome calculation
No of true negatives	Outcome calculation
No of false negatives	Outcome calculation
Sensitivity	Primary outcome
Specificity	Primary outcome
Single accuracy measures	For example, AUC, diagnostic accuracy, diagnostic OR; secondary outcome
PPV	Secondary outcome
NPV	Secondary outcome
LR+	Secondary outcome
LR−	Secondary outcome

AUC, area under the curve; ED, emergency department; EIA, enzyme immunoassay; HUS, haemolytic uraemic syndrome; LR, likelihood ratio; NAAT, nucleic acid amplification test; NPV, negative predictive value; PPV, positive predictive value; STEC, Shiga toxin-producing Escherichia coli.

Risk of bias assessments

To assess the risk of bias in individual studies, we will employ the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2).²⁵ We will follow the recommended process for tailoring the QUADAS-2 to our systematic review, including iteratively tailoring the QUADAS-2 assessment tool and piloting it on at least five studies until consensus has been reached on a version of the tool.²⁵ As part of this process, we will review the Standards for Reporting of Diagnostic Accuracy²⁶ and prior QUADAS-2 modifications for comparative accuracy studies²⁷ for relevant criteria. For comparative accuracy studies, we will add a signalling question regarding the assessment of EIA and NAATs in the same group of patients. The risk of bias in individual studies (for all outcomes reported) will be rated as low/unclear/high.²⁸ Assessments will be made independently by two reviewers, and disagreements will be resolved by discussion, or where necessary, by a third reviewer. Risk of bias will be reported for all included studies.

Data synthesis

Separate synthesis will be conducted for EIA and NAAT. For each of test type, data will be quantitatively synthesised if at least four studies have been identified. If the number of included studies for either EIA or NAAT is insufficient, point estimates and CIs from the individual papers will be shown, and the comparison of EIA and NAAT will be based on the range of estimates reported in individual papers.

If four or more studies are included for a given test type, a bivariate random-effects model²⁹ will be used to calculate summary estimates and confidence intervals of primary outcomes and secondary outcomes, and a hierarchical summary receiver operating characteristic (ROC) curve³⁰ will be constructed.³¹ The summary point for sensitivity and specificity with confidence ellipse and the hierarchical summary ROC curve will be graphed. These analyses take into account the correlation between sensitivity and specificity and potential threshold effects (eg, due to cycle thresholds used in PCR).³¹ Meta-analysis packages in R³² and RevMan³³ will be used to conduct all analyses.

Comparative accuracy

To compare EIA and NAAT, we will meta-analyse only comparative accuracy studies that evaluate both types of the test against the same reference standard. If no comparative accuracy studies are identified, we will graphically compare point estimates and CIs for sensitivity and specificity resulting from the separate meta-analysis of each type of test. If there is adequate consistency in reference standards used to assess single test accuracy, we will pool EIA and NAAT studies in a single meta-analysis and include test type as a covariate to test the difference in accuracy between EIA and NAAT.

Subgroup analysis

To identify study characteristics that may be contributing to heterogeneity, we will conduct subgroup analyses when at least four studies are available per subgroup:

• Funding (industry vs other).

• Data source (published vs unpublished).

• Age (<10 years old and <18 years old).

• Location of care.

• Diarrhoea duration (<7 days, ≥7 days, not specified).

• Presence of bloody diarrhoea.

• Specimen type.

• Test brand.

• Test targets.

• Reference standard.

Other subgroup analyses not prespecified here will be identified as such in all reports. Subgroup analyses will illustrate the magnitude of differences in accuracy, and thus allow readers to interpret whether they are clinically meaningful. We will obtain statistical evidence of whether these factors contribute to heterogeneity in the primary analysis by adding each to the bivariate random-effects model as a predictor.

A sensitivity analysis excluding studies with a high risk of bias will be conducted. Additional sensitivity analyses will be added if other potential biases become apparent during the review.

Quality of evidence assessment

For the quality of evidence for each test type, two reviewers, one with clinical and one with methodological expertise, will independently use the Grading of Recommendations Assessment, Development and Evaluation approach to assess the quality of evidence for sensitivity and specificity.^{34 35} The test will be considered in the context of how it relates to patient-important outcomes to assign importance to the consequences of summary sensitivity and specificity findings (eg, frequency of false negatives). The domains of study design, limitations/risk of bias, directness, consistency, precision and publication bias will be assessed and combined into a summary grade for all important outcomes of the test. Publication bias will be assessed based on differences in accuracy reported in industry-funded versus non-industry-funded studies.

For the comparison of EIA and NAAT, we will use a similar approach to grade the quality of evidence, with the same domains as for single test accuracy. Risk of bias will reflect the modifications we make to QUADAS-2 for comparative accuracy studies. Indirectness will be affected by the number of comparative accuracy studies including both EIA and NAAT; if few comparative accuracy studies are identified and the comparison is based on single test accuracy from different studies, quality will be downgraded due to indirectness.

Study results will be reported according to the PRISMA-DTA guidelines.²⁴

Patient and public involvement

This protocol was designed without patient involvement. Patients were not invited to comment on the systematic review design and were not consulted to develop patient-relevant outcomes. Patients were not invited to contribute to the writing or editing of this protocol for readability or accuracy.

Ethics and dissemination

Findings will be disseminated in conferences, through a peer-reviewed journal and via personal interactions with relevant stakeholders.