Skip to main content
NCBI home page
Journal of AOAC International logoLink to Journal of AOAC International
. 2023 Apr 17;106(5):1424–1430. doi: 10.1093/jaoacint/qsad047

Amplicon Sequencing Minimal Information (ASqMI): Quality and Reporting Guidelines for Actionable Calls in Biodefense Applications

Ishi Keenum 1,✉, Robert Player 2,3, Jason Kralj 4, Stephanie Servetas 5, Michael D Sussman 6, Joseph A Russell 7, Jennifer Stone 8, Sailaja Chandrapati 9, Shanmuga Sozhamannan 10,11
PMCID: PMC10472743  PMID: 37067472

Abstract

Background

Accurate, high-confidence data is critical for assessing potential biothreat incidents. In a biothreat event, false-negative and -positive results have serious consequences. Worst case scenarios can result in unnecessary shutdowns or fatalities at an exorbitant monetary and psychological cost, respectively. Quantitative PCR assays for agents of interest have been successfully used for routine biosurveillance. Recently, there has been increased impetus for adoption of amplicon sequencing (AS) for biosurveillance because it enables discrimination of true positives from near-neighbor false positives, as well as broad, simultaneous detection of many targets in many pathogens in a high-throughput scheme. However, the high sensitivity of AS can lead to false positives. Appropriate controls and workflow reporting can help address these challenges.

Objectives

Data reporting standards are critical to data trustworthiness. The standards presented herein aim to provide a framework for method quality assessment in biodetection.

Methods

We present a set of standards, Amplicon Sequencing Minimal Information (ASqMI), developed under the auspices of the AOAC INTERNATIONAL Stakeholder Program on Agent Detection Assays for making actionable calls in biosurveillance applications. In addition to the first minimum information guidelines for AS, we provide a controls checklist and scoring scheme to assure AS run quality and assess potential sample contamination.

Results

Adoption of the ASqMI guidelines will improve data quality, help track workflow performance, and ultimately provide decision makers confidence to trust the results of this new and powerful technology.

Conclusion

AS workflows can provide robust, confident calls for biodetection; however, due diligence in reporting and controls are needed. The ASqMI guideline is the first AS minimum reporting guidance document that also provides the means for end users to evaluate their workflows to improve confidence.

Highlights

Standardized reporting guidance for actionable calls is critical to ensuring trustworthy data.

Amplicon sequencing (AS) is widely applied for microbial pathogen detection and identification (1, 2), and has emerged as a leading technique for biosurveillance applications requiring high sensitivity and specificity (3). AS is preferred over traditional PCR approaches for its method performance, e.g., lower LOD compared to PCR alone. Sequencing PCR products (amplicons) can rapidly confirm the PCR target without size separation reducing the risk of false negatives and positives (see Supplemental Text S1). AS can also enable higher throughput than quantitative PCR (qPCR) which is limited to only four fluorophores (probes) per sample in order to achieve a distinct wavelength band signal per unique amplification target. Reliable detection has been reported in a limited multiplex format qPCR assay (2500 to 50 000 CFU/mL) for a variety of bacterial targets, compared to 10 to 50 CFU/mL for AS (4). When the goal is qualitative detection of a particular pathogen, e.g., an outbreak investigation, AS with pathogen-specific primers is an incredibly powerful tool and can provide genomic enrichment in low-abundance samples, while the sequencing allows for variant identification (single nucleotide variants, indels, etc.) within the amplified region. Amplified sequence targets can then go on to be used for characterizing genetic drift in the target pathogen (5), enabling phylogenetic studies (1), near-neighbor discrimination (6), and leveraged for primer optimization. While the U.S. Food and Drug Administration (FDA) provides some guidelines on AS-based diagnostics (7), there are currently no specified standards or guidelines for the implementation of AS to support actionable calls in biosurveillance. Consistent with current literature including diagnostic guides, we identify specific requirements applicable to biosurveillance/biodetection of microbial pathogens.

Specific guidelines and minimum requirements, including appropriate controls, are critical for data quality, and underpin confidence in analytical workflows to enable actionable results. AS methods require multi-step workflows consisting of sample collection and handling, nucleic acid extraction, PCR amplification, library preparation, and sequencing and bioinformatic processing (Figure 1). Due to the sequential nature of the workflow, systematic errors can be compounded. Minimum information guidelines for reporting and QA/QC exist for qPCR (MIQE; 8), genome sequences (MIGS; 9), marker gene sequences (MIMARKS; 10), and environmental monitoring (Environmental Microbiology Minimum Information [EMMI]; 11); however, these guidelines do not extend to AS approaches. Past work by Murray et al. (12) also identified the need for further development of AS workflow guidelines as bench methods and controls are often ignored. PCR is the current gold standard for detection of biodefense pathogens, and the further implementation of AS helps address several limitations including: (1) false-negative results due to target sequence variations, (2) false-positive results due to near-neighbor target matches, and (3) variable amplification efficiencies impacting LOD measurements. However, both methods are still impacted by inhibitors.

Figure 1.

Figure 1.

Open in a new tab

Known workflow steps that introduce biases in an AS workflow.

The objective of this document is to outline a set of requirements for AS workflows. The Amplicon Sequencing Minimal Information (ASqMI) details reporting guidelines and controls that will support reproducibility in day-to-day operations by providing guidance to mitigate/capture biases and enable actionable calls based on robust sequencing data from operational samples. AOAC INTERNATIONAL has a long-standing tradition in developing Standard Method Performance Requirements (SMPRs®) and has developed many SMPRs for agent detection assays and methods in the past; unfortunately, there are no guidelines for data elements, collection, and reporting for AS assays. Hence, we used the AOAC model to generate these guidelines and an SMPR will be developed in the future based on the ASqMI guidelines. Herein, we have provided platform-agnostic guidelines as much as possible, allowing and requiring the end user to optimally adapt these guidelines to their specific operational requirements. Additionally, we have provided routine and process checklists for laboratories to use to identify key elements of QA/QC in an AS workflow (Supplemental Worksheet 1). The objective of the Process Reporting Checklist is to provide end users with a comprehensive list of metadata that should be recorded to enable better troubleshooting when methods and controls do not perform as expected. A mock data set is provided along with an example reporting sheet, to serve as a guide (Supplemental Worksheet 2). Laboratories can utilize the point system to begin to gain consensus on their routine performance and identify potential areas for improvement.

Challenges with Sample Matrices

The EMMI guidelines are a good source for sample collection controls. Although they will not be covered in detail here (11), we would like to emphasize the impact of sample matrix on the interpretability of subsequent results and conclusions. Broadly, samples for biosurveillance can be separated into two categories: complex background, high bioburden (e.g., soils, wastewater) and low genomic yield samples (e.g., aerosol collection filters). Sample processing and bioinformatic workflows vary depending on the expected complexity of the sample (thresholds and the stringency for near-neighbors may need adjustment), as well as the amount of recoverable DNA or RNA. Ultimately, the sample matrix will greatly influence the conclusions, and it is critical that matrix effects are considered during workflow development. It is critical to well document the sampling method so that if field blanks are positive (Supplemental Worksheet 1), it is possible to identify how contamination could have occurred in the field.

The Need for Controls

Every step in an AS workflow, from sampling to data processing, can introduce systematic errors (13–16; Figure 1). Method and material controls are important for allowing the characterization of these systematic errors (Figure 2). Controls should be implemented at multiple stages in the sampling-to-analysis workflow to enable the identification of potential contamination (see Supplemental Text S2). Commonly, one control introduced early in the workflow can serve as a verification for downstream steps.

Figure 2.

Figure 2.

Open in a new tab

Summary of controls and reporting guidelines for an AS workflow.

Negative Controls

Negative controls such as sampling blanks (matrix blanks taken to the field and opened with each sample collected to account for sampling-based contamination) and no template controls (NTCs), serving as true-negative controls, should always be included in AS methods to control for false positives and ensure confidence. As a minimum, three biological sample replicates should be included to enable the practical calculation of the mean and SD of aligned read counts. If any NTC sample contains reads that align to a target reference above a practical threshold for a positive result, then these metrics indicate the aligned read counts for an amplicon are the result of contamination or another issue (17). For example, a threshold used in the Diagnostic targETEd seQuencing adjudicaTion analysis software uses three SDs (95% confidence) above the mean aligned read count of the NTCs for an amplicon reference as the minimum threshold that a sample needs to exceed for that same amplicon to be reported as detected in a sample (18).

PCR Assay Controls

These guidelines are focused on detection and identification of specific targets discriminating between a narrow range of microbes, potentially including a subset of a species or strain, in contrast to more general, broad target AS schemes such as 16S rRNA. AS method validation controls should always include ground truth material containing true positives (or simulated true positives) and true negatives to ensure PCR primer performance. Primer performance should be validated in the same format that will be used in the method, e.g., a multiplex assay. Initial primer performance validation serves as a go/no-go screen for the primer assays and should be evaluated to confirm the correct band size and specificity. Synthetic positive controls with unique barcodes for target amplicons that are distinguishable by sequence content can be used in the PCR workflow. It is critical to further interrogate sample positive or negative controls that are not performing as intended to identify where the workflow can be better optimized or where laboratory contamination is being introduced (Supplemental Worksheet 1).

Inhibition Control

Assay performance should also be evaluated in the specific matrix of interest to assess the contribution of inhibitors. When isolating nucleic acids from complex environmental matrices such as soil or wastewater, it is critical to consider whether there are inhibitors present in the sample that could disrupt the performance of PCR amplification leading to false negatives (19). There are multiple ways to detect the presence of inhibitors including internal amplification controls, parallel amplification controls, and dilution curves. Inhibition controls should be included in the method for all sample types and from each sample site because differing soils and environmental locations will have different levels and types of inhibitors. A dilution curve can be used to select an optimal sample dilution depending on balancing the inhibitors detected and the expected variability and abundance of target gene copies (i.e., that which maximizes signal). Supplements, such as proteinase inhibitors, can also be added to the qPCR reaction to enhance amplification (20).

Library Preparation and Amplicon Sequencing

Library preparation (LP) compatibility with the sequencing platform is important. In general, best practices that apply to all AS LP methods include: (1) minimizing amplification cycles to obtain sufficient yield without introducing unnecessary amplification errors and bias, (2) reducing cross-contamination risk by minimizing LP steps and transfers, and (3) using high-fidelity polymerases compatible with the LP chemistry (21, 22). (Note: If deoxy-UTP [dUTP] is included in initial PCR amplification, any subsequent PCR must be performed with a dUTP-compatible polymerase.)

Library Preparation

Most AS LP methods and sequencing platforms require DNA purification steps to remove extraneous oligonucleotides (e.g., primers, adaptors), as well as other reaction components (e.g., proteins, salts) that can interfere with downstream processes. LP batches should include appropriate control samples/libraries to monitor for cross-contamination, because there is a potential risk for contamination at this step (23). The yield and fragment size of the LP should be assessed not only to ensure that the LP was successful, but to determine the volume of library material that should be loaded onto the sequencer and allow for sample concentration normalization when multiple indexed/barcoded libraries will be sequenced together. Although some LP methods make yield and size determination optional, e.g., by using bead binding methods to normalize and limit loading concentrations, a pre-sequencing analysis of libraries to ensure adequate yield and representation on the sequencing run is a critical QC measure, especially in the field of biodefense, where high-confidence calls are a requirement. Consistently evaluating the performance of the LP method is critical for confidence in the AS results (Supplemental Worksheet 1).

Sequencing

Laboratories performing PCR and next-generation nucleotide sequencing methods may already have the existing required equipment or are planning to purchase new equipment. A number of next generation sequencing platforms are suitable for AS. Factors to consider when selecting a platform include: throughput, cost, turnaround time, ease of use, contamination risks, laboratory setting (mobile or stationary), amplicon length, and assay type (low-level or no multiplexing versus highly multiplexed, and presence/absence versus sequencing of an informative region).

Additionally, sequencing objectives dictate the required number of reads and read length. This could mean adjusting the number of cycles for synthesis-based sequencers or adjusting the run time for direct detection sequencers. More reads may be necessary to obtain high-confidence calls when read quality is low, when the assay is highly multiplexed (multiple amplicons competing for coverage), and for mixed/complex samples that may produce different amplicon sequences for the same region/assay. Regardless of sequencer and assay specifics, quality checks should be performed that include confirmation that sequence quality scores meet expectations for the sequencer, and that the number of reads produced per run—and more importantly per sample—are adequate to meet analytical objectives.

Depending on the sequencing approach, users should monitor and mitigate the possibility of on-instrument “contamination.” This includes intra-run crossover via read mis-assignment (e.g., index hopping; 24), as well as inter-run carryover for some sequencers that wash and re-use fluidics, flow-cells, or other components exposed to sample libraries. Although typically occurring at very low frequencies, even a very small percentage of crossover/carryover from a sample with a highly abundant amplicon can be significant, once again highlighting the need for contamination monitoring using control samples, and the importance of considering both laboratory context and appropriate detection thresholds during data analysis.

Bioinformatic Processing

Assessing Read and Base Call Quality

Sequence quality is a critical factor to ensure accurate detection with AS methods, and one of the most important factors missing from many repositories (25, 26). AS using any system can produce high quality results (27, 28). Accurate single-nucleotide base calls enable the identification of rare single-nucleotide polymorphisms, often important in near-neighbor discrimination (29). Base calling algorithms differ by the type of sequencing used. Understanding the different algorithms used for base calling can help identify relevant read quality thresholds. Quality scores indicate the amount of measurement uncertainty present for a base call (30, 31). For example, for Phred (Q) scores, high-quality scores indicate a low probability of error. Low-quality scores indicate a significant portion of the read is unusable. If used, it may lead to false variant calls and inaccurate conclusions. A related AOAC Stakeholder Program on Agent Detection Assays (SPADA) effort, lays the framework for read quality and reference sequences used in biothreat agent detection, identification, and quantification.

Sequence Cleanup and Filtering

Quality control of AS read data begins with removing adapters and contaminating sequences from the raw reads. A set of read metrics (e.g., reads generated, quality scores [Supplemental Worksheet 1]) should be reported and acceptable criteria for each metric should be defined before beginning the analysis for both reference (32) and amplicon target sequences.

Read Classification and Database Selection

Classification is the process of assigning a given quality-filtered sequence from an amplicon to the most closely matching representative sequence in a database of reference genomes, genome segments, or amplicons. The AS classification problem presents unique challenges, including: (1) a sequencing platform error rate that can be mis-assigned as meaningful biological variation in short target sequences; (2) pair-wise sequence alignments that can constrain or obscure sequence-based information; and (3) ensuring appropriate resolution amongst clades sharing high genome homology. Choosing the most appropriate classification tool for a given application can help ameliorate some of these challenges. There are a myriad of alignment and classification tools available, and each tool has strengths and weaknesses. For these reasons, and given the rapid development in this space, the authors cannot prescribe a specific tool. Rather, we offer some suggestions for selecting the appropriate tool (see Supplemental Text S3). Options include traditional sequence-composition (e.g., k-mer matching [33–36]) and alignment-based tools (37–42). Additionally, newly emerging machine learning methods show increasing promise (43, 44).

The database requirements for AS approaches are somewhat straightforward as the goal is to map the reads from a known selected target following PCR amplification. The quality of the database alignment can be established based on preset criteria, and these presets should be based on the known similarities of near-neighbor sequences in the non-targeted organisms (e.g., perfect [100% match over the entire length of the amplicon] or conditional [≥90% match over ≥90% of the amplicon sequence]). In a complex sample, identifying targeted versus near-neighbor organisms solely based on AS may pose some challenges; therefore, the reference database should contain whole-genome sequences of the targeted organisms as well as whole-genome sequences of organisms that can be found in the sample matrix enabling read mapping to an inclusivity and exclusivity set. The latter may aid in identifying non-target reads (reads that potentially align to other parts of the genome as well as those that can align to multiple genomes from the target environments). When including whole-genome sequences it is recommended to follow the sequence quality recommendations established by the ISO standard 23418 and AOAC SPADA SMPR (32).

Creating Response Actionable Calls

Establishing Thresholds for Action

Amplicon-based detection is suited to qualitative detection (i.e., returning a binary presence/absence result), rather than quantitative detection unless system response is characterized using a controlled dosage of the target. Prescribing read counts for detection cutoffs is not currently feasible due to the number of workflow variables.

Appropriate detection thresholds rely on both (1) pathogen levels associated with an increased risk and (2) the LOD or probability of detection of analyte in a defined matrix (45, 46) (see Supplemental Text S4). The threshold for risk depends on the pathogen, and ranges from any level above the LOD to some prescribed level (e.g., fecal coliform monitoring in recreation waters). Users must also consider how to treat ambiguous results based upon an organism’s risk and historical data (see Supplemental Text S5). If control experiments produce starkly different signals (e.g., read counts) for the positive and negative controls, but a test sample produces a signal between those, then one must have a plan for interpreting the results.

Finally, time requirements are another variable/limitation that must be considered. Real-time monitoring and analyses of sequence read data is feasible in Oxford Nanopore Technology (ONT) sequencing which does not have a fixed time for analysis. For routine monitoring, establishing a fixed run time and/or number of reads is good practice to ensure consistent monitoring and reduce the likelihood of false positives. One may also establish the practice where detection of some pre-established number of “hits” before a given time window would also constitute a positive signal. Because amplicon workflows pre-amplify targets, one could establish an LOD based upon threshold run time using well-controlled numbers of target copies. Ultimately, a clear separation between positive and negative controls needs to be identified and the time for sequencing in a real-time workflow will be dictated by the program requirements and the concept of operations.

Conclusions

AS workflows can provide robust, confident calls for biodetection; however, due diligence in reporting and controls are needed. The ASqMI guidelines represent the first AS minimum reporting guidance document that also provides detailed checklists of methods for end users to evaluate their workflows to improve confidence. Laboratories should employ the FAIR reporting guidelines (findability, accessibility, interoperability, and reusability) as well as ensuring that metadata is appropriately collected (47). While publicly available reporting may not be possible in many biodefense scenarios, workflows must still be validated, and this data structure should still be employed for method consistency and confidence.

CRediT Author Statement

Ishi Keenum: Conceptualization; Writing—original draft, review & editing. Robert Player: Conceptualization; Writing—original draft, review & editing. Jason Kralj: Conceptualization; Writing—original draft, review & editing. Stephanie Servetas: Conceptualization; Writing—original draft, review & editing. Michael D. Sussman: Conceptualization; Writing—original draft, review & editing. Joe Russell: Conceptualization; Writing—original draft, review & editing. Jennifer Stone: Conceptualization; Writing—original draft, review & editing. Sailaja Chandrapati: Conceptualization; Writing—original draft, review & editing. Shanmuga Sozhamannan: Conceptualization; Writing—original draft, review & editing.

Supplementary Material

qsad047_Supplementary_Data
Click here for additional data file. (140.4KB, zip)

Acknowledgments

We acknowledge the expertise provided by all members of the AOAC SPADA, NGS Working Group 2: Linda Beck (JRAD, USA), Sharon Brunelle (AOAC Technical Consultant, USA), Wes Colangelo (Neogen, USA), James Crill (Syracuse University, USA), Paul Davis (MRI Global, USA), Jonathan Jacobs (ATCC, USA), Paul Jackson (LLNL [retired]), Tim Minogue (USAMRIID, USA), Hiroki Nakae, (Japan bio Measurement & Analysis Consortium), Sanjiv Shah (EPA, USA), Kathleen Verratti (JHU/APL, USA), Conrad Turner (US Army, Germany), and Willy Valdivia (Orion Biosciences, USA).

The views expressed in this paper are those of the authors and do not necessarily represent the views or official position of NIST, DBPAO, JPEO or Department of Defense. The authors IK, JK, and SS are US Government employees, and this work was prepared as part of their official duties. Title 17 U.S.C. §105 provides that “Copyright protection under this title is not available for any work of the United States Government.” Title 17 U.S.C. §101 defines a U.S. Government work as a work prepared by a military service member or employee of the U.S. Government as part of that person’s official duties. Certain commercial equipment, instruments, or materials are identified in this paper to foster understanding. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.

Contributor Information

Ishi Keenum, National Institute of Standards and Technology, Biosystems and Biomaterials Division, Complex Microbial Systems Group, Gaithersburg, MD 20899, USA.

Robert Player, The Johns Hopkins University, Applied Physics Laboratory, Laurel, MD 20723, USA; Datirium, LLC, Cincinnati, OH 45526, USA.

Jason Kralj, National Institute of Standards and Technology, Biosystems and Biomaterials Division, Complex Microbial Systems Group, Gaithersburg, MD 20899, USA.

Stephanie Servetas, National Institute of Standards and Technology, Biosystems and Biomaterials Division, Complex Microbial Systems Group, Gaithersburg, MD 20899, USA.

Michael D Sussman, US Department of Agriculture, Agricultural Analytics Division, Livestock and Poultry Programs, Agricultural Marketing Service, Washington, DC 20250 USA.

Joseph A Russell, MRIGlobal, Gaithersburg, MD 20878, USA.

Jennifer Stone, MRIGlobal, Gaithersburg, MD 20878, USA.

Sailaja Chandrapati, Neogen Food Safety, Maplewood, MN 55125, USA.

Shanmuga Sozhamannan, Joint Program Executive Office for Chemical, Biological, Radiological and Nuclear Defense (JPEO-CBRND), Joint Project Lead for CBRND Enabling Biotechnologies (JPL CBRND EB), Frederick, MD 21702, USA; Joint Research and Development, Inc., Stafford, VA 22556, USA.

Conflicts of Interest

The authors report no conflicts of interest.

Funding

Funding for the AOAC SPADA work was co-funded by Joint Program Executive Office (JPEO), Joint Project Lead—Enabling Biotechnologies (CBRND), Defense Biological Product Assurance Office (DBPAO) and the Department of Defense, Deputy Under Secretary of the Army, Test and Evaluation (DoD, DUSA TE). Funding for NIST work was provided by JPEO. The funding agencies played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Supplemental Information

Supplemental information is available on the J. AOAC Int. website.

References

  • 1. Gilbert J.A., Steele J.A., Caporaso J.G., Steinbrück L., Reeder J., Temperton B., Huse S., McHardy A.C., Knight R., Joint I., Somerfield P., Fuhrman J.A., Field D. (2012) ISME J. 6, 298–308. doi: 10.1038/ismej.2011.107 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Paden C.R., Tao Y., Queen K., Zhang J., Li Y., Uehara A., Tong S. (2020) Emerg. Infect. Dis. 26, 2401–2405. doi: 10.3201/eid2610.201800 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Gardner S.N., Frey K.G., Redden C.L., Thissen J.B., Allen J.E., Allred A.F., Dyer M.D., Mokashi V.P., Slezak T.R. (2015) BMC Res. Notes 8, 682. doi: 10.1186/s13104-015-1530-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Player R., Verratti K., Staab A., Bradburne C., Grady S., Goodwin B., Sozhamannan S. (2020) BMC Genomics 21, 166. doi: 10.1186/s12864-020-6557-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Taitt C.R., Leski T.A., Chen A., Berk K.L., Dorsey R.W., Gregory M.J., Sozhamannan S., Frey K.G., Dutt D.L., Vora G.J. (2020) IJMS. 21, 1669.doi: 10.3390/ijms21051669 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Slezak T., Kuczmarski T., Ott L., Torres C., Medeiros D., Smith J., Truitt B., Mulakken N., Lam M., Vitalis E., Zemla A., Zhou C.E., Gardner S. (2003) Brief. Bioinform. 4, 133–149. doi: 10.1093/bib/4.2.133 [DOI] [PubMed] [Google Scholar]
  • 7. US Food and Drug Administration (2018) Considerations for Design, Development, and Analytical Validation of Next Generation Sequencing (NGS) – Based In Vitro Diagnostics (IVDs) Intended to Aid in the Diagnosis of Suspected Germline Diseases Guidance for Stakeholders and Food and Drug Administration Staff. https://www.fda.gov/media/99208/download
  • 8. Bustin S.A., Benes V., Garson J.A., Hellemans J., Huggett J., Kubista M., Mueller R., Nolan T., Pfaffl M.W., Shipley G.L., Vandesompele J., Wittwer C.T. (2009) Clin. Chem. 55, 611–622 [DOI] [PubMed] [Google Scholar]
  • 9. Field D., Garrity G., Gray T., Morrison N., Selengut J., Sterk P., Tatusova T., Thomson N., Allen M.J., Angiuoli S.V., Ashburner M., Axelrod N., Baldauf S., Ballard S., Boore J., Cochrane G., Cole J., Dawyndt P., De Vos P., dePamphilis C., Edwards R., Faruque N., Feldman R., Gilbert J., Gilna P., Glöckner F.O., Goldstein P., Guralnick R., Haft D., Hancock D., Hermjakob H., Hertz-Fowler C., Hugenholtz P., Joint I., Kagan L., Kane M., Kennedy J., Kowalchuk G., Kottmann R., Kolker E., Kravitz S., Kyrpides N., Leebens-Mack J., Lewis S.E., Li K., Lister A.L., Lord P., Maltsev N., Markowitz V., Martiny J., Methe B., Mizrachi I., Moxon R., Nelson K., Parkhill J., Proctor L., White O., Sansone S.-A., Spiers A., Stevens R., Swift P., Taylor C., Tateno Y., Tett A., Turner S., Ussery D., Vaughan B., Ward N., Whetzel T., San Gil I., Wilson G., Wipat A. (2008) Nat. Biotechnol. 26, 541–547. doi: 10.1038/nbt1360 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Yilmaz P., Kottmann R., Field D., Knight R., Cole J.R., Amaral-Zettler L., Gilbert J.A., Karsch-Mizrachi I., Johnston A., Cochrane G., Vaughan R., Hunter C., Park J., Morrison N., Rocca-Serra P., Sterk P., Arumugam M., Bailey M., Baumgartner L., Birren B.W., Blaser M.J., Bonazzi V., Booth T., Bork P., Bushman F.D., Buttigieg P.L., Chain P.S.G., Charlson E., Costello E.K., Huot-Creasy H., Dawyndt P., DeSantis T., Fierer N., Fuhrman J.A., Gallery R.E., Gevers D., Gibbs R.A., San Gil I., Gonzalez A., Gordon J.I., Guralnick R., Hankeln W., Highlander S., Hugenholtz P., Jansson J., Kau A.L., Kelley S.T., Kennedy J., Knights D., Koren O., Kuczynski J., Kyrpides N., Larsen R., Lauber C.L., Legg T., Ley R.E., Lozupone C.A., Ludwig W., Lyons D., Maguire E., Methé B.A., Meyer F., Muegge B., Nakielny S., Nelson K.E., Nemergut D., Neufeld J.D., Newbold L.K., Oliver A.E., Pace N.R., Palanisamy G., Peplies J., Petrosino J., Proctor L., Pruesse E., Quast C., Raes J., Ratnasingham S., Ravel J., Relman D.A., Assunta-Sansone S., Schloss P.D., Schriml L., Sinha R., Smith M.I., Sodergren E., Spo A., Stombaugh J., Tiedje J.M., Ward D.V., Weinstock G.M., Wendel D., White O., Whiteley A., Wilke A., Wortman J.R., Yatsunenko T., Glöckner F.O. (2011) Nat. Biotechnol. 29, 415–420. doi: 10.1038/nbt.1823 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Borchardt M.A., Boehm A.B., Salit M., Spencer S.K., Wigginton K.R., Noble R.T. (2021) Environ. Sci. Technol. 55, 10210–10223. doi: 10.1021/acs.est.1c01767 [DOI] [PubMed] [Google Scholar]
  • 12. Murray D.C., Coghlan M.L., Bunce M. (2015) PLoS One 10, e0124671. doi: 10.1371/journal.pone.0124671 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Choo J.M., Leong L.E., Rogers G.B. (2015) Sci. Rep. 5, 16350. doi: 10.1038/srep16350 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Li A.-D., Metch J.W., Wang Y., Garner E., Zhang A.N., Riquelme M.V., Vikesland P.J., Pruden A., Zhang T. (2017) FEMS Microbiol. Ecol. 94, fix189. [DOI] [PubMed] [Google Scholar]
  • 15. McCarthy A., Chiang E., Schmidt M.L., Denef V.J. (2015) PLoS One. 10, e0121659. doi: 10.1371/journal.pone.0121659 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Frostegård Å., Courtois S., Ramisse V., Clerc S., Bernillon D., Le Gall F., Jeannin P., Nesme X., Simonet P. (1999) Appl. Environ. Microbiol. 65, 5409–5420. doi: 10.1128/AEM.65.12.5409-5420.1999 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. AOAC International Methods Committee (2011) J. AOAC Int. 94, 1359–1381 [PubMed] [Google Scholar]
  • 18. Conrad T.A., Lo C.-C., Koehler J.W., Graham A.S., Stefan C.P., Hall A.T., Douglas C.E., Chain P.S., Minogue T.D. (2019) J. Mol. Diagn. 21, 99–110. doi: 10.1016/j.jmoldx.2018.08.008 [DOI] [PubMed] [Google Scholar]
  • 19. Wilson I.G. (1997) Appl. Environ. Microbiol. 63, 3741–3751 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Schrader C., Schielke A., Ellerbroek L., Johne R. (2012) J. Appl. Microbiol. 113, 1014–1026 [DOI] [PubMed] [Google Scholar]
  • 21. Endrullat C., Glökler J., Franke P., Frohme M. (2016) Appl. Transl. Genom. 10, 2–9 doi: 10.1016/j.atg.2016.06.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Gargis A.S., Kalman L., Berry M.W., Bick D.P., Dimmock D.P., Hambuch T., Lu F., Lyon E., Voelkerding K.V., Zehnbauer B.A., Agarwala R., Bennett S.F., Chen B., Chin E.L.H., Compton J.G., Das S., Farkas D.H., Ferber M.J., Funke B.H., Furtado M.R., Ganova-Raeva L.M., Geigenmüller U., Gunselman S.J., Hegde M.R., Johnson P.L.F., Kasarskis A., Kulkarni S., Lenk T., Liu C.S.J., Manion M., Manolio T.A., Mardis E.R., Merker J.D., Rajeevan M.S., Reese M.G., Rehm H.L., Simen B.B., Yeakley J.M., Zook J.M., Lubin I.M. (2012) Nat. Biotechnol. 30, 1033–1036. doi: 10.1038/nbt.2403 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Esling P., Lejzerowicz F., Pawlowski J. (2015) Nucleic Acids Res. 43, 2513–2524. doi: 10.1093/nar/gkv107 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Guenay-Greunke Y., Bohan D.A., Traugott M., Wallinger C. (2021) Sci. Rep. 11, 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Rajesh A., Chang Y., Abedalthagafi M.S., Wong-Beringer A., Love M.I., Mangul S. (2021) Genome Biol. 22, 106. doi: 10.1186/s13059-021-02332-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Benton B., King S., Greenfield S.R., Puthuveetil N., Reese A.L., Duncan J., Marlow R., Tabron C., Pierola A.E., Yarmosh D.A., Combs P.F., Riojas M.A., Bagnoli J., Jacobs J.L. (2021) Microbiol. Resour. Announc. 10, e00818-21. doi: 10.1128/MRA.00818-21 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. James P., Stoddart D., Harrington E.D., Beaulaurier J., Ly L., Reid S.W., Turner D.J., Juul S. (2020) medRxiv 2020.08.07.20161737. doi: 10.1101/2020.08.07.20161737 [DOI]
  • 28. Ptasinska A., Whalley C., Bosworth A., Poxon C., Bryer C., Machin N., Grippon S., Wise E.L., Armson B., Howson E.L.A., Goring A., Snell G., Forster J., Mattocks C., Frampton S., Anderson R., Cleary D., Parker J., Boukas K., Graham N., Cellura D., Garratt E., Skilton R., Sheldon H., Collins A., Ahmad N., Friar S., Burns D., Williams T., Godfrey K.M., Deans Z., Douglas A., Hill S., Kidd M., Porter D., Kidd S.P., Cortes N.J., Fowler V., Williams T., Richter A., Beggs A.D. (2021) Clin. Microbiol. Infect. 27, 1348.e1–1348-e7 doi: 10.1016/j.cmi.2021.04.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Ross M.G., Russ C., Costello M., Hollinger A., Lennon N.J., Hegarty R., Nusbaum C., Jaffe D.B. (2013) Genome Biol. 14, R51. doi: 10.1186/gb-2013-14-5-r51 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Ewing B., Hillier L., Wendl M.C., Green P. (1998) Genome Res. 8, 175–185. doi: 10.1101/gr.8.3.175 [DOI] [PubMed] [Google Scholar]
  • 31. Ewing B., Green P. (1998) Genome Res. 8, 186–194. doi: 10.1101/gr.8.3.186 [DOI] [PubMed] [Google Scholar]
  • 32. In ternational Organization for Standardization. (2022) Microbiology of the food chain — Whole genome sequencing for typing and genomic characterization of bacteria — General requirements and guidance. ISO 23418:2022. https://www.iso.org/standard/75509.html (accessed May 1, 2023)
  • 33. Altschul S.F., Gish W., Miller W., Myers E.W., Lipman D.J. (1990) J. Mol. Biol. 215, 403–410 doi: 10.1016/S0022-2836(05)80360-2 [DOI] [PubMed] [Google Scholar]
  • 34. Langmead B., Salzberg S.L. (2012) Nat. Methods. 9, 357–359 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Li H. (2013) arXiv preprintarXiv:1303.3997. doi: 10.48550/arXiv.1303.3997 [DOI]
  • 36. Li H. (2018) Bioinformatics 34, 3094–3100. doi: 10.1093/bioinformatics/bty191 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Kim D., Song L., Breitwieser F.P., Salzberg S.L. (2016) Genome Res. 26, 1721–1729 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Wood D.E., Salzberg S.L. (2014) Genome Biol. 15, R46. doi: 10.1186/gb-2014-15-3-r46 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Ounit R., Wanamaker S., Close T.J., Lonardi S. (2015) BMC Genomics 16, 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Dilthey A.T., Jain C., Koren S., Phillippy A.M. (2019) Nat. Commun. 10, 3066. doi: 10.1038/s41467-019-10934-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Ondov B.D., Treangen T.J., Melsted P., Mallonee A.B., Bergman N.H., Koren S., Phillippy A.M. (2016) Genome Biol. 17, 132. doi: 10.1186/s13059-016-0997-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Bui V.-K., Wei C. (2020) BMC Bioinformatics 21, 468. doi: 10.1186/s12859-020-03777-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Zhang J.X., Yordanov B., Gaunt A., Wang M.X., Dai P., Chen Y.-J., Zhang K., Fang J.Z., Dalchau N., Li J., Phillips A., Zhang D.Y. (2021) Nat. Commun. 12, 4387. doi: 10.1038/s41467-021-24497-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Wan Y.K., Hendra C., Pratanwanich P.N., Göke J. (2022) Trends Genet. 38, 246–257. doi: 10.1016/j.tig.2021.09.001 [DOI] [PubMed] [Google Scholar]
  • 45. Int ernational Organization for Standardization. (2019) Molecular biomarker analysis — Determination of the performance characteristics of qualitative measurement methods and validation of methods. ISO/TS 16393:2019. https://www.iso.org/standard/67450.html (accessed May 1, 2023)
  • 46. Wehling P., LaBudde R.A., Brunelle S.L., Nelson M.T. (2011) J. AOAC Int. 94, 335–347 doi: 10.1093/jaoac/94.1.335 [DOI] [PubMed] [Google Scholar]
  • 47. Wilkinson M.D., Dumontier M., Aalbersberg I.J.J., Appleton G., Axton M., Baak A., Blomberg N., Boiten J.-W., da Silva Santos L.B., Bourne P.E., Bouwman J., Brookes A.J., Clark T., Crosas M., Dillo I., Dumon O., Edmunds S., Evelo C.T., Finkers R., Gonzalez-Beltran A., Gray A.J.G., Groth P., Goble C., Grethe J.S., Heringa J., 't Hoen P.A.C., Hooft R., Kuhn T., Kok R., Kok J., Lusher S.J., Martone M.E., Mons A., Packer A.L., Persson B., Rocca-Serra P., Roos M., van Schaik R., Sansone S.-A., Schultes E., Sengstag T., Slater T., Strawn G., Swertz M.A., Thompson M., van der Lei J., van Mulligen E., Velterop J., Waagmeester A., Wittenburg P., Wolstencroft K., Zhao J., Mons B. (2016) Sci. Data 3, 160018. doi: 10.1038/sdata.2016.18 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

qsad047_Supplementary_Data
Click here for additional data file. (140.4KB, zip)

Articles from Journal of AOAC International are provided here courtesy of Oxford University Press

RESOURCES