Contemporary Considerations for Establishing Reference Methods for Antibacterial Susceptibility Testing

ABSTRACT

Antibacterial susceptibility testing (AST) is performed to guide therapy, perform resistance surveillance studies, and support development of new antibacterial agents. For 5 decades, broth microdilution (BMD) has served as the reference method to assess in vitro activity of antibacterial agents against which both novel agents and diagnostic tests have been measured. BMD relies on in vitro inhibition or killing of bacteria. It is associated with several limitations: it is a poor mimic of the in vivo milieu of bacterial infections, requires multiple days to perform, and is associated with subtle, difficult to control variability. In addition, new reference methods will soon be needed for novel agents whose activity cannot be evaluated by BMD (e.g., those that target virulence). Any new reference methods must be standardized, correlated with clinical efficacy and be recognized internationally by researchers, industry, and regulators. Herein, we describe current reference methods for in vitro assessment of antibacterial activity and highlight key considerations for the generation of novel reference methods.

INTRODUCTION

Antimicrobial resistance is a pressing global concern, challenged by both a limited number of new antimicrobial agents in development and the absence of rapid tests for diagnosis of infections caused by antimicrobial resistant pathogens. Programs such as the Combating Antibiotic-Resistant Bacteria Biopharmaceutical Accelerator (CARB-X, www.carb-x.org) aim to address these challenges by providing incentives for the development of new therapeutics and novel diagnostic tests for assessing the activity of both old and new antibacterial agents. As research progresses on these fronts, there is a need for a single, comprehensive resource that explains reference methods for in vitro assessment of antibacterial activity, against which both novel antibacterial agents and diagnostic tests can be measured. Of note, these reference methods evaluate in vitro activity and should correlate with in vivo (clinical) efficacy. The correlation of in vitro activity to clinical efficacy is accomplished by the application of clinically relevant breakpoints that are used to interpret the test results into reporting categories, such as susceptible or resistant, that can be used to guide therapy choices. These breakpoints are defined from correlation of AST results (MICs) to pharmacokinetic-pharmacodynamic endpoints, clinical trial results and organism MIC distributions (1).

Antibacterial susceptibility testing (AST) is primarily performed to guide patient therapy but is also important for antimicrobial resistance surveillance studies and the development of new antibacterial agents. For 5 decades, broth microdilution (BMD) and agar dilution methods have been widely utilized to assess in vitro activity of antibacterial agents. Although both have been considered reference methods at times, BMD is now the commonly accepted reference method (2). The disk diffusion method is a standard method that relies on correlation with the BMD (or in some cases agar dilution) reference method to interpret the significance of a disk diffusion result. All three methods rely on in vitro inhibition or killing of bacteria, an endpoint that cannot be applied to some newer antibacterial agents, such as those that target a microbe’s virulence. It is likely that additional reference methods will ultimately be needed to examine the effects of these and other novel agents. These new reference methods must be standardized, correlated with clinical efficacy, and recognized internationally by researchers, industry, and regulators. Establishing a new reference method to assess antibacterial activity and clinical efficacy is a substantial challenge as existing methods have been intrenched for decades. Herein, we provide the background on current reference methods for evaluating in vitro activity of antibacterial agents and explore principles that must be considered when developing a new reference method. Specifically, we 1) define the term, “reference method” for AST and describe the desired features of a reference method, 2) discuss current AST reference methods and the limitations of these for testing new antibacterial agents or evaluating new AST methods, 3) describe measures to ensure the reliability of reference AST methods and the extent to which variability in a reference AST method can be controlled, and 4) discuss potential alternatives or additions to current reference AST methods.

Our intention is to initiate the pathway toward consensus on definitions, and more importantly, the future of reference AST methods, which are key for all stakeholders addressing antimicrobial resistance, including antibacterial drug researchers and developers, AST device manufacturers, regulators and diagnosticians.

CURRENT REFERENCE METHODS FOR ANTIBACTERIAL SUSCEPTIBILITY TESTING

The ideal AST reference method is one that is feasible to perform in a routine microbiology laboratory and for which as many parameters as possible have been standardized to ensure a reproducible and quantitative endpoint. Additionally, no components of the test should be proprietary – these tests should be performed using widely available reagents from multiple manufacturers. Reproducibility is key as reference results are used as a correlate to downstream clinical and microbiological outcomes, for both new and older antibacterial agents (e.g., assessment of ’susceptible’ versus ’resistant’). Additional consequential downstream applications of an AST reference method include qualifying novel agents for further development based on in vitro activity and serving as the comparator method when asssessing the accuracy of new AST methods. It is important to note that all methods (including current reference methods) display some level of imprecision, even when performed under standardized conditions (3–5). Nonetheless, the internationally recognized primary reference method for AST of rapidly growing aerobic bacteria is MIC determination using BMD according to the procedure outlined in the International Standards Organization (ISO) standard 20776-1 (2). The CLSI M07 standard (6) for BMD is essentially the same as ISO 20776-1 and in fact, CLSI M07 was a precursor to the ISO standard. The reference BMD (rBMD) is based on work conducted by Ericsson and Sherris in the early 1970s (7). A series of 2-fold dilutions of each agent that relate to concentrations achievable in vivo are tested. The MIC value is defined as the lowest concentration of an antibacterial agent that, under defined in vitro conditions, prevents the appearance of visible growth of a microorganism within a defined period, and is expressed in μg/mL. Reproducibility of this method is within one 2-fold dilution (i.e., +/-1 log₂), although individual isolates and antimicrobials may show a more extensive variability. The method is performed manually and requires standard microbiology laboratory equipment (e.g., incubators, autoclaves, and pipettes).

rBMD is the definitive method for AST used by the Clinical and Laboratory Standards Institute (CLSI) and the European Committee on Antimicrobial Susceptibility Testing (EUCAST), two international standards development organizations for AST. EUCAST relies exclusively on ISO 20776-1 while the CLSI rBMD method is published in their standards document M-07 (05) and as noted above is essentially the same as the ISO method. Fastidious organisms are only briefly addressed in the ISO standard, but CLSI (6) and EUCAST (8) describe a method similar to rBMD for these organisms, using different testing media (as listed below) and, in some cases, different incubation conditions.

rBMD is performed in polystyrene microtiter trays with a final test volume of 100 to 200 μL per well containing solutions with increasing concentrations of the antibacterial agent. Both ISO 20776-1 and CLSI M07 provide explicit instructions for preparation of the BMD panels and steps that must be taken to ensure panels were made properly by applying quality control procedures (2, 6). Freshly prepared panels can be stored at ≤–60°C for up to 3 months (2) or until quality control results or other evidence indicates degradation of the antimicrobial agent (6, 8).

Historically, CLSI recognized agar dilution and broth macrodilution (also known as tube dilution) as reference methods for many agents (6), but rBMD is currently used almost exclusively as the reference method for most bacteria and antibacterial agents. Neither ISO nor EUCAST recognize broth macrodilution as a reference method. However, both recognize agar dilution only for those agents and/or organisms for which BMD does not give reliable results (e.g., fosfomycin and amdinocillin) (2, 9). The test parameters for rBMD that must be strictly controlled include those listed in Box 1 (2).

BOX 1. TEST PARAMENTERS CONTROLLED FOR BY THE ISO 20776-1 REFERENCE BMD METHOD

• Formulation, quality, handling, and storage of antibacterial powders

• Preparation and storage of antibacterial agent stock solutions

• Preparation of test media

• Panel preparation (including dispensing of media into test wells of the microdilution plate)

• Panel storage

• Preparation of the bacterial inoculum suspension and inoculum size

• Panel inoculation method

• Incubation atmosphere, temperature and length

• Method by which MIC endpoints are interpreted

The standard medium for rBMD is Mueller-Hinton broth (MHB), an undefined medium that is supplemented with defined concentrations of the cations, calcium and magnesium, which is referred to as cation-adjusted MHB (CAMHB). The concentration of these cations and also zinc and thymidine are known to have significant effects on MIC results for some antibacterial agents with some bacteria and must be carefully controlled (2, 6, 10). In addition, CAMHB must be modified further in specific situations, e.g., higher concentration of Ca²⁺ for daptomycin testing and iron depleted CAMHB for cefiderocol testing (2, 6, 9). ISO recommends adding 2.5 to 5.0% lysed horse blood to CAMHB for testing of Streptococcus spp. but does not address other bacterial pathogens that fail to grow satisfactorily in CAMHB. CLSI also recommends CAMHB supplemented with 2.5 to 5.0% lysed horse blood for Streptococcus spp. and Neisseria meningitidis and recommends Haemophilus Test Medium (HTM) for Haemophilus spp. (11) EUCAST has developed a common broth for BMD testing of Streptococcus spp. and Haemophilus spp., and several other fastidious organisms, which is CAMHB with 5% lysed horse blood and 20 mg/L beta-NAD (MH-F broth) (8). For Neisseria gonorrhoeae, CLSI recommends agar dilution with GC agar base and 1% defined growth supplement. Neither ISO nor EUCAST describe a reference method for AST of N. meningitidis or N. gonorrhoeae.

The MIC value is determined by visual inspection of the wells, but not all organism-agent combinations give clear endpoints. CLSI (6) and EUCAST (12) provide specific instructions, including photographs with examples, to facilitate reliable determination of some difficult-to-read MIC endpoints, yet such subjective determination of MIC endpoints is often challenging.

AST methods other than rBMD are calibrated to produce results comparable to those obtained with rBMD, including the disk diffusion methods described by both CLSI and EUCAST. These alternative methods provide step by step instructions which must be strictly adhered to and are often referred to as “standard” methods. These methods should not be confused with “reference” methods, although disk diffusion is often referred to as a “reference method,” as both CLSI (M02) and EUCAST (13) publish standards which describe disk diffusion testing and both are based on the descriptions published by Bauer et al. in 1966 (14) and Ericsson and Sherris in 1971 (15). Results from disk diffusion tests are calibrated to match the results of rBMD (2, 6, 16), i.e., correlates back to the rBMD method. Additionally, the criteria to interpret disk diffusion zone measurements (disk diffusion breakpoints) are established based on correlates to categorical interpretations defined from MIC values, and thus disk diffusion does not fit the description of a reference AST method, as described here as it is not a stand-alone reference, but rather a standardize method calibrated to the MIC. However, it is considered a reference method by CLSI (11) and the U.S. FDA.

ENSURING QUALITY OF REFERENCE METHODS

The earliest reports of AST included recommendations for testing strains of bacteria of known susceptibility (17). With time, several strains of bacteria similar to those encountered in clinical infections, so-called quality control (QC) strains, were selected and repeatedly tested with antibacterial agents available at the time to determine the reproducibility of MIC values for each organism/antimicrobial agent. Subsequently, an acceptable MIC range was established for each QC strain. For the test to be in control, the MIC for a specific QC strain must fall within the established QC range. Most of the QC strains used early on are still recommended by ISO, CLSI, and EUCAST today.

The QC strains listed for rBMD testing of nonfastidious bacteria with antibacterial agents with Gram-negative activity include Escherichia coli ATCC 25922 and Pseudomonas aeruginosa ATCC 27853. For Gram-positive agents, the recommended QC strains are Staphylococcus aureus ATCC 29213 and Enterococcus faecalis ATCC 29212. Staphylococcus aureus ATCC 29213 produces beta-lactamase because of acquired resistance due to blaZ genes. The other three strains are wild-type strains. There are several additional QC strains suggested for use when testing beta-lactam combination agents that produce one or more specific type of beta-lactamase. It is essential to include these strains to ensure the beta-lactamase inhibitor component of the beta-lactam combination agent is present and functional. Both ISO (4), CLSI (6) and EUCAST (18) recommend Streptococcus pneumoniae ATCC 49619 for quality control of rBMD testing of Streptococcus species. CLSI recommends this strain for quality control of rBMD for Neisseria meningitidis and H. influenzae ATCC 49247 and H. influenzae ATCC 49766 QC strains for quality control of rBMD testing for Haemophilus spp. (6). H. influenzae ATCC 49766 is also recommended by EUCAST for rBMD of Haemophilus spp. (18).

ISO and CLSI and EUCAST describe specific recommendations for establishing QC ranges for antibacterial agents tested by rBMD (2, 6, 16, 19). This involves testing the agent with the designated QC strain(s) multiple times at several independent laboratories. Replicate testing is performed using CAMHB from at least three different manufacturers. Over two hundred MIC values are generated for the antimicrobial agent-QC strain in question and the QC range is established statistically, normally by, including >95% of the MIC values. This usually encompasses a 2-, 3-, or 4-fold dilution range. QC ranges are evaluated over time for drift, and routinely reassessed.

Once QC ranges are established, it is incumbent on investigators performing rBMD to use them and it is generally recommended to test the QC strains each day a run of experiments is performed. It is common, and desirable, for an investigator to supplement the recommended QC strains with additional QC strains and/or perform additional steps to ensure the quality of rBMD testing. This is particularly important to control the performance of more resistant strains in the rBMD test, as QC strains are generally highly susceptible to most agents. For example, the QC range for E. coli 25922 is several dilutions below the susceptible breakpoint for most agents tested in clinical practice and that would be encountered among isolates tested routinely in a clinical laboratory. Because of this, the lowest concentrations tested for rBMD panels may be greater than the expected QC results, and the quality control of the test not appropriately evaluated, if these evaluations rely solely on the routine QC strains.

USE OF REFERENCE METHODS FOR AST

Reference methods for AST are used during development of new antibacterial agents and results from such testing are reviewed by regulatory agencies to assess the in vitro activity of the agent during the drug approval process and to set MIC susceptibility interpretive criteria or reference broth microdilution and corresponding disk diffusion zone correlates for the indicated drug-organism combinations. In the USA, the Food and Drug Administration (FDA) recognizes the CLSI reference method (https://www.fda.gov/media/77442/download) and in Europe, the European Medicines Agency (EMA) recognizes the reference method used by EUCAST as described in ISO 20776-1.

Reference methods for AST are also used during commercial AST development. In the USA, commercial methods for AST (including disks, gradient diffusion strips and other manual or automated AST methods) must receive clearance by the FDA prior to marketing for diagnostic testing in clinical laboratories. FDA clearance of AST devices requires the manufacturer to perform extensive in vitro studies to demonstrate the commercial method produces results that are comparable or have been correlated with those obtained using the CLSI rBMD method. The FDA has established performance thresholds for both essential and categorical agreement (major, very major and minor errors) that must be achieved for the commercial system to receive FDA clearance (20). Additionally, the ISO document 20776-2 provides performance guidelines focused on essential agreement for commercial AST systems in comparison to reference broth microdilution (21). If the breakpoints (interpretive criteria) are updated, the manufacturer must reapply for clearance with the U.S. FDA for the updated breakpoints (22). For FDA clearance of disk brands that were not evaluated as part of the NDA, the manufacturer needs to submit data to FDA to demonstrate that the disk performed comparably to a predicate FDA cleared disk and/or to MIC. However, FDA now requires more data and comprehensive guidelines for disk clearance are in development, which is summarized in the publicly available Decision Summary for K181975, the Oxoid Meropenem-Vaborbactam disk (30 μg) (https://www.accessdata.fda.gov/cdrh_docs/reviews/K181975.pdf) (Ribhi Shawar, personal communication to RMH). Molecular technologies, such as whole-genome sequencing, are being increasingly used to study antibacterial resistance. When a gene for antibacterial resistance is identified, it can be useful to determine if and how the gene is expressed in biological terms. Reference methods for AST represent one approach for to examine if gene expression results in a change in MIC, although there are no standards for such investigations.

LIMITATIONS OF CURRENT REFERENCE METHODS AND SUGGESTIONS FOR ADDRESSING THESE LIMITATIONS

The rBMD method has been successfully used for evaluating in vitro activity of antibacterial agents for decades, however, this method has its limitations. With newer technologies, it is possible that some of these limitations might be overcome as suggested in Tables 1 and 2.

TABLE 1.

Limitations of rBMD related to simulating in vivo activity

rBMD test requirement	Limitation	How might this limitation be overcome
Use bacteria isolated in pure culture	May introduce bias for testing populations of bacteria that have a growth advantage in culture May miss synergistic activity of bacteria growing in a mixture	Test patient’s clinical specimen (vs bacterial isolate) directly Enable testing of mixed populations
Use of a simple microbiological broth culture media	Poor simulation of the physiological environment at the infection site	Test using conditions comparable to those found at the infection site, or perform test in situ
Use of an inoculum size (i.e., no. of bacteria) that is infrequently observed in clinical specimens	May miss resistant subpopulations that occur at frequencies below the recommended inoculum size	Test inoculum size(s) that occurs at infection site
Performed on planktonic bacteria	Infections are often due to bacteria in biofilms or include bacterial spores, which are frequently more resistant to antibacterial agents	Test bacteria at a state comparable to that occurring at the infection site
Tests twofold dilutions of antibacterial concentrations at a single time point	Results in MIC that is an estimation of actual MIC that can be related to drug concentrations in vivo	Test a continuum of drug concentrations and examine their effects over time

TABLE 2.

Limitations of rBMD related to technical aspects of testing

rBMD test requirement	Limitation	How might this limitation be overcome?
Antibacterial powder procurement may involve legally binding agreement between supplier and recipient	May present challenges depending on investigator’s workplace rules	Develop plan for access of antibacterial powders for use in in vitro testing
Use of a simple microbiological broth culture media	Variability exists in the manufacturing method for current Mueller-Hinton broth, which may lead to variability in components May not support the growth of all bacterial pathogens May require adjustment for agents that have special chemical requirements for antibacterial activity Many agents unstable at incubator temp in broth for long periods of time	Identify a medium that might be appropriate for a wider range of rBMD testing or one that could easily be adjusted for specific organisms/antimicrobial agents
Requirement for16–20 hours of culture growth for the inoculum used in rBMD	Inability to quickly proceed to testing when performed from patient specimens	Reducing the time of subculture before setting up a reference (overnight) test
16−24 hours incubation of rBMD	Inability to quickly proceed to next steps in investigations	Determine if results could be accelerated with use of instrumentation or novel AST methods
Tests twofold dilution concentrations of antibacterial agent	The “gaps” between dilutions may be significant May yield results reported as susceptible or resistant when isolate is on the “borderline” of resistance	Tighten range of concentrations tested and/or test a continuum of concentrations Create an Area of Technical Uncertainly (as per EUCAST) to alert clinicians of potential “borderline” results
Results read visually	Subjective interpretation Some endpoints difficult to interpret	Utilize instrumentation that enables objective measurement of endpoint
Accuracy and reproducibility is monitored with a few QC strains, mostly wild type	QC strains often not representative of patient’s isolates and/or have MIC values outside range of concentrations tested	Describe additional measures for QC such as: assaying drug content in media; comparing performance of new lots with old lots; defining optimal QC strains for a specific antibacterial agent based on range of concentrations tested
Method not conducive to high-throughput testing	Limits practicality of use for large scale evaluations	Evaluate instrumentation / automation to enable high-throughput testing

NOVEL (NOT RBMD) PHENOTYPIC METHODS AS A REFERENCE

In recent years, several novel methods for AST have been developed. These have primarily focused on reducing time to results from patient specimen acquisition, from a traditional 2 to 3 days to methods that can be performed directly on patient specimens with results available within hours (23). Improvements include use of novel inoculum preparation methods and alternative methods of endpoint determination for antibacterial susceptibility, including observation of antimicrobial-induced changes to bacterial cell morphology, division rates and gene expression (23). The cost of goods for these new methods is significantly higher than rBMD, which is relatively inexpensive to perform.

Regulatory clearance of some of these new AST methods have been based on correlation to rBMD. The rBMD result provides a reasonable, albeit imperfect, correlation to treatment outcomes. Sometimes referred to as the 90-60 rule, infections caused by isolates defined as ‘susceptible’ respond well to therapy in 90% of cases whereas isolates defined as ‘resistant’ by the reference method respond well in 60% of cases (24). Inference is made that correlation of a new AST method with the reference method (typically rBMD) equates to similar correlation of the new method with treatment outcomes. In both the U.S. and Europe, substantial equivalence determination against rBMD is sufficient to qualify a new AST as suitable for patient testing.

The imperfect correlation between rBMD and clinical outcomes is due to many factors outside the test that impact treatment outcome (e.g., host immune response, pharmacokinetics, site of infection, source control, organism virulence factors, among others) (25–28), but also due to limitations inherent to the rBMD method (see Table 1). The idea that novel methods could overcome these limitations is enticing, but paradoxically if the novel method overcomes the limitations of rBMD, then the novel method could poorly correlate to rBMD. Since correlation of results to rBMD is unlikely or even impossible, widespread adoption of a novel method will be challenging and require substantial data to replace the current rBMD. Correlation of results from these novel tests to clinical outcome is needed to determine whether the phenotypes observed are artifact of the testing conditions or true representations of bacterial susceptibility to a given antibacterial agent with in vivo relevance.

Novel AST methods are penalized even if they do not share the limitations of rBMD, as “truth” is assumed to be the MIC result obtained by the rBMD. Theoretically, a paradigm shift away from rBMD is needed, but such a major change requires solid justification, by providing substantial analytical, cost benefit analysis and clinical improvement over existing reference methods. Demonstrating such improvement would require costly, large-scale clinical assessments (29). Careful selection of endpoints for such trials is important, as traditional outcome measures (e.g., mortality, microbiological cure, and/or duration of symptoms) are challenging to measure for diagnostic tests, as trial outcomes are contingent not only on the test’s performance in vitro, but also on timely clinical action based on the test results. Additionally, trial size requirements, already onerous for diagnostic companies, will likely need to be larger than simple correlation studies to rBMD, to account for enrolled but culture-negative patients (if testing direct-from specimen without preculture). As an example, randomized controlled trials conducted to assess the impact of more rapid AST results on management of patients with bacteremia have been unable to determine impact of these on mortality, adverse drug events, or length of hospitalization due to insufficient patients enrolled (30). Nonetheless, AST device results must be shown to correlate to clinical outcomes. Alternative, composite endpoints, such as the Antimicrobial Resistance Leadership Group’s desirability of outcome ranking (DOOR) approach, whereby patient outcome is ranked on both benefits and harms experienced (31). Other scoring approaches have been used in clinical trials for antimicrobials that could be similarly applied to AST (32). Similarly, correlation of novel outputs of antimicrobial susceptibility will need to be used to correlate to pharmacokinetic/pharmacodynamic endpoints, as this is foundational for developing clinical breakpoints, given the limitations to clinical trial data sets (1).

Protocolized trials with predefined action points have eluded diagnostic development to date and data suggest even very rapid (15 to 30 min time to result) ASTs are unlikely to impact the first dose of antimicrobials, so studies must focus on antimicrobial optimization (29). In the context of critically ill patients, the decision to rapidly deescalate therapy is unlikely, even if more rapid results are available (RMH, unpublished data) (33), and as such foundational noninferiority studies that evaluate impact of earlier AST results alone, let alone one that does not necessarily correlate with rBMD, are required.

For certain novel agents that may require a companion diagnostic (i.e., specific in vitro device that provides information essential for the safe and effective use of a corresponding antimicrobial) (34), codevelopment of new antibacterial agents together with specific AST may help better position both in the clinical market and provide synergistic data on the clinical impact of both the AST and the antibacterial. Due to the nature of several novel antibacterials, discussed below, such studies are inevitable. However, this approach requires very early collaboration between pharmaceutical and diagnostic companies, due to significant mismatch between development time frames of diagnostics and pharmaceuticals, differing returns on investment models, time scales and success rates between these two industries.

MOLECULAR METHODS AS A REFERENCE

Molecular methods include genotypic (assessment for one or more resistance genes) or genomic (assessment of resistance determinants across a genome). Both have been used as surrogates for phenotypic AST under certain conditions (23). Several features inherent to molecular methods make these attractive as a potential reference AST. Molecular methods may be more reproducible than rBMD as they are not as impacted by the slight variations in testing parameters as are phenotypic methods. Furthermore, a more objective interpretation of endpoints is possible, and the analytical sensitivity is high, enabling direct-from-specimen testing. Time to results can be very rapid (e.g., minutes for genotypic methods) and the skills required for test performance are similar to those for any molecular assay, although to date standardization and quality control is not yet well defined for more complex methods, like whole-genome sequencing.

Despite these advantages, some major limitations remain which prohibit use of molecular tests as a reference AST. Primarily, this includes the fact that molecular methods are best at predicting resistance to a given antimicrobial (i.e., due to the presence of a resistance determinant), but not necessarily at predicting susceptibility (i.e., absence of a gene). This is additionally complicated by the fact that the presence of a gene may not always indicate expression of resistance in vivo. Furthermore, genotypic tests are by definition limited by their ability to only assess for known resistance determinants; new mutations and novel mechanisms will be missed. Molecular methods described to date also provide a qualitative assessment of susceptibility, in contrast to the quantitative (MIC) results derived from rBMD. Nonetheless, there are instances where a genotypic result is used as the reference. The most straightforward example is the use of genotypic tests for mecA and/or mecC to define oxacillin resistance among the staphylococci. These tests are used on primary specimens, bacteria propagated in culture and are also recommended for testing of small colony variants of Staphylococcus aureus that do not grow satisfactorily to assess oxacillin susceptibility using phenotypic tests based on growth, including by rBMD. Use of genotypic tests as a reference for other types of resistance, such as beta-lactam resistance among Gram-negative bacteria is more complicated since multiple genes may encode resistance to a single agent and the presence of these genes may not correlate with clinical resistance. For example, there are many genes and gene variants that confer carbapenem resistance among isolates of Enterobacterales. While the presence of a specific gene(s) is often considered to correlate with resistance, there are a significant number of exceptions. Alternatively, absence of a gene or a negative test result may indicate susceptibility but does not exclude resistance due to another mechanism of action.

Increased use of DNA sequencing techniques, including whole-genome sequencing and metagenomics, bring the possibility of using genomic data as a reference for AST closer to reality. For Mycobacterium tuberculosis, this method is becoming increasingly well-established (35). However, much like genetic testing, genomic methods suffer from incomplete/unknown resistance profiles for given antimicrobials or microorganisms. Given the often complex and rapidly evolving array of resistance mechanisms for bacteria, methods that incorporate machine learning (i.e., computer prediction of AST results when unknown sequences are encountered) (36) may be required to improve correlations between genomic data and antimicrobial susceptibility results and/or clinical outcomes. At present, genomic data are processed using custom-based analysis pipelines with little standardization. Many databases that include both genomic and phenotypic AST data are either based on nonreference phenotypic results or include only susceptibility category and not MIC (37). Standardization of sequencing approaches and database standards are required before these methods can be universally applicable.

NEW REFERENCE METHODS FOR NOVEL ANTIBACTERIAL AGENTS

Current reference methods were developed to assess the inhibitory or killing effect of bacteria in an in vitro setting with the intention that in vitro results would correlate with in vivo efficacy and be useful in assessing and/or predicting clinical outcome. rBMD, therefore, is useful for drugs that work directly on bacterial cells by either inhibiting growth or killing bacteria. As researchers continue to explore improved therapies for treating bacterial infections, novel mechanisms of action are being explored. These include agents that target bacterial growth, antibacterials that cannot be evaluated in the current rAST method, phage therapy, monoclonal/polyclonal antibodies, agents that target bacterial virulence, and agents that target the host. For these agents, the current rAST may need to be significantly modified or may be mechanistically irrelevant. In cases where the current rAST are irrelevant, new standard reference methods will need to be developed that can assess the drug effect on the bacteria as well as provide a test result that correlates with clinical outcomes. The topic of phage susceptibility testing has recently been evaluated by the Antimicrobial Resistance Leadership Group, which highlighted the lack of a standard reference method for phage susceptibility testing and absence of in vitro parameters that will predict (as yet) undefined clinically efficacy endpoints (38). Clearly, the solution to this challenge is complex, and will require consensus among the FDA, standards development organizations and pharmaceutical/biotechnology companies. Development of a consensus group to assess these challenges, led by standards development organizations and including input from researchers and pharmaceutical/biotechnology companies, is needed to develop a standardized framework for how to approach testing these compounds.

CONCLUSIONS

In the current era of antimicrobial resistance and novel therapeutics, consensus on reference test methods, and an assessment of the need for new reference AST methods is needed. While associated with some limitations (Table 1 and 2), the rBMD method serves as a current gold standard for AST. As new antimicrobials and diagnostics are developed, it is certain that additional standard methods will be needed for assessing antibacterial activity and novel agents. Drawing on over 50 years’ experience with rBMD, a robust, systematic evaluation of the current reference method, and how the limitations outlined herein might be overcome, will help to inform this work. In addition, systematic study of newer methods, including novel phenotypic methods and genotypic approaches are needed to identify modern reference standards. The latter task is complex as the comparator for such assessments should be something other than rBMD, given its limitations. The ultimate assessment would be to imbed these newer technologies into the clinical trials for new antibacterial agents, so that the results might be correlated with clinical success and failures. Precedent for such outputs includes the work of the MERINO trial, which demonstrated a close link between in-vitro MIC results and clinical outcome, dependent on the test methodology. However, the complexity of such studies cannot be over emphasized, as they are confounded by patient heterogeneity and careful selection of endpoints is required. Regardless of approach, we maintain there remains a need for reference methods that demonstrate the biological activity of current and future antibacterial agents and that can also detect novel emerging resistance mechanisms.