Skip to main content
BMJ Open logoLink to BMJ Open
. 2019 Oct 30;9(10):e030747. doi: 10.1136/bmjopen-2019-030747

Bacterial antibiotic resistance development and mutagenesis following exposure to subminimal inhibitory concentrations of fluoroquinolones in vitro: a systematic literature review protocol

Carly Ching 1, Ebiowei SF Orubu 2, Veronika J Wirtz 3, Muhammad H Zaman 1,
PMCID: PMC6830604  PMID: 31666265

Abstract

Introduction

Antibiotic resistance (AR) is among the most pressing global health challenges. Fluoroquinolones are a clinically important group of antibiotics that have wide applicability in both humans and animals. While many drivers of AR are known, the impact of medicine quality on AR remains largely unknown. The aim of this review is to systematically evaluate the evidence of the impact of in vitro subinhibitory antibiotic exposure, a major tenet of substandard antibiotics, on the development of AR and mutagenesis, using fluoroquinolones as a case study.

Methods and analysis

EMBASE, Web of Science and PubMed will be systematically searched for primary experimental in vitro studies, from earliest available dates within each database (1947, 1965 and 1966, respectively) through 2018, related to subinhibitory fluoroquinolone exposure and AR. A specifically developed non-weighted tool will be used to critically assess the evidence. Subgroup analyses will be performed for different variables and outcomes.

Ethics and dissemination

Ethical approval is not required as no primary data are to be collected. The completed systematic review will be disseminated through conference meeting presentations and a peer-reviewed publication.

Keywords: microbiology, infectious diseases, health policy


Strengths and limitations of this study.

  • This study aims to be the first systematic review of this body of evidence.

  • · NCBI PubMed, ISI Web of Science and Elsevier Embase will be searched for studies on the impact of subinhibitory concentrations of fluoroquinolones on antimicrobial resistance from earliest available dates within each database through 2018 without language limitation.

  • Basic microbiological experimental studies will be reviewed following Preferred Reporting Items for Systematic Review and Meta-Analyses protocol standards, which is currently not best practice.

  • Data will be placed in the context of the important global health issue of medicine quality, with broad implications in mortality, morbidity and antibiotic resistance. The knowledge afforded by the review can provide a foundation for further research studies on substandard antibiotics.

  • Review is limited to in vitro studies of bacterial monocultures, limiting translation to the clinic.

Introduction

Antibiotic resistance (AR) is a rapidly growing global health threat. To provide evidence for improved clinical and public health interventions and policies, it is paramount to understand both the social drivers of AR development and the underlying scientific mechanisms. These drivers include antibiotic usage in the environment and the clinic, as well as access and quality of antibiotics.1 2

Poor-quality antibiotics, specifically substandard antibiotics, is one possible understudied driver of AR.3 Substandard drugs are defined by the WHO as ‘authorized medical products that fail to meet either their quality standards or their specifications, or both’.3 The prevalence, or failure rate, of substandard antibiotics and other anti-infectives in low/middle-income countries has been reported to be about 7%.3 Prevalence estimates are currently limited to low/middle-income countries, with more data needed for high-income countries.4 Substandard antibiotic products often contain inadequate levels of the active pharmaceutical ingredient (API) (not falling within the stated concentration or quality standards) or have lower than expected/specified bioavailability arising from poor dissolution. This can result in the treatment of bacteria at subinhibitory concentrations below their minimum inhibitory concentration (MIC). In this case, there is not enough API to completely clear the bacterial infection but there may be enough API to provide selective pressure for AR development. Thus, medicine quality may be a potentially important driver of AR; however, there is currently a lack of direct evidence to support this hypothesis.5

While systematic reviews of observational studies provide critical evidence for developing clinical interventions and public health policies, there is a lack of a similar systematic approach in reviews of experimental bench research—the science which underlies and explains what occurs clinically. To identify important scientific trends and bring awareness to the topic of medicine quality, we have extracted an underlying scientific question for a systematic review: Does subinhibitory fluoroquinolone exposure increase bacterial antibiotic resistance development and mutagenesis?

Here, we seek to systematically synthesise and critically appraise experimental evidence on how subinhibitory concentrations of one specific class of antibiotics, fluoroquinolones, impacts AR. We have chosen fluoroquinolones as they are a commonly used class of synthetic antibiotics, effective against both Gram-negative and Gram-positive bacteria, in both human and animals. Resistance emergence against fluoroquinolones has been widely reported for several decades.6–9 Second-generation to fourth-generation fluoroquinolones stem from the initial non-fluorinated first-generation quinolone class; these synthetic molecules share a bicyclic quinolone-related core structure, with a fluorine on the sixth carbon position. 9–11 In addition to substandard antibiotic exposure clinically, bacteria are exposed to subinhibitory antibiotic concentrations in other situations, such as in the environment from wastewaters or agricultural soils which can have implications in AR development and transmittance.12

Currently, there are only a few broad narrative literature reviews on the impacts of subinhibitory concentrations of antibiotics.13–16 To our knowledge, there are currently few systematic reviews of basic or fundamental microbiological bench research.17–19 Thus, we seek to perform an unbiased systematic literature review according to Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines on the topic of subinhibitory fluoroquinolone exposure and AR development.

The results of this systematic review will contribute to the understanding of the impact of exposure of bacteria to subinhibitory levels of fluoroquinolones on AR acquisition. Secondarily, we seek to identify gaps in evidence related to medicine quality in an effort to inform policy-making on the control of substandard medicines. This work can contribute to a rigorous evidence-base of bench research based on systematic review which includes critical appraisal of existing literature instead of narrative review and selective reporting.

Systematic review questions

This review seeks to address the following questions:

  1. Does subinhibitory fluoroquinolone exposure increase bacterial antibiotic resistance development and mutagenesis in vitro? (Primary).

  2. What is the potential for substandard fluoroquinolone drugs to lead to antibiotic resistance development? (Secondary).

Methods

Our methodology will conform to the PRISMA reporting standards (online supplementary appendix 1, Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols checklist). The protocol does not currently exist elsewhere and is ineligible for hosting on PROSPERO because it the study participants are not people or animals. The duration of this study is estimated to be 6 months.

Supplementary data

bmjopen-2019-030747supp001.pdf (114.6KB, pdf)

Patient and public involvement

It was not appropriate or possible to involve patients or the public in this work.

Eligibility criteria

To define the search approach and inclusion and exclusion criteria, we applied a Population Intervention Comparator Outcome Study search tool. The criteria are presented in table 1.

Table 1.

Population Intervention Comparator Outcome Study design criteria

Include  Exclude 
Population 
  • Bacteria (all isolates of Gram-negative and Gram-positive species)

  • Eukaryotes (all).

  • Archaea.

Intervention
(exposure)
  • Exposure to ranges of fluoroquinolone (second to fourth-generation) concentrations with levels below the defined MIC*, under controlled in vitro experimental conditions.

* Defined as the concentration visibly inhibiting growth in the experimental set-up. Methods employed would include broth and agar dilution methods and commercially available MIC test strips.
  • Exposure to first-generation quinolone antibiotics, for example, nalidixic acid, or other classes of antibiotics.

  • Exposure to sub-MIC fluoroquinolone concentrations in combination with another class of antibiotic or compound.

  • Purely computational models.

  • Studies involving animals.

Comparator
  • No treatment, MIC at 0% API of parental strain.

Outcomes
  • Quantitative experimental microbiological data related to:

(1) resistance acquisition (to same or other antibiotic) and (2) mutagenesis rate.
  • Examples of data include standard microbiological assays (ie, phenotypical tests, commercially available antibiotic susceptibility tests and molecular and PCR assays for identification of mutations).

  • Whether any mention of substandard of medicine quality within the paper (yes/no).

  • Outcomes from studies that treat bacteria with subinhibitory levels but do not follow-up with results related to resistance acquisition or mutagenesis. Examples of results to exclude include community behaviour, such as surface cell adhesion and biofilm formation, virulence (persister formation, toxin/antitoxin systems) and plasmid curing.

Study design
  • Primary experimental studies (all languages) published from 1966 to 2018 on NCBI PubMed, from 1965 to 2018 on ISI Web of Science and from 1947 to 2018 on Elsevier Embase.

  • Conference abstracts.

  • Review articles (no primary data).

  • Observational studies.

API, active pharmaceutical ingredient;MIC, minimum inhibitory concentration.

Outcomes, prioritisation and data extraction

The primary outcome extracted will be the effect of exposure to subinhibitory concentrations of fluoroquinolones on (1) AR acquisition (monoresistance and multidrug resistance) and (2) mutagenesis. A secondary outcome extracted will be whether these papers discuss substandard or poor quality medicines. Our rationale for prioritisation is that we first need to determine the link between exposure and resistance acquisition. After quantifying and evaluating the evidence, we aim to assess how frequently primary scientific papers mention or discuss medicine quality. Other variables extracted from each study will include year of publication, bacterial species and number of strains, type of bacterial isolate (clinically isolated vs reference strain), drug name and concentration, and study method (duration of exposure, growth conditions and so on). Study quality and limitations after quality assessment, and gaps in evidence for review questions will also be extracted. Data will be extracted to a standardised Excel table. The data will be summarised and standardised as described in the Data synthesis section.

Each paper will be analysed and key results extracted to a standardised table for comparison by a single reviewer. For a random sample of 10% of the publications, a second reviewer will extract the data. The results will be compared with the first. If the interrater reliability is moderate or low all data extraction will be done independently by two reviewers.

Search strategy

The search strategy was based on review questions and a preliminary search of PubMed to determine relevant Medical Subject Headings (MeSH) terms. Using MeSH terms and keyword synonyms along with identified terms for subinhibitory and substandard, a search strategy was designed in PubMed and translated to Web of Science and Embase to search all fields for articles that fit the inclusion criteria above (table 1). Identified search terms are listed below. Search strings were designed with a medical librarian. Additional records will be identified through searching the bibliographies of identified studies and searching through papers that have cited key studies. The complete search terms are provided in online supplementary appendix 2.

Supplementary data

bmjopen-2019-030747supp002.pdf (53.1KB, pdf)

Identified terms: subinhibitory, sub-inhibitory, sub inhibitory, sub-lethal, sublethal, sub lethal, subminimal, sub-minimal, sub minimal, sub-therapeutic, subtherapeutic, sub therapeutic, sub MIC, sub-MIC, low-dose, low dose, substandard, sub-standard, counterfeit, falsified.

Study records

Records will be managed through reference management software Endnote and Mendeley. Additionally, search histories will be saved. Abstract screening and selection of studies will be performed by two independent reviewers using software Rayyan QCRI.20 A third researcher will resolve discrepancies between reviewers selections. The full text of articles from the initial screening will be reviewed for inclusion.

Risk of bias in individual studies

Risk of bias for laboratory microbiology experimentation will be assessed with criteria formulated by considering and adapting the Systematic Review Centre for Laboratory animal Experimentation’s risk of bias tool for animal studies21 and the Effective Public Health Practice Project quality assessment tool.22 The criteria are presented in table 2. Here, we present a non-weighted assessment of individual study quality, including risk of bias. For each of five domains, studies will be assessed for a series of criteria listed below. For each unmet review criteria within the domain, an increased risk of bias point will be assigned. The more points assigned, the higher the risk of bias associated with the study. There will be no defined cut-off for exclusion of papers, in order for the review to be reflective of the evidence base as a whole. This will allow us to determine how strong the body of evidence is as a whole and to perform a qualitative assessment of the most frequent types of gaps in quality to inform recommendations for future studies. Papers will also have to meet a minimum criteria of ability to extract data on methods and results; for example, appropriate quantitative numerical data on study outcome.

Table 2.

Criteria for assessment of the quality of laboratory microbiology experimentation

Domain Description of domain Review criteria
Selection and confounding bias Describe possible genetic or environmental variations to determine how results for different strains and isolates of the same species can be compared. For clinical isolates, genotype is not required.
  • Were the groups compared individually or were differences discussed in the analysis?

  • Were species and strain details provided?

Study design/methods Reproducibility and detail of study design and methods. Description of analysis methods.
  • Are there any discrepancies between methods and in-text?

  • Is the methodological section missing any steps or appropriate detail? (including but not limited to below)

Steps/details:
  • Media used.

  • Temperature.

  • Time.

  • Incubation conditions (static, rolling, shaking, aeration).

  • Reagents used.

  • Concentrations used.

  • Appropriate control experiments.

  • Replication of experiments.

Incomplete outcome data Completeness of outcome data being analysed, including loss and exclusion of data from analysis.
  • Is there missing outcome data that was not addressed?

  • Is the control outcome data mentioned in the paper present?

Selective outcome reporting
(reporting bias)
Reporting of aim and all outcomes of the study.
  • Was all data reported for all conditions or just select/statistically significant results?

  • Was it clear whether no change results were reported?

  • Was statistical significance noted (if possible)?

  • Is the appropriate comparison to baseline provided?

Other sources of bias Potential bias not covered by other domains.
  • Was the study apparently free of additional concerns about bias?

Global bias score Summary of all five domains Calculate total quality points. The more points the higher the risk of bias.

Data synthesis

Meta-analysis may not be possible based on findings and will be defined by the limitations of the raw data extracted. It will be dependent on the magnitude of heterogeneity between independent studies and ability to assign an effect-size that would be appropriate. If heterogeneity is too large meta-analyses will not be performed in order to avoid over-interpretation. If we cannot assign a true appropriate control group and true ‘sample size’, meta-analysis will also not be possible. However, despite these potential limitations this is a novel review of experimental evidence that aims to provide a comprehensive synthesis of data that is much more complete than one individual study and which may reveal trends. It is clear that more tools need to be developed to move the field of basic science towards systematic reviews.

Quantitative subgroup analyses and summarisation will be performed. The following protocol, in brief, will be used: data will be extracted into a standardised Excel spreadsheet. From here, data will be sorted and grouped for each independent variable, such as bacterial species, concentration of exposure and antibiotic. The rationale for subgroups is as follows. First, different bacterial species often respond differently to stress or have different genetic responses to different stimuli. For example, the clinically relevant pathogen Acinetobacter baumannii, which has a propensity to gain multidrug resistances, has a different DNA damage response compared with the conserved paradigm of Escherichia coli.23 This impacts how these two bacteria respond to stress and such differences between bacterial species may lead to differences in responses to subinhibitory fluoroquinolone exposure. Concentration of exposure is an important factor, as different concentrations may present different selective compartments.24 Similarly, it is of clinical interest to determine if certain fluoroquinolones impact bacteria differently, given that fluoroquinolones (with different usage and prescription patterns), display differences in pharmacodynamics and resistance profiles.25 26 The dependent outcome of change in resistance and mutagenesis will be plotted against these factors. The values of outcomes (relative change in resistance) will also be binned. This will allow us to determine the range and frequency of magnitudes of resistance changes given different concentrations and different antibiotics.

Meta-bias(es)

Based on data synthesis parameters, the overall quality of the body of evidence will be determined, if possible. Since we will not be able to make direct clinical recommendations due to the limitations of our review being focused on in vitro studies, we will focus on confidence in our overall summary of results and trends. For this we will take into account publication bias across studies. Carroll et al 27 identified how publication bias may exist in scientific literature and described potential solutions; however, these are not best practice. Publication bias could arise from, but is not limited to, rejection of negative data, researchers not submitting research that present negative data, publication based on results rather than the quality and rigour of the study design and influence of industry and funding sources. All of these factors can lead to a skewed set of data that does not fully represent the phenomena being investigated. Narrative literature reviews of basic science typically do not critically assess the bias of each study and hence, do not take into account quality in their summary which is an important limitation of narrative reviews. We will use Grading of Recommendations, Assessment, Development and Evaluations (GRADE) guidelines on publication bias to aid us in rating the quality of our evidence.28 Specifically, to assess publication bias, we will look at the group of studies to determine how many studies had increased bias for not reporting negative results, or only reporting statistically significant results. This criteria is already present in our risk of bias analysis for individual studies. We will look at the studies that published negative results and determine if there are differences in the impact factor/prestige of journal that they are published in, and whether data are coming from the same research groups. We will also look at the final number of studies and time of publication in order to identify if there is any potential ‘lag bias’.28 Funding sources, specifically frequency of industry funded studies, will be noted. Acknowledging that publication bias is difficult to assess, as suggested by GRADE, we will then determine if publication bias is ‘undetected’ or ‘strongly suspected’ and rate down a maximum of one level for suspected bias.28 Additionally, biases in our set of evidence towards certain bacteria, antibiotics and inconsistencies in methodology and outcomes will be assessed and taken into account in determining the confidence of our reported data summary.

Ethics and dissemination

Ethical approval is not required as no primary data are to be collected. The completed systematic review will be disseminated through conference meeting presentations and a peer-reviewed publication.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

We would like to acknowledge David Flynn from Boston University Alumni Medical Library for assistance with search terms and members of the Zaman lab for their input and feedback.

Footnotes

Contributors: CC, VW and MZ initiated work. CC, ESFO, VW and MZ designed and wrote protocol.

Funding: This work was funded by a fellowship to CC by the United States Pharmacopeia.

Competing interests: None declared.

Patient consent for publication: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

  • 1. Chatterjee A, Modarai M, Naylor NR, et al. Quantifying drivers of antibiotic resistance in humans: a systematic review. Lancet Infect Dis 2018;18:e368–78. 10.1016/S1473-3099(18)30296-2 [DOI] [PubMed] [Google Scholar]
  • 2. Okeke IN, Klugman KP, Bhutta ZA, et al. Antimicrobial resistance in developing countries. Part II: strategies for containment. Lancet Infect Dis 2005;5:568–80. 10.1016/S1473-3099(05)70217-6 [DOI] [PubMed] [Google Scholar]
  • 3. World Health Organization A study on the public Heath and Socieconomic impact of substandard and Falsified medical products 2017.
  • 4. World Health Organization A study on the public Heath and Socieconomic impact of substandard and Falsified medical products, executive summary 2017:1–29.
  • 5. Holmes AH, Moore LSP, Sundsfjord A, et al. Understanding the mechanisms and drivers of antimicrobial resistance. Lancet 2016;387:176–87. 10.1016/S0140-6736(15)00473-0 [DOI] [PubMed] [Google Scholar]
  • 6. Kim ES, Hooper DC. Clinical importance and epidemiology of quinolone resistance. Infect Chemother 2014;46:226–38. 10.3947/ic.2014.46.4.226 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. McEwen SA, Fedorka‐Cray PJ. Antimicrobial use and resistance in animals. Clin Infect Dis 2002;34:S93–106. 10.1086/340246 [DOI] [PubMed] [Google Scholar]
  • 8. Acar JF, Goldstein FW. Trends in bacterial resistance to fluoroquinolones. Clin Infect Dis 1997;24 Suppl 1:S67–73. 10.1093/clinids/24.Supplement_1.S67 [DOI] [PubMed] [Google Scholar]
  • 9. Dalhoff A. Global fluoroquinolone resistance epidemiology and implictions for clinical use. Interdiscip Perspect Infect Dis 2012;2012:1–37. 10.1155/2012/976273 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Tillotson GS. Quinolones: structure-activity relationships and future predictions. J Med Microbiol 1996;44:320–4. 10.1099/00222615-44-5-320 [DOI] [PubMed] [Google Scholar]
  • 11. Redgrave LS, Sutton SB, Webber MA, et al. Fluoroquinolone resistance: mechanisms, impact on bacteria, and role in evolutionary success. Trends Microbiol 2014;22:438–45. 10.1016/j.tim.2014.04.007 [DOI] [PubMed] [Google Scholar]
  • 12. Van Doorslaer X, Dewulf J, Van Langenhove H, et al. Fluoroquinolone antibiotics: an emerging class of environmental micropollutants. Sci Total Environ 2014;500-501:250–69. 10.1016/j.scitotenv.2014.08.075 [DOI] [PubMed] [Google Scholar]
  • 13. Andersson DI, Hughes D. Microbiological effects of sublethal levels of antibiotics. Nat Rev Microbiol 2014;12:465–78. 10.1038/nrmicro3270 [DOI] [PubMed] [Google Scholar]
  • 14. Zhanel GG, Kim SO, Davidson RJ, et al. Effect of subinhibitory concentrations of ciprofloxacin and gentamicin on the adherence of Pseudomonas aeruginosa to Vero cells and voided uroepithelial cells. Chemotherapy 1993;39:105–11. 10.1159/000239110 [DOI] [PubMed] [Google Scholar]
  • 15. Blázquez J, Rodríguez-Beltrán J, Matic I. Antibiotic-Induced genetic variation: how it arises and how it can be prevented. Annu Rev Microbiol 2018;72:209–30. 10.1146/annurev-micro-090817-062139 [DOI] [PubMed] [Google Scholar]
  • 16. Broom LJ. The sub-inhibitory theory for antibiotic growth promoters. Poult Sci 2017;96:3104–8. 10.3382/ps/pex114 [DOI] [PubMed] [Google Scholar]
  • 17. O'Hagan EC, Matalon S, Riesenberg LA. Systematic reviews of the literature: a better way of addressing basic science controversies. Am J Physiol Lung Cell Mol Physiol 2018;314:L439–42. 10.1152/ajplung.00544.2017 [DOI] [PubMed] [Google Scholar]
  • 18. Anon Why basic microbiology still matters. Nat Rev Microbiol 2011;9:480 10.1038/nrmicro2611 [DOI] [PubMed] [Google Scholar]
  • 19. van der Putten BCL, Remondini D, Pasquini G, et al. Quantifying the contribution of four resistance mechanisms to ciprofloxacin MIC in Escherichia coli: a systematic review. J Antimicrob Chemother 2019;74:298–310. 10.1093/jac/dky417 [DOI] [PubMed] [Google Scholar]
  • 20. Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan—a web and mobile APP for systematic reviews. Syst Rev 2016;5:1–10. 10.1186/s13643-016-0384-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Hooijmans CR, Rovers MM, de Vries RBM, et al. SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol 2014;14:43 10.1186/1471-2288-14-43 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Thomas BH, Ciliska D, Dobbins M, et al. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews Evid Based Nurs 2004;1:176–84. 10.1111/j.1524-475X.2004.04006.x [DOI] [PubMed] [Google Scholar]
  • 23. Macguire AE, Ching MC, Diamond BH, et al. Activation of phenotypic subpopulations in response to ciprofloxacin treatment in Acinetobacter baumannii. Mol Microbiol 2014;92:138–52. 10.1111/mmi.12541 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Baquero F. Low-Level antibacterial resistance: a gateway to clinical resistance. Drug Resist Updat 2001;4:93–105. 10.1054/drup.2001.0196 [DOI] [PubMed] [Google Scholar]
  • 25. Rolston KVI, Frisbee-Hume S, LeBlanc BM, et al. Antimicrobial activity of a novel des-fluoro (6) quinolone, garenoxacin (BMS-284756), compared to other quinolones, against clinical isolates from cancer patients. Diagn Microbiol Infect Dis 2002;44:187–94. 10.1016/S0732-8893(02)00433-9 [DOI] [PubMed] [Google Scholar]
  • 26. Becnel Boyd L, Maynard MJ, Morgan-Linnell SK, et al. Relationships among ciprofloxacin, gatifloxacin, levofloxacin, and norfloxacin MICs for fluoroquinolone-resistant Escherichia coli clinical isolates. Antimicrob Agents Chemother 2009;53:229–34. 10.1128/AAC.00722-08 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Carroll HA, Toumpakari Z, Johnson L, et al. The perceived feasibility of methods to reduce publication bias. PLoS One 2017;12:1–19. 10.1371/journal.pone.0186472 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Nolting A, Perleth M, Langer G, et al. [GRADE guidelines: 5. Rating the quality of evidence: publication bias]. Z Evid Fortbild Qual Gesundhwes 2012;106:670–6. 10.1016/j.zefq.2012.10.015 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2019-030747supp001.pdf (114.6KB, pdf)

Supplementary data

bmjopen-2019-030747supp002.pdf (53.1KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES