Skip to main content
Wellcome Open Research logoLink to Wellcome Open Research
. 2020 Jun 1;5:13. Originally published 2020 Jan 27. [Version 2] doi: 10.12688/wellcomeopenres.15681.2

ACORN (A Clinically-Oriented Antimicrobial Resistance Surveillance Network): a pilot protocol for case based antimicrobial resistance surveillance

Paul Turner 1,2,a, Elizabeth A Ashley 2,3, Olivier J Celhay 4, Anousone Douangnouvong 3, Raph L Hamers 2,5, Clare L Ling 2,6, Yoel Lubell 2,4, Thyl Miliya 1, Tamalee Roberts 3, Chansovannara Soputhy 1, Pham Ngoc Thach 7, Manivanh Vongsouvath 3, Naomi Waithira 2,4, Prapass Wannapinij 4, H Rogier van Doorn 2,8
PMCID: PMC7250055  PMID: 32509968

Version Changes

Revised. Amendments from Version 1

The updates to this version are minor. In response to reviewer comments the following points have been added: 1. Included a definition of diagnostic stewardship; 2. Clarified the rationale for the target pathogens; 3. Added detail of how sites made ward selections; 4. Confirmed that multiple enrolments are possible; 5. Updated data visualisation section to confirm that infections due to non-target species can be summarised; 6. Included additional detail of enrolment process.

Abstract

Background: Antimicrobial resistance (AMR) / drug resistant infections (DRIs) are a major global health priority. Surveillance data is critical to inform infection treatment guidelines, monitor trends, and to assess interventions. However, most existing AMR / DRI surveillance systems are passive and pathogen-based with many potential biases. Addition of clinical and patient outcome data would provide considerable added value to pathogen-based surveillance.

Methods: The aim of the ACORN project is to develop an efficient clinically-oriented AMR surveillance system, implemented alongside routine clinical care in hospitals in low- and middle-income country settings. In an initial pilot phase, clinical and microbiology data will be collected from patients presenting with clinically suspected meningitis, pneumonia, or sepsis. Community-acquired infections will be identified by daily review of new admissions, and hospital-acquired infections will be enrolled during weekly point prevalence surveys, on surveillance wards. Clinical variables will be collected at enrolment, hospital discharge, and at day 28 post-enrolment using an electronic questionnaire on a mobile device. These data will be merged with laboratory data onsite using a flexible automated computer script. Specific target pathogens will be Streptococcus pneumoniae, Staphylococcus aureus, Salmonella spp ., Klebsiella pneumoniae, Escherichia coli, and Acinetobacter baumannii. A bespoke browser-based app will provide sites with fully interactive data visualisation, analysis, and reporting tools.

Discussion: ACORN will generate data on the burden of DRI which can be used to inform local treatment guidelines / national policy and serve as indicators to measure the impact of interventions. Following development, testing and iteration of the surveillance tools during an initial six-month pilot phase, a wider rollout is planned.

Keywords: Antimicrobial Resistance, Surveillance, Clinical

Introduction

Antimicrobial resistance (AMR) surveillance serves three main purposes: to provide local evidence for empiric treatment guidelines and clinical decision making, to characterise trends in space and time, and to serve as benchmark to measure the impact of interventions. Current AMR surveillance systems are typically passive, pathogen-focused, and based on routine antimicrobial susceptibility testing (AST) results generated by clinical microbiology laboratories, alone. These systems lack the relevant patient-level metadata and clinical syndromic denominators to appropriately inform treatment guidelines and decision making and understand the burden of drug-resistant infections (DRIs) 1.

Surveillance data may suffer from various biases due to lack of diagnostic stewardship (“coordinated guidance and interventions to improve appropriate use of microbiological diagnostics to guide therapeutic decisions” 2) and underuse of diagnostic microbiology resources, especially in low- and middle-income countries (LMICs) 3. Collection of samples for microbiologic testing is often not part of a standard diagnostic work-up for many clinical syndromes. This can be due to many factors, including lack of trust between clinicians and the microbiology laboratory and (national) insurance systems that do not reimburse microbiological diagnostics. Therefore, it is common for samples to be collected only in more severe cases or in case of treatment failure. This limits direct assessment and subsequent modelling of the clinically relevant impacts and burden of DRI. Microbiologists often do not receive all clinical information important for interpreting laboratory results and surveillance data, e.g. whether an infection is community- or hospital-acquired. In addition, many patients have access to over-the-counter antibiotics in the community and are often already taking these when admitted to hospital. All of these biases favour an overrepresentation of results from DRI among surveillance data 4. Therefore, if one were to use current surveillance results and resistance proportions to inform clinical guidelines, there is a risk of contributing to the problem of AMR rather than the solution and advocating the use of broader spectrum antibiotic regimens than would be justified if data were more representative.

In addition to the bias-related problems noted above, several key patient-level questions that are not answered adequately by passive pathogen-focussed AMR surveillance are:

  • What is the impact and cost of DRI at the patient level?

  • What are the patient, hospital and environmental risk factors for DRI in a particular setting?

  • Which AMR-syndrome combinations are associated with the poorest outcomes in particular patient groups?

In 2014 the WHO introduced the Global Antimicrobial Resistance Surveillance System (GLASS) that provides guidance towards standardised global surveillance of AMR focusing on a number of pathogens ("bugs") and antimicrobials ("drugs") 5. GLASS allows submission of "sample-based" in addition to "isolate based" data. Although this approach at least offers more clinical denominator data, it is still subject to the same biases of underuse of microbiology services and lacks clinical metadata on antimicrobial use and duration of hospitalisation 6.

The utility of fully integrated patient and laboratory-based surveillance was highlighted in a recent Fleming Fund funded report on AMR surveillance 7. High-quality patient-level surveillance data from LMICs are necessary to inform models to determine the impact of AMR using large datasets and to identify opportunities for intervention 6. Additionally, results based on patient-level data will be critical to generate reports that resonate with policy makers, i.e. how many people die from DRI and how much does it cost? Whilst these factors argue strongly in favour of more clinically focused surveillance, especially in LMICs, successful examples of such surveillance are limited.

The purpose of ACORN and this pilot study protocol is to establish efficient and pragmatic capture of clinical data with automated linkage to corresponding diagnostic microbiology data. Clinical variable selection was informed by an AMR stakeholder workshop, held in Bangkok in May 2019. The protocol implementation package includes tools to capture site and laboratory capacity information, guidelines on diagnostic stewardship, and a web-based data visualisation and analysis platform. The surveillance protocol is summarised below and key implementation documents are included in the accompanying Extended data 816.

Protocol

Ethics, regulatory approvals and governance

The protocol and participant information sheet has been approved by the Oxford Tropical Research Ethics Committee (OXTREC 536-19; 21 st June 2019), Cambodia National Ethics Committee for Health Research (215-NECHR; 30 th August 2019), Laos Ministry of Health – University of Health Sciences Ethics Committee (211/19; 23 rd September 2019), and National Hospital for Tropical Disease Institutional Review Board, Hanoi, Vietnam (13/HDDD-NDTU; 18 th November 2019).

Surveillance staff will ensure that the participants’ anonymity is maintained. Personal information (i.e. name and telephone number) necessary for post-discharge follow-up will not be entered into the electronic surveillance database and will be recorded only in a paper logbook (subject identification log) at the study site. This logbook will be destroyed as soon as it no longer required. Participants will be identified only by a participant ID number on other surveillance documents and electronic databases. All documents will be stored securely and only accessible by surveillance staff and authorised personnel. The surveillance will comply with the General Data Protection Regulation (GDPR), which requires that personal data must not be kept as identifiable data for longer than necessary for the purposes concerned.

Aims and objectives

The aim of this project is to develop an efficient clinically-orientated AMR surveillance system, implemented alongside routine clinical care in hospitals in LMIC settings. The data collected will expand on WHO GLASS, to enable classification of infection syndromes and outcomes. These data will be used to estimate syndromic and pathogen outcomes along with associated costs.

The primary objective is to develop, implement and assess a hospital-based system for patient-centred surveillance of DRI. Secondary objectives are to systematically characterise DRI based on important clinical syndromes, to adequately inform treatment guidelines; to implement clinical syndrome-guided diagnostic stewardship of patients with suspected infection; and to determine the duration, cost of hospitalisation and patient outcome of DRI and non-DRI. Finally, the tertiary objective is to evaluate the feasibility and acceptability of the surveillance system and package of tools.

The first phase of the project is development and pilot implementation over six months (which started in November 2019). Pilot implementation is occurring in three locations, focusing on a narrow range of syndromes whilst the methodology is established, and will be followed by review and refinement of the surveillance procedures, tools and results.

Surveillance design

This protocol describes collection of clinical and laboratory data of hospitalised patients with clinically suspected meningitis, pneumonia or sepsis, for surveillance purposes. Specific target pathogens will be Streptococcus pneumoniae, Staphylococcus aureus, Salmonella spp., Klebsiella pneumoniae, Escherichia coli, and Acinetobacter baumannii, i.e. the blood culture/invasive infection relevant species from WHO GLASS.

Surveillance sites

The sites included in the pilot phase are the Angkor Hospital for Children, Siem Reap, Cambodia; Mahosot Hospital, Vientiane, Lao People’s Democratic Republic; and the National Hospital for Tropical Diseases, Hanoi, Vietnam. These University of Oxford partner/host organisations were selected on the basis that they cover primary- to tertiary-level government and non-governmental facilities, include the full spectrum of patient groups, and the investigators are familiar with their diagnostic, laboratory and data procedures.

Pre-surveillance site visits will ensure that microbiology laboratory diagnostics and existing data capture procedures meet required standards ( Extended data 8). Summary information about the site will be documented, including number of beds, clinical services, staffing levels, medical records, and investigation/management of surveillance-relevant infections ( Extended data 9).

Investigators at each site will identify appropriate personnel to be included in surveillance training and implementation activities. Surveillance staff will be asked to complete anonymous feedback questionnaires on the surveillance tools, reports, and data visualisations. Early during ACORN site implementation, clinicians working on the selected surveillance wards will be asked to participate in a knowledge, attitudes, and practices (KAP) survey covering AMR, surveillance, and infection diagnosis and management ( Extended data 10). All surveys will be opt-out and formal informed consent will not be obtained.

Sample size

There is no formal sample size or target. This surveillance will enrol all eligible and consenting patients admitted to the surveillance wards during the surveillance period.

Participant selection and recruitment

The surveillance population will consist of hospitalised patients of any age (children and adults) on pre-selected surveillance wards. Sites will choose 2–3 surveillance wards based on their patient populations (“Each site should identify appropriate wards for ACORN surveillance. It may be desirable to start with a small number of wards/departments (e.g. medical admissions ward and the main intensive care unit) and scale up over time. Consideration should be given to harmonisation with other surveillance activities, where possible”) ( Extended data 9). Patients with clinically suspected community- or hospital-acquired infection (CAI/HAI) are eligible to participate in the surveillance. There is no formal consent procedure, but patients will be notified of the project and will be given the option to opt out (see below).

Surveillance procedures

Recruitment. Surveillance participants will be identified, screened and those who meet the eligibility criteria will be consecutively enrolled by surveillance personnel during daily review of new hospital admissions to surveillance wards (CAI) and during scheduled weekly point-prevalence surveys on these wards (HAI). For those patients who are screened and excluded, the reason for exclusion will be recorded. A surveillance screening log will be maintained for this purpose. Surveillance staff will be trained in the protocol and relevant surveillance procedures prior to the start of the project. With the aim of improving diagnostic consistency, international standard clinical case definitions for surveillance syndromes will be used in all site-based training activities and diagnostic stewardship materials, but will not be required to be met for enrolment during the pilot phase ( Extended data 11) 17, 18.

Screening and eligibility assessment. For CAI, a member of the surveillance team (clinician, nurse or research assistant) will review the clinical notes of each new admission on the surveillance wards Monday to Friday. The notes of patients admitted over the weekend or on public holidays shall be reviewed on the following workday. Patients in whom there is a clinical suspicion of meningitis, pneumonia, or sepsis on admission, and also those meeting formal case definitions (Appendix 1, ACORN Diagnostic Stewardship SOP/Form, Extended data 11), will be deemed eligible for inclusion in surveillance ( Extended data 12).

For HAI, patients will be identified during weekly surveys of all patients resident in a bed on the surveillance ward at 8am on the day of the survey, excluding day case patients expected to be admitted and discharged on the same day ( Extended data 13). Patients meeting the following case definition will be deemed eligible for inclusion in surveillance (based on the European Centre for Disease Prevention and Control definition 19):

  • Day 3 of admission onwards OR (Day 1–2 AND patient discharged from acute care hospital in preceding 48 hours) OR (Day 1–2 AND patient has relevant device inserted on this admission prior to onset)

    AND

  • Clinical diagnosis of suspected pneumonia or sepsis (hospital-acquired meningitis is not expected in the pilot sites/wards), or meets formal case definition, on the day of survey OR (patient is receiving treatment AND HAI diagnosis made between Day 1 of treatment and survey day)

    AND

  • HAI syndrome was not active during the previous weekly review (i.e. HAI syndrome onset at least one day following the most recent previous survey)

Patients may be enrolled into surveillance more than once, for example for both CAI and HAI on the same admission, multiple HAI on the same admission, or multiple admissions with CAI and /or HAI.

Informed consent. The research ethics committees detailed above agreed to waive the need for explicit individual informed consent as this surveillance is a minimal/negligible risk activity, consisting of implementation of accepted quality improvement tools (diagnostic stewardship) and collection and use of limited clinical data that is expected to be collected as part of standard of care. No patient samples will be collected other than for clinical diagnostic purposes. All patients admitted to participating wards will be given an information sheet with details about the surveillance ( Extended data 14). There will also be information posters visible on these wards. The information sheet and poster will inform patients regarding the purpose and procedures of the surveillance, what it will involve for the participant, and any risks involved in taking part as well as how to get more information about the surveillance. At the time of enrolment into surveillance, the patient or parent/legal acceptable representative will be approached by a surveillance team member and asked to confirm agreement for participation in surveillance. For those unable to read the information sheet, it will be read to them at this stage. It will be clearly stated that patients have the right to refuse participation at any time, for any reason, without prejudice to future care, and with no obligation to give the reason for withdrawal. It will also be stated how to withdraw from surveillance. Any patient who requests not to be included in surveillance will be recorded accordingly in the surveillance screening logbook and will be diagnosed and treated according to standard clinical care. Surveillance staff will be readily available to provide further information and answer any questions.

Formal consent will not be obtained from clinician or surveillance staff prior to completion of anonymous surveys and feedback questionnaires. It will be explained that participation in these activities is entirely voluntary and that there will be no penalty for refusal. Participants will be informed that the data will be used to help understand better the challenges to implementation of AMR surveillance and to guide further development of the protocol.

Baseline assessments. On the day of enrolment, baseline clinical data will be extracted from the patient clinical records or electronic hospital information systems and by brief interview of the patient:

  • Patient hospital ID code (or other locally-used unique patient identifier).

  • Date of birth or age (if date of birth not known).

  • Sex.

  • Co-morbidity status: cancer, chronic lung disease, chronic renal failure, diabetes mellitus, malnutrition.

  • Date of admission and original hospitalisation, if transferred directly from another hospital.

  • Ward name and type: medical, surgical, paediatric, intensive care unit.

  • Hospitalisation in the last three months before this admission: yes or no.

  • Surgery in the last three months: yes or no.

  • Surveillance category: CAI or HAI.

  • Surveillance diagnosis: meningitis, pneumonia, or sepsis.

  • Severity score on date of admission (CAI) or symptom onset (HAI): qSOFA score for adults (≥18 years) 20 or the “sepsis six” recognition features for children 21.

  • Presence of medical devices: peripheral IV catheter, central IV catheter, urinary catheter, endotracheal tube.

  • Microbiology: whether a blood culture was collected within 24 hours of admission (CAI) or symptom onset (HAI); whether the patient received ≥1 dose of a systemic antibiotic in the 24 hours before blood culture collected.

  • Empiric antibiotic treatment: names of systemic antibiotics prescribed on date of admission (CAI) or symptom onset (HAI).

Subsequent assessments. During hospitalisation. A surveillance clinician will review pathogen positive cases to provide further diagnostic/treatment advice to the responsible clinician. Clinical notes and electronic hospital information systems will be reviewed to capture ICD10 code for infection episode (if routinely generated by the hospital), final classification of the surveillance diagnosis (confirmed or rejected, plus likely source in sepsis cases), hospitalisation outcome and date, and the total number of days admitted to an intensive care unit during the hospitalisation ( Extended data 15).

Day 28 assessment. The participant or parent/legal acceptable representative will be contacted by telephone on day 28 (+/- 7 days) post enrolment to determine post-discharge health status and date of death, if appropriate. For patients enrolled more than once in a single hospital admission (e.g. a CAI and an HAI episode, or multiple HAI episodes), a single assessment will be made, 28 days (+/- 7 days) following the final enrolment date. The day 28 assessment may occur before hospital discharge ( Extended data 15).

Investigations. Specimens taken in the study are those required for routine clinical care only with no extra specimens for surveillance purposes. However, treating clinicians will be reminded of good practices for investigation of patients with suspected infection ( Extended data 11). This diagnostic stewardship will include encouragement to request blood cultures on all patients meeting the surveillance case definitions, and other specimens as indicated (e.g. cerebrospinal fluid on patients with suspected meningitis), as well as appropriate radiologic investigations on selected patients (e.g. abdominal ultrasound scan on patients with sepsis and clinically-suspected liver abscess). Microbiology specimens will be processed by onsite laboratories, to identify pathogens and their antibiotic susceptibility profiles, following approved standard operating procedures (SOPs).

Data management and analysis

Data management. Clinical data will be captured using a short (approximately 5 minute) questionnaire on password protected Android devices via the Open Data Kit Collect (ODK) software ( Figure 1; Extended data 16). These data will be uploaded to a secure server located at the Mahidol Oxford Tropical Medicine Research Unit (MORU) in Bangkok, Thailand and downloaded periodically to password-protected computers at each site via the ODK Briefcase software. Laboratory data will be captured using the sites' existing Laboratory Information Management System (LIMS), WHONET software or using a MORU-designed ACORN LIMS (Microsoft Access) for sites with no existing LIMS. Data will be extracted from these systems into WHONET file format for further analysis. Data validation procedures will be built into the ODK forms and are included in both the ACORN LIMS and WHONET workflows. Automated script-based linkage between the two systems will be performed locally at each site using R Statistical Software 22. Linkages will be made using the participant hospital identification (ID) code, date of birth, and hospitalisation/specimen dates. Participant name and any other explicitly identifying detail will not be included in any analysis. Hospital sites will be identified by a unique code rather than name to reduce the possibility of linking a participant hospital ID to a specific hospital. Each time clinical and laboratory data are merged, surveillance ID will be generated for each participant and the participant hospital ID and date of birth will be deleted automatically, rendering a fully de-identified dataset for analysis and further sharing. For data visualisation and analysis, the de-identified data files will be uploaded to a secure cloud-based server and will be visualised using a bespoke R Shiny interactive dashboard ( Figure 2) 23. Each site will have access only to its own data, which will be available in real-time to maximise local utility.

Figure 1. ACORN data capture and flow.

Figure 1.

CAI, community-acquired infection; HAI, hospital-acquired infection; LIMS, Laboratory Information Management System.

Figure 2. ACORN site-level data dashboard.

Figure 2.

( a) Patient enrolment summary. ( b) Antimicrobial susceptibility summary for Escherichia coli.

Surveillance laboratory assessment, site survey, clinician KAP survey, and feedback questionnaire data will be captured via secure online surveys ( www.jisc.ac.uk). Only aggregate summaries of survey responses will be included in surveillance reports, presentations, and publications to ensure participant anonymity is maintained.

AMR surveillance data analysis. Data will be summarised in tables and graphs on the interactive dashboard. Descriptive statistics will be used where appropriate. For syndrome-based analyses, the denominator will be the number of participants meeting the clinical case definitions. The proportions of participants in whom a blood culture and other syndrome-appropriate diagnostic specimens were collected will be calculated, along with estimation of hospital and 28-day mortality rates. For HAI prevalence, the percentage of ward patients with HAI will be determined, using the total number of patients admitted to the ward on the survey date as the denominator. For specimen-based analyses, the denominator will be the number of participants with a specific syndrome in whom a blood culture was collected. The proportions of cases in whom a pathogen was detected (specifically the selected key pathogens) or where the blood culture was considered contaminated will be calculated. For isolate-based analyses, the denominator will be the number of participants from whom a pathogen was isolated. Summaries will include the proportions of isolates resistant to key antibiotics, as defined by WHO GLASS 5 or categorised as multi-drug resistant, using standard definitions 24. Whilst the surveillance is focussed on a set of key pathogens, it is recognised that other bacterial species will be cultured from surveillance participant specimens. These species will not be included in formal analyses, but the organism and antimicrobial susceptibility data will be summarised on the interactive dashboard.

The impact of DRIs will be defined by compiling the mortality and morbidity data for patients admitted at the sites converted into disability adjusted life years (DALYs) using patient age and discharge diagnoses. The costs of their care will be estimated using data on length of stay (i.e. healthcare direct costs) and for antibiotic treatment, with hospital- and country- specific unit costs attached, respectively. These will be reported for patients with no infection, susceptible infections, and resistant infections. Modelling approaches previously described 25 will be applied to ascertain the incremental costs and DALYs lost that can be attributed to resistant infections as compared with susceptible or no (in the case of HAIs, assuming many of them are preventable) infections, therefore conservatively assuming that resistant infections replace, rather than add to, the burden of susceptible ones.

Surveillance monitoring and evaluation data. Quantitative data will be summarised in tables and graphs. Simple descriptive statistics will be used where appropriate. Qualitative data will be reviewed to identify key themes. Clinician KAP survey data will be used to understand potential barriers to implementation of AMR surveillance at the site level as well as to contextualise the microbiologic sampling data acquired during surveillance, i.e. these data will be used to aid implementation and iteration of the surveillance activities. Feedback questionnaire data will be used to inform updates to surveillance tools.

Dissemination

Results of the pilot will be presented at appropriate local, national and international scientific meetings and a summary manuscript will be prepared. At the conclusion of the pilot phase, an international AMR stakeholder workshop will be held to discuss the results and lessons learned during implementation of the ACORN protocol. It is anticipated that plans for the wider roll out of ACORN will be finalised during this workshop.

Software generated during the development of ACORN will be made available via GitHub at completion of the pilot phase (MS Access LIMS, ODK form templates, R-scripts to merge clinical and laboratory data, and code for the R Shiny data visualisation app).

Study status

Enrolment commenced in Cambodia on 18 th November 2019, in Laos on 12 th December 2019, and in Vietnam on 27 th February 2020.

Discussion

In the pilot phase, ACORN will establish patient-centred and efficient AMR surveillance at three locations within Asian LMICs. Here, preliminary data on key infection syndromes and associated pathogens will be gathered, along with identification of challenges and potential solutions to implementation of the surveillance protocol. The effectiveness of the diagnostic stewardship activities will be monitored as the proportion of cases with linked diagnostic specimens. These data will be used to iterate the protocol and tools prior to wider roll out.

Case definitions are a current area of uncertainty which will be reviewed specifically at the conclusion of the pilot phase. Clinical records are often not extensive in LMIC hospitals and the use of rigid inclusion criteria in surveillance is challenging, especially for hospital acquired infections. For this reason, very pragmatic definitions/clinical criteria have been employed in the pilot phase of ACORN, with capture of simple severity markers to enable patient stratification within a syndrome. Patients may be enrolled based on clinician diagnosis or, in the absence of clear clinician diagnosis, if they are assessed by the surveillance team as meeting the clinical case criteria for meningitis, pneumonia or sepsis. Widely used clinical definitions are used in diagnostic stewardship activities in an attempt to improve standardisation of clinician diagnosis and specimen collection. However, it is recognised that this approach may lead to limitations in inter-site data comparisons. Streamlined clinical case definitions for HAI are under development by US-CDC and WHO and these may be of use in future versions of the ACORN protocol. A hybrid solution would be to document whether a patent was included in surveillance based solely on clinical suspicion or whether a formal case definition was met, although this would require additional training and oversight. Another approach to improve specificity may be to include "prescribed antibiotics" as an inclusion criterion, i.e. specifically document whether the patient was prescribed an antibiotic. This would also add a simple antimicrobial use monitoring component to ACORN. Another area of uncertainty is around comprehensive capture of HAI cases. The use of point prevalence surveys for identification of HAI is time efficient but may result in underestimation of case numbers due to early death, discharge or recovery. However, this issue is common to all point prevalence surveys and not restricted to ACORN.

Importantly, implementation of ACORN in parallel to laboratory capacity building in LMICs will maximise the opportunities for rapid generation of actionable AMR and DRI data. It is hoped that inclusion of a simple to use data visualisation and analysis dashboard will help with local buy in and data use. In settings unfamiliar with diagnostic microbiology, without dedicated clinical support and/or surveillance activities, laboratories are likely to remain under-used and pathogen data summaries will retain current levels of ambiguity. The advantages of case-based surveillance with reporting of full AST profiles was recently highlighted 26. The authors commented that electronic medical records would facilitate such surveillance. These are not yet available in many LMICs and the ACORN clinical data capture tool offers an opportunity to collect such data in the absence of such systems.

In summary, ACORN will generate AMR and DRI data in LMICs that can be used to inform local treatment guidelines/national policy, in addition to inclusion in international pathogen-focused AMR surveillance systems and burden of disease studies. Generation and use of such data is a global health priority.

Data availability

Underlying data

No data are associated with this article.

Extended data

Figshare: ACORN Laboratory Assessment SOP / Form. https://doi.org/10.6084/m9.figshare.11577702.v2 8

Figshare: ACORN Site Preparation SOP / Form. https://doi.org/10.6084/m9.figshare.11577726.v2 9

Figshare: ACORN Clinician KAP Survey / Form. https://doi.org/10.6084/m9.figshare.11577729.v2 10

Figshare: ACORN Diagnostic Stewardship SOP / Form. https://doi.org/10.6084/m9.figshare.11577735.v1 11

Figshare: ACORN Community Acquired Infection Enrolment SOP. https://doi.org/10.6084/m9.figshare.11577738.v1 12

Figshare: ACORN Hospital Acquired Infection Point Prevalence Survey SOP. https://doi.org/10.6084/m9.figshare.11577741.v1 13

Figshare. ACORN Participant Information Sheet. https://doi.org/10.6084/m9.figshare.11676453.v1 14

Figshare: ACORN Patient Follow-up SOP. https://doi.org/10.6084/m9.figshare.11577747.v1 15

Figshare: ACORN ODK Data Entry SOP / Form. https://doi.org/10.6084/m9.figshare.11577750.v1 16

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Acknowledgments

The authors are grateful to the attendees of the ACORN AMR stakeholder meeting held in Bangkok 16–17 th May 2019, who provided insight and input into the protocol and selection of the clinical variables to be captured.

Funding Statement

This work was supported by the Wellcome Trust [215867; to PT].

[version 2; peer review: 4 approved]

References

  • 1. Ashley EA, Recht J, Chua A, et al. : An inventory of supranational antimicrobial resistance surveillance networks involving low- and middle-income countries since 2000. J Antimicrob Chemother. 2018;73(7):1737–49. 10.1093/jac/dky026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Global AMR Surveillance System: Diagnostic stewardship: A guide to implementation in antimicrobial resistance surveillance sites (WHO/DGO/AMR/2016.3). Geneva: World Helath Organization;2016. Reference Source [Google Scholar]
  • 3. Rempel OR, Laupland KB: Surveillance for antimicrobial resistant organisms: potential sources and magnitude of bias. Epidemiol Infect. 2009;137(12):1665–73. 10.1017/S0950268809990100 [DOI] [PubMed] [Google Scholar]
  • 4. Yam ELY, Hsu LY, Yap EPH, et al. : Antimicrobial Resistance in the Asia Pacific region: a meeting report. Antimicrob Resist Infect Control. 2019;8(1):202. 10.1186/s13756-019-0654-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Global Antimicrobial Resistance Surveillance System: Manual for Early Implementation. Geneva: World Helath Organization;2015. Reference Source [Google Scholar]
  • 6. Hay SI, Rao PC, Dolecek C, et al. : Measuring and mapping the global burden of antimicrobial resistance. BMC Med. 2018;16(1):78. 10.1186/s12916-018-1073-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Seale A, Hutchison C, Fernandes S, et al. : Supporting surveillance capacity for antimicrobial resistance: Laboratory capacity strengthening for drug resistant infections in low and middle income countries [version 1; peer review: 2 approved, 1 approved with reservations]. Wellcome Open Res. 2017;2:91. 10.12688/wellcomeopenres.12523.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Turner P: ACORN Laboratory Assessment SOP / Form. figshare.Online resource.2020. 10.6084/m9.figshare.11577702.v2 [DOI] [Google Scholar]
  • 9. Turner P: ACORN Site Preparation SOP / Form. figshare.Online resource.2020. 10.6084/m9.figshare.11577726.v2 [DOI] [Google Scholar]
  • 10. Turner P: ACORN Clinician KAP Survey / Form. figshare.Online resource.2020. 10.6084/m9.figshare.11577729.v2 [DOI] [Google Scholar]
  • 11. Turner P: ACORN Diagnostic Stewardship SOP / Form. figshare.Online resource.2020. 10.6084/m9.figshare.11577735.v1 [DOI] [Google Scholar]
  • 12. Turner P: ACORN Community Acquired Infection Enrolment SOP. figshare.Online resource.2020. 10.6084/m9.figshare.11577738.v1 [DOI] [Google Scholar]
  • 13. Turner P: ACORN Hospital Acquired Infection Point Prevalence Survey SOP. figshare.Online resource.2020. 10.6084/m9.figshare.11577741.v1 [DOI] [Google Scholar]
  • 14. Turner P: ACORN Participant Information Sheet. figshare.Online resource.2020. 10.6084/m9.figshare.11676453.v1 [DOI] [Google Scholar]
  • 15. Turner P: ACORN Patient Follow-up SOP. figshare.Online resource.2020. 10.6084/m9.figshare.11577747.v1 [DOI] [Google Scholar]
  • 16. Turner P: ACORN ODK Data Entry SOP / Form. figshare.Online resource.2020. 10.6084/m9.figshare.11577750.v1 [DOI] [Google Scholar]
  • 17. AMR Surveillance in low- and middle-income settings - A roadmap for participation in the Global Antimicrobial Surveillance System (GLASS). London School of Hygiene & Tropical Medicine;2016. Reference Source [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Dat VQ, Geskus RB, Wolbers M, et al. : Continuous versus intermittent endotracheal cuff pressure control for the prevention of ventilator-associated respiratory infections in Vietnam: study protocol for a randomised controlled trial. Trials. 2018;19(1):217. 10.1186/s13063-018-2587-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. European Centre for Disease Prevention and Control: Point prevalence survey of healthcare associated infections and antimicrobial use in European acute care hospitals – protocol version 5.3. Stockholm: ECDC;2016. Reference Source [Google Scholar]
  • 20. Seymour CW, Liu VX, Iwashyna TJ, et al. : Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):762–74. 10.1001/jama.2016.0288 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Plunkett A, Tong J: Sepsis in children. BMJ. 2015;350:h3017. 10.1136/bmj.h3017 [DOI] [PubMed] [Google Scholar]
  • 22. R Core Team: R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing;2018. Reference Source [Google Scholar]
  • 23. Chang W, Cheng J, Allaire JJ, et al. : shiny: Web Application Framework for R. R package version 1.2.0,2018. Reference Source [Google Scholar]
  • 24. Magiorakos AP, Srinivasan A, Carey RB, et al. : Multidrug-resistant, extensively drug-resistant and pandrug-resistant bacteria: an international expert proposal for interim standard definitions for acquired resistance. Clin Microbiol Infect. 2012;18(3):268–81. 10.1111/j.1469-0691.2011.03570.x [DOI] [PubMed] [Google Scholar]
  • 25. Fox-Lewis A, Takata J, Miliya T, et al. : Antimicrobial Resistance in Invasive Bacterial Infections in Hospitalized Children, Cambodia, 2007-2016. Emerg Infect Dis. 2018;24(5):841–51. 10.3201/eid2405.171830 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Ryu S, Cowling BJ, Wu P, et al. : Case-based surveillance of antimicrobial resistance with full susceptibility profiles. JAC-Antimicrobial Resistance. 2019;1(3):dlz070. 10.1093/jacamr/dlz070 [DOI] [PMC free article] [PubMed] [Google Scholar]
Wellcome Open Res. 2020 Jun 2. doi: 10.21956/wellcomeopenres.17566.r38906

Reviewer response for version 2

Rebecca Lester 1

No further comments.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

AMR/Antimicrobial Stewardship in LMICs.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 Jun 2. doi: 10.21956/wellcomeopenres.17566.r38904

Reviewer response for version 2

Maya Nadimpalli 1

No further comments.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Reviewer Expertise:

AMR transmission/One Health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 May 21. doi: 10.21956/wellcomeopenres.17186.r38653

Reviewer response for version 1

Andrew Whitelaw 1

Thanks for the opportunity to review this protocol. The protocol describes an "enhanced surveillance" system focusing on antimicrobial resistance, bringing together patient level clinical and laboratory data. The intention is to generate AMR-data that is more clinically relevant, and less subject to bias. Overall the design is appropriate and is likely to result in better quality AMR data than is currently achieved using lab-based routine data (the default position in many settings). It will also hopefully provide information that can be used to implement similar systems elsewhere

There are a few suggestions/criticisms, but these are relatively minor, and some may not be practical.

Page 4: The Surveillance design focuses on a set number of pathogens. It may help to clarify why these were chosen. What happens if a patient is infected with one of the other pathogens - if I understand correctly, they would still be included (or their clinical data will be included), but the organism data would not be part of the dashboard. I think this should be clarified.

Page 4: Recruitment: the protocol states that standardised clinical case definitions will be used for training, but will not be required for enrollment during the pilot phase. I think this should be clarified. Why are case definitions not required; and is this specific to the pilot study (i.e. will case definitions be required subsequently)? The discussion makes reference to "pragmatic definitions" being used; and that the issue of case definitions will be reviewed at the end of the pilot phase. The challenge of case definitions is well understood. I am not suggesting that standardised case definitions have to or even can be used for the pilot, but I think the question of what criteria will be used for enrollment needs to be clearer. Linked to this, the definitions in SOP 6 are not clearly laid out (at least to my mind) as case definitions. They come across as clinical criteria that can be used as a guide to suggest pneumonia/sepsis/meningitis.

Page 4/5 - HAI surveillance. This will be done as a point prevalence, weekly. The intention is to capture any patient who may have had an HAI during the previous week. What happens if a patient developed symptoms of an HAI in the preceding week, and either demised, was transferred, or recovered before the surveillance team do the weekly survey? Is there a risk of underestimating HAIs? Conversely, if a patient is included due to a clinically suspected HAI, which is subsequently shown not to be an infection, will this patient be removed from the HAI database? I suspect there is no easy way around this, and illustrates the challenge of any clinical surveillance system.

Data management: If I understand, lab data and clinical data will be merged locally to allow for assigning the correct lab data to the correct clinical data. At that stage data will be anonymised and then further analysed. At what time point/s will this de-identification take place? Is it an ongoing process or only at the end of the pilot? Given that patients are being followed up for 28 days, and thus there are multiple sets of clinical data linked to one patient, and potentially multiple sets of lab data linked to one patient, linking the clinical and lab datasets will need to be an ongoing process, with de-identification only taking place once you are sure that no more data will be forthcoming. This is not explicitly stated - maybe it doesn't need to be.

The study will use a MORU-designed LIMS for labs that have no electronic LIMS. Will this LIMS be accessible to clinical staff at the study sites - it may serve as a positive side effect of the study, and improve access to lab results.

The dashboard includes provision of resistance rates for the target pathogens. The example shown has 6 E. coli isolates; and at a facility level the numbers of individual isolates are likely to be low (at least initially). Please be careful of displaying resistance rates for small numbers of organisms as this can be misleading. CLSI recommends at least 30 isolates before showing cumulative resistance/susceptibility rates.

Laboratories will use either CLSI or EUCAST criteria. When merging data from different labs, this will create problems when displaying combined susceptibility data - sets of data generated using two different interpretive criteria should not be merged.

Interpretation of susceptibility may vary depending on site of infection (penicillin for S. pneumoniae). How will this be managed? It would be better to record MIC data and report MICs, rather than categorical interpretation.

Did the authors consider adding an AMS component to the study? One could include data related to whether the prescribed antibiotic for the infection was in line with guidelines; whether de-escalation occurred; whether doses were appropriate etc. (I know this adds to the workload so probably not practical - but worth asking).

It appears that the surveillance will be done by clinical staff already employed at the hospital. Is this sustainable?

SOP-2 - Laboratory Assessment:

On the whole this is quite a comprehensive process. However I think there should be more detail about the AST methods used in the lab. The current form just differentiates manual and automated and asks for free text details. I would suggest tick boxes for disc diffusion, gradient diffusion MIC, broth dilution MIC. It is also complicated by the fact that many labs will use different methods for different organisms. Its not practical to capture all the variations, but identify critical ones - such as penicillin and S. pneumoniae which needs an MIC method.

I would suggest a question about whether the lab is accredited to an international standard (such as ISO 15189).

I would suggest more information about EQA - maybe review the last set of EQA results; or look for what processes are in place to monitor and resolve faults identified by the EQA process.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Partly

Reviewer Expertise:

Clinical microbiology, with a focus on antimicrobial resistance, antimicrobial stewardship, and infection control.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 May 26.
Paul Turner 1

Responses to reviewer comments: Wellcome Open Research 5-13 V1

We thank all four reviewers for their extremely helpful and constructive comments. These are all of considerable value as ACORN is iterated prior to wider roll out. Our responses are included as bullet points below.

Reviewer 4

Thanks for the opportunity to review this protocol. The protocol describes an "enhanced surveillance" system focusing on antimicrobial resistance, bringing together patient level clinical and laboratory data. The intention is to generate AMR-data that is more clinically relevant, and less subject to bias. Overall the design is appropriate and is likely to result in better quality AMR data than is currently achieved using lab-based routine data (the default position in many settings). It will also hopefully provide information that can be used to implement similar systems elsewhere

  • We thank the reviewer for these positive comments.

There are a few suggestions/criticisms, but these are relatively minor, and some may not be practical.

Page 4: The Surveillance design focuses on a set number of pathogens. It may help to clarify why these were chosen. What happens if a patient is infected with one of the other pathogens - if I understand correctly, they would still be included (or their clinical data will be included), but the organism data would not be part of the dashboard. I think this should be clarified.

  • The organisms selected as the blood culture relevant subset of the WHO GLASS organism list. This detail has been added to the "surveillance design" section of the manuscript. These will form the basis of the reporting dashboard / downloadable data summaries. However, it is possible to summarise data regarding infections due to other organisms using the dashboard (but not for now possible to download this as a word document). It is fully anticipated that additional organisms may be added to the core surveillance list as time passes (for example Enterococcus spp and Pseudomonas aeruginosa). This has been clarified by adding "Whilst the surveillance is focussed on a set of key pathogens, it is recognised that other bacterial species will be cultured from surveillance participant specimens. These species will not be included in formal analyses, but the organism and antimicrobial susceptibility data will be summarised on the interactive dashboard."

Page 4: Recruitment: the protocol states that standardised clinical case definitions will be used for training, but will not be required for enrollment during the pilot phase. I think this should be clarified. Why are case definitions not required; and is this specific to the pilot study (i.e. will case definitions be required subsequently)? The discussion makes reference to "pragmatic definitions" being used; and that the issue of case definitions will be reviewed at the end of the pilot phase. The challenge of case definitions is well understood. I am not suggesting that standardised case definitions have to or even can be used for the pilot, but I think the question of what criteria will be used for enrollment needs to be clearer. Linked to this, the definitions in SOP 6 are not clearly laid out (at least to my mind) as case definitions. They come across as clinical criteria that can be used as a guide to suggest pneumonia/sepsis/meningitis.

  • We are grateful for this comment, which highlights the most challenging aspect of this project. We agree that the case definitions are worded and formatted with clinical guidance in mind and this was our preferred route for enrolment during the pilot, i.e. that these "definitions" would be used at site training to reinforce clinical diagnosis of the surveillance syndromes and that enrollable cases would thus be "clinically suspected" (but with a greater degree of standardisation). Identification of cases where clinical diagnosis is not well recorded in the medical notes has required the surveillance clinicians to assess patient fit to the case definitions as written. From site feedback, this has been relatively straightforward for pneumonia and meningitis (where the specific criteria required to make the clinical diagnosis are specified in SOP6) but, predictably, has been more of a challenge for sepsis. The manuscript discussion has been updated to: "For this reason, very pragmatic definitions / clinical criteria have been employed in the pilot phase of ACORN, with capture of simple severity markers to enable patient stratification within a syndrome. Patients may be enrolled based on clinician diagnosis or, in the absence of clear clinician diagnosis, if they are assessed by the surveillance team as meeting the clinical case criteria for meningitis, pneumonia or sepsis. Widely used clinical definitions are used in diagnostic stewardship activities in an attempt to improve standardisation of clinician diagnosis and specimen collection."

  • Given the current stage of the surveillance (i.e. nearing the end of the pilot), we have not updated the study documents in response to this comment but would very much like to affirm that the point is noted and will be dealt with. On-going discussions at planned pilot investigator and stakeholder meetings will resolve this issue prior to wider rollout.

Page 4/5 - HAI surveillance. This will be done as a point prevalence, weekly. The intention is to capture any patient who may have had an HAI during the previous week. What happens if a patient developed symptoms of an HAI in the preceding week, and either demised, was transferred, or recovered before the surveillance team do the weekly survey? Is there a risk of underestimating HAIs? Conversely, if a patient is included due to a clinically suspected HAI, which is subsequently shown not to be an infection, will this patient be removed from the HAI database? I suspect there is no easy way around this, and illustrates the challenge of any clinical surveillance system.

  • We agree that HAI surveillance is extremely challenging. HAI cases may be missed because of the reasons stated (indeed we note that this has happened during the pilot). Under ideal circumstances those that remain hospitalised but have recovered from their HAI within a week ought to be included, but this will require the surveillance team to maintain diligence in case identification. A comment on this has been added to the discussion: "Another area of uncertainty is around comprehensive capture of HAI cases. The use of point prevalence surveys for identification of HAI is time efficient but may result in underestimation of case numbers due to early death, discharge or recovery. However, this issue is common to all point prevalence surveys and not restricted to ACORN".

  • Those patients included in HAI surveillance but subsequently determined not to infected will be identified as syndrome "rejected" at the hospital outcome data collection point. These cases can then be removed from subsequent analyses.

Data management: If I understand, lab data and clinical data will be merged locally to allow for assigning the correct lab data to the correct clinical data. At that stage data will be anonymised and then further analysed. At what time point/s will this de-identification take place? Is it an ongoing process or only at the end of the pilot? Given that patients are being followed up for 28 days, and thus there are multiple sets of clinical data linked to one patient, and potentially multiple sets of lab data linked to one patient, linking the clinical and lab datasets will need to be an ongoing process, with de-identification only taking place once you are sure that no more data will be forthcoming. This is not explicitly stated - maybe it doesn't need to be.

  • Anonymisation occurs each time clinical and laboratory data are merged which enables the follow-up data to be correctly linked together prior to upload for analysis / sharing. The relevant sentence in "data management" has been updated to: " Each time clinical and laboratory data are merged, a unique surveillance ID will be generated for each participant and the participant hospital ID and date of birth will be deleted automatically, rendering a fully de-identified dataset for analysis and further sharing."

The study will use a MORU-designed LIMS for labs that have no electronic LIMS. Will this LIMS be accessible to clinical staff at the study sites - it may serve as a positive side effect of the study, and improve access to lab results.

  • Unfortunately, this is not possible at the current stage of development (although the LIMS does include generation of printed and pdf clinician reports). Further development of the LIMS as part of an associated project will address this important gap.

The dashboard includes provision of resistance rates for the target pathogens. The example shown has 6 E. coli isolates; and at a facility level the numbers of individual isolates are likely to be low (at least initially). Please be careful of displaying resistance rates for small numbers of organisms as this can be misleading. CLSI recommends at least 30 isolates before showing cumulative resistance/susceptibility rates.

  • We agree very much with this comment and will add an appropriate disclaimer to the dashboard. The following wording appears already in reports downloaded from the dashboard: "Care should be taken when interpreting rates and AMR profiles where there are small numbers of cases or bacterial isolates: point estimates may be unreliable."

Laboratories will use either CLSI or EUCAST criteria. When merging data from different labs, this will create problems when displaying combined susceptibility data - sets of data generated using two different interpretive criteria should not be merged.

  • Thanks for this comment – we agree it is a problem. The AST criteria used by the laboratory is stored within the merged clinical-lab data file for each site and we will take care not to combine CLSI and EUCAST data.

Interpretation of susceptibility may vary depending on site of infection (penicillin for S. pneumoniae). How will this be managed? It would be better to record MIC data and report MICs, rather than categorical interpretation.

  • For now the dashboard displays categorical data interpreted using the epidemiologic (i.e. meningitis) breakpoints for penicillin and S. pneumoniae. Moving forwards, our intention was to include a second interpretation of the penicillin data using the non-meningitis breakpoints. However, since the final site datafile retains the raw MICs (and zone sizes) alongside categorical interpretations of these, a summary table of MIC values may be a more appropriate way to display these data.

Did the authors consider adding an AMS component to the study? One could include data related to whether the prescribed antibiotic for the infection was in line with guidelines; whether de-escalation occurred; whether doses were appropriate etc. (I know this adds to the workload so probably not practical - but worth asking).

  • This is very much something we are considering for future versions of the surveillance protocol. As it stands the documentation of initial treatment will at least permit sites to determine whether empiric guidelines were adhered to.

It appears that the surveillance will be done by clinical staff already employed at the hospital. Is this sustainable?

  • Discussions around site-level sustainability are on-going but it will ultimately depend on resources available at the hospital and whether management / clinicians feel that the surveillance yields data of sufficient utility to allocate staff time to it.

SOP-2 - Laboratory Assessment:

On the whole this is quite a comprehensive process. However I think there should be more detail about the AST methods used in the lab. The current form just differentiates manual and automated and asks for free text details. I would suggest tick boxes for disc diffusion, gradient diffusion MIC, broth dilution MIC. It is also complicated by the fact that many labs will use different methods for different organisms. Its not practical to capture all the variations, but identify critical ones - such as penicillin and S. pneumoniae which needs an MIC method.

  • Agreed - we will add this for key bug-drug combinations into the next version of this document.

I would suggest a question about whether the lab is accredited to an international standard (such as ISO 15189).

  • This is included in the online document: section 12, ALAF-2: accreditation.

I would suggest more information about EQA - maybe review the last set of EQA results; or look for what processes are in place to monitor and resolve faults identified by the EQA process.

  • Agreed - we will add this into the next version of this document.

Wellcome Open Res. 2020 May 20. doi: 10.21956/wellcomeopenres.17186.r38141

Reviewer response for version 1

Mercy Jemima Newman 1

Summary of the study 

  1. Shows the importance of diagnostic microbiology for the treatment of systemic infections.

  2. Burden of drug resistant serious infections in specific hospitals.

  3. Causes and resistance of organisms of HAI and community infections.

  4. It will expand on the WHO GLASS and enable classification of infectious syndromes, outcome and associated costs.

  5. Surveillance training of hospital personnel.

  6. Post-discharge information will be collected e.g. health status and death of patients.

  7. Case definitions will be well defined at the end of the study.

Peer Review Report 

  1. Rational for the study - this is clearly described.

  2. Study design - this is appropriate for the research.

  3. Sufficient detail for the research has been provided.

  4. Datasets are adequate.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Reviewer Expertise:

I am a medical microbiologist. Researched on antimicrobial resistance, healthcare associated infections and investigation of disease.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 May 26.
Paul Turner 1

Responses to reviewer comments: Wellcome Open Research 5-13 V1

We thank all four reviewers for their extremely helpful and constructive comments. These are all of considerable value as ACORN is iterated prior to wider roll out. Our responses are included as bullet points below.

Reviewer 3

Summary of the study

  1. Shows the importance of diagnostic microbiology for the treatment of systemic infections.

  2. Burden of drug resistant serious infections in specific hospitals.

  3. Causes and resistance of organisms of HAI and community infections.

  4. It will expand on the WHO GLASS and enable classification of infectious syndromes, outcome and associated costs.

  5. Surveillance training of hospital personnel.

  6. Post-discharge information will be collected e.g. health status and death of patients.

  7. Case definitions will be well defined at the end of the study.

Peer Review Report

  1. Rational for the study - this is clearly described.

  2. Study design - this is appropriate for the research.

  3. Sufficient detail for the research has been provided.

  4. Datasets are adequate

 

  • We thank the reviewer for these positive comments.

Wellcome Open Res. 2020 May 20. doi: 10.21956/wellcomeopenres.17186.r38656

Reviewer response for version 1

Maya Nadimpalli 1

This protocol describes the framework for an AMR surveillance program targeted at LMIC hospitals that will combine clinical data, including follow-up data after discharge, with microbiological characterization of the etiologic agent. Combining these data streams is standard practice in many higher-income counties but not in LMICs, and successful implementation will be enormously helpful for generating accurate estimates of the prevalence of drug-resistant infections, and the mortality rate of drug-resistant versus susceptible infections. An important component of this surveillance program is that it will build trust in and demonstrate the value of routine laboratory diagnostics.

Overall, the protocol and supporting documents were clear and well-written. I only have minor comments:

P 3 – Please provide a brief definition or a citation for the term “diagnostic stewardship”. Clinicians and researchers based in settings where diagnostic microbiology is routinely conducted may not be familiar with the concept.

P 4 – You mention that surveillance sites will include “the full spectrum of patient groups” and that these sites will “choose 2-3 surveillance wards based on their patient populations.” Can you be more specific? Will site + ward selection help you address one of the key questions (typically missed by passive pathogen-focused AMR surveillance) laid out on page 3 – “Which AMR-syndrome combinations are associated with the poorest outcomes in particular patient groups?”

P 4 – For your definition of HAI, in addition to admission criteria, you write: “Clinical diagnosis of suspected pneumonia or sepsis…, or meets formal case definition, on the day of survey”. Are you referring to formal case definitions of pneumonia and sepsis? I was confused because you later discuss in detail on p. 8 how “case definitions are a current area of uncertainty.”

P 4 – Your definition of HAI seemed a bit circular. If I am interpreting correctly, a patient could be classified as having an HAI if they met the admission criteria AND patient is receiving treatment AND HAI diagnosis made between Day 1 of treatment and survey day. In this scenario, it seems like the classification of HAI is contingent on prior HAI diagnosis. Please clarify.

P 5 – for patients with HAIs transferred from other hospitals, how will you note a) severity score at symptom onset, b) whether a blood culture was collected at symptom onset, c) which antibiotics were prescribed at symptom onset? Will you rely on patient recall or will you be able to access prior hospital records?

P 5- You mention in the “Day 28 assessment” section that patients can be enrolled multiple times in a single hospital admission. Please clarify this earlier, maybe in “Screening and Eligibility Assessment”.

P 6 – For calculating HAI prevalence within a ward, you propose using the percentage of ward patients with an HAI as the numerator and the total number of patients admitted to the ward on the survey date as the denominator. It is unclear to me if this is the appropriate denominator, as not all patients admitted to the ward that day would be eligible to experience the outcome (classification as an HAI). Specifically, patients without prior hospitalization and without a device for whom disease onset occurred <3 days ago would not be eligible.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Reviewer Expertise:

AMR transmission/One Health.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 May 26.
Paul Turner 1

Responses to reviewer comments: Wellcome Open Research 5-13 V1

We thank all four reviewers for their extremely helpful and constructive comments. These are all of considerable value as ACORN is iterated prior to wider roll out. Our responses are included as bullet points below.

Reviewer 2

This protocol describes the framework for an AMR surveillance program targeted at LMIC hospitals that will combine clinical data, including follow-up data after discharge, with microbiological characterization of the etiologic agent. Combining these data streams is standard practice in many higher-income counties but not in LMICs, and successful implementation will be enormously helpful for generating accurate estimates of the prevalence of drug-resistant infections, and the mortality rate of drug-resistant versus susceptible infections. An important component of this surveillance program is that it will build trust in and demonstrate the value of routine laboratory diagnostics.

  • Thank you.

Overall, the protocol and supporting documents were clear and well-written. I only have minor comments:

P 3 – Please provide a brief definition or a citation for the term “diagnostic stewardship”. Clinicians and researchers based in settings where diagnostic microbiology is routinely conducted may not be familiar with the concept.

  • Added a definition from, and reference to, the WHO GLASS diagnostic stewardship document.

P 4 – You mention that surveillance sites will include “the full spectrum of patient groups” and that these sites will “choose 2-3 surveillance wards based on their patient populations.” Can you be more specific? Will site + ward selection help you address one of the key questions (typically missed by passive pathogen-focused AMR surveillance) laid out on page 3 – “Which AMR-syndrome combinations are associated with the poorest outcomes in particular patient groups?”

  • For the ward selection, reviewer 1 made a similar comment and the response was…Suggestions for appropriate surveillance wards are made by the central surveillance team. However, sites are encouraged to make their own decisions around ward inclusion based on case mix and practical considerations (size, staffing, patient numbers). The relevant section from ref (7) has been added into the main text: "Each site should identify appropriate wards for ACORN surveillance. It may be desirable to start with a small number of wards / departments (e.g. medical admissions ward and the main intensive care unit) and scale up over time. Consideration should be given to harmonisation with other surveillance activities, where possible".

  • Re. question on page 3. Yes: if one includes patient group X (e.g. adult ICU patients) and systematically collects infection and clinical data on them, then, when enough data are available, it should be possible to identify which pathogens / AMR profiles are associated with worse outcomes in that patient group, correcting for clinical things that might make a difference. At scale, inclusion of multiple sites will permit analyses which also address differences between sites. Pathogen-focussed surveillance tells us only about the bugs. Passive pathogen-focussed surveillance potentially only tells us about some of the bugs (and critically gives us no information about the patients who were not sampled).

P 4 – For your definition of HAI, in addition to admission criteria, you write: “Clinical diagnosis of suspected pneumonia or sepsis…, or meets formal case definition, on the day of survey”. Are you referring to formal case definitions of pneumonia and sepsis? I was confused because you later discuss in detail on p. 8 how “case definitions are a current area of uncertainty.”

  • This is correct. For the pilot either "clinician suspicion" (i.e. whatever is recorded in the clinical notes) or surveillance staff confirmation that the patient meets the case definition / criteria (SOP6) is required for patient enrolment. Criteria for enrolling patients into AMR surveillance is indeed an area of uncertainty, especially in settings where clinician diagnoses are inconsistent, physical examination may be cursory, clinical notes are frequently incomplete, and pathology support lacking. The uncertainty is true for both CAI and HAI, but currently accepted case definitions for HAI are almost all unfit for purpose in low-resource settings.

P 4 – Your definition of HAI seemed a bit circular. If I am interpreting correctly, a patient could be classified as having an HAI if they met the admission criteria AND patient is receiving treatment AND HAI diagnosis made between Day 1 of treatment and survey day. In this scenario, it seems like the classification of HAI is contingent on prior HAI diagnosis. Please clarify.

  • It isn't. The patient can be enrolled as an HAI case if they:
    • (1) meet the timing criteria
    • AND
    • (2) meet the syndromic / treatment criteria either on the day of the survey or at some point since the last survey and the day of the current survey
    • AND
    • (3) were not enrolled during a previous survey for the current HAI episode (for those HAI that have a clinical course >1 week)
  • We have not encountered any user issues with the definition as stated, so have not updated in the revised manuscript.

P 5 – for patients with HAIs transferred from other hospitals, how will you note a) severity score at symptom onset, b) whether a blood culture was collected at symptom onset, c) which antibiotics were prescribed at symptom onset? Will you rely on patient recall or will you be able to access prior hospital records?

  • In many settings none of these things will be able to be obtained, so they will be missed.

P 5- You mention in the “Day 28 assessment” section that patients can be enrolled multiple times in a single hospital admission. Please clarify this earlier, maybe in “Screening and Eligibility Assessment”.

  • Done: "Patients may be enrolled into ACORN more than once, for example for both CAI and HAI on the same admission, multiple HAI on the same admission, or multiple admissions with CAI and / or HAI."

P 6 – For calculating HAI prevalence within a ward, you propose using the percentage of ward patients with an HAI as the numerator and the total number of patients admitted to the ward on the survey date as the denominator. It is unclear to me if this is the appropriate denominator, as not all patients admitted to the ward that day would be eligible to experience the outcome (classification as an HAI). Specifically, patients without prior hospitalization and without a device for whom disease onset occurred <3 days ago would not be eligible.

Wellcome Open Res. 2020 Apr 17. doi: 10.21956/wellcomeopenres.17186.r37955

Reviewer response for version 1

Rebecca Lester 1

This paper describes a pilot study protocol for a multi-site AMR surveillance project, which aims to prospectively link clinical and diagnostic microbiological data and which can be implemented alongside routine clinical care. The study has gained ethical approval and the pilot phase is ongoing. 

The type of prospective AMR surveillance described by this protocol is much needed and, as the authors identify, should help to minimise the biases associated with many AMR studies which are frequently pathogen/drug specific, or which do not link microbiology to clinical data. The site-level data dashboard is a great development which will provide user-friendly and interactive data visualisation.

p4-5: The authors begin to address this in the discussion, but is the definition of HAI/CAI too narrow/specific? Will this approach miss patients who have attended other types of healthcare setting than acute care hospitals - for example outpatient services and health centres? Likewise, will definition of community acquired be able to distinguish community onset/hospital acquired from true community onset infections? The extended data describes possible reclassification of some CA cases based on transfer and admission dates. Although standardisation across sites may be difficult, the authors might consider making a judgement on this classification prospectively at enrolment and taking into account more nuanced clinical history. 

p5: Likewise, It's not completely clear whether a blood culture collected within 24 hours of admission (CAI) automatically put someone in CAI category? 

p5: What will be the purpose of qSOFA and sepsis six in the analysis? It is extremely challenging to come up with a unifying sepsis severity score for all age groups, particularly in LMICs where qSOFA is not well validated, but may be worth highlighting these limitations in the discussion if sepsis severity will play an important role in the burden models. Will any blood tests be taken into account?

General/Methods: A lot of information on methods is contained comprehensively within the SOPs in extended data. Whilst appreciating the word count limitations, could some of these be summarised in the main paper - in particular?

  • Cost data could be more comprehensively described - is it only antibiotics included in the unit costs and are these healthcare direct costs or patient indirect costs as well?

  • Likewise, the qualitative aspect to this study will be hugely informative but the description of this is somewhat hidden in the main text. Could this be separated out and made clearer in the confines of the work count?

  • Discussion p 8: "The effectiveness and cost-effectiveness of the diagnostic stewardship activities will be monitored as the proportion of cases with linked diagnostic specimens." How does this address cost-effectiveness? Again could this be made clearer in a costing paragraph in methods?

Minor comments:

  • Are there any references for paragraph two of the introduction, which describes the various biases associated with existing AMR surveillance data?

  • What level of QC is available to the diagnostic laboratories?

  • How were surveillance wards selected? Any space to explain this in the main text?

  • Why were the specific target pathogens chosen? If possible a justification would be informative.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

AMR/Antimicrobial Stewardship in LMICs.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 May 4.
Paul Turner 1

Many thanks for the review and comments. We will address these and upload a revised manuscript once the second peer review comes in.

To anyone out there who'd like to be a peer reviewer...please don't hesitate to get in touch with the editorial team. No prizes for doing so, but we'd be very grateful.

Wellcome Open Res. 2020 May 26.
Paul Turner 1

Responses to reviewer comments: Wellcome Open Research 5-13 V1

We thank all four reviewers for their extremely helpful and constructive comments. These are all of considerable value as ACORN is iterated prior to wider roll out. Our responses are included as bullet points below.

Reviewer 1

This paper describes a pilot study protocol for a multi-site AMR surveillance project, which aims to prospectively link clinical and diagnostic microbiological data and which can be implemented alongside routine clinical care. The study has gained ethical approval and the pilot phase is ongoing.

The type of prospective AMR surveillance described by this protocol is much needed and, as the authors identify, should help to minimise the biases associated with many AMR studies which are frequently pathogen/drug specific, or which do not link microbiology to clinical data. The site-level data dashboard is a great development which will provide user-friendly and interactive data visualisation.

  • We thank the reviewer for these positive comments.

p4-5: The authors begin to address this in the discussion, but is the definition of HAI/CAI too narrow/specific? Will this approach miss patients who have attended other types of healthcare setting than acute care hospitals - for example outpatient services and health centres? Likewise, will definition of community acquired be able to distinguish community onset/hospital acquired from true community onset infections? The extended data describes possible reclassification of some CA cases based on transfer and admission dates. Although standardisation across sites may be difficult, the authors might consider making a judgement on this classification prospectively at enrolment and taking into account more nuanced clinical history.

  • We appreciate this comment. Our experience is that it is incredibly difficult for clinical surveillance to be both comprehensive and done consistently / well, especially in resource limited settings. We felt that there was a need for pragmatism in this step that would inevitably result in a compromise and the potential to miss some cases. Simplified case definitions (e.g. to all patients with a fever) may increase sensitivity but this would be at the expense of specificity, which may result in overload and compromise of the entire system. That said, the enrolment criteria are the subject of on-going discussion as we approach the end of the pilot phase.

  • For the second point, it was concluded that, on balance, being able to re-classify CAI as healthcare-associated based on objective evidence recent healthcare exposure and/or direct transfer from another facility was more objective / standardised compared with asking clinicians (of differing experience) to make judgement calls on the origin of infection.

p5: Likewise, It's not completely clear whether a blood culture collected within 24 hours of admission (CAI) automatically put someone in CAI category?

  • This is an important point. We request that surveillance staff record these cases as “CAI”. However, as noted above, the intention is that patients will be stratified on whether they are transferred in (or have recent healthcare exposure) or are direct admissions. In this respect ACORN will be more nuanced than the simple CAI / HAI classification of WHO GLASS which is based entirely on timing of culture.

p5: What will be the purpose of qSOFA and sepsis six in the analysis? It is extremely challenging to come up with a unifying sepsis severity score for all age groups, particularly in LMICs where qSOFA is not well validated, but may be worth highlighting these limitations in the discussion if sepsis severity will play an important role in the burden models. Will any blood tests be taken into account?

  • We completely agree with the difficulties around sepsis severity scores, especially for use in children and/or in LMICs where access to supporting laboratory tests is patchy. The purpose of including qSOFA and the sepsis six signs is to give an indication of patient severity, to help contextualise the clinician diagnosis. We felt that including the requirement for blood test results would be unhelpful given the desire for ACORN to be useful to settings with limited resources. This use of the severity score is noted in the discussion and will not form a major component of burden modelling – the results are intended to be more for site-level data reviews (e.g. correlation of qSOFA score against blood culture positivity). As such, we have not updated the existing statement: "For this reason, very pragmatic definitions have been employed in the pilot phase of ACORN, with capture of simple severity markers to enable patient stratification within a syndrome."

General/Methods: A lot of information on methods is contained comprehensively within the SOPs in extended data. Whilst appreciating the word count limitations, could some of these be summarised in the main paper - in particular?

Cost data could be more comprehensively described - is it only antibiotics included in the unit costs and are these healthcare direct costs or patient indirect costs as well?

  • As currently stated, these calculations will include length of stay (i.e. healthcare direct costs) and antibiotic treatment: " The costs of their care will be estimated using data on length of stay and for antibiotic treatment, with hospital- and country- specific unit costs attached, respectively.". This has been further clarified by inclusion of "(i.e. healthcare direct costs)".

Likewise, the qualitative aspect to this study will be hugely informative but the description of this is somewhat hidden in the main text. Could this be separated out and made clearer in the confines of the work count?

  • This component is intentionally relatively small in the pilot phase and the results are largely intended to inform updates to the surveillance. With this in mind, we prefer not to expand the current description.

Discussion p 8: "The effectiveness and cost-effectiveness of the diagnostic stewardship activities will be monitored as the proportion of cases with linked diagnostic specimens." How does this address cost-effectiveness? Again could this be made clearer in a costing paragraph in methods?

  • We agree, this sentence is flawed. The words "cost-effectiveness" have been removed.

Minor comments:

Are there any references for paragraph two of the introduction, which describes the various biases associated with existing AMR surveillance data?

  • Most of the biases are covered in existing reference (2). We have added the following reference, which includes a brief summary of the issues and points to further relevant references: Yam ELY, Hsu LY, Yap EP-H, Yeo TW, Lee V, Schlundt J, Lwin MO, Limmathurotsakul D, Jit M, Dedon P, Turner P, Wilder-Smith A. Antimicrobial Resistance in the Asia Pacific region: a meeting report. Antimicrobial Resistance & Infection Control. 2019;8(1):202.

What level of QC is available to the diagnostic laboratories?

  • The laboratory assessment tool includes a review of quality control / quality assurance procedures in place. ACORN does not explicitly provide laboratory QC/QA but makes recommendations based on the laboratory assessment. The intention is that sites will not be included in ACORN if their laboratories are not able to meet the required quality standards. Initial site rejection could be reviewed on documentation of quality / technical improvements.

How were surveillance wards selected? Any space to explain this in the main text?

  • Suggestions for appropriate surveillance wards are made by the central surveillance team. However sites are encouraged to make their own decisions around ward inclusion based on case mix and practical considerations (size, staffing, patient numbers). The relevant section from ref (7) has been added into the main text: "Each site should identify appropriate wards for ACORN surveillance. It may be desirable to start with a small number of wards / departments (e.g. medical admissions ward and the main intensive care unit) and scale up over time. Consideration should be given to harmonisation with other surveillance activities, where possible".

Why were the specific target pathogens chosen? If possible a justification would be informative.

  • Target pathogens are the invasive infection / blood stream relevant organisms from the WHO GLASS surveillance scheme. This detail has been added to the "surveillance design" section of the manuscript.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    Underlying data

    No data are associated with this article.

    Extended data

    Figshare: ACORN Laboratory Assessment SOP / Form. https://doi.org/10.6084/m9.figshare.11577702.v2 8

    Figshare: ACORN Site Preparation SOP / Form. https://doi.org/10.6084/m9.figshare.11577726.v2 9

    Figshare: ACORN Clinician KAP Survey / Form. https://doi.org/10.6084/m9.figshare.11577729.v2 10

    Figshare: ACORN Diagnostic Stewardship SOP / Form. https://doi.org/10.6084/m9.figshare.11577735.v1 11

    Figshare: ACORN Community Acquired Infection Enrolment SOP. https://doi.org/10.6084/m9.figshare.11577738.v1 12

    Figshare: ACORN Hospital Acquired Infection Point Prevalence Survey SOP. https://doi.org/10.6084/m9.figshare.11577741.v1 13

    Figshare. ACORN Participant Information Sheet. https://doi.org/10.6084/m9.figshare.11676453.v1 14

    Figshare: ACORN Patient Follow-up SOP. https://doi.org/10.6084/m9.figshare.11577747.v1 15

    Figshare: ACORN ODK Data Entry SOP / Form. https://doi.org/10.6084/m9.figshare.11577750.v1 16

    Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).


    Articles from Wellcome Open Research are provided here courtesy of The Wellcome Trust

    RESOURCES