Table 7.

Synthesis of the underlying reasoning for the scoring according to the eight themes describing the scope of six AMR surveillance evaluation tools.

Themes	ISSEP (CA and UK)	ECoSur (DK and VN)	ATLASS (DK)	PMP (BE, DK, IT, and NO)	NEOH (DK, BE, IT, and NO)	SURVTOOLS (DK and NL)
AMU and AMR	Framework developed specifically for AMU and AMR	Not specific for AMU and AMR but can be easily applied for AMU and AMR	Designed for AMU and AMR and residues	Designed for AMU and AMR. Misses components besides farm animals	Not designed for this purpose but can be adapted (e.g., under “objectives of the initiatives”). Most, if not all, of these questions are expected to be included as part of elements 2 and 3	Not developed for AMU and AMR
Collaboration	Allows evaluation of collaboration between the different organizations involved	Collaboration at the heart of the tool, e.g., cross sectors/professions/disciplines/ public/private organizations/geographical/ governance/implementation	Between sectors, all actors, and all levels	Reporting, not data exchange. Participation stakeholders/actors considered for institutions. Gender not considered. Promotes knowledge sharing	Collaboration included in all aspects (in element 1).	No particular guidance; difficult to understand how to evaluate the amount of collaboration
Resources	Questions not included, but data can be collected if economic analysis is part of evaluation	Financial aspects addressed in detail at different levels: planning, allocation, and use	Ask for unlimited or limited budget	Only present in “governance”	Only covered in “planning” and “sharing” aspects of OH-ness evaluation. Focus on allocation: resources to achieve objectives of the initiative (human/physical/financial resources and training). In NEOH handbook, chapter about economic evaluation of OH	Generates a framework for economical evaluation. Epi-calculator available
Output and use of information	Allows to evaluate the outputs of integration and the impacts of integration on decision making and on health and economic outcomes	Allows conclusion about appropriateness of collaborative activities for the expected collaborative outputs (e.g., improving the epidemiological performance). No quantification of impacts on the surveillance value and of costs.	Intermediate-level outputs best addressed	Outputs evaluated (better than impacts), e.g., production of guidelines on prudent use of AM, data reporting to organizations. Not covered in “awareness”	Reveals gaps in OH and where impact of the initiative being evaluated might be improved. Outcomes/impacts depend on type of OH initiative and boundaries of the contextual “system” and resulting ToC. Hence, the evaluator must take into account the appropriate parameters (data and disciplinary paradigms)	If full evaluation, most of the aspects would be covered and impact/output might be possible to measure. Unclear how to measure for intermediate outputs/impacts
Integration	Allows evaluating impacts of integration on decision making/health /economic outcomes	Assessment of the organization and functions of collaboration to achieve the desired level of integration, in coherence with the context	Addressed for many areas, not in-depth	Questions on data reporting, adherence to international testing/data standards/level of knowledge/shared decision making. Not across sectors	Integration measures on many levels, e.g., data integration in organizations, national, regional, or international level, and systems interoperation between different sectors. International testing/data standards not included, unless it is included in “initiative” being evaluated	Not included or advanced to evaluate
Governance	Partly considered when looking at the overall organization/management	Inclusion of many aspects: rationale and objective of collaboration, responsibilities of stakeholders, functionality of governance mechanisms, etc.	Addressed for many areas	Well covered, one main focus of the tool	Partially in the thinking and systemic organization of the OH-ness evaluation. The tool includes consideration of legislation and National Action Plan, if nation is identified as dimensions in the “system”	Not included, but some aspects might be covered if conducting process evaluation
Adaptivity	The tool does not cover this aspect	No monitoring of the progress of collaboration. Monitoring and evaluation of collaboration performance	Measure progress	Designed for measuring improvement	Can be assessed through repeated evaluations. If a dedicated process evaluation is done, the progress can be studied. Evaluator and framework design are “key”	Obtainable if evaluation is done twice (over time) to identify improvement
Technical operations	Includes questions on technical aspects, e.g., sampling/methodology	No evaluation of surveillance performance, even if taken into account evaluation of certain collaboration attributes	Quality of epidemiological designs not covered	Includes questions on the targets of surveillance (e.g., pathogens). Low without ATLASS	Not among evaluation objectives. Include few questions probing for capacities/data handling. Could be part of operations assessment of OH-ness, but extent lies upon evaluator and framework followed	Cover technical efficiency/performance, other laboratory aspects not guided/covered