A user-friendly decision support tool to assist one-health risk assessors

Abstract

One-Health risk assessments are integral to developing efficient responses to disease threats, including global pandemics. However, short timeframes, inadequate disease-specific information and an insufficient skill-base make it hard for inexperienced assessors to distinguish between a large portfolio of approaches. The wrong choice can detract from the disease response. Here, we present an interactive decision support tool to help with this choice. A workshop with participants from diverse professional backgrounds provided six themes that should be considered when deciding on the best approach. Questions based on these themes were then developed to populate a decision tree which guides users to their most appropriate approach. One-Health risk assessment tools and literature were used as examples of the different approaches. The tool provides links to these examples and short descriptions of the approaches. Answers are easily changed, facilitating exploration though different approaches. The simple data structure of the tool means it is easy to update with more resources and approaches. It provides a valuable source of guidance and information for less experienced risk assessors.

Highlights

• •One-Health risk assessment is a diverse field with many distinct approaches
• •It is important that the approach used is applicable to the circumstances
• •We present a web-based decision tree to aid decision-making between approaches
• •It lists relevant approaches with descriptions, along with published examples
• •Examples include tools, guidance documents, and academic publications.

1. Introduction

Risk assessments predict the spread and consequence of potential disease threats. They often heavily inform a country's choice of disease response strategy. Where diseases vary in scope, from endemic diseases such as Salmonella to global pandemics such as SARS-CoV-2, so must the approach to risk assessment. One-Health risk assessment focusses on zoonotic disease threats, requiring collaboration between public health, veterinary health, food safety, and environmental health [1]. One-Health risk assessment approaches are varied and numerous, with ranging objectives and skill barriers. Thus, choosing the risk assessment approach that best suits the circumstances brings many challenges.

Firstly, the number of approaches can be overwhelming. All risk assessment approaches along the qualitative-quantitative spectrum are different. Some differences are clear cut. For example, where stochastic risk assessments rely on good data and sufficient resources, qualitative risk assessments can use limited data in shorter timeframes [2,3]. Other differences are more subtle, such as those between semi-quantitative and qualitative disease prioritisation. There is even variation within approaches. For example, the United Kingdom (UK) and France have similar disease prioritisation frameworks with only country-specific differences [4,5]. Therefore, it can be challenging to recognise the nuance between similar approaches without thorough research.

Secondly, risk assessors may require new skills to implement their desired approach. For example, quantitative modelling may be appropriate, but the risk assessor may lack mathematical experience. If an answer is required quickly, a timely output with lower resolution may be preferred. Understanding and prioritising new training is challenging without prior knowledge of different methods.

Thirdly, choosing an inappropriate approach can have unintended consequences, and may result in the implementation of an inappropriate control strategy. A previous study comparing the incursion risk of African swine fever into Finland compared to The Netherlands, found that each of the seven tools they tested yielded different relative risk scores under the same conditions [6]. In one scenario, one of the tools appeared to contradict all others by predicting a higher risk to Finland than the Netherlands. This tool only considered the risk of entry and not the subsequent exposure or consequence steps. Hence, if the risk question to be addressed needs to consider these steps then it would be inappropriate to use this tool. It is important, therefore, that chosen approaches consider all the characteristics of the risk question.

Despite the importance of risk assessments in supporting policy decisions, there is currently very limited support for the decision-making challenges faced by assessors in the field of One-Health risk assessment. Contrastingly, in business, decision support tools are widely used. The Harvard Business Review has even published guidance on how to decide on the correct decision support tool [7].

As part of the One-Health European Joint Program (EJP) project, COHESIVE, we present the first decision support tool to help One-Health risk assessors choose an appropriate approach. This tool is aimed at inexperienced risk assessors, researchers moving into the One-Health field and policy-makers keen to commission new types of risk assessment. The tool covers risk assessment approaches at the interface between Veterinary Health, Public Health and Food Safety.

2. Methods

2.1. Overview

Building a tool to facilitate the decision between One-Health risk assessment approaches required several stages of development. First, we assembled risk assessment resources: publications, tools and frameworks relating to risk assessment across Veterinary Health, Public Health and Food Safety. From these, we synthesized a list of generic approaches and grouped the compiled resources under each approach. We then conducted a workshop to determine the information that is consistently required to assess the user's needs with respect to the approaches available. From this workshop, six themes were elicited, against which each of the risk assessment approaches were classified. These themes facilitated the development of a decision tree. The decision tree was then converted into a functional online tool, listing assembled publications, tools and resources below each approach.

2.2. Resource assembly

Tools, guidance documents, and publications were assembled to establish core approaches to One-Health risk assessment using a range of methods. An initial questionnaire was used to elicit risk assessment tools and guidance documents used or built by member organisations within the COHESIVE consortium. The consortium is made up of practitioners from across veterinary health, public health, and food safety. The questionnaire asked members of the consortium to list the risk assessment tools they had previously used, or built, and to provide details of their strengths and limitations. This was sent to all members of the consortium by email and responses were recorded as written statements. Engagement in the questionnaire was not prescriptive to particular disciplines and relied on voluntary response from COHESIVE members. This questionnaire was built on during a face-to-face meeting with members of the consortium, where tools were suggested by attendees, and their utility described.

Though systematic methods of literature review such as the PRISMA guidelines exist [8], a non-systematic method of literature review based on snowball sampling [9,10] was deemed sufficient for the purposes here. It was used to expand the existing list of tools and guidance documents. The search databases used were Google scholar, PubMed, Scopus and Web of Science. To search for existing risk assessment tools, we used the starting search terms: ‘One-Health’, ‘risk assessment’, ‘tools’, ‘risk ranking’ and ‘guidance’, both individually and in multiple combinations.

Examples of One-Health risk assessments were also sourced, by expanding these search terms to include: ‘qualitative’, ‘quantitative’, ‘semi-quantitative’ and ‘prioritisation’. Any risk assessments within the public health, veterinary health or food safety sectors were included in the final list.

2.3. Theme identification

A workshop exercise was conducted as part of the COHESIVE project. At this workshop, small, multidisciplinary teams were asked to categorise a series of hypothetical risk assessment tools (supplementary information). From this, several themes emerged, which classified each risk assessment approach.

2.4. Technical development

Core approaches to risk assessment were established based upon the compiled risk assessment tools, guidance documents and examples, and the themes from the workshop. Examples were categorised under each approach. These were limited to 10 per approach, prioritising the most relevant examples.

The themes were used to frame questions and create a decision tree to facilitate choice between these approaches. The decision tree was coded using JavaScript Object Notation (JSON). Each question was linked to a list of responses. Responses to each question were linked to unique follow-up questions with their own list of responses. Approaches were provided at the terminal branches of the data structure, with a description of the approach and published examples where they have been applied. JavaScript code was written to read this data-structure and convert it to an interactive sheet.

3. Results

3.1. Resource assembly

Seven examples were taken from a review of risk ranking tools by Smeu and Taylor (2019) [11]. Further literature research yielded 23 publications relating to One-Health risk assessment. All of these, along with expert opinion, formed the basis for the categorisation of 15 risk assessment approaches. These approaches, 24 associated tools and guidance documents, and 22 risk assessment examples, are listed in Table 1.

Table 1.

examples of tools and risk assessments for each approach used in the decision-support tool.

Approach	Tools / guidance documents	Risk assessment examples
Qualitative risk assessment	OIE qualitative risk assessment framework [12]	RVF risk assessment UK [13]
	Codex Alimentarius microbial risk assessment framework [14]	RVF risk assessment EU [15]
		Foot-and-mouth Spain [16]
Deterministic risk assessment	OIE quantitative risk assessment framework [17]	Risks to animals from catering waste [18]
Stochastic risk assessment	OIE quantitative risk assessment framework [17]	E-coli in salad bars [19]
	Optimising surveillance systems [20]	Salmonellosis in Europe [21]
	One-health modelling overview [22]
Bespoke modelling techniques	Prioritisation of wildlife pathogens [23]	African Swine Fever spread [24]
		AMR spread in a hospital setting [25]
Qualitative disease prioritisation	ECDC tool for disease prioritisation [26]	Stakeholder prioritisation in Quebec [27]
	ECDC tool guidance (ECDC) [28]
Semi-quantitative disease prioritisation	Simplified generic prioritisation tool (France) (ANSES) [29]	MINTRISK in action [30]
	SPARE [31]
	SPARE explanation [32]
	D2R2 [33]
	MINTRISK [34]
	G-RAID comparison of tools [6]
	Prioritisation using DALY and H-index [35]	Stakeholder opinion on prioritisation in Quebec [27]
Quantitative disease prioritisation	WHO prioritisation for R&D [36]	Stakeholder opinion on prioritisation in Quebec [27]
Multi-country disease prioritisation		Disease prioritisation in Europe [35]
Regional disease prioritisation	Zoonotic surveillance in One-Health context [37]	Localised One-Health disease prioritisation in India [38]
		Stakeholder opinion in prioritisation in Quebec [27]
Cost benefit analysis		Cost-benefit assessment in UK pig industry [39]
Import risk assessment: stochastic	Europe-level QMRA [40]	Entry framework bat-borne viruses [41]
		Introduction of rabies into Japan [42]
Farm-to-consumption QMRA	OIE qualitative risk assessment framework [12]	Salmonella in pork products [43]
Preliminary outbreak assessment	ECDC rapid risk assessment tool [44]
	Veterinary risk assessments [45]	Bluetongue outbreak assessment Europe [46]
Rapid risk assessment	Horizon scanning in fisheries products [47]
Horizon scanning	HAIRS RA framework [48]	Global-level horizon scanning [49]
		HAIRS in action [50]

3.2. Decision tree construction

The theme identification workshop split participants into four groups and asked each group to consider a list of tools and reasons why they would or wouldn't use each tool. From this, they were asked to produce a generic decision-tree which listed important considerations when deciding a risk assessment approach (decision-trees shown in Supplementary Information 1). Every group felt it was important to consider the time available. Two groups considered the type of hazard that was being assessed. Three considered the hazard's geographical distribution, for example, whether the hazard was endemic in that area or not. One asked about the level of data available whereas one considered the level of expertise available. Two had more specific questions relating to the context of the assessment. Themes were subsequently clarified from these questions. Six themes in total were derived: 1) time available, 2) hazard identification (ID) 3) geographic specificity, 4) data availability, 5) available expertise, and 6) event specificity (see supplementary information). Time available splits timeframes for performing risk assessment into long, medium and short. Hazard ID splits outcomes based on whether hazards have been predetermined, i.e. whether the risk assessment is for scanning surveillance or in response to a specific disease risk. Geographic specificity splits outcomes based on whether the assessment should be made at the local, regional or country level. Availability of expertise splits outcomes based on prior experience of the risk assessor. Data availability splits outcomes by the quality and abundance of data, grouping these in to low, medium and high. Event specificity splits outcomes by further specifics of the risk question, including disease type (for example endemic or exotic) and assessment type (for example an incursion or impact assessment).

Each of the 15 risk assessment approaches have unique profiles across these six criteria. Stochastic risk assessments, for example, are medium and long-term risk assessments with unlimited geographic specificity. They require medium-to-high data availability, and high-level expertise. They are also useful when integrating uncertainty and variability and are broader than bespoke assessments (Fig. 1).

Fig. 1.

The layout of the decision support tool. 1. A question segment 2. The answer panel to that question. 3. The node, or output of the tool. 4. Links to examples of where this approach has been put in to practice. 5. The purpose, scope and caveats section is accessible at any point. 6. A key, showing which coloured examples link to which sector.

3.3. Tool output

A fully staged example is provided in supplementary information 2. The tool is publicly available and accessible from the following link: http://cohesive.onehealthejp.eu/. When a user opens the tool, they view the purpose, scope and caveats of the tool, which they can then access at any point when using it by clicking a button at the top of the screen. This ensures the user is aware of what the tool can and cannot provide for them. All approaches are listed at the start of the tool, allowing for the quick selection and exploration of approaches at the user's convenience. The user can then step through each question until they reach an outcome. Each question is prefaced with the core theme covered by that question. Each outcome shows the name of the approach to be used, provides a description of that approach, and lists published examples from across One-Health (Fig. 1). At any point, the user can also change answers to previous questions.

4. Discussion

The decision support tool described here defines the needs of its users by asking questions relating to six themes, synthesized from a workshop exercise with experts from across One-Health. Approaches to risk assessment and examples of these approaches were derived from literature searches using snowball sampling. Overall, the tool suggests risk assessment approaches that suit the user's needs using a decision tree, providing a brief description of each approach and linking to examples that support suggested approaches.

This decision tree suggests approaches using a simple question and answer format. However, decision-support methodologies vary in complexity thanks to their wide application across different disciplines [[51], [52], [53]]. Some are software based and some paper based [53]. Some, like the ambulance relocation tool by Anderson and Varbund, have simulation and optimisation facets [54]. Decision trees simply suggest actions based on the user's response to a series of questions, used for example, in the safety assessments of new pharmaceuticals [55]. Unlike many decision-tree flowcharts, our tool is software-based, allowing information to be provided on demand, and simplifying the user interface. It has advantages over other software-based tools. For example, it is small, allowing it to be easily shared and hosted. It is also easily updated. Unlike decision support IOS and Android applications, however, it is less adapted to small screens, and future improvements could be made to increase its compatibility in this area. The tool's simple question and answer format also has limitations. The user's needs can only be determined from their answers to the questions provided. Hence, compared to the ambulance relocation tool by Anderson and Varbund, for example, which uses a range of circumstantial data to cater to the user's needs [54], this tool has relatively limited information to draw a suitable approach from. This limits the specificity of output approaches it can provide.

However, it does allow the tool to remain non-prescriptive. Unlike other decision tree flowcharts used, for example, in nursing [56], the tool does not require a particular approach for a particular circumstance. In contrast, it is more suggestive than definitive, encouraging users to extract what is relevant to them without limiting them strictly to one approach or another. Thus, implementation guidance for each approach is limited to short descriptions. While, in many cases, this leaves the user to apply their own interpretation of a given approach, it also allows the tool to cater to a broader spectrum of user needs. With the reactive format, approaches can be easily cycled through to combine several resources or adapt aspects of different approaches. This means that users whose needs sit ‘between’ the question criteria, can still extract value from it. For example, risk pathways that combine steps of high complexity and uncertainty with those of low uncertainty and linearity could favour a mix of stochastic and deterministic approaches. Stochastic approaches sample parameter estimates from probability distributions rather than fixed values, meaning that subtle differences can be returned on each model run. This is particularly useful for parameters which are uncertain or show variability throughout the population, for example, rates of transmission following direct contact with an infected individual. In contrast, deterministic approaches take single values for each relevant parameter and integrate these directly into the risk estimate, therefore favouring circumstances with low uncertainty or complexity. With this tool, examples and descriptions of both stochastic and deterministic approaches can easily be assimilated.

With its simplicity, the tool's decision tree structure could be applied to further decision-making challenges. Selection between One-Health toolboxes like OH SMART, could easily be facilitated by this decision tree format [57]. More broadly, it could aid decision-making between the One-Health activities listed in the Tripartite Zoonoses Guide, such as surveillance, risk assessment, risk management and communication [1]. It could facilitate decision-making at a detailed technical level, for example in selecting different cost-benefit analysis tools and methods. In this case, it could define when to use qualitative, deterministic, or stochastic methods for cost-benefit estimation, or when to apply quality of life metrics such as disability adjusted life years (DALY) and quality adjusted life years (QALY) [[58], [59], [60]]. This framework could even be used to address decision-making challenges outside One-Health, for example, in deciding between approaches to environmental impact and sustainability assessment [61,62].

Stakeholder engagement through the written questionnaire was based on voluntary responses. Therefore, it is likely that respondents were not evenly distributed across the fields of veterinary health, public health, and food safety. Similarly, in the face-to-face meeting, involvement was limited by attendance and may also have seen an unfair distribution across disciplines. Furthermore, as the tool was designed primarily to show proof of concept and to provide a platform of resources to be built on and maintained over time, the review method employed was not systematic. Snowball sampling, in this context, was expected to capture the majority of relevant available literature, but some tools, publications, and guidance documents may have been missed in the search. As such, the complete list of tools, guidance documents, and publications is expected to expand over time. With feedback, the tool will receive periodic updates to improve its functionality and update examples and descriptions. In doing so, equal representation of One Health disciplines could also be improved. Further work could be to create an open submission platform to allow experts to add further examples and descriptions on demand via an online form.

The workshop exercise was essential to determining the core themes underpinning One-Health risk assessment. However, these themes relied on the subjective opinions from a sample of experts. Consequently, the distilled themes were broad. For example, time availability only divides timeframes into low, medium and high. Because time availability does not further sub-divide, approaches such as stochastic risk assessment appear in two decision pathways. Broad themes do, however, provide consistency across approaches and keep sub-divisions across themes evenly weighted. The small sample of experts may also not have captured viewpoints from all sectors under One Health, as teams were not divided with equal participation from all sectors. Hence, user feedback will be crucial to improve the balance of breadth and utility in these themes, to expand or constrict options where needed, or to provide more even representation from all One-Health sectors.

This is the first decision support tool for One-Health risk assessment. It defines a broad range of approaches against six core themes, listing a multitude of cross-sectoral examples. It is non-prescriptive, suggesting the approach best suited to the circumstances without limiting other options. The simple data-structure allows for regular updates and provides great potential utility for many decision-making challenges.

Ethics statement

No ethical approval was required for this work.

Data accessibility statement.

The tool is now publicly available and accessible from the following link: http://cohesive.onehealthejp.eu/. All code is available from: https://github.com/RDewar/One-health-RA-decisionsupport

Authors' contribution statement

The contribution of the authors is broken down according to their roles as defined in the Contributor Role Taxonomy (CRediT) [63].

Funding

This manuscript is part of the European Joint Programme One Health EJP. This project has received funding from the European Union's Horizon 2020 research and innovation programme under Grant Agreement No 773830 and additional funding from the Department for Environment, Food and Rural Affairs (DEFRA) UK.

Declaration of Competing Interest

None

Appendix A. Supplementary data

Supplementary material