SUMMARY
Nipah virus (NiV) is a recently emerged zoonotic virus that causes severe disease in humans. The reservoir hosts for NiV, bats of the genus Pteropus (known as flying-foxes) are found across the Asia-Pacific including Australia. While NiV has not been detected in Australia, evidence for NiV infection has been found in flying-foxes in some of Australia's closest neighbours. A qualitative risk assessment was undertaken to assess the risk of NiV establishing in Australian flying-foxes through flying-fox movements from nearby regions. Events surrounding the emergence of new diseases are typically uncertain and in this study an expert opinion workshop was used to address gaps in knowledge. Given the difficulties in combining expert opinion, five different combination methods were analysed to assess their influence on the risk outcome. Under the baseline scenario where the median was used to combine opinions, the risk was estimated to be very low. However, this risk increased when the mean and linear opinion pooling combination methods were used. This assessment highlights the effects that different methods for combining expert opinion have on final risk estimates and the caution needed when interpreting these outcomes given the high degree of uncertainty in expert opinion. This work has provided a flexible model framework for assessing the risk of NiV establishment in Australian flying-foxes through bat movements which can be updated when new data become available.
Key words: Expert opinion, flying-foxes, henipavirus, Nipah virus, risk assessment, zoonosis
INTRODUCTION
Nipah virus (NiV) (genus Henipavirus) is a zoonotic virus that first emerged in Malaysia in 1998. It caused a large outbreak of respiratory disease in pigs, and severe encephalitis in humans with a high mortality rate (∼40%) [1]. Seasonal outbreaks of NiV have also been reported in Bangladesh with ∼75% mortality rates and some human-to-human transmission, highlighting the threat NiV poses to public health [2–4].
Worldwide there are about 65 recognized species of bats in the genus Pteropus. Flying-foxes (genus Pteropus) are considered the reservoir hosts for NiV [5, 6], with spillover into pigs and humans thought to occur through close contact with infected body fluids [1, 2, 7]. Widespread evidence exists for infection of NiV or related henipaviruses in flying-foxes which span tropical and subtropical regions of the Western Pacific to the east coast of Africa [8]. Although NiV disease has not been reported in Australia, evidence of NiV infection has been found in flying-foxes in some of Australia's closest neighbours including Sumatra, Java and Timor-Leste [9, 10]. Additionally, of the four flying-fox species found on the Australian mainland, two (Pteropus alecto and P. conspicillatus) are also found in Papua New Guinea (PNG) and Indonesia [11]. P. alecto has expanded its range southwards along the east coast of Australia, from the Mary River in the 1930s [12] to Sydney in 2007 [13], a distance of more than 950 km. These facts, and the ability of individual flying-foxes to fly long distances, provide an opportunity for pathogens to enter Australia through flying-fox movements [14].
In this study we used the Office International des Epizooties (OIE) risk assessment framework [15] to assess the risk of NiV establishing in Australian flying-fox populations through flying-fox movements from neighbouring regions of the eastern archipelago of Indonesia (Lesser Sunda and Molucca Islands), Timor-Leste and PNG, referred to as ‘pre-border’ regions in this study. The qualitative approach was chosen primarily because quantitative data is sparse, and a qualitative approach provides a transparent and systematic means for identifying the basic model structure, key input parameters, and areas of data scarcity [16, 17].
Events surrounding the emergence of new diseases such as NiV are typically highly uncertain. In this study there are a number of uncertainties identified in the risk pathway. Formal expert elicitation provides a structured and transparent method to address these uncertainties and data gaps [18]. Generally, a group of experts tend to provide better estimates than the average individual expert [19, 20]. The modified Delphi method is an elicitation technique designed to capture the judgement of multiple experts without the biases and heuristics that can result from group discussions [19, 21–24]. However, as consensus is not necessarily achieved with this technique, further mathematical combination is needed after the interactive process.
Conflicting opinion exists in the literature on which method performs best when combining expert judgements. A number of methods have been described, and Knol et al. [18], Clemen & Winkler [25], and O'Hagan et al. [24] provide good reviews. Simple averaging techniques for combining expert opinion perform well in comparison to more complex techniques [24, 25]. The objectives of this study were to qualitatively assess the risk of NiV establishing in Australian flying-fox populations through the movements of pre-border flying-foxes using literature and expert opinion where data gaps exist; and to compare five methods of combining expert opinion and use these as inputs in the model to assess changes in overall risk.
METHODS
Risk assessment model
The OIE framework for import risk analyses was used for this assessment [15]. It is divided into ‘release’, ‘exposure’, and ‘consequence’ assessments, where ‘release’ refers to the probability of entry of NiV into Australia via flying-foxes, ‘exposure’ considers the probability Australian flying-foxes are exposed to NiV, and ‘consequence’ considers the probability NiV establishes itself in Australian flying-fox populations. These are subdivided into events in a pathway, where each event is conditional on the previous event occurring (Fig. 1).
Fig. 1.
Pathway describing the events (P1–P5) necessary for Nipah virus establishment in Australian flying-foxes through flying-fox movements from pre-border regions (eastern archipelago of Indonesia, Papua New Guinea and Timor-Leste).
The probability of an event occurring was defined in qualitative terms on a linear six-level scale, ranging from negligible to high probability (Table 1). This was derived from the scale used by the OIE and that used by experts in the expert opinion workshop. Since events described in this model are uncommon, more precision was required at the lower end of the scale, so six levels were used.
Table 1.
Qualitative categories used to describe the probability of occurrence of an event in the model assessing the risk of Nipah virus establishment in Australian flying-foxes
| Probabilities used in the model | Probabilities used in the expert opinion workshop | ||
|---|---|---|---|
| Qualitative probability estimate | Description | Probability score | Description |
| Negligible | Probability of event occurring is so rare that it does not merit consideration | 1 | In your opinion the event is biologically implausible |
| Extremely low | Probability of event occurring is very rare but cannot be excluded | 2 | In your opinion the event is plausible but extremely unlikely (e.g. could occur once in 500 years) |
| Very low | Probability of event occurring is rare but does occur | 3 | In your opinion the event is very unlikely but not biologically surprising, e.g. could occur once in a human lifetime (50–100 years) |
| Low | Probability of event occurring is occasional | 4 | In your opinion the event could occur very occasionally (e.g. likely to occur once every 10 years) |
| Medium | Probability of event occurring is regular | 5 | In your opinion the event is likely (e.g. could occur as often as annually) given the suggested scenario |
| High | Probability of event occurring is very often | ||
Due to the conditional nature of each event occurring in the pathway, probabilities for each event must be sequentially multiplied together for the final probability estimation. However, in steps 2 (2a and 2b) and 3 (3a and 3b), events are independent of each other, so must be ‘added’ together. A matrix was used to determine the result of multiplying two qualitative probabilities together following the methods of a previous assessment which uses the fact that probabilities lie between 0 and 1, so the result of multiplying two probabilities together cannot be higher than the lower probability [26, 27] (Table 2).
Table 2.
Matrix used for the multiplication of two qualitative probabilities
| Results of probability 2 | Results of probability 1 | |||||
|---|---|---|---|---|---|---|
| Negligible | Extremely low | Very low | Low | Medium | High | |
| Negligible | Negligible | Negligible | Negligible | Negligible | Negligible | Negligible |
| Extremely low | Negligible | Extremely low | Extremely low | Extremely low | Extremely low | Extremely low |
| Very low | Negligible | Extremely low | Very low | Very low | Very low | Very low |
| Low | Negligible | Extremely low | Very low | Low | Low | Low |
| Medium | Negligible | Extremely low | Very low | Low | Medium | Medium |
| High | Negligible | Extremely low | Very low | Low | Medium | High |
Data sources
Input data were derived from the scientific literature and expert opinion. Expert opinion was elicited through a two-stage modified Delphi technique comprising a workshop and a questionnaire completed in pre-workshop (stage 1) and post-workshop (stage 2). This technique was used to preserve independence and anonymity of the experts through the questionnaire, and exploit the benefits of group interactions with the workshop [19, 22, 23, 28]. The workshop was run during a 2-day Henipavirus Research Adoption Forum in Australia on 16–17 July 2007 with participants including leading scientists in the area from Australia, Bangladesh, Malaysia and the USA. The 2-h workshop was led by a professional facilitator and started with a short background to the topic followed by discussion on each of the steps in the risk pathway. Only post-workshop results were used in the analysis.
A multi-disciplinary group of nine experts were selected from the research adoption forum to provide expertise in flying-fox ecology, virology, disease ecology, epidemiology, and risk assessment. All experts that were asked to participate in the workshop completed the exercise. Experts gave probability and uncertainty scores for ten questions on NiV introduction into Australia through flying-fox movements (Tables 3 and 4). Only responses from questions directly relevant to the model were used in this study.
Table 3.
Questionnaire used in the expert opinion elicitation workshop to assess the risk of Nipah virus (NiV) establishment in flying-foxes in Australia
| Scenario 1 |
| Flying-fox populations on several of the Lesser Sunda and Moluccan Islands are endemically infected with NiV. These islands are 320–450 km from the Australian mainland. Several times a year small groups of Nipah infected bats are blown out to sea, south of their island of residence. |
| Question 1: What is the likelihood that one or more of these bats will fly to mainland Australia? |
| Scenario 2 |
| A small group of flying-foxes from a population where NiV infection is endemic are blown out to sea and end up landing in Australia. |
| Question 2: What is the likelihood that one or more of these flying-foxes will actively excrete NiV in Australia? |
| Scenario 3 |
| A small group of flying-foxes, actively excreting NiV, are blown across the Arafura Sea and end up landing in mangroves on mainland Australia. |
| Question 3: What is the likelihood that these bats will survive and start co-roosting with Australian flying-foxes? |
| Scenario 4 |
| Several flying-foxes, actively excreting NiV, are blown across the Arafura Sea and join a flying-fox roost in mangroves on mainland Australia. Hendra virus is endemic in the Australian flying-fox population. |
| Question 4: What is the likelihood that some of the Australian flying-foxes will become infected with NiV? |
| Scenario 5 |
| A small number of Australian flying-foxes become infected with NiV following contact with flying-foxes from outside Australia. |
| Question 5: What is the likelihood that some of the Australian flying-foxes will excrete NiV resulting in infection of other Australian flying-foxes? |
| Scenario 6 |
| Australian flying-foxes from one roost site become infected with NiV and infection spreads between individuals at the site. |
| Question 6: What is the likelihood that NiV will become established in the Australian flying-fox population? |
| Scenario 7 |
| Two species of flying-fox, Pteropus species x. and P. alecto, both occur on one of the Lesser Sunda Islands of the Indomalayan region. Anthropogenic environmental change leads to leads to an increase in contact between the species. They now feed on similar fruit and flowers throughout the year and share roosts for several months of the year. NiV infection is endemic in Pteropus x. on this island and P. alecto was previously naive to NiV. |
| Question 7: What is the likelihood that NiV transmission from Pteropus x. to P. alecto will occur resulting in NiV infection in P. alecto? |
| Scenario 8 |
| Regular NiV transmission from Pteropus species x. to P. alecto occurs in an area of overlapping distribution between the two species. |
| Question 8: What is the likelihood that active NiV infection of P. alecto will occur resulting in shedding of NiV by P. alecto? |
| Scenario 9 |
| Regular transmission of NiV from Pteropus sp. to P. alecto occurs resulting in excretion of NiV by P. alecto. |
| Question 9: What is the likelihood that NiV will become established in the P. alecto population at this location? |
| Scenario 10 |
| NiV becomes established in a P. alecto population outside Australia. This population has regular migratory contact (allowing a high level of gene flow) with P. alecto on mainland Australia. |
| Question 10: What is the likelihood that NiV will become established in Australia through the P. alecto population? |
Table 4.
Experts' probability and uncertainty scores for the ten questions presented in the expert opinion elicitation workshop
| Expert | Individual probability/uncertainty scores for the ten questions | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
| 1 | 3/3 | 2/4 | 4/3 | 4/4 | 4/4 | 3/3 | 5/4 | 5/4 | 5/4 | 4/4 |
| 2 | 3/3 | 2/3 | 4/4 | 5/4 | 4/4 | 5/5 | 4/3 | 5/4 | 5/5 | 5/5 |
| 3 | 4/4 | 3/3 | 3/4 | 3/3 | 4/4 | 3/3 | 4/4 | 4/4 | 3/3 | 3/3 |
| 4 | 4/4 | 3/3 | 5/4 | 4/4 | 4/4 | 3/3 | 4/4 | 4/4 | 2/3 | 2/3 |
| 5 | 3/3 | 3/3 | 4/4 | 4/4 | 4/4 | 3/4 | 5/5 | 5/4 | 4/3 | 4/4 |
| 6 | 4/3 | 2/4 | 5/3 | 5/4 | 5/4 | 5/3 | 5/5 | 5/5 | 4/2 | 5/4 |
| 7 | 3/3 | 2/4 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| 8 | 5/5 | 3/4 | 5/4 | 4/4 | 5/4 | 4/3 | 5/4 | 5/4 | 5/4 | 3/3 |
| 9 | 4/4 | 2/3 | 5/4 | 4/4 | 4/4 | 3/3 | 4/4 | 4/4 | 4/3 | 3/3 |
Expert opinion combination
No single ideal technique is described for combining experts' judgements. In this study we used five different averaging methods for combining experts' probability scores: median, mean, uncertainty-weighted median, expertise-weighted median and linear opinion pooling (LOP). The median was used as the reference combination method as it is simple and robust [21, 26, 29]. The first two approaches are simple central tendency measures while the last three methods provide weighting of the responses. Weighting was used to assess the effects of ‘better’ expert opinion on risk outcomes. LOP [25] is a weighted linear average of each expert's probability distribution. Probabilities were provided by experts as scores rather than distributions. Higher weights were given to expert responses when experts were more certain of their answer. For example, in question 2 of the questionnaire the sum of the experts' uncertainty scores was 31. Expert 1 had an uncertainty score of ‘4’ and a probability score of ‘2’. Thus the weighting for expert 1 would be 4/31 and their probability estimate would be 4/31 × 2. Figures were rounded to the nearest whole number.
Similarly, for the uncertainty-weighted median and expertise-weighted median methods, experts' responses were weighted higher if they were more certain of their estimates or were regarded as having more expertise, respectively. Expertise was weighted according to their level of background knowledge and involvement in NiV and flying-fox research. Three categories were used, with the lowest weight allocated to someone who had expertise in a relevant discipline but no direct involvement with NiV or flying-fox research; the middle category was for experts who had worked on NiV or flying-foxes and contributed fewer than 20 peer-reviewed papers in the area; and the top category was for experts who had contributed over 20 peer-reviewed papers on NiV or flying-foxes and worked extensively in the area. The allocation of weights to each expert follows the same technique as described by Gale et al. [27].
The final probability scores were integrated into the model where expert opinion was required (P2a, P3a, P4, P5).
Uncertainty
Uncertainty in the input parameters was categorized with the scale used in Table 5. When combining uncertainty of the risk estimates the highest uncertainty score was used, based on a precautionary approach [15].
Table 5.
Qualitative categories used to describe uncertainty in this risk assessment, adapted from the European Food Safety Authority (2006) [64]
| Description | Uncertainty category |
|---|---|
| Scarce or no data available; evidence is not provided in references but rather in unpublished reports, based on observations, or personal communications; authors report conclusions that vary considerably between them | High |
| Some but no complete data available; evidence provided in small number of references; authors report conclusions that vary from one another | Medium |
| Solid and complete data available; strong evidence provided in multiple references; authors report similar conclusions | Low |
Sensitivity analysis
Two approaches were used to assess uncertainty in the model. To assess the effect of changing the combination method on the model, outputs of five methods of combining expert opinion were compared in the model. The baseline model used the median combination method, and was compared with the other four combination methods to assess changes in overall risk.
To assess the effects of changes to highly uncertain events on the model outputs, baseline probability estimates for the highly uncertain events (P2a, P3a, P4, P5) were changed consecutively, so that the baseline probability estimates for a single event were increased and decreased by one qualitative category while keeping other parameters constant.
RESULTS
Risk assessment
Probability a flying-fox is infected in a pre-border region (P1)
The prevalence of NiV in bats and flying-fox ecology will influence the probability a flying-fox is infected in pre-border areas. Evidence for NiV has been found in flying-foxes in Malaysia, Thailand, Cambodia and Indonesia (Java and Sumatra) [6, 10, 30, 31]. In addition, evidence for NiV in flying-foxes has been found in the pre-border regions considered in this study: PNG, Indonesia (Sumba) and Timor-Leste [9]. Snary et al. [32] provide a list of seroprevalence estimates of henipaviruses in flying-foxes from nearby countries to Australia. However, the extent of NiV infection in flying-foxes in pre-border regions is largely unknown. Flying-foxes can fly considerable distances and have been observed, via telemetry studies, to travel between Malaysia and Sumatra [33] and PNG and Queensland [14]. There also appears to be a reasonable level of connectivity between bats in pre-border regions since genetic studies of P. vampyrus show high levels of gene flow between populations in Indonesia, Malaysia and Timor-Leste [34]. Given these long-distance movements and connectivity, it is possible that NiV could spread between flying-fox populations in pre-border regions. Indeed, in Australia Hendra virus (HeV) is found in all flying-fox species which have overlapping ranges along the east coast [11]. Consequently P1 was estimated to be ‘medium’.
Probability a flying-fox enters Australia (P2)
Flying-fox entry into Australia is considered from two routes.
Probability a flying-fox arrives in Australia via non-migratory routes (P2a)
The ‘unintentional’ entry of flying-foxes moving beyond the recognized species distribution can occur through episodic climatic events such as storms and strong winds. Severe storms are not uncommon throughout the range of flying-foxes, and periodically affect island fauna in the Asia Pacific [35]. For example, P. scapulatus was found in New Zealand after storm activity, 1600 km from its residence in Australia [36]. From these observations and expert opinion (question 1), P2a was estimated to be ‘low’.
Probability a flying-fox migrates to Australia (P2b)
The only potential migratory route considered for flying-foxes into Australia is across the Torres Strait from PNG to Cape York, a distance of 150 km. The distances between Australia and the Lesser Sunda Islands (Timor-Leste and Indonesia) are several hundred kilometres further and there is currently no evidence of bats regularly travelling between these islands and Australia.
Flying-foxes can migrate long distances, and satellite telemetry studies of P. alecto have shown movement occurs across the Torres Strait [14]. Therefore P2b was estimated to be ‘high’.
Probability a flying-fox contacts Australian flying-foxes (P3)
Flying-foxes must survive and contact local flying-foxes, so ecology, entry route into Australia, and food sources will influence the probability a flying-fox will contact resident Australian flying-foxes. Flying-foxes are gregarious by nature and are known to share camps with other flying-fox species where their distributions overlap [11]. Flying-foxes also time large-scale movements with the seasonal availability of food [37, 38], so when food is abundant, increased contact rates are expected between individuals.
Probability a ‘non-migratory’ flying-fox contacts Australian flying-foxes (P3a)
In northern Australia, a relatively undisturbed ecosystem provides a reasonable food supply for flying-foxes, so large and relatively stable camps of P. alecto and P. scapulatus exist [39, 40]. Since the likelihood of survival and contact is difficult to determine expert opinion (question 3) was needed, and P3a was estimated to be ‘low’.
Probability a ‘migratory’ flying-fox contacts Australian flying-foxes (P3b)
Given the short distance and interceding islands en route to Australia, survival is likely. It is assumed migratory flying-foxes are somewhat familiar with local habitats and bat populations. Indeed, P. alecto have been observed roosting with P. conspicillatus in north Queensland and with P. neohibernicus in PNG [14]. Therefore, P3b was estimated to be ‘medium’.
Probability flying-fox transmits NiV to Australian flying-foxes (P4)
Transmission requires excretion of virus from the infected flying-fox to a susceptible host. Although transmission of NiV among bats is poorly understood, transmission in other species is believed to be through close contact with infected body fluids or tissues [41]. NiV has been isolated from urine, uterine and kidney tissues in bats [42]. Australian flying-foxes are susceptible to NiV as experimental infection shows episodic low-level viral excretion [42]. This may be sufficient to maintain NiV infection through aerosol transmission of urine particles in high-density roosts, or directly through contact with urine used for grooming [40]. The high seroprevalence of HeV in Australian flying-foxes suggests highly efficient transmission, or that infection is maintained for long periods of time [43]. Different species of flying-fox share roosts together and HeV isolates from different Australian flying-fox species show almost identical nucleotide sequences [44, 45]. Whether the same transmission characteristics can be applied to NiV is unknown. Some studies suggest similarities exist, since there may be a higher risk of henipavirus transmission from flying-foxes to domestic animals or humans during the gestation period of flying-foxes [30, 43, 46]. It is conceivable that prior infection and immunity to HeV may limit or prevent infection with NiV given the shared cross-reactive antigenic domains in both viruses [47, 48]. Given these uncertainties expert opinion was used (question 7) and P4 was estimated to be ‘low’.
Probability infection establishes in Australian flying-fox populations (P5)
Infection could spread easily and widely among Australian flying-foxes, given their overlapping distributions, close genetic relationship, and co-roosting behaviour [49]. However, endemic HeV infection in Australian flying-foxes places uncertainty on NiV establishment, so expert opinion was used (question 6) and P5 was estimated to be ‘very low’.
Final risk estimate
The probability of NiV establishing in Australian flying-foxes through non-migratory or migratory routes were both estimated to be ‘very low’. The overall risk of establishment via either route was estimated to be ‘very low’ with high uncertainty (Table 6).
Table 6.
Results for the assessment of Nipah virus establishing in Australian flying-foxes through flying-foxes from the eastern archipelago of Indonesia, Timor-Leste and Papua New Guinea (pre-border regions)
| Framework | Event in pathway | Qualitative probability estimate | Level of uncertainty | Key reference | |
|---|---|---|---|---|---|
| Release assessment | 1. Probability pre-border flying-fox infected (P1) | Medium | Medium | Medium | Breed et al. (2013) [9] |
| 2a. Probability non-migratory flying-fox arrives in Australia (P2a) | Low | High | Expert opinion (Q. 1, probability score 4) | ||
| 2b. Probability flying-fox migrates to Australia (P2b) | High | Low | Breed et al. (2010) [14] | ||
| Exposure assessment | 3a. Probability non-migratory flying-fox contacts Australian flying-foxes (P3a) | Low | High | Expert opinion (Q. 3, probability score 4) | |
| 3b. Probability migratory flying-fox contacts Australian flying-foxes (P3b) | Medium | Medium | Breed et al. (2010) [14] | ||
| 4. Probability flying-fox transmits infection to Australian flying-foxes (P4) | Low | Low | High | Expert opinion (Q. 7, probability score 4) | |
| Consequence assessment | 5. Probability NiV infection becomes established in Australian flying-foxes (P5) | Very low | Very low | High | Expert opinion (Q. 6, probability score 3) |
| Probability via non-migratory route (Pn)* | Very low | ||||
| Probability via migratory route (Pm)* | Very low | ||||
| Final risk (P) | Very low | High | |||
There are two final risk estimates that describe the probability of flying-foxes establishing in Australia via migratory or non-migratory routes of entry.
Combining expert opinion
Results of comparing the methods for combining expert opinion showed minor differences (Fig. 2). When incorporated into the model and compared to the baseline final risk, both weighted-median methods produced the same final risk of ‘very low’, and the mean and LOP methods increased the risk to ‘low’ (Table 7).
Fig. 2.
[colour online]. Comparison of the five methods (mean, median, linear opinion pooling, uncertainty-weighted median, expertise-weighted median) for combining experts' probability scores.
Table 7.
Comparison of final risk estimates for the five methods of combining expert opinion when used in the model to assess the risk of Nipah virus establishment in Australian flying-foxes. Events highlighted in bold are based on expert opinion
| Event in pathway* | Qualitative probability estimates for five methods of combining expert opinion | ||||
|---|---|---|---|---|---|
| Median (baseline model) | Mean | Unertainty-weighted median | LOP | Expertise-weighted median | |
| P1 | Medium | Medium | Medium | Medium | Medium |
| P2a | Low | Low | Low | Low | Low |
| P2b | High | High | High | High | High |
| P3a | Low | Low | Low | Low | Medium |
| P3b | Medium | Medium | Medium | Medium | Medium |
| P4 | Low | Low | Medium | Low | Low |
| P5 | Very low | Low | Very low | Low | Very low |
| Probability establishment via non-migratory route (Pn) | Very low | Low | Very low | Low | Very low |
| Probability establishment via migratory route (Pm) | Very low | Low | Very low | Low | Very low |
| Final risk (P)† | Very low | Low | Very Low | Low | Very low |
LOP, Linear opinion pooling.
For a description of events refer to Figure 1.
P = Pn + Pm = (P1*P2a*P3a*P4*P5)+(P1*P2b*P3b*P4*P5).
Sensitivity analysis
Model output was insensitive to the changes made to the highly uncertain events except for P5 (probability of NiV establishing in an Australian flying-fox population), where the final risk increased to ‘low’ when a higher category was used and decreased to ‘extremely low’ when a lower category was used (Table 8).
Table 8.
Sensitivity analysis of the highly uncertain events in the model assessing the risk of Nipah virus establishment in flying-foxes. Changes to the final risk outcome were assessed when baseline estimates for the highly uncertain events were increased and then decreased by one qualitative category while keeping other parameters constant. The change to the event is highlighted in bold italic
| Event in pathway* | Qualitative probability estimates for event | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| P2a | P3a | P4 | P5 | ||||||
| Baseline model | Higher | Lower | Higher | Lower | Higher | Lower | Higher | Lower | |
| P1 | M | M | M | M | M | M | M | M | M |
| P2a | L | VL | M | L | L | L | L | L | L |
| P2b | H | H | H | H | H | H | H | H | H |
| P3a | L | L | L | VL | M | L | L | L | L |
| P3b | M | M | M | M | M | M | M | M | M |
| P4 | L | L | L | L | L | VL | M | L | L |
| P5 | VL | VL | VL | VL | VL | VL | VL | EL | L |
| Probability establishment via non-migratory route (Pn) | VL | VL | VL | VL | VL | VL | VL | EL | L |
| Probability establishment via migratory route (Pm) | VL | VL | VL | VL | VL | VL | VL | EL | L |
| Final risk (P): | VL | VL | VL | VL | VL | VL | VL | EL | L |
EL, Extremely low; VL, very low; L, low; M, medium; H, high.
For a description of events please refer to Figure 1.
DISCUSSION
The risk of NiV establishing in Australian flying-foxes through pre-border flying-fox movements was estimated to be low, with an associated high level of uncertainty. This outcome is strongly influenced by the probability of NiV establishing in Australian flying-fox populations (P5) following introduction. The sensitivity analysis also supports this finding. Hence the results highlight the importance of step P5 in the model. Further, the likelihood of NiV entry and spread (release and exposure assessments) is non-negligible (low and medium for the release assessment for non-migratory and migratory routes, respectively), so how the virus behaves once it arrives in Australia is critical. Given the high uncertainty associated with P5, further research in this area would be valuable. Recent studies show that African Green Monkeys vaccinated with a HeV subunit vaccine are protected against challenge with NiV [50], suggesting that cross-protection may be afforded against NiV infection following previous exposure to HeV. The findings of a seroepidemiological study by Breed et al. [43] suggest the probability of an Australian flying-fox population or subpopulation having a very low level of herd immunity to HeV at any particular time is less than previously thought. Hence it seems plausible that the presence of continually moderate to high herd immunity to HeV in Australian flying-foxes may act as a barrier to NiV incursion. However the limited knowledge of various aspects of henipavirus disease ecology (e.g. differences in infection dynamics between host and virus species, potential for co-infection of henipaviruses, role of cross-neutralizing antibodies in preventing infection) curtail better understanding of the factors that may influence this scenario and preclude an accurate assessment of its probability. If, however, Australian flying-fox populations were entirely susceptible to NiV infection, the final risk may increase significantly. NiV establishment in Australian flying-foxes has obvious animal and public health concerns, particularly given the overlapping human, domestic and feral pig, and flying-fox populations in eastern Australia. Further research on the infection dynamics of HeV and NiV (including pathogenesis and immunity) could greatly reduce the uncertainty in several parts of this risk assessment and hence improve the accuracy of the risk estimates.
The movement of bats between Australia and the immediate countries to the north is known to occur, although the frequency is not. Satellite telemetry studies of P. alecto have shown movements occur across the Torres Strait [14]. Evidence of movement and exchange of viruses between flying-fox populations in Australia and pre-border countries could be derived from genetic analyses of the henipaviruses present in pre-border countries. The infection dynamics of henipaviruses and other bat-borne viruses in pre-border countries is largely unknown. More data on the distribution of henipaviruses in bat populations, including clear differentiation between the virus species, and increased knowledge of the movement patterns of flying-foxes would contribute to a greater understanding of this area.
The different methods for combining expert opinion showed similar results. This may be influenced by the limited variation between experts' scores before combination and the moderating effects of averaging [29, 51]. If opinions were more skewed or ‘extreme’ then more variation between combination methods may have resulted. The modified Delphi approach is designed to provide synthesis and analysis of knowledge through open discussions, while retaining the benefits of multiple judgements without the biases of group discussions [19, 21–23]. In the workshop used for this study, experts in one subject area provided insights and knowledge to those in other areas. For example, the two experts in flying-fox ecology discussed the distances flying-foxes are likely to travel, behaviour of different species and level of contact between and within species of flying-fox. This resulted in greater knowledge of the entire group. However, the success of group interactions is dependent on the ability of the facilitator to encourage the sharing of knowledge and recognition of expertise, and to avoid the biases that can result from group discussions such as dominant personalities and overconfidence [24]. During the workshop conducted in this study, subject-matter experts as defined by Knol et al. [18], tended to contribute more to the discussion than the normative experts who provided more input on probabilities and pathways. This was one of the objectives of the workshop, allowing information on the subject to be shared between participants, and more harmonized estimates to result. It is not surprising then that the model output was relatively robust to the different combination methods used. However, some notable differences were apparent. When using the mean and LOP combination methods higher risk estimates resulted, where risk increased from very low to low. It is interesting to note that weighted methods did not influence the final risk estimates anymore than non-weighted methods. In light of the subjective nature of weighting experts, simpler methods for combining expert opinion may suffice. This may be particularly the case where little variation exists between experts' opinions, and simple averaging techniques are appropriate. Similar findings have been reported by Scholz & Hansmann [26]. An alternative approach may be based on a precautionary approach, where the technique that yields the highest risk could be used. Ultimately the decision to use one method over another is a difficult one, and will depend on the elicitation method used, type of assessment undertaken, and data obtained for the study.
There are few published risk assessments on the introduction and spread of a pathogen into a country through the movement of wildlife and hence validation of the assessment is limited. Snary et al. [32] performed a qualitative release assessment of henipavirus entry into the UK via different routes of introduction. In that study, researchers found a medium probability of importing infected flying-foxes from regions where henipaviruses are found, due to the medium probability that a fruit bat is infected, survives and goes undetected through the importation process.
The present study assumes that flying-foxes are the reservoir host for NiV and non-pteropid bats do not play a significant role in the risk of introduction of NiV to Australia. While some data indicate henipavirus infection may occur in bats of several other genera, available evidence suggests flying-foxes are the predominant host [8, 52, 53]. The assessment also assumes that NiV transmission to other flying-foxes is dependent on direct or indirect contact with an infected flying-fox, i.e. via contaminated urine, saliva, or other bodily fluids. Significant uncertainty exists on the mode of transmission of NiV within and between bat species. HeV transmission between Australian flying-fox species almost certainly occurs, since HeV isolates from different species show almost identical nucleotide sequences [54]. However, transmission is expected to be higher within species than between species, since contact rates are higher within species. Although different species share roosts together, they tend to segregate within roosts so contact is reduced [44]. Additionally, the apparent lack of clinical illness in flying-foxes infected with NiV is based on very limited information [42]. The publication of further information in these areas may warrant revision of this risk assessment.
Qualitative risk assessments provide a systematic way of assessing risk that can be communicated to decision makers readily. However, assigning probability estimates in qualitative categories is subjective without a standardized methodology. This can lead to inconsistent, unrepresentative or misleading outcomes in the risk pathway. Indeed, uncertainties arising from words with imprecise or different meanings, or differences between verbal and numerical probability estimates can lead to risk being interpreted differently by different individuals, such as risk assessors, decision makers and experts [55–58]. Despite these limitations, subjectivity can be reduced through a transparent approach. For example, the OIE provides a scale to classify and standardize qualitative probability categories [15]. National agencies in Australia (Biosecurity Australia), Canada (Canadian Food Inspection Agency) and the USA (United States Department of Agriculture) have used numerical ordinal scales that correspond to verbal expressions of risk for their qualitative risk assessments [59]. For example, very low, low, …, high is equivalent to 1, 2, …, n, where n is the number of points on the scale. These scales offer a relative measure of risk and provide an option to differentiate risk when it is impossible to quantify using probability measures [51]. In this assessment, we used the OIE scale to qualify our probability estimates and aligned this scale to the one used by experts in the workshop.
Difficulties also arise in qualitative risk assessments when combining steps in the pathway that are independent of each other. While there is a large number of matrices used to combine qualitative probabilities that are dependent on each other (see [17, 27, 60] for examples), there is no such rule for addition of qualitative probabilities. A case by case approach was taken in a risk assessment performed by Snary and co-workers [32]. In this assessment, independent pathways (Pn and Pm) were provided with separate probability estimates. To obtain an overall probability estimate, probabilities were ‘combined’ using the same mathematical rules that would be used for a quantitative assessment.
Expert opinion is subjective by nature. In highly uncertain events such as those presented in this model, experts' probability estimates can be difficult to quantify [61]. Consequently, the high level of uncertainty placed on many events in this model implies caution when considering the risk outputs. However, it provides a useful tool for communicating these risks to decision makers, and provides a clear model structure, key information needs and areas of uncertainty. The assessment also highlights the influence that different methods for combining expert opinion has on final risk estimates and the need to be aware of these methods when interpreting the findings.
Despite the large gaps in knowledge as outlined above, sensible and ethical policy and management options are currently required given the probable presence of NiV within a bat's ‘flying-distance’ of Australia [9]. Suggestions for reducing the probability of transmission of henipaviruses to domestic animals in Australia are outlined in Breed et al. [62]. These include: the planting of trees that are not attractive to flying-foxes in preference to those that are currently often planted around areas where livestock are kept (i.e. figs, melaleucas, various eucalypts and introduced fruit trees should be avoided); ensuring that feed bins and water troughs are not placed under trees in which flying-foxes feed or roost; and the placing of feed bins and water troughs under cover. The risk to human health from contact with livestock can be managed by adoption of infection control protocols and risk-related biosecurity measures, including the use of appropriate personal protective equipment. While the culling of flying-foxes has been suggested by some as a management activity, there is strong evidence from other wildlife diseases, including rabies in bats, that culling may well exacerbate the problem rather than provide a solution [62, 63]. Human, livestock and environmental health authorities are increasingly adopting a One Health approach to infectious diseases, recognizing that these three sectors are inextricably linked and interdependent.
ACKNOWLEDGEMENTS
This work was supported by the Australian Biosecurity Cooperative for Research Centre Emerging Infectious Disease and the consulted experts. We thank Katarina Staerk, Emma Snary, Robin Simons and Kim Stevens for advice on incorporation of expert opinion to risk analysis, and Lisa Adams and Deb Cousins for facilitating and helping to organize the workshop.
DECLARATION OF INTEREST
None.
REFERENCES
- 1.Chua KB, et al. Nipah virus: a recently emergent deadly paramyxovirus. Science 2000; 288: 1432–1435. [DOI] [PubMed] [Google Scholar]
- 2.Hsu V, et al. Nipah virus encephalitis reemergence, Bangladesh. Emerging Infectious Diseases 2004; 10: 2082–2087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chadha M, et al. Nipah virus-associated encephalitis outbreak, Siliguri, India. Emerging Infectious Diseases 2006; 12: 235–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Gurley ES, et al. Person-to-person transmission of Nipah virus in a Bangladeshi community. Emerging Infectious Diseases 2007; 13: 1031–1037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Eaton BT, et al. Hendra and Nipah viruses: different and dangerous. Nature Reviews Microbiology 2006; 4: 23–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Chua K, et al. Isolation of Nipah virus from Malaysian Island flying foxes. Microbes and Infection 2002; 4: 145–151. [DOI] [PubMed] [Google Scholar]
- 7.Epstein JH, et al. Nipah virus: impact, origins, and causes of emergence. Current Infectious Disease Reports 2006; 8: 59–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Calisher CH, et al. Bats: important reservoir hosts of emerging viruses. Clinincal Microbiology Review 2006; 19: 531–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Breed AC, et al. The distribution of henipaviruses in southeast Asia and Australasia: is Wallace's Line a barrier to Nipah virus? PLoS ONE 2013; 8: e61316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sendow I, et al. Henipavirus in Pteropus vampyrus bats, Indonesia. Emerging Infectious Diseases 2006; 12: 711–712. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Field H, et al. The natural history of Hendra and Nipah viruses. Microbes and Infection 2001; 3: 307–314. [DOI] [PubMed] [Google Scholar]
- 12.Ratcliffe F. Notes on the fruit bats (Pteropus spp.) of Australia. Journal of Animal Ecology 1932, 1: 32–57. [Google Scholar]
- 13.Roberts BJ, et al. The outcomes and costs of relocating flying-fox camps: insights from the case of Maclean, Australia. In: Law B, Eby P, Lunney D, Lumsden L, eds. Symposium on the Biology and Conservation of Australasian Bat. Sydney, 12–14 April 2007, pp. 277–287. [Google Scholar]
- 14.Breed AC, et al. Bats without borders: long-distance movements and implications for disease risk management. EcoHealth 2010; 7: 204–212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.OIE. Handbook on Import Risk Analysis for Animals and Animal Products, volume 1, Introduction and Qualitative Risk Analysis. Paris: Office International des Epizooties, 2004. [Google Scholar]
- 16.Clough HE, Clancy D, French NP. Vero-cytotoxigenic Escherichia coli O157 in pasteurized milk containers at the point of retail: a qualitative approach to exposure assessment. Risk Analysis 2006; 6: 1291–1309. [DOI] [PubMed] [Google Scholar]
- 17.Peeler EJ, et al. The application of risk analysis in aquatic animal health management. Preventive Veterinary Medicine 2007; 81: 3–20. [DOI] [PubMed] [Google Scholar]
- 18.Knol AB, et al. The use of expert elicitation in environmental health impact assessment: a seven step procedure. Environmental Health 2010; 9: 19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mosleh A, Bier VM, Apostolakis G. A critique of current practice for the use of expert opinions in probabilistic risk assessment. Reliability Engineering & System Safety 1988; 20: 63–85. [Google Scholar]
- 20.Van der Fels-Klerx IHJ, et al. Elicitation of quantitative data from a heterogeneous expert panel: formal process and application in animal health. Risk Analysis 2002; 22: 67–81. [DOI] [PubMed] [Google Scholar]
- 21.Larreche J, Moinpour R. Managerial judgment in marketing: the concept of expertise. Journal Marketing Research 1983; XX: 110–121. [Google Scholar]
- 22.Gustafson DH, et al. A comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups. Organizational Behavior and Human Performance 1973; 9: 280–291. [Google Scholar]
- 23.Riggs WE. The Delphi technique: an experimental evaluation. Technological Forecasting and Social Change 1983: 23: 89–94. [Google Scholar]
- 24.O'Hagan A, et al. Uncertain Judgments: Eliciting Expert's Probabilities. Chichester, UK: John Wiley & Sons, 2006, pp. 323. [Google Scholar]
- 25.Clemen RT, Winkler RL. Combining probability distributions from experts in risk analysis. Risk Analysis 1999; 19: 187–203. [DOI] [PubMed] [Google Scholar]
- 26.Scholz RW, Hansmann R. Combining experts’ risk judgments on technology performance of phytoremediation: self-confidence ratings, averaging procedures, and formative consensus building. Risk Analysis 2007; 27: 225–240. [DOI] [PubMed] [Google Scholar]
- 27.Gale P, et al. Assessing the impact of climate change on vector-borne viruses in the EU through the elicitation of expert opinion. Epidemiology and Infection 2009; 7: 1–12. [DOI] [PubMed] [Google Scholar]
- 28.Normand ST, et al. Eliciting expert opinion using the Delphi technique: identifying performance indicators for cardiovascular disease. International Journal for Quality in Health Care 1998; 10: 247–260. [DOI] [PubMed] [Google Scholar]
- 29.Ariely D, et al. The effects of averaging subjective probability estimates between and within judges. Journal of Experimental Psychology – Applied 2000; 6: 130–147. [DOI] [PubMed] [Google Scholar]
- 30.Wacharapluesadee S, et al. Bat Nipah virus, Thailand. Emerging Infectious Diseases 2005; 11: 1949–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Reynes JM, et al. Nipah virus in Lyle's flying foxes, Cambodia. Emerging Infectious Diseases 2005; 11: 1042–1047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Snary EL, et al. Qualitative release assessment to estimate the likelihood of henipavirus entering the United Kingdom. PLoS ONE 2010; 7: 2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Daszak P, et al. The emergence of Nipah and Hendra virus: pathogen dynamics across a wildlife-livestock-human continuum. In: Collinge S, Ray C, eds. Disease Ecology: Community Structure and Pathogen Dynamics. Oxford, UK: Oxford University Press, 2006, pp. 186–201. [Google Scholar]
- 34.Olival KJ. Population genetic structure and phylogeography of Southeast Asian flying foxes: implications for conservation and disease ecology (dissertation). New York, NY, USA: Columbia University, 2008. [Google Scholar]
- 35.Robertson PB. Small islands, natural catastrophes, and rapidly disappearing forests: a high vulnerability recipe for island populations of flying-foxes. In: Wilson DE, Graham GL, eds. Pacific Island Flying Foxes: Proceedings of an International Conservation Conference. Washington, DC: Department of the Interior Fish and Wildlife Service, 1992, pp. 176. [Google Scholar]
- 36.Daniel M. First report of an Australian fruit bat (Megachiroptera: Pteropodidae) reaching New Zealand. New Zealand Journal of Zoology 1975; 2: 227–231. [Google Scholar]
- 37.Pierson ED, Rainey WE. The biology of flying foxes of the genus Pteropus: a review. In: Wilson DE, Graham GL, eds. Pacific Island Flying Foxes: Proceedings of an International Conservation Conference. Washington, DC: US Department of the Interior Fish and Wildlife Service, 1992, pp. 176. [Google Scholar]
- 38.Palmer C, Woinarski JCZ. Seasonal roosts and foraging movements of the black flying fox (Pteropus alecto) in the Northern Territory: resource tracking in a landscape mosaic. Wildlife Research 1999; 26: 823–838. [Google Scholar]
- 39.Vardon MJ, Tidemann CR. Flying-foxes (Pteropus alecto and P. scapulatus) in the Darwin region, north Australia: patterns in camp size and structure. Australian Journal of Zoology 1999; 47: 411–423. [Google Scholar]
- 40.Hall L, Richards G. Flying-foxes, Fruit and Blossom Bats. Sydney: University of New South Wales Press, 2000, pp. 148. [Google Scholar]
- 41.Fogarty R, et al. Henipavirus susceptibility to environmental variables. Virus Research 2008; 132: 140–144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Middleton DJ, et al. Experimental Nipah virus infection in pteropid bats (Pteropus poliocephalus). Journal of Comparative Pathology 2007; 136: 266–72. [DOI] [PubMed] [Google Scholar]
- 43.Breed AC, et al. Evidence of endemic Hendra virus infection in flying-foxes (Pteropus conspicillatus) – implications for disease risk management. PLoS ONE 2011; 6: e28 816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Birt P, Markus N. Notes on the temporary displacement of Pteropus alecto and P. poliocephalus by P. scapulatus within a daytime campsite. Australian Mammalogy 1999; 21: 107–110. [Google Scholar]
- 45.Halpin K, et al. Isolation of Hendra virus from pteropid bats: a natural reservoir of Hendra virus. Journal of General Virology 2000; 81: 1927–1932. [DOI] [PubMed] [Google Scholar]
- 46.Plowright RK, et al. Reproduction and nutritional stress are risk factors for Hendra virus infection in little red flying foxes (Pteropus scapulatus). Proceedings of the Royal Society of London, Series B: Biological Sciences 2008; 275: 861–869. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Defang GN, et al. Induction of neutralizing antibodies to Hendra and Nipah glycoproteins using a Venezuelan equine encephalitis virus in vivo expression system. Vaccine 2011; 29: 212–220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Chan YP, et al. Biochemical, conformational, and immunogenic analysis of soluble trimeric forms of henipavirus fusion glycoproteins. Journal of Virology 2013; 86: 11457–11471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Webb NJ, Tidemann CR. Mobility of Australian flying-foxes, Pteropus spp. (Megachiroptera): evidence from genetic variation. Proceedings of the Royal Society of London, B: Biological Sciences 1996; 263: 497–502. [DOI] [PubMed] [Google Scholar]
- 50.Bossart KN, et al. Neutralization assays for differential henipavirus serology using Bio-Plex protein array systems. Journal of Virological Methods 2007; 142: 29–40. [DOI] [PubMed] [Google Scholar]
- 51.Holt J. Score averaging for alien species risk assessment: a probabilistic alternative. Journal of Environmental Management 2006; 81: 58–62. [DOI] [PubMed] [Google Scholar]
- 52.Chua KB. Nipah virus outbreak in Malaysia. Journal of Clinical Virology 2003; 26: 265–75. [DOI] [PubMed] [Google Scholar]
- 53.Hayman DTS, et al. Evidence of henipavirus infection in west African fruit bats. PLoS ONE 2008; 3: e2739. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Smith I, et al. Identifying Hendra Virus Diversity in Pteropid Bats. PLoS ONE 2011; 6: e25 275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Gigerenzer G, et al. ‘A 30% chance of rain tomorrow’: how does the public understand probabilistic weather forecasts? Risk Analysis 2005; 25: 623–629. [DOI] [PubMed] [Google Scholar]
- 56.Teigen KH. When equal chances equals good chances: verbal probabilities and the equiprobability effect. Organizational Behavior and Human Decision Processes 2001; 85: 77–108. [DOI] [PubMed] [Google Scholar]
- 57.Carey JM, Burgman MA. Linguistic uncertainty in qualitative risk analysis and how to minimize it. Strategies for Risk Communication: Evolution, Evidence, Experience 2008; 1128: 13–17. [DOI] [PubMed] [Google Scholar]
- 58.Franklin J, et al. Evaluating extreme risks in invasion ecology: learning from banking compliance. Diversity and Distributions 2008; 14: 581–591. [Google Scholar]
- 59.Holt J, Black R, Abdallah R. A rigorous yet simple quantitative risk assessment method for quarantine pests and non-native organisms. Annals of Applied Biology 2006; 149: 167–173. [Google Scholar]
- 60.Moutou F, Dufour B, Ivanov Y. A qualitative assessment of the risk of introducing foot and mouth disease into Russia and Europe from Georgia, Armenia and Azerbaijan. Revue Scientifique et Technique de l'Office International des Epizooties 2001; 20: 723–730. [DOI] [PubMed] [Google Scholar]
- 61.Vose D. Risk Analysis. A Quantitative Guide, 2nd edn. Chichester: John Wiley & Sons, 2000, pp. 418. [Google Scholar]
- 62.Breed AC, et al. Re: flying foxes carrying Hendra virus in Queensland pose a potential problem for other states. Australian Veterinary Journal 2010; 88: 24. [Google Scholar]
- 63.Streiker D, et al. Ecological and anthropogenic drivers of rabies exposure in vampire bats: implications for transmission and control. Proceedings of the Royal Society of London, Series B: Biological Sciences 2012; 279: 3384–3392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.EFSA. Migratory birds and their possible role in the spread of highly pathogenic Avian Influenza (EFSA-Q-2005-243). European Food Safety Authority Journal 2006; 357: 1–46. [DOI] [PMC free article] [PubMed] [Google Scholar]