Can foodborne illness estimates from different countries be legitimately compared?: case study of rates in the UK compared with Australia, Canada and USA

Abstract

Objective

Mathematical models have gained traction when estimating cases of foodborne illness. Model structures vary due to differences in data availability. This begs the question as to whether differences in foodborne illness rates internationally are real or due to differences in modelling approaches.

Difficulties in comparing illness rates have come into focus with COVID-19 infection rates being contrasted between countries. Furthermore, with post-EU Exit trade talks ongoing, being able to understand and compare foodborne illness rates internationally is a vital part of risk assessments related to trade in food commodities.

Design

We compared foodborne illness estimates for the United Kingdom (UK) with those from Australia, Canada and the USA. We then undertook sensitivity analysis, by recreating the mathematical models used in each country, to understand the impact of some of the key differences in approach and to enable more like-for-like comparisons.

Results

Published estimates of overall foodborne illness rates in the UK were lower than the other countries. However, when UK estimates were adjusted to a more like-for-like approach to the other countries, differences were smaller and often had overlapping credible intervals. When comparing rates by specific pathogens, there were fewer differences between countries. The few large differences found, such as virus rates in Canada, could at least partly be traced to methodological differences.

Conclusion

Foodborne illness estimation models are country specific, making international comparisons problematic. Some of the disparities in estimated rates between countries can be shown to be attributed to differences in methodology rather than real differences in risk.

WHAT IS ALREADY KNOW ABOUT THIS SUBJECT

• In recent years, several countries have used mathematical models to produce estimates of foodborne illness. The models vary by country and are dependent on the source data available. These estimates were developed to aid within country policy development and prioritisation.

• Since EU Exit the results from these models have been used to make comparisons of foodborne illness rates between countries as part of risk assessments related to trade in food commodities.

• Published estimates of total infectious intestinal disease and total foodborne illness rates are lower in the UK than Australia, Canada and USA.

WHAT ARE THE NEW FINDINGS

• When UK estimates of total infectious intestinal disease and foodborne illness rates were adjusted to a more similar approach to the other three countries, differences were smaller and often had overlapping credible intervals.

• Differences in estimates in foodborne illness rates for individual pathogens, while generally smaller, can in some instances also be demonstrated to be affected by differences in methodology.

HOW MIGHT THIS IMPACT OF CLINICAL PRACTICES IN THE FORESEEABLE FUTURE

• This study highlights the uncertainty in estimates of foodborne illness and how differences in methodology can impact results making comparisons between countries problematic. However, whichever methodology is used, foodborne illness remains a significant issue in the UK.

Introduction

Foodborne illness (FBI), also referred to as foodborne disease or food poisoning, is caused by consuming contaminated food or drink. The majority of over 250 known FBIs are due to pathogenic micro-organisms. Typical symptoms include nausea, vomiting, stomach cramps and diarrhoea. The WHO estimates that just 31 foodborne diseases cause as many as 600 million cases of illness worldwide each year.¹ That is equivalent to almost 1 in 10 people on the planet becoming ill annually. These illnesses lead to around 420 000 deaths, with children under 5 years of age bearing the brunt (125 000 deaths in this age group). Diarrheal diseases alone, caused by bacteria, viruses and parasites, are associated with 550 million illnesses and 230 000 deaths. Norovirus and Campylobacter spp are the most common viral and bacterial causes of FBI, respectively. Non typhoidal salmonellas, Salmonella Typhi, Taenia solium, hepatitis A virus and aflatoxin are the most common causes of death.

Coverage of COVID-19 has highlighted the difficulties of comparing infectious disease case rates between countries.² There are similar problems when comparing infectious intestinal disease (IID) and FBI rates. Following the result of the 2016 United Kingdom (UK) European Union (EU) membership referendum one of the many questions raised was whether changes in the UK’s trading partners might increase food safety risks. International trade in food is big business with 28% of food and drink consumed in the UK coming from the EU and a further 19% from third countries.³ To try to answer this question, several commentators have quoted publicly available FBI rates. In particular, media focus has been on differences between the UK and USA due to the potential of a substantial trade deal between the two nations. The US Centers for Disease Control and Prevention estimate that 48 million people get ill from FBI each year—a rate of one in six people getting FBI a year.⁴ For comparison, in the UK, the Food Standards Agency (FSA) estimates 2.4 million cases of FBI a year—a rate of 1 in 28 people getting FBI a year.⁵ On the face of it, this would support the assertion that the risk of acquiring FBI in the USA is over four and half times greater than in the UK. However, such top-level figures disguise differences in the ways in which baseline data are collected, the pathogens included, the manner in which overall disease burden is attributed to foodborne transmission, and other important nuances in calculations. Furthermore, all estimates provide credible intervals to show uncertainty. These intervals can be quite wide but are seldom reported in the press or by others. Without a full understanding of the context and methods involved in producing estimates, comparisons can be misleading at best, or at worst, be used to distort the facts deliberately.

Following discussion in the press and elsewhere on the relative risks of UK and US food in relation to potential trade deals, we re-analysed the estimates from the UK and the USA to determine what evidence there is for real differences in FBI rates between the two countries, considering both overall FBI levels and estimates for individual pathogens. We added studies from additional countries to the analysis based on a literature review, commissioned by the UK’s FSA, of the different methodologies used to estimate foodborne disease in the UK and in other countries.⁶ This review found 33 studies between 1994 and 2017. Of these studies, those for Australia and Canada used very similar approaches to the study for the USA, with all three providing estimates for at least 12 of the 13 individual pathogens covered in the UK study. We, therefore, decided to add the Australian and Canadian studies to the analysis, to test whether arguments for the differences between the UK and USA would hold for them. For total IID, we also included research from Netherlands and Sweden, which were the only other two studies referenced in the literature review that, like the UK, used cohort studies. Unfortunately, these two studies did not include estimates for individual pathogens or for FBI so could not be included in the rest of the analyses.

Methods

This re-analysis was based on data and results from studies for the UK,^{5 7} Australia,⁸ Canada⁹ and USA.^10–12 The main features of the studies in the four countries are summarised in the online supplemental table 1.

The studies for each country produced the following estimates (see also online supplemental table 2):

1. Total IID: this was based on the specific case definition of IID (or acute gastroenteritis illnesses as it is often referred to outside the UK) used in each study. For Australia, Canada and USA, telephone surveys with 28-day recall were used, while for the UK, three different estimates were produced. These were based on a cohort study and two telephone surveys using 7-day and 28-day recall, respectively.

2. Total cases for specific pathogens: for Australia, Canada and USA, the studies used two different approaches depending on pathogen and data availability. The first used surveillance data scaled up for underreporting and underdiagnosis. They were then adjusted for cases acquired abroad. The second used an estimate of the proportion of IID due to each pathogen and applied this to the total IID in 1 above.

   For the UK, estimates were based on a cohort study undertaken in 2009: where data were sufficiently robust, the UK had subsequently adjusted estimates for some pathogens based on changes in confirmed laboratory reports to reflect changes between 2009 and 2018.

   As well as producing estimates for pathogens that cause IID, all four countries produced estimates for one or more non-IID pathogen (pathogens that do not cause diarrhoea or vomiting) using approach 1. For Australia, the IID and non-IID pathogens were reported separately and only total FBI figures for the IID pathogens are included in this paper. For the other three countries, the impact of excluding non-IID pathogens is explored later.

3. Cases of FBI caused by specific pathogens: for all four countries, these were based on the estimates from all sources (2 above) multiplied by the proportion estimated to be due to food. This is with the exception of norovirus in the UK where the estimate was based on the Norovirus Attribution study,^{13 14} which assessed the contribution made by the food chain to the burden of UK-acquired norovirus infection.

4. Total FBI: for all four countries, these estimates were based on subtracting the IID estimates for each specific IID pathogen (2 above) from total IID (1 above) to get an estimate of unattributed cases (also known as unspecified agents or cases of unknown aetiology). These are cases where no specific pathogen was identified, either because there was insufficient data to attribute it to a specific pathogen, or the specific pathogen was not one of the ones included in the study. This figure was then multiplied by the foodborne proportion, estimated by taking the weighted foodborne proportion of all the specific IID pathogens. The foodborne estimate for the unattributed cases was then added to all the foodborne estimates for the specific pathogens (with the exception of Australia, these included one or more non-IID pathogen) to get total FBI. These unattributed cases make a large contribution of overall FBI, accounting for around 60% of total FBI in the UK and Canada, 80% in Australia and the USA.

Estimates from 2 to 4 all used Monte Carlo simulations to model the uncertainty in the estimates (note while the USA study used estimates under 2 to produce 3 and 4, the figures were not published. We, therefore, recreated these estimates based on the information in the technical annex 3 of the study).

Using the data available from these studies, we compared the estimates of overall FBI and IID as well as the estimates for individual pathogens. Estimates were converted into rates per 1 million person-years. To compare results between countries, more easily we re-ran the UK models to produce estimates of the mean and 90% credible intervals (UK published results gave the median with 95% credible intervals), so that the outputs were consistent. We recreated the Monte Carlo simulation models for each country based on information in the published papers and relevant technical appendices to provide comparable outputs. This allowed us to produce comparable results for all countries and to try some sensitivity analysis under different assumptions. Each model, including the one for the UK, was produced using @risk V.7.6 (http://www.palisade.com/). Each simulation was run for 100 000 iterations. As the reported results were from simulations using random numbers, it was not possible to get complete agreement; however, differences between our results and those published were small.

For comparisons between countries, we compared 90% credible intervals.

Results

Overall IID rates

Figure 1 (see online supplemental table 3) shows 90% credible intervals for the overall IID rates. The studies for Australia, Canada and USA used telephone surveys with 28-day recall to estimate total IID. The UK produced three estimates, a cohort study plus two telephone studies with 7-day and 28-day recall, respectively.¹⁵ We have also added two cohort studies from Netherlands and Sweden for comparison (cohort studies tend to be uncommon, largely due to their complexity and expense).^{16 17}

Figure 1.

Infectious Intestinal Disease rates per 1 million population per annum (90% credible intervals shown with the mean).

Differences between countries, using similar telephone surveys with 28-day recall, are smaller than the differences between the three UK approaches, with the telephone surveys giving larger IID estimates than the cohort study. The cohort studies from Netherlands and Sweden give estimates that are closer to the UK cohort study.

Overall FBI rates

Figure 2 (see online supplemental table 4) shows 90% credible intervals for the overall FBI rates for all four countries. The highest mean rate is for Australia with 193 000 cases per 1 million person-years. This compares with the UK cohort rate of 36 000 cases per 1 million person-years.

Figure 2.

Foodborne illness rates per 1 million population per annum (90% credible intervals shown with the mean).

We re-ran the UK model using the figures from the UK telephone surveys with 7 and 28-day recall for a fairer comparison. Figure 2 shows the mean rate from the 28-day recall is two times the rate from the UK cohort model, but it is only about a third of that from the 7-day recall. The mean estimate for the UK using 28-day recall is still less than the other three countries. However, its 90% credible interval overlaps with those of Australia, Canada and the USA (note the cohort surveys in Netherlands and Sweden mentioned previously did not provide foodborne disease rates so could not be included).

Sensitivity of FBI rates to the inclusion of certain pathogens

Each study used a slightly different set of pathogens to produce estimates of total FBI. We, therefore, reproduced estimates for each country based on the 10 IID pathogens common to all four studies plus estimates for unattributed cases. The 10 common pathogens were C. perfringens, Campylobacter, Salmonella non typhoidal, Shigella, Cryptosporidium, Giardia, astrovirus, norovirus, rotavirus and sapovirus. These estimates were then compared with those published in the individual papers (see online supplemental figure S1)

Despite the large variation in the number of pathogens included in each study, the differences between the new estimates and those in the papers were relatively small and certainly much smaller than differences in rates seen between countries. There are a few reasons for this. First, these 10 pathogens contribute a large percentage of the total estimates from the known pathogens in each country, that is, 98.4% for the UK, 64.7% for Australia (96% of the rest is from other pathogenic E. coli), 91.0% for Canada and 91.7% for USA (see online supplemental table 5). Second, when the additional IID pathogens are removed from the calculations, this does not reduce overall IID, it just moves it from known pathogens to unspecified agents. Third, the proportion of IID from unspecified agents that is attributed to food is based on the weighted proportion for each of the known pathogens included in the calculation. As previously stated, the majority of cases are from the 10 common IID pathogens, so the weighting used only changes by a small amount when the other IID pathogens are excluded (see online supplemental table 6 for percentages). Finally, the number of cases attributed to non-IID pathogens is very small, making up 0.02% of the total for the UK, 0.6% for Canada and 1.1% for USA (for Australia only IID pathogens were included in the estimates in this paper). All four of these effects, even in combination, only make a small difference to overall estimates.

To further analyse the impact of using a cohort study compared with a telephone survey with either 7-day or 28-day recall, we produced estimates for IID for known pathogens only, as these estimates do not use any of these studies. As the different studies included, different known pathogens in their estimates of FBI, we estimated the total IID rate for the 10 common IID pathogens used in all four studies, as listed above, plus Listeria monocytogenes which was also included in all four studies. The total IID rates for USA and Australia for these 11 pathogens (see online supplemental table 7 and figure S2) are much closer to those in the UK, with large overlaps in the credible intervals. However, the IID rate for the 11 pathogens is two times as high in Canada than in the UK.

A key reason for this difference to Canada is the approach used to estimate virus rates. As no Canadian virus data were available, the Canadian study calculated these proportions using the UK IID2 study data. To do this, they used the UK estimate for each virus divided by the overall estimate of IID from the UK cohort study to get a proportion of the total IID caused by the virus. The Canadian estimate of total IID obtained by their telephone survey was then multiplied by this proportion. As the Canadian estimate of total IID (based on a telephone survey with 28-day recall) was 2.3 times greater than the UK cohort study, this means their estimate for each virus was also greater. If the study had used the UK’s telephone survey with 28-day recall instead as the estimate of total IID in the UK, then the Canadian virus estimates would have decreased by 51%. The Canadian estimate for C. perfringens was estimated in the same way as the viruses and so has the same issue. This is explored further in the comparisons for these individual pathogens later on.

We similarly considered just the 11 known pathogens common to all studies and estimated their contribution to foodborne disease. FBI rates on this basis are still lower in the UK than in the other three countries (see online supplemental table 8 and figure S3). Again, the large difference between the UK and Canadian figures is largely due to the way viruses and C. perfringens were estimated in the Canadian study as discussed above.

IID rates for individual pathogens

Figure 3 (see online supplemental table 9) shows the 90% credible intervals for the estimated IID rates for each pathogen and country. These are for the 13 pathogens included in the UK study.

Figure 3.

Infectious Intestinal Disease rates for individual pathogens per 1 million population per annum (90% credible intervals shown with the mean).

For most pathogens, the 90% credible intervals largely overlap between countries, although there are some exceptions. One pathogen where there are differences is E. coli O157. In the UK models, it is assumed that there is no under-reporting due to the seriousness of the illness, which explains the much lower estimates compared with other countries, where adjustments for underreporting have been included in the models. A second example is rotavirus, where estimates in UK and Australia are lower than the other two countries. For the UK at least, a large reason for this difference will be the years on which the estimates are based. In the UK, estimates are based on 2018 data (Australia on 2010) while Canadian and US estimates are based on 2006 data (see online supplemental table 1). This difference in timescales is particularly important due to roll out rotavirus vaccination programmes in infants over this period, which has decreased overall rates of the illness.¹⁸

As previously mentioned, no virus or C. perfringens data were available for Canada, so estimates were based on UK data using the proportion of IID associated with each pathogen. The UK IID figures used were those from the cohort study. If they were based on the telephone survey with 28-day recall, which is 2.3 times larger (see figure 1), then the proportion of IID attributed to each pathogen would reduce as would the Canadian estimates. As the estimates of IID for Canada also used a telephone survey with 28-day recall, this seems a reasonable adjustment. Figure 4 demonstrates the impact of this change. It shows estimates for Canada for C. perfringens, adenovirus, astrovirus, norovirus, rotavirus and sapovirus. New estimates have been calculated using the proportion of the IID attributed to each pathogen based on the UK telephone survey with 28-day recall. These estimates are then compared with those from the UK and the previous estimates for Canada based on the cohort study.

Figure 4.

Infectious Intestinal Disease (IID) rates per 1 million population per annum for Canada for selected pathogens using different estimates of the proportion of IID attributed to these pathogens (90% credible intervals shown with the mean).

As is seen in figure 4 (see online supplemental table 10), the estimates for Canada are now much more similar to those of the UK. This is with the exception of rotavirus where UK rates are still lower due to the infant vaccination programme mentioned earlier.

FBI rates for individual pathogens

Figure 5 (see online supplemental table 11) shows the 90% credible intervals for the estimated FBI rates for each pathogen and country.

Figure 5.

Foodborne illness rates for individual pathogens per 1 million population per annum (90% credible intervals shown with the mean).

As with estimates of IID, for most pathogens, the 90% credible intervals largely overlap between countries, with E. coli O157 again an outlier. Estimates for viruses are more similar than for IID. The reason for this narrowing of the gap is that although estimates of the proportion of IID attributed to food are low for the UK for viruses, they are still higher than for some of the other countries.

Estimates of the proportion of cases attributed to food by pathogen by country are found in online supplemental table 12 and figure S4. Notable differences in these estimates between countries include those for Campylobacter and Giardia. For Campylobacter, the mean estimate for the UK (48%) is smaller than in the other three countries (77% for Australia, 68% for Canada and 80% for the USA). The 90% credible intervals for the UK overlap with those for Australia and Canada, but not those for the USA. For Giardia, median estimates for the proportion of cases attributed to food in the UK (22%) are higher than the other three countries (6% for Australia, 7% for Canada and 7% for the USA), although the 90% credible intervals overlap. Despite these differences, the 90% credible intervals for foodborne estimates for both Campylobacter and Giardia overlap for all four countries.

Discussion

Overall FBI

The three different IID estimates from the UK IID2 study varied considerably in size. This highlights the impact of methodology. Using the estimates of IID from either of the telephone surveys rather than the cohort study will lead to large increases in the FBI estimate, everything else being equal. An earlier study has previously shown the impact of such differences on estimates of IID,¹⁵ including the effect of length of the recall period of telephone surveys, but here we also show the impact of such differences on estimates of FBI.

As shown in figures 1 and 2, the total UK IID and FBI rates based on the UK telephone survey with 28-day recall are closer to the estimates in the comparator countries. However, mean estimates are still less than the other three countries. Methodological differences in the estimation of individual pathogen rates may explain some of the variation, as can differences in the case definition of IID (see online supplemental table 1). The impact of the latter has been explored more fully in a study looking at a common symptom-based case definition of gastroenteritis.¹⁹

Future studies are likely to include online surveys which may introduce further methodological differences.

Individual pathogens

Researchers in each country have estimated cases based on the best data available to them. This varies considerably between countries and pathogens. For instance, the UK estimates of IID are based on a cohort study, while the other three main studies reviewed used one of two different models for each pathogen depending on the data availability. Even then, further tweaks were needed. These included adjusting for partial geographical surveillance for some pathogens in Australia and the USA, and using proxy parameters from other pathogens where data were missing. Arguably then, it is inappropriate to compare estimates for individual pathogens that are not produced on a like-for-like basis.

A further consideration is the way the foodborne proportion is estimated for in each study, as the accuracy of this clearly makes a difference to the results. Both the Australia and Canadian studies used expert elicitation to estimate these proportions,^{20 21} while the UK and USA use outbreak data to estimate what proportion of cases from outbreaks was foodborne (with the exception of norovirus in the UK where foodborne estimates are based on a quantitative risk assessment model). This is an area of major uncertainty due to the difficulty of attributing cases to specific sources, meaning estimates are often based on limited data. Consequently, a few percentage point difference in proportion can lead to a large difference in the final estimate, particularly when the proportion is small. This is illustrated with sapovirus, where the estimated proportion attributed to food in the UK is 2.6%, which is fivefold higher than the Canadian and USA figures of 0.5%. Where the proportion of cases attributed to food is higher, a similar percentage point difference will have a much smaller impact, for example, a change from 90% to 92.1% would be only a 2.3% increase.

For Salmonella non typhoidal, the USA and Canada had significantly higher rates than the UK. We could find no obvious methodological reason for this difference. A real difference may be due to the mass vaccination of poultry in the UK from the late 1990s.²² Following this, human cases of Salmonella non typhoidal decreased by 40% between 2000 and 2009 in the UK (Food Standards Agency, 2018).²³

A general point is that the studies on which these estimates are based are resource intensive and so are infrequently undertaken. This means that the comparisons made are based on research undertaken several years ago and at different periods. The study for the UK used data from 2008 to 2009 and 2018, Australia 2006 to 2010, Canada 2001 to 2006 and USA 2000 to 2008. The number of cases for each pathogen will vary by year anyway, but there may also been more fundamental long-term changes over time, which could explain differences.

Conclusions

Estimates of FBI are important for policymakers in making decisions on resource allocation and prioritising interventions. Recently, such estimates have been used by various commentators to make statements on relative food safety risks in different countries. This paper shows that such comparisons need to be undertaken with considerable caution and appropriate caveats. Comparisons for individual pathogens between the four studies are possible but need very careful interpretation. While we have tried to control for methodological differences in approach where possible, there are other sources of variations between countries which we were unable to adjust for which may also have an impact on estimates. These factors include surveillance data, geographical coverage, access to healthcare and laboratory testing practices.

For overall estimates of FBI, comparing published rates between countries can be misleading due to major differences in methodology and data sources. Our sensitivity analysis adjusted UK estimates to a similar basis as the other three countries. Then the differences in estimates, which initially appear large, are much smaller, often with overlapping credible intervals.

By far the biggest impact on estimates for overall IID and FBI is the type of study used, for example, cohort study or telephone survey. In the case of the latter, the length of the recall period is also a major contributor to differences. We, therefore, recommend that only estimates using the same approach are compared, and ideally undertaken at a similar period of time. Differences in the number of pathogens included tend to have less impact, as most studies all include the major pathogens which contribute most of the total burden.

For individual pathogens, one big consideration is the year the estimates are for, which can make a large difference, particularly where interventions directed at specific pathogens have been undertaken between the different time periods. Another big factor we found is when estimates are based on the proportion of IID caused by a specific pathogen based on figures from a different country, particularly when that country uses a different method to estimate total IID.

Finally, differences between FBI rates are only part of the story in terms of whether changes in trading partners may change risks for the UK. These risks are more likely to be dependent on which specific foods are imported from each country and the relative risks of these compared with the equivalent domestically produced food, rather than overall FBI rates. Genuine differences in FBI rates between countries are also likely to be influenced by factors such as consumer preferences in terms of which foods are consumed and how they are prepared, diet, hygiene practices both in food business and at home and wider environmental factors such as climate.

Data availability statement

All data relevant to the study are either included in ther article, in the papers referenced or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.