Section	Guidance point no.	Evidence/reasoning
3.1.1 Alignment with the assessment	1	Reasoning: It is essential for transparency and consistency that all parts of the communication should convey the same message
	2	Reasoning: Minimises ambiguity about what has been assessed and what conditions and outcome the scientific conclusion and uncertainty relate to, which is essential for the information to be understood, and to allow unambiguous expression of its uncertainty in terms of probability
	3	Reasoning: This is critical for correct understanding of the conclusion and its associated uncertainty
3.1.2 Describing uncertainty with words	1 and 2	Evidence: Many studies demonstrate that verbal expressions of probability (e.g. likely) are ambiguous and interpreted by different people in widely different ways (see Sections 4.1.4.3 and 4.1.4.4) Reasoning: To be understood, a meaningful and unambiguous metric is needed for expressing the magnitude of uncertainty. This requirement is met by probability but not by other quantitative or qualitative means of expressing uncertainty. See EFSA (2018b) for further discussion and evidence on these points
3.1.3 Describing uncertainty with numbers	1	Evidence: Multiple studies show that interpretation is more variable for verbal probabilities than numeric probabilities (see Sections 4.1.4.3 and 4.1.4.4)
	2	Reasoning: Because both probabilities and percentages can be used to express other things as well as uncertainty, there is potential for misunderstanding. To avoid this, it was proposed in the public consultation draft of this guidance to refer to ‘% certainty’. This also helps to indicate the subjective nature of the probability and provides a framing in terms of certainty rather than uncertainty, which assessors and communicators may prefer. This approach could be misleading for low probabilities (5% certain sounds low but is actually 95% certainty of non‐occurrence), so it is recommended to communicate % certainty for the outcome that EFSA considers more likely (e.g. 10% probability of X would be communicated as 90% certainty of X not occurring). Some comments received in the public consultation suggested that varied framing (% certainty for some outcomes occurring, and for others not occurring) might lead to misunderstanding, and it is unclear whether recipients would understand % certainty as equivalent to the corresponding probability. On balance, it is judged preferable to retain the use of % certainty due to the advantages stated above, and to address the issues raised by the commenters by a) ensuring the outcome each % certainty refers to is clearly stated, and b) ensuring that it is clearly explained in FAQs linked to the communication that % certainty is to be interpreted as % probability. The use of % certainty in communication has not been tested in any existing study, so it is recommended to evaluate it by monitoring its effects in practice and through appropriate research studies, and to revise the guidance if this appears necessary There is a body of literature that recommends communicating with frequencies rather than probabilities (e.g. Gigerenzer, 1994). However, many of the studies supporting this involved tasks which required subjects to make inferences involving combination of conditional probabilities and base rates. EFSA assessments and communications generally provide direct estimates of the outcomes of interest to the audience, and hence do not require them to deduce these from conditional probabilities. Also, applying EFSA's guidance on uncertainty analysis results in subjective probabilities expressing uncertainty about single outcomes or events and it is unclear whether these would be better expressed using frequencies. Gigerenzer and Hoffrage (1995) have argued that they would, but a study by Joslyn and Nichols (2009) in which the tasks did not require conditional inferences found that subjects understood weather forecast information better when expressed using probability than frequencies. Furthermore, expressing single‐event probabilities in terms of frequency requires framing each assessment in terms of a hypothetical population of similar cases (e.g. other chemical risks with similar evidence and risk to the one under assessment), which increases the complexity of the communication and therefore poses different challenges to understanding. The Working Group therefore decided to retain the recommendation to communicate subjective probabilities as % certainty, but recommends that this be reviewed when further research on the issue is available
	3	One idea emerging from the literature was that it might be desirable to present probabilities both for occurrence and for non‐occurrence of an event, to avoid audience understanding or perception being biased by the framing An obvious strategy to reduce bias would be to communicate both positive and negative framings. However, in the EFSA focus group study (summarised in the draft guidance) where this was done in a simple way (there is a P% probability of X occurring, and a 100 − P% probability of X not occurring), subjects said they considered this form of communication to be unnecessarily complex We therefore propose a layered strategy, where only a positive framing (the outcome EFSA considers more likely) is used but, at the informed level, this is accompanied by concise information on reasons why both the positive and negative outcomes have some probability. Taking the more likely outcome as positive framing is necessary to avoid the problem of expressing low probabilities as % certainty, and also has some support from the finding of Teigen and Brun (2003) that positive and negative frames were induced by complementary ways of presenting numeric chances. We are unaware of any evidence on whether presenting reasons for both positive and negative outcomes at informed level would reduce framing bias or affect communication in other ways, so this should be subject to future review of its performance in practice, based on appropriate studies
	4	Evidence: There is evidence that when both verbal and numeric probabilities are presented together, interpretation is less variable when the numeric probability is presented first (Jenkins et al., 2018)
	5	Reasoning for indicating a probability: When a range is given on its own, the probability for it is ambiguous and could in principle be anything, although it is usually implicit that it is at least possible that the value lies in the range (probability greater than zero). To avoid ambiguity, a probability must be indicated Evidence and reasoning for indicating which values within the range are more likely: Several studies provide evidence that, when uncertainty is represented by a range, people tend to focus on one or other end of the range (Viscusi et al., 1991; Johnson and Slovic, 1998; Dieckmann et al., 2015, 2017). It seems likely that this results not from misunderstanding of the range (which is a primary concern in this guidance) but from subsequent interpretation and value judgements by the recipient (e.g. choosing to be precautionary), which are outside EFSA's remit. Nevertheless, it seems prudent to communicate both a central estimate and range, and to distinguish clearly between these, because this indicates which part of the range is most likely (except in rare cases, when the underlying distribution is multimodal) and provides more understanding for the recipients
	6	Reasoning: The term ‘best’ has a normative content, as it embeds already a preference for a value, whereas ‘central’ just communicates the place of that value in a set of relevant values
	7	Evidence: Several studies have shown that people perform poorly at inferring central estimates from graphical representations of distributions (e.g. Ibrekk and Morgan, 1987)
	8	Reasoning: It is essential to provide explanation and background on the concept of uncertainty and the methods used for expressing it, and of any unavoidable technical terms. Providing this in the form of general FAQs on EFSA's website avoids repeating the same information in every communication, which would be impractical and result in significant risk of inconsistencies between communications
3.1.4 Precautions when using numbers	1	Evidence: Hedging is understood in different ways by different users, with important inter‐individual variation and overlap. Furthermore, their use is modulated by context, including the magnitude of the number, its roundness and even its units (Ferson et al., 2015)
	2	Reasoning: The potential for this type of misunderstanding was raised in a comment received from the public consultation. Careful wording is the only practical option for countering it
	3	Reasoning: Referring to % certainty about an outcome that is expressed in % (e.g. % of people affected) may cause confusion for some readers
	4	Reasoning and evidence: The risk of this type of misinterpretation in the example of hazard has been raised by EFSA experts and others on several occasions and instances of it were observed during the trial period for EFSA's Uncertainty Analysis GD
	5	Reasoning: most people are more familiar with the use of ranges and graphs to communicate variability, so may not recognise when they are used for uncertainty unless this is communicated clearly
3.1.5 Describing sources of uncertainty	1	Evidence: Many publications emphasise the importance of specifying the sources of uncertainty and/or describing their nature (Budescu et al., 2009; also based on Kuhn, 2000; Maxim et al., 2012; Maxim and Mansier, 2014; Visschers, 2018; Wardekker et al., 2008). Also, people's response to uncertainty depends in part on what caused the uncertainty (Frewer et al., 2002; Miles and Frewer, 2011)
	2	Reasoning: assessors and risk managers have in the past raised concerns that communication of conclusions might be undermined if accompanied by description of sources of uncertainty, so it needs to be communicated clearly that conclusions take the uncertainties into account
3.1.6 Addressing the uncertainties	1 and 2	Evidence and reasoning: Frewer et al. (2002) found that uncertainty associated with the scientific process was more readily accepted than uncertainty due to lack of action of the government. This suggests that communication of uncertainty is less likely to cause public alarm if it is accompanied by information on what actions are being taken by the relevant authorities to address that uncertainty. While such actions are risk management measures and are therefore outside the remit of EFSA, as is current practice, such communication may take place if carried out in coordination with decision‐makers (e.g. the European Commission)
	3	Evidence and reasoning: Frewer et al. (2002) found that uncertainty associated with the scientific process was more readily accepted than uncertainty due to lack of action of the government. Investigations aimed at reducing uncertainty are one type of action that can be taken. However, it is important to make distinguish actions that have already been decided (e.g. in areas in which EFSA has the authority to set data requirements) and those that are options for risk managers to consider
3.1.7 Guidance for assessors	1	Reasoning: This is necessary to ensure communicators can interpret correctly what the scientific conclusion refers to and convey this correctly in their communications
	2	Reasoning: this is a generally accepted principle in science, to avoid implying more precision than is justified
	3	Evidence: Multiple studies show that interpretation is more variable for verbal probabilities than numeric probabilities (see Sections 4.1.4.3 and 4.1.4.4). Some of these show that, when both verbal and numeric probabilities are presented together, interpretation is less variable when the numeric probability is presented first (e.g. Jenkins et al., 2018)
	4	Evidence and reasoning: the findings of Gigerenzer and Hoffrage (1995), Gigerenzer et al. (2007), and other publications) show that people (including doctors and people with statistical training) perform very poorly at deducing probabilities of interest from conditional probability information and perform much better if provided with information in the form of frequencies. It is better to avoid the problem altogether by providing frequencies for the outcome of interest directly and not requiring readers to make deductions from conditional information
	5	See Evidence and reasoning for 3.1.3 point 5 (above)
	6	Evidence: when receiving a message such as ‘more than X’, people tend to infer extreme values (Teigen (2008), Hohle and Teigen (2018), Hoorens and Bruckmüller (2015))
	7	Reasoning: This is desirable because it requires assessors to draft an expression of the uncertainty that is concise enough to serve as a starting point for crafting communications, without the communicators having to interpret it (and potentially misinterpret it) from more detailed technical material. It also makes the information more accessible to the communicators
Box  1	Reasoning: Standardised procedures usually have standard language for expressing their conclusions (e.g. no concern), which should be used also in communications. If there is a requirement for conclusions to be unqualified, then this should be respected by both assessors and communicators Reasoning: If there is not a requirement for the conclusion to be unqualified then any information about uncertainty that is provided by the assessors may optionally be included in the communication, following the recommendations applicable to the forms of expression involved
Box  2	Evidence: Many publications emphasise the importance of specifying the sources of uncertainty and/or describing their nature (Budescu et al., 2009; also based on Kuhn, 2000; Maxim et al., 2012; Maxim and Mansier, 2014; Visschers, 2018; Wardekker et al., 2008) Also, people's response to uncertainty depends in part on what caused the uncertainty (Frewer et al., 2002; Miles and Frewer, 2011) Reasoning: The relative importance of uncertainties is important information for targeting further analysis or research, when needed. Conflicting evidence is an important source of uncertainty (as stated in a comment to the public consultation) but is not explicitly listed in EFSA Scientific Committee (2018a,b). The opportunity is therefore taken to highlight it here and refer assessors to EFSA's guidance on weight of evidence assessment, which addresses it directly
Box  3	Reasoning: This form of expression is discouraged by the Uncertainty Analysis GD due to its ambiguity. If the assessors have nevertheless used such expressions, then it is important that the communicators avoid rewording them in ways that the assessors would consider incompatible with their judgement of the uncertainty It seems likely that the informed‐level audience would like to receive some description of the uncertainties that contribute to the qualitatively described magnitude, and how they were assessed. Various qualitative scales have been proposed for expressing uncertainty, confidence or the strength and quality of evidence, but the Scientific Committee is not aware that any of these have a formal operational definition of the type that exists for probability itself (see section 5.10 of EFSA Scientific Committee, 2018b). Therefore, qualitative scales and expressions are not recommended in EFSA Scientific Committee (2018a) or the present guidance for expressing the overall uncertainty of assessment conclusions. However, they can be useful for expressing individual sources of uncertainty and in weight of evidence assessment (EFSA Scientific Committee, 2017), provided the overall uncertainty of the conclusion is expressed quantitatively when possible Evidence: Findings from EFSA studies support the use of general qualitative statements on the direction of the uncertainties in communication aimed at the informed‐level and technical‐level audiences. The +/− table currently used in some EFSA assessments is considered helpful by some users, but limited or misleading by others including some users with high scientific literacy. Providing quantitative definitions will reduce ambiguity when such scales are used
Box  4	Reasoning: If the assessors were unable to make any conclusion then it is essential that the communication should reflect this clearly and not use any language that implies more is known When there is this much uncertainty, it seems prudent to give some indication of the reasons for it at entry level as well as informed level. At the informed level it seems desirable to mention also any options that have been identified for reducing uncertainty
Box  5	Most unqualified conclusions follow a standard wording that has special meaning and should therefore be retained in all communications. For transparency, the special meaning should be made explicit at the informed level and/or in a linked FAQ. To provide this, actions are needed by assessors and risk managers at the technical level It seems likely that the informed‐level audience may want to have access to a concise plain language explanation of how the conclusion was reached (evidence and methods), and of the types of uncertainty that are present, and to know that the latter were taken into account in reaching the conclusion
Box  6	Reasoning: For entry level, see reasoning for general guidance It seems likely that the informed‐level audience may want to have access to a concise plain language explanation of how the conclusion was reached (evidence and methods), and of the types of uncertainty that are present; to know that the latter were taken into account in reaching the conclusion; and to have a plain language outline of how that was carried out
Box  7	Evidence: Many studies have shown that using verbal expressions to communicate approximate probabilities results in widely varying interpretations, and that this is reduced when the numeric range is presented alongside the verbal expression (Patt and Schrag, 2003; Patt and Dessai, 2005; Budescu et al., 2009; Harris and Corner, 2011; Budescu et al., 2012, 2014; also based on Wallsten et al., 1986; Mosteller and Youtz, 1990) Reasoning: It is recommended to communicate the quantitative range of probabilities, because this expresses the assessors’ conclusion without ambiguity. Although other options may be simpler (e.g. replacing 66‐90% with ‘likely’ or ‘about 80%’), and there was some indication from the EFSA studies that more subjects preferred them, they will be interpreted by different people in different ways. If a simpler option is used at the entry level, it is essential that the assessors’ range of probabilities is provided at the informed level, so that the meaning is clear to those who read both Reasoning: It seems likely that the informed‐level audience may also want to have access to a concise plain language explanation of the basis for the approximate probability (evidence and methods), and of the types of uncertainty that were taken into account
Box  8	Evidence and reasoning: Some studies in the literature review compare alternative means for visual representations of probability distributions. In summary, box plots appear to be best understood, although they only provide partial information about distribution shape. CDFs and PDFs provide more information but will be badly misinterpreted by some people. Therefore it is proposed to use only box plots in communications except at the technical level, in which the box plot may be accompanied by a PDF and/or CDF if this is useful to communicate more information about distribution shape (PDF) or enable readers to read off additional quantiles (CDF); in all cases all graphics should be well labelled and the key results should be marked on them Experience at EFSA has shown that it is important to make clear when box plots and PDFs are used to communicate uncertainty rather than variability, as people are more familiar with the latter use There is evidence that the choice of quantiles to report will influence how people respond (their perception of the uncertainty) but it is unclear whether this implies a failure to understand what is communicated or subsequent interpretation by the recipient. There is a trade‐off between giving excessive weight to extreme values and understating the uncertainty. Also, it is unlikely to be prudent for EFSA to limit reporting to ranges that are expected not to contain the outcome in a substantial proportion of cases. Therefore, it is proposed in general to communicate the median together with both the 50% and 95% range (the latter because this is conventionally used in science). In addition to this, results for specific values or quantiles of interest when these are known
Box  9	Reasoning: None of the selected studies examined the communication of 2D probability distributions, but it is expected that they will be less well understood than 1D distributions. Therefore it is proposed to focus communication for entry and informed levels on selected results extracted from the 2D distribution, and communicate quantiles for these numerically at entry level with the addition of box plots at informed level 2D distributions should be used only at the technical level, and then with great care and explanation