Pilot study on uncertainty analysis in EFSA Reasoned Opinions on the modification of pesticide maximum residue levels

Abstract

Applicability of the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment is tested in the context of an EFSA Reasoned Opinion on the modification of pesticide maximum residue levels (MRLs). EFSA purchased services for the preparation of a non‐regulatory Evaluation Report with example non‐standard uncertainties related to a fictitious application for the modification of MRLs. The Evaluation Report was assessed by EFSA in the format of a Reasoned Opinion and case‐specific examples of non‐standard uncertainty in the acute and chronic dietary risk assessments were analysed. Methods were selected from the general framework outlined in the Scientific Committee draft Guidance in order to apply a relatively simple strategy that could be considered for use in a regulatory context. The individual non‐standard uncertainties were assessed by sensitivity analysis with iterative back‐calculation of the parameter values that would lead to exceedance of the toxicological reference value (exceedance limit calculation), and quantified by subjective probability estimation. Non‐standard uncertainties affecting the chronic risk assessment were quantified by subjective upper bound probability percentile estimation and the combined estimated non‐standard uncertainty calculated by probability bounds analysis. Probability bounds analysis provides a relatively simple approach for calculating the probability related to a combination of uncertainties. The draft Guidance was found to provide a comprehensive range of methods for uncertainty analysis. However, process‐specific guidelines and practical procedures may need to be developed in order to implement the uncertainty assessment framework in routine pesticide risk assessments. The uncertainty assessment is intended to provide additional information on how certain the conclusions of the risk assessment are and thereby support the risk‐based decision‐making process by enabling risk managers to take account of uncertainty. The outcome of the pilot study will inform the EFSA Scientific Committee Working Group on how to further tailor the draft Guidance on uncertainty for the needs of the EFSA panels and units.

Summary

Food safety risk assessment is subject to many sources of uncertainty and those uncertainties that have an impact on the risk assessment should be explicitly considered at each step in the risk assessment and documented in a transparent manner. The EFSA Science Strategy for the period 2012–2016 recognises the importance of characterising the uncertainties underlying EFSA's risk assessments, and of communicating these uncertainties and their potential impact on the decisions in a transparent manner.

In February 2016, the EFSA Scientific Committee endorsed revised draft Guidance on uncertainty in EFSA scientific assessment for internal testing for a one‐year trial phase. All EFSA Panels and Units that produce scientific outputs were requested to apply the Scientific Committee draft Guidance to at least one assessment during the trial period, involving relevant decision‐makers and supported by specialists in uncertainty analysis where needed.

For the purposes of testing the Scientific Committee draft Guidance in the area of pesticide risk assessment, the assessment of the risks to the consumer associated with a fictitious (and thus non‐regulatory) application for the modification of the European Union (EU) maximum residue levels (MRLs) in the format of an EFSA Reasoned Opinion was selected to pilot uncertainty analysis.

The pesticide risk assessment methodology that is applied in the standard procedure for the assessment of MRL applications has been agreed by risk managers with an accepted provision for uncertainty. The risk assessment methodology and the data requirements contain standard elements that are accepted by risk managers as providing adequate cover for uncertainty, such as uncertainty factors, default values and conservative assumptions (standard uncertainties). The assessment of MRL applications may also contain non‐standard case‐specific sources of uncertainty in situations where there is no standard procedure for the type of assessment or there may be sources of uncertainty that are not included, or not adequately covered, by the standard procedure (non‐standard uncertainties).

It was decided to work with a fictitious example pesticide active substance in order to differentiate the pilot study from regulatory assessments. The European Food Safety Authority (EFSA) purchased services for the preparation of an example non‐regulatory Evaluation Report related to the modification of MRLs for a fictitious pesticide active substance. The Netherlands Board for the Authorisation of Plant Protection Products and Biocides (Ctgb) generated and summarised the data that would be submitted by the (fictional) applicant.

In the pilot study, it was postulated that, in accordance with Article 6 of Regulation (EC) No 396/2005, a fictional applicant submitted an application to a competent national authority (evaluating Member State (EMS)) to set new MRLs for the fictitious active substance MAS in potatoes, tomatoes and cauliflower. The Ctgb, acting as the EMS, drafted an Evaluation Report in accordance with Article 8 of Regulation (EC) No 396/2005, which was submitted to EFSA on 13 December 2016. To accommodate for the intended uses of MAS, the EMS proposed to raise the existing MRL for the intended use in tomatoes from the limit of quantification (LOQ) to 0.60 mg/kg, based on the data set for cherry tomatoes only. The EMS did not propose MRLs for the intended uses in potatoes and cauliflower.

EFSA based its assessment on the Evaluation Report submitted by the EMS. In a real‐case regulatory Reasoned Opinion, EFSA would also base its assessment on the draft assessment report (DAR) (and its addenda) prepared under either Council Directive 91/414/EEC or Regulation (EC) 1107/2009, as well as the Commission review report on the active substance, the EFSA Conclusion on the peer review of the pesticide risk assessment of the active substance and, where available, the Joint FAO/WHO Meeting on Pesticide Residues (JMPR) Evaluation Report, as well as previous EFSA opinions on the active substance. However, since the current assessment concerns a fictitious active substance, these reports are not available, and EFSA used the Evaluation Report as the basis of the assessment.

In the Evaluation Report, the EMS reported that the metabolism of MAS following foliar application was investigated in crops belonging to the fruit (apple), leafy (lettuce) and cereals/grass (wheat) crop groups, and reviewed in the framework of the (fictitious) EU pesticides peer, and that for the current application a new metabolism study in stored potatoes was submitted for evaluation. The potato metabolism study revealed that metabolite ATO is present in stored potatoes at a significant level.

Studies investigating the effect of processing on the nature of MAS and metabolite ATO (hydrolysis studies) demonstrated that the active substance and its metabolite are stable.

Based on the metabolic pattern identified in the metabolism studies and hydrolysis studies, the residue definitions for plant products were proposed as MAS (only) for enforcement and risk assessment (applicable to primary crops, rotational crops and processed products; with the exception of the risk assessment of stored potatoes). EFSA concluded that for the crops assessed, in this application, metabolism of MAS in primary and in rotational crops and the possible degradation in processed products has been sufficiently addressed and that the previously derived residue definitions are applicable, except for stored potatoes. The residue definition for risk assessment of stored potatoes was proposed as: the sum of substance MAS and metabolite ATO, expressed as substance MAS.

Sufficiently validated analytical methods are reported in the Evaluation Report to be available to quantify residues in the crops assessed in this application according to the enforcement residue definition. The methods enable quantification of residues at or above 0.01 mg/kg in the crops assessed (LOQ).

The available residue trials data are sufficient to calculate MRL proposals of 0.50, 0.50 and 1.50 mg/kg for potatoes, tomatoes and cauliflower, respectively.

Processing factors for the crops under assessment were derived from processing studies provided and that would be recommended to be included in Annex VI of Regulation (EC) No 396/2005 are as follows:

– potatoes/flakes: 1.02 (MAS)	– potatoes/microwaving: 0.99 (MAS)
– potatoes/flakes: 1.05 (ATO)	– potatoes/microwaving: 1.01 (ATO)
– potatoes/chips: 1.02 (MAS)	– tomatoes/puree: 0.52
– potatoes/chips: 1.05 (AOT)	– tomatoes/juice: 0.34
– potatoes/fried: 1.02 (MAS)	– tomatoes/paste: 0.39
– potatoes/fried: 1.05 (ATO)	– cauliflower/boiling: 0.49

The occurrence of MAS residues in rotational crops was reported in the Evaluation Report to have been investigated in the framework of the EU pesticides peer review. Based on the available information on the nature and magnitude of residues, it was concluded that significant residue levels are unlikely to occur in rotational crops, provided that the active substance is used according to the intended good agricultural practice (GAP).

As the crops under consideration and their by‐products are used as feed products, a potential carry‐over into food of animal origin was assessed. The calculated livestock dietary burden exceeded the trigger value of 0.004 mg/kg body weight (bw) for all relevant animal species. Therefore, the possible occurrence of MAS residues in commodities of animal origin was investigated. The nature of MAS residues in livestock was reported in the Evaluation Report to have been investigated during the fictitious EU pesticides peer review of MAS and the residue definitions for enforcement and risk assessment were proposed as metabolite ATO, expressed as substance MAS. Based on the results of metabolism studies the MRLs, HRs and STMRs for residues in products of animal origin are expected to be at or below the LOQ, and therefore, MRLs were not proposed for products of animal origin in the present study.

The toxicological profile of MAS was reported in the Evaluation Report to have been assessed in the framework of the EU pesticides peer review under Regulation (EC) No 1107/2009 and the data were sufficient to derive an acceptable daily intake (ADI) of 0.0011 mg/kg bw per day and an acute reference dose (ARfD) of 0.045 mg/kg bw. The metabolite ATO, which is included in the residue definition for risk assessment of stored potatoes, is of similar toxicity as the parent active substance.

The consumer risk assessment was performed with revision 2 of the EFSA Pesticide Residues Intake Model (PRIMo).

The data submitted in support of this fictitious MRL application were found to be sufficient to calculate MRLs for all crops under consideration. Adequate analytical methods for enforcement are available to control the residues of MAS in the plant matrices and commodities under consideration.

Based on the risk assessment results for the intended use of MAS on cauliflower, EFSA concluded that according to the internationally agreed methodology, the expected short‐term exposure exceeds the toxicological reference value (international estimated short‐term intake (IESTI) 1 is 104.3% ARfD) and a risk to consumer health from short‐term intake cannot be excluded for the intended use of MAS on cauliflower. However, it was judged as very likely (90–99% subjective probability) that the worst‐case refined IESTI would not exceed the toxicological reference value because a proportion of the population mean consumption of the large portion of cauliflower would be cooked. This is a non‐standard refinement of the risk assessment for illustrative purposes.

For the intended use on potatoes, EFSA concluded that the expected short‐term exposure according to the internationally agreed methodology was below the toxicological reference value and therefore an acute consumer intake exceedance was not identified; however, the safety margin for acute exposure as regards MAS residues on potatoes is very narrow (IESTI 1 is 98.4% of the ARfD) and the risk assessment is affected by non‐standard uncertainty regarding the conversion factor (CF_Risk) derived from metabolism study data. EFSA conducted a sensitivity analysis and uncertainty analysis to evaluate the impact of the non‐standard uncertainty on the estimated short‐term exposure. On this basis, taking account of the overall non‐standard uncertainty, it was judged as as‐likely‐as‐not (33–66% subjective probability) that the short‐term exposure (IESTI 1) would exceed the toxicological reference value (ARfD).

For the intended use on tomatoes, the expected short‐term exposure was below the toxicological reference value (IESTI 1 is 36.2% of the ARfD) and is affected by a minor non‐standard uncertainty that was judged to have no material impact on the risk assessment.

The estimated long‐term consumer exposure was below the toxicological reference value, however, the safety margin for chronic exposure is narrow (highest international estimated daily intake (IEDI) 92.2% of the ADI) and the risk assessment was affected by non‐standard uncertainties affecting the contributions of potatoes and cauliflower. Non‐standard uncertainty involved for the residues trials in tomatoes was judged to have no material impact on the chronic risk assessment. The individually quantifiable non‐standard uncertainties affecting the long‐term consumer exposure estimate were estimated and combined by probability bounds analysis. From this analysis, taking account of the overall non‐standard uncertainty affecting the chronic risk assessment it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value.

The present pilot study tests the applicability of methods for uncertainty analysis to non‐standard uncertainties in the acute and chronic consumer risk assessments performed in EFSA Reasoned Opinions on the modification of pesticide MRLs. The methods for the uncertainty analyses were selected from the ‘toolbox of methods’ provided in the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment in order to apply a relatively simple strategy to analyse non‐standard uncertainties that could be considered for use in routine pesticide risk assessments such as for the setting or modification of pesticide MRLs.

The individual non‐standard uncertainties were assessed by sensitivity analysis with iterative back‐calculation of the parameter values that would lead to exceedance of the toxicological reference value (exceedance limit calculation), and quantified by subjective probability estimation. The methods for sensitivity analysis were found to be readily applicable. However, quantification of the uncertainty by subjective probability estimation may be one of the more challenging areas for implementation of the uncertainty assessment framework in the context of pesticide risk assessment. The present pilot study is based on a fictitious data set and therefore formal expert knowledge elicitation was not applied to provide probability judgements. The probability judgements in a real‐case assessment must be based on reasoned consideration of the available evidence, which might not be measured data, but could be other forms of evidence including expert knowledge and experience and the process must be documented in order to meet the required level of transparency.

The non‐standard uncertainties affecting the chronic risk assessment were additionally quantified by subjective upper bound probability percentile estimation and the combined estimated non‐standard uncertainty calculated by probability bounds analysis. The probability bounds analysis method was found to provide a relatively simple approach for calculating the probability related to a combination of uncertainties.

The draft Guidance was found to provide a comprehensive range of methods from which applicable methods could be selected for each step in the pilot study uncertainty assessment. However, process‐specific guidelines and practical procedures may need to be developed in order to implement the uncertainty assessment framework in routine pesticide risk assessments and to ensure consistent identification and analysis of non‐standard uncertainties.

Pesticide risk assessments may be affected by more than one source of non‐standard uncertainty and an assessment strategy that is suitable for case‐specific combinations of non‐standard uncertainties would be required. The level of complexity and extent of the uncertainty analyses should be fit‐for‐purpose, taking into consideration the available time and resources. Simpler methods of uncertainty analysis may be appropriate where they provide sufficient information for decision‐making. The uncertainty assessment is intended to provide additional information on how certain the conclusions of the risk assessment are and thereby support the risk‐based decision‐making process by enabling risk managers to take account of uncertainty.

The outcome of the pilot study will inform the EFSA Scientific Committee Working Group on how to further tailor the draft Guidance on uncertainty for the needs of the EFSA panels and units.

The fictional recommended MRLs are summarised in the table below.

Codea	Commodity	Existing EU MRL (mg/kg)	Recommended EU MRL (mg/kg)	Comment/justification and non‐standard uncertainties
Enforcement residue definition: MAS (only)
211000	Potatoes	0.01*	0.50	The submitted data are sufficient to derive a MRL proposal for the post‐harvest treatment use in potatoes The estimated short‐term exposure for the intended use in potatoes (IESTI 1) is 98.4% of the toxicological reference value (ARfD) and therefore an acute consumer intake exceedance was not identified; however, the safety margin for acute exposure as regards MAS residues on potatoes is very narrow and the risk assessment is affected by non‐standard uncertainty regarding the CFRisk derived from metabolism study data. Taking account of the non‐standard uncertainty affecting the acute risk assessment it was judged as as‐likely‐as‐not (33–66% subjective probability) that the short‐term exposure (IESTI 1) would exceed the toxicological reference value (ARfD) The estimated long‐term exposure from the intended use in potatoes contributed to up to 83.6% of the ADI (highest theoretical maximum daily intake: 92.2% of ADI for NL child) and the risk assessment was affected by non‐standard uncertainties. Taking account of the overall non‐standard uncertainty affecting the chronic risk assessment, it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value
231010	Tomatoes	0.01*	0.50	The residues trials on tomato were involved with a non‐standard uncertainty which was judged to have no material impact on the trials data. The submitted data are considered sufficient to derive a MRL proposal for the intended use on tomatoes from the indoor supervised residues trials The estimated short‐term exposure for the intended use in tomatoes is below the toxicological reference value (IESTI 1 is 36.2% ARfD) and is affected by a minor non‐standard uncertainty that was judged to have no material impact on the risk assessment The estimated long‐term exposure from the intended use in tomatoes contributed to up to 7.4% of the ADI (highest theoretical maximum daily intake: 92.2% of ADI for NL child) and the risk assessment was affected by non‐standard uncertainties. Taking account of the overall non‐standard uncertainty affecting the chronic risk assessment, it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value
241020	Cauliflower	0.01*	Further risk management considerations required	The submitted data are sufficient to calculate a MRL of 1.50 mg/kg for the intended use in cauliflower The estimated short‐term exposure (IESTI 1) from the intended use in cauliflower exceeded the toxicological reference value (104.3% of the ARfD) and consequently a risk to consumer health could not be excluded for the intended use of MAS on cauliflower. However, it was judged as very likely (90–99% subjective probability) that the worst‐case refined IESTI would not exceed the toxicological reference value because a proportion of the population mean consumption of the large portion of cauliflower would be cooked The estimated long‐term exposure from the intended use in cauliflower contributed to up to 1.2% of the ADI (highest TMDI: 92.2% of ADI for NL child) and the risk assessment was affected by non‐standard uncertainties. Taking account of the overall non‐standard uncertainty affecting the chronic risk assessment, it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value

MRL: maximum residue level; IESTI: international estimated daily intake; ARfD: acute reference dose; CF_risk: conversion factor for enforcement to risk assessment residue definition; ADI: acceptable daily intake; TMDI: theoretical maximum daily intake.

* Indicates that the MRL is set at the limit of analytical quantification (LOQ).

^aCommodity code number according to Annex I of Regulation (EC) No 396/2005.

Introduction

Food safety risk assessment is subject to many sources of uncertainty and those uncertainties that have an impact on the risk assessment should be explicitly considered at each step in the risk assessment and documented in a transparent manner (Codex Alimentarius Commission, 2007). The Codex Alimentarius of the Joint FAO/WHO Food Standards Programme specifies that ‘the report of a risk assessment should indicate any constraints, uncertainties, assumptions and their impact on the risk assessment’. Furthermore, the risk communication should include a transparent explanation of the risk assessment policy and of the assessment of risk, including the uncertainty.

Regulation (EC) No 178/2002 of the European Parliament and of the Council laying down the general principles and requirements of food law, establishing the European Food Safety Authority (EFSA) and laying down procedures in matters of food safety,1 refers to uncertainty assessment in specific circumstances where the possibility of harmful effects on health is identified but scientific uncertainty persists following an assessment of available information. The Regulation sets out the obligations to clarify the contentious scientific issues and identify the relevant uncertainties in the data in specific circumstances where a substantive divergence over scientific issues has been identified.

The EFSA Science Strategy for the period 2012–2016 recognised the importance of characterising the uncertainties underlying EFSA's risk assessments, and of communicating these uncertainties and their potential impact on the decisions in a transparent manner (EFSA, 2012). The first key objective of the Science Strategy is to further develop excellence of EFSA's scientific advice, which encompasses a core EFSA value of transparency; that is to ensure that the processes and the basis for EFSA's opinions are documented and understood. Demonstrating how data provided to EFSA are used and managed, as well as the mechanisms by which an opinion is developed and scientific consensus is reached, was identified in the Science Strategy as an issue where EFSA still needed to develop further, including the documentation of the underlying uncertainties and their potential impact on the decisions to be made. Another key objective of the Science Strategy is to develop and harmonise methodologies and approaches to assess risks associated with the food chain, which identified the need to work towards improvement and harmonisation of risk assessment terminology, including for addressing uncertainties and expressing these with transparency and relevance.

In 2006, the EFSA Scientific Committee adopted a first Guidance to address scientific uncertainties in dietary exposure assessment, which recommended a tiered approach (qualitative, deterministic or probabilistic) to progressively refine the characterisation of uncertainty and provide an increasingly clear picture of the likelihood of exceeding (or in the case of nutrients, failure to attain) a health‐based guidance value (EFSA, 2007a). The Guidance identified the communication of uncertainty as a key challenge, with the aim to provide a balanced picture of what is known and what is uncertain, and avoid giving an exaggerated impression of either certainty or uncertainty. The 2006 Scientific Committee Guidance strongly encouraged the EFSA Scientific Panels to incorporate the systematic evaluation of uncertainties in their risk assessments and to communicate this clearly in their opinions.

1.

1.1.

Pesticides risk assessment and uncertainty

In the area of pesticides, Regulation (EC) No 1107/20092, concerning the placing of plant protection products on the market, is underpinned by the precautionary principle in order to ensure that active substances or products placed on the market do not adversely affect human or animal health or the environment. In particular, this regulation establishes that Member States shall not be prevented from applying the precautionary principle where there is scientific uncertainty as to the risks with regard to human or animal health or the environment. The procedure and criteria for the approval of active substances include a specific requirement in the area of ecotoxicology for the assessment to take into account the uncertainty of data.

The general principles for the evaluation and authorisation of plant protection products (so called ‘Uniform Principles’) implemented by Commission Regulation (EU) No 546/20113 require Member States to interpret the results of evaluations taking into consideration possible elements of uncertainty in the information obtained during the evaluation, in order to ensure that the chances of failing to detect adverse effects or of underestimating their importance are reduced to a minimum. This regulation requires the decision‐making process to be examined to identify critical decision points or items of data for which uncertainties could lead to a false classification of risk.

EFSA has previously applied a qualitative estimation of uncertainties in a Reasoned Opinion on the potential chronic and acute risk to consumers’ health arising from proposed temporary EU MRLs (EFSA, 2007b). The Reasoned Opinion identified the major sources of uncertainties and estimated the directions and magnitudes of influence of uncertainties for both the chronic and acute risk assessments.

The EFSA Panel on Plant Protection Products and their Residues (PPR) has characterised the uncertainties affecting the cumulative risk assessment for a selected group of pesticides from the triazole group (EFSA PPR Panel, 2009). The Opinion characterised the uncertainties (quantified and unquantified), conducted sensitivity analyses, evaluated the unquantified uncertainties, and provided an overall assessment of the influence of uncertainties.

In a 2012 Opinion, the EFSA PPR evaluated the qualitative influence of uncertainties on assessing the toxicological relevance of pesticide metabolites for dietary risk assessment (EFSA PPR Panel, 2012). The Opinion considered the applicability of safety factors and where a conservative approach is not assured, the qualitative evaluation of the uncertainties affecting the approaches (uncertainty assessment) could be used to identify the critical areas that need further refinement.

EFSA Scientific Committee draft Guidance

In 2013, EFSA agreed a self‐tasking mandate of the EFSA Scientific Committee to develop guidance on how to characterise, document and explain uncertainties in risk assessment. The guidance should cover uncertainties related to the various steps of the risk assessment, i.e. hazard identification and characterisation, exposure assessment and risk characterisation. A working group was requested to aim as far as possible at developing a harmonised framework applicable to all relevant working areas of EFSA. The EFSA Scientific Committee was requested to demonstrate the applicability of the proposed framework with case studies.

The EFSA Scientific Committee developed documents to introduce uncertainty analysis to general risk assessment approaches across the work of EFSA (e.g. guidance on transparency and uncertainty). In 2015, the EFSA Scientific Committee issued draft Guidance on uncertainty in EFSA scientific assessment, which reviews the general applicability of principles and approaches to EFSA's work, in order to establish a general framework for addressing uncertainty. The aim was to guide EFSA Panels and staff on how to deal with sources of uncertainty in scientific assessments and to provide a toolbox of methods from which assessors can select those methods which most appropriately fit the purpose of their individual assessment. EFSA conducted a public consultation on the draft Guidance on uncertainty in EFSA scientific assessment in June–September 2015, and the draft Guidance was subsequently revised based on the comments received.

The EFSA Scientific Committee endorsed the revised draft Guidance on uncertainty in EFSA scientific assessment in February 2016, for internal testing for a one‐year trial phase (EFSA Scientific Committee, 2016). All EFSA Panels and Units that produce scientific outputs were requested to apply the draft Scientific Committee Guidance to at least one assessment during the trial period, involving relevant decision‐makers and supported by specialists in uncertainty analysis where needed. It is intended that when the trial period is completed and any resulting improvements to the guidance document have been agreed, uncertainty analysis will be unconditional for EFSA Panels and staff and must be embedded into scientific assessment in all areas of EFSA's work.

Uncertainty analysis in EFSA Reasoned Opinions

For the purposes of testing applicability of the EFSA Scientific Committee draft Guidance on uncertainty to the area of pesticide risk assessment, the assessment of the risks to the consumer associated with an application for the modification of EU maximum residue levels (MRLs) in the format of an EFSA Reasoned Opinion was selected as a case study to pilot the inclusion of uncertainty analysis.

In the European Union (EU), applications concerning pesticide MRLs are submitted to and evaluated by the competent authorities of the Member States in accordance with Article 8 of Regulation (EC) No 396/20054 (hereinafter referred to as ‘the MRL regulation’). The evaluating Member State (EMS) submits an Evaluation Report to the European Commission and to EFSA. The application, the Evaluation Report and the supporting dossier are subsequently assessed by EFSA in accordance with Article 10 of the MRL Regulation. The EFSA risk assessment is reported in the format of a Reasoned Opinion on, in particular, the risks to the consumer and where relevant to animals associated with the setting of the requested MRLs. The Reasoned Opinion is required to clearly define the basis for each conclusion reached.

The process for the assessment of an application for the modification of the existing MRLs or for setting of import tolerances is a standard risk assessment procedure which is conducted in accordance with the relevant provisions and against the relevant data requirements established in either Regulation (EU) No 544/20115 or Regulation (EU) No 283/20136, and according to the guidance documents that were applicable at the date of submission of the application. The evaluation is performed in accordance with the legal provisions of the Uniform Principles for the Evaluation and the Authorisation of Plant Protection Products adopted by Commission Regulation (EU) No 546/2011.

The pesticide risk assessment methodology that is applied in the standard procedure for the assessment of MRL applications has been agreed by risk managers with an accepted provision for uncertainty. The risk assessment methodology and the data requirements contain standard elements that are accepted by risk managers as providing adequate cover for uncertainty, such as uncertainty factors, default values and conservative assumptions (standard uncertainties). However, currently, there is no guidance available to systematically identify and evaluate the applicable standardised uncertainties in pesticide risk assessment and this may be identified as an area requiring future work.

The assessment of MRL applications may also contain case‐specific sources of non‐standard uncertainty in situations where there is no standard procedure for the type of assessment or there may be sources of uncertainty that are not included, or not adequately covered, by the standard procedure (non‐standard uncertainties). The present scientific report applies uncertainty analysis to case‐specific non‐standard uncertainties in the format of an EFSA Reasoned Opinion on the modification of pesticide MRLs, following the general framework outlined in the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment.

Background to the assessment

The MRL regulation (Regulation (EC) No 396/2005) establishes the rules governing the setting of pesticide MRLs at European Union (EU) level. Article 6 of the MRL regulation lays down that any party having a legitimate interest or requesting an authorisation for the use of a plant protection product in accordance with Council Directive 91/414/EEC7, repealed by Regulation (EC) No 1107/2009, shall submit an application to a Member State to modify a MRL in accordance with the provisions of Article 7 of the MRL regulation.

EFSA decided to pilot the assessment of uncertainties with a fictitious (and thus non‐regulatory) application to modify MRLs for a fictitious pesticide active substance in order to differentiate the pilot study from regulatory assessments. EFSA purchased services for the preparation of a non‐regulatory example Evaluation Report that contains case‐specific non‐standard uncertainties that are not adequately covered by the standard procedure and which require uncertainty analysis, in order to test the applicability of the draft Guidance.

The procurement for the preparation of the non‐regulatory Evaluation Report was included in the EFSA Register of Questions with the reference number EFSA‐Q‐2016‐00662 and the subject: Preparation of an evaluation report related to an Article 10 Reasoned Opinion as case study applying the Scientific Committee draft guidance on uncertainty in EFSA scientific assessment.

The Netherlands Board for the Authorisation of Plant Protection Products and Biocides (Ctgb) generated and summarised the data that would be submitted by the (fictional) applicant. It was agreed that the pilot study would consider three different crop types and include a crop which is consumed regularly in order to include a chronic dietary risk scenario. Furthermore, the data generated to simulate the fictitious application would include scenarios where the dietary exposure is close to or exceeding the health based guidance values (acute reference dose (ARfD) and acceptable daily intake (ADI)).

A fictional applicant was postulated to have submitted an application to a competent national authority, referred to as the EMS, to modify MRLs for the fictitious active substance ‘MAS’ in potatoes, tomatoes and cauliflower. This fictitious application was evaluated by the Ctgb, acting as the EMS, in accordance with Article 8 of the MRL regulation and following the general framework outlined in the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment. The EMS summarised the fictitious data in the Evaluation Report which was submitted to EFSA on 13 December 2016 (Schepens, 2017).

The EMS proposed to raise the MRL of fictitious active substance MAS from the limit of quantification (LOQ) to 0.6 mg/kg in tomatoes, based on the data set for cherry tomatoes only. The EMS did not propose MRLs for the intended uses in potatoes and cauliflower. EFSA assessed the application and Evaluation Report in line with the requirements of Article 10 of the MRL Regulation and following the general framework outlined in the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment.

This scientific report was prepared by the EFSA Pesticides Unit with the support of the EFSA Scientific Committee Working Group on Uncertainty. This pilot study includes the application of new methodology, i.e. uncertainty analyses based on the EFSA Scientific Committee draft guidance, which is not routine, and therefore was submitted to the EFSA PPR in the format of a draft Reasoned Opinion for peer review and possible endorsement. At the plenary meeting held on 23 March 2017, the Panel supported the overall approach to identify and analyse the non‐standard uncertainties and endorsed the draft report subject to finalisation by written procedure of the detailed comments of the Panel (EFSA PPR Panel, 2017).

Terms of Reference

In accordance with Article 10 of MRL regulation (Regulation (EC) No 396/2005), EFSA shall assess applications and the evaluation reports and give a reasoned opinion on the risks to the consumer and where relevant to animals associated with the setting of the requested MRLs. The opinion shall include:

• an assessment of whether the analytical method for routine monitoring proposed in the application is appropriate for the intended control purposes;

• the anticipated LOQ for the pesticide/product combination;

• an assessment of the risks of the ADI and ARfD being exceeded as a result of the modification of the MRL;

• the contribution to the intake due to the residues in the product for which the MRLs was requested;

• any other element relevant to the risk assessment.

In accordance with Article 11 of the MRL regulation, EFSA shall give its reasoned opinions as soon as possible and at the latest within three months from the date of receipt of the application.

The Evaluation Report submitted by the EMS (Schepens, 2017) and the exposure calculations using the EFSA Pesticide Residues Intake Model (PRIMo) are considered as supporting documents to this scientific report and, thus, are made publicly available.

The active substance and its use pattern

The detailed description of the fictitious intended uses of fictitious active substance MAS in potatoes, tomatoes and cauliflower, which are the basis for the current MRL application, is reported in Appendix A.

‘MAS’ is the name for the fictitious active substance in this pilot study. In a real‐case assessment, the chemical structures of the active substance and its main metabolites would be reported in Appendix F.

For the purposes of this study, MAS was postulated to have been evaluated in the framework of Regulation (EC) No 1107/2009 for representative uses as a foliar treatment on three different crop groups. A fictitious draft assessment report (DAR) is postulated to have been prepared by the RMS and to have been peer reviewed by EFSA. The process of renewal of the first approval is assumed to have not yet been initiated.

The fictitious active substance MAS was reported in the Evaluation Report to have been approved for use as a fungicide. The approval was assumed not to impose restrictions relevant for the dietary risk assessment.

The EU MRLs for active substances are established in Annexes II and III of Regulation (EC) No 396/2005. The review of existing MRLs according to Article 12 of Regulation (EC) No 396/2005 (MRL review) has not been completed for the fictitious active substance MAS and EFSA has not previously issued a Reasoned Opinion on the modification of MRLs for MAS.

Assessment

The present pilot study tests the application of new methodology for the assessment of non‐standard uncertainty in the context of pesticide risk assessment. Non‐standard uncertainties were introduced into the assessment in order to increase the illustrative value of the study and these non‐standard uncertainties may not be acceptable in a regulatory context. The assessment includes non‐standard refinement of the risk assessment for illustrative purposes and is not intended to provide a scheme for risk assessment in future EFSA Reasoned Opinions.

EFSA has based its assessment on the Evaluation Report submitted by Ctgb, acting in the place of the EMS (Schepens, 2017). The assessment by EFSA of real‐case regulatory applications concerning MRLs are usually also based on the DAR (and its addenda) prepared under either Directive 91/414/EEC or Regulation (EC) No 1107/2009, the European Commission Review Report on the active substance, the EFSA Conclusion on the peer review of the pesticide risk assessment of the active substance and, where available, the FAO JMPR evaluation reports, as well as previous EFSA Opinions on the active substance.

For the purposes of this pilot study, the new data requirements established in Regulation (EU) No 283/2013 and the guidance documents applicable at the date of the tender offer for the negotiated procedure for preparation of the Evaluation Report on 28 September 2016 are assumed as being applicable (European Commission, 1997a, b, c, d, e, f, g, 2000, 2010a,b, 2016, 2011; OECD, 2011, 2013). The assessment is performed in accordance with the legal provisions of the Uniform Principles for the Evaluation and the Authorisation of Plant Protection Products adopted by Commission Regulation (EU) No 546/2011. The attendant uncertainties are assessed in accordance with the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment (EFSA Scientific Committee, 2016).

A selected list of fictitious end points of studies assessed in the framework of the EU pesticides peer review, including the end points of fictitious studies submitted in support of the current MRL application, are presented in Appendix B.

1. Residues in plants

1.1. Nature of residues and methods of analysis in plants

1.1.1. Nature of residues in primary crops

In the Evaluation Report, it was reported that the metabolism of the fictitious active substance MAS in primary crops belonging to the fruit (apple), leafy (lettuce) and cereals/grass (wheat) groups of crops after foliar application has been investigated and reviewed in the framework of the (fictitious) EU pesticides peer review of MAS and was found to be similar.

An additional metabolism study was reported to have been submitted in the current fictitious MRL application in support of the use of MAS as a post‐harvest treatment on potatoes (Schepens, 2017). When stored potatoes were treated by gassing with ¹⁴C‐labelled MAS (relevant label position) at a rate (1.5 g/tonne) comparable to the intended post‐harvest treatment use for the current MRL application, parent compound was found to be the main residue. Most of the total radioactive residues (TRR) remained on the peel of the potatoes, with low levels migrating into the pulp (at 6 months after application 87% of the TRR was on the peel). The percentage TRR for substance MAS in the whole potato at 2 weeks after treatment was 80.1%, slightly decreasing to 78.9% TRR after 2 months and 78.6% TRR after 6 months. The major metabolite ATO was identified at 2 weeks after treatment (16.9% TRR), which remained relatively stable during the experiment (15.0% TRR after 2 months and 16.1% TRR after 6 months).

For the purposes of this study, it is postulated that the toxicological profile of metabolite ATO was assessed during the fictitious EU pesticides peer review, and that it was demonstrated that ATO is of similar toxicity as substance MAS.

For the intended uses, the metabolic behaviour in primary crops is sufficiently addressed.

1.1.2. Nature of residues in rotational crops

The fictitious active substance MAS is intended to be used on crops that can be grown in rotation with other crops. The Evaluation Report states that according to soil degradation studies evaluated in the framework of the fictitious EU pesticides peer review, the DT₉₀ value of MAS is 6 days (Schepens, 2017). Further studies investigating the nature and magnitude of residues in rotational crops are not required and have not been submitted. For the intended use of MAS for post‐harvest application, investigations of residues in rotational crops are not required.

1.1.3. Nature of residues in processed commodities

The effect of processing on the nature of MAS was reported to have been investigated in the framework of the fictitious EU pesticides peer review. These studies showed that MAS is hydrolytically stable under standard processing conditions.

Standard hydrolysis studies simulating conditions representative for pasteurisation, boiling and sterilisation on the nature of the metabolite ATO were also reported to have been assessed during the fictitious EU pesticides peer review process and it was concluded that the metabolite ATO is stable under these processing conditions.

1.1.4. Methods of analysis in plants

Analytical methods for the determination of MAS residues and residues of ATO reported to have been assessed during the fictitious EU pesticides peer review. The methods are sufficiently validated for residues of MAS and ATO in the crops under consideration. The methods allow quantifying residues at or above the LOQ of 0.01 mg/kg for each analyte in crops belonging to the group of high water content and high starch content.

1.1.5. Stability of residues in plants

The storage stability of MAS and ATO in plants stored under frozen conditions was reported in the Evaluation Report to have been investigated in the framework of the fictitious EU pesticides peer review. For the purposes of this study, it is postulated that it had been demonstrated that residues in crops assessed in the framework of this application were stable for at least 12 months when stored at −18°C.

1.1.6. Proposed residue definitions

Based on the metabolic pattern identified in metabolism studies, the results of hydrolysis studies, the toxicological significance of metabolites and/or degradation products, the capabilities of enforcement analytical methods and since the metabolism was similar in three different crop groups when substance MAS is foliarly applied, the residue definitions for plants are applicable to all crops with foliar application. The additional metabolism study on stored potatoes demonstrated that during storage the major metabolite ATO is present at relevant levels, and since the toxicity of metabolite ATO is similar to the parent active substance, a specific residue definition was proposed for risk assessment for stored potatoes. Consequently, the following residue definitions were proposed:

• residue for risk assessment (foliar application): MAS (only);

• residue for risk assessment (stored potatoes): the sum of substance MAS and metabolite ATO, expressed as substance MAS;

• residue definition for enforcement: MAS (only).

The same residue definitions are applicable to rotational crops and processed products.

For the purposes of this study, it is postulated that the residue definition for enforcement set in Regulation (EC) No 396/2005 is identical with the above‐mentioned residue definition.

Taking into account the intended use assessed in this application, EFSA concluded that these residue definitions are appropriate and no further information is required.

1.2. Magnitude of residues in plants

1.2.1. Magnitude of residues in primary crops

In support of the MRL application, the applicant submitted residue trials performed in potatoes, tomatoes and cauliflower. The samples were analysed for the parent compound only, in accordance with the residue definition for enforcement for foliar application. The metabolite ATO, which is included in the residue definition for risk assessment for use in stored potatoes, was not analysed. The EMS reported that the methods used were sufficiently validated and fit for purpose.

It was reported in the Evaluation Report that the samples in all the residue trials were stored under frozen conditions within the time periods for which integrity of the samples has been demonstrated. However, in order to increase the illustrative value of this pilot study, an additional example of a minor non‐standard uncertainty was introduced and it was postulated that samples from the supervised residues trials on tomatoes were stored for 1 month longer than the maximum validated time period for storage stability.

A summary of residues data from the supervised residue trials is presented in Appendix B.1.2.1.

1.2.1.1. Potatoes

In support of the good agricultural practice (GAP) requested for post‐harvest use, eight residue trials on potatoes reflecting the intended fumigation treatment were provided. The trials were conducted with sufficient independence of the experimental conditions. Sampling was conducted at 2 weeks after application, and 1, 2, 3, 4, 5 and 6 months after application. The minimum withholding period of the critical good agricultural practice (cGAP) is 14 days, and therefore, the highest residue from each trial was selected for MRL setting and risk assessment.

Samples in the supervised residues trials were analysed for the active substance MAS only. Metabolite ATO, which is included in the residue definition for risk assessment for stored potatoes, which were fumigated during storage, was not measured in the supervised residues trials. The EMS proposed to use a conversion factor (CF_Risk) derived from the potato metabolism study data for this specific case since the ratio of parent to metabolite ATO remained stable during the course of the metabolism study. The EMS calculated a CF_Risk of 1.21 at 14 days, a CF_Risk of 1.19 at 2 months after application, and a CF_Risk of 1.20 at 6 months after application. The EMS calculated a median CF_Risk of 1.2 based on the conversion factors calculated at the different waiting periods. EFSA considered the proposed approach for conversion of residue data from the residue definition for enforcement to the residue definition for risk assessment to be applicable in this specific circumstance. The use of conversion factors for converting residues measured using the residue definition for MRL enforcement to residues corresponding to the residue definition for risk assessment is a source of uncertainty which is not covered by the standard procedure (non‐standard uncertainty) and therefore assessed in the uncertainty analyses (Appendices D.3 and D.4).

The number and quality of the trials is sufficient to calculate a MRL of 0.50 mg/kg for potatoes. Empirical data on the levels of metabolite ATO in GAP‐compliant supervised residues trials reflecting the intended use would be desirable. Such data should be generated by re‐analysing samples from the supervised residues trials for the metabolite ATO.

COMMENTARY: The supervised residue trials should address all components of both the residue definition for enforcement and the residue definition for risk assessment by using appropriate methods (OECD, 2016). Conversion factors derived from empirical supervised residue trials data from the residue definition for enforcement to the residue definition for risk assessment (CF_Risk) may be used where a risk assessment is conducted on the basis of enforcement residue data. The OECD Draft Guidance Document on Crop Field Trials states that ‘Plant metabolism studies give indications and can be used to derive conversion factors for the crop investigated if the study parameters match the intended PHI but should not be used on regular basis as their main purpose is to identify the nature rather than the magnitude of the residue which may vary from crop to crop. In most cases conversion factors should be calculated using data from supervised field trials supported by metabolism data’ (OECD, 2016). Therefore, the use of a conversion factor derived from metabolism study data is not standard procedure and introduces a non‐standard uncertainty.

1.2.1.2. Tomatoes

In support of the GAP requested, eight GAP‐compliant residue trials on standard‐sized tomatoes and eight GAP‐compliant residue trials on cherry tomatoes were provided. All 16 trials were conducted indoor. In order to increase the illustrative value of this pilot study, an additional example of a minor non‐standard uncertainty was introduced to this scientific report and it was therefore postulated that samples from the supervised residues trials on tomatoes were stored for 1 month longer than the maximum validated time period for storage stability.

The residue levels in trials conducted with cherry tomatoes were consistently higher than in the trials conducted with standard‐sized tomatoes. EU MRLs do not distinguish between varieties of tomatoes (crop code 231010) and the standard procedure was applied using the combined data set for MRL setting (standard approach). The chronic dietary risk assessment was based on the supervised trials median residue (STMR) of the combined data (standard approach). The acute dietary risk assessment was based on the highest residue from the combined data (HR; standard approach, worst‐case scenario).

The data from all indoor trials (cherry tomato and standard‐sized tomato) are combined for the MRL calculation. The number and quality of the trials (combined data sets for cherry tomato and standard‐sized tomato) is sufficient to calculate a MRL of 0.50 mg/kg for tomatoes. The MRL calculation for tomatoes is involved with non‐standard uncertainty due to the extended storage period of samples from the supervised residues trials which was judged to have no material impact on the residues trials data (see Appendix D.5).

The calculated MRL for the use on tomatoes is applicable for indoor use only. A limited data set of trials for outdoor uses (around 50% of the required minimum) would be required in order to confirm that outdoor use is less critical.

COMMENTARY: The data generated in the Evaluation Report for the use on tomatoes was intended to present an example of uncertainty based on the consistently different residue levels between the trials conducted with cherry tomatoes and the trials conducted with standard‐sized tomatoes. The standard assessment procedure was applied using the combined data set from all trials for the MRL calculation, the chronic consumer risk assessment, and the acute consumer risk assessment. The standard procedure incorporates an agreed approach to the assessment with an accepted provision for uncertainty due to the difference between residue levels in trials with cherry tomatoes and in trials with standard‐sized tomatoes. Therefore, non‐standard uncertainty is not involved for the different‐sized tomatoes.

1.2.1.3. Cauliflower

In support of the northern Europe (NEU) GAP, eight GAP‐compliant residue decline trials on cauliflower were provided. For the purposes of this study, the trials were postulated to have been conducted at geographical locations and over growing seasons that ensure the independence of trials.

Residues of substance MAS were in the range of < 0.02–0.26 mg/kg with the addition of one high value of 0.71 mg/kg. There was no information provided in the study report to explain the high value. The EMS applied statistical tests to the residues trials data and considered whether the extreme value could be regarded as an abnormal outlier. However, the EMS concluded that, based on expert judgement, the value of 0.71 mg/kg should not be disregarded from the calculations. EFSA considered that, on the basis of the available information, there was no justifiable reason to exclude the highest residue value and agreed with the EMS proposal that the MRL calculation and risk assessments should be based on the complete data set.

The number and quality of the trials is sufficient to calculate a MRL of 1.50 mg/kg for cauliflower.

COMMENTARY: In cases where there is an absence of information and no justification to explain a high‐value outlier, the standard procedure is to not to exclude the high‐value data point. Regarding the handling of outliers, the OECD draft guidance document on crop field trials states: ‘Residue values above the majority of the data population are always suspicious and therefore are often characterised as outliers. Nevertheless, before disregarding a result as an outlier the study should be carefully examined to see if there is adequate information and/or experimental evidence to justify its exclusion. The exclusion of an apparent outlier must be justified by agricultural practice or other evidence deriving from the experimental set up or analytical conditions. Statistical results, in and of themselves, are generally not sufficient to exclude data from the MRL‐setting process’ (OECD, 2016). The EMS considered the MRL calculation for cauliflower to contain additional (non‐standard) uncertainty due to questioning whether the high‐value data point is an artefact outlier that should be excluded from the data set. EFSA proceeded on the basis that in the absence of evidence or justification to exclude the high‐value data point, it must be considered a real value within the distribution of residues levels and therefore should not be excluded from the MRL calculation. EFSA considered the high‐value data point to be valid and calculated the short‐term and long‐term exposure using the HR and the STMR, respectively, from the full data set of supervised residues trials in cauliflower. Therefore, the high value data point was not considered to be a source of non‐standard uncertainty and non‐standard uncertainty is not involved.

1.2.2. Magnitude of residues in rotational crops

The possible transfer of MAS residues to crops that are grown in crop rotation is reported in the Evaluation Report to have been assessed in fictitious EU pesticides peer review. For the purposes of this study, it is assumed that available studies demonstrated that no significant residues (residues below 0.01 mg/kg) are expected in succeeding crops planted in treated soil. In the fictitious EU peer review, soil degradation studies have demonstrated that the DT₉₀ of MAS in soil is 6 days, and therefore, it was concluded that no significant residues are expected and rotational crop studies are not required.

1.2.3. Magnitude of residues in processed commodities

Processing studies with potatoes were evaluated during the fictitious EU peer review. A processing factor of 1.02 for potato flakes, chips and fried potatoes, and a processing factor of 0.99 for potato microwaving were derived for substance MAS.

Processing studies with tomatoes evaluated during the fictitious EU peer review demonstrated that there was no concentration of residues in processed tomato products containing residues of substance MAS. Processing factors were calculated for tomato puree (0.52), tomato juice (0.34) and tomato paste (0.39).

Processing studies in cauliflower evaluated during the fictitious EU peer review demonstrated that cooking (boiling) of cauliflower leads to a reduction of the residues in the processed product. A processing factor of 0.49 was calculated for boiling of cauliflower.

The number and quality of the processing studies are for the purposes of this study postulated to be sufficient to derive robust processing factors which, in a real‐case assessment, would be recommended to be included in Annex VI of Regulation (EC) No 396/2005.

1.2.4. Calculated MRLs

The residues trials on tomato were involved with a non‐standard uncertainty due to the extended storage period of samples which was judged to have no material impact on the trials data (see Appendix D.5). The available data are considered sufficient to calculate MRL proposals as well as risk assessment values for all commodities under evaluation (see Appendix B.1.2.1). In Section 3, EFSA assessed whether residues on these crops resulting from the intended uses are likely to pose a consumer health risk.

2. Residues in livestock

Potatoes may be used for feed purposes. Hence, it was necessary to perform a dietary burden calculation for livestock to estimate whether the intended use of MAS would have an impact on the residues expected in food of animal origin.

The input values for the exposure calculations for livestock are presented in Appendix C. Residue levels according to the residue definition for risk assessment have been used as input. To calculate the input values for potatoes and potato by products, which need to include not only the residues of the parent compound but also the residues of the metabolite generated after the fumigation treatment (see Section 1.1.1), the CF_Risk derived from the metabolism study was used. The results of the dietary burden calculation are presented in Appendix B.2 and demonstrated that the exposure of cattle, sheep, swine and poultry exceeded the trigger values defined in the relevant guidance document.

2.1. Nature of residues and methods of analysis in livestock

2.1.1. Nature of residues in livestock

Metabolism studies in livestock (laying hen and lactating goat) were, for the purposes of this study, postulated to have been assessed previously in the framework of the fictitious EU pesticides peer review. The metabolism studies conducted on laying hen and lactating goat are summarised in Appendix B.2.1.1.

2.1.2. Method of analysis in products of animal origin

For the purposes of this study, sufficiently validated analytical methods are postulated to be available to enforce the MRLs for metabolite ATO in food of animal origin with an LOQ of 0.01 mg/kg. These methods are postulated to have been validated during the fictitious EU pesticides peer review of substance MAS.

2.1.3. Stability of residues in products of animal origin

For the purposes of this study, the stability of residues in products of animal origin is postulated to have been assessed during the fictitious EU pesticides peer review of substance MAS.

2.1.4. Proposed residue definitions

Based on the metabolic pattern identified in metabolism studies, the toxicological significance of metabolites and the capabilities of enforcement analytical methods, the following residue definitions for products of animal origin were proposed:

• Residue definition for risk assessment: metabolite ATO, expressed as substance MAS.

• Residue definition for enforcement: metabolite ATO, expressed as substance MAS.

The fat solubility of residues in products of animal origin was not reported. For metabolite ATO, the log P_ow is not available. The metabolism studies do not give clear indications that the residues accumulate in fat tissue or in milk fat.

For the purposes of this study, it is assumed that the residue definition currently set in Regulation (EC) No 396/2005 is identical to the before mentioned residue definition for enforcement.

2.2. Magnitude of residues in livestock

Feeding studies with lactating goat and laying hens were for the purposes of this study postulated to have been assessed during the fictitious EU pesticides peer review. Metabolism studies with a 18N dose level for lactating goat and 28N dose level for laying hen (relative to the maximum dietary burden) demonstrated that no residue levels above LOQ were present in animal matrices, and therefore, residues in products of animal origin are expected to be at or below the LOQ, and no further feeding studies are required.

2.2.1. Calculated MRLs

The fictitious MRL application in the present study does not include the setting or modification of MRLs for products of animal origin.

3. Consumer risk assessment

EFSA performed a dietary risk assessment using revision 2 of the EFSA PRIMo (Reich, 2007). This exposure assessment model contains food consumption data for different subgroups of the EU population and allows the acute and chronic exposure assessment to be performed in accordance with the internationally agreed methodology for pesticide residues (EFSA, 2007b). The PRIMo report summarises the results of the acute and chronic dietary exposure estimations and is presented in Appendix G.

The toxicological reference values for substance MAS used in the risk assessment (i.e. ADI and ARfD values) were for the purposes of this study postulated to have been derived in the framework of the fictitious EU pesticides peer review. The metabolite ATO, included in the residue definitions for risk assessment for stored potatoes and for products of animal origin, was considered to be of similar toxicity as the parent compound. Potato is a crop that is consumed only after processing and the processing factors for potato in this fictitious case are close to 1 (processing factors for microwaving are 0.99 for MAS and 1.01 for ATO); therefore, processing of potatoes would not have a significant impact on the exposure and the potato processing factors are not applied to the exposure estimation calculations.

The risk assessment is based on standard risk assessment methodology which has been agreed by risk managers with an accepted provision for standard uncertainty. EFSA performed analyses of the non‐standard uncertainties in the acute and chronic dietary risk assessments to include as many as possible of the identified sources of non‐standard uncertainty within the quantitative assessment of combined non‐standard uncertainty, and omitting only those which the assessors are unable to quantify, in accordance with the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment (EFSA Scientific Committee, 2016). The methods of uncertainty analyses were selected from the toolbox provided in the guidance in order to apply a relatively simple strategy that could be considered fit‐for‐purpose to meet the tight legal deadlines in the regulatory context.

Expert judgement is needed in order to quantify non‐standard uncertainty by subjective probability estimation. This can be done by a single assessor or collectively in a process of expert knowledge elicitation. The present pilot study is based on a fictitious data set and therefore formal expert knowledge elicitation was not applied to provide probability judgements. The probability estimations in a real‐case assessment must be based on reasoned consideration of the available evidence, which must be documented in order to meet the required level of transparency. It should be borne in mind that relevant evidence to be used in probability judgements might not be measured data, but could be other forms of evidence including expert knowledge and experience.

The individual non‐standard uncertainties identified in the acute and chronic risk assessments were assessed by sensitivity analyses through iterative back‐calculation of the parameter values that would lead to exceedance of the toxicological reference values (exceedance limit calculations). The individual quantifiable non‐standard uncertainties were estimated by the assessor on the harmonised scale of bounded probabilities proposed in the EFSA Scientific Committee draft guidance (EFSA Scientific Committee, 2016) and were subject to revised estimation where necessary.

The individual quantifiable non‐standard uncertainties affecting the chronic risk assessment were additionally quantified by subjective upper bound probability percentile estimation by the assessor prior to the uncertainty analysis calculations and were not subsequently revised in order to prevent the possible influence of post hoc reasoning. The subjective estimations of the individually quantifiable non‐standard uncertainties affecting the chronic risk assessment were combined by probability bounds analysis in accordance with the method proposed in the EFSA Scientific Committee draft Guidance.

The lists of identified sources of non‐standard uncertainties, subjective probability estimates and the uncertainty analyses calculations are presented in Appendix D, Tables D.3, D.4 and D.5.

3.1. Short‐term (acute) dietary risk assessment

The short‐term exposure estimates were performed for the commodities assessed in this application in accordance with the internationally agreed methodology. The calculations were based on the HR derived from supervised field trials and the complete list of input values can be found in Appendix C.2.

For the intended use in cauliflower, the estimated short‐term exposure (international estimated short‐term intake (IESTI) 1) exceeded the toxicological reference value, accounting for 104.3% of the ARfD (see Appendix D.1). The standard acute dietary risk assessment approach does not use a processing factor for cauliflower because cauliflower is a crop that can be consumed raw, and suitable information on the consumption of raw and cooked cauliflower is not available to perform a refined risk assessment covering all Member State diets. Therefore, the estimated short‐term dietary risk assessment did not apply the processing factor for cauliflower. However, for illustrative purposes, sensitivity analysis and uncertainty analysis were conducted in this pilot study to assess the potential impact and uncertainty due to cooking of cauliflower (see Appendix D.3.3). The short‐term exposure estimate is based on a large portion which is defined in the IESTI calculation as the highest large portion reported at the 97.5 percentile of eaters (FAO, 2016). The estimated short‐term exposure would be at or below the toxicological reference value (ARfD) if at least 8% of the population mean of the large portion of cauliflower is consumed after boiling. It was judged as very likely (90–99% subjective probability) that at least 8% of the population mean of the large portion of cauliflower is consumed cooked and therefore taking account of the non‐standard (additional) uncertainty it was judged as very likely (90–99% subjective probability) that the worst‐case refined IESTI would not exceed the toxicological reference value because a proportion of the population mean consumption of the large portion of cauliflower would be cooked. This is a non‐standard refinement of the risk assessment for illustrative purposes.

COMMENTARY: This example illustrates a non‐standard refinement of the risk assessment. However, a refinement that is outside the standard risk assessment procedure would require a full uncertainty analysis of the standard and non‐standard uncertainties in order to assess the degree of conservatism within the standard risk assessment methodology and to assess whether the conservatism of not using a processing factor for cauliflower maybe already be offset elsewhere within the standard methodology. It should be noted that the ‘Uniform Principles’ for pesticides evaluation, implemented by Commission Regulation (EU) No 546/2011, specify that pesticide evaluations should take elements of uncertainty into consideration in order to ensure that the chances of underestimating adverse effects are reduced to a minimum. Sensitivity analysis and uncertainty analysis are not tools to satisfy the data requirements for pesticides evaluation or to replace data that is specified in the data requirements.

The estimated short‐term exposure (IESTI 1) for the intended use in potatoes was below the toxicological reference value (98.4% of the ARfD; see Appendix D.1) but there was a very narrow margin of exposure and the risk assessment is affected by non‐standard uncertainty due to the use of CF_Risk derived from metabolism study data. EFSA conducted sensitivity analysis and uncertainty analysis to evaluate the impact of the non‐standard uncertainty on the estimated short‐term exposure (see Appendix D.3.1). On the basis of the uncertainty analysis taking account of the non‐standard uncertainty, it was judged as as‐likely‐as‐not (33–66% subjective probability) that the estimated short‐term exposure (IESTI 1) would exceed the toxicological reference value (ARfD).

The estimated short‐term exposure for the intended use in tomatoes is below the toxicological reference value (IESTI 1 is 36.2% ARfD; see Appendix D.1). The estimated short‐term exposure estimate is affected by a minor non‐standard uncertainty regarding extended storage of residues trials samples which was judged to have no material impact on the acute risk assessment for tomatoes (see Appendix D.3.2).

COMMENTARY: Difference between the present scientific report and the Evaluation Report in the calculated percent of the ARfD for potatoes in the short‐term dietary risk assessments is due to rounding of input values in the Evaluation Report. The estimated short‐term exposure in the present report was calculated as 98.4% of the ARfD using HR_Risk = HR_Mo × CF_Risk = 0.24 × 1.2 = 0.288. The estimated short‐term exposure in the Evaluation Report (99.1% of the ARfD) was calculated from rounded HR_Risk = 0.29.

3.2. Long‐term (chronic) dietary risk assessment

The long‐term exposure assessment was performed on the basis of the STMR values derived for the commodities assessed in this fictitious application. In regulatory EFSA Reasoned Opinions, the chronic consumer dietary risk assessment normally takes into account the dietary exposure contribution from commodities assessed in the application as well as the remaining commodities covered by the MRL regulation, the existing EU MRLs and, where relevant, the STMR values derived in previous MRL applications would be selected as input values for any remaining commodities. However, for the purposes of this study, it was assumed that substance MAS is not authorised for use on any other crop in the EU, and therefore, no further data need be included in the exposure assessment. The complete list of input values is presented in Appendix C.2.

The standard dietary risk assessment approach does not use a processing factor for cauliflower because cauliflower is a crop that can be consumed raw, and suitable information on the consumption of raw and cooked cauliflower is not available to perform a refined risk assessment covering all Member State diets. However, in order to introduce an additional example of non‐standard uncertainty for the purpose of increasing the illustrative value of this pilot study, the long‐term dietary risk assessment applied the processing factor for boiling of cauliflower. The EFSA Scientific Committee revised draft Guidance on uncertainty indicates that where the risk assessment includes a refinement that is not covered by the standardised procedure, then the assessment becomes case‐specific and a case‐specific uncertainty analysis is required in which both the standard and non‐standard uncertainties should be considered. However, currently there is no guidance available to analyse the standard uncertainties in the chronic dietary risk assessment of pesticides. Therefore, the additional uncertainty that results from use of a processing factor for cauliflower is analysed in the context of non‐standard uncertainties only.

The estimated long‐term dietary intake was in the range of 21–92% of the ADI and the risk assessment was affected by non‐standard uncertainties. The highest international estimated daily intake (IEDI), according to EFSA PRIMo, was 92.2% of the ADI (NL child). The contributions of residues expected in the commodities assessed in this application to the overall long‐term exposure were: potatoes: 83.6% ADI; tomatoes: 7.4% ADI; and cauliflower: 1.2% ADI (see Appendix D.2). The non‐standard uncertainty in the estimated long‐term dietary intake was assessed quantitatively. Non‐standard uncertainties affecting the contributions of potatoes and cauliflower were individually quantified as subjective probability bounds and subjective probabilities related to the combined non‐standard uncertainties calculated by probability bounds analysis, in accordance with the methods proposed in the EFSA Scientific Committee revised draft Guidance on uncertainty (see Annex B.13 of EFSA Scientific Committee, 2016). Non‐standard uncertainty was involved for the residues trials in tomatoes which was not individually quantified and was judged to have no material impact. In this analysis EFSA assessed the overall non‐standard uncertainty affecting the chronic risk assessment and it was judged as very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value.

Conclusions and recommendations

The data submitted in support of the fictitious MRL application were found to be sufficient to calculate MRLs for all crops under consideration. Adequate analytical methods for enforcement are reported to be available to control the residues of MAS in the plant matrices and commodities under consideration.

Based on the risk assessment results for the intended use of MAS on cauliflower, EFSA concluded that, according to the internationally agreed methodology, the expected short‐term exposure exceeded the toxicological reference value (IESTI 1 is 104.3% ARfD) and a risk to consumer health from short‐term intake cannot be excluded for the intended use of MAS cauliflower. However, it was judged as very likely (90–99% subjective probability) that the worst‐case refined IESTI would not exceed the toxicological reference value because a proportion of the population mean consumption of the large portion of cauliflower would be cooked. This is a non‐standard refinement of the risk assessment for illustrative purposes.

For the intended use on potatoes, EFSA concluded that the expected short‐term exposure according to the internationally agreed methodology was below the toxicological reference value and therefore an acute consumer intake exceedance was not identified; however, the safety margin for acute exposure as regards MAS residues on potatoes is very narrow (IESTI 1 is 98.4% of the ARfD) and the risk assessment is affected by non‐standard uncertainty regarding the CF_Risk derived from metabolism study data. EFSA conducted a sensitivity analysis and uncertainty analysis to evaluate the impact of the non‐standard uncertainty on the estimated short‐term exposure. On this basis, taking account of the non‐standard uncertainty, it was judged as as‐likely‐as‐not (33–66% subjective probability) that the short‐term exposure (IESTI 1) would exceed the toxicological reference value (ARfD).

For the intended use on tomatoes, the expected short‐term exposure was below the toxicological reference value (IESTI 1 is 36.2% of the ARfD) and is affected by a minor non‐standard uncertainty that was judged to have no material impact on the risk assessment.

The estimated long‐term consumer exposure was below the toxicological reference value; however, the safety margin for chronic exposure is narrow (highest IEDI 92.2% of the ADI) and the risk assessment was affected by non‐standard uncertainties affecting the contributions of potatoes and cauliflower. Non‐standard uncertainty involved for the residues trials in tomatoes was judged to have no material impact on the chronic risk assessment. The individually quantifiable non‐standard uncertainties were estimated and combined by probability bounds analysis. From this analysis, taking account of the overall non‐standard uncertainty affecting the chronic risk assessment it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value.

The fictional recommended MRLs are summarised in Appendix E.

1.

1.1.

Conclusions on applicability of the revised draft Guidance on uncertainty

The present pilot study tests the applicability of the EFSA Scientific Committee revised draft Guidance on uncertainty in EFSA scientific assessment to the acute and chronic consumer risk assessments performed in EFSA Reasoned Opinions on the modification of pesticide MRLs. The study included several example sources of non‐standard uncertainty in order to test various methods for uncertainty analysis and to increase the illustrative value of the study. Relatively simple methods of uncertainty analysis were selected, which could be considered for use in routine pesticide risk assessments such as for the setting or modification of pesticide MRLs. Pesticide risk assessments may be affected by more than one source of non‐standard uncertainty and an assessment strategy that is suitable for case‐specific combinations of non‐standard uncertainties would be required. The level of complexity and extent of the uncertainty analyses should be fit‐for‐purpose, taking into consideration the available time and resources. Simpler methods of uncertainty analysis may be appropriate where they provide sufficient information for decision‐making. In order to apply the revised draft Guidance in the present pilot study, additional time and resources were required to develop the uncertainty assessment strategy and structure the analysis tables to assess the non‐standard uncertainties. It should be noted that where uncertainty analyses are conducted on a routine basis, the extended explanation and commentary presented in this pilot study would not be required.

The evaluation report, which generated the data for the fictitious MRL application, included several example sources of uncertainty in order to test the applicability of the revised draft Guidance on uncertainty. The process of checking for and listing any non‐standard sources of uncertainty was conducted during the Reasoned Opinion risk assessment process; however, categorisation of the sources of uncertainty as either standard or non‐standard required judging whether they are adequately covered by the standardised procedure, which was not always evident. A number of example sources of uncertainty were identified as being included in or already covered by the standardised procedure for the assessment of MRL applications that has been agreed by risk managers with an accepted provision for uncertainty. These example sources of uncertainty were therefore classified as standard uncertainties and excluded from the analyses of non‐standard uncertainty.

The individual non‐standard uncertainties identified in the pilot study were assessed by sensitivity analysis and quantified by subjective probability estimation. The methods for sensitivity analysis, by back‐calculation of the parameter values that would lead to exceedance of the toxicological reference value, were found to be readily applicable. However, quantification of the uncertainty by subjective probability estimation may be one of the more challenging areas for implementation of the uncertainty assessment framework in the context of pesticide risk assessment. The present pilot study is based on a fictitious data set and therefore formal expert knowledge elicitation was not applied to provide probability judgements. The probability judgements in a real‐case assessment must be based on reasoned consideration of the available evidence, which might not be measured data, but could be other forms of evidence including expert knowledge and experience and the process must be documented in order to meet the required level of transparency.

The non‐standard uncertainties affecting the chronic risk assessment were quantified by subjective upper bound probability percentile estimation and combined by probability bounds analysis. The probability bounds analysis method was found to provide a relatively simple approach for calculating the probability related to a combination of uncertainties without the need to elicit a full probability distribution.

Overall, the revised draft Guidance was found to provide a comprehensive ‘toolbox of methods’ from which suitable methods could be selected for each step in the pilot study uncertainty assessment. It is suggested that the Guidance may include additional information on implementation of the uncertainty assessment framework in the general context of routine risk assessments for applications related to regulated products. Furthermore, process‐specific guidelines and practical procedures may need to be developed in order to implement the uncertainty assessment framework in routine pesticide risk assessments and to ensure consistent identification and analysis of non‐standard uncertainties.

The application of uncertainty assessment in EFSA Reasoned Opinions on the modification of pesticide MRLs is not intended to alter the conclusions that are reached by the standard risk assessment processes. The uncertainty analyses are intended to provide additional information by making the processes more transparent and the risk assessment results more explicit about how certain are the conclusions reached. This additional information on uncertainty in the risk assessment is intended to support the risk‐based decision‐making process. The additional information provided by uncertainty assessment in the context of EFSA Reasoned Opinions is expected to support the overall process of setting MRLs by enabling risk managers to take account of uncertainty in the decision‐making process. Ideally, the uncertainty assessment should quantify the overall standard and non‐standard uncertainty of the risk assessment, taking into account uncertainties in both the toxicology and exposure elements. An assessment of the overall standard and non‐standard uncertainty would require all elements of pesticide risk assessment process to have been subject to uncertainty analyses. Analysis of the elements of standard uncertainty in pesticide risk assessment, which are accepted by risk managers as providing adequate cover for uncertainty, is outside scope of the present project.

The EFSA PPR at its plenary meeting held on 23 March 2017 supported the overall principle of identifying and analysing the non‐standard uncertainties and the implementation of uncertainty analysis in the area of pesticides. The PPR recommended the preparation of ‘a reference document identifying and assessing the generic uncertainties inherent to the standard procedures of the various types of risk assessment with respect to the precise assessment question and respective desired level of protection’ (EFSA PPR Panel, 2017).

The outcome of the present study together with the results of the trial across the different EFSA panels and units will be presented at an EFSA internal workshop in June 2017 to inform the EFSA Scientific Committee Working Group on how to further tailor the revised draft Guidance on uncertainty and to make it fit‐for‐purpose for the needs of the EFSA panels and units.

Abbreviations

a.s.

active substance

ADI

acceptable daily intake

AR

applied radioactivity

ARfD

acute reference dose

BBCH

growth stages of mono‐ and dicotyledonous plants

bw

body weight

CAC

Codex Alimentarius Commission

CAS

Chemical Abstract Service

CF

conversion factor for enforcement to risk assessment residue definition

cGAP

critical GAP

Ctgb

The Netherlands Board for the Authorisation of Plant Protection Products and Biocides

DAR

draft assessment report

DAT

days after treatment

DM

dry matter

DT₉₀

period required for 90% dissipation (define method of estimation)

EC

emulsifiable concentrate

ECD

electron capture detector

EDI

estimated daily intake

EMS

evaluating Member State

FAO

Food and Agriculture Organization of the United Nations

FID

flame ionisation detector

GAP

Good Agricultural Practice

GC

gas chromatography

GC‐ECD

gas chromatography with electron capture detector

HN

hot fogging concentrate

HR

highest residue

IEDI

international estimated daily intake

IESTI

international estimated short‐term intake

ILV

independent laboratory validation

ISO

International Organisation for Standardisation

JMPR

Joint FAO/WHO Meeting on Pesticide Residues

LOQ

limit of quantification

MRL

maximum residue level

MS

Member States

NEU

northern Europe

OECD

Organisation for Economic Co‐operation and Development

PBI

plant back interval

PF

processing factor

PHI

preharvest interval

P_ow

partition coefficient between n‐octanol and water

PPR

EFSA Panel on Plant Protection Products and their Residues

PRIMo

(EFSA) Pesticide Residues Intake Model

PROFile

(EFSA) Pesticide Residues Overview File

RA

risk assessment

RAC

raw agricultural commodity

RD

residue definition

RMS

rapporteur Member State

SANCO

Directorate‐General for Health and Consumers

SEU

southern Europe

STMR

supervised trials median residue

TAR

total applied radioactivity

TMDI

theoretical maximum daily intake

TRR

total radioactive residue

UV

ultraviolet (detector)

WHO

World Health Organization

Appendix A – Summary of intended GAP triggering the amendment of existing EU MRLs

1.

Crop and/or situationa	MS country	NEU SEU G	Product name	F G or Ib	Pests or group of pests controlledc	Preparation		Application				Application rate per treatment			PHI (days)m	Remarks
						Typed , e , f	Conc. a.s.i	Method kindf , g , h	Growth stages & seasonj	Number min‐maxk	Interval between application min–max	g a.s/hL min–maxl	Water L/ha min‐max	g a.s./ha min–maxl
Potatoes	NL	I	RESTY	I	Post‐harvest fungi	HN	350 g/L	Post‐harvest treatment, gassing		1				1.2 g/tonne	14
Tomatoes	NL	G	RESTY	G	Foliar fungi	EC	500 g/L	Foliar treatment, spraying		1				75–100	7
Cauliflower	NL	NEU	RESTY	F	Foliar fungi	EC	500 g/L	Foliar treatment, spraying		1				75–100	7

GAP: good agricultural practice; MRL: maximum residue level; MS: Member State; NEU: outdoor uses in northern Europe; SEU: outdoor uses in southern Europe; G: Greenhouse (indoor uses); a.s.: active substance.

^aFor crops, the EU and Codex classifications (both) should be taken into account; where relevant, the use situation should be described (e.g. fumigation of a structure).

^bOutdoor or field use (F), greenhouse application (G) or indoor application (I).

^ce.g. biting and sucking insects, soil born insects, foliar fungi, weeds.

^dEmulsifiable concentrate (EC), hot fogging concentrate (HN).

^eCropLife International Technical Monograph no 2, 6th Edition. Revised May 2008. Catalogue of pesticide.

^fAll abbreviations used must be explained.

^gMethod, e.g. high volume spraying, low volume spraying, spreading, dusting, drench.

^hKind, e.g. overall, broadcast, aerial spraying, row, individual plant, between the plant‐type of equipment used must be indicated.

ⁱg/kg or g/L. Normally the rate should be given for the active substance (according to ISO) and not for the variant in order to compare the rate for same active substances used in different variants (e.g. fluoroxypyr). In certain cases, where only one variant is synthesised, it is more appropriate to give the rate for the variant (e.g. benthiavalicarb‐isopropyl).

^jGrowth stage range from first to last treatment (BBCH Monograph, Growth Stages of Plants, 1997, Blackwell, ISBN 3‐8263‐3152‐4), including where relevant, information on season at time of application.

^kIndicate the minimum and maximum number of applications possible under practical conditions of use.

^lThe values should be given in g or kg whatever gives the more manageable number (e.g. 200 kg/ha instead of 200,000 g/ha or 12.5 g/ha instead of 0.0125 kg/ha.

^mPHI: minimum preharvest interval.

Appendix B – Selected list of end points

B.1. Residues in plants

B.1.1. Nature of residues and methods of analysis in plants

B.1.1.1. Metabolism studies, methods of analysis and residue definitions in plants

Primary crops (available studies)	Crop groups	Crop(s)	Application(s)	Sampling (DAT)
	Fruit crops(a)	Apple	Foliar, 2 × 100 g/ha	7, 14
	Root crops(a)	–	–	–
	Leafy crops(a)	Lettuce	Foliar, 1 × 125 g/ha	14
	Cereals/grass(a)	Wheat	Foliar, 1 × 150 g/ha, BBCH 49	7, 14 and 80 (maturity)
	Pulses/oilseeds(a)	–	–	–
	Post‐harvest treatment(b)	Potatoes	Gassing, 1 × 1.5 g/tonne	14, 2 m, 6 m
	Radiolabelled active substance: 14C‐labelled MAS (relevant label position) Reference: (a)EFSA, fictitious EU peer review; (b)Schepens (2017)
Rotational crops (available studies)	Crop groups	Crop(s)	Application(s)	PBI (DAT)
	Root/tuber crops	–	–	–
	Leafy crops	–	–	–
	Cereal (small grain)	–	–	–
	Other	–	–	–
	Comments: MAS DT90 soil = 6 days. Studies investigating the nature and magnitude of residues in rotational crops are not required Reference: EFSA, fictitious EU peer review
Processed commodities (hydrolysis study)	Conditions			Investigated?
	Pasteurisation (20 min, 90°C, pH 4)			Yes
	Baking, brewing and boiling (60 min, 100°C, pH 5)			Yes
	Sterilisation (20 min, 120°C, pH 6)			Yes
	Comments: Investigated for both MAS and metabolite ATO Reference: EFSA, fictitious EU peer review

DAT: days after treatment; BBCH: growth stages of mono‐ and dicotyledonous plants; PBI: plant back interval; DT₉₀: period required for 90% dissipation.

B.1.1.2. Stability of residues in plants

Plant products (available studies)	Category	Commodity	T (°C)	Stability (months)
	High water content	Cauliflower	−18	> 12
	High oil content	–	–	–
	Dry/high starch	Potatoes	–	–
	High acid content	Tomatoes	−18	> 12
	Comments: Stability of residues in plants is for the purpose of this study postulated to have been assessed during the fictitious peer review Reference: EFSA, fictitious EU peer review

B.1.2. Magnitude of residues in plants

B.1.2.1. Summary of residues data from the supervised residue trials

Crop (supervised trials)	Region/indoora	Residue levels observed in the supervised residue trials (mg/kg)	Comments (OECD calculations; unrounded/rounded result)	Crop (MRL application/request)	Calculated MRL (mg/kg)	HRMo b (mg/kg)	STMRMo c (mg/kg)	CFd
Potatoes (RD‐Mo≠RD‐RA)	Indoor	Mo: 0.04, 0.09, 2x 0.12, 0.14, 0.16, 0.23, 0.24 RA: –	MRLOECD: 0.43/0.50 Residue trials results were not provided according to the residue definition for risk assessment	Potatoes	0.50	0.24	0.13	1.2**
Tomatoes (RD‐Mo=RD‐RA)	Indoor	Standard‐sized tomato: Mo/RA: 3x 0.03, 2x 0.05, 0.07, 0.09, 0.12	MRLOECD data set with cherry tomato only: 0.18/0.20	Tomatoes	–	0.12	0.05	–
		Cherry tomato: Mo/RA: 0.14, 2x 0.15, 0.18, 0.20, 0.24, 0.26, 0.28	MRLOECD, data set with standard‐sized tomato only: 0.60/0.60		–	0.28	0.19
		Data sets combined: Mo/RA: 3x 0.03, 2x 0.05, 0.07, 0.09, 0.12, 0.14, 2x 0.15, 0.18, 0.20, 0.24, 0.26, 0.28	MRLOECD, data sets combined: 0.39/0.50		0.50	0.28	0.13
Cauliflower (RD‐Mo=RD‐RA)	NEU	Mo/RA: < 0.02, 2x 0.02, 0.03, 0.09, 0.15, 0.26, 0.71	MRLOECD: 1.11/1.50	Cauliflower	1.50	0.71	0.06	–

RD: residue definition; Mo: monitoring; RA: risk assessment; MRL: maximum residue level; OECD: Organisation for Economic Co‐operation and Development.

* Indicates that the MRL is recommended at the limit of quantification.

** CF_Risk potatoes derived from the potato metabolism study data.

^aNEU: Outdoor trials conducted in northern Europe, SEU: Outdoor trials conducted in southern Europe, Indoor: indoor EU trials or Country code: if non‐EU trials.

^bHighest residue according to the residue definition for monitoring.

^cSupervised trials median residue according to the residue definition for monitoring.

^dConversion factor to recalculate residues according to the residue definition for monitoring to the residue definition for risk assessment.

COMMENTARY: Residue levels after a post‐harvest treatment in potatoes would normally be expected to be higher, however, for the purposes of this pilot study the EMS generated data with low residues in potatoes in order to increase the illustrative value.

B.1.2.2. Conversion factors for risk assessment in plant products

Crop	Median conversion factors at the different waiting periods
	14 days	2 months	6 months					Comments
Potatoes (post‐harvest treatment)	1.21**	1.19**	1.20**					Metabolism study data
Comments	** CFRisk derived from the potato metabolism study data A median CFRisk for potatoes of 1.2 was calculated based on the CFs derived for potatoes (post‐harvest treatment) at the different waiting periods

CF: conversion factor for enforcement to risk assessment residue definition.

B.1.2.3. Residues in succeeding crops

B.1.2.4. Processing factors

Processed commodity	Number of valid studiesa	Processing factor (PF)		CFP c
		Individual values	Median PFb
Potatoes, flakes	Not reported	Not reported	1.02 (MAS) 1.05 (ATO)	1.2
Potatoes, chips	Not reported	Not reported	1.02 (MAS) 1.05 (ATO)	1.2
Potatoes, fried	Not reported	Not reported	1.02 (MAS) 1.05 (ATO)	1.2
Potatoes, microwaving	Not reported	Not reported	0.99 (MAS) 1.01 (ATO)	1.2
Tomatoes, puree	Not reported	Not reported	0.52	–
Tomatoes, juice	Not reported	Not reported	0.34	–
Tomatoes, paste	Not reported	Not reported	0.39	–
Cauliflower, boiling	Not reported	Not reported	0.49	–

^aStudies with residues in the RAC at or close to the LOQ were disregarded (unless concentration may occur).

^bProcessing studies were postulated to have been evaluated during the fictitious EU peer review.

^cConversion factor for risk assessment in the processed potato commodities (CF_P) is taken from the CF_Risk for potatoes derived from the raw commodities using the potato metabolism study data.

B.2. Residues in livestock

Relevant groups	Dietary burden expressed in				Most critical dieta	Most critical commoditya	Trigger exceeded (Y/N)
	mg/kg bw per day		mg/kg DM
	Median	Maximum	Median	Maximum
Beef cattle	0.255	0.260	–	10.83	Potato	Process waste	Yes
Dairy cattle	0.309	0.317	–	8.23	Potato	Process waste	Yes
Ram/Ewe	0.354	0.361	–	10.83	Potato	Process waste	Yes
Lamb	0.228	0.233	–	5.49	Potato	Process waste	Yes
Pig (breeding)	0.129	0.137	–	5.92	Potato	Process waste	Yes
Pig (finishing)	0.052	0.062	–	2.07	Potato	Dried pulp	Yes
Poultry broiler	0.101	0.105	–	1.49	Potato	Dried pulp	Yes
Poultry layer	0.074	0.079	–	1.15	Potato	Dried pulp	Yes
Turkey	0.011	0.021	–	0.29	Potato	Culls	Yes

bw: body weight; DM: dry matter.

^aCalculated for the maximum dietary burden.

Comments: Livestock dietary burden calculated with CF_Risk potatoes derived from the potato metabolism study data.

COMMENTARY: Differences in the calculated residues in livestock between the Reasoned Opinion and Evaluation Report are due to rounding of input values for the dietary burden calculation in the Evaluation Report.

B.2.1. Nature of residues and methods of analysis in livestock

B.2.1.1. Metabolism studies, methods of analysis and residue definitions in livestock

Livestock (available studies)	Animal	Dose (mg/kg bw per day)	Duration (days)	N rate/comment
	Laying hen	3.1	14	ca. 28N
	Lactating goat	6.8	4	ca. 18N
	Pig	Not required	–	Metabolism in rat and ruminant is similar
	Comment: Metabolism of substance MAS is for the purpose of this study postulated to have been assessed during the fictitious peer review

bw: body weight.

B.2.1.2. Stability of residues in livestock

Animal products (available studies)	Animal	Commodity	T (°C)	Stability (months)
	Laying hen	Muscle	−18	> 12
		Liver	−18	> 12
		Kidney	−18	> 12
		Egg	−18	> 12
	Lactating goat	Muscle	−18	> 12
		Liver	−18	> 12
		Kidney	−18	> 12
		Milk	−18	> 12
	Comment: Stability of residues in livestock is for the purpose of this study postulated to have been assessed during the fictitious peer review Reference: EFSA, fictitious EU peer review

B.2.2. Magnitude of residues in livestock

B.2.2.1. Summary of the residue data from livestock feeding studies

Not relevant since residues in products of animal origin are expected to be at or below the LOQ. No further animal feeding studies are required.

Appendix C – Input values for the exposure calculations

C.1. Livestock dietary burden calculations

Feed commodity	Median dietary burden		Maximum dietary burden
	Input value (mg/kg)	Comment	Input value (mg/kg)	Comment
Residue definition for risk assessment for plant products (except stored potatoes): substance MAS Residue definition for risk assessment for stored potatoes: sum of substance MAS and metabolite ATO, expressed as substance MAS
Potatoes, culls	0.156	STMRMo × CFRisk (0.13 × 1.2)	0.288	HRMo × CFRisk (0.24 × 1.2)
Potatoes, process waste	3.12	STMRMo × CFRisk × default PFa (0.13 × 1.2 × 20)
Potatoes, dried pulp	5.928	STMRMo × CFRisk × default PFa (0.13 × 1.2 × 38)

STMR: supervised trials median residue; HR: highest residue; PF: processing factor; CF: conversion factor for enforcement to risk assessment residue definition; Mo: monitoring.

^aFor potatoes process waste and potatoes dried pulp, in the absence of processing factors supported by data, default processing factors of 20 and 38 were, respectively, included in the calculation to consider the potential concentration of residues in these commodities (Dujardin et al., 2016).

Comments: Calculated with CF_Risk potatoes derived from the potato metabolism study data.

C.2. Consumer risk assessment

Commodity	Chronic risk assessment		Acute risk assessment
	Input value (mg/kg)	Comment	Input value (mg/kg)	Comment
211000 Potatoes	0.156	STMRRisk = STMRMo × CFRisk (0.13 × 1.2)	0.288	HRRisk = HRMo × CFRisk (0.24 × 1.2)
231010 Tomatoes	0.13	STMR Combined data sets standard‐sized tomato and cherry tomato	0.28	HR Combined data sets standard‐sized tomato and cherry tomato
241020 Cauliflower	0.0294	STMR × PFBoiling (0.06 × 0.49)	0.71	HR

STMR: supervised trials median residue; HR: highest residue; PF: processing factor; CF: conversion factor for enforcement to risk assessment residue definition; Mo: monitoring.

Comments: CF_Risk potatoes derived from the potato metabolism study data. The chronic dietary risk assessment applied the processing factor for boiling of cauliflower; see Section 3.2.

Appendix D – Consumer risk assessment

D.1. Acute risk assessment

D.2. Chronic risk assessment

D.3. Acute risk assessments uncertainty analyses

COMMENTARY: The uncertainty analyses presented in this pilot study include extended explanation and commentary which would not be required where uncertainty analyses are conducted on a routine basis.

D.3.1. Acute risk assessment uncertainty analysis: potatoes (post‐harvest use)

Acute risk assessment uncertainty analysis: Potatoes (post‐harvest use)
Parametera	Source of non‐standard uncertaintyb	Qualitative assessment	Quantitative assessment
CFRisk (metabolite conversion factor for risk assessment)	Empirical data on levels of metabolite ATO in supervised residue trials is not available. The calculated CFRisk for potatoes is the median of CFs derived from the potato metabolism study at various waiting periods. Although preferably the CFRisk should be derived from residue trials, supported by metabolism studies, in exceptional cases based on expert judgement, it is acceptable to derive CFRisk from metabolism studies. This approach is considered a non‐standard procedure	With CFRisk = 1.2, the estimated short‐term exposure (IESTI 1) from the intended use in potatoes is 98.4% of the toxicological reference value (ARfD) and no further refinements were possible with the available data. The CFRisk potatoes derived from metabolism study data may underestimate or overestimate the quantity of metabolite ATO in the residue trials. Therefore, the uncertainty direction for CFRisk is both upward and downward (underestimation or overestimation of the risk: the real risk can be higher or lower than the estimate)	Sensitivity analysis Sensitivity analysis was conducted to assess the potential impact of the uncertainty of the CFRisk potatoes on the international estimated short‐term intake (IESTI 1). The value of the CFRisk potatoes that would lead to the IESTI 1 exceeding the toxicological reference value (ARfD) for the intended use in potatoes (exceedance limit to the scale of CFRisk) was calculated by iterative back‐calculation of the PRIMo as CFRisk ARfD Limit = 1.2195 (HRRisk ARfD Limit = 0.2927 mg/kg) Subjective probability The degree of uncertainty of the median CFRisk derived from the potato metabolism study was quantitatively estimated by subjective probability. It was judged that it is as‐likely‐as‐not (33–66% subjective probability***) that the median CFRisk potatoes in residue trials would exceed 1.2195 because requirements for experimental independence of residues trials is expected to lead to greater variability than the controlled experimental conditions of the metabolism study and it is judged as‐likely‐as‐not (33–66% subjective probability***) that this greater variability would lead to the increased level of metabolism of fictitious substance MAS to metabolite ATO than was observed under conditions of the potato metabolism study. Therefore, taking account of the degree of individual non‐standard uncertainty due to the median CFRisk derived from the potato metabolism study it was judged as as‐likely‐as‐not (33–66% subjective probability) that IESTI 1 would exceed the ARfD for to the intended use in potatoes COMMENTARY: The uncertainty in the median CFRisk derived from the potato metabolism study is quantified as subjective probability estimates in relation to the median CFRisk potatoes expected in residue trials (‘real value’), rather than to the median CFRisk potatoes expected for commercial use situations in the EU, because the degree of uncertainty as a result of extrapolation from residue trials data to commercial use situations is a standardised uncertainty that is accepted and already included in the established risk assessment procedure
Other parameters	No further non‐standard uncertainties identified	–	–
Overall assessment of non‐standard uncertainty	Since only one parameter is affected by non‐standard uncertainties, the overall uncertainty is the same as the uncertainty for that parameter alone. It was therefore concluded that, taking account of the non‐standard uncertainty, it is judged to be as‐likely‐as‐not (33–66% probability) that the estimated short‐term exposure (IESTI 1) for the intended use in potatoes would exceed the toxicological reference value (ARfD)

CF: conversion factor for enforcement to risk assessment residue definition; ARfD: acute reference dose; PRIMo: (EFSA) Pesticide Residues Intake Model.

^aParameter(s) or element(s) of the risk assessment affected by the non‐standard uncertainty. Indicate the smallest component of the risk assessment affected.

^bDescription to indicate the source and nature of non‐standard uncertainty.

*** Subjective probability judgement to the harmonised scale of bounded probabilities proposed in the EFSA Scientific Committee draft guidance (EFSA Scientific Committee, 2016). The present pilot study is based on a fictitious data set and therefore limited examples of reasoning and justification of probability judgement are provided. The probability estimations in a real‐case assessment must be based on reasoned consideration of the available evidence which must be documented in order to meet the required level of transparency. Relevant evidence might not be measured data, but could be other forms of evidence including expert knowledge and experience.

D.3.2. Acute risk assessment uncertainty analysis: Tomatoes (greenhouse application)

Acute risk assessment uncertainty analysis: tomatoes (greenhouse application)
Parametera	Source of non‐standard uncertaintyb	Qualitative assessment	Quantitative assessment
HR	Samples from the supervised residues trials were stored for 1 month longer than the maximum validated time period for storage stability	The uncertainty direction is upward (underestimation: the real value higher than estimate). The data from the supervised residues trials may underestimate the quantity of substance MAS in the residue trials	Taking account of the non‐standard uncertainty due to the additional one‐month sample storage period in the supervised residues trials, it was judged that the degree of additional degradation of substance MAS that can be expected to occur during an additional one‐month storage period would be within the range of normal measurement error for substance MAS in the supervised residues trials for tomatoes, and therefore, the extended storage period would have no material impact on the acute risk assessment for tomatoes
Other parameters	No further non‐standard uncertainties identified	–	–
Overall assessment of non‐standard uncertainty	Since only one parameter is affected by non‐standard uncertainty, the overall non‐standard uncertainty is the same as the uncertainty for that parameter alone. It is therefore concluded that the overall non‐standard uncertainty is judged to have no material impact on the acute risk assessment for tomatoes
COMMENTARY: Examples of standard uncertainties that are covered in the standard procedure are provided below for illustrative purposes. Uncertainties that are covered in the standard procedure would not normally be listed in the uncertainty analysis, but would be included in the overall assessment of uncertainties
Bias in residue trials conducted with cherry tomatoes versus standard‐sized tomatoes		Standard procedure uses the full data set for calculation of the MRL. The uncertainty due to different varieties of tomatoes is established in standard procedure. Therefore non‐standard uncertainty analysis is not required
Supervised residues trials are not available for outdoor use		Tomatoes are a crop that may be grown either under greenhouse (indoor) or field (outdoor) conditions and generally the use on protected crop leads to higher residue levels. The data submitted in the present application are from trials conducted under indoor conditions (only). The standard procedure proposes a MRL for indoor use only and therefore a non‐standard uncertainty assessment is not required

HR; highest residue; MRL: maximum residue level.

^aParameter(s) or element(s) of the risk assessment affected by the non‐standard uncertainty. Indicate the smallest component of the risk assessment affected.

^bDescription to indicate the source and nature of non‐standard uncertainty.

Greyed cells: Example standard uncertainties that would not normally be listed in the analysis of non‐standard uncertainty.

D.3.3. Acute risk assessment uncertainty analysis: Cauliflower (NEU)

Acute risk assessment uncertainty analysis: Cauliflower (NEU)
Parametera	Source of non‐standard uncertaintyb	Qualitative assessment	Quantitative assessment
Proportion of large portion of cauliflower consumed raw	The short‐term dietary risk assessment assumed that all cauliflower is eaten raw and did not apply the processing factor for boiling of cauliflower (PFBoiling). The standard risk assessment approach does not use a processing factor for cauliflower because suitable information on the consumption of raw and cooked cauliflower is not available. However, for illustrative purposes in this pilot study, sensitivity and uncertainty analyses were conducted to assess the uncertainty of the potential impact of cooking cauliflower. Use of the processing factor for cooking of cauliflower would introduce a source of non‐standard uncertainty because cauliflower is a crop which is usually consumed cooked	The PRIMo currently contains a mixture of raw and cooked cauliflower consumption data and the uncertainty assessment is based on the proportion of the population mean of the large portion of cauliflower that is consumed raw/cooked. This is a non‐standard refinement of the risk assessment. The short‐term dietary risk assessment did not use PFBoiling cauliflower, and therefore, the uncertainty direction is downward (overestimation: real value lower than estimate). The short‐term exposure (international estimated short‐term intake (IESTI) 1; without processing factor) exceeded the toxicological reference value for the intended use in cauliflower, accounting for 104.3% of the ARfD. The acute risk assessment assumes that all cauliflower is eaten raw (RRaw = 1). Cauliflower is a crop which is usually consumed cooked and the processing factor for boiling of cauliflower (PFBoiling=0.49) indicates that the acute risk assessment may overestimate the dietary exposure. However, use of PFBoiling cauliflower in the acute risk assessment would introduce a non‐standard uncertainty due to the lack of information on the proportion of the large portion of cauliflower consumed raw (by cauliflower consumers)	Sensitivity analysis The short‐term exposure estimate is based on a large portion which is defined in the IESTI calculation as the highest large portion reported at the 97.5 percentile of eaters (FAO, 2016). Sensitivity analysis was conducted to assess the potential impact of the uncertainty of the proportion of the large portion of cauliflower consumed raw on the IESTI 1. The value of the HRRisk cauliflower that would lead to the IESTI 1 exceeding the toxicological reference value (ARfD) for the intended use in cauliflower (exceedance limit to the scale of HRRisk) was calculated by iterative back‐calculation of the PRIMo as 0.681 mg/kg (HRRisk ARfD Limit) The population mean proportion of the large portion of cauliflower eaten raw by cauliflower consumers (as opposed to eaten boiled) that would lead to the IESTI 1 for the intended use in cauliflower exceeding the ARfD was calculated (RRaw ARfD Limit): HRRisk ARfD Limit = HRRisk × RRaw ARfD Limit + HRRisk  × PFBoiling × (1 − RRaw ARfD Limit) ∴ HRRisk ARfD Limit = HRRisk × RRaw ARfD Limit + HRRisk × PFBoiling × 1 − HRRisk × PFBoiling × RRaw ARfD Limit ∴ HRRisk ARfD Limit − HRRisk × PFBoiling = HRRisk × RRaw ARfD Limit × (1 − PFBoiling) ∴ RRaw ARfD Limit = (HRRisk ARfD Limit − HRRisk × PFBoiling)/(HRRisk × (1 − PFBoiling)) RRaw ARfD Limit = (0.681 − 0.71 × 0.49)/(0.71 × (1 − 0.49)) = 0.92 (92%) Therefore, the IESTI 1 would be at or below the toxicological reference value (ARfD) if at least 8% of the population mean of the large portion of cauliflower is consumed boiled Subjective probability estimation The non‐standard uncertainty due to the proportion of large portion of cauliflower consumed raw was estimated by subjective probability. It was judged as very likely (90–99% subjective probability***) that less than 92% of the population mean of the large portion of cauliflower is consumed raw because cauliflower is a crop for which the large portion size is usually consumed cooked. Raw cauliflower is typically consumed in significantly smaller portion size than cooked cauliflower. Therefore, taking account of the non‐standard uncertainty due to the proportion of large portion of cauliflower consumed raw, it was judged as very likely (90–99% subjective probability) that the worst‐case refined IESTI for the intended use in cauliflower would not exceed the toxicological reference value (ARfD). This is a non‐standard refinement of the risk assessment for illustrative purposes COMMENTARY: The present pilot study selected the proportion of cauliflower consumed as an example element to estimate uncertainty because the proportion of cauliflower consumed raw would allow comparison of calculations for both the acute and chronic risk assessments. However, it may be more appropriate to base the acute risk assessment uncertainty analysis on the size of the large portion of cauliflower eaten raw. Information on large portion size of raw cauliflower may be available in food consumption databases
Other parameters	No further non‐standard uncertainties identified	–	–
Overall assessment of non‐standard uncertainty	Since only one parameter is affected by non‐standard uncertainty, the overall non‐standard uncertainty is the same as the uncertainty for that parameter alone. The IESTI 1 exceeded the toxicological reference value for the intended use in cauliflower (104.3% of the ARfD). Therefore, taking account of the overall non‐standard uncertainty, it was judged as very likely (90–99% subjective probability) that the estimated short‐term exposure would be below the toxicological reference value for the intended use in cauliflower
COMMENTARY: Examples of standard uncertainties that are covered in the standard procedure are provided below for illustrative purposes. Uncertainties that are covered in the standard procedure would not normally be listed in the uncertainty analysis, but would be included in the overall assessment of uncertainties
Outlier (high) value in the supervised residues trials data set		The Evaluation Report assessed the uncertainty due to application of the processing factor for cauliflower (boiling) in combination with the high‐value outlier data point in the residues trials for cauliflower (which EFSA considered to be a standard uncertainty), and judged that the non‐standard uncertainty was such that it is very unlikely (0–10% subjective probability) the short‐term exposure from the intended use in cauliflower would exceed the toxicological reference value. Standard procedure does not exclude outlier data points in residues trials data from the MRL‐setting process unless experimental information or justification is available. Unjustified exclusion of a data point would introduce bias in the data set. Assessment of the impact (sensitivity analysis) of excluding the outlier data point without scientific justification would not be appropriate in the context of the risk assessment

NEU: northern Europe; PRIMo: (EFSA) Pesticide Residues Intake Model; ARfD: acute reference dose.

^aParameter(s) or element(s) of the risk assessment affected by the non‐standard uncertainty. Indicate the smallest component of the risk assessment affected.

^bDescription to indicate the source and nature of non‐standard uncertainty.

*** Subjective probability judgement to the harmonised scale of bounded probabilities proposed in the EFSA Scientific Committee draft guidance (EFSA Scientific Committee, 2016). The present pilot study is based on a fictitious data set and therefore limited examples of reasoning and justification of probability judgement are provided. The probability estimations in a real‐case assessment must be based on reasoned consideration of the available evidence which must be documented in order to meet the required level of transparency. Relevant evidence might not be measured data, but could be other forms of evidence including expert knowledge and experience.

Greyed cells: Example standard uncertainties that would not normally be listed in the analysis of non‐standard uncertainty.

D.4. Chronic risk assessment uncertainty analysis

Chronic risk assessment uncertainty analysis
Component(s) of the risk assessment		Source of non‐standard uncertaintyc	Qualitative and/or quantitative assessmentd
Crop or situation (GAP)a	Parameter and/or elementb
Individually quantified non‐standard uncertainties
Potatoes	CFRisk (metabolite conversion factor for risk assessment)	Empirical data on levels of metabolite ATO in supervised residue trials is not available. The calculated CFRisk for potatoes is the median of CFs derived from the potato metabolism study at various waiting periods. The use of metabolism study data to derive CFRisk potatoes is a non‐standard procedure based on expert judgement	Quantitative uncertainty assessment The uncertainty direction is both upward and downward (underestimation or overestimation of risk: the real risk can be higher or lower than the estimate). The CFRisk potatoes derived from metabolism study data may underestimate or overestimate the quantity of metabolite ATO in residue trials. The estimated long‐term intake (TMDI) from the intended use in potatoes contributed to up to 83.6% of the ADI (highest theoretical maximum daily intake: 92.2% of ADI for NL child) and no further refinements were possible with the available data. Median CFRisk potatoes calculated from the metabolism study data is 1.2 Sensitivity analysis The value of the median CFRisk potato that would lead to exceedance of the toxicological reference value (ADI) for the long‐term dietary intake (TMDI) (exceedance limit to the scale of median CFRisk potato) was calculated by iterative back‐calculation of the PRIMo as CFRisk ADI Limit = 1.292 (STMRRisk ADI Limit = 0.168 mg/kg) Individual uncertainty subjective probability The individual non‐standard uncertainty due to the median CFRisk derived from the metabolism study was estimated as very unlikely (1–10% subjective probability***) that the median CFRisk derived from potato residues trials would be equal to or greater than 1.292 because, although requirement for residues trial experimental independence is expected to lead to greater variability than the controlled experimental conditions of the metabolism study, it was judged very unlikely (1–10% subjective probability***) that potatoes under post‐harvest use residues trial conditions would lead to the significantly greater extent of metabolism of fictitious substance MAS to metabolite ATO than was observed under the conditions of the potato metabolism study. Therefore, taking account of the individual non‐standard uncertainty for the median CFRisk derived from metabolism study, it was judged very unlikely (1–10% subjective probability) that the estimated long‐term intake (TMDI) would exceed the toxicological reference value (ADI) COMMENTARY: It is noted the EMS estimated a greater degree of uncertainty for CFRisk when considering the chronic as well as the acute exposure calculation and stated that ‘since the chronic as well as the acute exposure calculation are both close to 100% of the toxicological reference values, the uncertainty related to the use of the CF could potentially (33–66% probability) lead to an exceedance of the toxicological reference values’ (Schepens, 2017) Subjective probability percentiles In order to enable calculation of combined non‐standard uncertainty for the long‐term dietary risk assessment, the uncertainty of the median CFRisk derived from the potato metabolism study was quantified by estimation of subjective upper bound probability percentiles (see ‘Combined individually quantified non‐standard uncertainty’, below) Rational and evidence base for the probability percentiles judgements The present pilot study is based on a fictitious data set and therefore formal expert knowledge elicitation was not used to provide probability judgement. The probability estimations in a real‐case assessment must be based on reasoned consideration of the available evidence which must be justified and documented in order to meet the required level of transparency. The EFSA Scientific Committee stresses that where suitable data are available, this should be used in preference to relying solely on expert judgement (EFSA Scientific Committee, 2016). Where the data are limited, expert judgement based on evidence such as expert knowledge and experience, supported by the available data, may be necessary to quantify uncertainty. Where justification for quantification is not possible, the uncertainty should be expressed qualitatively only. For the purposes of the present fictitious‐case pilot study, the subjective upper bound probability percentiles of the median CFRisk were estimated by the assessor as follows: 97.5 probability percentile (CFRisk P97.5%) = 1.28 99 probability percentile (CFRisk P99%) = 1.35 99.9 probability percentile (CFRisk P99.9%) = 1.40
Cauliflower	Long‐term dietary exposure estimate	The long‐term dietary risk assessment used the processing factor for boiling of cauliflower assuming that all cauliflower is eaten boiled Use of a processing factor for cooking of cauliflower is a source of non‐standard uncertainty because cauliflower is a crop which can also be consumed raw The standard risk assessment approach does not use a processing factor for cauliflower because suitable information on the consumption of raw and cooked cauliflower is not available COMMENTARY: The application of the processing factor for boiling of cauliflower used to introduce a non‐standard uncertainty example and increase the illustrative value of this pilot study	Quantitative uncertainty assessment The uncertainty direction is upward (underestimation of risk: real risk can be higher than the estimate). The long‐term dietary risk assessment used the processing factor for boiling cauliflower (PFBoiling = 0.49) assuming that all cauliflower is eaten boiled (RRaw = 0). Cauliflower is a crop which may be consumed raw and the use of the processing factor may underestimate the dietary exposure and introduces a non‐standard uncertainty in the chronic risk assessment over the proportion of cauliflower consumed raw Sensitivity analysis (method 1) Sensitivity analysis to assess the potential impact of the processing factor for boiling of cauliflower (0.49) assuming that if all cauliflower were to be consumed raw (RRaw = 1) indicates the estimated long‐term dietary intake from the intended use in cauliflower would contribute to maximally 2.5% of the ADI (highest theoretical maximum daily intake: 93.5% of ADI for NL child) (overestimation: real value lower than estimate) Sensitivity analysis (method 2) The value of the STMRRisk that would lead to the estimated long‐term dietary intake (TMDI) exceeding the toxicological reference value (ADI) for the intended use in cauliflower (exceedance limit to the scale of STMRRisk) was calculated by iterative back‐calculation of the PRIMo as 0.2125 mg/kg (STMRRisk ADI Limit). The population mean proportion of cauliflower eaten raw that would lead to the estimated long‐term dietary intake (TMDI) exceeding the toxicological reference value (ADI) for the intended use in cauliflower (RRaw ADI Limit) was calculated as follows: STMRRisk ADI Limit = STMRRisk × RRaw ADI Limit + STMRRisk × PFBoiling × (1 − RRaw ADI Limit) ∴ RRaw ADI Limit = (STMRRisk ADI Limit − STMRRisk × PFBoiling)/(STMRRisk × (1 − PFBoiling)) RRaw ADI Limit = (0.2125 − 0.06 × 0.49)/(0.06 × (1 − 0.49)) = 5.98 (598%) The proportion of cauliflower eaten raw that would lead to exceedance of the toxicological reference value (ADI) is greater than 1 (exceedance limit to the scale of the population mean proportion of cauliflower eaten raw (RRaw ADI Limit) = 5.98); therefore, the toxicological reference value (ADI) would not be exceeded even if all cauliflower were to be consumed raw (RRaw = 1) Individual uncertainty subjective probability The individual non‐standard uncertainty was quantified on the proposed harmonised scale of bounded probabilities as extremely unlikely (0–1% subjective probability***) that the population mean proportion of cauliflower consumed raw would lead to the estimated long‐term intake (TMDI) exceeding the toxicological reference value (ADI) for the intended use in cauliflower because the sensitivity analysis calculation indicates that the toxicological reference value would not be exceeded even if all cauliflower were to be consumed raw Probability percentiles The intended use in cauliflower contributed to maximally 1.2% of the ADI in the estimated long‐term dietary intake (TMDI; with processing factor; underestimation: real value higher than estimate) and the non‐standard uncertainty due to the proportion of cauliflower consumed raw is expected to have a minor impact on the long‐term dietary consumer risk assessment. However, the margin of safety is narrow (highest estimated long‐term dietary intake is 92.2% of ADI for NL child; with PFBoiling cauliflower; underestimation: real value higher than estimate) and therefore, the combined non‐standard uncertainty in the long‐term dietary consumer risk assessment was assessed. In order to enable calculation of combined non‐standard uncertainty for the long‐term dietary risk assessment, the uncertainty of the population mean proportion of cauliflower consumed raw was quantified by estimation of subjective upper bound probability percentiles (see ‘Combined individually quantified non‐standard uncertainty’, below) Rational and evidence base for the probability percentiles judgements The present pilot study is based on a fictitious data set and therefore formal expert knowledge elicitation was not used to provide probability judgement. The probability estimations in a real‐case assessment must be based on reasoned consideration of the available evidence which must be justified and documented in order to meet the required level of transparency. The EFSA Scientific Committee stresses that where suitable data are available, this should be used in preference to relying solely on expert judgement (EFSA Scientific Committee, 2016). Where the data are limited, expert judgement based on evidence such as expert knowledge and experience, supported by the available data, may be necessary to quantify uncertainty. Where justification for quantification is not possible, the uncertainty should be expressed qualitatively only. The probability percentile was not reported where the assessor had a low degree of confidence in the uncertainty estimation. For the purposes of the present fictitious‐case pilot study, the subjective upper bound probability percentiles of the population mean proportion of cauliflower consumed raw were estimated by the assessor as follows: 97.5 probability percentile: 30% 99 probability percentile: 40% 99.9 probability percentile: Not provided (assessor not able to quantify) The subjective upper bound 99.9 probability percentile was considered by the assessor but estimation was not provided due to a low degree of confidence in the uncertainty estimation of the 99.9 probability percentile for the population mean proportion of cauliflower consumed raw COMMENTARY: Extreme fringe probability percentiles may in practice be problematic to estimate. This example illustrates a case where an estimate cannot be made (or ‘forced’) because the assessor does not have sufficient evidence, ‘information base’ or experience to estimate the uncertainty. Further evidence on extreme dietary intake of raw cauliflower would be required to estimate the extreme probability percentile. Indicative data on the consumption of raw cauliflower may be available in food consumption databases or from other sources; see for example, IARC, 2004
Combined individually quantified non‐standard uncertainty			Combining individually quantified non‐standard uncertainty subjective upper bound probability percentiles by probability bounds analysis The combined individually quantifiable non‐standard uncertainty was calculated by probability bounds analysis in accordance with the method proposed in the EFSA Scientific Committee draft guidance (see Annex B.13 of EFSA Scientific Committee, 2016). PRIMo input values were calculated for each subjective upper bound probability percentile Uncertainty 1: Median CFRisk potatoes Subjective upper bound 97.5 probability percentile (CFRisk P97.5%) = 1.28 STMRRisk Potatoes P97.5% = STMRMo Potatoes × CFRisk P97.5% STMRRisk Potatoes P97.5% = 0.13 × 1.28 = 0.166 mg/kg (PRIMo input value) Subjective upper bound 99.0 probability percentile (CFRisk P99.0%) = 1.35 STMRRisk Potatoes P99% = STMRMo Potatoes × CFRisk P99% STMRRisk Potatoes P99% = 0.13 × 1.35 = 0.176 mg/kg (PRIMo input value) Subjective upper bound 99.9 probability percentile (CFRisk P99.9%) = 1.40 STMRRisk Potatoes P99.9% = STMRMo Potatoes × CFRisk P99.9% STMRRisk Potatoes P99.9% = 0.13 × 1.40 = 0.182 mg/kg (PRIMo input value) Uncertainty 2: Population mean proportion of cauliflower consumed raw Subjective upper bound 97.5 probability percentile: 30%; RRaw P97.5% = 0.3 STMRRisk Cauliflower P97.5% = STMRRisk Cauliflower × RRaw P97.5% + STMRRisk Cauliflower × PFBoiling × (1 − RRaw P97.5%) STMRRisk Cauliflower P97.5% = 0.06 × 0.3 + 0.06 × 0.49 × (1 − 0.3) = 0.039 mg/kg (PRIMo input value) Subjective upper bound 99 probability percentile: 40%; RRaw P99% = 0.4 STMRRisk Cauliflower P99% = STMRRisk Cauliflower × RRaw P99% + STMRRisk Cauliflower × PFBoiling × (1 − RRaw P99%) STMRRisk Cauliflower P99% = 0.06 × 0.4 + 0.06 × 0.49 × (1 − 0.4) = 0.042 mg/kg (PRIMo input value) Subjective upper bound 99.9 probability percentile: Not provided (assessor not able to quantify) STMRRisk Cauliflower P99.9% = Not calculated Combined non‐standard uncertainty probability bounds calculations The chronic dietary exposure was calculated at three combined non‐standard uncertainty probability bounds using the EFSA PRIMo with two probability bound input values. Other PRIMo input values were the same as the risk assessment calculation Combined non‐standard uncertainty: subjective upper bound P97.5% + P97.5% probability percentiles Probability bound 1 PRIMo input: STMRRisk Potatoes P97.5% = 0.166 mg/kg Probability bound 2 PRIMo input: STMRRisk Cauliflower P97.5% = 0.039 mg/kg Combined probability bounds P97.5% + P97.5% ≈ P≥95% PRIMo output: highest IEDI P≥95% = 98.2% ADI (NL child); Contribution of crops assessed: Potatoes (P≥95%): 89.2% of ADI; Tomatoes (P≥95%): 7.4% of ADI; Cauliflower (P≥95%): 1.6% of ADI Combined non‐standard uncertainty: subjective upper bound P99% + P99% probability percentiles Probability bound 1 PRIMo input: STMRRisk Potatoes P99% = 0.176 mg/kg Probability bound 2 PRIMo input: STMRRisk Cauliflower P99% = 0.042 mg/kg Combined probability bounds P99% + P99% ≈ P≥98% PRIMo output: highest IEDI P≥98% = 103.2% ADI (NL child); Contribution of crops assessed: Potatoes (P≥98%): 94.1% of ADI; Tomatoes (P≥98%): 7.4% of ADI; Cauliflower (P≥98%): 1.8% of ADI Combined non‐standard uncertainty: subjective upper bound P99.9% + P99% probability percentiles Probability bound 1 PRIMo input: STMRRisk Potatoes P99.9% = 0.182 mg/kg Probability bound 2 PRIMo input: STMRRisk Cauliflower P99% = 0.042 mg/kg Probability bounds analysis P99.9% + P99% ≈ P≥98.9% PRIMo output: highest IEDI P≥98.9%: 106.7% ADI (NL child); Contribution of crops assessed: Potatoes (P≥98.9%): 97.5% of ADI; Tomatoes (P≥98.9%): 7.4% of ADI; Cauliflower (P≥98.9%): 1.8% of ADI Combined individually quantified non‐standard uncertainty subjective upper bound probability percentiles The combined individually quantified non‐standard uncertainty in the chronic risk assessment was calculated by subjective probability bounds analysis. Taking account of the individually quantified non‐standard uncertainties in the chronic risk assessment, there was ≥ 95% subjective probability that the highest IEDI would not exceed 98.2% ADI (NL child), and ≥ 98% subjective probability that the highest IEDI would not exceed 103.2% ADI (NL child), and ≥ 98.9% subjective probability that the highest IEDI would not exceed 106.7% ADI (NL child). Based on these calculations, the combined individually quantified non‐standard uncertainty was assesses as very likely (90–99% subjective probability***) the long‐term consumer exposure from the intended uses of MAS would be below the toxicological reference value. Taking account of the non‐standard uncertainties, it was therefore judged as very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value COMMENTARY: Probability bounds analysis provides bounds for combinations of bounded probabilities. Here, probability bounds analysis is applied to the exposure assessment, but it could also be applied to uncertainties affecting toxicity assessment or to the risk assessment as a whole
Non‐standard uncertainties which are not individually quantified
Tomatoes	STMR	Samples from the supervised residues trials were stored for 1 month longer than the maximum validated time period for storage stability	Qualitative uncertainty assessment The uncertainty direction is upward (underestimation: the real value higher than estimate). The data from the supervised residues trials may underestimate the quantity of substance MAS in the residue trials. Taking account of the non‐standard uncertainty due to the additional one‐month sample storage period in the supervised residues trials, it was judged that the degree of additional degradation of substance MAS that can be expected to occur during an additional one‐month storage period would be within the range of normal measurement error for substance MAS in the supervised residues trials for tomatoes, and therefore, the extended storage period would have no material impact on the chronic risk assessment for tomatoes COMMENTARY: In a situation where several uncertainties are not individually quantified the assessor would need to make a judgement about their collective contribution to the uncertainty. The non‐standard uncertainties which are not quantified could not be entered into the PRIMo calculation and so would have to be considered together with the other unquantified uncertainties
Overall assessment of non‐standard uncertainty (Individually quantified non‐standard uncertainty and non‐standard uncertainties which are not individually quantified)			Taking account of the combined individually quantified non‐standard uncertainty, it was judged as very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value. The non‐standard uncertainty which was not individually quantified was judged to have no material impact on the chronic risk assessment. Therefore, taking account of the overall non‐standard uncertainty, it was judged as very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value

GAP: good agricultural practice; CF: conversion factor for enforcement to risk assessment residue definition; TMDI: theoretical maximum daily intake; ADI: acceptable daily intake; PRIMo: (EFSA) Pesticide Residues Intake Model; EMS: evaluating Member State; STMR: supervised trials median residue; IEDI: international estimated daily intake.

^aCrop(s) and/or situation(s) should indicate the affected intended GAP(s) triggering the amendment of existing EU MRLs.

^bParameter(s) or element(s) of the risk assessment affected by the non‐standard uncertainty. Indicate the smallest component of the risk assessment affected.

^cDescription to indicate the source and nature of non‐standard uncertainty.

^dIndicate: Quantitative assessment/qualitative assessment/Not individually assessed (combined assessment only)/Not able to quantify individually or collectively.

*** Subjective probability judgement to the harmonised scale of bounded probabilities proposed in the EFSA Scientific Committee draft guidance (EFSA Scientific Committee, 2016). The present pilot study is based on a fictitious data set and therefore limited examples of reasoning and justification of probability judgement are provided. The probability estimations in a real‐case assessment must be based on reasoned consideration of the available evidence which must be documented in order to meet the required level of transparency. Relevant evidence might not be measured data, but could be other forms of evidence including expert knowledge and experience.

D.5. MRL calculation uncertainty analysis

MRL calculation		Source of non‐standard uncertaintyb	Qualitative and/or quantitative assessmentc
Commodity (code)	Parameter and/or element affecteda
Potatoes (211000)	–	None identified	Not applicable COMMENTARY: Metabolite ATO is not included in the residue definition for monitoring and therefore the MRL calculation is not affected by the non‐standard uncertainty of CFRisk for potatoes
Tomatoes (231010)	STMR	Samples from the supervised residues trials were stored for 1 month longer than the maximum validated time period for storage stability (see Section 1.2.1)	Qualitative uncertainty assessment The non‐standard uncertainty direction is upward (underestimation: the real value higher than estimate). The data from the supervised residues trials may underestimate the quantity of substance MAS in the residue trials. Taking account of the non‐standard uncertainty due to the additional one‐month sample storage period in the supervised residues trials, it was judged that the degree of additional degradation of substance MAS that can be expected to occur during an additional one‐month storage period would be within the range of normal measurement error for substance MAS in the supervised residues trials for tomatoes, and therefore, the extended storage period was judged to have no material impact on the residues trials data. It is therefore concluded that the number and quality of the trials (cherry tomato and standard‐sized tomato data sets combined) is sufficient to calculate a MRL for tomatoes
Cauliflower (241020)	–	None identified	Not applicable

MRL: maximum residue level; STMR: supervised trials median residue; CF: conversion factor for enforcement to risk assessment residue definition.

^aParameter(s) or element(s) of the risk assessment affected by the non‐standard uncertainty.

^bDescription to indicate the source and nature of non‐standard uncertainty.

^cIndicate: Quantitative assessment/qualitative assessment/Not individually assessed (combined assessment only)/Not able to quantify individually or collectively.

Appendix E – Recommended MRLs

1.

Codea	Commodity	Existing EU MRL (mg/kg)	Recommended EU MRL (mg/kg)	Comment/justification and uncertainties
Enforcement residue definition: MAS (only)
211000	Potatoes	0.01*	0.50	The submitted data are sufficient to derive a MRL proposal for the post‐harvest treatment use in potatoes The estimated short‐term exposure for the intended use in potatoes (IESTI 1) is 98.4% of the toxicological reference value (ARfD) and therefore an acute consumer intake exceedance was not identified; however, the safety margin for acute exposure as regards MAS residues on potatoes is very narrow and the risk assessment is affected by non‐standard uncertainty regarding the CFRisk derived from metabolism study data. Taking account of the non‐standard uncertainty affecting the acute risk assessment it was judged as as‐likely‐as‐not (33–66% subjective probability) that the short‐term exposure (IESTI 1) would exceed the toxicological reference value (ARfD) The estimated long‐term exposure from the intended use in potatoes contributed to up to 83.6% of the ADI (highest theoretical maximum daily intake: 92.2% of ADI for NL child) and the risk assessment was affected by non‐standard uncertainties. Taking account of the overall non‐standard uncertainty affecting the chronic risk assessment, it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value
231010	Tomatoes	0.01*	0.50	The residues trials on tomato were involved with a non‐standard uncertainty which was judged to have no material impact on the trials data. The submitted data are considered sufficient to derive a MRL proposal for the intended use on tomatoes from the indoor supervised residues trials The estimated short‐term exposure for the intended use in tomatoes is below the toxicological reference value (IESTI 1 is 36.2% ARfD) and is affected by a minor non‐standard uncertainty that was judged to have no material impact on the risk assessment The estimated long‐term exposure from the intended use in tomatoes contributed to up to 7.4% of the ADI (highest theoretical maximum daily intake: 92.2% of ADI for NL child) and the risk assessment was affected by non‐standard uncertainties. Taking account of the overall non‐standard uncertainty affecting the chronic risk assessment, it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value
241020	Cauliflower	0.01*	Further risk management considerations required	The submitted data are sufficient to calculate a MRL of 1.50 mg/kg for the intended use in cauliflower The estimated short‐term exposure (IESTI 1) from the intended use in cauliflower exceeded the toxicological reference value (104.3% of the ARfD), and consequently, a risk to consumer health could not be excluded for the intended use of MAS on cauliflower. However, it was judged as very likely (90–99% subjective probability) that the worst‐case refined IESTI would not exceed the toxicological reference value because a proportion of the population mean consumption of the large portion of cauliflower would be cooked The estimated long‐term exposure from the intended use in cauliflower contributed to up to 1.2% of the ADI (highest TMDI: 92.2% of ADI for NL child) and the risk assessment was affected by non‐standard uncertainties. Taking account of the overall non‐standard uncertainty affecting the chronic risk assessment, it was judged very unlikely (1–10% subjective probability) that the long‐term consumer exposure from the intended uses of MAS would exceed the toxicological reference value

MRL: maximum residue level; IESTI: international estimated daily intake; ARfD: acute reference dose; CF: conversion factor for enforcement to risk assessment residue definition; ADI: acceptable daily intake; TMDI: theoretical maximum daily intake.

* Indicates that the MRL is set at the limit of analytical quantification (LOQ).

^aCommodity code number according to Annex I of Regulation (EC) No 396/2005.

Appendix F – Used compound codes

1.

Code/trivial name	Chemical name/SMILES notation	Structural formula
MAS	MAS is the name for a fictitious active substance	Not applicable
ATO	ATO is the name for a fictitious metabolite of MAS	Not applicable

Appendix G – Pesticide Residue Intake Model (PRIMo)

1.

Suggested citation: EFSA (European Food Safety Authority) , Pedersen R, 2017. Scientific Report on the pilot study on uncertainty analysis in EFSA Reasoned Opinions on the modification of pesticide maximum residue levels. EFSA Journal 2017;15(7):4906, 52 pp. 10.2903/j.efsa.2017.4906