ASPECT OF THE MODEL TO BE EVALUATED BY THE RISK ASSESSOR – DEBtox model applications for sublethal effectsPlease note that when a Yes/No answer is not possible or not applicable, the box is highlighted in yellow.		Yes	No
0. Evaluation of the physiological DEB part DEBtox models consist of two parts: (iii) the physiological DEB part describing the basic metabolic processes combined with species‐specific details; this part is assumed to have been evaluated and approved beforehand; (iv) the TKTD part, describing the effects of toxicant on life‐history traits through changes in the DEB parameters. This checklist is adapted to the evaluation of the TKTD part of the DEBtox models (see Sections 2.3 and 5), but the first thing to check is whether the physiological part, meaning the DEB part, was evaluated beforehand for the chosen species. Without that check, the use of a DEBtox model cannot be suggested, even if the TKTD model part is evaluated positively.
(a)	Was the evaluation of the physiological DEB part conducted by a regulatory authority or a group delegated to this at the EU level?		X
(b)	If (a) is yes, did the above evaluation of the physiological DEB part conclude that it is suitable for use in risk assessment?
(c)	If (a) and (b) are yes, can the physiological DEB part of the model describe the behaviour of the control data?
1. Evaluation of the problem definition The problem definition needs to explain how the modelling fits into the risk assessment and how it can be used to address the specific protection goals (Chapter 3). Please check if due attention is paid to the following points:
(a)	Is the regulatory context for the model application documented?	X
(b)	Is the question that has to be answered with the model clearly formulated?	X
(c)	Is the model output suitable to answer the formulated questions?		X
(d)	Is the choice of the test species clearly described and justified, also considering all the available valid information (including literature)?	X
(e)	Is the species to be modelled specified? – Is it clear whether the model is being used with a Tier‐1 test species i.e. Tier‐2C1 or with one or more relevant species (which might include the Tier‐1 species) i.e. Tier‐2C2?		X
2. Evaluation of the quality of the supporting experimental data In this part of the evaluation, it is checked whether the experimental data with which the model is compared (both calibration and validation data sets) have been subjected to quality control. The focus is on the data quality, i.e. the laboratory conditions, set‐up, chemical analytics and similar. Additional specific criteria for the suitability of the data sets for model calibration and validation are evaluated later in more detail (Sections 7 and 9 of this checklist).
(a)	Are all types of data fully described, meaning that factors like temperature, food conditions, measurements, handling, etc. are documented?	X
(b)	Has the quality of the used data been considered and documented? (see the list of OECD test guidelines in Section 7, Table 6).		X
(c)	Have all available data been used (either for calibration or for validation)? If not, is there a justification why some information has not been used?	X
(d)	Is it checked whether the actual exposure profile in the study matches the intended profile in the test (+/‐ 20%); if not, are then measured concentrations used for the modelling, instead of nominal ones?	X
3. Evaluation of the conceptual model The conceptual model provides a general and qualitative description of the system to be modelled. The physiological DEB part should have been evaluated and approved beforehand (see Section 0 of this checklist). Please check the following items for the TKTD part (refer to Section 7.3):
(a)	Is the reference to the evaluation of the physiological DEB part given?		X
(b)	Is the potential mode of action of the toxicant specified (effects on assimilation, growth, maintenance costs, reproductive output and/or survival)?	X
(c)	Are the links between the scaled damage or the internal concentration and the DEB parameters logical?	X
(d)	In case environmental factors (e.g. temperature) are explicitly considered, is their influence on the physiological and/or TKTD processes documented in the conceptual model?		X
(e)	Are the modelling endpoints relevant to the specific protection goal?	X
4. Evaluation of the formal model The formal model contains equations used in the model. The physiological DEB part should have been evaluated and approved beforehand (see Section 0 of this checklist). Nevertheless, all equations (from either the physiological or the TKTD part) that include one or more parameters that are impacted by the effects of the toxicant need to be documented. Please check the following items:
(a)	Are all mathematical equations relevant for the effect of the toxicant described, including the equations of the toxicokinetic (TK) part and the equations relating the used scaled damage or internal concentrations with the DEB parameter(s)?	X
(b)	Is there a list or a summary of all variables and parameters including their meaning and unit?		X
(c)	Are both the deterministic part (equations describing the mean tendency of the data) and the stochastic part (the probability law describing the variability around the mean tendency) of the model fully described? See Section 5.1 for an example.		X
5. Evaluation of the computer model The computer model corresponds to the implementation of the formal model that can run on a computer. Please check the following items:
(a)	Is the computer code available, including explaining comments for the most important functions?	X
(b)	Is enough information provided to allow non‐expert user to re‐run the model independently (e.g. parameter sets, input data)?	X
(c)	Is it demonstrated that the mathematical model is correctly implemented (model verification), e.g. by checking and documenting the internal consistency of the model results based on a set of default or extreme cases (see e.g. Section 4.1.2.3 for an example with GUTS, ‘reality check’ in chapter 10.4 of EFSA PPR Panel, 2014)?		X
6. Evaluation of the regulatory model – the environmental scenarios The environmental scenarios determine the environmental context in which the model is run. For the application of DEBtox for the prediction of toxicological effects in a regulatory context, the environmental scenario need to be fixed to the laboratory conditions of the experiments used for calibrating the TKTD part of the model. Please check the following items:
(a)	Are all the relevant conditions, recorded in the experiments used for calibrating the TKTD part of the model, used consistently for the simulations (i.e. for generating EPx) used in the risk assessment?
7. Evaluation of the regulatory model – parameter estimation Parameter estimation requires a suitable data set, the correct application of a parameter optimisation routine, and the comprehensive documentation of methods and results. Model parameters are always estimated for a specific combination of species and compound (see Section 3 for background information). Supporting data for DEBtox have to be of sufficient quality (Section 2 in this checklist), be relevant to the risk assessment problem and fulfil a set of basic criteria. Typical supporting data for the TKTD part of DEBtox models are experimental toxicity data for growth or growth + reproduction, to which mortality or immobility data can also be added (see Sections 2.3 and 7.2 for background information, Section 5.1.3 for an example). Specific requirements on the time resolution and temporal scale for the calibration data cannot be given, but in general, the number of time‐points has to be sufficient to provide enough degrees of freedom for the model calibration. Please check the following items to evaluate the calibration data, the parameter optimisation process and the results (see Sections 4.1.3 and 7.6.2):
(a)	Is it clear which parameters have been taken from literature or other sources and which have been fitted to data?If used, are values from literature reasonable and justified?	X
(b)	Are the calibration data sufficient to provide enough degrees of freedom for the model calibration, i.e. are endpoints at least reported at several intermediate time‐points?		X
(c)	Does calibration data span from treatment levels with no effects up to strong effects, ideally up to full effects (e.g. no growth, no reproduction)?	X
(d)	Have all data available for calibration been used? If not, is there a justification why this information has not been used?	X
(e)	Has attention been paid in terms of adjusting the time course of the experiment to capture the full toxicity of the pesticide?	X
(f)	Has the model parameter estimation been adequately documented, including settings of optimisation routines, and type and settings of the numerical solver that was used for solving the differential equations?		X
(g)	If Bayesian inference has been used, are priors on model parameters reported?If a frequentist approach has been used, are starting values for the optimisation reported?		X
(h)	Are the estimated parameter values reported including confidence/credible limits?		X
(i)	Is the method to get these limits reported and documented?		X
(j)	Are the optimal values of the objective function for calibration (e.g. log‐likelihood function) as the result of the parameter optimisation reported?		X
(k)	Are plots of the calibrated DEBtox model in comparison with the calibration data over time provided, and does the visual match appear of acceptable quality?	X
(l)	Has a posterior predictive check been performed and documented?		X
8. Evaluation of the sensitivity and uncertainty analysis It is assumed that sensitivity and uncertainty analyses for the physiological DEB model part have been evaluated and approved beforehand (see Section 0 of this checklist).Sensitivity analysis identifies the influence of the parameters on the model outputs and can hence identify the most influential parameters. Uncertainty analysis aims at identifying how uncertain the model output is regarding the uncertainty in parameter estimates.Please check the following items for the TKTD part:
(a)	Has a sensitivity analysis for the TKTD part been performed and been adequately documented? (The range of parameter variation in the sensitivity analysis should be justified by an analysis of the expected variation of model parameters.)		X
(b)	Are the results of the sensitivity analysis presented so that the most sensitive parameters can be identified?		X
(c)	Is the parameter uncertainty for the most important TKTD model parameters propagated to the model outputs and the results of the uncertainty propagation been documented?		X
(d)	Are the model outputs reported including confidence/credible intervals?		X
9. Evaluation of the model by comparison with independent measurements (model validation) The model performance is usually evaluated by comparing relevant model outputs with independent experimental measurements (i.e. data from other experiments than those used for calibration, often referred to as model validation). These independent measurements (or validation data sets) are used to test the model performance in predicting the chosen endpoints under time‐variable exposure profiles. For DEBtox models, relevant outputs may be simulated growth and/or reproduction, sometimes together with simulated survival, or EPx values. The following checklist is mandatory only for invertebrates, for vertebrates a case‐by‐case basis check needs to be done (see also Sections 7.7.2 and 4.1.4.5):
(a)	Are effect data available from experiments under time‐variable exposure?	X
(b)	If non‐destructive measurement is possible, are sufficient endpoint measurements provided in order to cover at least before, during and after each pulse exposure?Are these time‐points enough to allow for evaluation of changes in growth rates different from the control treatment?		X
(c)	Are two exposure profiles tested with at least two pulses each, separated by no‐exposure intervals of different duration length?		X
(d)	Is the individual depuration and repair time (DRT95) calculated based on toxicokinetic parameters, and is the duration of the no‐exposure intervals defined accordingly?[Ideally one of the profiles should show a no‐exposure interval shorter than the DRT95, the other profile clearly larger than the DRT95]		X
(e)	Is each profile tested at least at 3 concentration levels, in order to obtain low, medium and high effects at the end of the respective experiment?		X
(f)	Has attention been paid to the duration of the experiments considering the time course of development in toxicity of the specific pesticide?	X
(g)	Does the visual match (‘visual fit’ in FOCUS Kinetics, (2006)) of the model prediction quality indicate acceptability of the model predictions in comparison with the validation data?	X
(h)	Do the reported quantitative model performance criteria (e.g. PPC, NRMSE, SPPE) indicate a sufficient model performance?For DEBtox models, the adequacy of the quantitative criteria listed above (set on basis of GUTS models) needs still to be fully tested and may need future adaptation. However, for the time being, this set of performance criteria is also suggested for DEBtox models (see Section 7.7.2 ‘Model performance criteria’).		X
(i)	Has the performance of the model been reported in an objective and reproducible way?	X
10. Evaluation of model use When using a TKTD model for regulatory purposes, the inputs of species‐ and compound‐specific model parameters and of exposure profile data are required to run the model under new conditions. In this stage, it is important that the model is well documented and that it is clear how the model works. Please check the following items:
(a)	Is the use of the model sufficiently documented? Is a user manual available? See items provided in Section 10.7 of the EFSA PPR Panel (2014).	X
(b)	Is an executable implementation of the model made available to the reviewer, or Is at least the source code provided?	X
(c)	Has a summary sheet been provided by the applicant? The summary sheet should provide quick access to the comprehensive documentation with sections corresponding to the ones of this checklist.		X
(d)	Does the exposure profile used with the TKTD model come from the same source as the PEC used with the Tier‐1 effects data? For example, if FOCUS Step 3 maximum values were used at Tier‐1 are the exposure profiles used Tier‐2C from the same FOCUS Step 3 modelling? If the exposure profile comes from any other source (e.g. different scenarios, different inputs, different model) has this been checked?
(e)	In case parameters of the DEB (physiological) part of the model were changed from calibration to validation in order to better describe control data:‐ Is the difference in the parameters inducing considerable difference in the model predictions?And, if yes:‐ Are the model simulations relevant for the risk assessment carried out with the parameter set producing the more conservative (worst‐case) predictions?