Table 2.

Abbreviated list of recommendations on best practice to maximize the use and reuse of QSP models

QSP workflow step		Recommendations		Relevant references
		Mathematical	Computational
1. Purpose and context of the model		Ask “Do I need a model?” and “What is the purpose of the model?” Engage with stakeholders: “end users” and “domain experts” Formulate clearly the questions addressed, their context, expected impact of the decisions derived from the model, and rationale for the selection of QSP as modeling methodology		Peterson & Riggs (2015)11 Timmis et al. (2017)26 Ribba et al. (2017)27 Gadkar et al. (2016)28 Friedrich (2016)29
2. Model structure and modeling methodology	i. Model domain and general structure	Define clearly the model domain: therapeutic area, biological scale, biological/clinical system Provide a schematic representation of the model domain and general structure (e.g., Figure 2) Whenever possible, follow standard graphical notation (e.g., SBGN)		Gadkar et al. (2016)28 Figure 2 Le Novère et al. (2009)36
	ii. Model formulation or algorithm	Provide all equations and boundary conditions (e.g., Box  1) Explain all the terms and their biological/pharmacological meaning	Clearly state the algorithm using pseudo‐code and clearly state any associated equations Explain all the rules and parameters and their biological/pharmacological meaning	Box  1 Timmis et al. (2017)26
		Explain any abstractions and/or simplifications made Report units for each element in the model
	iii. Model solving and simulation method	State the method used to solve the system of equations (e.g., Runge‐Kutta fourth/fifth order implemented via the ode45 solver in MATLAB64) Provide absolute/relative tolerance value	Clearly state simulation engine used (and version)	Timmis et al. (2017)26
		Provide software package used and version
	iv. Code files	Share code and model files generated to build and run the model via the following: ○Supplementary material of an article ○Public online model repositories (e.g., BioModels4, DDMore6) ○Academic author websites, or ○Public platforms for computational code (e.g., GitHub37) Ensure code is easy to follow, adequately annotated, and as error free as possible Whenever possible, use a standard format (e.g., SBML, PharmML)		Chelliah et al. (2015)4 Lloyd et al. (2008)5 DDMore‐Foundation (2012–2018)6 GitHub (2018)37 Hucka et al. (2003)9 Golebiewski (2019)16 Swat et al. (2015)17 Smith et al. (2017)18 Cuellar et al. (2003)38
3. Input data, knowledge and assumptions going into the model		Use input data from systems under experimental conditions as relevant as possible to the system being modeled Provide a detailed model parameter description, including the following: ○Symbol/name of parameter ○Definition ○Parameter value (or range of values) ○Units ○Sources used to obtain it (literature citation, database, derivation from other parameters, experiment presented in the same report/article, in silico estimations, etc.) ○Details of how the parameter value was determined (measured directly, fitted or assumed) and whether the underlying data has any limitations (suspected errors, outliers, high variability, excluded data points, etc.) Consider using a tabular format to present this information (e.g., Table  1) Consider providing actual data files along with code files (see 2. Model structure and modeling methodology, iv. Code files in this table) Describe the following in detail: ○Qualitative and/or semiquantitative knowledge obtained firsthand from stakeholders ○Assumptions (pharmacological, physiological, disease, data, mathematical, statistical) and how they were tested Discuss potential limitations of model in the context of available input data, knowledge, and assumptions		Table  1 Sarkans et al. (2018)40 Marshall et al. (2016)41 Ribba et al. (2017)27 Bonate et al. (2012)42
4. Model verification		Test code for consistency: ○Eliminate detected coding errors ○Ensure solutions or limit conditions reached by the model are correct (e.g., A + B ‐> C yields no C when A and B are set to zero) Determine the steady states of the system Run a sensitivity analysis to identify which parameters have the most effect on model responses and how significant is that effect		Anderson et al. (2007)43 Hicks et al. (2015)45 Nestorov et al. (1999)67 Nestorov et al. (1997)68 Kirouac (2018)69 Thomaseth & Cobelli (1999)70
		When model parameters are assumed, that is, not supported by independent, reliable input data or knowledge (see 3. Input data, knowledge and assumptions going into the model in this table): ○Check that those parameters are identifiable ○Consider techniques to establish model parameter redundancy Consider running a bifurcation analysis to define the scope of extrapolations from the model Consider model reduction methods		Walter et al. (1987)48 Janzen et al. (2016)53 Raue et al. (2014)55 Karlsson et al. (2012)56 Villaverde et al. (2019)57 Meshkat et al. (2014)58 Saccomani et al. (2010)59 Choquet et al. (2012)60 Cole et al. (2010)61 Back et al. (1992)65 Snowden et al. (2017)66
5. Model validation		Describe and clearly reference the data or knowledge used to validate the model and explain its relevance to the model context Plot model simulations overlaying the corresponding experimental data onto them with measures of potential/perceived variability (e.g., standard error bars, confidence intervals, shadows from ensemble simulations)		Anderson et al. (2007)43 Hicks et al. (2015)45 Lu et al. (2014)72 Kanodia et al. (2014)73 Ortega et al. (2013)74 Karelina et al. (2012)75 Peterson and Riggs (2012)76 Agoram (2014)78
6. Model results, application, and impact		Articulate a clear answer to the questions originally posed for the model (see 1. Purpose and context of the model in this table) Provide the simulation plots and/or outcome numerical values that underpin those answers Qualify the type of knowledge acquired through the modeling exercise: a positive new discovery, a confirmation, and/or a realization of a misconception. Describe the decisions that the modeling exercise enabled for the different stakeholders (user, domain expert, academic, industry, regulatory)—qualitatively and, whenever possible, quantitatively Describe the impact of the QSP modeling exercise beyond the initial stakeholders, especially if the impact is societal and/or can be translated into financial figures		Marshall et al. (2016)41 Shepard (2011)79 Peterson & Riggs (2015)11Hendricks (2013)77 Kansal & Trimmer (2005)80 Milligan et al. (2013)81 Allerheiligen (2014)82 Bueters et al. (2013)83 Nayak et al. (2018)84

PharmML, pharmacometrics markup language; QSP, quantitative and systems pharmacology; SBGN, systems biology graphical notation; SBML, systems biology markup language.