Table 5. Summary of observations on considerations for PSM credibility assessment in relation to V&V40 credibility factors and gradations.

PSMs as clinical tools (PSM-CT) or PSM virtual cohort studies (PSM-VC) are considered separately.

Category	V&V40 Credibility factor	PSM-CT	PSM-VC
Verification	Software quality assurance	No unique PSM considerations identified.
	Numerical code verification
	Discretization error	Gradations could account for number of patients these activities are performed on (see Table 6 & S1 Text, Section S2).
	Numerical solver Error
	Use error	For PSMs, with manual stages, there is a possibility of user variability related to subjectively chosen inputs. See S1 Text, Section S2 for example gradation accounting for this.	No unique PSM considerations identified.
Validation–model	Model form	No unique PSM considerations identified.
	Model inputs–quantification of sensitivities	For PSMs, sensitivity analysis and uncertainty quantification are intimately linked, so an alternative approach is have a single sensitivity analysis and uncertainty quantification factor with possible subfactors: • Inputs analyzed • Rigor of input uncertainty characterization • Number of patients • Output quantities analyzed
	Model inputs–quantification of uncertainties
Validation -comparator	Quantity of Test Samples	There may be unique PSM considerations, but these will be dependent on the specific validation activities performed. For some cases where a PSM is validated against clinical data, these factors could be interpreted as: • Number of validation subjects • Range of characteristics of validation subjects • Patient data • Patient measurements See S1 Text, Section S2 for example gradations.
	Range of Characteristics of Test Samples
	Characteristics of Test Samples
	Measurements of Test Samples
	Quantity of Test Conditions	There may be unique PSM considerations, but these will be dependent on the specific validation activities performed.
	Range of Test Conditions
	Measurements of Test Conditions
	Uncertainty of Test Condition Measurements
Validation—comparison	Equivalency of Input Parameters	No unique PSM considerations identified.
	Output Comparison–quantity
	Equivalency of Output Parameters
	Agreement of Output Comparison
	Rigor of Output Comparison
Applicability	Relevance of the QOIs
	Relevance of the Validation Activities to the COU	For PSMs, assessing the relevance of the validation subjects to the full patient population / full virtual cohort is a component of applicability assessment