Table 3.

Type-I-error for the NPDE-based model evaluation approach (positive-control simulation study)

Number of Subjects	Samples per Subject	Total Samples	Iterationsa	P-value (μNPDE)b	P-value (NPC)c
5	3	15	467	0.028	0.61
10	3	30	491	0.047	0.894
20	3	60	498	0.042	1
30	3	90	500	0.06	1
5	6	30	493	0.053	0.85
10	6	60	496	0.026	0.974
20	6	120	498	0.038	0.998
30	6	180	497	0.038	1
5	9	45	492	0.03	0.868
10	9	90	497	0.054	0.988
20	9	180	499	0.036	1
30	9	270	499	0.056	1

μ_NPDE mean of normalized prediction distribution errors; NPC, numerical predictive check

^aIterations (out of 500) where the standard deviation of NPDE were not optimized to within ±0.01 of the target value (i.e., 1) by the developed fitting algorithm were excluded from the analysis.

^bType-I-error for the NPDE-based model evaluation approach. Computed as the proportion of iterations where the μ_NPDE was statistically different than 0 (p-value<0.05; two-sided Student’s t-test).

^cType-I-error for the conventional evaluation approach. Computed as the proportion of iterations where the proportion of observed data exceeding the model’s 90% prediction interval was statistically > 0.10 (p-value < 0.05; exact binomial test)