Table 3.

Simulation results comparing model selection procedures. Table entries include the median model error (ME), false positive (FP) rate, and false negative (FN) rate. Cost-scale coefficient estimation is a function of the calibration coefficient vector.

n	σ	Regression & Variable Selection Procedures	Calibration Coefficients	Time-scale Estimates			Cost-scale Estimates
				ME	FP	FN	ME	FP	FN
60	3	Unregularized	ϑ̂0	1.65	1.00	0	2.39	1.00	0
		Oracles	ϑ̂𝒜	0.48	0	0	0.64	0	0
		Time-scale only	ϑ̂λ	1.47	0.17	0.34	-	-	-
		Cost- & time-scale	ϑ̃λ	1.33	0.31	0.22	1.80	0.38	0.04
		Cost-scale only	ϑ0	0	0	0	1.68	0.39	0.04
			ϑ̂𝒜	0.48	0	0	1.65	0.38	0.05
			ϑ̂0	1.65	1.00	0	1.70	0.37	0.04
60	1	Unregularized	ϑ̂0	0.19	1.00	0	0.22	1.00	0
		Oracles	ϑ̂𝒜	0.06	0	0	0.06	0	0
		Time-scale only	ϑ̂λ	0.10	0.14	0	-	-	-
		Cost- & time-scale	ϑ̃λ	0.10	0.19	0	0.21	0.23	0
		Cost-scale only	ϑ0	0	0	0	0.14	0.09	0
			ϑ̂𝒜	0.06	0	0	0.12	0.11	0
			ϑ̂0	0.19	1.00	0	0.10	0.10	0
90	1	Unregularized	ϑ̂0	0.09	1.00	0	0.15	1.00	0
		Oracles	ϑ̂𝒜	0.03	0	0	0.05	0.00	0
		Time-scale only	ϑ̂λ	0.04	0.07	0	-	-	-
		Cost- & time-scale	ϑ̃λ	0.06	0.14	0	0.14	0.19	0
		Cost-scale only	ϑ0	0	0	0	0.07	0.03	0
			ϑ̂𝒜	0.03	0	0	0.07	0.04	0
			ϑ̂0	0.09	1.00	0	0.06	0.04	0