Table 3:

Metrics that in-scope articles used to evaluate both point and density forecasts. A preferred term is listed (metric column), given an abbreviation and related names reported. Whether the evaluative metric operates on a continuous or binary variable is stated and the computational formula presented.Question

Metric	Abbreviation	Other names	Binary or Continuous target	Formula
Absolute Loss	AS	-	Categorical	$|P\left(F_i\right)-O_i|$
Quadratic Loss	QS	-	Categorical	${\left[P\left(F_i\right)-O_i\right]}^2$
Prediction Accuracy	PA	-	Categorical	$N^{-1}{\displaystyle {\sum }_{i=1}^N}{\displaystyle \sum _{}^{}}1\left(F_i=O_i\right)$
Brier Score	BS	-	Categorical	$N^{-1}{\displaystyle {\sum }_{i=1}^N{\left[P\left(F_i\right)-O_i\right]}^2}$
Mean Error	ME	-	Continuous	$N^{-1}{\displaystyle {\sum }_{i=1}^N\left(F_i-O_i\right)}$
Mean Absolute Error	MAE	Mean Absolute Deviation (MAD)	Continuous	$N^{-1}{\displaystyle {\sum }_{i=1}^N\left|\right(O_i-F_i\left)\right|}$
Mean Absolute	MAPE	Mean Percent Error (MPE)	Continuous	$N^{-1}{\displaystyle {\sum }_{i=1}^N\left|\left(F_i/O_i-1\right)\right|}$
Percent Error		Average percentage error (APE)
Mean Squared Error	MSE	-	Continuous	$N^{-1}{\displaystyle {\sum }_{i=1}^N{\left(F_i-O_i\right)}^2}$
Root mean squared error	RMSE	Root mean squared prediction error (RMSPE)	Continuous	$\sqrt{N^{-1}{\displaystyle {\sum }_{i=1}^N{\left(F_i-O_i\right)}^2}}$
Proportion higher density	PHD		Continuous	$N^{-1}{\displaystyle {\sum }_{i=1}^N\mathbb{1}\left\{P\left[F\left(x_i\right)\right]>P\left[G\left(x_i\right)\right]\right\}}$
95% Coverage probability	CP	-	Continuous	$N^1{\displaystyle {\sum }_{i=1}^N\mathbb{1}\left(F_{2.5}<O_i<F_{97.5}\right)}$
Judgemental Adjustment	JA	-	Continuous	$\left(F_i-G_i\right)/G_i$
Forecast Improvement	FCIMP	-	Continuous	$\left(\left|O_i-F_i\right|-\left|O_i-G_i\right|\right)/O_i$