Table 1:

The formulas for the six common calibration metrics. y_i denotes the actual outcome, ${\widehat{y}}_i$ represents the estimated risk probability, N indicates the total number of subjects, and G_g denotes the patients in group g.

Metric	Formula	Perfect Calibration
Brier score	$\frac{1}{N}{\displaystyle {\displaystyle \sum _{i=1}^N}{\left(y_i-{\widehat{y}}_i\right)}^2}$	0
Scaled Brier score	$1-\frac{{\displaystyle {\displaystyle \sum _{i=1}^N}{\left(y_i-{\widehat{y}}_i\right)}^2}}{N\overline{y}(1-\overline{y})}$	1
Hosmer-Lemeshow statistic	${\displaystyle {\displaystyle \sum _{g=1}^{10}}\frac{{\left({\displaystyle \sum _{i\in {\mathcal{G}}_g}{y}_i}-{\displaystyle \sum _{i\in {\mathcal{G}}_g}{\widehat{y}}_i}\right)}^2}{N_{\text{g}}{\pi }_g\left(1-{\pi }_g\right)}},\text{where}{\pi }_{\text{g}}=\frac{1}{N_g}{\displaystyle \sum _{i\in {\mathcal{G}}_g}{\widehat{y}}_i}$	1
Spiegelhalter z-statistic	$\frac{{\displaystyle {\displaystyle \sum _{i=1}^N}\left(y_i-{\widehat{y}}_i\right)}\left(1-2{\widehat{y}}_i\right)}{\sqrt{{\displaystyle {\displaystyle \sum _{i=1}^N}{\left(1-2{\widehat{y}}_i\right)}^2}{\widehat{y}}_i\left(1-{\widehat{y}}_i\right)}}$	1
Reliability-in-the-small	$\frac{1}{10}{\displaystyle {\displaystyle \sum _{g=1}^{10}}{\left(\frac{1}{N_g}{\displaystyle \sum _{i\in G_g}{\widehat{y}}_i}-\frac{1}{N_g}{\displaystyle \sum _{i\in G_g}{y}_i}\right)}^2}$	0
Reliability-in-the-large	${\left(\frac{1}{N}{\displaystyle {\displaystyle \sum _{i=1}^N}{\widehat{y}}_i}-\frac{1}{N}{\displaystyle {\displaystyle \sum _{i=1}^N}{y}_i}\right)}^2$	0