Skip to main content
. 2020 Nov 24;28(3):549–558. doi: 10.1093/jamia/ocaa283

Table 2.

Model performance measures across subpopulations: before and after fairness algorithm processing

PCE
FRAX
Performance measure Before After Before After
Expectation across subpopulations [CITL] 1.247 (1.227–1.277) 1.039 (1.025–1.058) 1.076 (1.065–1.089) 1.001 (0.992–1.012)
Variance across subpopulations (CITL) 0.500 (0.450–0.633) 0.006 (0.006–0.013) 0.070 (0.064–0.082) 0.004 (0.003–0.006)
CITL 20th percentile 0.805 (0.787–0.824) 0.983 (0.965–0.992) 0.865 (0.856–0.882) 0.962 (0.945–0.967)
CITL 50th percentile 1.068 (1.043–1.087) 1.019 (1.005–1.036) 1.004 (0.991–1.018) 0.997 (0.987–1.006)
CITL 80th percentile 1.492 (1.453–1.565) 1.090 (1.071–1.119) 1.249 (1.237–1.285) 1.034 (1.028–1.058)
Expectation across subpopulations [CS] 1.739 (1.698–1.781) 0.876 (0.855–0.898) 1.414 (1.377–1.449) 0.906 (0.884–0.927)
Variance across subpopulations (CS) 1.179 (1.048–1.382) 0.019 (0.018–0.030) 0.484 (0.434–0.560) 0.014 (0.014–0.023)
CS 20th percentile 0.884 (0.846–0.907) 0.772 (0.738–0.798) 0.835 (0.800–0.851) 0.819 (0.786–0.843)
CS 50th percentile 1.280 (1.235–1.322) 0.883 (0.866–0.910) 1.238 (1.151–1.276) 0.913 (0.898–0.933)
CS 80th percentile 2.885 (2.683–2.952) 0.969 (0.956–1.01) 2.027 (1.924–2.135) 0.982 (0.973–1.014)
Area Under the Receiver Operating Curve 0.730 (0.727–0.733) 0.736 (0.734–0.739) 0.712 (0.710–0.715) 0.714 (0.712–0.716)
Area Under the Precision-Recall Curve 0.109 (0.107–0.111) 0.116 (0.114–0.119) 0.180 (0.177–0.183) 0.183 (0.180–0.185)
Brier Score 0.043 (0.043–0.044) 0.043 (0.042–0.043) 0.068 (0.068–0.069) 0.068 (0.067–0.068)

Note: Values are point estimates and 95% confidence intervals, derived via the percentile bootstrap method with 200 repetitions.

Abbreviations: CITL, calibration in the large; CS, calibration slope; FRAX, fracture risk assessment tool; NA, not applicable; PCE, pooled cohort equations.