Table 3.

Performance Metrics of Life Expectancy (LE) Predictions for 5-Year Mortality and Overall Survival Time by Physicians, the Chicago Model, the Average of the Physician and the Chicago Model, the “And” Model, and the “Or” Model

	5-Year Mortality					Overall Survival
	Sensitivity	Specificity	PPVa	NPVb	c-Statistic ± SE (95% CI)c	Harrell’s c-Statistic ± SE (95% CI)d	IBSe
Physicianf	0.53	0.76	0.41	0.83	0.69 ± 0.027 (0.67–0.74)	0.65 ± 0.027 (0.64–0.74)	0.148
Chicago modelg	0.47	0.75	0.37	0.82	0.68 ± 0.028 (0.63–0.78)	0.66 ± 0.028 (0.63–0.74)	0.147
Average modelh	0.35	0.87	0.46	0.81	0.73 ± 0.026 (0.68–0.78)	0.69 ± 0.026 (0.68–0.78)	0.140
And modeli	0.26	0.90	0.45	0.79	0.58 ± 0.023 (0.53–0.63)	NAj	NAj
Or modelk	0.74	0.60	0.37	0.88	0.67 ± 0.024 (0.62–0.72)	NAj	NAj

Note: PPV = positive predictive value; NPV = negative predictive value; SE = standard error; CI = confidence interval; IBS = integrated Brier score.

^a.Positive predictive value: true positives/(true positives + false positives).

^b.Negative predictive value: true negatives/(true negatives + false negatives).

^c.Area under the receiver operating characteristic (ROC) curve (AUC or c-statistic). The c-statistic is equivalent to the area under the receiver-operating characteristic curve, incorporating both sensitivity and specificity (a c-statistic of 0.5 indicates predictive value no better than random chance and c-statistic of 1.0 indicates perfect prediction).

^d.Harrell’s c-statistic.²⁹

^e.Integrated Brier score, a predictive accuracy score function that takes one values between 0 and 1, with lower values indicating better predictive performance.^30,31

^f.Physician’s answer on a patient-specific questionnaire to the question, “How many years do you estimate that this patient will live?” The physician’s answer to this question was converted to a binary indicator of predicted five-year mortality when the answer to this question was 5 years or less.

^g.Chicago model generates a point estimate of the patient’s LE which is turned into a binary indicator of 5-year mortality when it was equal to 5 years or less.

^h.Average model takes the mean of the point estimates generated by the physician and the Chicago model and uses this as its predictor. The average model is also converted into a binary indicator of predicted 5-year mortality when the average is equal to 5 years or less.

^i.“And” model predicts 5-year mortality when both the physician and the Chicago model predict 5-year mortality.

^j.NA = not applicable. Performance metrics for overall survival time cannot be computed for these models because they generate a binary classification of limited LE rather than a point estimate of LE.

^k.“Or” model predicts 5-year mortality when either the physician or the Chicago model predict 5-year mortality.