TABLE 2.
Performance Results for the Four Models Predicting Three Toxicity Outcomes and OS in the 20% Test Set and in the Three Largest Cancer Diagnosis Groups
| Outcome Modeled | Diagnosis (N) | AUC (95% bootstrap CI) | Sensitivity (95% bootstrap CI) | Specificity (95% bootstrap CI) | PPV (95% bootstrap CI) | NPV (95% bootstrap CI) | IR, Half the Patients on the Basis of Model Predictions, % | IR, Random Selection of Patients, % |
|---|---|---|---|---|---|---|---|---|
| Pneumonitis 1y | All cancer types (n = 416) | 0.739 (0.638 to 0.823) | 0.711 (0.561 to 0.845) | 0.701 (0.653 to 0.746) | 0.193 (0.129 to 0.261) | 0.960 (0.937 to 0.981) | 3.8 | 9.1% |
| Melanoma (n = 166) | 0.590 (0.353 to 0.841) | 0.444 (0.125 to 0.800) | 0.917 (0.874 to 0.956) | 0.235 (0.059 to 0.462) | 0.966 (0.936 to 0.993) | 6.0 | 5.4% | |
| Lung cancer (n = 111) | 0.723 (0.600 to 0.840) | 0.952 (0.846 to 1.000) | 0.256 (0.172 to 0.346) | 0.230 (0.146 to 0.317) | 0.958 (0.867 to 1.000) | 10.7 | 18.9 | |
| GU cancer (n = 67) | 0.845 (0.686 to 0.960) | 0.857 (0.500 to 1.000) | 0.617 (0.484 to 0.741) | 0.207 (0.069 to 0.357) | 0.974 (0.914 to 1.000) | 0.0 | 10.4 | |
| Hepatitis 1y | All (n = 392) | 0.729 (0.655 to 0.804) | 0.455 (0.333 to 0.574) | 0.859 (0.822 to 0.899) | 0.395 (0.284 to 0.508) | 0.886 (0.851 to 0.921) | 8.2 | 16.8 |
| Melanoma (n = 136) | 0.567 (0.373 to 0.777) | 0.214 (0.000 to 0.455) | 0.893 (0.838 to 0.943) | 0.188 (0.000 to 0.400) | 0.908 (0.850 to 0.958) | 8.8 | 10.3 | |
| Lung cancer (n = 107) | 0.884 (0.792 to 0.965) | 0.500 (0.200 to 0.818) | 0.907 (0.845 to 0.959) | 0.357 (0.125 to 0.636) | 0.946 (0.896 to 0.989) | 0.0 | 9.3 | |
| GU cancer (n = 53) | 0.639 (0.376 to 0.863) | 0.444 (0.111 to 0.777) | 0.795 (0.674 to 0.909) | 0.308 (0.077 to 0.583) | 0.875 (0.769 to 0.958) | 15.4 | 17.0 | |
| Colitis 1y | All (n = 427) | 0.755 (0.638 to 0.856) | 0.750 (0.581 to 0.909) | 0.608 (0.563 to 0.654) | 0.134 (0.086 to 0.182) | 0.968 (0.943 to 0.988) | 3.3 | 7.5 |
| Melanoma (n = 161) | 0.747 (0.631 to 0.850) | 0.957 (0.857 to 1.000) | 0.174 (0.112 to 0.238) | 0.162 (0.100 to 0.223) | 0.960 (0.864 to 1.000) | 5.0 | 14.3 | |
| Lung cancer (n = 112) | 0.853 (0.717 to 0.964) | 0.750 (0.000 to 1.000) | 0.833 (0.764 to 0.900) | 0.143 (0.000 to 0.308) | 0.989 (0.958 to 1.000) | 0.0 | 3.6 | |
| GU cancer (n = 68) | 0.560 (0.447 to 0.672) | 0.000 (0.000 to 0.000) | 0.806 (0.712 to 0.894) | 0.000 (0.000 to 0.000) | 0.982 (0.934 to 0.983) | 0.0 | 1.5 | |
| OS 1y | All (n = 403) | 0.752 (0.706 to 0.796) | 0.817 (0.764 to 0.865) | 0.558 (0.487 to 0.627) | 0.664 (0.609 to 0.719) | 0.741 (0.673 to 0.806) | 67.8 | 51.6 |
| Melanoma (n = 158) | 0.794 (0.716 to 0.861) | 0.917 (0.861 to 0.868) | 0.500 (0.362 to 0.639) | 0.798 (0.723 to 0.866) | 0.735 (0.585 to 0.882) | 83.5 | 68.4 | |
| Lung cancer (n = 115) | 0.681 (0.580 to 0.781) | 0.674 (0.533 to 0.814) | 0.611 (0.500 to 0.714) | 0.509 (0.387 to 0.630) | 0.759 (0.642 to 0.8680 | 50.0 | 37.4 | |
| GU cancer (n = 63) | 0.741 (0.600 to 0.858) | 0.742 (0.576 to 0.889) | 0.625 (0.448 to 0.814) | 0.657 (0.487 to 0.814) | 0.714 (0.542 to 0.880) | 65.6 | 49.2 |
NOTE. Detailed interpretation of the results can be found in the Data Supplement.
Abbreviations: 1y, 1-year prediction time window; GU, genitourinary; IR, incidence rate; NPV, negative predictive value; OS, overall survival; PPV, positive predictive value.