Table 4.
ROC values estimated with the direct ROC modeling (DM) and ordinal regression (OR) algorithms applied to the Breast Cancer Surveillance Consortium mammography data. Shown are point estimates and confidence intervals in parentheses. Confidence intervals were calculated using 500 bootstrapped samples. Covariate effects are denoted by ‘ignored’ when X = breast density is not included in the model, by ‘stratified nonparametric’ when FPR estimates pertaining to each category of breast density are calculated without modeling assumptions, and as ‘ordinal model’ when an ordinal regression model is assumed for effects of the covariate on FPRs or equivalently on θrX . The binormal ROC model allows X to affect intercept but not the slope (equation (5)). Results for the DM estimation algorithm correspond to curves shown in Figure 1.
| Covariate Effects Modeled | |||||||
|---|---|---|---|---|---|---|---|
| ROC | False Positive Rates |
Breast Density Category |
Estimation Method |
AUC | ROC(0.1) | ROC(0.3) | ROC(0.5) |
| ignored | ignored | --- | DM | 0.929 (0.914,0.940) | 0.786 (0.751,0.823) | 0.939 (0.911,0.958) | 0.981 (0.961,0.990) |
| --- | OR | 0.921 (0.907,0.934) | 0.787 (0.751,0.824) | 0.917 (0.893,0.939) | 0.963 (0.942,0.978) | ||
| Binormal | stratified | Not dense | DM | 0.955 (0.939,0.966) | 0.867 (0.826,0.903) | 0.968 (0.940,0.982) | 0.991 (0.973,0.996) |
| Model | nonparametric | Not dense | OR | 0.950 (0.933,0.962) | 0.865 (0.829,0.897) | 0.951 (0.927,0.968) | 0.979 (0.960,0.989) |
| Binormal | stratified | Medium | DM | 0.903 (0.879,0.925) | 0.714 (0.652,0.788) | 0.905 (0.863,0.939) | 0.966 (0.933,0.984) |
| Model | nonparametric | Medium | OR | 0.895 (0.868,0.920) | 0.731 (0.670,0.795) | 0.879 (0.841,0.914) | 0.939 (0.909,0.964) |
| Binormal | stratified | Extreme | DM | 0.848 (0.765,0.913) | 0.571 (0.419,0.747) | 0.822 (0.696,0.917) | 0.925 (0.830,0.976) |
| Model | nonparametric | Extreme | OR | 0.820 (0.742,0.893) | 0.578 (0.457,0.733) | 0.773 (0.665,0.876) | 0.871 (0.783,0.940) |
| Binormal | ordinal model | Not dense | DM | 0.951 (0.936,0.962) | 0.857 (0.820,0.891) | 0.965 (0.937,0.979) | 0.990 (0.974,0.996) |
| Model | Not dense | OR | 0.943 (0.927,0.954) | 0.842 (0.809,0.874) | 0.944 (0.919,0.962) | 0.977 (0.958,0.988) | |
| Binormal | ordinal model | Medium | DM | 0.908 (0.886,0.928) | 0.727 (0.666,0.794) | 0.912 (0.873,0.943) | 0.970 (0.942,0.984) |
| Model | Medium | OR | 0.902 (0.879,0.924) | 0.741 (0.684,0.799) | 0.891 (0.856,0.922) | 0.949 (0.922,0.970) | |
| Binormal | ordinal model | Extreme | DM | 0.844 (0.771,0.906) | 0.559 (0.414,0.734) | 0.816 (0.703,0.908) | 0.923 (0.844,0.969) |
| Model | Extreme | OR | 0.848 (0.778,0.908) | 0.622 (0.479,0.766) | 0.814 (0.713,0.897) | 0.903 (0.831,0.953) | |
ROC: receiver operating characteristic, FPR: false positive rate, AUC: area under the ROC curve.
DM: direct ROC modeling algorithm, OR: ordinal regression algorithm,