ROC - Test out for NNM ROCKIT (Windows95 version 0.9.1 BETA): Maximum Likelihood Estimation of a Binormal ROC Curve From CONTINUOUSLY-Distributed Test Results ----------------------------------------------------- Original input of 44 Actually-NEGATIVE cases ----------------------------------------------------- .870 .722 .829 .927 .908 .741 .931 .675 .727 .912 .513 .912 .614 .607 .531 .717 .829 .881 .868 .647 .701 .778 .629 .513 .601 .754 .614 .571 .607 .845 .862 .529 .784 .846 .777 .881 .539 .646 .576 .602 .784 .912 .613 .912 ---------------------------------------------------- Original input of 56 Actually-POSITIVE cases ---------------------------------------------------- .893 .859 .972 .949 .951 .882 .927 .791 .849 .883 .663 .959 .505 .913 .937 .783 .934 .570 .961 .951 .941 .704 .900 .978 .622 .938 .743 .622 .925 .883 .936 .619 .879 .820 .900 .931 .914 .733 .934 .964 .947 .931 .746 .951 .847 .927 .756 .929 .902 .890 .959 .888 .949 .934 .906 .928 ROCKIT (Windows95 version 0.9.1 BETA): ROC data by Philip Maximum Likelihood Estimation of the Parameters a Single Binormal ROC Curve Name of Input File being used: Clinical NN test in. Condition 1: ROC Total number of actually-negative cases = 44. Total number of actually-positive cases = 56. Data collected on a nominally continuous scale. Larger values of the test result represent stronger evidence that the case is actually-positive (e.g., that the patient is actually abnormal) Operating Points Corresponding to the Input Data Categorized by the LABROC5 Scheme: FPF: .000 .000 .023 .023 .045 .045 .159 .159 .205 .205 .273 TPF: .000 .339 .375 .411 .446 .500 .500 .679 .679 .696 .696 FPF: .273 .364 .409 .455 .477 .500 .614 .682 .886 1.000 TPF: .750 .750 .786 .804 .857 .875 .893 .964 .964 1.000 ----------------------------------------------------- Initial Estimates of the Binormal ROC Parameters: ----------------------------------------------------- a = 1.0272 b = .6808 z(k) = 1.193 .596 .296 .048 -.025 -.157 -.267 -.406 -.573 -.661 -.796 -.826 -.924 -1.242 -1.577 -1.701 -1.885 -1.984 -2.216 Procedure Converges after 4 Iterations ===================================================== Final Estimates of the Binormal ROC Parameters ===================================================== Binormal Parameters and Area Under the Estimated ROC : a = 1.0538 b = .6991 Area (Az) = .8061 Area (Wilc) = .8028 1: z(k) = -1.174 -.592 -.271 -.032 .044 .189 .294 .433 .576 .684 .795 .833 .911 1.299 1.516 1.669 1.831 1.947 2.136 Estimated Standard Errors and Correlation of these Values: Std. Err. (a) = .2055 Std. Err. (b) = .1363 Corr(a,b) = .3898 Std. Err. (Az) = .0427 Std. Err.(Wilc)= .0433 Symmetric 95% Confidence Intervals For a : ( .6510, 1.4565) For b : ( .4320, .9663) Asymmetric 95% Confidence Interval For Az: ( .7119, .8787) Variance-Covariance Matrix: =========================== a b z( 1) z( 2) z( 3) z( 4) z( 5) z( 6) z( 7) z( 8) z( 9) z(10) z(11) z(12) z(13) z(14) z(15) z(16) z(17) z(18) z(19) a .0422 b .0109 .0186 z( 1) .0170 .0095 .0559 z( 2) .0167 .0066 .0306 .0378 z( 3) .0167 .0047 .0236 .0290 .0334 z( 4) .0167 .0031 .0199 .0245 .0282 .0319 z( 5) .0167 .0026 .0189 .0233 .0269 .0304 .0317 z( 6) .0167 .0014 .0171 .0212 .0246 .0279 .0291 .0316 z( 7) .0168 .0005 .0159 .0199 .0232 .0263 .0275 .0299 .0319 z( 8) .0167 -.0008 .0145 .0184 .0215 .0245 .0256 .0279 .0298 .0327 z( 9) .0167 -.0024 .0131 .0170 .0200 .0229 .0240 .0263 .0281 .0308 .0340 z(10) .0166 -.0036 .0121 .0160 .0190 .0219 .0230 .0252 .0270 .0297 .0328 .0355 z(11) .0166 -.0050 .0110 .0150 .0181 .0209 .0220 .0242 .0260 .0286 .0318 .0344 .0374 z(12) .0165 -.0056 .0107 .0147 .0178 .0206 .0217 .0239 .0257 .0283 .0314 .0341 .0371 .0382 z(13) .0164 -.0067 .0099 .0140 .0171 .0200 .0211 .0233 .0251 .0278 .0309 .0335 .0365 .0376 .0400 z(14) .0156 -.0131 .0062 .0110 .0144 .0176 .0187 .0211 .0231 .0259 .0292 .0320 .0351 .0362 .0387 .0538 z(15) .0148 -.0174 .0040 .0093 .0130 .0165 .0177 .0203 .0223 .0254 .0289 .0318 .0351 .0363 .0389 .0547 .0659 z(16) .0141 -.0207 .0023 .0080 .0121 .0157 .0171 .0198 .0220 .0252 .0288 .0319 .0354 .0367 .0394 .0559 .0674 .0767 z(17) .0132 -.0245 .0004 .0067 .0111 .0150 .0164 .0193 .0217 .0251 .0290 .0323 .0359 .0373 .0402 .0575 .0696 .0792 .0906 z(18) .0125 -.0273 -.0009 .0057 .0104 .0145 .0160 .0190 .0215 .0251 .0292 .0326 .0364 .0378 .0409 .0589 .0714 .0813 .0930 .1020 z(19) .0111 -.0321 -.0033 .0041 .0092 .0137 .0153 .0186 .0213 .0251 .0296 .0333 .0374 .0389 .0422 .0615 .0747 .0853 .0975 .1070 .1240 Correlation Matrix: =================== a b z( 1) z( 2) z( 3) z( 4) z( 5) z( 6) z( 7) z( 8) z( 9) z(10) z(11) z(12) z(13) z(14) z(15) z(16) z(17) z(18) z(19) a 1.0000 b .3898 1.0000 z( 1) .3495 .2942 1.0000 z( 2) .4174 .2481 .6671 1.0000 z( 3) .4437 .1897 .5468 .8154 1.0000 z( 4) .4554 .1288 .4714 .7049 .8649 1.0000 z( 5) .4574 .1060 .4488 .6727 .8262 .9557 1.0000 z( 6) .4583 .0583 .4070 .6146 .7572 .8774 .9184 1.0000 z( 7) .4566 .0203 .3775 .5746 .7103 .8248 .8639 .9415 1.0000 z( 8) .4509 -.0339 .3391 .5238 .6516 .7595 .7964 .8696 .9247 1.0000 z( 9) .4408 -.0938 .3000 .4734 .5941 .6964 .7314 .8008 .8531 .9245 1.0000 z(10) .4301 -.1414 .2706 .4361 .5522 .6508 .6845 .7516 .8021 .8710 .9438 1.0000 z(11) .4166 -.1911 .2409 .3990 .5108 .6062 .6389 .7038 .7528 .8196 .8901 .9445 1.0000 z(12) .4113 -.2084 .2308 .3864 .4969 .5912 .6235 .6878 .7363 .8025 .8723 .9262 .9811 1.0000 z(13) .3996 -.2440 .2101 .3609 .4687 .5610 .5928 .6558 .7034 .7683 .8369 .8897 .9435 .9621 1.0000 z(14) .3278 -.4140 .1131 .2431 .3401 .4248 .4541 .5126 .5569 .6175 .6817 .7311 .7813 .7985 .8337 1.0000 z(15) .2814 -.4972 .0652 .1856 .2780 .3594 .3878 .4445 .4876 .5468 .6095 .6578 .7069 .7237 .7581 .9191 1.0000 z(16) .2482 -.5488 .0349 .1493 .2387 .3183 .3461 .4018 .4443 .5026 .5645 .6122 .6608 .6774 .7114 .8698 .9484 1.0000 z(17) .2138 -.5968 .0061 .1147 .2015 .2792 .3065 .3613 .4032 .4608 .5220 .5693 .6173 .6338 .6675 .8239 .9006 .9504 1.0000 z(18) .1901 -.6268 -.0123 .0926 .1776 .2542 .2811 .3354 .3768 .4340 .4948 .5418 .5896 .6060 .6395 .7948 .8705 .9192 .9673 1.0000 z(19) .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 .0000 Estimated Binormal ROC curve, with Lower and Upper Bounds of the Asymmetric Point-wise 95% Confidence Interval for True-Positive Fraction at a Variety of False-Positive Fractions: FPF TPF (Lower Bound, Upper Bound) .005 .2274 ( .0815 , .4604 ) .010 .2833 ( .1215 , .5086 ) .020 .3511 ( .1783 , .5624 ) .030 .3969 ( .2211 , .5970 ) .040 .4323 ( .2564 , .6230 ) .050 .4616 ( .2868 , .6442 ) .060 .4867 ( .3137 , .6623 ) .070 .5087 ( .3378 , .6780 ) .080 .5284 ( .3597 , .6921 ) .090 .5463 ( .3799 , .7048 ) .100 .5627 ( .3986 , .7165 ) .110 .5778 ( .4160 , .7272 ) .120 .5918 ( .4323 , .7373 ) .130 .6050 ( .4476 , .7467 ) .140 .6173 ( .4621 , .7555 ) .150 .6290 ( .4758 , .7639 ) .200 .6792 ( .5352 , .8003 ) .250 .7199 ( .5835 , .8300 ) .300 .7541 ( .6241 , .8551 ) .400 .8097 ( .6898 , .8959 ) .500 .8540 ( .7425 , .9274 ) .600 .8908 ( .7873 , .9520 ) .700 .9222 ( .8277 , .9710 ) .800 .9497 ( .8665 , .9852 ) .900 .9744 ( .9079 , .9949 ) .950 .9862 ( .9334 , .9982 ) Estimated Relationship between the Critical Test-Result Value (which separates 'positive' results form 'negative' results) and the Corresponding Operating Point on the Fitted Binormal ROC Curve: *********************************************************************** Critical Test ( FPF , TPF ) Result Value .9325 ( .016, .330) .9300 ( .026, .379) .9275 ( .034, .410) .9260 ( .048, .455) .9125 ( .065, .498) .9070 ( .097, .558) .8815 ( .181, .662) .8800 ( .203, .681) .8745 ( .213, .691) .8605 ( .247, .718) .8465 ( .282, .743) .8245 ( .332, .774) .7835 ( .384, .802) .7665 ( .425, .822) .7420 ( .482, .847) .7300 ( .513, .859) .6690 ( .607, .893) .6165 ( .723, .929) .5705 ( .880, .970)