Table 7.
Model Performance at Sample Size 10% of Primary Analyses
| Performance Measure | SPCA | Regularization | Boosting | Naïve MLE |
|---|---|---|---|---|
| Scale Length (number of items) | 9 | 6 | 11 | 57 |
| Training/Test Sample N = 247 | 57 | |||
| Peak Accuracy (% Correct Classification) | 65% | 62% | 69% | 78% |
| Point Biserial Correlation | .28 | .29 | .42 | .59 |
| Spearman Correlation | .28 | .28 | .41 | .60 |
| Pseudo R2 | .08 | .10 | .18 | .39 |
| HR for +1 SD Scale Score | 1.53 | 1.56 | 1.80 | 3.8 |
| AUC | .66 | .60 | .74 | .85 |
| Hold-Out Sample N = 124 | ||||
| Peak Accuracy (% Correct Classification) | 64% | 63% | 62% | 63% |
| Biserial Correlation | .26 | .26 | .24 | .30 |
| Spearman Correlation | .27 | .26 | .25 | .30 |
| Pseudo R2 | .07 | .10 | .04 | 0 |
| HR for +1 SD Scale Score | 1.46 | 1.58 | 1.29 | 1.51 |
| AUC | .66 | .65 | .64 | .68 |