Table 4.
External model validation diagnostics*
| Model | Observed case count within SEPEA overall prediction intervals? (rank) | EIS (N=6) |
LAD (N=21) |
Mean ranking (rank of mean ranking) | ||
|---|---|---|---|---|---|---|
| Number correct (rank) | RMSE (rank) | Number correct (rank) | RMSE (rank) | |||
| Model 1 | No (5) | 3 (6) | 26.9 (6) | 18 (2) | 8.9 (6) | 5.0 (6) |
| Model 2 | No (5) | 4 (4) | 17.0 (4) | 18 (2) | 6.5 (4) | 4.8 (5) |
| Model 3 | Yes (1) | 5 (1) | 15.1 (2) | 17 (5) | 6.2 (3) | 2.4 (2) |
| Model 4 | Yes (1) | 4 (4) | 15.1 (2) | 17 (5) | 6.1 (1) | 2.6 (3) |
| Model 5 | Yes (1) | 5 (1) | 18.0 (5) | 18 (2) | 6.7 (5) | 2.8 (4) |
| Model 6 | Yes (1) | 5 (1) | 11.6 (1) | 19 (1) | 6.1 (1) | 1.0 (1) |
| Model 7 | No (5) | 2 (7) | 39.4 (7) | 13 (7) | 11.7 (7) | 6.6 (7) |
*For each diagnostic, models are placed in rank order (1=best model, 7=worst model) with ties given the same ranking. The mean ranking and rank provide an estimate of the overall performance of various models.
EIS, early intervention services; LAD, local authority district; RMSE, root mean squared error; SEPEA, Social Epidemiology of Psychoses in East Anglia.