Table 4.
Summary of automated spindle detection results in the research literature and in this study.
| Study | Spindle assessment | Participants and data collected | Database | Algorithm requires hypnogram | Spindle detector TP evaluation | |||
|---|---|---|---|---|---|---|---|---|
| Sensitivity (%) | Specificity (%) | FDR (%) | Weighted Cohen kappa | |||||
| Schonwald et al., 2006 | 81.2 | 81.2 | N/R | N/R | 9 healthy adults, extracted 24 segments from each subject using 20 s epochs, removed epochs with artifacts | Private (N = 9) | Yes | Second-by-second analysis |
| Huupponen et al., 2007 | 70.0 | 98.6 | 32 | N/R | 12 healthy adults, entire night recordings | Private (N = 12) | Yes | The absolute difference between the detected spindle onset and the spindle onset determined by the experts was less than 0.5 s. |
| Causa et al., 2010 | 88.2 | 89.7 | 11.9 | N/R | 56 healthy children overnight recordings, 27 recordings used for training, 10 recordings for validation, and 19 for testing performance | Private (N = 56) | No | At least 75% spindle duration overlap between detected and expert assessed spindle |
| Warby et al. (2014) applying a1 | 74 | 81 | 89 | N/R | 110 healthy adults, (4 min of artifact-free stage 2 sleep from 100 subjects and ~38 min of stage 2 sleep from 10 subjects) | Private (N = 110) | Yes | At least 20% spindle duration overlap between detected and expert assessed spindle |
| Warby et al. (2014) applying a2 | 17 | 99 | 48 | N/R | See above entry | Private (N = 110) | Yes | See above entry |
| Warby et al. (2014) applying a3 | 71 | 81 | 89 | N/R | See above entry | Private (N = 110) | Yes | See above entry |
| Warby et al. (2014) applying a4 | 43 | 98 | 58 | N/R | See above entry | Private (N = 110) | Yes | See above entry |
| Warby et al. (2014) applying a5 | 33 | 99 | 44 | N/R | See above entry | Private (N = 110) | Yes | See above entry |
| Warby et al. (2014) applying a6 | 57 | 96 | 70 | N/R | See above entry | Private (N = 110) | Yes | See above entry |
| Devuyst et al., 2011 | 70.2 | 98.6 | N/R | N/R | 8 diagnosed with various sleep disorders (30 min segments), two raters for all signals; one rater only for two signals. Use only six signals and only cases where raters agree | DREAMS sleep spindle database (publicly available) (N = 6) | No | N/R |
| Nonclercq et al., 2013 | 75.1 | 96.7 | N/R | N/R | See above entry | DREAMS (N = 6) | No | There is overlap between the duration of the detected spindle and the spindle duration assessed by experts |
| Present study a1 | 56.0 | 82.4 | 86.8 | 0.37 | 8 from various sleep disorders (30 min segments), two raters for all signals; one rater only for two signals. Use all eight signals including “difficult” cases where raters do not agree | DREAMS (N = 8) | Yes | The absolute difference between the detected spindle onset and the spindle onset determined by the experts was less than 0.5 s |
| Present study a2 | 14.4 | 99.3 | 48.2 | 0.17 | See above entry | DREAMS (N = 8) | Yes | See above entry |
| Present study a3 | 77.7 | 81.4 | 76.1 | 0.55 | See above entry | DREAMS (N = 8) | Yes | See above entry |
| Present study a4 | 57.9 | 97.1 | 53.1 | 0.59 | See above entry | DREAMS (N = 8) | Yes | See above entry |
| Present study a5 | 45.1 | 97.9 | 49.1 | 0.47 | See above entry | DREAMS (N = 8) | Yes | See above entry |
| Present study a6 | 75.8 | 84.1 | 80.0 | 0.55 | See above entry | DREAMS (N = 8) | Yes | See above entry |
| Present study a7 | 75.9 | 91.8 | 66.7 | 0.66 | See above entry | DREAMS (N = 8) | No | See above entry |
| Present study a8 | 83.2 | 74.9 | 85.3 | 0.50 | See above entry | DREAMS (N = 8) | No | See above entry |
| Present study a1 | 65.5 | 85.1 | 82.7 | 0.46 | 19 overnight PSG from healthy controls; two raters for 15 signals, one rater for four signals | MASS database S2 (publicly available) (N = 19) | Yes | See above entry |
| Present study a2 | 16.5 | 99.2 | 49.5 | 0.20 | See above entry | MASS (N = 19) | Yes | See above entry |
| Present study a3 | 73.5 | 78.2 | 75.3 | 0.46 | See above entry | MASS (N = 19) | Yes | See above entry |
| Present study a4 | 66.2 | 97.5 | 48.1 | 0.64 | See above entry | MASS (N = 19) | Yes | See above entry |
| Present study a5 | 41.3 | 98.8 | 45.3 | 0.43 | See above entry | MASS (N = 19) | Yes | See above entry |
| Present study a6 | 73.0 | 90.5 | 69.1 | 0.60 | See above entry | MASS (N = 19) | Yes | See above entry |
| Present study a7 | 83.8 | 90.2 | 82.6 | 0.64 | See above entry | MASS (N = 19) | No | See above entry |
| Present study a8 | 77.2 | 76.9 | 86.5 | 0.46 | See above entry | MASS (N = 19) | No | See above entry |
Sensitivity (%) = TP/(TP + FN), Specificity (%) = TN/(TN + FP), False Discovery Rate (FDR) (%) = FP/(TP + FP). TP stands for true positive, TN for true negative, FP for false positive, and FN for false negative. The last column briefly explains the method used to assess how the automatic sleep spindle detector was deemed to succeed in detecting the spindle as registered by the experts. See Section Evaluation of Sleep Spindle Detection Algorithms for more details.