Table 3. Detailed evaluation results.
In bold best accuracy values. CLFS and ELM metrics are similarity measures the biggest the best, MAE and EMD are distance measures the lowest the best.
| Metric | Size | Type of source | Event log | GRU | LSTM | LSTM–GAN | SIMOD |
|---|---|---|---|---|---|---|---|
| CLFS | LARGE | REAL | POC | 0.63141 | 0.67176 | 0.28998 | |
| BPI17W | 0.63751 | 0.71798 | 0.36629 | 0.58861 | |||
| BPI12W | 0.58375 | 0.70228 | 0.35073 | 0.53744 | |||
| CALL | 0.82995 | 0.83043 | 0.24055 | 0.62911 | |||
| SYNTHETIC | CVS | 0.83369 | 0.85752 | 0.20898 | 0.71359 | ||
| CFM | 0.81956 | 0.60224 | 0.11412 | 0.77094 | |||
| SMALL | REAL | INS | 0.50365 | 0.51299 | 0.25619 | 0.61034 | |
| ACR | 0.78413 | 0.78879 | 0.18073 | 0.67959 | |||
| MP | 0.27094 | 0.23197 | 0.06691 | 0.34596 | |||
| SYNTHETIC | CFS | 0.69543 | 0.66782 | 0.10157 | 0.76648 | ||
| P2P | 0.41179 | 0.65904 | 0.13556 | 0.45297 | |||
| MAE | LARGE | REAL | POC | 801147 | 778608 | 603105 | |
| BPI17W | 868766 | 603688 | 828165 | 961727 | |||
| BPI12W | 701892 | 327350 | 653656 | 662333 | |||
| CALL | 160485 | 174343 | 159424 | 679847 | |||
| SYNTHETIC | CVS | 859926 | 667715 | 952004 | 1067258 | ||
| CFM | 25346 | 15078 | 956289 | 252458 | |||
| SMALL | REAL | INS | 1586323 | 1516368 | 1302337 | 1090179 | |
| ACR | 344811 | 341694 | 296094 | 230363 | |||
| MP | 335553 | 321147 | 210714 | 298641 | |||
| SYNTHETIC | CFS | 30327 | 33016 | 717266 | 15297 | ||
| P2P | 2407551 | 2495593 | 2347070 | 1892415 | |||
| ELS | LARGE | REAL | POC | 0.58215 | 0.65961 | 0.28503 | |
| BPI17W | 0.63643 | 0.70317 | 0.35282 | 0.58412 | |||
| BPI12W | 0.57862 | 0.67751 | 0.33649 | 0.52555 | |||
| CALL | 0.79336 | 0.81645 | 0.19123 | 0.59371 | |||
| SYNTHETIC | CVS | 0.65160 | 0.70355 | 0.16854 | 0.70154 | ||
| CFM | 0.68292 | 0.43825 | 0.09505 | 0.66301 | |||
| SMALL | REAL | INS | 0.49625 | 0.50939 | 0.23070 | 0.57017 | |
| ACR | 0.75635 | 0.45737 | 0.15884 | 0.71977 | |||
| MP | 0.25019 | 0.21508 | 0.04570 | 0.31024 | |||
| SYNTHETIC | CFS | 0.54433 | 0.57392 | 0.07930 | 0.67526 | ||
| P2P | 0.22923 | 0.39249 | 0.09968 | 0.43202 | |||
| EMD | LARGE | REAL | POC | 0.00036 | 0.00011 | 0.00001 | |
| BPI17W | 0.00060 | 0.01010 | 0.00072 | 0.00057 | |||
| BPI12W | 0.00077 | 0.00061 | 0.00006 | 0.00002 | |||
| CALL | 0.00084 | 0.15794 | 0.00090 | 0.00072 | |||
| SYNTHETIC | CVS | 0.61521 | 0.57217 | 0.40006 | 0.13509 | ||
| CFM | 0.00472 | 0.00828 | 0.03529 | 0.06848 | |||
| SMALL | REAL | INS | 0.03343 | 0.00308 | 0.33336 | 0.00001 | |
| ACR | 0.49996 | 0.68837 | 0.25012 | 0.58674 | |||
| MP | 0.12609 | 0.33375 | 0.28577 | 0.31411 | |||
| SYNTHETIC | CFS | 0.08253 | 0.10784 | 0.06924 | 0.03461 | ||
| P2P | 0.25306 | 0.33747 | 0.23898 | 0.03888 |