Table 3.
Meth. | M | Scanner E | Scanner F | Scanner G | Scanner H | BWT | FWT |
---|---|---|---|---|---|---|---|
DM (Ours) | 128 | 0.722 ± 0.020 | 0.526 ± 0.021 | 0.592 ± 0.041 | 0.330 ± 0.015 | 0.030 ± 0.018 | 0.063 ± 0.016 |
DM-PD (Ours) | 128 | 0.750 ± 0.006 | 0.565 ± 0.067 | 0.624 ± 0.024 | 0.355 ± 0.038 | 0.028 ± 0.019 | 0.066 ± 0.030 |
Random | 128 | 0.752 ± 0.019 | 0.514 ± 0.021 | 0.600 ± 0.021 | 0.394 ± 0.013 | 0.007 ± 0.016 | 0.084 ± 0.026 |
Naive | 0.682 ± 0.014 | 0.506 ± 0.017 | 0.561 ± 0.020 | 0.369 ± 0.008 | 0.000 ± 0.008 | 0.091 ± 0.027 | |
GEM19 | 128 | 0.754 ± 0.012 | 0.568 ± 0.022 | 0.622 ± 0.038 | 0.366 ± 0.024 | 0.034 ± 0.016 | 0.067 ± 0.018 |
ER-MIR20 | 128 | 0.754 ± 0.012 | 0.588 ± 0.038 | 0.611 ± 0.039 | 0.363 ± 0.027 | 0.031 ± 0.016 | 0.075 ± 0.016 |
DSM | 0.653 ± 0.047 | 0.441 ± 0.074 | 0.643 ± 0.067 | 0.454 ± 0.096 | – | – | |
JModel | 0.716 ± 0.063 | 0.522 ± 0.114 | 0.711 ± 0.058 | 0.419 ± 0.087 | – | – | |
Base | 0.645 | 0.372 | 0.509 | 0.136 | – | – |
± indicates the interval over n = 5 independent runs with different seeds. Dynamic memory (DM) is compared to DM with a pseudo-domain module (DM-PD), naive continual learning, random replacement strategy (Random), domain-specific models (DSM), a joint model (JModel) and using base training only (Base). In addition, GEM and ER-MIR are shown for reference, noting that they require information about domain membership. For base training only one model was trained to avoid influence of base training results on subsequent continual training, therefore no standard deviations are indicated. For a visual presentation of the results, see Supplementary Fig. 2.