Table 6.

List of XAI studies with time series data performing clinical validation of explanations

References	# Cit.	Application	Input Data	AI model(s)	XAI method(s)	Dataset(s)
Shashikumar et al. (2021)	15	Sepsis prediction	Longitudinal EHR	WCPH-RNN	Saliency	Retrospective study
Sun et al. (2021)	1	Survival prediction	Longitudinal EHR	RNN, GNN	Attention	WHAS Lemeshow et al. (2011), SUPPORT Knaus et al. (1995), METABRIC Curtis et al. (2012), MIMIC-III, PLAGH
Zheng et al. (2020)	9	Mortality prediction, AKI prediction	Longitudinal EHR	TITV network	FiLM, attention	NUH-AKI, MIMIC-III
Kwon et al. (2018)	196	HF diagnosis	Longitudinal EHR	RETAIN variant	Attention	HIRA-NPS Kim et al. (2014)
Ivaturi et al. (2021)	4	AF detection	ECG	CNN	Feature ablation/permutation, LIME, guided saliency	PhysioNet 2017 Clifford et al. (2017)
Mousavi et al. (2020)	42	AF detection	ECG	Bi-LSTM ensemble	Attention	PhysioNet 2017, MIT-BIH AFIB${}^1$
Filtjens et al. (2021)	3	pre-FoG movement detection in PD patients	3-D kinematic joint trajectories	CNN	LRP	Gait dataset Spildooren et al. (2010)