Table A.1:

Training Details and Hyperparamters for Experiments.

Experiment	Synthetic Dataset
Main Network Main Network Layer Size Prediction Network Temporal Module Optimization	2-Layer FCN ReLU Activation (input-dimension)2→16→3(embedding-space) Linear Layer with Softmax Activation 3→2 1-Layer LSTM 3→3 Optimizer: Adam, Learning Rate: 1e-4, Number of Epochs: 20, Batch-Size: 32
Experiment	MIMIC
Main Network Main Network Layer Size Prediction Network Temporal Module Optimization	2-Layer FCN ReLU Activation (input-dimension)55→32→32(embedding-space) Linear Layer with Softmax Activation 32→3 1-Layer LSTM 32→32 Optimizer: Adam, Learning Rate: 1e-4, Number of Epochs: 100, Batch-Size: 128
Experiment	ADNI
Main Network Main Network Layer Size Prediction Network Temporal Module Optimization	2-Layer FCN ReLU Activation (input-dimension)21→50→16(embedding-space) Linear Layer with Softmax Activation 16→3 1-Layer LSTM 16→16 Optimizer: Adam, Learning Rate: 1e-4, Number of Epochs: 100, Batch-Size: 128