Figure 2:

Framework of CAT-LSTM. (A) CNN with contrastive attention module. we adopt DenseNet and an attention module to represent an individual eye image as a vector $e_t^{c,a}$. Contrastive attention module removes the common feature $e_t^{c,p}$ and generate an abnormal feature vector $e_t^{c,d}$, which is used to predict AMD steps $y_t^c$ in current visit. (B) T-LSTM. We obtain t^th visit’s feature x_t by combining the CFP feature $e_t^{c,d}$, genotype feature e^g, sociodemographic feature $e_t^d$ and progression feature $e_t^p$ Then x_t and time gaps Δ_t between successive visits are sent to T-LSTM to model the temporal CFP sequences. T-LSTM output the AMD prediction results y_t at each time step. (C) Details of T-LSTM. (D) Progression embedding. We generate an AMD stage history vector based on previous visits and impute the missing AMD steps. Then we build a AMD progression time matrix to represent how many years it takes for the eye to progress from earlier AMD step to later AMD step. We flatten the matrix and concatenate it with the imputed AMD step history vector to generate the progression embedding $e_t^p$.