Algorithm 1 Profit-aware Reinforcement Learning State Construction

Require: Aligned multimodal sequence $\mathcal{S}={\left\{S_t\right\}}_{t=1}^T$, historical management actions $\mathcal{M}={\left\{M_t\right\}}_{t=1}^T$, intervention sequence $\mathcal{A}={\left\{A_t\right\}}_{t=1}^T$ Ensure: Profit-aware RL state sequence $\mathcal{X}={\left\{x_t\right\}}_{t=1}^T$   1:for  $t=1,\dots ,T$  do   2:       $z_t^s\leftarrow f_s\left(S_t\right)$   3:       $({\widehat{Y}}_t,{\widehat{C}}_t,{\widehat{R}}_t)\leftarrow g_{\mathrm{causal}}(S_t,A_t)$   4:       $z_t^e\leftarrow f_e({\widehat{Y}}_t,{\widehat{C}}_t,{\widehat{R}}_t)$   5:       $z_t^m\leftarrow f_m\left(M_t\right)$   6:       $x_t\leftarrow \left[z_t^s;z_t^e;z_t^m\right]$   7:       $a_t\leftarrow {\pi }_{\theta }\left(x_t\right)$   8:       $(r_t,x_{t+1})\leftarrow \mathcal{E}\left(a_t\right)$   9:       Update critic network 10:       Update actor network 11:end for 12:return  $\mathcal{X}$