Algorithm 1 Training/Prediction Scheme of Our Hierarchical Framework

$$\begin{array}{cc}\hfill & 1:\text{Initialization:}\hfill \\ \hfill & \text{Split each conversation}C\text{into segments}{C}_1,C_2,\dots C_n\hfill \\ \hfill & \text{denote the session-level score of}C\text{as}s\hfill \\ \hfill & \text{initialize the segment quality scores as}{y}_1^0=s,y_2^0=s,\dots ,y_N^0=s\hfill \\ \hfill & 2:\hfill \\ \hfill & 3:\mathbf{for}k=0\text{to}K\mathbf{do}\hfill \\ \hfill & 4:\text{Fine-tune BERT by using the local quality scores}{y}_1^k,y_2^k,\dots ,y_N^k\text{with a}\hfill \\ \hfill & \text{regression task to learn representations of segments}.\hfill \\ \hfill & 5:\text{Feed segment representations into the LSTM-based model to train the}\hfill \\ \hfill & \text{segment quality estimatior (SQE) with a regression task}.\hfill \\ \hfill & 6:\text{Use the trained SQE to predict global quality}\widehat{s}\text{and local qualities}\hfill \\ \hfill & {\widehat{s}}_1,{\widehat{s}}_2,\dots ,{\widehat{s}}_N.\hfill \\ \hfill & 7:\text{Correcting shift and update local quality:}\hfill \\ \hfill & {\overline{s}}_i={\widehat{s}}_i+s-\widehat{s},y_i^{k+1}={\overline{s}}_i\hfill \\ \hfill & 8:\mathbf{end}\mathbf{for}\hfill \\ \hfill & 9:\hfill \\ \hfill & 10:\text{Train the predictor with either regression or classification task to make a final}\hfill \\ \hfill & \text{prediction of the global quality}.\hfill \end{array}$$