Low-Complexity Multiple Transform Selection Combining Multi-Type Tree Partition Algorithm for Versatile Video Coding

. 2022 Jul 25;22(15):5523. doi: 10.3390/s22155523

Algorithm 2 The proposed MTS early termination method

Input: the prediction modes list L
Output: the minimum RD cost of second pass

R D C o s t_{m i n}

and the best results

1:
initialize the minimum RD cost of MTS $R D C o s t_{m i n m t s}$
2:
reorder MTS candidates according to the frequency of transform used in neighboring CUs
3:
calculate the RD cost of the current CU $R D C o s t_{m i n}$ using DCT-II
4:
for each transform $t r a n s_{j}$ in MTS candidates do
5:
if all neighboring CUs choose DCT-II then
6:
break;
7:
end if
8:
for each predicition mode $m o d e_{i}$ in list L do
9:
derive the RD cost $R D (m o d e_{i}, t r a n s_{j})$ ;
10:
if $R D (m o d e_{i}, t r a n s_{j})$ < $R D C o s t_{m i n m t s}$ then
11:
update $R D C o s t_{m i n m t s}$ by $R D (m o d e_{i}, t r a n s_{j})$ ;
12:
load the prediction results of $m o d e_{i}$ ;
13:
else
14:
contine;
15:
end if
16:
end for
17:
if $R D C o s t_{m i n}$ > $R D C o s t_{m i n m t s}$ then
18:
update $R D C o s t_{m i n}$ by $R D C o s t_{m i n m t s}$ ;
19:
else
20:
break;
21:
end if
22:
end for
23:
return $R D C o s t_{m i n}$