Anonymous Traffic Detection Based on Feature Engineering and Reinforcement Learning

. 2024 Apr 4;24(7):2295. doi: 10.3390/s24072295

Algorithm 1 Transitions between states.

1:
procedure Transition( $t h r e s h o l d_T o r, t h r e s h o l d_n o n T o r, f e a t u r e_s e t, b a t c h_s i z e$ ) ▹ Function
2:
for i in $r a n g e ($ 0, len(feature_set), batch_size) do

// Create batches
3:
$b a t c h_f e a t u r e s = f e a t u r e_s e t [i : i + b a t c h_s i z e]$

// Calculate reward value of each sample in the batch
4:
$r_b a t c h = f i r s t_a c t i o n (b a t c h_f e a t u r e s)$
5:
for j in range(len(batch_features), batch_size) do
6:
if $r [j] > = t h r e s h o l d_t o r$ then ▹ Classified as Tor
7:
$S t a t e \leftarrow T o r$
8:
else if $r [j] < = t h r e s h o l d_n o n_t o r$ then ▹ Classified as non_Tor
9:
$S t a t e \leftarrow N o n_T o r$
10:
else ▹ Classified as Ambiguous
11:
while $r [j] > t h r e s h o l d_n o n_t o r and r [j] < t h r e s h o l d_t o r$ do
12:
if $n o a c t i o n s r e m a i n i n g i n a c t i o n s p a c e$ then
13:
$L a b e l (A n o n y m o u s & & M a l i c i o u s)$ ▹ Further processing required
14:
$b r e a k$
15:
$S t a t e \leftarrow A m b i g u o u s$
16:
$r_b a t c h = n e x t_a c t i o n (b a t c h_f e a t u r e s)$ ▹ $S e l e c t n e x t a c t i o n$