A Hybrid Deep Learning and Visualization Framework for Pushing Behavior Detection in Pedestrian Dynamics

. 2022 May 26;22(11):4040. doi: 10.3390/s22114040

Algorithm 2 Patch-Based Approach.
Inputs: dataset ← collection of MIMs with the first frame of each MIM ROI $\leftarrow m a t r i x [l e f t_t o p : [x_c o o r d i n a t e, y_c o o r d i n a t e], r i g h t_b o t t o m : [x_c o o r d i n a t e, y_c o o r d i n a t e]]$ $n, m$ ← the numbers of rows and columns that are used to divide ROI into $n \times m$ regions. trajectory ← CSV file, each row represents $〈 o r d e r o f f r a m e (f_{t}), p e d e s t r i a n n o ., p i x e l x - c o o r d i n a t e, p i x e l y - c o o r d i n a t e 〉$ ground_truth ← CSV file, each row represents $〈 o r d e r o f c_{i} o r M I M, b e h a v i o r o f p e d e s t r i a n 1, b e h a v i o r o f p e d e s t r i a n 2, \dots,$ $b e h a v i o r o f l a s t p e d e s t r i a n 〉$ Outputs: pushing_folder, non-pushing_folder
1: $r e g i o n \leftarrow m a t r i x [[]]$	▹ Automatic patches labeling
2: $p a t c h_w i d t h \leftarrow (R O I [1, 0] - R O I [0, 0]) / m$
3: $p a t c h_h e i g h t \leftarrow (R O I [1, 1] - R O I [0, 1]) / n$
4: for $i \leftarrow 0, 1, \dots, n - 1$ do
5: for $j \leftarrow 0, 1, \dots, m - 1$ do
6: $r e g i o n . a p p e n d ([R O I [0, 0] + j \times p a t c h_w i d t h, R O I [0, 1] + i \times p a t c h_h e i g h t, R O I [0, 0] + (j + 1) \times p a t c h_w i d t h, R O I [0, 1] + (i + 1) \times p a t c h_h e i g h t])$
7: end for
8: end for
9: $p a t c h_w i d t h \leftarrow (R O I [1, 0] - R O I [0, 0]) / (m - 1)$
10: $p a t c h_h e i g h t \leftarrow (R O I [1, 1] - R O I [0, 1]) / (n - 1)$
11: for $i \leftarrow 0, 1, \dots, n - 2$ do
12: for $j \leftarrow 0, 1, \dots, m - 2$ do
13: $r e g i o n . a p p e n d ([R O I [0, 0] + j \times p a t c h_w i d t h, R O I [0, 1] + i \times p a t c h_h e i g h t, R O I [0, 0] + (j + 1) \times p a t c h_w i d t h, R O I [0, 1] + (i + 1) \times p a t c h_h e i g h t])$
14: end for
15: end for
16: $f i l e \leftarrow C S V f i l e$
17: for each $M I M \in d a t a s e t$ do
18: $f r a m e_o r d e r \leftarrow M I M n a m e$
19: $p e d \leftarrow F i l t e r (t r a j e c t o r y . f r a m e_o r d e r) [1]$
20: $p a t c h_n o \leftarrow 1$
21: for each $p a t c h_r e g i o n \in r e g i o n$ do
22: $b e h a v i o r \leftarrow 1$ //non-pushing
23: for each $p e d \in p a t c h_r e g i o n$ do
24: if $F i l t e r (g r o u n d_t r u t h . f r a m e_o r d e r & p e d) = = 2$ then
25: $b e h a v i o r \leftarrow 2$ //pushing
26: break
27: end if
28: end for
29: $r e c o r d \leftarrow [p a t c h_n o, f r a m e_o r d e r, b e h a v i o r]$
30: $f i l e . w r i t e (r e c o r d)$
31: $p a t c h_n o \leftarrow p a t c h_n o + 1$
32: end for
33: end for
	▹ Visualization
34: for each $f r a m e \in d a t a s e t$ do
35: $f r a m e_o r d e r \leftarrow f r a m e n a m e$
36: $p e d \leftarrow F i l t e r (t r a j e c t o r y . f r a m e_o r d e r) [1]$
37: for each $p e r s o n \in p e d$ do
38: $b e h a v i o r \leftarrow F i l t e r (g r o u n d_t r u t h . f r a m e_o r d e r & p e r s o n)$
39: if behavior ==2 then
40: draw a circle around the position $〈 p e r s o n [2], p e r s o n [3]] 〉$ of pedestrian $p e r s o n [1]$ over $f r a m e$
41: end if
42: end for
43: for $p a t c h_n o \leftarrow 1, 2, \dots, l e n (r e g i o n)$ do
44: if $F i l t e r (f i l e . f r a m e_o r d e r & p a t c h_n o) [2] = = 2$ then
45: draw a red rectangle around $r e g i o n [p a t c h_n o - 1]$ over $f r a m e$
46: else
47: draw a green rectangle around $r e g i o n [p a t c h_n o - 1]$ over $f r a m e$
48: end if
49: end for
50: end for
	▹ Manual revision
51: for each $f r a m e \in d a t a s e t$ do
52: for each $p a t c h_r e g i o n \in r e g i o n$ do
53: manual revision of $p a t c h_r e g i o n$ in $f r a m e$
54: if $p a t c h_r e g i o n c o n t a i n s o n l y a p a r t o f o n e p u s h i n g b e h a v i o r a n d i t s l a b e l i s 2$ then
55: manually updating the label of the patch_region in $f i l e$ to 6, where 6 means unknown patch
56: end if
57: end for
58: end for
	▹ Patch-based MIM dataset creation
59: for each $M I M \in d a t a s e t$ do
60: $M I M_o r d e r \leftarrow M I M n a m e$
61: for $p a t c h_n o \leftarrow 1, 2, \dots, l e n (r e g i o n)$ do
62: $p a t c h \leftarrow M I M [r e g i o n [p a t c h_n o - 1, 1] : r e g i o n [p a t c h_n o - 1, 3], [r e g i o n [p a t c h_n o - 1, 0] : r e g i o n [p a t c h_n o - 1, 2]]$
63: if $F i l t e r (f i l e . M I M_o r d e r & p a t c h_n o) [2] = = 2$ then
64: save $p a t c h$ to pushing_folder under name “ $M I M_o r d e r - p a t c h_n o$ "
65: else if $F i l t e r (f i l e . M I M_o r d e r & p a t c h_n o) [2] = = 1$ then
66: save $p a t c h$ to non-pushing_folder under name “ $M I M_o r d e r - p a t c h_n o$ "
67: end if
68: end for
69: end for