Algorithm 1: Management Algorithm

1: Initialize communication between management algorithm and robot controller

2: camera ← Set up the camera

3: model ← Load deep learning model

4: storage_detected_classes ← Reset the classes detected in the last n frames

5: tasks_performed ← Reset the registered tasks as performed by the robot

6: num_frames ← 0 Reset the frame count captured by the camera

7: running_flag ← true

8: while (running_flag)

9:  image ← Capture_Image (camera)

10:  detected_classes ← model (image)

11:  storage_detected_classes ← Update (detected_classes)

12:  tasks_performed ← The robot provides feedback about task execution

13:  num_frames ← num_frames + 1

14:  if (num_frames == n)

15:   if (Existence and non-existence of certain key categorical classes within the workspace detected in m frames (storage_detected_classes) and certain tasks performed by the robot were and were not performed (tasks_performed) as required by robot task 1)

16:     Command the robot to perform the robot task 1

17:     num_frames ← 0

18:     storage_detected_classes ← Reset

19:    elseif (Existence and non-existence of certain key categorical classes within the workspace detected in m frames (storage_detected_classes) and certain tasks performed by the robot were and were not performed (tasks_performed) as required by robot task 2)

20:     Command the robot to perform the robot task 2

21:     num_frames ← 0

22:     storage_detected_classes ← Reset

23:    elseif (…)

24:     …

25:    elseif (All k robot tasks were performed (tasks_performed))

26:     tasks_performed ← Reset

27:    endif

28:    if (num_frames == n)

29:     num_frames ← num_frames-1

30:    endif

31:   endif

32:   running_flag ← Check if the human has turned off the HRC application

33: endwhile