Skip to main content
. 2021 May 31;7(2):e24418. doi: 10.2196/24418

Table 5.

Time taken for the manual and automation teams to learn and complete each systematic review (SR) task.

SR task No. SR task Total time,
hours:minutes
Time doing task,
hours:minutes
Time learning task,
hours:minutes


Manual Automation Manual Automation Manual Automation
1 Formulate the question 1:00a N/Ab 1:00a N/A 0:00 N/A
2 Check for similar reviews 1:00a N/A 1:00a N/A 0:00 N/A
3 Write the protocol 4:00a N/A 4:00a N/A 0:00 N/A
4 Design the systematic search 13:00a N/A 13:00a N/A 0:00 N/A
5 Run the systematic search 6:15 1:20 5:00 0:37 1:15 0:43
6 Deduplicate the search results 2:09 0:36 2:09 0:12 0:00 0:24
7 Screen the titles and abstracts 5:10 3:33 4:40 3:28 0:30 0:05
8 Find the full text 0:50 0:23 0:50 0:18 0:00 0:05
9 Screen the full text 3:29 3:44 3:29 3:44 0:00 0:00
10 Conduct a citation analysis 7:43 N/A 7:43 N/A 0:00 N/A
11 Extract the data 9:42 N/A 9:42 N/A 0:00 N/A
12 Assess the risk of bias 23:40 2:12c 19:20 2:11c 4:20 0:01c
13 Synthesize the data 10:00 N/A 8:00 N/A 2:00 N/A
14 Rerun the systematic search 0:22 N/A 0:22 N/A 0:00 N/A
15 Conduct a meta-analysis 16:00 N/A 10:00 N/A 6:00 N/A
16 Write the results 21:20 N/A 10:40 N/A 10:40 N/A
All tasks Tasks done by both teams 41:33 11:48 35:28 10:30 6:05 1:18
All tasks Tasks done by manual team 125:40a N/A 100:55a N/A 24:45a N/A

aApproximate time only.

bN/A: not applicable; task not done by automation team.

cTask partially completed; four of seven domains assessed.