Table 5.

Results, summary of the judgements made by the 17 evaluators of the overall quality in the 12 examples in the pilot study

Example	High	Moderate	Low	Very low	Consensus	Comments
1	2/15	4/15	5/15	4/15	Moderate
2	12/17	5/17	-	-	High
3	1/17	6/17	5/17	5/17	Need more information before consensus
4	4/17	13/17	-	-	High	Based on the new rule
5	12/16	4/16	-	-	High
6	7/17	10/17	-	-	High	Based on new rule
7	-	11/17	6/17	-	Moderate
8	-	6/17	3/17	8/17	High	Based on new rule
9	2/16	3/16	5/16	6/16	High/Moderate depending if there are fatal flaws
10	-	1/17	4/17	12/17	Very low
11	1/17	3/17	8/17	5/17	High	Changed question
12	11/13	2/13	-	-	High