Table 12.

IPS results: comparison of normalization results on the IPS task test data.

	Calculated by interactor			Calculated by article			Calculated by article with interactions
	P	R	F	P	R	F	P	R	F
Run 1	0.57	0.12	0.19	0.15	0.13	0.13	0.56	0.46	0.48
Median	0.18	0.25	0.19	0.16	0.28	0.17	0.21	0.39	0.24

'Calculated by interactor' indicates each interactor extracted was given equal weight. 'Calculated by article' means the measure was calculated by averaging over articles. 'Calculated by article with interactions' means that only articles in which at least a single prediction submitted by a team was considered in the calculation. IPS, interaction pair subtask.