Table 9.

F-measures computed prior to and after feature selection for the test dataset relationship prediction. Results are computed using the annotated concepts (columns 1 and 2), and using the predicted concepts as identified in the concept recognition step (columns 3 and 4).

Results of relationship identification	Annotated concepts		Predicted concepts
	Before feature selection	After feature selection	Before feature selection	After feature selection
Improves	0.735	0.727	0.558	0.728
Worsens	0.598	0.600	0.398	0.541
Causes	0.655	0.664	0.401	0.632
Given	0.735	0.738	0.536	0.735
Not Given	0.628	0.427	0.494	0.458
Reveals	0.814	0.823	0.625	0.821
Conducted	0.593	0.583	0.404	0.531
Relates	0.591	0.588	0.390	0.588
Overall weighted average	0.712	0.711	0.516	0.704

The F-measures are computed using the i2b2 evaluation package, and the Average is computed weighting each individual F-measure by the number of all positive examples in that particular relationship category.