Figure 1.

Overview of a TeamTat annotation project. A project manager selects the documents to be annotated, specifies the types of entities and relations to be considered, enables the participation of the annotator(s), and distributes the documents among annotators. An annotation round consists of team members working independently to annotate. Team members can also review annotations where they have a disagreement among annotation partners. TeamTat maintains anonymity to prevent annotator bias. At the end of each round, the project manager calculates inter-annotator agreement statistics, and decides to continue or finalize the corpus. Annotations are trackable for every annotation round, and data can be downloaded at any time.