An annotation guide is a document that contains the data labeling instructions for a given project. It is normally prepared before starting any labeling project, given the importance of having well-established guidelines to direct the team.
The annotation guide is written by an expert (the client or one of our project managers) at the beginning of the project. With that, the annotators can familiarize themselves with the instructions before beginning the labeling process.
The format and size of this guide can vary significantly: it can be concise or have multiple sections and chapters. Varying on the data type, the document may take the form of :
- a Word,
- a PDF document or
- a PowerPoint presentation.
All of that will depend on the complexity of the project. In the Foodvisor case study we had more than 1500 labeling classes, so there were a lot of annotation guides in the form of PowerPoint presentations.
Writing this annotation guide effectively requires putting oneself in the position of someone who has no prior knowledge of your specific challenges.
Moreover, it’s probable that the guide will see further additions upon completing initial annotation phases (pilot project and/or training project). When a project begins, it’s common for new and insightful questions about labeling to emerge. However, constantly revising the guide for every minor change can be quite time-consuming. As a result, we only make changes to the annotation guide if there are significant modifications.
Otherwise, project managers handle smaller questions and answers by summarizing and reorganizing them in another document (usually a Q&A document).
Synonyms: Labeling instructions; Annotation Book; Guidelines
