Basic Linguistic Resources and Baselines for Bhojpuri, Magahi and Maithili for Natural Language Processing
Table 17. Entropy and TTR calculations for POS Tag, Word_POS_Tag pair, and Chunk Tag annotation
['[BOLD] Languages', '[BOLD] POS Tag', '[BOLD] Entropy  [BOLD] Word_POS_Tag', '[BOLD] Chunk Tag', '[BOLD] POS Tag', '[BOLD] TTR  [BOLD] Word_POS', '[BOLD] Chunk Tag']
Although it is possible to perform annotation in a simple text editor, since the annotated file is stored in a text file, such an approach can lead to easily avoidable errors. It is, therefore, a common practice to use some annotation tool with Graphical User Interface and some kind of error checking. As far as actual file format or representation is concerned, we used the Shakti Standard Format (SSF), which is a robust recursive representation. It allows storing a wide variety of linguistic and non-linguistic information in one place. The detailed description of the SSF representation can be found is given in (Bharati et al. We are try to get the same kind of statistics as above for POS tags, pairs of words and POS tags and of chunks.