Building a Word Segmenter for Sanskrit Overnight
Table 1: Micro-averaged Precision, Recall and F-Score for the competing systems on the test dataset of 4200 strings.
['[BOLD] Model', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F-Score']
We can find that the system ‘attnSegSeq2Seq’ outperforms the current state of the art with a percent increase of 16.29 % in F-Score. The model ‘segSeq2Seq’ falls short of the current state of the art with a percent decrease of 6.29 % in F-Score. It needs to be noted that the systems ‘attnSegSeq2Seq’ and ‘segSeq2Seq’ are exactly the same architectures other than the addition of attention in the former. But there is a percentage increase of 23.61 % for both the systems. One probable reason for this is due to the free word order nature of sentences in Sanskrit. Since there are multiple permutations of words in a sentence which are valid syntactically and convey the same semantic meaning, the entire input context is required to understand the meaning of a sentence for any distributional semantic model.