Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit
Table 2: Performance of Clique-EBM with pruned edges in G.
['k', 'WPT P', 'WPT R', 'WPT F', 'WP3T P', 'WP3T R', 'WP3T F']
For any two words appearing in an exhaustive segmentation, we keep an edge only if both the words overlap within a distance of k characters. We experiment with k = 5, 10, 15 and 20. Hence, for K=20, a word will form edges with all the words that fall within 20 characters to left and 20 characters to right. The average length of an input sequence in DCS10K is 40.79 characters. We do not modify our inference procedure in system 8 other than to take care of the possibility that a clique need not always be returned. k. Interestingly, the results show a monotonic increase with the increase in context window, and the results with the entire context are still better than those with k=20, even though only marginally. It is interesting to note that, keeping the entire context does not adversely affect the predictions as none of the pruned models outperforms System 8.