Abstract
Table 6.2: A qualitative overview of the various algorithmic changes proposed in this thesis. The best results in every metric shown in boldface. Quality is measured both directly on clusters through final average mutual information (the 5th column) and on the resulting class-based language models. The test corpus TT is used to measure language model characteristics such as class prediction accuracy (3rd column) and perplexity (4th column).
['[BOLD] Corpus', '[BOLD] Language model', '[BOLD] CPA (%)', '[BOLD] PerplexityN', '[BOLD] Final AMI']
This is due to the fact that every time ALLSAME encounters multiple words with the same frequency, it chooses its own order for merging these words based on their average mutual information with all words in the merge window. Actually because ALLSAME makes better merge decisions than BROWN every time there are multiple words with the same frequency, ALLSAME should be expected to perform better than BROWN the more words with a given frequency there are (the larger the saw teeth), and the more often this occurs (the more teeth there are). The RESORT algorithm is a little more subtle. Its average mutual information tends to be just slightly better than BROWN while the algorithms runs. This, however, should not be considered a null result. Experiments on much larger vocabularies and with different sorting strategies are necessary in order to determine whether RESORT is actually not better than BROWN. And, of course, performance of the resulting class-based language models should also be taken into account.