Approximating probabilistic models as weighted finite automata
Table 1: Test perplexity of models when approximated onto smaller topologies.
['Topology', 'Test perplexity', '# Transitions']
However, the test perplexity also increases some. To control for this, we prune the original model to a 1.08M n-gram topology A′greedy instead of the 1M as before and apply the same procedure to obtain an approximation on Minimize(A′greedy). We achieve a 0.4% perplexity reduction compared to the approximation on Agreedy with very nearly the same number of transitions.