Leveraging Twitter for Low-Resource Conversational Speech Language Modeling
TABLE VI: Queue Ranking Strategy LM Perplexity Comparison
['[EMPTY]', '[BOLD] 150 Users', '[BOLD] 300 Users', '[BOLD] 500 Users']
We created language models from the downloaded data to compare the effectiveness of the ranking strategies. The language models used the data from the 300 users in the training set as well as the additional users selected by each of the respective ranking strategies. The predictive ranking LM achieved a 17% reduction in perplexity over the random baseline after downloading 500 users from the queue. When less than 500 additional users are downloaded, the predictive ranking has a better perplexity than the edge ranking. However, once enough data has been acquired the difference in perplexity between the two ranking systems is small. We reached two conclusions from this experiment: either of the two ranking systems is better than not doing any ranking and the predictive ranking has a definite advantage at covering the in-domain vocabulary.