Online Representation Learning in Recurrent Neural Language Models
Table 2: Dataset sizes
['[EMPTY]', 'Train', 'Dev', 'Test']
We constructed a dataset from English Wikipedia to evaluate language modelling performance over individual sentences. The text was tokenised, sentence split and lowercased. The sentences were shuffled, in order to minimise any transfer effects between consecutive sentences, and then split into training, development and test sets. The final sentences were sampled randomly, in order to obtain reasonable training times for the experiments.