Low-Rank RNN Adaptation for Context-Aware Language Modeling
Table 3: Perplexity and classification accuracy on the test set for the four word-based datasets.
['[BOLD] Model', '[BOLD] AGNews PPL', '[BOLD] AGNews ACC', '[BOLD] DBPedia PPL', '[BOLD] DBPedia ACC', '[BOLD] TripAdvisor PPL', '[BOLD] TripAdvisor ACC', '[BOLD] Yelp PPL', '[BOLD] Yelp ACC']
In each of the four datasets, the FactorCell model gives the best perplexity. For classification accuracy, there is a bigger difference between the models, and the FactorCell model is the most accurate on three out of four datasets and tied with the SoftmaxBias model on AgNews. For DBPedia and TripAdvisor, most of the improvement in perplexity relative to the unadapted case is achieved by the SoftmaxBias model with smaller relative improvements coming from the increased power of the ConcatCell and FactorCell models. For Yelp, the perplexity improvements are small; the FactorCell model is just 1.3% better than the unadapted model. The figure compares perplexity on the x-axis with accuracy on the y-axis with both metrics computed on the development set. Each point in this figure represents a different instance of the model trained with random hyperparameter settings and the best results are in the upper right corner of each plot. The color/shape differences of the points correspond to the three classes of models: FactorCell, ConcatCell, and SoftmaxBias.