Long-Short Range Context Neural Networks for Language Modeling
Table 2: LMs performance on the PTB test set.
['N-1=', 'model 1', 'model 2', 'model 4', 'model+KN5 1', 'model+KN5 2', 'model+KN5 4', 'NoP 4']
As a first observation, we can clearly see that the proposed approach outperforms all other models for all configurations, in particular, RNN and LSTM. More particularly, we can conclude that LSRC, with an embedding size of 100, achieves a better performance than all other models while reducing the number of parameters by ≈29% and ≈17% compared to RNN and LSTM, respectively. Increasing the embedding size to 200, which is used by the other models, improves significantly the performance with a resulting NoP comparable to LSTM. The significance of the improvements obtained here over LSTM were confirmed through a statistical significance t-test, which led to p-values ≤10−10 for a significance level of 5% and 0.01%, respectively. In fact, the additional layer bridges the gap between the LSRC models with an embedding size of 100 and 200, respectively.