Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
Table 6: Test accuracy for LSTM model trained on original/shuffled training set.
['[BOLD] Datasets', '[BOLD] Yahoo', '[BOLD] Yelp P.', '[BOLD] SNLI']
Somewhat surprisingly, for Yahoo and SNLI datasets, the LSTM model trained on shuffled training set shows comparable accuracies to those trained on the original dataset, indicating that word-order information does not contribute significantly on these two problems, i.e., topic categorization and textual entailment. However, on the Yelp polarity dataset, the results drop noticeably, further suggesting that word-order does matter for sentiment analysis (as indicated above from a different perspective).