Do sequence-to-sequence VAEs learn global features of sentences?
Table 5: Performance of bag-of-word classifiers when using all words as features versus only the first three words. Ratios of performance vary a lot across datasets.
['Dataset', 'F1(All)', 'F1(3)', 'Ratio']
have a performance somewhat close to the classifier trained on all the words, reaching 80.8% and 85.4% of its scores respectively. On AGNews, for instance, the first words are often nouns that directly gives the topic of the news item: country names for the politics category, firm names for the technology category, athlete or team names for the sports category, etc. On the two other datasets, the performance decays a lot if we only use the first three words: three-words F1-scores make up for 60.7% and 30.3% of all-words F1-scores on Amazon and Yahoo. This explains why the original VAE can perform on par or slightly better than our variants on certain datasets for which the first words are very predictive of the labels.