Deep Learning for Hindi Text Classification : A Comparison
Table 1: Classification accuracies of different models
['Model / Dataset', '[EMPTY]', 'TREC', 'SST-1', 'SST-2']
Performance of different models based on CNN and LSTM were evaluated on translated versions of TREC, SST-1, and SST-2 datasets. Different versions of input word vectors were given to the models for comparison. Pre-trained fast text embeddings trained on Hindi corpus were compared against random initialization of word vectors. The random values were sampled from a continuous uniform distribution in a half-open interval [0.0, 1.0). Moreover, in one setting pre-trained fast-text embeddings were fine-tuned whereas in other settings they remained static. Keeping the word vector layer un-trainable allows better handling of words that were not seen during training as all the word vectors follow the same distribution. However, the domain of the corpus on which the word vectors were pre-trained may be different from the target domain. In such cases, fine-tuning the trained word vectors helps model adapt to the domain of the target corpus. So re-training the fast text vectors and keeping them static has its pros and cons. The three versions of word vectors are indicated as random for random initialization, fast text for trainable fast-text initialization, and fast text-static for un-trainable fast text initialization. Out of all the models vanilla CNN performs the best for all the datasets. CNNs have known to perform best for short texts and same is visible here as the datasets under consideration do not have long sentences. There is a small difference in the performance of different LSTM model. However, Bi-LSTM with max-pooling performed better than its attention version and unidirectional LSTM. Bag of words based on attention fared better than the simple bag of words model. Attention was particularly helpful with the usage of static fast text word vectors. Stacked CNN-LSTM models were somewhere between LSTM and CNN based models. We did not see a huge drop in performance due to random initialization of word vectors. But the performance across different epochs was very stable with fast text initialization. Finally, as compared to generic sentence embeddings obtained from BERT and LASER, specific embeddings obtained from custom models performed better. LASER was able to reach close to the best performing model. This shows that LASER was able to capture important discriminative features of a sentence required for the task at hand whereas BERT failed to capture the same.