Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification
Table 1: Data used for training the CNN model.
['[BOLD] Language', '[BOLD] Dataset', '[BOLD] Total', '[BOLD] Neutral', '[BOLD] Neg.', '[BOLD] Pos.']
Training. During the first distant-supervised phase, we use emoticons to infer noisy labels on tweets in the training set [ Note that the data used for this pre-training phase do not necessarily overlap with the data used to create the word embeddings. The neural network was trained on these datasets for one epoch, before finally training on the supervised data for about 20 epochs. The word embeddings X∈Rd×n are updated during both the distant- and supervised training phases by applying back-propagation through the entire network. We used a large set of 300M tweets to create the word embeddings for each language, as well as a distant supervision corpus of 40-60M tweets for each language, where each tweet contained at least one emoticon (positive or negative smiley). Smileys were used to automatically infer weak labels and subsequently removed from the tweets. The German corpus was newly created by the authors and is available at http://spinningbytes.com/resources. In our experiments, we compare the performance of the following sentiment analysis systems: Random forest (RF) as a common baseline classifier. In a set of experiments, the amount of training in the three phases is gradually reduced. The system using all available training data for one language is also referred to as ’fully-trained CNN’ Multi-language CNN (ML-CNN), where the distant-supervised phase is performed jointly for all languages at once, and the final supervised phase independently for each language. The pre-training data is the same as in ML-CNN SemEval benchmark. It then uses the SL-CNN classifier trained for the target language to classify the tweets For French, German and Italian, we created a validation set by randomly sampling 10% of the data. For English we used the test2015 set as validation set and test2016 for testing from the SemEval-2016 challenge, see Validation set in Table