Towards a Seamless Integration of Word Senses into Downstream NLP Applications
Table 4: Accuracy performance on five polarity detection datasets. Given that polarity datasets are balanced1717footnotemark: 17, we do not report F1 which would have been identical to accuracy.
['[BOLD] Initialization  [BOLD] Random', '[BOLD] Input type  [BOLD] Word', '[BOLD] Input type  [BOLD] Word', '[BOLD] RTC 83.6', '[BOLD] IMDB 87.7', '[BOLD] PL05 77.3', '[BOLD] PL04 67.9', '[BOLD] Stanford 91.8']
Results are generally better than those of \newcitekim2014convolutional, showing that the addition of the recurrent layer to the model (cf. However, interestingly, no consistent performance gain is observed in the polarity detection task, when the model is provided with disambiguated input, particularly for datasets with relatively short reviews. We attribute this to the nature of the task. Firstly, given that words rarely happen to be ambiguous with respect to their sentiment, the semantic sense distinctions provided by the disambiguation stage do not assist the classifier in better decision making, and instead introduce data sparsity. Secondly, since the datasets mostly contain short texts, e.g., sentences or snippets, the disambiguation algorithm does not have sufficient context to make high-confidence judgements, resulting in fewer disambiguations or less reliable ones. In the following section we perform a more in-depth analysis of the impact of document size on the performance of our sense-based models.