Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis
Table 2: Comparison results in terms of F1 scores.
['Models', 'Restaurant Aspect', 'Restaurant Opinion', 'Laptop Aspect', 'Laptop Opinion']
We compare our model with several baselines: CRF-1: a linear-chain CRF with standard linguistic features including word string, stylistics, POS tag, context string, and context POS tags. CRF-2: a linear-chain CRF with both standard linguistic features and dependency information including head word, dependency relations with parent token and child tokens. LSTM: We try different hidden layer dimensions (50, 100, 150, 200) and reported the best result with size 50. LSTM+F: the above LSTM model with the three additional types of hand-crafted features as with RNCRF. SemEval-1, SemEval-2: the top two winning systems for SemEval challenge 2014 task 4 (subtask 1). Note that we provide the same annotated dataset (both aspect labels and opinion labels are included for training) for CRF-1, CRF-2 and LSTM for fair comparison. It is clear that our proposed model RNCRF achieves superior performance compared with most of the baseline models. The performance is even better by adding simple hand-crafted features, i.e., RNCRF+F, with 0.92% and 3.87% absolute improvement over the best system in the challenge for aspect extraction for the restaurant domain and the laptop domain, respectively. This shows the advantage of combining high-level continuous features and discrete hand-crafted features. Though CRFs usually show promising results in sequence tagging problems, they fail to achieve comparable performance when lacking of extensive features (e.g., CRF-1). By adding dependency information explicitly in CRF-2, the result only improves slightly for aspect extraction. Alternatively, by incorporating dependency information into a deep learning model (e.g., RNCRF), the result shows more than 7% improvement for aspect extraction and 2% for opinion extraction.