Selective Attention Based Graph Convolutional Networks for Aspect-Level Sentiment Classification
Table 4: Ablation study of our model
['Model', 'Restaurant Acc', 'Restaurant Macro-F1', 'Laptop Acc', 'Laptop Macro-F1']
Ablation Study. To demonstrate effectiveness of different modules in our model, we conduct ablation studies by the following settings: [leftmargin=10pt] 1-layer GCN on DT + 1 SA-GCN block utilizes 1-layer GCN on dependency trees and then employs one SA-GCN block. 2 SA-GCN blocks directly applies two SA-GCN blocks, and removes the GCN over dependency trees. 1-layer GCN on DT + 2 SA-GCN blocks firstly applies the 1-layer GCN on dependency trees and then employs 2 SA-GCN blocks. From this table, we observe that: [leftmargin=10pt] Effect of SA-GCN. Compared with 2-layer GCN on DT, in Laptop and Restaurant domains, our 1-layer GCN on dependency trees + 1 SA-GCN block model improves by 1.3%, 0.6% on accuracy and 2.5%, 1.5% on Macro-F1 respectively, which shows that SA-GCN block is beneficial to this task. Effect of Syntax Information. Compared with 1-layer GCN on dependency trees + 1 SA-GCN block, the 2 SA-GCN blocks model without GCN on dependency trees decreases the accuracy and Macro-F1 on both datasets. This implies that the syntax information introduced by the dependency trees is important and needed. Effect of Head-independent and Head-dependent Selection. This is because the mechanism of head-independent selection is similar to voting. By summing up the weight scores from each head, context words with higher scores in most heads get emphasized, and words that only show importance in few heads are filtered out. Thus all heads reach to an agreement and the top-k context words are decided. However for head-dependent selection, each head selects different top-k context words, which is more likely to choose certain unimportant context words and introduce noise to the model prediction.