DRTS Parsing with Structure-Aware Encoding and Decoding
Table 2: GAT settings results on development set.
['[BOLD] Model', '[BOLD] BLEU']
In particular, we vary the value of l in the set {1, 2, 3, 4, 5} and measure the corresponding BLEU scores. The structural-aware model equipped with GAT achieves the best performance when l is 2, which justifies the selection on the number of layers in the experimental setting section. Moreover, a dropping trend on both metrics is present as l increases. For a larger l, the GAT module becomes more difficult to train due to larger amounts of parameters. One intuitive reason is that each layer of the GAT module aggregates the direct neighbor information of a node. After 2 layers, each node can obtain sufficient information, and further more layers can bring noise. We make comparison with multi-head attention, varying the heads in the set {1, 2, 3, 4, 5} and checking the corresponding BLEU scores. Theoretically, the larger the number of heads, the better the performance of the model. We thus choose the head to be 4 for the remaining experiments.