Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations
Table 6: Results of the improvements in isolation (without post-processing).
['[BOLD] Baseline', '[BOLD] Type seq2seq', '[BOLD] Dev 54.8', '[BOLD] Diff', '[BOLD] Test 53.1', '[BOLD] Diff']
Re-ordering has a clear positive effect, both for using the best re-ordering (+2.0) and adding that re-ordering to the existing data set (+5.2). Constructing super characters and adding POS-tags both lead to a similar increase in performance. Pre-training and subsequently fine-tuning also results in a substantial improvement, but creating an average model only has a slight positive effect. The biggest improvement comes from adding silver standard data to our training set, reaching a maximum of 65.8 on the dev set. However, there is a limit with regards to adding silver data, since adding 500k silver AMRs performed worse than adding 50k, 75k or 100k silver AMRs. Finding the optimal number of silver AMRs is difficult due to the long training times and is therefore left for future work.