ShanghaiTech at MRP 2019: Sequence-to-Graph Transduction with Second-Order Edge Inference for Cross-Framework Meaning Representation Parsing
Table 1: Comparison of cross-framework F1 scores achieved by our system and best scores of other teams for each metric. all represents the F1 score over the full test set for each framework. lpps represents a 100-sentence sample from the little prince containing graphs over all the frameworks.
['[EMPTY]', 'DM', 'PSD', 'EDS', 'UCCA', 'AMR']
Due to an unexpected bug in UCCA anchor prediction, we failed to submit our UCCA prediction. Our results are still competitive to those of the other teams and we get the \nth3 place for the DM framework in the official metrics. Our system performs well on the DM framework with an F1 score only 0.4 percent F1 below the best score on DM. Note that our system does not learn to predict node labels for DM and PSD and simply uses lemmas from the companion data as node labels. We find that compared to gold lemmas from the original SDP dataset, lemmas from the companion data have only 71.4% accuracy. We believe that it is the main reason for the F1 score gap between our system and the best one on DM and PSD. For PSD, EDS and AMR graph, our system ranks \nth6, \nth5 and \nth7 among 13 teams.