Bootstrapping a Crosslingual Semantic Parser
Table 5: Test denotation accuracy on ATIS and Overnight for reference model for English. Best accuracy is bolded. Note that Inan et al. (2019) evaluate on ATIS, but use the non-executable λ−calculus logical form and are therefore not comparable to our results. Domains are Basketball, Blocks, Calendar, Housing, Publications, Recipes, Restaurants, and Social Network.
['[EMPTY]', 'ATIS', 'Overnight Ba.', 'Overnight Bl.', 'Ca.', 'Ho.', 'Pu.', 'Rec.', 'Res.', 'So.', 'Avg']
We acknowledge our system performs below the state of the art for ATIS by -7.8% and Overnight by -3.9%, but this is most likely because we omit any English-specific feature augmentation other than BERT. In comparison to prior work, we do not use entity anonymization, paraphrasing, execution-guided decoding or a mechanism to incorporate feedback for incorrect predictions from humans or neural critics. The closest comparable model to ours is reported by Wang et al. However, this result uses entity anonymization for ATIS. We attempted to implement such an approach but found it to be unreliable and largely incompatible with the vocabularies of pre-trained models.