Decoupling Structure and Lexicon for Zero-Shot Semantic Parsing
Table 4: Development accuracy for all ablations.
['Model', 'Blocks', 'Calendar', 'Housing', 'Publications', 'Recipes', 'Restaurants', 'Social', 'Avg.']
S4SS2SSS0Px1 Model Ablations We now measure the effect of different components of our framework on denotation accuracy. We examined the effect of removing components completely, or replacing them with simpler ones. Thus, the following ablated models can be viewed as additional baselines. [topsep=0pt,itemsep=0pt,parsep=0pt,wide=0pt, widest=99,leftmargin =labelsep=*] -Aligner: Replacing the alignment distribution from the aligner with alignment distribution from the decoder of the structure mapper. -Inference: Discarding the global scoring function and maximizing each slot independently. -Aligner,Inference: Discarding both of our main technical contributions. -GlobalHeur : Discarding the global inference heuristics (denoted as once(z)). Discarding our two main technical contributions results in 31.2% accuracy compared to 54.5% in the full model. Performing inference with global constraints dramatically improve performance, showing that using the alignment model alone results often in incoherent logical forms. Our dedicated aligner also improves performance compared to alignments learned by the decoder of the structure mapper. This is pronounced without global constraints (a drop from 42.8% to 31.2%), but is less severe when global inference is used (a drop from 54.5% to 48.8%).