Learning with Latent Language
Table 3: Inference and representation experiments for string editing. Italicized numbers correspond to entries in Table 2. Allowing the model to use multiple samples rather than the 1-best decoder output substantially improves performance. The full model does better with inferred natural language descriptions than either regular expressions or ground-truth natural language.
['Annotations', 'Samples 1', 'Samples 100', 'Oracle Ann.', 'Oracle Eval.']
Because all of the transduction rules in this dataset were generated from known formal descriptors, these tasks provide an opportunity to perform additional analysis comparing natural language to more structured forms of annotation (since we have access to ground-truth regular expressions) and more conventional synthesis-based methods (since we have access to a ground-truth regular expression execution engine). We additionally investigate the effect of the number of samples drawn from the proposal model.