Self-Attentional Models for Lattice Inputs
Table 1: BLEU scores on Fisher (4 references) and Callhome (1 reference), for proposed method and several baselines.
['Encoder model', 'Inputs', 'Fisher', 'Callh.']
Lattice models tested on Callhome are pretrained on Fisher and finetuned on Callhome lattices (Fisher+Callhome setting), while lattice models tested on Fisher use a Fisher+Fisher training setting. All sequential baselines are trained on the reference transcripts of Fisher. The first set of baselines operates on 1-best (sequential) inputs and includes a bidirectional LSTM, an unmasked self-attentional encoder (SA) of otherwise identical architecture with our proposed model, and a variant with directional masks Shen et al. Next, we include a graph-attentional model that masks all but adjacent lattice nodes Veličković et al. We conduct a manual inspection and showcase several common patterns in which the lattice input helps improve translation quality, as well as one counter example.