Morphological Inflection Generationwith Hard Monotonic Attention
Table 1: Results on the CELEX dataset
['[EMPTY]', '13SIA', '2PIE', '2PKE', 'rP', 'Avg.']
In addition, it significantly outperforms our implementation of the soft attention model (Soft). It is also, to our knowledge, the first model that surpassed in overall accuracy the latent variable model on this dataset. We attribute our advantage over the soft attention models to the ability of the hard attention control mechanism to harness the monotonic alignments found in the data. The advantage over the FST models may be explained by our conditioning on the entire output history which is not available in those models. While both models perform similarly on the train-set (with the soft attention model fitting it slightly faster), the hard attention model performs significantly better on the dev-set. This shows the soft attention model’s tendency to overfit on the small dataset, as it is not enforcing the monotonic assumption of the hard attention model. ; Hulden et al. ; Nicolai et al. Our work was mainly inspired by \newcitefaruquiTND15 which trained an independent encoder-decoder neural network for every inflection type in the training data, alleviating the need for feature engineering. \newcitekann2016,kann2016medtl tackled the task with a single soft attention model Bahdanau et al. Cotterell et al. In another closely related work, Rastogi et al. (2016) model the task with a WFST in which the arc weights are learned by optimizing a global loss function over all the possible paths in the state graph, while modeling contextual features with bi-directional LSTMS. This is similar to our approach, where instead of learning to mimic a single greedy alignment as we do, they sum over all possible alignments. Another recent work Kann et al.