Linguistic Input Features Improve Neural Machine Translation
Table 1: Vocabulary size, and size of embedding layer of linguistic features, in system that includes all features, and contrastive experiments that add a single feature over the baseline. The embedding layer size of the word feature is set to bring the total size to 500.
['feature', 'input vocabulary EN', 'input vocabulary DE', 'input vocabulary model', 'embedding all', 'embedding single']
To ensure that performance improvements are not simply due to an increase in the number of model parameters, we keep the total size of the embedding layer fixed to 500. If we include the lemma feature, we roughly split the embedding vector one-to-two between the lemma feature and the word feature. The table also shows the network vocabulary size; for all features except lemmas, we can represent all feature values in the network vocabulary – in the case of words, this is due to BPE segmentation. For lemmas, we choose the same vocabulary size as for words, replacing rare lemmas with a special UNK symbol.