QCRI Machine Translation Systems for IWSLT 16
Table 12: Fine-tuning with/without freezing the Embeddings
['[BOLD] System', '[BOLD] ted-11', '[BOLD] ted-12', '[BOLD] ted-13', '[BOLD] ted-14', '[BOLD] Avg']
The default version of Nematus applies fine-tuning by freezing the weights of embedding layer. The intuition behind freezing a layer is to not allow the weights in that layer to change with additional data. This is sometimes useful when we can learn certain layers better from out-domain data. One such layer in our case is the word embedding layer. We tried a variation in which we do not freeze any layer. This latter variant was found to outperform the default setting