FRAGE: Frequency-Agnostic Word Representation
Table 10: BLEU scores on test set of the WMT14 English-German task and IWSLT14 German-English task. Our method is denoted as “FRAGE”, “Reweighting” denotes reweighting the loss of each word by reciprocal of its frequency, and “Weight Decay” denotes putting weight decay rate (0.2) on embeddings.
['[BOLD] WMT En→De  [BOLD] Method', '[BOLD] WMT En→De  [BOLD] BLEU', '[BOLD] IWSLT De→En  [BOLD] Method', '[BOLD] IWSLT De→En  [BOLD] BLEU']
We compare some other simple methods with ours on machine translation tasks, which include reweighting method and l2 regularization (weight decay). We notice that those simple methods do not work for the tasks, even have negative effects.