MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models
Table 3: Performance with different pre-trained machine comprehension models for our NMT model on De→En of WMT’14. EM means the exact match score, which represents the performance of the MC model on the SQuAD dev set, BLEU is the results of our NMT model.
['[BOLD] MC Attention', '[BOLD] EM', '[BOLD] BLEU']
Their performance on the machine comprehension task drops a lot, and it seems to affect the results of the NMT models as well. Unfortunately, the result of the NMT model fails to keep pace with the performance of its pre-train MC model. Finally, we apply memory network, which is also very popular among MC models , the performance on the SQuAD rises a lot but the NMT result is similar to the original model. This series of experiments denote that the model’s performance with our MacNet is not always in positive correlation to the improvement of the MC architecture. We conjecture that it might depend on many potential factors such as the complexity of the extracted parts, the heterogeneity of different tasks, etc.