Solving Math Word Problems with Double-Decoder Transformer
Table 1: Results of RNN models and retrieval models are from C18-1018. Results of Transformer models are from our experiments. Our Transformer models do not use copy or align. MLE stands for maximum likelihood learning. RL stands for reinforcement learning.
['[BOLD] Generative Models - RNN', '[BOLD] Accuracy']
Results of the RNN models and retrieval models are from \newciteC18-1018. When using the MLE objective, we can see that Transformer outperforms GRU, especially when it has no copy and align mechanisms. Another important observation is that the results with two decoders outperform the one with one decoder by a large margin. Even if we just pick the results from one decoder and ignore the other, the two-decoder model still yields better results. It suggests that the whole system (including the encoder) benefits from having two decoders.