Natural- to formal-language generation using Tensor Product Representations
Table 1: Results on MathQA dataset testing set
['[BOLD] MODEL', '[BOLD] Operation Accuracy(%)', '[BOLD] Execution Accuracy(%)']
Given a natural-language math problem, we need to generate a sequence of operations (operators and corresponding arguments) from a set of operators and arguments to solve the given problem. Each operation is regarded as a relational tuple by viewing the operator as relation, e.g., (add,n1,n2). The MathQA dataset consists of about 37k math word problems, each with a corresponding list of multi-choice options and the corresponding operation sequence. In this task, TP-N2F is deployed to generate the operation sequence given the question. The generated operations are executed with the execution script from Amini et al. As there are about 30% noisy data (where the execution script returns the wrong answer when given the ground-truth program; see Sec. TP-N2F is compared to a baseline provided by the seq2prog model in Amini et al. Our model outperforms both the original seq2prog, designated SEQ2PROG-orig, and the best reimplemented seq2prog after an extensive hyperparameter search, designated SEQ2PROG-best. To verify the importance of the TP-N2F encoder and decoder, we conducted experiments to replace either the encoder with a standard LSTM (denoted LSTM2TP) or the decoder with a standard attentional LSTM (denoted TP2LSTM). We observe that both the TPR components of TP-N2F are important for achieving the observed performance gain relative to the baseline.