aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model
Table 3: The comparision of aNMM-1/aNMM-2 with aNMM-IDF which is a degenerate version of our model where we use IDF to directly replace the output of question attention network.
['Training Data Method', 'TRAIN MAP', 'TRAIN MRR', 'TRAIN-ALL MAP', 'TRAIN-ALL MRR']
An interesting question is how the learned term importance compare with traditional IR term weighting methods such as IDF. We design an experiment to compare aNMM-1/aNMM-2 with aNMM-IDF, which is a degenerate version of our model where we use IDF to directly replace the output of question attention network. We find that if we replace the output of question attention network of aNMM with IDF, it will decrease the answer ranking performance, especially on TRAIN data. Thus, we can see that with the optimization process in the back propagation training process, aNMM can learn better question term weighting score than heuristic term weighting functions like IDF.