FAQ-based Question Answering via Word Alignment
Table 1: Evaluation on the test set
['[EMPTY]', 'English', 'Spanish', 'Japanese']
In this section, we evaluated our systems on the test sets. We tested three systems: (1) “Dense” takes the dense features, (2) “Sparse” takes both dense and sparse features, and (3) “SparseHidden” adds 300 hidden neurons to the second system. We also designed three baseline systems: (1) “BagOfWord” calculates question similarity by counting how many query words also occur in the candidate; (2) “IDF-VSM” represents each question with vector space model (VSM) (each dimension is the IDF score of the corresponding word), and calculates the cosine distance as the question similarity; (3) “Similarity” only uses our “similarity” feature. The “BagOfWord” and ”IDF-VSM” systems only counted the exactly matched words, so they didn’t work well. The “Similarity” system got some improvements by matching words with distributed word representations. The performance of the baseline systems also showed the difficulty of this task. By adding dense and sparse features extracted from word alignment, our systems significantly outperformed the baseline systems.