Simple Question Answering with Subgraph Ranking and Joint-Scoring
Table 4: Subgraph Selection Results
['Rank Method', 'Top-N', 'Recall']
Subgraph selection The subgraph selection results for our approach and one of the state-of-the-art baselines Yin et al. Both the baseline and our approach use unigrams to retrieve candidates. We see that the literal score used in the baseline performs well and using the semantic score (the log-likelihood) This is due to the nature of how the questions in the SimpleQuestions dataset are generated—the majority of the questions only contain mentions matching the subjects in the Freebase in the lexical level, making the literal score sufficiently effective. However, we see that combining the literal score and semantic score outperforms the baseline by a large margin. For top-1, 5, 10, 20, 50 recall our ranking approach surpasses the baseline by 11.9%, 5.4%, 4.6%, 3.9%, 4.1%, respectively. Our approach also surpasses other baselines Lukovnikov et al. ; Yu et al. under the same settings. We note that the recall is not monotonically increasing with the top-n. The reason is that, as opposed to conventional methods which rank the entire subgraph returned from unigram matching to select the top-n candidates, we choose only the first 200 candidates from the subgraph and then rank them with our proposed ranking score. This is more efficient, but at the price of potentially dropping the correct facts. One could trade efficiency for accuracy by ranking all the candidates in the subgraph.