Question Answering on Scholarly Knowledge Graphs
Table 5: Performance comparison using the two datasets TabMCQ and ORKG-QA against JarvisQA and the baselines. The results suggest that JarvisQA outperforms the baselines by substantially on both datasets. Best results are highlighted for both datasets.
['[BOLD] System', '[BOLD] Dataset', '[BOLD] Precision @K #1', '[BOLD] Precision @K #3', '[BOLD] Precision @K #5', '[BOLD] Precision @K #10', '[BOLD] Recall @K #1', '[BOLD] Recall @K #3', '[BOLD] Recall @K #5', '[BOLD] Recall @K #10', '[BOLD] F1-Score @K #1', '[BOLD] F1-Score @K #3', '[BOLD] F1-Score @K #5', '[BOLD] F1-Score @K #10']
4.0.4 Experiment 4 - Performance on TabMCQ. We also show the performance of our system on the TabMCQ dataset against the ORKG-QA dataset. We see the same trend in both datasets, that JarvisQA outperforms the baselines by many folds. TabMCQ is not directly related to scholarly knowledge. However, it shows that JarvisQA can generalize to related data and can answer questions about it.