Knowledge Base Inference for Regular Expression Queries
Table 7: Performance of compositional training of KBC and Path Query dataset.
['[BOLD] Dataset', '[BOLD] Trained on', '[BOLD] KBC Task  [BOLD] MRR', '[BOLD] KBC Task  [BOLD] HITS@1', '[BOLD] KBC Task  [BOLD] HITS@10', '[BOLD] Path Queries  [BOLD] MRR', '[BOLD] Path Queries  [BOLD] HITS@1', '[BOLD] Path Queries  [BOLD] HITS@10']
We found that jointly training for KBC queries and Path queries allowed the embeddings of entities and relations to be trained for both the tasks. We also observed that combining multiple copies of KBC dataset with Path query dataset helped in path training as well as in stabilizing the performance metrics of original KBC task. The KBC to Path dataset ratio of 4:1 gave the best performance on path queries for both FB15k-Path and Wiki100-Path. Overall, it resulted in a massive increase in performance in path queries with a marginal reduction in KBC performance. Our empirical evaluation reveals a few interesting insights. First of all, it is worth noting that Wiki100, even with much lower performance on KBC task as compared to FB15k, admits a stronger baseline under regex queries. The performance gains of MLP and FP models observed on FB15k dataset – e.g., about 33% improvement in HITS@10, do not seem to carry over to Wiki100 setting, which shows just 6% improvement over the baseline (HITS@10 with UNIT-AVG). This shows that Wiki100 is a challenging KB to model regex queries.