Building Memory with Concept Learning Capabilities from Large-scale Knowledge Base
Table 4: Link prediction results (of unseen entities) on FB4M-desc.
['[BOLD] Model', '[BOLD] Mean rank 1k samples', '[BOLD] Mean rank 10k samples', '[BOLD] Mean rank 80k samples', '[BOLD] Hits@10 (%) 1k samples', '[BOLD] Hits@10 (%) 10k samples', '[BOLD] Hits@10 (%) 80k samples']
It has been shown in Section 4.2 that our framework well regularizes the neural embedding model for memory storage. Next we use FB4M-desc to evaluate the capability of our framework on learning new concepts and performing reasoning based on learnt embeddings. Note that the test set contains millions of triples, which is very time-consuming in the ranking-based evaluation. So we randomly sample 1k, 10k and 80k triplets from the test set to report the evaluation statistics. We can see that CNN consistently outperforms MLP in terms of both mean rank and hits@10. All the triplets in the test set of FB4M-desc include an entity unseen in the training set on one side, requiring the model to understand natural language descriptions and to do reasoning based on it. As far as we know, no traditional knowledge base embedding model can compete with us on this task, which again claims the novelty of our framework.