Learning Better Word Embedding by Asymmetric Low-Rank Projection of Knowledge Graph
Table 1: Number of head entities per tail entity and number of tail entities per head entity in FB13.
['[BOLD] #Tail per Head Mean', '[BOLD] #Tail per Head Std.Deviation', '[BOLD] #Head per Tail Mean', '[BOLD] #Head per Tail Std.Deviation']
Actually such kind of complex relationships are very common in knowledge graphs. For every relationship in FB13, we calculate the average number of head entities corresponding to one tail entity and the average number of tail entities corresponding to one head entity. Then we obtain the means and standard deviations of such values under different relationships. In addition, the standard deviation for #Head per Tail is fairly large, indicating that the degrees of non one-to-one mappings from tail entity to head entity vary drastically across different relationships. This clearly shows that the issue is very serious and we should tackle it in order to learn a reasonable word embedding. Second, the loss function LK adopts simple arithmetic operations on the embedding vectors of the head and tail entities, implying that both entities are located in the same space. However, the fact is that head entities are usually more concrete and tail entities are more abstract, making it unreasonable to simply regard them as in a homogeneous space. Still use the above example, for the relationship r=“cause of death”, all the head entities are real human names whereas all the tail entities are abstract reasons of death.