ℂ𝕆𝕄𝔼𝕋: Commonsense Transformers for Automatic Knowledge Graph Construction
Table 12: Human score of generations of Atomic commonsense for the regular ℂ𝕆𝕄𝔼𝕋 model and the ℂ𝕆𝕄𝔼𝕋 + category meta tokens
['[BOLD] Model', '[BOLD] oEffect', '[BOLD] oReact', '[BOLD] oWant', '[BOLD] xAttr', '[BOLD] xEffect', '[BOLD] xIntent', '[BOLD] xNeed', '[BOLD] xReact', '[BOLD] xWant', '[BOLD] Total']
Leveraging the prior knowledge that certain relation types in the Atomic knowledge graph are linked to each other, we explore providing these group identities as additional tokens in the relation. For example, when generating the completion of a xReact relation, the model would receive as input the following meta-tokens:  ,  ,  ,   – thereby providing common context with other relations that are part of the same groupings (e.g., generating a phrase for a xWant relation would receive the   and < POST > tokens as input, but not < Involuntary>). Depending on the relation for a particular training example (e.g., xReact), a set of meta-tokens are appended to the relation tokens, Xr, that provide hierarchical relational information, allowing the model to share information across relation types. Because the model with the hierarchical meta-tokens performed worse than the regular ℂ𝕆𝕄𝔼𝕋, we did not run additional experiments on this ablations.