Question-Answering with Grammatically-Interpretable Representations
Table 3: Symbol 27
['[BOLD] Token', '[BOLD] Similarity']
To interpret the lexical-semantic content of the TPR symbols s(t) learned by the TPRN network: s(t)=SaS(t)∈R10 is calculated for all (120,950) word tokens w(t)in the validation set. The cosine similarity is computed between aS(t) and the embedding vector of each symbol. The symbol with maximum (cosine) similarity is assigned to the corresponding token. For each symbol, all tokens assigned to it are sorted based on their similarity to it; tokens of the same type are removed, and the top tokens from this list are examined to assess by inspection the semantic coherence of the symbol assignments (see The results provide significant support for our hypothesis that each symbol corresponds to a particular meaning, assigned to a cloud of semantically-related word tokens. For example, symbol 27 and symbol 6 can be respectively interpreted as meaning ‘occupation’ and ‘geopolitical unit’. Symbol 11 is assigned to multiple forms of the verb to be, e.g., was (85.8% of occurrences of tokens in the validation set), is, (93.2%) being (100%) and be (98%). Symbol 29 is selected by 10 of the 12 month names (along with other word types; more details in supplementary materials). Other symbols with semantically coherent token sets are reported in the supplementary materials.