IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings
Table 3: Performance on subset 2 (MR = mean rank, ACC = accuracy)
['[BOLD] Embedding', '[BOLD] Dim', '[BOLD] Unsupervised  [BOLD] Cosine', '[BOLD] Unsupervised  [BOLD] Cosine', '[BOLD] Unsupervised  [BOLD] L1', '[BOLD] Unsupervised  [BOLD] L1', '[BOLD] Unsupervised  [BOLD] L2', '[BOLD] Unsupervised  [BOLD] L2', '[BOLD] Supervised  [BOLD] Naïve Bayes', '[BOLD] Supervised  [BOLD] Naïve Bayes', '[BOLD] Supervised  [BOLD] Naïve Bayes', '[BOLD] Supervised  [BOLD] Logistic Regression', '[BOLD] Supervised  [BOLD] Logistic Regression', '[BOLD] Supervised  [BOLD] Logistic Regression']
Contrary to subset 1, supervised approaches perform better than unsupervised approaches on subset 2. Naïve Bayes dominates among the supervised classifiers for the Word2vec word-embeddings while logistic regression dominates for BERT embeddings. Since we obtain 100% accuracy for subset 1, as assumed based on the rule, and the training dataset is small, we add the terms in subset 1, with their predicted labels, in the training dataset. Hence, we present results for subset 2 on 100 training data points (original train dataset) as well as on 166 training data points (original train dataset + subset 1). In the following systems, we use results with 166 training data points on subset 2 for consistency.