Table Search Using a Deep Contextualized Language Model
Table 5. Results using feature-based approaches. The superscript ‡ denotes statistically significant improvements over all baseline methods.
['Method Name', 'MAP', 'MRR', 'NDCG@5', 'NDCG@10', 'NDCG@15', 'NDCG@20']
In the experiment, we tried different methods to incorporate additional features. For example, we can directly concatenate additional features without any transformation with BERT features and feed the concatenated vector to the regression layer. We also tried to predict two relevance scores with BERT features and additional features separately, and then linearly transform them into a weighted relevance score. However, all of those variants perform worse than BERT-text. It is possible that BERT performance highly depends on the optimization strategy and adding other components for joint training can have negative impact on the fine-tuning process. To avoid such a case, we adapt BERT to a feature-based approach. For the three item types, we only include the results of models using the best content selectors. All methods have significant improvements compared with fine-tuned approaches. Among the baselines, Hybrid-BERT-Rand-Col has the most improvement, which is even better than the best performance of BERT using content selectors for column items. Hybrid-BERT-Row-Max still achieves the best performance and the improvements over baselines are statistically significant at the level of p=0.005.