.
Table 6. Further analysis on column type annotation: Model performance for 5 selected types. Results are F1 on validation set.
['Method', 'person', 'pro_athlete', 'actor', 'location', 'citytown']
Our model substantially outperforms the baseline, even when using the same input information (only entity mention vs Sherlock). Adding table metadata information and entity embedding learned during pre-training further boost the performance to 94.75 under F1. In addition, our model achieves such performance using only 10 epochs for fine-tuning, which demonstrates the efficiency of the pre-training/fine-tuning paradigm. However, fine-grained types like actor and citytown are much more difficult to predict. More specifically, it is hard for a model to predict such types for a column only based on entity mentions in cells. On the other hand, using table metedata works much better than using entity mentions (e.g., 70.86 vs 58.86 for actor and 72.54 vs 60.13 for citytown). This indicates the importance of table context information for predicting fine-grained column types.