Establishing Strong Baselines for the New Decade:Sequence Tagging, Syntactic and Semantic Parsing with BERT
(e) Results from the in-domain (ID) test sets.
['[EMPTY]', '[BOLD] DM', '[BOLD] PAS', '[BOLD] PSD', '[BOLD] AVG']
The baseline, \BERT, and Our baseline is a simplified version of dozat:18a; its average scores are 1.2% higher and 1.0% lower than the original model for ID and OOD, due to different hyperparameter settings. +BERT shows good improvement over \BERT for both test sets, implying that BERT embeddings are complementary to those pre-trained embeddings, and surpasses the previous state-of-the-art scores by 3% and 2% for ID and OOD, respectively. For DM and PAS, the baseline scores drop faster than those of \BERT as sentences get longer. For PSD, the score drop rates are similar between the two, due to the challenging nature of this dataset oepen:15a. This reflects that BERT embeddings can handle far-distant dependencies in longer sentences better. Prague Semantic Dependencies (PSD) is used for our labeling analysis because it is manually annotated and well-documented cinkova2006annotation.