Evaluation of Neural NER Models on Turkish
Table 4: Settings of Masked Language Models
['Model', '# Hidden Layers', '# Hidden Units', '# Attention Heads', 'Vocabulary Size']
In our preliminary experiments, we observe that tokens produced by multilingual BERT tokenizer do not correlate well with morphological units given in the dataset. Although this is not the case for other tokenizers, BERTurk has a small vocabulary size and XMLR is not trained solely for Turkish language. Due to these reasons, we do not report any results on the use of default tokenizers in BiLSTM networks. Moreover, in our preliminary experiments, we observe around 20%-40% mismatches between subword tokens obtained by our trained Tr SentPiece tokenizer and vocabularies used in pretrained models. Thus, we do not report results regarding the use of Tr SentPiece tokenizer in any of our transformer-based networks. Networks are trained by Adam optimizer with fixed weight decay, with initial learning rate of 5e−05, and gradient clipping of 1.