User Generated Data: Achilles’ heel of BERT
Table 2: Comparative results on STS-B dataset with different tokenizers
['Error', 'WordPiece', 'WhiteSpce', 'N-gram(n=6)']
Even though wordPiece tokenizer has the issues stated earlier, it is still performing better than whitespace and character n-gram tokenizer. This is primarily because of the vocabulary overlap between STS-B dataset and BERT vocabulary.