Multimodal Named Entity Recognition for Short Social Media Posts
Table 3: NER performance (F1) on SnapCaptions with varying word embeddings vocabulary size. Models being compared: (W+C) Bi-LSTM/CRF + Bi-CharLSTM w/ and w/o modality attention (M.A.)
['Vocab Size', 'w/o M.A.', 'w/ M.A.']
By increasing the number of missing tokens artificially by randomly removing words from the word embeddings matrix (original vocab size: 400K), we observe that while the overall performance degrades, the modality attention module is able to suppress the peformance degradation. Note also that the performance gap generally gets bigger as we decrease the vocabulary size of the word embeddings matrix. This result is significant in that the modality attention is able to improve the model more robust to missing tokens without having to train an indefinitely large word embeddings matrix for arbitrarily noisy social media text datasets.