Contextual Text Denoising with Masked Language Models
Table 4: SNLI classification accuracy with artificial noise and natural noise. {}^{*}: Applying denoising algorithm on original texts.
['Method', 'Original', 'Artificial', 'Natural']
Here we use state-of-the-art 400 dimensional Hierarchical BiLSTM with Max Pooling (HBMP) talman2019sentence. We use the same noise setting as the NMT experiments. We observe performance improvement with our method. To see if the denoising algorithm would induce noises to the clean texts, we also apply the algorithm to the original sentence and check if performance will degrade. It can be seen that, unlike the traditional robust model approach, applying a denoising algorithm on a clean sample has little influence on performance.