meProp: Sparsified Back Propagation for Accelerated Deep Learningwith Reduced Overfitting
Table 4: Adding the dropout technique.
['[BOLD] POS-Tag (Adam)', 'Dropout', 'Test Acc (%)']
As we can see, meProp can achieve further improvement over dropout. In particular, meProp has an improvement of 0.46 UAS on Parsing. The results suggest that the type of overfitting that meProp reduces is probably different from that of dropout. Thus, a model should be able to take advantage of both meProp and dropout to reduce overfitting. The POS tagging task is a well-known benchmark task, with the accuracy reports from 97.2% to 97.4%