On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification
Table 5: Encoder ablation analysis on multiple binary classification.
['Model', 'Comp.', 'Cont.', 'Exp.', 'Temp.']
We can find that the main gaps between encoding without concatenating (BMGF-RoBERTa-Siamese) and encoding without segment embeddings (BMGF- RoBERTa w/o SE) come from the Comp. and Temp. classes. And adding SE brings a qualitative leap in all four classes because BMGF-RoBERTa finally captures clues of two arguments by multiple attention and relative segment order embeddings. That is, a more powerful representation layer can bring unexpected improvement and understand implicit relations better. But how to inject useful information to the representation module looks more important. In our paper, we inject the “previous-next” context relationship by the concatenation and segment embeddings for implicit discourse relation classification.