Deepening Hidden Representations from Pre-trained Language Models
Table 6: Ablation study over GRU layer number in fusion network on the development set of CoLA. The results are a median Matthews correlation of five random runs. The best result is in bold.
['Number', '0', '1', '2', '3']
To investigate the effect of GRU layer number in the fusion network, we set the GRU layer number from 0 to 3 and conduct the ablation experiments on the development dataset of CoLA. We observe that the modest number 2 would be a better choice.