Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
Table 3: Influences of graph layer numbers on the development set.
['Model', 'LA.F1', 'SA.F1']
We can see the increase in the performance of our models when the number of layers increases from 0 to 2 (The 0-layer setting means that only the graph initialization module is used to obtain the graph representations). Then the model performance does not improve with the number of network layers increasing. We attribute it to the fact that the information between every two nodes in our proposed graph can be passed through in no more than two edges, and that increasing the size of randomly initialized parameters may not be beneficial for BERT fine-tuning.