Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers
Table 2: Our model with different prediction modules.
['[BOLD] Method', '[BOLD] dev', '[BOLD] bc', '[BOLD] cts', '[BOLD] wl', '[BOLD] avg']
Results show that the usage of the linear predictor gives better results. This is consistent with the motivation of the pre-trained encoders: by unsupervised pre-training the encoders are expected to be sufficiently powerful thus adding more complex layers on top does not improve the capacity but leads to more free parameters and higher risk of over-fitting.