Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction
Table 4: Ablation of attention mechanisms with WordDecoding (WDec) and PtrNetDecoding (PNDec) model.
['Model', 'NYT29 Prec.', 'NYT29 Rec.', 'NYT29 F1', 'NYT24 Prec.', 'NYT24 Rec.', 'NYT24 F1']
For our pointer network-based decoding model, we use three variants of the single attention model. First, we use hDt−1 to calculate qit in the attention mechanism. Next, we use yprev to calculate qit, where Wq∈R(8dp+dr)×dh. In the final variant, we obtain the attentive context vector by concatenating the two attentive vectors obtained using hDt−1 and yprev. This gives the best performance with the pointer network-based decoding model. : This model uses an encoder-decoder approach with N-gram attention mechanism for knowledge-base completion using distantly supervised data. The encoder takes the source sentence as input and the decoder outputs the two entity IDs and relation ID for every tuple. During training, it uses the mapping of entity names and their Wikidata IDs of the entire Wikidata for proper alignment. Our task of extracting relation tuples with the raw entity names from a sentence is more challenging since entity names are not of fixed length. Our more generic approach is also helpful for extracting new entities which are not present in the existing knowledge bases such as Wikidata. WordDecoding with single attention achieves the highest F1 score on both datasets. We also see that our copy mechanism improves F1 scores by around 4–7% in each attention mechanism with both datasets. PtrNetDecoding achieves the highest F1 scores when we combine the two attention mechanisms with respect to the previous hidden vector of the decoder LSTM (hDt−1) and representation of all previously extracted tuples (yprev).