Improving Slot Filling by Utilizing Contextual Information
Table 2: Test F1-score for the ablated models
['[BOLD] Model', '[BOLD] SNIPS', '[BOLD] ATIS', '[BOLD] EditMe']
Our model consists of three major components: (1) MI: Increasing mutual information between word and its context representations (2) WP: Predicting the label of the word using its context to increase word level task-specific information in the word context (3) SP: Predicting which labels exist in the given sentence in a multi-label classification to increase sentence level task-specific information in word representations. In order to analyze the contribution of each of these components, we also evaluate the model performance when we remove one of the components and retrain the model. This Table shows that all components are required for the model to have its best performance. Among all components, the word level prediction using the contextual information has the major contribution to the model performance. This fact shows that contextual information trained to be informative about the final task is necessary to obtain the representations which could boost the performance.