Sequential Convolutional Neural Networks for Slot Filling in Spoken Language Understanding
Table 2: F1-score (%) of uni vs. bi-directional sequential CNNs trained with two different ranking loss functions
['Objectives', 'Methods', 'Score']
Five left context words, five right context words and the current word form the input of a feed-forward neural network with one hidden layer with size 100. We obtained an F1-score of 94.23% and 94.14% with this simple feed-forward network using ranking loss and hinge loss respectively. The results show that the context information from the past is more important than the future context. The future context, however, appears to provide meaningful information because their combination leads to better results. Moreover, the comparison between two different kinds of combinations of previous and future context (concatenation vs. addition) suggests to not mix up the information using addition.