Alquist: The Alexa Prize Socialbot
Table 1: Accuracy of intent detection
['Method', 'Accuracy']
One of the most important parts of our dialogue system is the intent detection used in dialogue manager. We report the accuracy of four methods we tested to recognize intent in the user’s message. Tested methods of intent detection are: TF-IDF: We found the hyper-parameters of TF-IDF by grid search. They are following, analyzer: word, ngram_range: (1,2), max_df: 0.9, min_df: 0.0, norm: l1, smooth_idf: False, sublinear_tf: True. The rest of parameters has default values. Embeddings: We convert training examples to vector representation by calculating the average of embeddings of example’s words. We normalize vector representations of training examples to unit length. We do the same for the user’s input. We calculate cosine similarity of user’s input to each training examples and use the label of the most similar training example as the recognized intent. We achieved the best accuracy with the glove.42B.300d variant of vectors. Logistic regression: We use the scikit-learn implementation with default parameters and balanced classes. The input is the vector of word uni- and -bi grams concatenated with the vector of POS uni- and bi-grams. The number of convolutions is set to N/2, the activation function is tanh, and max-pooling is over the whole sentence. Since the convolutions of message and focus input are separate layers, the resulting vectors are then concatenated. Additionally, the label predicted for the previous message is concatenated to the vector as well. The concatenated output is then fed into a dense layer with output dimension equal to 300, followed by dropout with rate 0.5 and finally with softmax activated dense layer with output dimension equal to a number of classes.