Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks
Table 4: Accuracy (%) of our models and other methods from the literature. The majority class model predicts the most frequent class. SVM: [Dernoncourt et al.2016]. Graphical model: [Ji and Bilmes2006]. Naive Bayes: [Lendvai and Geertzen2007]. HMM: [Stolcke et al.2000]. Memory-based Learning: [Rotaru2002]. All five models use features derived from transcribed words, as well as previous predicted dialog acts except for Naive Bayes. The interlabeler agreement could be obtained only for SwDA. For the CNN and LSTM models, the presented results are the test set accuracy of the run with the highest accuracy on the validation set.
['[BOLD] Model', 'DSTC\xa04', 'MRDA', 'SwDA']
Overall, our model shows competitive results, while requiring no human-engineered features. Rigorous comparisons are challenging to draw, as many important details such as text preprocessing and train/valid/test split may vary, and many studies fail to perform several runs despite the randomness in some parts of the training process, such as weight initialization.