A Hybrid Architecture for Multi-Party Conversational Systems
Table 16: Evaluation of different classifiers in the most recent version of the training set
['[EMPTY]', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F1', '[BOLD] Accuracy']
As we mentioned, the use of an 1NN classifier has allowed the developer of the system to easily add new intent classes and samples whenever they judged it necessary, so that the system could present new actions, or the understanding of the intents could be improved. As a consequence, the initial training set grew from 37 to 63 classes, and from 415 to 659 samples, with the number of samples per class varying from 2 to 63. we present the accuracy of the same classifiers used in the previous evaluation, in the new set. In this case, we observe some drop in accuracy for 1NN, showing that this classifier suffers in dealing with scalability. On the other hand, SVM has shown to scale very well to more classes and samples, since its accuracy kept at a very similar level than that with the other set, with a difference of about only 1 percentage point.