Detecting Egregious Conversations between Customers and Virtual Agents
Table 3: Cross domain performance ( models trained on company A’s data, tested on company B’s data).
['Model', 'Egregious P', 'Egregious R', 'Egregious F', 'Non-Egregious P', 'Non-Egregious R', 'Non-Egregious F']
the model’s performance when it was trained and tested on the same domain), with a degradation of only 9% in F1-score In addition, the results also show that the text-based model performs poorly when applied to a different domain (F1-score of 0.11). This may occur since textual features are closely tied to the training domain.