Paying Attention to Attention: Highlighting Influential Samples in Sequential Analysis
Table 1: The influence on escalation of each user turn in a conversation. Higher weight turns are darker in color. As the Han weights are uniform, and, therefore, similar in color, it is difficult to infer the cause of escalation. In contrast, the weights of our visual on the same conversation show distinct turn importances, thus, quickly indicating the cause of escalation in this conversation.
['[BOLD] Turn', '[BOLD] User Text', '[BOLD] Han Weight', '[BOLD] Our Weight']
Unfortunately, there are instances where the attention weights for every turn at the point of escalation are nearly equal, requiring the representative to carefully read the history to determine the cause of escalation unassisted. Our application requires that the visualizations be generated in real-time at the point of escalation. The user must wait for the human representative to review the IVA chat history and resume the failed task. Therefore, we seek visualization methods that do not add significant latency to the escalation transfer. Using the attention weights for turn influence is fast as they were already computed at the time of classification. To overcome this problem, we develop a visualization method to be applied in the instances where the attention weights are uniform. Note that we present a technique that only serves to resolve situations when the existing attention weights are ambiguous; we are not developing a new attention mechanism, and, as our method is external, it does not require any changes to the existing model to apply. To avoid bias, the chosen reviewers had never used the specific IVA and were not familiar with its knowledge base although they may have performed similar tagging tasks in the past. The annotators were reminded that if a turn is given a darker color, then that turn supposedly has greater influence in determining escalation. They were, thus, given the task of determining if they agree with the visualization’s decision. A rating of 0 was instructed to be given on complete disagreement, and 10 upon perfect agreement.