Who Did They Respond to? Conversation Structure Modeling Using Masked Hierarchical Transformer
Table 7: Conversation results on the Ubuntu IRC test set (dev set in [kummerfeld-etal-2019-large]). The methods with suffix ‘*’ are using the additional hand-crafted features in [kummerfeld-etal-2019-large]. The baseline models with ‘+F’ suffix are with the additional features.
['Method', 'VI', '1-1', 'P', 'R', 'F']
In particular, a cluster in a conversation is defined as a connected component in the conversation graph. Three types of metrics are considered as the modified Variation of Information (VI) in [kummerfeld-etal-2019-large], One-to-One Overlap (1-1) of the cluster [elsner-charniak-2008-talking], and the precision, recall, and F1 score between the cluster prediction and ground truth. Our approach combined with the additional features achieves the best performance among all the metrics. This suggests our model is both good at finding the correct parent of an utterance and reconstructing the whole conversation.