Who Did They Respond to? Conversation Structure Modeling Using Masked Hierarchical Transformer
Table 6: Graph results on the Ubuntu IRC test set (Dev set in [kummerfeld-etal-2019-large]). The methods with suffix ‘*’ are using the additional hand-crafted features in [kummerfeld-etal-2019-large]. The baseline models with ‘+F’ suffix are with the additional features.
['Method', 'P', 'R', 'F']
Graph metrics for Ubunut IRC dataset are first considered including the precision, recall and F1 scores for the parent relationship prediction. We first consider the performance of the baseline models without any additional features. For the decomposable attention model and ESIM, we use the Glove embedding by [kummerfeld-etal-2019-large]. Besides, we consider models that use those features in [kummerfeld-etal-2019-large]. Since our test set is the development set in their paper and no results have been reported, we train the Linear and FeedForward model in their paper under our train/dev split and evaluate them on our test set. Again, the performance of our model is the best among all those models.