Hierarchical Multi-Label Dialog Act Recognition on Spanish Data
Table 11: Results on Level 3 using the two segment representation approaches.
['[ITALIC] n', '[BOLD] MR  [ITALIC] m', '[BOLD] MR  [ITALIC] s', '[BOLD] Acc  [ITALIC] m', '[BOLD] Acc  [ITALIC] s', '[BOLD] P  [ITALIC] m', '[BOLD] P  [ITALIC] s', '[BOLD] R  [ITALIC] m', '[BOLD] R  [ITALIC] s', '[BOLD] F1  [ITALIC] m', '[BOLD] F1  [ITALIC] s', '[BOLD] HL  [ITALIC] m', '[BOLD] HL  [ITALIC] s']
However, in this case, the difference is less pronounced. This is due to the fact that the Level 3 focuses on the information that is explicitly referred to in the segments and, thus, is even more keyword-oriented than Level 2. That also explains the average results above 96% on every non-loss metric. The average training times per epoch are the same as those for Level 2. Once again, this is explained by the nature of Level 3 and its focus on what is explicitly referred to in the current segment. Thus, the preceding segments are not relevant. In this case, considering the Level 2 label of the same segment leads to a statistically significant improvement (p≈0.03). This can be explained by the fact that when a certain kind of information is explicitly referred to in a segment, it is typically also focused by the segment and, thus, overlaps between the Level 2 and 3 labels of a segment are common. Considering Level 2 information from preceding segments does not lead to statistically significant improvements (p≈0.13). Since the Level 1 labels are related to the generic intention of the segment, they have no direct relation to what is explicitly referred to in the segment and, thus, to the Level 3. Once again, this is explained by the fact that the Level 3 is highly focused on keywords and, thus, the fact that the system segments are scripted does not have the same influence on classification.