Towards Understanding and Answering Multi-Sentence Recommendation Questions on Tourism
Table 6: Performance of operator detection using gold sequence labels, and system generated labels
['[EMPTY]', 'Gold P', 'Gold R', 'System P', 'System R']
Operator Labeling: We create rules for commonly occurring operators – disjunctions, negations, and preferences. The default operator between semantic labels is a conjunction. The ‘Gold’ columns denote the performance when using gold semantic label mentions. The ’System’ columns are the performance when using labels generated by our sequence tagger. We find a significant drop in detection of disjunctions and preferences. A detailed study reveals that nearly 70% of all disjunction clauses occur in the context of attribute labels and almost all preferences are expressed for attributes. Since system’s recall for x.attr is low, the pipeline under-performance is not entirely surprising. However, as the next section shows, the low recall for these tags is not very problematic for the end QA task, as they are relatively infrequent. Moreover, even if we miss some attributes, the other attributes can often lead us to correct answers.