\papertitle
Table 1: Model performance on the blind, held-out test set of ShARC. The evaluation metrics are micro and macro-averaged accuracy in classifying bewteen the decisions yes, no, irrelevant, and inquire. In the event of an inquiry, the generated follow-up question is further evaluated using the BLEU score. In addition to official evaluation metrics, we also show a combined metric (“Comb.”), which is the product between the macro-averaged accuracy and the BLEU4 score.
['Model', 'Micro Acc.', 'Macro Acc.', 'BLEU1', 'BLEU4', 'Comb.']
Compared to previous results, E3 achieves a new state-of-the-art, obtaining best performance on micro and macro-averaged decision classification accuracy and BLEU4 scores while maintaining similar BLEU1 scores. These results show that E3 both answers the user’s original question more accurately, and generates more coherent and relevant follow-up questions.