Using Multi-Label Classification for Improved Question Answering
Table 4: Performance of QA systems on QALD-6.
['[BOLD] Id', '[BOLD] Question', '[BOLD] KWGAnswer', '[BOLD] NbFramework', '[BOLD] PersianQA', '[BOLD] SemGraphQA', '[BOLD] UIQA WME', '[BOLD] UTQA English', '[BOLD] Optimal', '[BOLD] Metasystem']
The overall goal was to develop a metasystem that is able to perform better than the underlying systems to benefit from the multitude of existing QA research and development activities. An optimal selection algorithm (which would always choose the best performing QA system) would achieve 0.89 F-measure. Our best performing metasystem, trained on the 100 questions alone using the PSt classifier and only four features–namely #T, Loc, QW and QRT– is able to improve the best single system performance by 14.1% and reaches an F-measure of 0.78. This result supports our assumptions about the diversity of existing QA solutions and shows how a good feature design allows characterization and the effective use of QA systems. The overall results however also show clear weaknesses of existing QA solutions. In particular, questions which require solution modifiers (e.g., 9, 88, 17, 28, 33, 36, 49) remain a difficult problem that need to be tackled.