UH-PRHLT at SemEval-2016 Task 3:Combining Lexical and Semantic-based Features forCommunity Question Answering
Table 3: Results of Subtask C: English Question-External Comment Similarity. (a) Baselines; (b) proposed approach.
['[EMPTY]', '[BOLD] Model', '[BOLD] Ranking measures  [BOLD] MAP', '[BOLD] Ranking measures  [BOLD] AvgRec', '[BOLD] Ranking measures  [BOLD] MRR', '[BOLD] Classification measures  [BOLD] Acc.', '[BOLD] Classification measures  [BOLD] P', '[BOLD] Classification measures  [BOLD] R', '[BOLD] Classification measures  [BOLD] F1']
In this case, we are ranking 100 comments (10 times more compared to the other subtasks). Therefore, this has been the most difficult subtask. However, we obtained improvements in line with those reported for the other subtasks. Compared to the search engine baseline, the MAP, AvgRec, and MRR improved 8.7%, 8.5%, and 7.5% respectively when using the development partition. The accuracy and F1-measure improved 61.5% and 12.2% respectively. The largest number of comments to rank, and the use of top 10 results when measuring results, benefited our approach with this especially high difference in accuracy.