Adaptability of Neural Networks on Varying Granularity IR Tasks
Table 1: Comparison of a CNN and LSTM network after hyper-parameter tuning over Yahoo’s Webscope L4 collection.
['Method', 'MRR', 'P@1']
Due to the span of relevant information across the length of candidate answers, LSTM networks are uniquely suited to this task as they are able to model syntactic and semantic dependencies across positions in a sequence and focus less on matching than the CNN does. The purpose of this test was to demonstrate the ability of the two network structures to retain information across long sequences, therefore the candidate answer pool for each query consisted of 10 randomly sampled answers from the collection. The filter lengths of CNNs are unable to capture long term dependencies that span multiple sentences, which results in its poor performance relative to the LSTM network.