Generative Stock Question Answering
Table 9: Evaluation of diversity at different granularities, ranging from 1-gram to 4-gram. Higher number denotes higher diversity.
['[BOLD] Model', '[BOLD] Diversity at  [ITALIC] n-gram 1', '[BOLD] Diversity at  [ITALIC] n-gram 2', '[BOLD] Diversity at  [ITALIC] n-gram 3', '[BOLD] Diversity at  [ITALIC] n-gram 4']
We analyze the diversity of our model comparing to the three baselines quantitatively and qualitatively. If the diversity for one model is better, the n-grams of the generated sentences will be significantly larger than other models. Starting from this intuition, we count the number of unique n-grams appears in each answer and normalize it with the total number of n-grams. As the retrieved answers are searched from the training data, the diversity of the retrieved answers are significantly better than other models. The generation model tends to generate generic responses, which reconfirm our previous findings. By utilizing retrieved answers, the diversity of the hybrid model increases significantly, which shows the benefits of external information source.