Transferability of Natural Language Inference to Biomedical Question Answering
Table 3: Experiments of Context Length Discrepancy. Factoid evaluation metrics are strict accuracy (SAcc), lenient accuracy (LAcc) and mean reciprocal rank (MRR). List evaluation metrics are precision (Prec), recall (Recall) and score of macro f1 (F1). Original indicates training in full document in SQuAD and using snippet in BioASQ. Document recurs to train in full document in SQuAD and use full abstract in BioASQ. Snippet denotes to train in a unified distribution of minimal context. All scores are averaged of 5 batch results. Bold denotes the best score of the columns in each task.
['Context Length Discrepancy # of Task', 'Context Length Discrepancy Setting', 'Context Length Discrepancy Sequence of Transfer Learning', 'Context Length Discrepancy Factoid (%)', 'Context Length Discrepancy Factoid (%)', 'Context Length Discrepancy Factoid (%)', 'Context Length Discrepancy List (%)', 'Context Length Discrepancy List (%)', 'Context Length Discrepancy List (%)']
When leveraging the MNLI dataset in the factoid and list types, we have to consider the discrepancy of context lengths. In the original setting, the SQuAD dataset is trained with full documents and the snippet is utilized in learning the BioASQ dataset. There are no performance gains in 6B test dataset. However, we observe that the performance enhances as the size of the training dataset increases as shown in the 7B test dataset.