Look at the First Sentence:Position Bias in Question Answering
Table 3: Position bias in different positions. Each model is trained on a biased SQuAD dataset (SQuADktrain) and evaluated on SQuADdev.
['[EMPTY]', '[BOLD] SQuADdev EM', '[BOLD] SQuADdev F1', '[BOLD] SQuADdev EM', '[BOLD] SQuADdev F1', '[BOLD] SQuADdev EM', '[BOLD] SQuADdev F1', '[BOLD] SQuADdev EM', '[BOLD] SQuADdev F1']
Due to the blurred sentence boundaries, position bias is less problematic when k is large. We observe a similar trend in BERT and XLNet while a huge performance drop is observed in BiDAF even with a large k.