Unsupervised Natural Question Answering with a Small Model
Table 2: Question answering accuracy.
['Embedding', 'dim', '[ITALIC] α', 'Score']
Adjusting the reported GPT-2 figures (downward) for this effect shows that the proposed model has higher performance for a much lower parameter count, even when using purely unsupervised training methods.