EQuANt (Enhanced Question Answer Network)
(a) SQuAD 2 dev set results.
['Name', 'No. of Params', 'Training Iterations', 'EM', 'F1', 'Accuracy']
As well as using this process to gain an understanding of the inner workings of QANet, we utilised light QANet to provide a principled initialisation for the training of our extended architecture. In Yu et al. As having access to this dataset would likely result in improved outcomes for our model, we initiated contact with the QANet authors, however access to the augmented dataset was not granted. The quoted number of training iterations for other models in tables These EM/F1 scores are 9.98/6.853 lower than the corresponding results for the full QANet architecture Yu et al. , implying that our choice to employ a lightweight architecture has a noticeable impact on performance. We evaluated these trained light QANet models on the SQuAD 2 dev set, implicitly treating all questions as answerable. In fact, their performance is identical. This is explained by both models learning to output a constant answerability probability of 0.69, independent of the query-context pair considered. Note that this probability matches the proportion of SQuAD2 training examples which are answerable, meaning that these models have been unable to extract the necessary features for accurately predicting answerability and have defaulted to the most basic frequentist approach of predicting the mean.