Pay More Attention: Neural Architectures for Question-Answering
Table 4: Hyperparameter Tuning for DCA Model
['[BOLD] dropout', '[BOLD] learning rate', '[BOLD] hidden size', '[BOLD] glove', '[BOLD] batch-size', '[BOLD] F1', '[BOLD] EM']
Ideally, hyperparameter tuning for neural network architectures should be done using bayesian hyperparameter optimization but due to the lack of time we tried to do a random search on a small set of hyperparameters that we guessed could be more suitable. While, we didn’t find any significantly good set of parameters, we noticed that reducing the hidden size has a minor effect on improving the performance. This is probably because it reduces the system complexity which makes the model easier to train.