Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies
Table 2: Hyper-parameters in our experiments.
['Name', 'Value']
This section includes details of our implementation. We use 100-dimensional, pre-trained Glove Pennington et al. Words or lemmas in the training corpora that do not appear in pre-trained embeddings are associated with randomly initialized vector representations. Embeddings of POS tags and other special symbol are also randomly initialized. We apply Adam as our optimizer. Preliminary experiments show that mini-batching led to a degradation in performance. When we apply policy gradient, we pre-train our model using supervised learning. We then use policy gradient for task-specific fine-tuning of our model. We find that updating parameters of BiLSTM and word embeddings during policy gradient makes training quite unstable. Therefore we fix the BiLSTM parameters during policy gradient. Hashimoto et al. We use task-specific MLPs for the three different linguistic formalisms: DM, PAS and PSD. We train the shared BiLSTM using multi-task learning beforehand, and then we fine-tune the task-specific MLPs with policy gradient.