Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML
Table 4: Sentence-level BLEU (S-B, training reward) and corpus-level BLEU (C-B, standard evaluation metric) scores for RAML with different τ.
['Metric', 'NER Acc.', 'NER F1', 'NER E.M.', 'Parsing UAS', 'Parsing E.M.', 'MT S-B', 'MT C-B', 'MT E.M.']
Our approach, albeit simple, surprisingly outperforms previous works. Specifically, all previous methods require a pre-trained ML baseline to initialize the model, while RAML learns from scratch. This suggests that RAML is easier and more stable to optimize compared with existing approaches like RL (e.g., Ranzato et al. Finally, we remark that RAML performs consistently better than the ML (27.66) across most temperature terms. We observe that RAML outperforms ML on both the directly optimized rewards (token-level accuracy for NER, UAS for dependency parsing and sentence-level BLEU for MT) and task-specific evaluation metrics (F1 for NER and corpus-level BLEU for MT). This is in line with our theoretical analysis, in that RAML and ML achieve better prediction functions w.r.t. their corresponding rewards they try to optimize. For the sake of completeness, we also experimented using the negative Hamming distance as the reward function for RAML, as proposed in Norouzi et al.