Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
Table 2: BLEU score on Chinese-English and English-German translation tasks for MRT and BR-CSGAN.
['Model', 'Chinese-English', 'English-German']
We show that MRT Shen et al. Considering a sentence pair (x,y), the training objective of MRT is calculated as ˆJ(θ′)=∑ys∈S(x)p(ys|x;θ′)Δ(ys,y) where Δ(ys,y) is a loss function (i.e., the sentence-level BLEU used in this paper) that measures the discrepancy between a predicted translation ys and the training example y, S(x) represents the set which contains all of the predictions given the input x, and θ′ is the parameters of the NMT model. Unfortunately, this objective is usually intractable due to the exponential search space. To alleviate this problem, a subset of the search space is sampled to approximate this objective. In this paper, when we set λ as zero, the objective for the proposed BR-CSGAN comes to J(θ)λ=0=∑Y1: TGθ(Y1: T|X)⋅Q(Y1:T,Y∗) where the Q(Y1:T,Y∗) is also a loss function between the predicted translation Y1: T and the training example Y∗. It is easy to be found that, under this condition (i.e., λ set as zero), the proposed BR-CSGAN optimizes almost the same objective with MRT. The only difference is that the reinforcement learning procedure is utilized in BR-CSGAN to maximize the total reward and MRT instead applies random sampling to approximate the risk. Actually, the BR-CSGAN is a weighted sum of the naive GAN (λ=1) and MRT (λ=0), and it incorporates the advantages of the two approaches. Specifically, compared to naive GAN which is trained without specific objective guidance, BR-CSGAN utilizes the BLEU objective to guide the generator to generate sentences with higher BLEU points. And compared to MRT which is trained only with the static objective, the BR-CSGAN applies a dynamic discriminator which updates synchronously with the generator, to feedback the dynamic rewards for the generator. Results show that the proposed BR-CSGAN consistently outperforms the MRT on the Chinese-English and English-German translations.