Modeling Voting for System Combination in Machine Translation
Table 5: Generalization ability evaluation. We report the results of single MT systems and system combination methods (i.e. Hier and Ours) on the Chinese-English task. N denotes the number of single MT systems. Note that both system combination methods were trained on the outputs of three single MT systems (i.e., N=3) and tested on various number of single systems (i.e., N=2,3,4,5). “†” and “††”: significantly better than the best system among inputs (p<0.05 and p<0.01). “‡” and “‡‡”: significantly better than “Hier” (p<0.05 and p<0.01).
['N 2', 'Single MT Systems 47.44', 'Single MT Systems 46.18', 'Single MT Systems -', 'Single MT Systems -', 'Single MT Systems -', 'Training No', 'Test Yes', 'Hier 47.35', 'Ours  [BOLD] 47.76{}^{{\\dagger}{\\ddagger}}', '\\Delta +0.41']
We are interested in evaluating whether a system combination method performs well when the number of single MT systems during testing is different from that during training. We shared the parameters of each hypothesis encoder \mathrm{Encoder}^{\mathrm{hyp}}(\cdot) so that the system combination model has the ability to generalize to different number of single systems.