Exploring Prediction Uncertainty in Machine Translation Quality Estimation
Table 2: Asymmetric loss experiments results. The first line in each table corresponds to a standard GP while the others are Warped GPs with different warping functions. All models use the Matèrn52 kernel. The optimistic setting corresponds to w=1/3 for AL and w=0.75 for linex. The pessimistic setting uses w=3 for AL and w=−0.75 for linex, except for English-German, where w=−0.25.
['[BOLD] English-Spanish', '[BOLD] English-Spanish  [BOLD] Optimistic', '[BOLD] English-Spanish  [BOLD] Optimistic', '[BOLD] English-Spanish  [BOLD] Pessimistic', '[BOLD] English-Spanish  [BOLD] Pessimistic']
In the optimistic scenario the tanh-based warped GP models give consistently better results than standard GPs. The log-based models also gives good results for AL but for linex the results are mixed except for en-es. This is probably again related to the larger sizes of the fr-en and en-de datasets, which allows the tanh-based models to learn richer representations.