Learning Structural Kernels for Natural Language Processing
Table 4: Error scores for the Quality Estimation task (lower is better). Results are in terms of post-editing time per word. Bold scores are the best ones for each dataset.
['[EMPTY]', 'French-English MAE', 'French-English RMSE', 'English-Spanish MAE', 'English-Spanish RMSE']
They indicate a number of interesting findings: For the fr-en dataset, the GP models combining tree kernels with an RBF kernel outperform all other models. Results for the en-es dataset are less consistent, probably due to the small size of the dataset, but on average they are better than their SVM counterparts. The SVMs using a combination of kernels performs worse than using the RBF kernel alone. Inspecting the models, we found that grid search actually harms performance. For instance, for the fr-en dataset, MAE and RMSE for the RBF + SSTK × model before grid search are 0.4681 and 0.6016, respectively. On the other hand, for this dataset all GP models achieve better results after optimization. Unlike in the Emotion Analysis task, fixing α results in better performance, even though the resulting models have lower marginal likelihood than the ones with free α. The same effect happened when comparing the SASSTK models with the SSTK ones for the en-es dataset. Both cases are evidence of model overfitting.