Character-Level Models versus Morphology in Semantic Role Labeling
Table 4: Results of ensembling via averaging (Avg) and stack generalization (SG). IOB: Improvement Over Best of baseline models
['[EMPTY]', '[BOLD] char+char3 Avg', '[BOLD] char+char3 SG', '[BOLD] char+char3  [ITALIC] IOB%', '[BOLD] char+oracle Avg', '[BOLD] char+oracle SG', '[BOLD] char+oracle  [ITALIC] IOB%', '[BOLD] char3+oracle Avg', '[BOLD] char3+oracle SG', '[BOLD] char3+oracle  [ITALIC] IOB%']
IOB shows the improvement over the best of the baseline models in the ensemble. Averaging and stacking methods gave similar results, meaning that there is no immediate nonlinear relations between units. We observe two language clusters: (1) Czech and agglutinative languages (2) Spanish, Catalan, German and English. The common property of that separate clusters are (1) high OOV% and (2) relatively low OOV%. Amongst the first set, we observe that the improvement gained by character-morphology ensembles is higher (shown with green) than ensembles between characters and character trigrams (shown with red), whereas the opposite is true for the second set of languages. It can be interpreted as character level models being more similar to the morphology level models for the first cluster, i.e., languages with high OOV%, and characters and morphology being more diverse for the second cluster.