Controlling Text Complexity in Neural Machine Translation
Table 6: Adjacency ARI accuracy within grade level given by Adjacency level for the system output with respect to the target grade: Multitask model is able to better capture the target grade than the pipeline model when the difference between the source and the target grade is greater than 3.
['Adj. level', 'Model', 'Source Grade - Target Grade 1', 'Source Grade - Target Grade 2', 'Source Grade - Target Grade 3', 'Source Grade - Target Grade 4', 'Source Grade - Target Grade 5', 'Source Grade - Target Grade 6', 'Source Grade - Target Grade 7', 'Source Grade - Target Grade 8', 'Source Grade - Target Grade 9', 'Source Grade - Target Grade 10']
We aim to better understand to what degree models simplify the input text: how often does the output complexity exactly matches that of the reference? Does this change depend on the distance between input and output complexity levels? We derive the reading grade levels from ARI SenterSmith1967 and conduct this analysis for the best pipeline (“Translate then Simplify”) and multitask models (“All Tasks”). These adjacency scores are broken down according to the distance between input and target grade levels.