Does BLEU Score Work for Code Migration?
TABLE II: Correlation between Metrics with Semantic Score
['[EMPTY]', 'STS', 'TRS', 'GRS']
These results follow a similar trend: for each model, the correlation coefficients increase accordingly to the abstraction levels of code from text, syntax, to semantic representations in three metrics. For example, for mppSMT, the correlation coefficient between STS and semantic score is 0.549, whereas that for TRS is much greater at 0.820. The highest value is the correlation coefficient between GRS and semantic score at 0.910. A reason for this phenomenon is that for a pair of source code, if they are textually identical, they have the same AST representation, and the exact-match in AST leads to the equivalence between the corresponding PDGs. Meanwhile, the equivalence in a higher abstraction level does not necessarily mean the equivalence in a lower abstraction. For example, Fig.