Intelligent Translation Memory Matching and Retrieval with Sentence Encoders
Table 7: Result comparison between Okapi and the Universal Sentence Encoder for each partition. Fuzzy score column represents the each partition. Okapi column shows the average METEOR score between the matches provided by the Okapi and the actual translations in that partition. USE column shows the average METEOR score between the matches provided by the Universal Sentence Encoder and the actual translations in that partition. Amount column shows the number of sentences in each partition. Bold shows the best result for that partition
['[BOLD] Fuzzy score', '[BOLD] Okapi', '[BOLD] USE', '[BOLD] Amount']
However, this is not a surprise given that METEOR score is largely based on overlapping ngrams, and therefore will reward segments that have a high fuzzy match score. However, we noticed that in most cases, the difference between the actual translation and the suggested match from either Okapi or Universal Sentence Encoder is just a number, a location, an organisation or a name of a person. We thought this might affect the results since we are depending on the Universal Sentence Encoder’s ability to retrieve semantically similar segments from the TM. For this reason, we applied a Named Entity Recognition (NER) pipeline on the actual translations, segments retrieved from Okapi and the segments retrieved from Universal Sentence Encoder. We detected locations, organisations and person names with the NER pipeline and replaced them with a placeholder. Last, we used a regular expression to detect number sequences in the segments and replaced them too with a place holder.