Training a Code-Switching Language Model with Monolingual Data
Table 2: Results for unsupervised bilingual word translation using different approaches, all with monolingual training: translation (A) from Mandarin to English and (B) from English to Mandarin.
['[width=5.0cm]ApproachMetric', '(A) Mandarin \\rightarrow English MRR', '(A) Mandarin \\rightarrow English P@10', '(B) English \\rightarrow Mandarin MRR', '(B) English \\rightarrow Mandarin P@10']
To analyze whether words with equivalent semantics in different languages are mapped together with the proposed approaches, we conducted experiments on unsupervised bilingual word translation. If the translated word of w is ranked as the r-th candidate, then the reciprocal rank is \frac{1}{r}. The mean reciprocal rank (MRR) is used as an evaluation metric, which is the average of the reciprocal ranks; thus the MRR should be less than 1.0, and the closer to 1.0 the better. The proportion of correct translations that are in the top 10 candidate list (r\leq 10) is also reported as “P@10” [Xing2015NormalizedWE]. In order to mitigate the degradation in performance caused by low-frequency words, we selected words only with a frequency greater than 80, resulting in about 200 vocabulary words in Mandarin and English respectively, and 55 bilingual pairs used for unsupervised bilingual word translation. We see performance for Mandarin-English translation (column (A)) in both MRR and P@10 that is worse than that in the reverse direction (column (B)). Row (i) demonstrates that the unconstrained baseline performs poorly, whereas additional constraints and normalization in rows (ii) and (iii) yield significantly improved MRR and P@10 compared with row (i). This suggests that constraints and normalization for CS language modeling indeed enhance semantic mapping.