Multi-Domain Neural Machine Translation
Table 6: BLEU scores and p-values for test on Wikipedia-only data to compare the effect of Unsupervised clustering (Usup12, Usup5), supervised clustering (Super) and no-clustering approach (Ref). The p-values are shown in respect to the version where the value is -.
['[BOLD] NClust', '[BOLD] Usup12', '[BOLD] Usup5', '[BOLD] Super', '[BOLD] Ref']
In the final experiment, three models were trained: Supervised 5-domain source tag model Unsupervised 5-domain source tag model Unsupervised 12-domain source tag model Regular not domain-tagged model Unsupervised 5-domain model was included to compare the performance of supervised and unsupervised approach with the same amount of domains, giving an indication of the ”goodness” of these cluster assignments. The Unsupervised 12-domain model was included to compare the performance of best unsupervised clustering and the intuitively optimal supervised clustering. Supervised 12-domain model is not presented because we were not able produce such reasonable structure from ET Wikipedia. The models were trained for 48 hours. slightly outperforms Unsupervised 5-cluster approach (Usup5). The best option for Unsupervised clustering (Usup12) performs as well as the Supervised approach. The results show that Unsupervised approach is comparable in performance to the Supervised approach, which means that at least in this setting both of the approaches are viable. Even more so, when obtaining labelled data for supervised clustering can often require a lot of additional effort, the unsupervised approach is not chained by the (lack) of pre-existing knowledge about the data.