Knowledge Adaptation: Teaching to Adapt
Table 2: Average results for domain adaptation from multiple sources for the comparison models and ours on the sentiment analysis benchmark. For the results in each column, the domain in the column header is used as target domain and the remaining three domains are used as source domains.
['[EMPTY]', '[BOLD] Book', '[BOLD] DVD', '[BOLD] Electronics', '[BOLD] Kitchen']
Results. Evaluating the combination of the source teacher models directly on the target domain (Teacher-only) produces the worst results, which underscores the need for methods that allow adaptation to the target domain. Training the student model on the soft targets of the teachers allows us to improve upon the teacher-only baseline significantly, which demonstrates the appropriateness of the teacher-student paradigm to the domain adaptation scenario. This showcases the usefulness of learning from soft targets in the domain adaptation scenario. Training on a general teacher model as well as on a combination of the general teacher and the source domain teachers allows us to improve results even further. Both models improve over existing approaches to domain adaptation from multiple sources and outperform approaches that rely on sentiment analysis-specific information Wu and Huang