Share your Model instead of your Data: Privacy Preserving Mimic Learning for Information Retrieval
Table 2. Performance of teacher and student models with different training strategies.
['[BOLD] Training strategy', '[BOLD] model', '[BOLD] MAP', '[BOLD] P@20', '[BOLD] nDCG@20']
The results generally suggest that we can train a neural ranker using mimic learning. Using weak supervision to train the teacher model, the student model performs as good as the teacher model. In case of training the teacher with full supervision, as the original training data is small, the performance of the teacher model is rather low which is mostly due to the fact that the big teacher model overfits on the train data and is not able to generalize well.