Knowledge Distillation in Document Retrieval
Table 2: Teacher Models Metrics
['Model', 'R(1)', 'Rmicro(3)', 'Rmacro(3)', 'Rmicro(5)', 'Rmacro(5)', 'DCG']
As our teacher models, we experimented with architectures that have been shown to give state of the art performance on the SQuAD task (Rajpurkar et al. Two models that performed the best were DCN (Xiong et al. without highway maxout network layers and Transformer (Vaswani et al. Note that the purpose of our study is not to find the best teacher model, but a teacher model that significantly outperforms the baseline student model. We picked the DCN model as the teacher for further experiments. Please note that the models were modified to be used in our \textlessd,c\textgreater classification task.