Adversarial Domain Adaptation for Machine Reading Comprehension
Table 5: Semi-supervised domain adaptation experiment with varied labeling ratio on the target-domain dataset. Setting: adaptation from SQuAD to NewsQA.
['Ratio', 'SAN', 'AdaMRC + SAN']
As an additional experiment, we also evaluate the proposed AdaMRC framework for semi-supervised domain adaptation. We randomly sample k portion of labeled data from the target domain, and feed them to the MRC model. The ratio of labeled data ranges k from 0% to 100%. However, the gap is decreasing when the labeling ratio increases. When the ratio is 20% or smaller, there is noticeable improvement. When the ratio is set to 50%, the two methods result in similar performance. When the ratio is increased to 100%, i.e., fully supervised learning, the performance of AdaMRC is slightly worse than SAN. This is possibly because in a supervised learning setting, the encoder is trained to preserve domain-specific feature information. The overall results suggest that our proposed AdaMRC is also effective in semi-supervised setting, when a small portion of target-domain data is provided.