Sampling Bias in Deep Active Classification: An Empirical Study
Table 3: Proportion of Support Vectors intersecting with our actively selected set calculated by |SSV∩^Sb||SSV|. Actively selected sets share large overlap with supports of an SVM (critical for classification).
['Dsets', 'Common%', 'Chance%', '# [ITALIC] SV']
Indeed, recent work suggests that the learning in deep classification networks may focus on small part of the data closer to class boundaries, thus resembling support vectors Xu et al. To investigate whether uncertainty sampling also exhibits this behavior, we perform below a direct comparison with support vectors from a SVM. For this, we train a FTZ model on the full training data and train a SVM on the resulting features (sentence embeddings) to obtain the support vectors and compute the intersection of support vectors with each selected set. The high percentage overlap is a surprising result which shows that the sampling is indeed biased but in a desirable way. Since the support vectors represent the class boundaries, a large percentage of selected data consists of samples around the class boundaries. This overlap indicates that the actively acquired training sample covers the support vectors well which are important for good classification performance. The overlap with the support vectors of an SVM (a fixed algorithm) also suggests that uncertainty sampling using deep models might generalize beyond FastText, to other learning algorithms.