Sampling Bias in Deep Active Classification: An Empirical Study
Table 5: Intersection of samples obtained with different values of b. We see the intersection of samples selected with different number of intersections comparable to highest possible (different seeds) in FastText, far higher compared to chance intersection. This indicates similar samples are selected regardless of sample size. NaiveBayes does not show clear trends but occasionally the queried percentage drops significantly when increasing iterations, occasionally it remains unaffected.
['Dsets', 'Chance', 'FTZ 9∩19∩39', 'FTZ 39∩39∩39', 'MNB 9∩19∩39', 'MNB 39∩39∩39']
Hoi et al. We experiment with various query sizes - (0.25%, 0.5%, 1%) for DBP, SGN, YRP and AMZP and (0.5%, 1%, 2%) for the rest corresponding to 9, 19 and 39 iterations. We observe a high overlap of the acquired samples across different query sizes indicating that the performance is independent of the query size (compare column 3 to column 4 where the size is held constant) while MNB results in lower overlap with more erratic behavior due to change in the query size (compare column 5 compared to column 6).