Iterative Data Programming for Expanding Text Classification Corpora
Table 1: Accuracy results for Random, I-MLT, and I-DP on Reddit, English and Japanese Customer Service test sets. We compare the initial and optimal train sets to the terminal train sets, using a downstream classifier for evaluation.
['Method', 'Reddit', 'CS-En', 'CS-Ja']
The accuracy results there reflect the quality of a classifier trained on labeled examples obtained by each method, and are evaluated on the held out examples. Another assessment of the quality of the obtained training set is given by the estimated KL divergence of the word distributions in the training set and the word distribution in the optimal set. It is clear in all cases that the I-DP approach is able to construct a training set that performs best on the held-out test sets.