Learning Multi-faceted Representations of Individuals from Heterogeneous Evidence using Neural Networks
Table 5: Performance on the manually-labeled devset at different iterations of data harvesting.
['iteration 1', 'tweet-model 1', 'Pre 0.86', 'Rec 0.40', 'F1 0.55']
Stopping Condition: To decide the optimum number of steps for the algorithm to stop, we manually labeled a dataset which contains 200 positive tweets (100 like and 100 dislike) with entities. selected from the original raw tweet dataset rather than the automatically harvested data. The dataset contains 800 negative tweets . For each iteration of data harvesting, we evaluate the performance of the classification models and labeling model on this human-labeled dataset, which can be viewed as a development set for parameter tuning. As can be seen, the precision score decreases as the algorithm iterates, but the recall rises. The best F1 score is obtained at the end of the third round of iteration.