Towards Automated Sexual Violence Report Tracking
Table 5: Ranking evaluation of sexual violence reports
['k', '25', '50', '100', '300', '500', '1000', '2500']
Recall that the dataset has about 11K sexual violence reports and 7K other tweets. MVC classifier achieves the best performance in detecting sexual violence reports in tweets. It can identify a report with a precision, recall, and F1-measure of 0.804, 0.834, and 0.808, respectively. To further understand the effectiveness of MVC, we used its SVR class probabilities to conduct ranking evaluation. We measured the accuracy of the top-k sentences by two commonly used measures- Precision-at-k (P@k), and Average Precision-at-k (AvgP@k). In general, MVC achieves excellent performance in ranking. For top 1000 sentences, its precision is 0.951. This indicates that MVC has a strong agreement with human annotators on detecting sexual violence reports.