Towards Automated Sexual Violence Report Tracking
Table 6: Performance of perpetrator labeling
['Class', 'Precision', 'Recall', 'F1-measure', 'Support']
We used Conditional Random Field (CRF) We used the BIE (begin-inside-end) notation to tag the perpetrator word sequences. Words those were not part of the perpetrator sequence, were assigned the O tag. For instance, the tweet “in the subway an old man grabbed me” would be tagged as- {in-O, the-O, subway-O, an-B, old-I, man-E, grabbed-O, me-O}. We used the words, their prefix and suffix, and their parts-of-speech tags as features while training the sequence labeling model. In total, 8,614 sexual violence reporting tweets were labeled with perpetrator words. We did a 5-fold cross-validation. The model achieves a weighted-average F1-measure of 0.943 (0.70 if class O is not considered). We observe that the performance for class I is low. As most of the highlighted perpetrator text (6,653 out of 8,614) was less than or equal two words, we didn’t have enough sample for class I. For the B and E classes, the F1-measures were above 0.76.