\text{A}^{4}\text{NT}: Author Attribute Anonymityby Adversarial Training of Neural Machine Translation
Table II: F1-scores of the attribute classifiers. All of them do well and better than the document-level random chance (0.62 for speech), (0.53 for age), and (0.50 for gender).
['[BOLD] Setting', 'Training Set Sentence', 'Training Set Document', 'Validation Set Sentence', 'Validation Set Document']
Before analyzing the performance of our A4NT network, we evaluate the attribute classifiers on the three settings we use. Document-level scores are obtained from accumulating the class log-probability scores on each sentence in a document before picking the maximum scoring class as the output label. We also tried hard voting to accumulate sentence level decisions, and observed that the hard voting results follow the same trend across datasets and splits. On the smaller political speech dataset, the attribute classifier is able to easily discriminate between the two authors, Barack Obama and Donald Trump, achieving perfect F1-score of 1.0 on both the training and the validation splits. The model also performs well on the age-group classification, achieving F1-score of 0.88 on the validation set at the document-level. Gender classification turns out to be the hardest to generalize, with a significant drop in F1-score on the validation set compared to the training set (down to 0.75 from 0.93). In all three tasks, the performance on sentence-level is worse than on document-level classification. Henceforth, we will use document-level F1-score as our primary metric when evaluating the effectiveness of A4NT networks.