Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?
Table 6: Attack accuracy on probe groups. In addition to the original Alice set, we have adjusted set where the feature values are adjusted by subtracting the mean BLEU difference between Alice and Bob models.
['[EMPTY]', 'Bob train', 'Bob valid', 'Bob test', 'Alice original', 'Alice adjusted']
We can see that the accuracy is much higher than 50%, not only for Bob’s training set but also for his validation and test sets. However, for Alice, we found that classifiers were almost always predicting in, resulting the accuracy to be around 50%. This is due to the fact that classifiers were trained on shadow models that have lower BLEU scores than Alice. This suggests that we need to incorporate the information about the Alice / Bob MT performance difference. One way to adjust the difference is to directly manipulate the input feature values. If the classifier took advantage of the absolute values in its decision, the adjustment may give improvements. If that is not the case, then improvements are less likely. Before the adjustment, all classifiers were predicting everything to be in for Alice probes. Classifiers like NB and MLP apparently did not change how often it predicts in even after the normalization, whereas classifiers like P and DT did. In a real scenario this BLEU difference can be reasonably estimated by Bob, since he can use Alice’s translation API to calculate BLEU score on a held-out set, and compare it with his shadow models.