Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?
Table 4: Membership inference accuracy per subcorpus. Right 4 columns are results for out-of-domain subcorpora. Note that ParaCrawl is out-of-domain for Bob and his classifier, whereas in-domain for Alice and her MT model.
['[EMPTY]', 'ParaCrawl', 'CommonCrawl', 'Europarl', 'News', 'Rapid', 'EMEA', 'Koran', 'Subtitles', 'TED']
Carol prepared out-of-domain (OOD) subcorpora, Aood, that are separate from Atrain and Ball. The accuracy for OOD subcorpora are much higher than that of original in-domain subcorpora. For example, the accuracy with Decision Tree was 50.3% and 51.1% for ParaCrawl and CommonCrawl (in-domain), whereas 67.2% and 94.1% for EMEA and Koran (out-of-domain). This suggests that for OOD data Bob has a better chance to infer the membership. Note that the OOD subcorpora only have out-probes; By definition none of the samples from OOD subcorpora are in the training data. We believe this behavior is caused by applying Perceptron to inseparable data, and this particular model happened to be trained to act this way. To confirm this we have trained variations of Perceptrons by shuffling the training data, and observed that the resulting models had different output ratio of in and out, and in some cases always predicting in for both in and OOD subcorpora.