Imitation Attacks and Defenses for Black-box Machine Translation Systems
Table 4: Results for targeted flips and malicious nonsense. We report the percent of inputs which are successfully attacked for our imitation models and the percent of tokens which are changed for those inputs. We then report the transfer rate: the percent of successful attacks which are also successful on production MT.
['[ITALIC] Targeted Flips  [BOLD] Model', '[ITALIC] Targeted Flips  [BOLD] % Inputs (\\uparrow)', '[ITALIC] Targeted Flips  [BOLD] % Tokens (\\downarrow)', '[BOLD] Transfer % (\\uparrow)']
The attacks break our imitation models and successfully transfer to production systems. For our imitation models, we are able to successfully perturb the input in the majority (>3/4) of cases. For the targeted flips attack, few perturbations are required (usually near 10% of the tokens). Both attacks transfer at a reasonable rate, e.g., the targeted flips attack transfers 23% of the time for Systran.