Learning Autocomplete Systems as a Communication Game
Table 2: We measure robustness as the performance of trained decoders with respect to different encoders. Selected uniform and constrained models have similar retention rates. The diagonal numbers represent the accuracy of matched encoder-decoder pairs.
['Encoder', 'Kept (%)', 'Decoder (ACC@1 (%)) Unif(0.1)', 'Decoder (ACC@1 (%)) Unif(0.3)', 'Decoder (ACC@1 (%)) Unif(0.5)', 'Decoder (ACC@1 (%)) Constr(0.8)', 'Decoder (ACC@1 (%)) Constr(0.6)', 'Decoder (ACC@1 (%)) Constr(0.4)', 'Decoder (ACC@1 (%)) Constr(0.2)']
We quantify the robustness of communication schemes by measuring how well a decoder that was trained with respect to one encoder works with another encoder. For instance, using keywords from the trained encoder with a 55% retention rate ({{Constr}}(0.2)) improves the performance of the Unif decoders by 200% compared to the matched uniform random encoder ({{Unif}}(0.5)). Furthermore, trained decoders are robust to being given more tokens than they were trained on (e.g. going from {{Unif}}(0.1) to {{Unif}}(0.5) encoders and {{Constr}}(0.8) to {{Constr}}(0.2) encoders), which ensures that users can increase the accuracy by passing more tokens.