Learning Compressed Sentence Representationsfor On-Device Text Processing
Table 3: Ablation study for the AE-binary-SP model with different choices of λsp (evaluated with test accuracy on the MR dataset).
['[ITALIC] λsp', '0.0', '0.2', '0.5', '0.8', '1.0']
5.3.1 The effect of semantic-preserving loss To investigate the importance of incorporating the locality-sensitive regularizer, we select different values of λsp (ranging from 0.0 to 1.0) and explore how the transfer results would change accordingly. The λsp controls the relative weight of the semantic-preserving loss term.