MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Table 4: The comparison between the inference speed of MUSE and Transformer.
['Model', 'Inference Speed (tokens/s)']
Under the settings, where the numbers of parameters are similar for MUSE and Transformer, about 31% increase in inference speed can be obtained. The experiments use MUSE with 6 MUSE-simple modules and Transformer with 6 base blocks. The hidden size is set to 512.