Improving the Performance of Online Neural Transducer Models
Table 1: WER for NT, Varying Chunks Looked Over
['System', 'Chunk Size', 'WER']
Attention Window Our first set of experiments analyzes the behavior of NT as we vary the window used to compute attention. For these experiments, we use an encoder which consists of five layers of 768 uni-directional LSTM cells and a decoder with two layers of 768 LSTM cells. Allowing the model to compute attention over the last 20 chunks in addition to the current chunk, however, slightly improves performance of the NT system.