Parallelizable Stack Long Short-Term Memory
Table 2: Dependency parsing result with various training batch size b and without composition function. The results marked with asterisks were reported in the Ballesteros et al. (2017).
['[ITALIC] b', '[BOLD] dev UAS', '[BOLD] dev LAS', '[BOLD] test UAS', '[BOLD] test LAS']
Our implementation is able to yield better test set performance than that reported in Ballesteros et al. for all batch size configurations except 256, where we observe a modest performance loss. Like Goyal et al. Keskar et al. et al. , we find warmup very helpful for stabilizing large-batch training. We did not run experiments with batch size below 8 as they are too slow due to Python’s inherent performance issue.