Simple Recurrent Units for Highly Parallelizable Recurrence
Table 1: Test accuracies on classification benchmarks (Section 4.1). The first block presents best reported results of various methods. The second block compares SRU and other baselines given the same setup. For the SST dataset, we report average results of 5 runs. For other datasets, we perform 3 independent trials of 10-fold cross validation (3×10 runs). The last column compares the wall clock time (in seconds) to finish 100 epochs on the SST dataset.
['Model', 'Size', 'CR', 'SUBJ', 'MR', 'TREC', 'MPQA', 'SST', 'Time']
We select the best number reported in previous methods when multiple model variants were explored in their experiments. Despite our simple setup, SRU outperforms most previous methods and achieves comparable results compared to the state-of-the-art but more sophisticated model of Zhao et al. LSTM and the CNN model. On the movie review (MR) dataset for instance, SRU completes 100 training epochs within 40 seconds, while LSTM takes over 320 seconds.