Very Deep Convolutional Networks for Text Classification
Table 6: Test error on the Yelp Full data set for all depths, with or without residual connections.
['depth', 'without shortcut', 'with shortcut']
We evaluate the impact of shortcut connections by increasing the number of convolutions to 49 layers. When looking at the column “without shortcut”, we observe the same degradation problem as in the original ResNet article: when going from 29 to 49 layers, the test error rate increases from 35.28 to 37.41 (while the training error goes up from 29.57 to 35.54). When using shortcut connections, we observe improved results when the network has 49 layers: both the training and test errors go down and the network is less prone to underfitting than it was without shortcut connections.