Squeezed Very Deep Convolutional Neural Networks for Text Classification *Authors contributed equally and are both first writers.
Table I: Number of convolutional layers for each different VDCNN depth architecture
['Depth', '9', '17', '29', '49']
The following layer (3, Temp Convolution, 64) applies 64 temporal convolutions of kernel size 3, so the output tensor has size 64∗s. Its primary function is to fit the lookup table output with the modular network segment input composed by convolutional blocks. Each aforenamed block is a sequence of two temporal convolutional layers, Besides, the different network depths are obtained varying the number of convolutional blocks. As a convention, the depth of a network is given as its total number of convolutions. Considering the first convolutional layer of the network, we obtain the depth 2∗(2+2+2+2)+1=17. The following rule is employed to minimize the network’s memory footprint: Before each convolutional block doubling the number of feature maps, a pooling layer halves the temporal dimension. This strategy is inspired by the VGG and ResNets philosophy and results in three levels of feature maps: 128, 256 and 512 (see Fig. Lastly, since each standard temporal convolution turns into two (Depthwise and Pointwise), the number of convolutions per convolutional block has doubled. Nevertheless, these two convolutions work as one because it is not possible to use them separately keeping the same propose. In this way, we count them as one layer in the network depth.