genCNN: A Convolutional Architecture for Word Sequence Prediction
Table 3: FBIS results. The upper block (row 1-6) compares genCNN and the competitor models, and the bottom block (row 7-9) compares different variants of genCNN.
['[BOLD] Model', '[BOLD] Perplexity']
Interestingly genCNN outperforms its variants also quite significantly (bottom block): 1) with only Time-Arrow (same number of feature-maps), the performance deteriorates considerably for losing the ability of capturing long range correlation reliably; 2) with only Time-Flow the performance gets even worse, for partially losing the sensitivity to the prediction task. It is quite remarkable that, although αCNN (with Lα=30) can achieve good results, the recursive structure in full genCNN can further decrease the perplexity by over 3 points, indicating that genCNN can benefit from modeling the dependency over range as long as 30 words.