Time-aware Large Kernel Convolutions
Table 5: Test perplexity on WikiText-103. We used adaptive inputs similar to DBLP:journals/corr/abs-1809-10853 and show that our method yields better perplexity than dynamic convolutions and comparative performance with self-attention.
['[EMPTY]', 'Param', 'Test']
We evaluated our method on the task of language modeling. We considered the WikiText-103 benchmark dataset and we compared against recent methods in the literature. Particularly, we followed the setup that was implemented in the adaptive inputs baseline (DBLP:journals/corr/abs-1809-10853). This work suggest the use of self-attention with adaptive input representations. We substituted the self-attention module with our method. In order to assimilate the number of parameters used in their experiments, we increased the number of layers by one. In addition, we get comparative performance with self-attention. Moreover, we use less number of parameters than the best comparison methods.