Any-gram Kernels for Sentence Classification: A Sentiment Analysis Case Study
Table 4: Hyper-parameters of the deep neural net models tuned on development sets (Up to 3 hidden layers were tried. The activation function for all settings is tanh.)
['[EMPTY]', 'Laptop batch size', 'Laptop #hidden layer', 'Laptop hidden layer size', 'Laptop learning rate', 'Laptop dropout rate', 'Restaurant batch size', 'Restaurant #hidden layer', 'Restaurant hidden layer size', 'Restaurant learning rate', 'Restaurant dropout rate', 'SSTB batch size', 'SSTB #hidden layer', 'SSTB hidden layer size', 'SSTB learning rate', 'SSTB dropout rate']
One model is unidirectional, where information only flows forward in time (i.e., from the previous word to the next in a sentence) and the other is bidirectional, where it flows in both directions to model the dependence of previous words on the next words. The input to the models are the same pre-trained word embeddings used by the any-gram models. A dropout layer is added before the output layer for regularization.