Expanding the Text Classification Toolbox with Cross-Lingual Embeddings
Table 5: HyperParameters for different text classification architectures and sentence alignment
['[BOLD] Param', '[BOLD] Val']
For FT-MLP, we use a first dense layer with 512 units and rectified linear unit activation prior to the second dense layer that directly precedes softmax activation, a dropout layer of 0.7, an Adam optimizer with learning rate 10-2. For MF-CNN, we use 3 types of filters with kernel sizes 3, 4 and 5 consisting of 200 filters each, a dropout of 0.3 and Adam optimizer with learning rate 10-3. bi-GRU-Att uses 150 GRU units with tanh as an activation function, dropout layer of 0.3 and Adam optimizer 10-3. For multi-tasking experiments, we design hierarchical attention network use bidirectional GRUs consisting of 50 units, and tanh activation function. We use dropout layer of rate 0.5 and alternate training of the two tasks with two different optimization learning rates in order to make them synchronized to each other.