1 Introduction
Table 6: Test Accuracy for various model architectures.
['[EMPTY]', '[BOLD] Baseline', '[BOLD] LSTM', '[BOLD] LASER  [BOLD] FC', '[BOLD] CNN', '[BOLD] LASER  [BOLD] CNN', '[BOLD] DeepSubQE']
We found that the existing convolution networks Liu et al. that try to classify bilingual dataset are not able to learn the linguistic nuances of text and fail in many cases. However, recurrent networks that are good with parsing variable length inputs with temporal dependencies complement them. For LSTM network, we concatenated the output of both BiLSTMs and fed to a fully connected layer. For CNN model, we had three convolution modules followed by a fully connected layer. We use the 1024-dimension language agnostic sentence embeddings from LASER. An only-LSTM network works just about as well as the baseline but only-CNN network brings significant gains. The convolutional layer is possibly evaluating semantic retention better Kim et al. LASER FC is a classifier trained with LASER embeddings fed into a fully connected layer that performs worse than baseline. LASER CNN is a CNN on top of LASER embeddings and performs about as well as only-CNN. Proposed hybrid model, however, outperformed all other networks including the CNN by 3%. The LSTM when used in combination with the CNN is consistently improving prediction accuracies across languages. One notable observation from models trained using LASER embeddings was that LASER models took about 10 epochs on average to meet our stopping criterion while DeepSubQE model took around 34 epochs on average.