Language Identification in Code-Mixed Data using Multichannel Neural Networks and Context Capture
Table 1: Data distribution.
['[EMPTY]', '[BOLD] Train', '[BOLD] Dev', '[BOLD] Test']
We wanted to test our approach on two different language pairs, which were Bengali-English (Bn-En) and Hindi-English (Hi-En). For Bn-En, we used the data prepared in Mandal et al. The number of instances of each type we selected for our experiments was 6000. All indic tagged tokens were used instead of just unique ones of respective languages. The whole model was compiled using loss as binary cross-entropy, optimization function used was adam Kingma and Ba The batch size was set to 64, and number of epochs was set to 30. Other parameters were kept at default. The training accuracy and loss graphs for both Bn and Hi are shown below. As the MNN model produces a sigmoid output, to convert the model into a classifier, we decided to use a threshold based technique identical to the one used in Mandal et al. For this the development data was used, threshold for Bn was calculated to be θ ≤ 0.95, while threshold for Hi was calculated to be θ ≤ 0.89. Using these, the accuracies on the development data was 93.6% and 92.87% for Bn and Hi respectively. The targets here were the actual language labels (0 for the Bn/Hi and 1 for En). The hyper-parameters which we set mostly follow Yang et al. Both the models (Bn-En & Hi-En) had identical parameters. L2 regularization λ was set at 1e-8. Learning rate η was set to 0.015. Batch size was kept at 16 and number of epochs was set to 210. Mini-batch stochastic gradient descent (SGD) with decaying learning rate (0.05) was used to update the parameters. All the other parameters were kept at default values. This setting was finalized post multiple experiments on the development data. Final accuracy scores on the development data was 93.91% and 93.11% for Bn and Hi respectively. For comparison purposes, we decided to use the character encoding architecture described in Mandal et al. The thresholds for the baseline models calculated on the development data was found to be θ ≤ 0.91 and θ ≤ 0.90 for Bn and Hi respectively. The results (in %) for each of the language pairs are shown below.