Universal Word Segmentation: Implementation and Interpretation
Table 4: Hyper-parameters for segmentation.
['Character embedding size', '50']
We use one set of parameters for all the experiments as we aim for a simple universal model, although fine-tuning the hyper-parameters on individual languages might result in additional improvements. The encoder-decoder is trained prior to the main network. The weights of the neural networks, including the embeddings, are initialised using the scheme introduced in \newciteglorot2010understanding. The network is trained using back-propagation. All the random embeddings are fine-tuned during training by back-propagating gradients. The initial learning rate η0 is updated with a decay rate ρ.