Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations
Table 3: The result of Thai sentence segmentation for each model. For the Orchid dataset, we report the average of each metric on 10-fold cross-validation. Meanwhile, average metrics from 5-fold cross-validation are shown for the UGWC dataset.
['Model', 'Orchid precision', 'Orchid recall', 'Orchid F1', 'UGWC precision', 'UGWC recall', 'UGWC F1']
Thai sentence segmentation to 88.7% (row (f)) on the UGWC dataset. These results occur because many word groups exist that can be used to signal the beginning and end of a sentence in Thai. Word groups always found near sentence boundaries can be categorized into 2 groups. The first group consists of final particles, e.g., “ นะ|คะ ” (na | kha), “ นะ|ครับ ” (na | khrạb), “ เลย|ครับ ” (ley | khrạb), “ แล้ว|ครับ ” (laêw | khrạb), and others. These word groups are usually used at the ends of sentences to indicate the formality level. For instance, the model with local representation can detect the sentence boundary at “ ครับ ” (khrạb) that is followed by “ แล้ว ” (laêw), as shown in Fig. The second group consists of conjunctions that are always used at the beginnings of sentences, e.g., “ จาก|นั้น (after that) ”, “ ไม่|งั้น (otherwise) ” and others. The model that uses n-gram to capture word group information is better able to detect word groups near sentence boundaries. Thus, this model can identify these sentence boundaries easily in the Thai language.