Language and Dialect Identification of Cuneiform Texts
Table 5: The macro F1-scores attained by the baseline methods with the CLI dataset.
['[BOLD] Method', '[ITALIC]  [BOLD] n [BOLD] -gram range', '[BOLD] F1 dev', '[BOLD] F1 test']
The parameters and the best possible language models are determined by training the identifier using the training set and evaluating its performance on the development set. Once the best parameters are decided, the texts in the development set can also be added to the training set for the final evaluation against the test set. We used the macro F1-score as the measure for language identification performance. For each of the methods, we evaluated all possible sign n-gram ranges from 1 to 15 using the development set. In the voting ensemble, we used the best parameters for the methods from the individual experiments, and in case of a tie, the result from the product of relative frequencies was used. The diagonal values represent correct identifications. Standard Babylonian and Neo-Babylonian were the most difficult varieties to distinguish, mostly being erroneously identified as each other. Late Babylonian was the easiest to identify, with a recall of over 96%.