Data Cleaning for XML Electronic Dictionaries via Statistical Anomaly Detection
TABLE V: Language expert annotations on 175 anomalies proposed by the text length system and the uncommon character system for data in fields POS and MAIN.
['System', 'Field', 'Real Error', 'No Error', 'Total']
We conducted an additional evaluation of our systems using the annotations of a language expert in the process of correcting errors in a Tamasheq dictionary containing 5968 lexical entries. The language expert evaluated 175 anomalies proposed by the single-field text length system and the single-field uncommon character system for data in fields POS (part of speech) and MAIN (headword). The language expert marked each proposed anomaly as either a real error or not a real error. These results are further evidence that the error detection systems could substantially increase the efficiency with which domain experts can clean XML data.