Hedge detection as a lens on framing in the GMO debates: A position paper
Table 6: Best performance after parameter tuning based on the 153 labeled WOS+LEXIS sentences; this gives some idea of the upper-bound potential of our Georgescul-based method. The training set is Bio, which gave the best performance in Table 5.
['Evaluation set', '( [ITALIC] C, [ITALIC] γ)', 'P', 'R', 'F']
It is clear that there exist domain differences between the CoNLL dataset and WOS+LEXIS. To better understand the poor cross-domain performance of the classifier, we tuned another model based on the performance on the 153 labeled sentences using Bio as training data. Also, LEXIS articles are hard to classify even with the tuned model, which challenges the effectiveness of a cue-words frequency approach beyond professional scientific texts. Indeed, the simplicity of our reimplementation of Georgescul ’s algorithm seems to cause longer sentences to be classified as uncertain, because cue phrases (or n-grams extracted from cue phrases) are more likely to appear in lengthier sentences. Analysis of the best performing model shows that the false-positive sentences are significantly longer than the false-negative ones.