Automatic Mapping of French Discourse Connectives to PDTB Discourse Relations
Table 4: Error analysis of the potential false positive entries. ✓indicates newly discoursed mappings which are not included in LEXCONN.
['[BOLD] FR-DC  [ITALIC] /', '[BOLD] Relation', '[BOLD] Jdg', '[BOLD] FR-DC  [ITALIC] /', '[BOLD] Relation', '[BOLD] Jdg [head to column names]manual-judge2.csv']
In addition to the quantitative evaluation, we also performed a manual analysis of the false-positive errors to see if they really constituted errors. To do so, we looked at the tuples with a probability higher than 0.01 but which did not appear in LEXCONN. To evaluate if these 14 cases were true mistakes, we randomly selected five English-French parallel sentences from Europarl that contained the French DC and one of its English DC translations signalling the discourse relation. Then, we showed the French DCs within their sentence to two native French speakers and asked them to confirm if the discourse relation identified was indeed signaled by the French DCs or not. The Kappa agreement between the two annotators was 0.72. For 9 French connectives, both annotators agreed that indicated that in at least one of the five sentences, the discourse relation was signalled by the connective. This indicates that 64% (9/14) are in fact true-positives, i.e. correct mappings that are not listed in LEXCONN. FR-DC/English translation, Discourse relation> used in the manual evaluation and indicates the newly discovered mappings by ✓.