Cross-lingual and cross-domain discourse segmentation of entire documents
Table 1: Number of documents, EDUs, sentences and words (according to UDPipe, see Section 7).
['Corpus', '#Doc', '#EDU', '#Sent', '#Words']
For English, we use four corpora, allowing us to evaluate cross-domain performance: the RST-DT (En-DT) composed of Wall Street Journal articles; the SFU review (En-SFU-DT) containing product reviews; the instructional corpus (En-Instr-DT) containing interviews, news, travel guides and how-tos. The “cross” models are trained using multi-task learning. Note that the sentence-based baseline has a high precision (e.g. 96.6 on Es-DT against 59.8 for the cross-lingual system), but a much lower recall, since it mainly predicts the sentence boundaries. Using the punctuation (UDP-P) could be a better approximation for corpora with more varied EDUs, see the large gain for the Pt-DT and the En-Instr-DT.