Leveraging Newswire Treebanks for Parsing Conversational Data with Argument Scrambling
Table 1: The table shows theoretically possible orders of Subject (S), Object (O) and Verb (V) in transitive sentences in the HTB training data with their percentages of occurrence.
['S.No.', 'Order', 'Percentage']
\sidecaptionvpos figuret Despite word order freedom, most of the morphologically-rich languages exhibit a preferred word order in formal registers such as newswire. Word order alternations are more commonplace in informal language use such as day-to-day conversations and social media content. For statistical parsing, word order alternations (argument scrambling) are a major bottleneck. Given appropriate pragmatic conditions a transitive sentence in a morphologically-rich language allows n factorial (n!) permutations, where n is the number of verb arguments and/or adjuncts. Argument scrambling often leads to structural discontinuities. Moreover, these scramblings worsen the data sparsity problem since data-driven parsers are usually trained on a limited size treebank where most of the valid structures may never show up. The chances of any deviation from the canonical word order are smaller, thereby creating sampling bias. Newswire data usually represent only a sample of possible syntactic structures and, therefore, suffer from non-representation of certain classes that encode valid arc directionalities. In the Hindi dependency treebank (HTB) Bhat et al. Therefore, we instead generate training examples for varied arc directionalities by transforming the gold syntactic trees in the training data. We experiment with the Hindi dependency treebank and show that such transformations are indeed helpful when we deal with data with diverse word-orders such as movie dialogues. Our work is in conformity with earlier attempts where modifying source syntactic trees to match target distributions benefited parsing of noisy, conversational data Van der Plas et al.