Neural Net Models of Open-domain Discourse Coherence
Table 2: Performance on the open-domain binary classification dataset of 984 Wikipedia paragraphs.
['Model', 'Accuracy']
Contrary to the findings on the domain specific dataset in the previous subsection, the discriminative model does not yield compelling results, performing only slightly better than the entity grid model. We believe the poor performance is due to the sentence-level negative sampling used by the discriminative model. Due to the huge semantic space in the open-domain setting, the sampled instances can only cover a tiny proportion of the possible negative candidates, and therefore don’t cover the space of possible meanings. By contrast the dataset in \newcitebarzilay2008modeling is very domain-specific, and the semantic space is thus relatively small. By treating all other sentences in the document as negative, the discriminative strategy ’s negative samples form a much larger proportion of the semantic space, leading to good performance.