Durkheim Project Data Analysis Report
Table 13: Choosing Significant Words
['Word Source', 'Mean Accuracy', 'Std. Dev.']
Creating the set of corpus-pairs requires having previously computed a list of ’significant words’. Creating that list is time-consuming, since it requires training an ensemble, extracting the words, and then training again, with pairs. Thus, it is natural to ask if there are simpler ways of obtaining a list of ’significant words’ that are just as good. To make the two methods comparable, a simple mutual-information-maximizing feature selection step was performed to select 371 words, the same number of words as obtained from the ensemble. Feature selection runs in seconds, whereas training an ensemble of 40 models can take hours. Perhaps it is a mistake, then to cut rare words and rare word-pairs, when using a bag-of-pairs model? In this experiment, no rare words or pairs were cut; only the cut for the MI was altered. Thus, somehow, when word-pairs come into play, failing to cut rare words and phrases makes things worse!