HappyDB: A Corpus of 100,000 Crowdsourced Happy Moments Part of the work was done while Asai, Evensen, and Xu were at Recruit Institute of Technology.
Table 16: The top shows the most used words in happy moments for instruction sets I1 and I2. The bottom shows the frequencies in I1 and I2 w.r.t. V, the vocabulary in our positive examples.
['[ITALIC] I1 got', '[BOLD] Count 37', '[ITALIC] I2 made', '[BOLD] Count 42']
Upon collecting our first batch of 300 happy moments, we tabulated the top 20 most frequent words (nouns/verbs/adjectives) that occur in the happy moments. It was surprising to observe that the words used in the positive examples often appear in the top 20 most frequent words. For example, the bold words “morning”, “enjoy”, “woke”, “son”, and “great” ( excluding “went”), which are words we use in our positive examples, appear highly among the top 20 words. Consequently, we experimented with an instruction set without the positive examples (I2) and collected another 300 happy moments. When we compare the moments collected under I1 and I2, it is evident that the bold words no longer appear highly ranked. Here, V denotes the set of all words (no duplicates and no stopwords) obtained from our 5 positive examples in the I1 instruction set. We counted the number of times words in V occur in the happy moments obtained from I1 instruction and, respectively, I2 instruction. In addition, we also counted the number of times words outside V occur in the respective batches of happy moments. Specifically, our null hypothesis is that the word usage of the happy moments is independent of whether the instruction set contained positive examples or not. The χ2-test rejected the null hypothesis with p-value <0.001. Hence, we conclude that MTurk workers were influenced by the positive examples in our instructions when reporting their happy moments and decided against positive example sentences in the instructions for collecting happy moments.