ParaBank: Monolingual Bitext Generation and Sentential Paraphrasingvia Lexically-constrained Neural Machine Translation
Table 2: Tokens assessed, along with their IDFs, of the sentence “I told her I was proud to work for them.”
['[BOLD] Token proud', '[BOLD] IDF 11.1', '[BOLD] Token told', '[BOLD] IDF 7.9']
First, tokens with only lower-cased alphabetical letters are assessed; they are listed in Tab. After applying the IDF thresholds and exception for prepositions, the following tokens are in the candidate pool, from which we choose tokens to constrain on (listed in descending order of IDF values): proud, told, work, for, and to.