A Continuously Growing Dataset of Sentential Paraphrases
Table 6: Impact of filtering of manual retweets.
['[EMPTY]', '[BOLD] avg #tweets (STD)', '[BOLD] avg PINC', '[BOLD] avg Jaccard']
Retweeting is an important feature in Twitter. There are two types: automatic and manual retweets. An automatic retweet is done by clicking the retweet button on Twitter and is easy to remove using the Twitter API. A manual retweet occurs when the user creates a new tweet by copying and pasting the original tweet and possibly adding some extras, such as hashtags, usernames or comments. We then filtered out manual retweets using a set of rules, checking if one tweet was a sub- string of the other, or if it only differed in punctuation, or the contents of the “twitter:title” or “twitter:description” tag in the linked HTML file of the news article. After filtering, the dataset contains tweets with more significant rephrasing as indicated by higher PINC and lower Jaccard scores.