Few-shot tweet detection in emerging disaster events
Table 1: Hashtags chosen for the CrisisLexT26 data set. (Two events did not contain frequent hashtags and were therefore excluded.)
['[BOLD] Event', '[BOLD] Hashtag', '[BOLD] Occurrences/1001']
In order to train and test our few-shot models, sub-sets of positive and negative supports for a class (examples and counterexamples) plus an either positive or negative query are necessary. Training steps performed on such sets are called “episodes”. For our purposes, tweets are considered to belong to the same class if they are associated with the same event. We perform experiments on two different constellations of data: Event-vs.-event training, and event-vs.-all training. Event-vs.-event In these experiments, we attempt to detect tweets related to a specific disaster event out of tweets related to other disaster events. CrisisLexT26 is used to generate the training episodes, and CrisisNLP is used for the validation episodes. To generate episode data packs, tweets for an event containing a specific hashtag are selected randomly to create the positive support set. Upper- and lower-case versions are equalized, and this hashtag is removed in a pre-processing step so as not to bias the classifier towards it. For positive queries, tweets without the hashtag, but coming from the same event are picked randomly. For negative supports and queries, random tweets from other events are used. Hashtags were chosen manually according to their prevalence in the event’s tweets. The practical reasoning for this is that during an ongoing event, a user could quickly search for example tweets by a certain hashtag, then use the found examples to detect more event-related tweets without this hashtag using the models proposed by us. (Of course, a user may just as well use any other method for selecting examples of the class they are interested in). Event-vs.-all For the second set of experiments, we train models for the more general task of detecting tweets pertaining to a specific event out of random other tweets. Episodes are generated similarly to event-vs.-event training, but selection of positive and negative examples is based on the event set these tweets come from rather than contained hashtags. Training episodes are generated from Events2012, and validation episodes from the combined CrisisLexT26 and Sentiment140 data sets. The Sentiment140 data is used only for generating negative support and query examples.