Towards Real-Time Summarization of Scheduled Events from Twitter Streams \titlenoteThe present paper gives more technical and experimental details about the work published as a poster at HT’2012 [8].
Table 1: Evaluation of sub-event detection approaches.
['[EMPTY]', '[BOLD] P', '[BOLD] R', '[BOLD] F1', '[BOLD] #']
This evaluation method enables us to compare the two systems to infer which of them performs best. Our outliers approach clearly outperforms the baseline, improving both precision (75.8% improvement) and recall (3.7%) for an overall 40% gain in F1. At the same time, the compression rate for the outliers approach almost doubles that of the baseline (56.4%). From the average of 32k tweets sent per game, the summarization to 25.6 tweets represents a drastic reduction to only 0.079% of the total. Keeping the number of sub-events small while effectiveness improves is important for a summarization system in order to provide a concise and accurate summary. The outperformance of the outlier-based approach shows the importance of taking into account the audience of a specific game, as well as the helpfulness of learning from previous activity throughout a game.