Video Highlight Prediction Using Audience Chat Reactions
Table 2: Ablation Study: Effects of various models. C:Chat, V:Video, UF: % of frames Used in highlight clips as positive training examples; P: Precision, R: Recall, F: F-score.
['Method', 'Data', 'UF', 'P', 'R', 'F']
To deal with the large number of frames total, we sample only 5k positive and 5k negative examples in each epoch. We use batch size of 32 and run 60 epochs in all experiments. Weight decay is 10−4 and learning rate is set as 10−2 in the first 20 epochs and 10−3 after that. Cross entropy loss is used. Highlights are generated by fans and consist of clips. We match each clip to when it happened in the full match and call this the “highlight clip” (non-overlapping). The action of interest (“kill”, “objective control”, etc.) often happens in the later part of a highlight clip, while the clip contains some additional context before that action that may help set the stage. During evaluation, we used all frames in the highlight clip. For the basic L-Char-LSTM and V-CNN models, using only the last 25% of frames in highlight clips in training works best. In order to evaluate the performance of L-Char-LSTM model, we also train a Word-LSTM model by tokenizing all the chats and only considering the words that appeared more than 10 times, which results in 10019 words. We use this vocabulary to encode the words to 1-hot vectors. The L-Char-LSTM outperforms L-Word-LSTM by 22.3%.