A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos
Table 4: Key clip selection results.
['[EMPTY]', 'Acc', 'P', 'R', 'F1']
We evaluate using the accuracy, precision, recall and F1 score for the binary classification problem of whether a given clip in the video is a key clip. This is despite the fact that the dataset used for training neural models has a different data distribution and domain from the test set. Among heuristic methods, pretrained SRL is better than Kiddon2015MiseEP even though the second is trained on transcript text from YouCook2 videos. One possible reason for this is that the unsupervised segmentation method was specially designed for recipe texts, which are mostly simple, concise and imperative sentences found in recipe books, while our transcript text is full of noise and tends to have longer, more complicated, and oral-style English. Post-processing significantly improves the SRL model, showing that filtering unrelated arguments and incorporating the cooking lexicon helps, especially with reducing false positives. Among neural method ablations, the model using only visual features performs worse than that using only text features. The best model for identifying key clips among proposed baselines uses both visual and text information in the neural model.