Procedural Reasoning Networks for Understanding Multimodal Procedures
Table 1: Quantitative comparison of the proposed PRN model against the baselines.
['Model', 'Single-task Training Cloze', 'Single-task Training Coherence', 'Single-task Training Ordering', 'Single-task Training Average', 'Multi-task Training Cloze', 'Multi-task Training Coherence', 'Multi-task Training Ordering', 'Multi-task Training All']
In single-task training setting, PRN gives state-of-the-art results compared to other neural models. Moreover, it achieves the best performance on average. These results demonstrate the importance of having a dynamic memory and keeping track of entities extracted from the recipe. In multi-task training setting where a single model is trained to solve all the tasks at once, PRN and BIDAF w/ static memory perform comparably and give much better results than BIDAF. Note that the model performances in the multi-task training setting are worse than single-task performances. We believe that this is due to the nature of the tasks that some are more difficult than the others. We think that the performance could be improved by employing a carefully selected curriculum strategy McCann et al.