Visual Commonsense Graphs: Reasoning about the Dynamic Context of a Still Image visualcomet.xyz
Table 7: Generated Inference Results. Human score for the generated inferences on the Test split. We select 200 random images and generate 5 sentences for each of the three inference type (3000 sentences total). Then, we assign three annotators to determine if each inference sentence is correct, and take the majority vote. Refer to Table 2 and Section 6.2 for model details. We see that the best model using generated event and place as input provides a worse performance than the best model without the text input.
['[BOLD] Modalities ', '[BOLD] Human  [BOLD] Before ', '[BOLD] Human   [BOLD] Intent ', '[BOLD] Human   [BOLD] After ', '[BOLD] Human   [BOLD] Avg ']
Inference using Generated Event Can the generated event be used as text input to generate the inferences? Under the section With Generated Text Input, we see that the Image + Text model performs better than Text only model, when generated event and place is given as input. However, the scores are lower than the best model without text input (36.0 vs 38.2). Note that this does not indicate that event and place information are not useful. As mentioned in the main paper, the model trained to generate event, place, and inference [Image + Event + Place + PG + EP Loss] performs the best when image is only given as input.