CLEVRER: Collision Events forVideo Representation and Reasoning
Table 3: Quantitative results of NS-DR on CLEVRER. We evaluate our model on all four question types. We also study a variation of our model NS-DR (NE) with “no events” but only motion traces from the dynamics predictor. Our question parser is trained with 1000 programs.
['Methods', 'Descriptive', 'Explanatory per opt.', 'Explanatory per ques.', 'Predictive per opt.', 'Predictive per ques.', 'Counterfactual per opt.', 'Counterfactual per ques.']
On descriptive questions, our model achieves an 88.1% accuracy when the question parser is trained under 1,000 programs, outperforming other baseline methods. On explanatory, predictive, and counterfactual questions, our model achieves a more significant gain. We also study a variation of our model, NS-DR (No Events/NE), which uses the dynamics predictor to generate only the predictive and counterfactual motion traces, and collision events are identified by the velocity change of the colliding objects. NS-DR (NE) performs comparably to the full model, showing that our disentangled system is adaptable to alternative methods for dynamics modeling. We also conduct ablation study on the number of programs for training the question parser.