Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA
Table 4: IOFSM and BBCE help improve the model’s performance by changing in and out-frame scores.
['[EMPTY]', 'Loss', 'Val Score (%)', 'IFS avg', 'IFS std', 'OFS avg', 'OFS std']
This can mean two things: (1) IOFSM helps increase the scores of in-frames, and (2) increased in-frame scores help improve the model’s performance. On the other hand, when we compare row 1 and 2, the average in-frame score of IOFSM is higher than BCE’s. But, the average out-frame score of IOFSM is also much higher than BCE’s. This can mean that out-frame scores have a large impact on the performance as well as in-frame scores. This is intuitively reasonable. Because information from out-frames also flows to the next layer (i.e., classifier) after being multiplied by the frame scores, the score for the ‘negative’ label also has a direct impact on the performance. So, making the scores as small as possible is also important. Also, when we compare the row 2 and others (2 vs. 1 and 3), the gap between in-frame scores is much larger than the gap between out-frame scores. But, considering the scores are average values, and the number of out-frames is usually much larger than in-frames, the difference between out-frame scores would affect more than the gap itself.