Normalized and Geometry-Aware Self-Attention Network for Image Captioning
Table 4: Comparison of various variants of GSA.
['Approach', '#params', 'B@4', 'M', 'R', 'C', 'S']
‘+absolute" denotes adding absolute geometry information of each individual object to their input representations at the bottoms of the encoder. We have the following findings. 1) Adding the absolute geometry information (‘‘absolute") is not beneficial to the performance. That is probably because it is too complex for SA to infer the 2D layout of objects from their absolute geometry information. 2) All the proposed variants of GSA can improve the performance of SAN, showing the advantages of using relative geometry information. 3) ‘‘query-dependent" brings the best performance and outperforms the content-independent variant, proving that incorporating the content information of the associated query can help infer a better geometric bias. 4) ‘‘key-dependent" is inferior to ‘‘query-dependent". That is because when using key-dependent geometric bias, the scores ϕ3ij= K′j⊤Gij condition on different keys K′j, thus the differences in Gij may be overwhelmed by the differences in K′j when performing softmax on the keys’ dimension. In comparision, when using query-dependent geometric bias, the effect of Gij could be highlighted since the scores condition on a common query Q′i when performing softmax. We did not observe further improvement when combing these variants into ϕ in Eq.