Relationship-Embedded Representation Learning for Grounding Referring Expressions
TABLE IV: Comparison of different phrase designs on RefCOCO, RefCOCO+ and RefCOCOg.
['[EMPTY]', 'RefCOCO testA', 'RefCOCO testB', 'RefCOCO+ testA', 'RefCOCO+ testB', 'RefCOCOg test']
The performance of the variant using words to highlight the vertices of the spatial relation graph (row 1) is worse than that of the final version using both words and phrases (row 3), which demonstrates the effectiveness of phrases in improving the accuracy of vertex highlighting. It is worth noting that we implicitly considered the word context (phrase-level information) in our conference version (row 2) by using the contextual features of words to attend the vertices instead of using the word embeddings. However, the contextual features of words introduce the global noise of the expressions, which increases the difficulty of learning the correspondence between words and vertices. The performance of CMRIN with implicit phrases is worse than that of it with explicit phrases in the object category (i.e., test B), because the visual contents of object category is sensitive to contextual noise. In addition, explicit use of phrases can help align between the linguistic words and visual objects.