Can Neural Image Captioning be Controlled via Forced Attention?
Table 2: The degree of controllability as the percentage of box captions containing their category in relation to all 117,798 box captions. The degree for the distinct share is based on 87,033 (k@1) or 58,407 (k@5) box captions with before unmentioned categories.
['[EMPTY]', 'Controllable k@1', 'Controllable k@5', 'and distinct k@1', 'and distinct k@5']
As the set of COCO categories is rather restrictive, and e.g. only states “person” where a caption might say “woman” or “man” , we also check for the five nearest neighbors in cosine distance of the model’s learned word embedding space (k@5). Still, the limited-6 configuration results in the highest score with 55.58%, in which, for example, the box caption contains the category name “person” or one of its neighbors (“man”, “woman”, “guy” or “girl”). The unlimited configuration has also the highest degree with 21.39%, when we also allow the five nearest neighbors to be included.