Modality-Balanced Models for Visual Dialogue
Table 3: Intersection and Union of the answers from image-only model and joint model which contribute to scoring for R@1 and NDCG metrics.
['[EMPTY]', 'Img-Only Model', 'Joint Model', 'Intersection', 'Union']
If the image-only model is good at NDCG, can we exploit its ability by combining it with the joint model? To figure out this possibility, we compare each answer from the image-only model and the joint model. To be specific, for R@1, we list up the correct answers from each model and count answers which are in both sets, i.e., the intersection. From the intersection, we obtain the union of the two sets. For NDCG, there is not one single correct answer. So we roughly calculate the intersection by taking minimum values between the two models’ scores and averaging them. This could mean image-only and joint models have room to be improved by combining them together.