Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO
Table 3: Number of annotations per task (STS, SIS, SITS) and per split (val, test).
['SplitTask', 'STS', 'SIS', 'SITS', 'Total (per split)']
Based on the pair selection and annotation methodology given in the previous section, we obtained similarity ratings for 247,314 text-text, image-image, and image-text pairs (corresponding to 1,216,570 total individual judgments). For STS and SITS, it also shows the rating distribution for pairs included from other-modality selection and from existing MS-COCO links. The test set distributions are similar. We also count the number of positive examples for each example in all three type of tasks, where a score ≥3 is considered positive (and roughly equivalent in all task instructions). These positive examples can improve retrieval evaluation. We discuss each of the STS, SIS and SITS ratings below.