Sherlock: Scalable Fact Learning in Images
Table 4: Large Scale Experiment
['[EMPTY]', '[BOLD] Language View retrieval %  [BOLD] Top1', '[BOLD] Language View retrieval %  [BOLD] Top 5', '[BOLD] Language View retrieval %  [BOLD] Top 10', '[BOLD] Visual view Retrieval %  [BOLD] mAP100', '[BOLD] Visual view Retrieval %  [BOLD] mAP10']
Details about the nearest-neighbor database creation and the large scale evaluation could be found in the supplementary. Model 2 is also multiple orders of magnitude better than chance and is also significantly better than the competing methods. To test the value of structure, we ran an experiment where we averaged the S, P, and O parts of the visual and language embedding vectors instead of keeping the structure. Previous smaller scale experiments are orders of magnitudes smaller and also less challenging since all facts were seen during training. There is an observable increase on the improvement of Model 2 compared to the baselines in the large scale setting. Additionally, the performance of the image-caption similarity methods degrade substantially. We think this is due to both the large scale of the facts and that the majority of the facts have zero or very few training examples. Interestingly, MV CCA is among the best performing methods in the large scale setting. On the language view retrieval, we have very competitive results to MV CCA but as we have notices several good visual retrieval results for which the metric gives zero-credit. These results show that Model 2 generally behaves better with compared other models with the increase of facts. We noticed a slight increase for Model 1 over Model 2.