Semantic Product Search
Table 8. Impact of Out-of-Vocabulary Bin Size
['Tokenization', 'Recall', 'MAP', 'Matching NDCG', 'Matching MRR', 'Ranking NDCG', 'Ranking MRR']
We see that matching performance improves as the bin size increases, although ranking performance peaks at lower bin sizes. These results confirm the intuition that adding OOV hashing leads to generalization to unseen tokens. This generalization improves matching performance as there are fewer spurious matches resulting from OOV tokens mapping to the same bucket and/or simply excluding OOV tokens.