Semantic Product Search
Table 5. Comparison with Baselines
['Model', 'Loss', 'Tokenization', 'Recall', 'MAP', 'Matching NDCG', 'Matching MRR', 'Ranking NDCG', 'Ranking MRR']
Deerwester et al. In 2013, Huang et al. Fully connected units in the DSSM architecture were subsequently replaced with Convolutional Neural Networks (CNNs) In an alternate approach, which articulated the idea of interaction models, Guo et al. Hui et al., Nevertheless, these interaction methods require memory and computation proportional to the number of words in the document and hence are prohibitively expensive for online inference. We only use word unigrams or character trigrams in our model, as it is not immediately clear how to extend the bag-of-tokens approach to methods that incorporate ordering. We compare the performance of using the 3-part L2 hinge loss to the original loss presented for each model. Across all baselines, matching performance of the model improves using the 3-part L2 hinge loss. ARC-II and Match Pyramid ranking performance is similar or lower when using the 3-part loss. Ranking performance improves for DSSM, possibly because the original approach uses only random negatives to approximate the softmax normalization. More complex models, like Match Pyramid and ARC-II, had significantly lower matching and ranking performance while taking significantly longer to train and evaluate. These models are also much harder to tune and tend to overfit.