Self-Attention and Ingredient-Attention Based Model for Recipe Retrieval from Image Queries
Table 1. Comparison between our method, our Joint Neural Embedding (JNE)(Marín et al., 2018) and AdaMine (Carvalho et al., 2018) re-implementation. For all models we were using selected matching pairs generated by reducing noisy instruction sentences as described above. Recall rates are averaged over the evaluation batches.
['Image to Recipe', 'Image to Recipe', 'Image to Recipe MedR', 'Image to Recipe R@1', 'Image to Recipe R@5', 'Image to Recipe R@10']
One sample of these subsets is composed of text embedding and image embedding in the shared latent space. Since our interest lies in the recipe retrieval task, we optimized and evaluated our model by using each image embedding in the subsets as query against all text embeddings. By ranking the query and the candidate embeddings according to their cosine distance, we estimate the median rank. The model’s performance is best, if the matching text embedding is found at the first rank. Further, we estimate the recall percentage at the top K percent over all queries. The recall percentage describes the quantity of queries ranked amid the top K closest results.