Dual Attention Network for Product Compatibilityand Function Satisfiability Analysis
Table 4: Performance on Product Compatibility Analysis (PCA): PCA F1 is the averaged F1 scores over 3 types (compatible, incompatible and uncertain) of complementary entities; CER F1 only evaluates the performance of Complementary Entity Recognition without considering the polarity; Polarity Accuracy only evalutes the polarity detection given successfully identified complementary entities.
['Method', 'PCA F1', 'CER F1', 'Polar. Acc.']
We first evaluate the performance of product compatibility analysis. Note that the label space of this task is Lc={O, C, I, U}, as described in Section Model Overview and Preliminary. We consider an extracted entity that has more than or equal to 50% overlapping words with the ground truth entity as a positive extraction. The polarity of a positive extraction is computed as the majority type voted from all words in such a positive extraction. So a true positive example must have at least 50% overlapping words and a match on polarity. Any positive extraction with no corresponding ground truth entity is considered as a false positive example. Any example with mismatched polarity or negative extraction is considered as a false negative example. Further, by only considering a positive extraction as a true positive and a negative extraction as a false negative, we compute CER F1 for complementary entity recognition. Given a positive extraction, we further compute the classification accuracy over 3 polarity types to show the effectiveness of polarity detection.