Sememe Prediction: Learning Semantic Knowledge from Unstructured Textual Wiki Descriptions
Table 2: Comparison with different baseline models. All the models use two wiki resources in this table. “P” means Precision,“R” means recall rate.
['Model', 'P', 'R', 'F1']
Comparison with Baselines: From the results we can see that clustering based method ML-KNN performs the worst for sememe prediction. We assume that this is because the textual descriptions are very diverse, which makes KNN hard to determine the borders among space of different labels. Methods that aim to transform classifiers to multi-label task perform closely to each other, with F1 scores around 25%. Effect of Proposed Soft Loss Function: We think that this is because our novel loss function eases the restriction on the order between labels. For example, assume the target sememes are (s1,s2,s3) in order. At the first time step of decoding, the one-hot loss function would strongly punish the decoder from giving s2 or s3 probabilities, which may confuse the decoder, because at the moment the difference between time step 1 and time step 2 may not be significant when the order of the labels are not obvious. However, our soft loss function would still lead the decoder to firstly choose s1, while the two labels s2 and s3 are also encouraged with some probability less than s1. The experiment results show that this modification is very effective to make seq2seq work well on the multi-label problem.