Getting To Know You: User Attribute Extraction from Dialogues
Table 4: Oracle results of the predicate classifier and entity generator. The entity generator is evaluated given correct predicates as input.
['[EMPTY]', '[BOLD] ACC', '[BOLD] F1']
The predicate classifier achieves a 44.4% F1 score on the multi-label classification with 61 possible predicates. In the oracle study, the entity generator, which is given the correct predicates in the distant supervision data as input, can obtain a 46.03% F1 score. Therefore, the performance drop from 46.03% to 28.68% is because of the incorrect predicate prediction.