Visual Affect Around the World:A Large-scale Multilingual Visual Sentiment Ontology
Table 3: Crowdsourcing results via no. of input candidate ANPs #cand, #users, countries #coun, and perc. of ANPs accepted %correct and annotator agreement %agree.
['[EMPTY]', '[BOLD] #cand', '[BOLD] #users', '[BOLD] #coun', '[BOLD] %correct', '[BOLD] %agree']
On average, our crowdsourcing validated that a vast number of the input candidate ANPs from our automatic ANP discovery and filtering process were indeed correct ANPs. English, Spanish and Russian were the top three for which the automatic pipeline performed the best, where every three in five ANPs were approved by the crowd judgements. However, for certain languages, including German, Dutch, Persian and French, the number of ANPs rejected by the crowd was actually greater than accepted ANPs due to a higher occurrence of mixed language pairs, e.g. witzig humor.