Augmenting Visual Question Answering with Semantic Frame Information in a Multitask Learning Approach
TABLE VIII: Performance evaluation grouped by performance intervals showing verb frequency and role frequency in each group.
['Accuracy', 'Verb', 'Role']
Fine-grained evaluation. In order to perform a fine-grained analysis of the results, performance per question, per verb and per role are computed. Multi-task CNN-LSTM performs better for who (4%), what (8%) and where (5%) when compared to CNN-LSTM. Exploring performance per verb, we can see for example cooking improves from 30.12% to 44.58% and buying from 27.42% to 64.52%. Exploring performance per role, for example, the multi-task approach improves AGENT from 48.78% to 52.29%, PLACE from 34.75% to 39.52% and ITEM from 32.27% to 39.65%. About 55% of verbs improve by less than 10%. whipping, buying, sketching, scooping, making improve by more than 30%. spanking, ejecting, farming, hitting, harvesting, moistening decline by more than 15%.