Composing and Embedding theWords-as-Classifiers Model of Grounded Semantics
Table 1: Experiment 1 & 2 results: accuracy scores for the models and composition approaches using the refCOCO data.
['[BOLD] model', '[BOLD] Exp. 1 acc', '[BOLD] Exp. 2 acc']
The results for this experiment are in the Exp. We see that in all cases, the denotative summed-predictions for all three classifiers yield respectable performance (the first row verifies results reported in \newciteSchlangenetal2016 for this task). When considering connotative approaches, the results are more nuanced: for the mlp adj-noun approaches, we see similar results when the hidden layers are extended and when we use warm-start. This is a positive result in that composing adj-noun pairs together using warm-start is theoretically appealing because a single classifier can perform the work of two, though it needs additional training data to perform that function, whereas extending the hidden layers requires no additional retraining, and the compose-then-apply nature of it makes it more appealing than simply summing the predictions of each word applied to the objects, as has been done in prior work. Unfortunately, when extending the hidden layers to contain the neurons from the mlp classifiers for all words in the expression, the results take a hit when compared to the denotative summed predictions approach.