ShapeGlot: Learning Language for Shape Differentiation
Table 7: Optimal hyper parameters for literal image-based neural-speaker. The dropout numbers reflect keep probabilities and the Image-Dropout refers to the dropout applied at the VGG-image codes, before the FC-projection layer.
['LSTM Size', 'Learning rate', '[ITALIC] L2-reg.', 'Word-Dropout', 'Image-Dropout', 'LSTM-out Dropout']
Similarly, to what we did in the ablations of listener variants we conducted a two-stage grid search given a single random seed and the object generalization task. At the first stage, we searched models varying: a) the hidden neurons of the LSTM (100 or 200), b) the initial learning rate ([0.0005, 0.001, 0.003]), c) the drop-out keep probability applied on the word-embeddings ([0.8, 0.9, 1.0]) and d) the dropout keep probability applied at the LSTM ’s output ([0.8, 0.9, 1.0]). The two best performing models were further optimized by considering L2-weight regularization applied at the FC-projection layer (with values in [0, 0.005, 0.01]) and the dropout keep-probability applied before the FC-projection layer ([0.5, 0.7, 0.9 1.0]).