ShapeGlot: Learning Language for Shape Differentiation
Table 6: Optimal hyper-parameters for ablated neural listener architectures, using both geometric modalities and word-attention and various degrees of context. Dropout numbers reflect the keep probability.
['Hyper Parameters Architecture', '[ITALIC] Baseline', '[ITALIC] Early-Context', '[ITALIC] Combined-Interpretation']
We did a grid search over the space of HP associated with each architecture separately. To circumvent the exponential growth of this space, we search it into two phases. First, we optimized the learning rate (in the regime of [0.0001, 0.0005, 0.001, 0.002, 0.004, 0.005]) in conjunction with the drop-out (keep probability) applied at the LSTM’s input, in the range [0.4-0.7] with increments of 0.05. Given the acquired optimal values, we searched for the optimal L2 weight-regularization (in the range of [0.005, 0.01, 0.05, 0.1, 0.3, 0.9]) applied at the two projection layers, and label-smoothing ([0.8, 0.9, 1.0]). For these experiments we used a single random seed to control for the data splits with the object-generalization task. We note that for the Early-Context listener, using a single 1D convolutional layer to extract the grounding vector of each object, appeared to produce better results than using a single FC layer (or deeper alternatives). This single convolutional layer we used, converted the input signal [f(vj,vk)||g(vj,vk)||vi] ∈R100×3 to a R100×1 LSTM-grounding vector for each object vi, with an 8×3×1 kernel and stride 1.