Abstract Reasoning with Distracting Features
Table 1: Test performance of LEN trained on different trajectories. "−>" denotes to training order. The first row demonstrates two datasets (i.e., 1 and 2) without distracting features while the second row illustrates datasets (i.e., 3 and 4) with distracting features. FRAR demonstrates our algorithm which optimizes learning trajectory to prevent distracting features from affecting the training of learning algorithms.
['Dataset', '1', '2−>1', '2', '1−>2', '1+2', '1−>2 −>1+2', '1−>2−> 1−>1+2', '2−>1−> 2−>1+2', 'FRAR']
To tackle the problem of abstract reasoning with distraction, we take inspirations from human learning in which knowledge is taught progressively according to a specific order as our reasoning abilities build up. As we see from the results, when no distracting features are present (first row), changing the order of the training has little impacts on the actual results. When distracting features are present (second row), however, the trajectory of training data significantly affects the training outcome. The FRAR model that we propose to optimize training trajectory in order to prevent distracting features from affecting the training achieves a significant boost of 15.1% compares to training on a single dataset. This suggests that we are able to achieve better training performance by changing the order that the learning algorithm receives the training data. We set two groups of experiments to examine training trajectory generated by the teacher model. D1 denotes that we instantiate the “and” relation on the type of “shape” as reasoning attributes and does not set the distracting attribute. D2 denotes that the reasoning attribution is based on the “size shape” and do not set the distracting attribute. D3 is similar to D1, but “size” is set a random value as the distracting attribute. D4 is similar to D2, but “type” is set a random value as the distracting attribute. In summary, there not exist distracting attributes in D1 and D2. For D3 and D4, “size” and “type” are distracting attributes respectively. We conduct experiments as follows. Without distracting attributes, D1 and D2 can promote each other to encode the reasoning attributes, thus improving the accuracy of the model. Adjusting the training trajectory in the dataset without distracting attributes only provides a small increase in the performance. It demonstrates that a model without the influence of distracting attributes is able to encode all the attributes into satisfactory embedding and perform abstract reasoning. However, joint training in the dataset D3 and D4 with distracting attributes do not promote each other. Then, our proposed model is able to find a more proper training trajectory and achieve an obvious improvement.