Continual and Multi-Task Architecture Search
Table 4: Comparison of MAS cell on RTE task.
['Cell Structure', 'Performance on RTE']
In these experiments, we first find the best ENAS cell structures for the individual QNLI and WNLI tasks, and use these for training the RTE task. Next, we find a joint cell structure by training ENAS via joint rewards from both QNLI and WNLI datasets. We also include the LSTM cell and RTE-ENAS cell results for fair comparison. It is clear that the multi-task cell performs better than the single-task cells. This shows that a cell learned on multiple tasks is more generalizable to other tasks. Note that the multi-task cell does not necessarily need to be better than the RTE cell, because the latter cell will be over-optimized for its own data, while the former is a more generalized cell learned from two other datasets.