Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent
Table 5: Accuracy breakdown on datasets. The first column indicates how the model is trained, and the first row indicates which test set the model is evaluated on.
['Method', 'Test on MTD limit', 'Test on Baseline', 'Test on Pilot Study']
Here we also report the accuracy on different test sets obtained by different training procedures, including the pilot study dataset, the baseline test set, and the MTD test set. It is clear that MTD consistently outperforms the baseline on all test sets. The margin on the MTD test set is larger than on the baseline test set, because the baseline test set has a more similar distribution to the baseline training set (which the baseline agent is trained on).