Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Games
Table 1: Tiers of games. The tiers are selected by the difficulty level of games. Tier-1 is the simplest, containing only one ingredient in a recipe and one room to explore per game. Tier-6 is the most difficult, including up to three ingredients in a recipe, and twelve rooms to explore per game. The first three tiers only contain one room, which means there need be no go actions involved in these games. Maximum scores summed up over each tier are shown in the last column.
['tier', '#ingredients', '#rooms', '#games', 'max scores']
We divide the game types into six tiers of increasing difficulty. The easiest games take place inside a single room and require only one (tier-1), two (tier-2), or three (tier-3) ingredients. More complicated are the multi-room games; these may have six (tier-4), nine (tier-5), or twelve (tier-6) rooms. Intuitively, it should be very easy to learn a tier-1 game. Adding additional ingredients requires knowing how to prepare each ingredient correctly, and adding additional rooms requires finding the kitchen and other locations. During training, we use 50,000 observation steps and 500,000 replay memory entries. From a training run, we select the model with the highest score on the dev set for test inference. The maximum total steps of evaluating on one test set is thus 222×10×100. We use the percentage of scores as the evaluation criteria in the following sections. The higher the score, the better the agent. We also show a detailed percentage of scores for each tier. The average of scores of all tiers may not be equal to the all-tier result, because the total maximum scores for each tier may differ.