Learning Dynamic Belief Graphs to Generalizeon Text-Based Games
Table 7: Probing task results showing that belief graphs obtained from OG and COC do contain information about the game dynamics, i.e. node relationships.
['Model', 'Exact Match Train', 'Exact Match Test', 'F1 score Train', 'F1 score Test']
We first compare against a random baseline where each G is randomly sampled from N(0,1) and kept fixed throughout the probing task. We observe the model with random graph input can somewhat fit the training data, however, it achieves extremely low scores during test. This baseline suggests the validity of our probing task design — there is no way to perform good without having the information encoded in the belief graph G.