Learning Norms from Stories: A Prior for Value Aligned Agents
Table 4: Results for science fiction summary transfer experiments. The BERT-scifi and XLNet-scifi models were first trained on G&G and then additionally trained on the Sci-Fi corpus.
['[BOLD] Model', '[BOLD] Test acc', '[ITALIC] F1 [BOLD] -score', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] MCC']
We use the science fiction plot summary dataset to show the capability these models have for transfer learning in another narrative context. As before, we find that transformer-based models perform well on zero-shot transfer, though in this case they perform worse than they did with the Plotto task. As with the Plotto task, we also fine-tuned our models on the sci-fi training data using the same training protocol. We see a dramatic increase in performance when given access to even a small amount of task-specific normative labels for fine tuning.