Learning Norms from Stories: A Prior for Value Aligned Agents
Table 3: Results for Plotto transfer experiments. The BERT-Plotto and XLNet-Plotto models were first trained on G&G and then additionally trained on the Plotto corpus.
['[BOLD] Model', '[BOLD] Test acc', '[ITALIC] F1 [BOLD] -score', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] MCC']
Zero-shot transfer results are achieved by testing Bi-LSTM, DPCNN, BERT-GG and XLNet-GG on the Plotto dataset; these models were trained on G&G but have never seen Plotto plot events. BERT-GG outperforms all the other models in the zero-shot transfer in terms of accuracy and MCC. These results demonstrate that the knowledge of normative and non-normative behavior gathered from the G&G stories alone facilitates a strong prior over normative/non-normative behavior without overfitting to G&G scenarios and language.