Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators
Table 9: Percentage of Correct Items and Average Ratings and Naturalness Scores for Each Personality (Personage vs. Model_Context)
['[BOLD] Person.', 'Personage  [BOLD] Ratio Correct', 'Personage  [BOLD] Avg. Rating', 'Personage  [BOLD] Nat. Rating', 'Model_Context  [BOLD] Ratio Correct', 'Model_Context  [BOLD] Avg. Rating', 'Model_Context  [BOLD] Nat. Rating']
, the average rating the correct item received (range between 1-7), and an average “naturalness” score for the output (also rated 1-7). From the table, we can see that for Personage training data, all of the personalities have a correct ratio that is higher than 0.5. The Model_Context outputs exhibit the same trend except for unconscientious and agreeable, where the correct ratio is only 0.17 and 0.50, respectively (they also have the lowest correct ratio for the original Personage data). While human utterances would probably be judged more natural, it is not at all clear that similar experiments could be done with human generated utterances, where it is difficult to enforce the same amount of experimental control.