Evaluating Creative Language Generation: The Case of Rap Lyric Ghostwriting
Table 5: The maximum lengths of generated verses and % of training completed on which the verse is generated
['[BOLD] Artist', '[BOLD] Max Len', '[BOLD] % of training completed']
One aspect of the generated verse we have not discussed so far is the structure of the generated verse. For example, the length of the generated verses should be evaluated, since the models we examined do generate line breaks and also decide when to end the verse. We note that although 10 of the 11 models are able to generate long verses (up to a full standard deviation above the average verse length for that author), it takes a substantial amount of time, and the correlation between the average verse length for a given an artist and the verse length achieved by the model is weak (0.258). This suggests that modeling the specific verse structure, including length, is one aspect that requires improvement.