The E2E NLG Shared Task
Table 4: Lexical Sophistication (LS) and Mean Segmental Type-Token Ratio (MSTTR).
['[BOLD] Dataset', '[BOLD] Tokens', '[BOLD] Types', '[BOLD] LS', '[BOLD] TTR', '[BOLD] MSTTR']
We complement the traditional measure of lexical diversity type-token ratio (TTR) with the more robust measure of mean segmental TTR (MSTTR) The higher the value of MSTTR, the more diverse is the measured text. In addition, we measure lexical sophistication (LS), also known as lexical rareness, which is calculated as the proportion of lexical word types not on the list of 2,000 most frequent words generated from the British National Corpus.