SummEval: Re-evaluating Summarization Evaluation
(b) Model scores from other text generation evaluation metrics.
['[BOLD] Method', 'BLEU', 'CHRF', 'CIDEr', 'METEOR', 'Length', 'Stats (cov/comp/den)', 'Repeated (1/2/3)']
The results align with insights coming from human evaluation of models. We found that for most metrics, the highest scores were assigned to large models pretrained on vast quantities of data. However, several metrics, such as S3, SummaQA, SMS, CHRF, and METEOR tended to favor extractive models, assigning highest scores to their outputs.