A new paper by Pascale Fung's group describes a framework for quantitatively evaluating interactive LLMs using publicly available data sets. It then evaluates ChatGPT using 21 data sets covering 8 common NLP application tasks.