You might evaluate ChatGPT by randomly sampling first time questions by new users manually typing prompt into google and evaluating against top 3 results. You could start manually and then build a calibration model after you start to