Inference alone is not reasonable at that scale. Models like ChatGPT needs several seconds to process an answer. Now imagine how many GPUs would be needed to sustain that at a rate of 99K queries per second. I bet a million GPUs wouldn't