ML inference/deployment is not free - especially on scale. It's a common misconception that I encounter. Below a neat back-of-the-envelope calculation for ChatGPT. Seems sensible overall. I actually had the same idea on my todo list for the last two days.