OpenAI GPT 3.5 and ChatGPT trained on Microsofts Azure AI supercomputing infrastructure powered by Nvidia A100 GPUs. OpenAI supercomputer is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per sec of network connectivity for each GPU server.