If you are fascinated by chatGPT and want to peek under the hood, I highly recommend you take a look at It's an excellent explanation starting from the basics of matrix multiplication to implementing modern transformers.