'In terms of underlying techniques, ChatGPT is not particularly innovative, although well engineered. ChatGPT uses Transformer architectures (a Google invention) that are pre-trained in a self-supervised manner. There is no secret behind this.' Yann LeCun, Meta Chief AI Scientist