After all, you only asked it to pretend to be a REPL. If you specifically ask it to run a command, it will refuse. It creates this (a little disorienting) mixed experience: seeing actual output mixed with ChatGPT's memories and guesses.