TLDR: The model's responses to this question are different at different times and therefore unreliable. I can see a 'tug-of-war' going on between the autoregressive pre-training and the InstructGPT 'alignment' fine-tuning which was intended to reign in the model. 3/4