What is the point of the hypothetical? Is AI yet intelligent, and morally competent, enough to make a good decision in a case that risks human lives. The answer, for ChatGPT, is no. Before these systems go live say, as