OpenAI has shown with ChatGPT that you can pretty easily condition a language model to give the type of responses you want. The difficult problem is deciding what your values are not any technical "alignment" problem.