This is similar problem people face at Robotic where the aim is to translate human language from a user interface to predefined set of arm movements. I think LLMs like ChatGPT themselves can do these translations from human language to