I can see this work quite well with a narrow(er) domain and an embedding model in the middle instead of ChatGPT. This would also mean that you can host everything yourself or even run on device/on client afaiu.