Im not pursuing the idea, it was just an idle train of thought along the lines of but how could we make a ChatGPT-like assistant also process visual information? But since CLIP already exists I think something along those lines would be