The question itself is not stupid at all; otherwise society wouldn't be able to look at data with gender/ethnicity filters. If we knew enough about ChatGPT internals then you may have a point but I'm not convinced we do.