I get the general techbro sphere doing this because they're fucking idiots, but you are the maintainers of this technology. You know this is ultimately a language model built to synthesize information from pre-existing data sets. This is extremely fucking ghoulish.
I hate the promotion of the idea that we should be using generative adversarial networks as search engines or encyclopedias so much it's unreal, literal misinfo generation machines :agony-shivering:
You have to ask it not to do that explicitly. AI researchers call it "hallucinating" a response. You have to ask it to say "I don't know" if it doesn't know something.