I think you're misunderstanding how the model works. It predicts next tokens based on past tokens and the LLM trained on large bodies of text. It doesn't have an underlying database of "factual" elements it incorporates or searches, and its output doesn't have an underlying semantic structure that can be verified or reasoned about. The entirety of the quality of its output can only be judged by whether it "sounds" like the rest of the text on which it was trained.

I think making the connection between the predictive output and an underlying representation of reality is the next great step, but until that happens, chatGPT's output is just amazing mimicry of human language.