Did we ‘train’ AI LLMs to hallucinate?
This study from OpenAI argues that hallucinations “need not be mysterious—they originate simply as errors in binary classification”.
And, interestingly, hallucinations, persist, in part because the models “are optimized to be good test-takers, and guessing when uncertain improves test performance”.
That is, they’re rewarded in training for guessing answers compared to abstaining.
SafetyInsights.org 