AI makes you smarter but none the wiser: The disconnect between performance and metacognition

Cool study which explored how AI affects a person’s metacognitive judgements.

Thanks to Professor Erwin for sharing this.

Tl;dr:

· AI improved a user’s task performance – ‘AI makes you smarter’

· Despite improving performance, it resulted in large overestimation of a user’s abilities, e.g. they overestimated how well they performed; creating an illusion of knowledge – ‘AI makes you none the wiser’

· The ‘classic’ Dunning-Kruger effect wasn’t found, where overestimation was found across all skill levels

· Higher AI literacy (greater technical knowledge of AI) correlated with lower metacognitive judgements

Extracts:

· Metacognition is ‘the ability to monitor and regulate one’s cognitive processes”

· ‘..research on metacognition has shown that people typically estimate themselves to be better than average (Brown, 1986), also called the ‘‘better-than-average effect’’

· ‘Metacognitive accuracy is shaped by metacognitive bias (consistent over- or underestimation of one’s cognitive abilities or performance) and noise (encompasses random, unintentional fluctuations in self-assessments”

· ‘The [Dunning-Kruger Effect] DKE appears in the connection between metacognitive accuracy and skill. It suggests that less-skilled individuals overestimate their performance, while highly competent individuals underestimate theirs’

· “…AI users in our sample … consistently overestimated their performance by about four points”

· “moderate correlation between estimated and actual scores (Table 2), with many participants estimating their joint performance with AI higher than the most skilled in the sample (Fig. 4(a)), suggests that AI improves performance but leads to highly biased self-assessments”

· “This disconnect between actual and perceived performance mirrors earlier findings on overtrust and overreliance in AI systems …Overconfidence may impair users’ ability to evaluate their performance without AI”

· “The classic DKE, where lower performers overestimate and higher performers underestimate their performance, disappeared with AI use, suggesting that while AI levels performance, it does not correct inflated self-assessments”

· “Interestingly, higher AI literacy correlated with lower metacognitive accuracy, suggesting that those with more technical knowledge of AI were more confident but less precise in judging their own performance”

· “Familiarity with AI may enhance the better-than-average effect .. leading to the over-estimation of both relative and absolute performance”

· “High metacognitive bias leads users to overestimate their performance and over-rely on AI systems … reducing their ability to critically monitor HAI outcomes”

· “using an LLM had improved logical reasoning performance as compared to no AI, but that cognitive performance gains did not scaffold metacognition”

Ref: Fernandes et al. 2026. Computers in Human Behavior