The impact of generative AI on critical thinking skills: a systematic review, conceptual framework and future research directions

How do generative AI (GenAI) models affect critical thinking skills? This systematic review unpacked 68 studies to explore the good and the bad.

GenAI are “machine-learning algorithms, usually transformer-based large-language models (LLMs), that generate new text, code or other media from probabilistic patterns in enormous training corpora“.

Negative impact of GenAI on critical thinking:

· Review of 68 studies reveals “GenAI makes critical thinking worse by promoting automation bias, lowering metacognitive monitoring and embedding unverified mistakes into learners’ reasoning chains”

· “When solutions come right away, individuals cannot think about how to explain, defend and change their reasoning. Instead, they just accept preformed conclusions rather than constructing their own arguments”

· “ Medical education comments have already reported cases of students copying ChatGPT differentials into clinical write-ups without checking the primary sources”

· “LLM fallibility subtly affects individuals’ cognition. When the GenAI tool was asked biostatistics questions, it only got half of them right the first time, and it kept getting ANOVA wrong”

· “The same thing happens in physics research: accuracy drops from 62% on well-defined issues to 8% on poorly defined ones, creating models that make sense but are unattainable in real life”

· In geo engineering “ GenAI imagined conceptual steps that were not real until they were given bespoke educational cues. This showed “conceptual” and “grounding” flaws that new learners have a hard time finding”

· “People who unquestioningly accept authoritative responses are likely to make many argument mistakes, which discreetly undermine the analytic and evaluative parts of critical thinking”

· Metacognition is also affected, where the ease of use of GenAI “decouples perceived competence from real performance: students miscalibrate their assessments following AI assistance, overrating solution quality while overlooking errors”

· “ChatGPT helped make people feel more capable but failed to enhance their ability to critically evaluate their thinking, creating an “illusion of knowing”

Positive impact of GenAI on critical thinking:

· “GenAI can also help people think more critically when seen as an engaging partner instead of an all- knowing oracle”

· “LLMs provide an infinitely patient conversation partner that helps this cycle happen more often and clearly“

· GenAI when used in daily tasks “facilitates deeper analysis, stronger arguments and more complex creativity” but conversely “studies show that the same technologies may make people less active mentally, make them remember things less deeply and depend too much on eloquent but unverified writing”

· Other positives were found – e.g. helping teachers prepare for lessons or having students review intentionally inaccurate GenAI articles to find mistakes