[...] LLMs Can Be Useful, but not for any Task The reinforcement learning neural network models driving the functionality of current LLMs constitute major technological developments (McClelland, 2009; McClelland et al., 2003). The capabilities of these models have been gathering pace over the past decades mainly through advances in computing power and the amount of data available for training - i.e. for successively adjusting the weights of the network nodes used for stochastic predictions (Perfors, 2026). But the basic model functionality has remained constant: based on high-dimensional correlation matrices describing the frequency of co-occurrence of data units over time (language) or space (graphics), the models can take user input as the start of a pattern and use it to compute the most plausible continuation of that pattern. The capacity for high-dimensional pattern matching and extension (also referred to as "autocomplete", Bergstrom, 2025) can be useful in a variety of domains, not least because the distilled patterns allow for generalisation beyond the individual instances on which they are based (Lake & Baroni,2023; Peters & Chin-Yee, 2025; but see Becker et al., 2025). For example, when trained on the respective content domains, LLMs can help identify patterns in chemical structures (Jumper et al., 2021), in clinical samples (Epping et al., 2025); and between words of different languages (Gao et al., 2024; but see Maiberg, 2026). The mechanism of matching and generalising probabilistic patterns based on information from a given database is less useful for tasks that require other types of mechanisms for their solution. For instance, tasks requiring contextual sensitivity and hence a solution to the frame problem in artificial intelligence (Oaksford & Chater, 2009; Pylyshyn, 1987); high accuracy and precision (Hsu, 2025; Kalai et al., 2025); or novel, creative solutions for which no pattern or template has yet been built (Habib et al., 2023; Meincke et al., 2025). The mechanism based limitations in the scope of applicability of LLMs are often masked in current discourse about them, a problem complicated by the optimisation of LLMs for the production of generic, plausible and confident appearing output regardless of how the output relates to what is in fact the case (Kalai et al., 2025). This risks creating the illusion that LLMs can do things that they cannot, and that they have a connection to truth and understanding that they do not. LLMs Cannot Think The companies marketing their LLMs often describe them with anthropomorphising terms like "thinking" and "reasoning", which might create the impression that they can think (Mirzadeh et al., 2025; Shojaee et al., 2026). But for that impression to be accurate we would have to stretch the meaning of the term to refer trivially to whatever the LLMs produce as output - much like the meaning of intelligence has historically been watered down to whatever the tests used to operationalise the construct measured (Loru et al., 2025; Mitchell, 2023; Quattrociocchi & Capraro, 2025; van der Maas et al., 2021). The task of developing systems with non-trivial capability for human-like cognition is computationally intractable (van Rooij et al., 2024). Focussing on the foundation rather than on the endpoint, to me there is a simple and inescapable basis to any thinking and reasoning: logical consistency. Just as we cannot see both interpretations of an ambiguous image like the rabbit-duck illusion or the Necker Cube at the same time (Gopnik & Rosati, 2001), we are incapable of assigning meaning to the conjunction of two contradictory statements. We can focus our attention on the meaning of one statement and then move over to the meaning of the other, but we cannot integrate them into a single meaningful representation. Thinking and understanding break down when we encounter an inconsistency, like an alarm signal that prompts us to stop and reevaluate the situation (Johnson-Laird et al., 2004); and even thinking that is not outright contradictory but moves fast and loose from one representation to another one incompatible with it is classified as a formal thought disorder (Holyoak & Morrison, 2005). This does not imply people are good at detecting inconsistencies regardless of problem complexity (Oberauer et al., 2016); but merely that it is a foundation, however local and fragile, on which thinking and understanding depends (Oaksford & Chater, 2020; Wheeler, 2026). Now, one of the more notorious features of LLMs is their logical inconsistency. They routinely produce contradictory output or output that changes the topic mid-argument, and construct so-called "hallucinations" or "bullshit" responses (Frankfurt, 2005; Hicks et al., 2024; Kalai et al., 2025) in unforeseeable ways (Hägele et al., 2026). Further, LLMs seem incapable of detecting when such inconsistencies occur and just keep producing further output unabated - hence their functionality breaks down in ways different from how human thinking breaks down. This makes sense as their inconsistency is not a bug but a natural consequence of the stochastic mechanisms underlying them, together with their disconnection from any ground truth about which relatively stable conceptual representations could be formed (Kalai et al., 2025; Spencer-Brown, 1969; Wittgenstein, 1991). LLM developers have themselves stated that the problem of inconsistent, nonsensical output is impossible in principle to overcome, regardless of the amount of computing power and training data the models are based on (Shojaee et al., 2026; Song and Han, 2026). The path from LLMs to thinking machines thus seems impossible from the outset due to the absence by design of the requirement for consistency. Many older computational models exist that fulfil the consistency requirement. But the capacity for both consistency and scalability remains an open, potentially unsolvable problem (Gödel, 1931; Kwisthout et al., 2011; Pylyshyn, 1987). LLMs Can Undermine Thinking and Understanding Thinking and reasoning, and with them knowledge and understanding, can improve with practice, and they can deteriorate without practice. LLMs are sometimes compared to electronic calculators (Geuter, 2024; Voinea et al., 2026), which have greatly increased the speed and accuracy of everyday calculations. The concomitant reduction in the need for simple mental arithmetic may have led to a decrease in our average mental arithmetic skills - but it freed up time to engage in potentially more complex and creative tasks. At the same time, our collective understanding of simple arithmetic has arguably not declined because the arithmetic rules by which calculators operate are transparent, precise and can be looked up in reliable sources anytime we need them (Sloman & Fernbach, 2017). The situation is different in several ways for LLMs. They are being used to replace complex and creative tasks that draw on our capacity for critical thinking (Reuters, 2026). They have the feature of producing seemingly plausible but imprecise and sometimes wildly inaccurate output, and they are intransparent about their sources - although their training data tends to include any information from the internet, however unreliable and regardless of legal requirements for source acknowledgment (Blau et al., 2024; Gewirtz, 2025; Meyer, 2025). For example, if asked for a solution to Lord’s paradox (Lord, 1967), a LLM might produce different output each time it is asked, and every time the output may sound plausible but may be justified in part by false or nonexistent evidence that is difficult to detect by nonexperts in the field (Fisher, 2021; Walters & Wilder, 2023). The literature on the impact of LLMs on thinking and understanding is still very new and preliminary. But some studies have pointed to reduced task engagement and learning when relying on LLMs (Melumad, 2025; Shen, 2026; Stadler et al., 2024); and based on the existing literature on cognition we can expect the principle "use it or lose it" to apply here too (Bainbridge, 1983; Furman, 2025; Mızrak, 2020). In contrast to the calculator example, what we risk undermining in this case is our capacity for critical thinking, and the source reliability and transparency on which our collective understanding depends. This comes in addition to LLM enabled mass production of slop, mis- and disinformation (Clark & Lewandowsky, 2026; Furman, 2025; Köbis & Doležalová, 2021; Perfors, 2025; Thorp, 2026). Technology is arguably not value neutral, and the ways in which current LLMs have been built and deployed risk undermining not only our thinking and understanding as individuals but also our participation as active, diverse citizens in democratic decision making processes (Kant, 1784; Lewandowsky & Hertwig, 2025; Lewandowsky & Garcia, 2026). Huxley’s dystopic novel Brave New World (Huxley, 1932) might reflect a luddite position, which might sound pejorative in first instance. But it illustrates that technology can take us in different directions towards different societal goals, which are worth thinking about. There Are no Shortcuts to Understanding Understanding doesn’t work without thinking, which is often hard, cumbersome and full of errors. It will also keep trapping us in illusions, as Shiffrin et al. point out. But there is no free lunch to understanding. If we keep working on it we have reason to expect to keep escaping some of the illusions and increase our understanding over time - following the positive side of the "use it or lose it" principle. Some uses of LLMs may not undermine understanding, and in some cases we can avoid illusions by making an active decision about which parts of our thought processes, if any, to replace with their output. https://doi.org/10.1007/s42113-026-00288-6