Nei LLM, il concetto di "emergent ability” ha una definizione precisa: An ability that is “not present in small models but is present in large models.” https://www.jasonwei.net/blog/emergence Non mi pare che in questo contesto sia mai stata considerato di: "use it as a hypothesis to predict the outcome of a complex, unknown system, with the hope that a desired property will emerge;” Che il comportamento sorprendente e non facilmente spiegabile dei LLM, vada oltre le capacità per cui sono stati allenati, è un fatto appurato anche se controverso. Ne ho parlato con Giorgio Parisi, che sul tema dei sistemi complessi ha vinto il premio Nobel, e ha concordato con me che il fenomeno possa essere appunto spiegato come l’applicazione su larga scala di una semplice legge di probabilità: in questo caso la probabilità delle prossima parola in una sequenza. — Beppe

On 19 Jul 2023, at 10:37, nexa-request@server-nexa.polito.it wrote:

Date: Wed, 19 Jul 2023 14:36:59 +0000 From: Daniela Tafani <daniela.tafani@unipi.it <mailto:daniela.tafani@unipi.it>> To: "nexa@server-nexa.polito.it <mailto:nexa@server-nexa.polito.it>" <nexa@server-nexa.polito.it <mailto:nexa@server-nexa.polito.it>> Subject: [nexa] “Emergence” isn’t an explanation, it’s a prayer. A critique of Emergentism in Artificial Intelligence Message-ID: <8aced3bd15f24f548084f82dbf280719@unipi.it <mailto:8aced3bd15f24f548084f82dbf280719@unipi.it>> Content-Type: text/plain; charset="windows-1252"

“Emergence” isn’t an explanation, it’s a prayer A critique of Emergentism in Artificial Intelligence <https://ykulbashian.medium.com/?source=post_page-----ef239d3687bf-----------...> [cardboarddreams]

Emergence is the notion that in a complex system, the interactions of the whole may exhibit properties that are not present in the individual parts. It is most often applied to examples in physics and nature, such as the collective behaviours of ant colonies, the self-organizing principles of social groups, or the macro properties of molecules. In the last few decades, it has given rise to emergentist perspectives of human cognition<https://onlinelibrary.wiley.com/doi/10.1111/j.1756-8765.2010.01116.x>, and even of consciousness<https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597170/>. These are based on the recognition that the complexities and mysteries of the human mind, being a part of nature, may be characterized as emergent phenomena.

This approach has an intuitive appeal. It is supported by the superficial facts: the brain — the source of intelligence and consciousness— is most certainly a complex of interconnected neurons. Emergentist interpretations of human behaviour may also boast some recent wins—the proliferation of LLMs (e.g. ChatGPT) may be seen as one such success. This has reignited the discussion of whether emergence is the best way to frame intelligence.

You may have noticed that the last paragraph switched between two subtly different uses of “emergence”. The first use was to describe an observed emergent property; consciousness, we have seen it, likely emerges out of neuronal interactions. The second was to use it as a hypothesis to predict the outcome of a complex, unknown system, with the hope that a desired property will emerge; e.g. intelligence will arise from the interactions of artificial neurons at scale. The latter example is the driving motivation behind the multi-million dollar Human Brain Project<https://en.wikipedia.org/wiki/Human_Brain_Project>. The project justifies its cost by leaning on evidence from observed instances of emergence. Doing so paints emergence as a theory; that is, existing observations can be used to justify future predictions.

But emergence is not a theory. Emergence can only be ascribed to a phenomenon in retrospect, once you already know what has “emerged”. The higher-level properties that emerge are qualitatively different from those at the lower-level — otherwise it wouldn’t be “emergence”. So by necessity they could not have been predicted from the lower-level ones. The properties of “intelligence” could not have been logically foreseen from the properties of neurons unless you had already observed that property emerge in a similar substrate. And even then it’s just a guess that is likely to be wrong given the complexity of the interactions involved; small differences can easily invalidate the hypothesis. In both cases emergence gives no new information: when explaining existing examples it gives you no new insights about the processes except that they happen; and when predicting unknown behaviours it gives very poor guarantees that anything you expect to happen will do so.

Emergence is only really valid as a general metaphysical classification of certain phenomena. It’s a metaphysical category, like “cause”, “effect” or “change”. Using the word when explaining cognition is not wrong per se, it just has no real meaning or explanatory force. It’s like having a theory of “thing-happened-ness” — it’s correct, but void of content. Take, for example, the following quotes from a review article on emergence:

This process gives rise to an emergent tendency to facilitate perception of items consistent with the patterns of English orthography, without explicitly representing this knowledge in a system of rules, as in other approaches.

…

However, such modes of thought themselves might be viewed as emergent consequences of a lifetime of thought-structuring practice supported by culture and education.

Emergence in Cognitive Science, McClelland<https://onlinelibrary.wiley.com/doi/10.1111/j.1756-8765.2010.01116.x>

If you removed the word “emergent” from the above two sentences, would anything important change? Indeed any sentence that includes “emergent” would give the same information if you removed it; “it gives rise to emergent properties” means the same as “it gives rise to properties”, or “there is an emergent tendency” is not substantially different from “there is a tendency”.

Adding “emergent” to any sentence doesn’t increase its useful information content.¹

Emergence has no information that fundamentally differentiates it from a “miracle”. If I were to say that applying transformers<https://arxiv.org/abs/1706.03762> to Neural Networks creates intelligence through a miracle, I would be ridiculed. Were I to say that they create intelligence through emergent interactions, suddenly they gain an air of scientific credibility — but what have I added to the conversation with the use of that word? What quantifiable scientific facts are entailed in the term “emergent”? There are none.

In cognitive science, emergence is regularly used to “explain” the connection between two phenomena, when it is otherwise complex and difficult to predict: e.g. how neuronal firing gives rise to consciousness, or transformers to the appearance of language comprehension. Where there may be a connection, but nothing more is known or can be proved, emergence is a placeholder that fills the gap. The word gives weight and gravitas to what is essentially a blank space.

Despite emergence contributing nothing of substance to the discussion, as a concept it admittedly has a compelling intuitive appeal. There is a wonderful feeling about the notion of emergence. It does seem to be adding something valuable, as if you’ve discovered a magical ingredient by which you can explain mysterious phenomena. That’s the reason it continues to be popular, and gets inserted into scientific discussions. It convinces the listener that something has been explained with scientific rigour when all we’ve done is to say “it’s complicated”.

Besides the good feeling, however, emergence is void of any explanatory power. And so it has no scientific value in a predictive capacity. You can’t use it to say anything about what an unknown system will do; only what you hope it will do. When applied to pie-in-the-sky AI futurism, emergence has become synonymous with “I’m sure the system will work itself out”. It indicates that the author has a feeling that a complex system will align at some point, but no clear sense of how, why, or when. Insofar as intelligence does manifest in a specific instance, “emergence” doesn’t tell us anything interesting about how it happened. And insofar as intelligence hasn’t yet manifested, emergence doesn’t tell us when it will or what direction to take to get there.

In the field of AI development, emergence is invoked whenever someone encounters a phenomenon in the human mind and has no idea how to even start explaining it (e.g. art, socialization, empathy, transcendental aesthetics, DnD, etc). If said researcher already has a working theory of AI, this realization is disheartening. So they look deeper into the matter, find some point of overlap between the existing theory and the missing behaviour, and assume that with enough time and complexity the missing pieces will emerge.

Emergence is attractive in such cases because it puts the author’s mind at ease, by making it seem like they have a viable mechanism that only needs more time to be vindicated. It placates their inner watchdog, the one that demands concrete, scientific explanations. Emergence, being related to complexity and superficially validated by experiments such as Conway’s Game of Life, is enough to lull that watchdog back to sleep.

This justifies continuing to ignore any shortcomings in a theoretical model, and persisting on the current path. Like the proverbial man who searches for his lost keys under the lamplight, because that is where the light is, he hopes that with enough persistence his keys will “emerge”. The only other alternative is to admit failure, and to give up any hope of accomplishing what you want within this lifetime.

Scientists, it seems, can have superstitions too. And emergence has a powerful narcotic effect: it feels so reasonable and credible on a gut level². There are many factors that prevent a given researcher from investigating emergence too deeply and realizing that it lacks any substance. First, there appears to be a lot of external evidence to back it up in the natural world. This, as was pointed out, equivocates between retrospective and prospective uses of the term, and so legitimate uses are being conscripted to justify the illegitimate ones. Secondly, the fact that emergence exclusively concerns itself with intractably complex systems means anything behind its curtain by definition can’t be studied. So it conveniently excludes itself from exactly that analysis which would reveal it to be hollow.

In the end emergence isn’t an explanation; it’s an observation combined with a recognition of ignorance. Wherever emergence shows up there is an implicit acceptance that everyone involved is at a loss for how to approach the topic. It’s not that properties like intelligence won’t emerge from neural activity, it’s that emergence is a placeholder that justifies and promotes a lack of interest in exploring the details behind the connection. It discourages investigation. By invoking the term, we are merely thanking the nature gods for granting us this emergent property (aka property), and trying not to examine their gifts too profanely or with ingratitude. This impulse is understandable, since we don’t think we’ll discover an answer if we were to dig in. But we shouldn’t allow our insecurities to masquerade as science, or else they may become ingrained to the extent that they are difficult to uproot. A false answer stands in the way of a true one.

¹ This used to say ‘You can remove “emergent” from any sentence and it would mean the same thing’, but that has caused some confusion, so to clarify: the word “emergent” when used as an adjective doesn’t add new or useful information; you won’t know any more about the subject than you did before.

² A self-aware researcher should notice if they have a strong intuitive or emotional reason for holding on to the idea. If you ever feel that emergence is so self-evident that it can never be disproved, that should give you pause — perhaps you have strayed outside the bounds of scientific inquiry and into metaphysical expositions. Not that there’s anything wrong with the latter…

https://ykulbashian.medium.com/emergence-isnt-an-explanation-it-s-a-prayer-e...