- nexa - server-nexa.polito.it

watermarking AI-generatet content
by Andrea Bolioli July 28, 2023

July 28, 2023

Buongiorno, condivido un pezzo di newsletter di Andrea Ng sul watermarking dei contenuti generati automaticamente (THE BATCH, 26-07-2023, DeepLearning.AI). Dear friends, Last week, the White House announced voluntary commitments by seven AI companies, as you can read below. Most of the points were sufficiently vague that it seems easy for the White House and the companies to declare success without doing much that they don’t already do. But the commitment to develop mechanisms to ensure that users know when content is AI-generated, such as watermarks, struck me as concrete and actionable. While most of the voluntary commitments are not measurable, this one is. It offers an opportunity, in the near future, to test whether the White House’s presently soft approach to regulation is effective. I was pleasantly surprised that watermarking was on the list. It’s beneficial to society, but it can be costly to implement (in terms of losing users). As I wrote in an earlier letter, watermarking is technically feasible, and I think society would be better off if we knew what content was and wasn’t AI-generated. However, many companies won’t want it. For example, a company that uses a large language model to create marketing content may not want the output to be watermarked, because then readers would know that it was generated by AI. Also, search engines might rank generated content lower than human-written content. Thus, the government’s push to have major generative AI companies watermark their output is a good move. It reduces the competitive pressure to avoid watermarking. All the companies that agreed to the White House’s voluntary commitments employ highly skilled engineers and are highly capable of shipping products, so they should be able to keep this promise. When we look back after three or six months, it will be interesting to see which ones: - Implemented a robust watermarking system - Implemented a weak watermarking system that’s easy to circumvent by, say, paying a fee for watermark-free output - Didn’t implement a system to identify AI-generated content To be fair, I think it would be very difficult to enforce watermarking in open source systems, since users can easily modify the software to turn it off. But I would love to see watermarking implemented in proprietary systems. The companies involved are staffed by honorable people who want to do right by society. I hope they will take the announced commitments seriously and implement them faithfully. I would love to get your thoughts on this as well. How can we collectively hold the U.S. government and AI companies to these commitments? Please let me know on social media! Keep learning, Andrew

3 7

Re: [nexa] LLM (di serie "A"): possibili in Italia/Europa?
by Giuseppe Attardi July 28, 2023

July 28, 2023

Scusate se, rientrando, ritorno su questo tema per fare alcuni chiarimenti. La stima di 10^23 FLOPS, come qualcuno ha capito, è il conto totale dei FLOPS necessari per allenare GPT-3. Si tratta di stime, riportate ad esempio su https://www.hyro.ai/glossary/gpt-3/#:~:text=To%20be%20exact%2C%20GPT%2D3,am…., e riguardano le risorse necessarie per UN SINGOLO run. I costi complessivi delle risorse sono stimati in diverse centinaia di milioni di $, in quanto occorrono diversi run, per non parlare dei modelli successivi, GPT-3.5 (ChatGPT) e GPT-4. E ovviamente le macchine adesso non sono ferme, ma stanno lavorando al prossimo modello, oltre che a fornire accesso al servizio (a pagamento). Prosegue sotto. > On 19 Jul 2023, at 10:37, nexa-request(a)server-nexa.polito.it wrote: > > Date: Wed, 19 Jul 2023 16:07:06 +0200 > From: Damiano Verzulli <damiano(a)verzulli.it <mailto:damiano@verzulli.it>> > To: nexa(a)server-nexa.polito.it <mailto:nexa@server-nexa.polito.it> > Subject: [nexa] LLM (di serie "A"): possibili in Italia/Europa? [Era: > nexa Digest, Vol 171, Issue 53] > Message-ID: <737ccbae-8b9c-3adb-542d-259c837ce2ae(a)verzulli.it <mailto:737ccbae-8b9c-3adb-542d-259c837ce2ae@verzulli.it>> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Il 19/07/23 13:30, Giuseppe Attardi ha scritto: >> [...] >> Le risorse di calcolo per costruire GPT-3.5 sono stimate in 10^23 >> FLOPS per un costo di centinaia di milioni di $ >> Meta, per rilasciare i suoi modelli, ha costruito un Research >> Supercluster con 10.000 GPU Nvidia, che secondo Yann LeCun è già in >> overbooking. > > > Leggo da una fonte terza (Wikipedia) che "Leonardo" [1] ormai ha quasi > un anno, è costato 240M€ e di picco fa 250 petaFLOPS (aka: ~10^17), > anche grazie ai suoi 13.824 GPU-core. La questione non è l’esistenza di cluster di server adatti allo sviluppo di LLM: ci sono quelli di Azure, che usa OpenAI, quelli di GCP che usa Google, quelli di AWS, quelli di Nvidia. Il Cineca è solo in parte diverso da questi, in quanto circa metà delle risorse sono concesse in uso a progetti selezionati ed approvati dallo stesso Cineca: https://leonardo-supercomputer.cineca.eu/access-leonardo-hpc-resources/ Attualmente la call è chiusa. Personalmente ho avuto approvato un piccolo progetto, che viene messo in esecuzione con priorità bassa rispetto ad altri. Anche Colab di Google mette a disposizione server con TPU per lo svolgimento di piccoli progetti: i miei studenti lo usano, ma si lamentano del fatto che le risorse non bastano, che i run vengono interrotti improvvisamente e vengono a chiedermi di usare altre risorse. Invece mi è stato rifiutato un progetto più grande per produrre un sistema di Neural Machine Translation multilingue, per uso in una collaborazione con la European Broadcasting Corporation. Per sviluppare LLM o AI allo stato dell’arte, i ricercatori di OpenAI, Google, Meta, X.AI e le nuove startup come Converse.AI o Anthropic AI hanno a disposizione un intero cluster. Ossia come se il Cineca mettesse a disposizione quasi tutto Leonardo per UN SINGOLO progetto o gruppo di progetti! E non sono soggetti all’approvazione di terzi. Quindi dovrebbe essere chi finanzia i progetti a mettere a disposizione le risorse per svolgerli, come fa ad esempio la NSF, fornendo crediti di calcolo cloud, per evitare asincronie e valutazioni separate. https://www.nsf.gov/pubs/2021/nsf21058/nsf21058.jsp > > Leggo da altra fonte terza (Top500 [2]) che attualmente (06/2023) > risulta 4° al mondo, come potenza di calcolo. > > Leggo dal sito ufficiale [3] che: > "Leonardo's main goals are [...] The computational power of Leonardo > will boost scientific exellences and industrial strenght across Europe...." > > Non si parla di IA/ML, né si accenna agli LLM. Ma faccio comunque fatica > ad immaginare che queste tonnellate di ferro *NON* possano essere > utilizzate dalla comunita' della ricerca Italiana (...magari, in modo > coordinato con gli altri paesi EU, dove "giocattoli" simili sono > comunque presenti) a questo scopo. Ovviamente non mi aspetto che > parcheggiati davanti al Tecnolopolo ci siano una fila di TAXI, pronti a > scattare all'ordine di Cineca, per "prelevare" i ricercatori in giro per > l'Italia al fine di portarli al Tecnopolo... per conoscere il giocattolo > e iniziare ad usarlo. Come riportato sopra, il Cineca fa le sue call per progetti che utilizzino Leonardo. La gran parte delle risorse vanno ai progetti di fisica alte energie, come si desume dal Rapporto Annuale Cineca 2018. > > Certo: se il dottorando X, o l'assegnista Y (o anche il Ricercatore Z o > il docente K) sentono il bisogno di avere del ferro sul pianerottolo di > fianco al loro studio, in UNIV [come accade in UniPI, ad esempio, con i > sistemi NVIDIA qui discussi, qualche giorno fa]... allora il discorso > cambia... Non so di cosa parli: a UniPI io sono stato responsabile di un server con 4 GPU Nvidia P100 acquistato con fondi comuni di 5 gruppi di ricerca, che è installato nel Green Datacenter di S. Piero a Grado (che ho contribuito a progettare e realizzare), quando Cineca non aveva GPU a disposizione né parlava di ML e AI, ed è usato in condivisione al 100% 24h/365. Tra coloro che usano le mie GPU, ci sono ricercatori del progetto Virgo, che a quel tempo non le trovavano altrove. In generale, ho sempre parlato di un servizio cloud condivisibile con altri ricercatori europei, ma gestito dagli stessi ricercatori del settore, non da altri. Esattamente come il CERN decide da solo quali progetti sviluppare, e magari usa anche le risorse del Cineca, pagandole coi fondi propri. — Beppe PS. Ho avuto un ruolo di consulente del MISE al tempo della decisione di finanziare con 120 milioni € Leonardo, il maggiore finanziamento nazionale di EuroHPC, proprio sostenendo l’importanza di disporre di risorse per AI. > > Un'ultima nota a chiusura: sono cosciente che fra 10^17 e 10^23 c'e' > *MOLTA* differenza (a proposito: qual'e' la fonte di 10^23?). Prima di > preoccuparmi di questo, pero', attenderei di vedere che quei 10^17 > stiano lavorando almeno come 10^16 per un buon periodo di ore/mese. > Dopodiché sarei pronto ad alzare la mano e chiedere qualcosa... di piu' > performante. > > Un saluto, > DV > > > [1] https://en.wikipedia.org/wiki/Leonardo_(supercomputer) > [2] https://www.top500.org/lists/top500/2023/06/ > [3] https://leonardo-supercomputer.cineca.eu/ > > -- > Damiano Verzulli > e-mail: damiano(a)verzulli.it <mailto:damiano@verzulli.it> > ---

3 4

Re: [nexa] Rant against centralising e-mail in big-tech silos, and breaking the internet in the process
by Giuseppe Attardi July 28, 2023

July 28, 2023

Sacrosanta protesta. La mia università era tra le poche che fino a pochi anni fa gestiva un proprio servizio di posta per tutto l’ateneo, giustificato anche per il fatto che potesse garantire la consegna delle comunicazioni ufficiali a tutti gli interessati. Poi il nuovo prorettore all’informatica, Antonio Cisternino, ha deciso di trasferire tutto su MS Exchange, e ha anche introdotto un servizio di analisi dei link per contrastare il phishing, affidato a un prodotto commerciale di SonicWall, il quale legge tutte le mail e sostituisce ogni link con un link a un proprio server. Come riportato nel blog, questi link sono illeggibili e lunghissimi: quando ci si clicca, si viene portati sul sito del servizio che lo analizza e poi ridirige all’originale. Ciò è fastidioso perché allunga il tempo per aprire la pagina (una decina di secondi) e influenza anche persone esterne all’università se si fa un forward del messaggio. Ho protestato contro questa operazione, che ritenevo lesiva della privatezza delle comunicazioni, nella lista di discussione del mio dipartimento. Il risultato è stato, che il direttore, pochi minuti dopo, con strana solerzia, in un sabato pomeriggio, mi ha escluso dalla lista. — Beppe > On 1 Jul 2023, at 11:55, nexa-request(a)server-nexa.polito.it wrote: > > From: Enrico Nardelli <nardelli(a)mat.uniroma2.it <mailto:nardelli@mat.uniroma2.it>> > To: Nexa <nexa(a)server-nexa.polito.it <mailto:nexa@server-nexa.polito.it>> > Subject: [nexa] Rant against centralising e-mail in big-tech silos, > and breaking the internet in the process > Message-ID: <a0b8adba-ac91-9cff-071c-117cd7b192ca(a)mat.uniroma2.it <mailto:a0b8adba-ac91-9cff-071c-117cd7b192ca@mat.uniroma2.it>> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Universities and research institutes used to be at the forefront when it > comes to development of the fundamental technologies that power the > internet. As early as 1965, researchers at MIT developed a kind of > precursor to e-mail. In 1978, researchers at Berkeley leveraged the > already existing unix tool 'mail' (1971) to communicate over a larger > network. Universities and research institutes were among the first to > embrace this new way to communicate with peers across ever greater > distances and eventually, with the advent of the internet, all over the > globe. > > I'm a rather dismayed to see those universities and institutes nowadays > no longer as pioneers and innovators in this area, but instead as mere > consumers of ready-made corporate solutions, following corporate > interests and centralising solutions. > > ... > > https://proycon.anaproy.nl/posts/rant-against-centralising-e-mail/ > > (è di febbraio, ma non mi sembra di averlo visto passare in lista; mi > scuso qualora fosse successo) > > -- > > -- EN > > https://www.hoepli.it/libro/la-rivoluzione-informatica/9788896069516.html > ====================================================== > Prof. Enrico Nardelli > Presidente di "Informatics Europe" > Direttore del Laboratorio Nazionale "Informatica e Scuola" del CINI > Dipartimento di Matematica - Università di Roma "Tor Vergata" > Via della Ricerca Scientifica snc - 00133 Roma > home page: https://www.mat.uniroma2.it/~nardelli > blog: https://link-and-think.blogspot.it/ > tel: +39 06 7259.4204 fax: +39 06 7259.4699 > mobile: +39 335 590.2331 e-mail: nardelli(a)mat.uniroma2.it <mailto:nardelli@mat.uniroma2.it> > online meeting: https://blue.meet.garr.it/b/enr-y7f-t0q-ont > ======================================================

12 33

Re: [nexa] The Santiago Boys
by maurizio lana July 28, 2023

July 28, 2023

Le Monde Diplomatique appena uscito ha un articolo intitolato «Une multinationale contre Salvador Allende» https://www.monde-diplomatique.fr/2023/08/MOROZOV/65981 di Evgeny Morozov "auteur de The Santiago Boys, ... , et dont s’inspire cet article" Maurizio DEUX semaines après l’élimination de Salvador Allende et de la démocratie chilienne par le coup d’État sanglant d’Augusto Pinochet, le New York Times reçut tard dans la nuit un appel anonyme. « Notez, recommanda la voix au téléphone, car je ne répéterai pas. » En cette fin septembre 1973, quelque chose d’inouï était sur le point de survenir. « Dans quinze minutes, une bombe explosera dans l’immeuble d’International Telephone & Telegraph. » La cible, connue sous son sigle ITT, n’était pas choisie au hasard : « C’est en représailles des crimes commis par ITT contre le Chili (1). » À l’époque, ce géant de la technologie devenu un conglomérat tentaculaire compte au nombre des plus grandes multinationales de la planète. À son illustre conseil d’administration siègent un ancien directeur de l’Agence centrale de renseignement américaine (CIA) et un ex-président de la Banque mondiale — un casting idéal pour propulser l’un des plus gros contractants de l’armée américaine parmi les principaux profiteurs de la guerre du Vietnam. La compagnie affiche fièrement sa position au sein du complexe militaro-industriel. « Pour voir dans l’obscurité, voyez ITT. La nuit a cessé d’appartenir à la guérilla », proclame une publicité pour ses appareils de vision nocturne diffusée en 1967, l’année même où Ernesto« Che » Guevara est assassiné en Bolivie. La compagnie fait l’objet d’appels au boycott, comme celui dirigé contre le pain industriel produit par une filiale du groupe. « Achetez du pain, achetez des bombes : ITT au Vietnam », titre alors un journal de gauche. La redéfinition du sigle en Imperialism, Treason and Terror (« impérialisme, trahison, terreur ») se répand dans les milieux militants. Mais de là à déposer une bombe en plein Manhattan... L’engin explose finalement à 5 heures 40 du matin au 437 de l’avenue Madison, siège de la branche latino-américaine d’ITT. C’est la troisième attaque perpétrée contre la multinationale en moins de deux semaines, après Rome et Zurich. Et la série ne fait que commencer... À la différence du techlash actuel — terme à la mode pour décrire l’hostilité que provoque la Silicon Valley —, les actions menées contre ITT en 1973 occasionnent plus de dégâts que des tweets indignés. Pour ses détracteurs, le groupe incarne non seulement le capitalisme multinational mais également une puissance autonome, dotée de sa propre politique étrangère, de son propre service d’espionnage et même de son propre personnel politique, un attelage d’anciens militaires, de barbouzes, de diplomates et de journalistes lauréats du prix Pulitzer reconvertis en chargés de relations publiques. ITT semble détenir toutes les prérogatives d’une puissance étatique. D’où le titre du livre paru à son sujet en 1973 : L’État souverain (2). Un jeune avocat nommé Fidel Castro Les accusations de techno-féodalisme qui pleuvent aujourd’hui sur les géants de la Silicon Valley (3) — dépeints comme des seigneurs médiévaux qui décident du sort de leurs usagers — réactualisent en réalité des griefs vieux d’un demi-siècle : même un ouvrage à la gloire d’ITT, paru au début des années 1980 (4), convoquait l’imagerie seigneuriale en invitant ses lecteurs — dès la première page ! — à remonter jusqu’à « l’Europe médiévale des années 1200 » pour inscrire les opérations de la multinationale dans un « contexte féodal ». La comparaison n’est certes pas infondée. Mais elle souffre d’une erreur d’analyse majeure : tous les États ne se ressemblent pas. Et tous n’entretiennent pas les mêmes relations avec les géants de la technologie. Or il suffit d’examiner l’histoire d’ITT pour comprendre que la métamorphose d’un humble opérateur de lignes téléphoniques en mastodonte planétaire fut la conséquence directe de la domination militaire, financière et technologique exercée par un seul et même pays : jamais ITT — ni la Silicon Valley — n’aurait bénéficié d’une croissance aussi phénoménale sans le soutien inconditionnel des États-Unis. Les frères Hernán et Sosthenes Behn fondent ITT en 1920 à New York. À l’origine, l’entreprise leur sert de devanture pour gérer les installations téléphoniques qu’ils détiennent à Porto Rico et à Cuba. Nés à Saint-Thomas, dans les actuelles îles Vierges britanniques, les deux frères connaissent bien les Caraïbes et s’emploient à y attirer les capitaux américains. Les Behn possèdent une petite fortune familiale mais surtout une ambition dévorante. Avant de s’installer à Porto Rico, Sosthenes a travaillé quelques années à Wall Street, où il a noué avec JP Morgan et ce qui deviendrait plus tard la Citibank des liens qui s’avèrent fructueux. Au cours des années 1920, ITT se propage au Mexique, en Uruguay, au Brésil, au Chili, en Argentine et en Espagne. En 1929, elle contrôle les deux tiers des téléphones et la moitié des câbles en Amérique latine (5). Cette extension fulgurante repose sur l’endettement obtenu grâce aux connexions des Behn avec Wall Street. Elle coïncide avec l’effort des États-Unis, alors en pleine ascension comme puissance planétaire, pour évincer les intérêts britanniques d’Amérique latine. Comme le reconnaît l’ancien ministre de la guerre Elihu Root devant un comité du Congrès en 1921 : « Il y a une lutte à mort pour le contrôle des communications sud- américaines. » Sans surprise, les États-Unis la remportent, avec l’aide d’ITT. Selon un compte rendu fascinant publié en 1930, la compagnie des frères Behn « a fait davantage en neuf ans pour briser le monopole britannique sur les communications mondiales que tous les autres groupes et gouvernements réunis durant un demi-siècle (6) ». Ceux qui, plus tard, interpréteront le « i » d’ITT comme l’initiale d’« impérialisme » n’auront pas totalement tort. Dans l’ensemble, la guerre de conquête se déroula sans accroc. Pour s’attirer les faveurs de Washington, de nombreux pays sud-américains déroulèrent le tapis rouge à ITT, l’exemptant même des engagements coûteux généralement demandés aux opérateurs étrangers : investir dans les infrastructures ou éviter toute hausse unilatérale des tarifs. C’est seulement durant la seconde guerre mondiale que les liens entre ITT et Washington commencent à inquiéter certains gouvernements. Le premier souci concerne la sécurité des communications. L’autre tient à la montée du nationalisme économique. Ses représentants les plus fervents, comme Juan Perón en Argentine ou Francisco Franco en Espagne, congédient ITT, non sans lui verser un dédommagement confortable. Devenue entre-temps un fournisseur important de la défense américaine, la multinationale sait que ses jours comme opérateur de lignes téléphoniques sont comptés. Mais elle entend bien céder ses actifs au meilleur prix. En attendant une offre intéressante, ITT presse la poule aux œufs d’or, fait grimper ses tarifs et bloque les investissements. De sorte que le service devient à la fois plus médiocre et plus coûteux. Les populations locales enragent, mais ITT paraît intouchable. Qui oserait nationaliser une entreprise américaine aussi puissante ? Un homme a cette audace. Au début des années 1950, un jeune avocat cubain traîne le groupe devant un tribunal, l’accusant d’avoir trahi ses engagements. Son cabinet gagne le procès, mais le dictateur qui tient alors les rênes de Cuba, Fulgencio Batista, ignore le jugement du tribunal. Le jeune avocat s’appelle Fidel Castro. Il n’oubliera jamais cette humiliation : la filiale cubaine d’ITT sera l’une des premières sociétés étrangères nationalisées au lendemain de la révolution castriste de 1959. Le geste sonnera comme une gifle pour ITT — et comme un présage. Lorsqu’en 1962 le gouverneur d’un État brésilien prend le contrôle d’une de ses filiales locales, la compagnie mobilise ses liens avec Washington contre ce qu’elle présente comme un épisode de la guerre froide — un thème qui refera surface deux ans plus tard à la faveur du coup d’État militaire. Sa campagne de lobbying se révèle fructueuse, puisque le Brésil souffre l’humiliation de devoir payer une compensation exorbitante pour la filiale nationalisée. À la fin des années 1960, l’empire ITT réinvestit les énormes profits tirés de la revente de ses biens en Amérique latine dans des acquisitions de toutes sortes — compagnies d’assurances, hôtels, et même une société de location de voitures. La plupart sont domiciliées sur place et ne courent aucun risque de nationalisation. Au tournant de 1970, les seuls réseaux téléphoniques encore aux mains d’ITT se situent à Porto Rico, base arrière historique de la compagnie, ainsi qu’au Chili, où elle s’était installée en 1927. Les engagements d’ITT auprès de l’État chilien brillent par leur imprécision, en vertu d’un contrat exceptionnellement avantageux pour la compagnie (7). Dans les années 1960, le gouvernement d’Eduardo Frei, un chrétien-démocrate élu en 1964, tente de régler le problème sans faire de vagues, grâce à un plan prévoyant de racheter petit à petit les parts de la filiale locale d’ITT. Mais, pour les opposants de Frei, c’est à la fois trop peu et trop. Le socialiste Allende remporte l’élection présidentielle de 1970 en promettant de nationaliser ITT, d’y remplacer les managers par des ingénieurs et d’étendre le réseau téléphonique dans les zones les plus pauvres du pays. ITT craignait une présidence Allende bien avant 1970. Six ans plus tôt, déjà, l’un des membres de son conseil d’administration, l’ex-directeur de la CIA John McCone, avait pesé de tout son poids pour empêcher l’élection du socialiste chilien. Quelques mois avant le scrutin de 1970, ITT se met en relation avec la CIA et lui propose de l’argent pour faire obstacle à une possible victoire de la gauche. La CIA refuse, n’étant jamais à court de liquidités, ce qui ne décourage pas la compagnie d’arroser copieusement les opposants d’Allende. Après la victoire-surprise de ce dernier, c’est la CIA qui prend langue avec ITT. La compagnie ne pourrait-elle pas mettre l’État chilien sous pression, en refusant par exemple de fournir des pièces détachées ou du personnel de maintenance ? L’objectif de l’Agence consistait, selon les mots de Richard Nixon, à « faire crier l’économie chilienne » pour inciter les militaires à sortir de leurs casernes avant même qu’Allende ait le temps d’inaugurer son mandat. Entre espionnage et finance Cette stratégie tourne court. Une fois au pouvoir, Allende préfère négocier avec la compagnie plutôt que de la nationaliser sur-le-champ, alors que sa base — dont les syndicats de travailleurs d’ITT — réclame des mesures plus radicales. Comble de l’ingénuité, il demande même à l’entreprise de détecter d’éventuels micros au palais présidentiel... En septembre 1971, Allende se ravise et prend le contrôle de la filiale chilienne d’ITT, dont les dirigeants sont arrêtés pour avoir siphonné des profits indus via des sociétés fictives. En retour, la multinationale lance une virulente campagne à Washington. Ayant ses entrées chez le secrétaire d’État Henry Kissinger, elle lui suggère dix-huit mesures à prendre pour déstabiliser le président chilien dans un délai de six mois. Et continue par ailleurs d’encourager la CIA à financer El Mercurio, le principal journal de l’opposition. Au sein même de la compagnie, certains commencent à s’interroger. La presse publie des communications entre sa direction et des membres de l’administration Nixon, poussant le Sénat à mener des auditions pour clarifier l’influence d’ITT sur la politique étrangère américaine (8). Mais l’enquête ne parvient pas à mettre en cause les responsables et nul n’est condamné. Trois mois après, Allende perd la vie dans le coup d’État de Pinochet. Pour ITT, la nationalisation ne fut pas un choc trop rude : peu de temps après le coup d’État, la compagnie reçut 125 millions de dollars de Pinochet en guise d’indemnités, ainsi que 30 millions de la part de l’administration Nixon. En dépit — ou, peut-être, en raison — du rapport non concluant du Sénat américain, les soupçons au sujet du rôle d’ITT au Chili ne cessèrent de croître. Il n’était donc pas illogique que la multinationale représentât une cible toute trouvée pour de nombreux militants. L’inconnu qui avertit le New York Times de la présence d’une bombe au siège d’ITT se revendiquait du Weather Underground, une organisation clandestine d’extrême gauche. Au bout du compte, cette publicité négative indisposa même Porto Rico, foyer historique de la compagnie : en 1974, le territoire décida de racheter la filiale. La compensation massive qui lui fut accordée ne calma pas les esprits : son siège sauta quelques mois après la transaction. Au cours de la majeure partie de son existence, ITT fut le laboratoire d’un modèle d’expansion appelé à faire école, fondé sur les liens avec Wall Street et le Pentagone. Elle fut également pionnière de la mondialisation avec sa vision d’emblée globale et sa maîtrise du conglomérat — même si les synergies entre les filiales les plus hétéroclites relevaient surtout d’astuces comptables. De plus en plus obsédés par les profits à court terme et le cours des actions, ses dirigeants négligèrent les investissements à long terme dans ses services-clés. Là encore, elle fut en avance sur son temps : la plupart des autres compagnies américaines ne succomberaient à pareille tentation qu’à partir des années 1980. ITT, elle, embrassa la financiarisation dès le milieu des années 1960. À l’époque, il pouvait paraître étonnant qu’un manutentionnaire du téléphone travaillant pour la défense préférât racheter des compagnies d’assurances plutôt que d’investir dans la recherche et le développement. Encouragés par leurs amis de la banque Lazard, ses dirigeants réussirent à convaincre Wall Street que leur gloutonnerie s’inscrivait dans une ingénieuse stratégie de diversification. Mais son désir de croissance exponentielle marqua également le début de la fin : elle ne vit pas l’intérêt des recherches longues et coûteuses qui commençaient à fleurir dans la Silicon Valley. Le coup d’État au Chili abîma son image de manière irréversible pour les décennies à venir. Paradoxalement, la proximité d’ITT avec l’État américain et Wall Street — à laquelle elle dut sa prodigieuse croissance initiale — causa son déclin. De cette erreur, les actuels géants de la Silicon Valley, pareillement pris en étau entre espionnage et finance, ne paraissent pas avoir tiré toutes les leçons. EVGENY MOROZOV Auteur de The Santiago Boys, [https://the-santiago-boys.com/] une série de podcasts en neuf épisodes basée sur plus de deux cents entretiens, produite par Chora Media et Post-Utopia, et dont s’inspire cet article. Il 23/07/23 10:25, nexa-request(a)server-nexa.polito.it ha scritto: > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 23 Jul 2023 08:51:56 +0200 > From: "J.C. DE MARTIN"<juancarlos.demartin(a)polito.it> > To: Nexa<nexa(a)server-nexa.polito.it> > Subject: [nexa] The Santiago Boys > Message-ID:<d068ff21-3db5-8f84-c7c0-08985a7484ef(a)polito.it> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > E' online The Santiago Boys, il nuovo, splendido progetto di Evgeny Morozov: > https://the-santiago-boys.com/ > Qui sotto l'annuncio. > > Inoltre, oggi "La Lettura" dedica quattro pagine (!) all'iniziativa e > ieri il Guardian da pubblicato un articolo a uno dei protagonisti della > storia, Stafford Beer: > https://www.theguardian.com/world/2023/jul/22/stafford-beer-chile-allende-t… > > Bravo, Evgeny! > > juan carlos > / > > //The podcast that took 2+ years and 200+ interviews to produce is > finally online!// > // > //You can listen to it on the main podcasting platforms (including > Spotify and Apple Podcasts). // > // > //The website of the Santiago Boys offers plenty of extra materials for > those of you who want to dig deeper: footnotes, backgrounders, sources, > videos, a glossary, and so much else. // > // > //And we are also publishing the interviews with the many people we > interviewed (check out, for example, this interview with Brian Eno where > he talks about Stafford Beer and his own fascination with cybernetics). // > // > //Many thanks to dozens of people who worked on this ambitious project; > you can see all their names here. // > // > //I hope you take a break from the Barbenheimer hyper and spend some > time with the Santiago Boys instead! // > // > //Evgeny Morozov/ > -------------- next part -------------- > An HTML attachment was scrubbed... > URL:<http://server-nexa.polito.it/pipermail/nexa/attachments/20230723/7c590db9/a…> > > ------------------------------ > > Message: 2 > Date: Sun, 23 Jul 2023 10:07:13 +0200 > From: "Federico Guerrini"<federico.guerrini(a)hushmail.com> > To:nexa@server-nexa.polito.it > Subject: [nexa] Artificial Intelligence: A New Frontier for > Surveillance Capitalism? > Message-ID:<20230723080713.3D66B80E2BB(a)smtp.hushmail.com> > Content-Type: text/plain; charset="utf-8" > > "If you ask Alexa, Amazon’s voice assistant AI system, whether > Amazon is a monopoly, it responds by saying it doesn’t know. It > doesn’t take much to make it lambaste the other tech giants, but > it’s silent about its own corporate parent’s misdeeds. When Alexa > responds in this way, it’s obvious that it is putting its > developer’s interests ahead of yours. Usually, though, it’s not so > obvious whom an AI system is serving. To avoid being exploited by > these systems, people will need to learn to approach AI skeptically. > (...) As a security expert and data scientist, we believe that people > who come to rely on these AIs will have to trust them implicitly to > navigate daily life. That means they will need to be sure the AIs > aren’t secretly working for someone else. Across the internet, > devices and services that seem to work for you already secretly work > against you. Smart TVs spy on you. Phone apps collect and sell your > data. Many apps and websites manipulate you through dark patterns, > design elements that deliberately mislead, coerce or deceive website > visitors. This is surveillance capitalism, and AI is shaping up to be > part of it." > "Imagine asking your chatbot to plan your next vacation. Did it choose > a particular airline or hotel chain or restaurant because it was the > best for you or because its maker got a kickback from the businesses? > As with paid results in Google search, newsfeed ads on Facebook and > paid placements on Amazon queries, these paid influences are likely > to get more surreptitious over time. > > If you’re asking your chatbot for political information, are the > results skewed by the politics of the corporation that owns the > chatbot? Or the candidate who paid it the most money? Or even the > views of the demographic of the people whose data was used in > training the model? Is your AI agent secretly a double agent? Right > now, there is no way to know." > https://theconversation.com/can-you-trust-ai-heres-why-you-shouldnt-209283 > > Ciao, > > Federico > -------------- next part -------------- > An HTML attachment was scrubbed... > URL:<http://server-nexa.polito.it/pipermail/nexa/attachments/20230723/945774f2/a…> > > ------------------------------ > > Message: 3 > Date: Sun, 23 Jul 2023 10:24:58 +0200 > From: "J.C. DE MARTIN"<juancarlos.demartin(a)polito.it> > To: Nexa<nexa(a)server-nexa.polito.it> > Subject: [nexa] RFC "Reflections on Ten Years Past the Snowden > Revelations" > Message-ID:<f2d5a90c-b839-e654-20b7-369919bf9ff9(a)polito.it> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > RFC 9446 > *Reflections on Ten Years Past the Snowden Revelations** > * > > Stream: > Independent Submission > RFC: > 9446 > Category: > Informational > Published: > July 2023 > ISSN: > 2070-1721 > Authors: > S. Farrell > Trinity College, Dublin > F. Badii > Digital Medusa > B. Schneier > Harvard University > S. M. Bellovin > Columbia University > > *Abstract** > * > This memo contains the thoughts and recountings of events that > transpired during and after the release of information about the United > States National Security Agency (NSA) by Edward Snowden in 2013. There > are four perspectives: that of someone who was involved with sifting > through the information to responsibly inform the public, that of a > security area director of the IETF, that of a human rights expert, and > that of a computer science and affiliate law professor. The purpose of > this memo is to provide some historical perspective, while at the same > time offering a view as to what security and privacy challenges the > technical community should consider. These essays do not represent a > consensus view, but that of the individual authors. > > https://www.rfc-editor.org/rfc/rfc9446.html > -------------- next part -------------- > An HTML attachment was scrubbed... > URL:<http://server-nexa.polito.it/pipermail/nexa/attachments/20230723/1c545016/a…> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > nexa mailing list > nexa(a)server-nexa.polito.it > https://server-nexa.polito.it/cgi-bin/mailman/listinfo/nexa > > > ------------------------------ > > End of nexa Digest, Vol 171, Issue 63 > ************************************* ------------------------------------------------------------------------ quanti nella loro vita / si fecero custodi delle Termopili, / sono degni di più grande onore / se prevedono (e molti lo prevedono) / che all’ultimo comparirà un Efialte / e comunque i Persiani passeranno Kostantinos Kavafis, Termopili ------------------------------------------------------------------------ Maurizio Lana Università del Piemonte Orientale Dipartimento di Studi Umanistici Piazza Roma 36 - 13100 Vercelli

1 0

Never mind the delivery..
by Luigi Scorca July 26, 2023

July 26, 2023

https://www.wsj.com/articles/never-mind-the-delivery-more-online-consumers-…

1 0

Traduzione inglese dell'articolo "L’esperienza del Centro Nexa su internet e società" | A&RT - Atti e Rassegna Tecnica | Di Juan Carlos De Martin e Marco Ricolfi
by Nexa Media July 25, 2023

July 25, 2023

Gentilissime, gentilissimi, Vi segnaliamo che è disponibile online (open access) la *traduzione inglese* dell'articolo "L’esperienza del Centro Nexa su internet e società" <http://art.siat.torino.it/wp-content/uploads/2022/12/A_RT_LXXVI-demartin-ri…>, scritto dai co-fondatori e co-direttori del Centro Nexa *Juan Carlos De Martin* e *Marco Ricolfi*, e pubblicato all'interno del numero speciale LXXVI, 1-2-3 - Cultura tecnica e cultura umanistica. Il caso torinese <http://art.siat.torino.it/lxxvi/> della rivista*A&RT - Atti e Rassegna Tecnica*, la rivista della Società degli Ingegnerie degli Architetti in Torino (SIAT). Leggi la traduzione integrale e aggiornata dell'articolo al seguente link: https://nexa.polito.it/the-first-15-years-of-the-Nexa-Center-for-Internet-a… Grazie per l'attenzione, Cordiali saluti, <https://nexa.polito.it/articolo-esperienza-centronexa-demartin-ricolfi> -- Anita Botta Communication Manager Nexa Center for Internet & Society Politecnico di Torino – DAUIN Corso Duca degli Abruzzi, 24 - 10129 Torino web: https://nexa.polito.it/ mail: anita.botta(a)polito.it tel: 011 090 7219

1 0

Re: [nexa] “Emergence” isn’t an explanation, it’s a prayer. A critique of Emergentism in Artificial Intelligence
by Giuseppe Attardi July 25, 2023

July 25, 2023

Nei LLM, il concetto di "emergent ability” ha una definizione precisa: An ability that is “not present in small models but is present in large models.” https://www.jasonwei.net/blog/emergence Non mi pare che in questo contesto sia mai stata considerato di: "use it as a hypothesis to predict the outcome of a complex, unknown system, with the hope that a desired property will emerge;” Che il comportamento sorprendente e non facilmente spiegabile dei LLM, vada oltre le capacità per cui sono stati allenati, è un fatto appurato anche se controverso. Ne ho parlato con Giorgio Parisi, che sul tema dei sistemi complessi ha vinto il premio Nobel, e ha concordato con me che il fenomeno possa essere appunto spiegato come l’applicazione su larga scala di una semplice legge di probabilità: in questo caso la probabilità delle prossima parola in una sequenza. — Beppe > On 19 Jul 2023, at 10:37, nexa-request(a)server-nexa.polito.it wrote: > > Date: Wed, 19 Jul 2023 14:36:59 +0000 > From: Daniela Tafani <daniela.tafani(a)unipi.it <mailto:daniela.tafani@unipi.it>> > To: "nexa(a)server-nexa.polito.it <mailto:nexa@server-nexa.polito.it>" <nexa(a)server-nexa.polito.it <mailto:nexa@server-nexa.polito.it>> > Subject: [nexa] “Emergence” isn’t an explanation, it’s a > prayer. A critique of Emergentism in Artificial Intelligence > Message-ID: <8aced3bd15f24f548084f82dbf280719(a)unipi.it <mailto:8aced3bd15f24f548084f82dbf280719@unipi.it>> > Content-Type: text/plain; charset="windows-1252" > > “Emergence” isn’t an explanation, it’s a prayer > A critique of Emergentism in Artificial Intelligence > <https://ykulbashian.medium.com/?source=post_page-----ef239d3687bf----------…> > [cardboarddreams] > > Emergence is the notion that in a complex system, the interactions of the whole may exhibit properties that are not present in the individual parts. It is most often applied to examples in physics and nature, such as the collective behaviours of ant colonies, the self-organizing principles of social groups, or the macro properties of molecules. In the last few decades, it has given rise to emergentist perspectives of human cognition<https://onlinelibrary.wiley.com/doi/10.1111/j.1756-8765.2010.01116.x>, and even of consciousness<https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597170/>. These are based on the recognition that the complexities and mysteries of the human mind, being a part of nature, may be characterized as emergent phenomena. > > This approach has an intuitive appeal. It is supported by the superficial facts: the brain — the source of intelligence and consciousness— is most certainly a complex of interconnected neurons. Emergentist interpretations of human behaviour may also boast some recent wins—the proliferation of LLMs (e.g. ChatGPT) may be seen as one such success. This has reignited the discussion of whether emergence is the best way to frame intelligence. > > You may have noticed that the last paragraph switched between two subtly different uses of “emergence”. The first use was to describe an observed emergent property; consciousness, we have seen it, likely emerges out of neuronal interactions. The second was to use it as a hypothesis to predict the outcome of a complex, unknown system, with the hope that a desired property will emerge; e.g. intelligence will arise from the interactions of artificial neurons at scale. The latter example is the driving motivation behind the multi-million dollar Human Brain Project<https://en.wikipedia.org/wiki/Human_Brain_Project>. The project justifies its cost by leaning on evidence from observed instances of emergence. Doing so paints emergence as a theory; that is, existing observations can be used to justify future predictions. > > But emergence is not a theory. Emergence can only be ascribed to a phenomenon in retrospect, once you already know what has “emerged”. The higher-level properties that emerge are qualitatively different from those at the lower-level — otherwise it wouldn’t be “emergence”. So by necessity they could not have been predicted from the lower-level ones. The properties of “intelligence” could not have been logically foreseen from the properties of neurons unless you had already observed that property emerge in a similar substrate. And even then it’s just a guess that is likely to be wrong given the complexity of the interactions involved; small differences can easily invalidate the hypothesis. In both cases emergence gives no new information: when explaining existing examples it gives you no new insights about the processes except that they happen; and when predicting unknown behaviours it gives very poor guarantees that anything you expect to happen will do so. > > Emergence is only really valid as a general metaphysical classification of certain phenomena. It’s a metaphysical category, like “cause”, “effect” or “change”. Using the word when explaining cognition is not wrong per se, it just has no real meaning or explanatory force. It’s like having a theory of “thing-happened-ness” — it’s correct, but void of content. Take, for example, the following quotes from a review article on emergence: > > This process gives rise to an emergent tendency to facilitate perception of items consistent with the patterns of English orthography, without explicitly representing this knowledge in a system of rules, as in other approaches. > > … > > However, such modes of thought themselves might be viewed as emergent consequences of a lifetime of thought-structuring practice supported by culture and education. > > Emergence in Cognitive Science, McClelland<https://onlinelibrary.wiley.com/doi/10.1111/j.1756-8765.2010.01116.x> > > If you removed the word “emergent” from the above two sentences, would anything important change? Indeed any sentence that includes “emergent” would give the same information if you removed it; “it gives rise to emergent properties” means the same as “it gives rise to properties”, or “there is an emergent tendency” is not substantially different from “there is a tendency”. > > Adding “emergent” to any sentence doesn’t increase its useful information content.¹ > > Emergence has no information that fundamentally differentiates it from a “miracle”. If I were to say that applying transformers<https://arxiv.org/abs/1706.03762> to Neural Networks creates intelligence through a miracle, I would be ridiculed. Were I to say that they create intelligence through emergent interactions, suddenly they gain an air of scientific credibility — but what have I added to the conversation with the use of that word? What quantifiable scientific facts are entailed in the term “emergent”? There are none. > > In cognitive science, emergence is regularly used to “explain” the connection between two phenomena, when it is otherwise complex and difficult to predict: e.g. how neuronal firing gives rise to consciousness, or transformers to the appearance of language comprehension. Where there may be a connection, but nothing more is known or can be proved, emergence is a placeholder that fills the gap. The word gives weight and gravitas to what is essentially a blank space. > > Despite emergence contributing nothing of substance to the discussion, as a concept it admittedly has a compelling intuitive appeal. There is a wonderful feeling about the notion of emergence. It does seem to be adding something valuable, as if you’ve discovered a magical ingredient by which you can explain mysterious phenomena. That’s the reason it continues to be popular, and gets inserted into scientific discussions. It convinces the listener that something has been explained with scientific rigour when all we’ve done is to say “it’s complicated”. > > Besides the good feeling, however, emergence is void of any explanatory power. And so it has no scientific value in a predictive capacity. You can’t use it to say anything about what an unknown system will do; only what you hope it will do. When applied to pie-in-the-sky AI futurism, emergence has become synonymous with “I’m sure the system will work itself out”. It indicates that the author has a feeling that a complex system will align at some point, but no clear sense of how, why, or when. Insofar as intelligence does manifest in a specific instance, “emergence” doesn’t tell us anything interesting about how it happened. And insofar as intelligence hasn’t yet manifested, emergence doesn’t tell us when it will or what direction to take to get there. > > In the field of AI development, emergence is invoked whenever someone encounters a phenomenon in the human mind and has no idea how to even start explaining it (e.g. art, socialization, empathy, transcendental aesthetics, DnD, etc). If said researcher already has a working theory of AI, this realization is disheartening. So they look deeper into the matter, find some point of overlap between the existing theory and the missing behaviour, and assume that with enough time and complexity the missing pieces will emerge. > > Emergence is attractive in such cases because it puts the author’s mind at ease, by making it seem like they have a viable mechanism that only needs more time to be vindicated. It placates their inner watchdog, the one that demands concrete, scientific explanations. Emergence, being related to complexity and superficially validated by experiments such as Conway’s Game of Life, is enough to lull that watchdog back to sleep. > > This justifies continuing to ignore any shortcomings in a theoretical model, and persisting on the current path. Like the proverbial man who searches for his lost keys under the lamplight, because that is where the light is, he hopes that with enough persistence his keys will “emerge”. The only other alternative is to admit failure, and to give up any hope of accomplishing what you want within this lifetime. > > Scientists, it seems, can have superstitions too. And emergence has a powerful narcotic effect: it feels so reasonable and credible on a gut level². There are many factors that prevent a given researcher from investigating emergence too deeply and realizing that it lacks any substance. First, there appears to be a lot of external evidence to back it up in the natural world. This, as was pointed out, equivocates between retrospective and prospective uses of the term, and so legitimate uses are being conscripted to justify the illegitimate ones. Secondly, the fact that emergence exclusively concerns itself with intractably complex systems means anything behind its curtain by definition can’t be studied. So it conveniently excludes itself from exactly that analysis which would reveal it to be hollow. > > In the end emergence isn’t an explanation; it’s an observation combined with a recognition of ignorance. Wherever emergence shows up there is an implicit acceptance that everyone involved is at a loss for how to approach the topic. It’s not that properties like intelligence won’t emerge from neural activity, it’s that emergence is a placeholder that justifies and promotes a lack of interest in exploring the details behind the connection. It discourages investigation. By invoking the term, we are merely thanking the nature gods for granting us this emergent property (aka property), and trying not to examine their gifts too profanely or with ingratitude. This impulse is understandable, since we don’t think we’ll discover an answer if we were to dig in. But we shouldn’t allow our insecurities to masquerade as science, or else they may become ingrained to the extent that they are difficult to uproot. A false answer stands in the way of a true one. > > ¹ This used to say ‘You can remove “emergent” from any sentence and it would mean the same thing’, but that has caused some confusion, so to clarify: the word “emergent” when used as an adjective doesn’t add new or useful information; you won’t know any more about the subject than you did before. > > ² A self-aware researcher should notice if they have a strong intuitive or emotional reason for holding on to the idea. If you ever feel that emergence is so self-evident that it can never be disproved, that should give you pause — perhaps you have strayed outside the bounds of scientific inquiry and into metaphysical expositions. Not that there’s anything wrong with the latter… > > > https://ykulbashian.medium.com/emergence-isnt-an-explanation-it-s-a-prayer-…

3 3

‘A certain danger lurks there’: how the inventor of the first chatbot turned against AI | Artificial intelligence (AI) | The Guardian
by Alberto Cammozzo July 25, 2023

July 25, 2023

<https://www.theguardian.com/technology/2023/jul/25/joseph-weizenbaum-invent…> In 1966, an MIT professor named Joseph Weizenbaum created the first chatbot. He cast it in the role of a psychotherapist. A user would type a message on an electric typewriter connected to a mainframe. After a moment, the “psychotherapist” would reply. User: Men are all alike. Chatbot: IN WHAT WAY User: They’re always bugging us about something or other. Chatbot: CAN YOU THINK OF A SPECIFIC EXAMPLE User: Well, my boyfriend made me come here. Chatbot: YOUR BOYFRIEND MADE YOU COME HERE User: He says I’m depressed much of the time. Chatbot: I AM SORRY TO HEAR YOU ARE DEPRESSED Weizenbaum published this sample exchange in a journal article that explained how the chatbot worked. The software was relatively simple. It looked at the user input and applied a set of rules to generate a plausible response. He called the program Eliza, after Eliza Doolittle in Pygmalion. The cockney flower girl in George Bernard Shaw’s play uses language to produce an illusion: she elevates her elocution to the point where she can pass for a duchess. Similarly, Eliza would speak in such a way as to produce the illusion that it understood the person sitting at the typewriter. “Some subjects have been very hard to convince that Eliza (with its present script) is not human,” Weizenbaum wrote. In a follow-up article that appeared the next year, he was more specific: one day, he said, his secretary requested some time with Eliza. After a few moments, she asked Weizenbaum to leave the room. “I believe this anecdote testifies to the success with which the program maintains the illusion of understanding,” he noted. Eliza isn’t exactly obscure. It caused a stir at the time – the Boston Globe sent a reporter to go and sit at the typewriter and ran an excerpt of the conversation – and remains one of the best known developments in the history of computing. More recently, the release of ChatGPT has renewed interest in it. In the last year, Eliza has been invoked in the Guardian, the New York Times, the Atlantic and elsewhere. The reason that people are still thinking about a piece of software that is nearly 60 years old has nothing to do with its technical aspects, which weren’t terribly sophisticated even by the standards of its time. Rather, Eliza illuminated a mechanism of the human mind that strongly affects how we relate to computers. Early in his career, Sigmund Freud noticed that his patients kept falling in love with him. It wasn’t because he was exceptionally charming or good-looking, he concluded. Instead, something more interesting was going on: transference. Briefly, transference refers to our tendency to project feelings about someone from our past on to someone in our present. While it is amplified by being in psychoanalysis, it is a feature of all relationships. When we interact with other people, we always bring a group of ghosts to the encounter. The residue of our earlier life, and above all our childhood, is the screen through which we see one another. This concept helps make sense of people’s reactions to Eliza. Weizenbaum had stumbled across the computerised version of transference, with people attributing understanding, empathy and other human characteristics to software. While he never used the term himself, he had a long history with psychoanalysis that clearly informed how he interpreted what would come to be called the “Eliza effect”. As computers have become more capable, the Eliza effect has only grown stronger. Take the way many people relate to ChatGPT. Inside the chatbot is a “large language model”, a mathematical system that is trained to predict the next string of characters, words, or sentences in a sequence. What distinguishes ChatGPT is not only the complexity of the large language model that underlies it, but its eerily conversational voice. As Colin Fraser, a data scientist at Meta, has put it, the application is “designed to trick you, to make you think you’re talking to someone who’s not actually there”. But the Eliza effect is far from the only reason to return to Weizenbaum. His experience with the software was the beginning of a remarkable journey. As an MIT professor with a prestigious career, he was, in his words, a “high priest, if not a bishop, in the cathedral to modern science”. But by the 1970s, Joseph Weizenbaum had become a heretic, publishing articles and books that condemned the worldview of his colleagues and warned of the dangers posed by their work. Artificial intelligence, he came to believe, was an “index of the insanity of our world.” Today, the view that artificial intelligence poses some kind of threat is no longer a minority position among those working on it. There are different opinions on which risks we should be most worried about, but many prominent researchers, from Timnit Gebru to Geoffrey Hinton – both ex-Google computer scientists – share the basic view that the technology can be toxic. Weizenbaum’s pessimism made him a lonely figure among computer scientists during the last three decades of his life; he would be less lonely in 2023. There is so much in Weizenbaum’s thinking that is urgently relevant now. Perhaps his most fundamental heresy was the belief that the computer revolution, which Weizenbaum not only lived through but centrally participated in, was actually a counter-revolution. It strengthened repressive power structures instead of upending them. It constricted rather than enlarged our humanity, prompting people to think of themselves as little more than machines. By ceding so many decisions to computers, he thought, we had created a world that was more unequal and less rational, in which the richness of human reason had been flattened into the senseless routines of code. Weizenbaum liked to say that every person is the product of a particular history. His ideas bear the imprint of his own particular history, which was shaped above all by the atrocities of the 20th century and the demands of his personal demons. Computers came naturally to him. The hard part, he said, was life. What it means to be human – and how a human is different from a computer – was something Weizenbaum spent a lot of time thinking about. So it’s fitting that his own humanity was up for debate from the start. His mother had a difficult labour, and felt some disappointment at the result. “When she was finally shown me, she thought I was a bloody mess and hardly looked human,” Weizenbaum later recalled. “She couldn’t believe this was supposed to be her child.” He was born in 1923, the youngest son of an assimilated, upper-middle class Jewish family in Berlin. His father, Jechiel, who had emigrated to Germany from Galicia, which spanned what is now south-eastern Poland and western Ukraine, at the age of 12, was an accomplished furrier who had acquired a comfortable foothold in society, a nice apartment, and a much younger Viennese wife (Weizenbaum’s mother). From the start, Jechiel treated his son with a contempt that would haunt Weizenbaum for the rest of his life. “My father was absolutely convinced that I was a worthless moron, a complete fool, that I would never become anything,” Weizenbaum later told the documentary film-makers Peter Haas and Silvia Holzinger. By the time he was old enough to make memories, the Nazis were everywhere. His family lived near a bar frequented by Hitler’s paramilitaries, the SA, and sometimes he would see people getting dragged inside to be beaten up in the backroom. Once, while he was out with his nanny, columns of armed communists and Nazis lined up and started shooting at each other. The nanny pushed him under a parked car until the bullets stopped flying. Shortly after Hitler became chancellor in 1933, the government passed a law that severely restricted the number of Jews in public schools. Weizenbaum had to transfer to a Jewish boys’ school. It was here that he first came into contact with the Ostjuden: Jews from eastern Europe, poor, dressed in rags, speaking Yiddish. To Weizenbaum, they may as well have come from Mars. Nevertheless, the time he spent with them gave him what he later described as “a new feeling of camaraderie”, as well as a “sensitivity for oppression”. He became deeply attached to one of his classmates in particular. “If fate had been different, I would have developed a homosexual love for this boy,” he later said. The boy “led me into his world”, the world of the Jewish ghetto around Berlin’s Grenadierstrasse. “They had nothing, owned nothing, but somehow supported each other,” he recalled. One day, he brought the boy back to his family’s apartment. His father, himself once a poor Jewish boy from eastern Europe, was disgusted and furious. Jechiel was very proud, Weizenbaum remembered – and he had reason to be, given the literal and figurative distances he had travelled from the shtetl. Now his son was bringing the shtetl back into his home. Alienated from his parents, richer than his classmates, and a Jew in Nazi Germany: Weizenbaum felt comfortable nowhere. His instinct, he said, was always to “bite the hand that fed me”, to provoke the paternal figure, to be a pain in the backside. And this instinct presumably proceeded from the lesson he learned from his father’s hostility toward him and bigotry toward the boy he loved: that danger could lie within one’s home, people, tribe. In 1936, the family left Germany suddenly, possibly because Jechiel had slept with the girlfriend of an SA member. Weizenbaum’s aunt owned a bakery in Detroit, so that’s where they went. At 13, he found himself 4,000 miles from everything he knew. “I was very, very lonely,” he recalled. School became a refuge from reality – specifically algebra, which didn’t require English, which he didn’t speak at first. “Of all the things that one could study,” he later said, “mathematics seemed by far the easiest. Mathematics is a game. It is entirely abstract.” In his school’s metalworking class, he learned to operate a lathe. The experience brought him out of his brain and into his body. About 70 years later, he looked back on the realisation prompted by this new skill: that intelligence “isn’t just in the head but also in the arm, in the wrist, in the hand”. Thus, at a young age, two concepts were in place that would later steer his career as a practitioner and critic of AI: on the one hand, an appreciation for the pleasures of abstraction; on the other, a suspicion of those pleasures as escapist, and a related understanding that human intelligence exists in the whole person and not in any one part. In 1941, Weizenbaum enrolled at the local public university. Wayne University was a working-class place: cheap to attend, filled with students holding down full-time jobs. The seeds of social consciousness that had been planted in Berlin started to grow: Weizenbaum saw parallels between the oppression of Black people in Detroit and that of the Jews under Hitler. This was also a time of incandescent class struggle in the city – the United Auto Workers union won its first contract with Ford the same year that Weizenbaum entered college. Weizenbaum’s growing leftwing political commitments complicated his love of mathematics. “I wanted to do something for the world or society,” he remembered. “To study plain mathematics, as if the world were doing fine, or even didn’t exist at all – that’s not what I wanted.” He soon had his chance. In 1941, the US entered the second world war; the following year, Weizenbaum was drafted. He spent the next five years working as a meteorologist for the Army Air corps, stationed on different bases across the US. The military was a “salvation”, he later said. What fun, to get free of his family and fight Hitler at the same time. While home on furlough, he began a romance with Selma Goode, a Jewish civil rights activist and early member of the Democratic Socialists of America. Before long they were married, with a baby boy, and after the war Weizenbaum moved back to Detroit. There, he resumed his studies at Wayne, now financed by the federal government through the GI Bill. Then, in the late 1940s, the couple got divorced, with Goode taking custody of their son. “That was incredibly tragic for me,” Weizenbaum later said. “It took me a long time to get over it.” His mental state was forever unsteady: his daughter Pm – pronounced “Pim” and named after the New York leftwing daily newspaper PM – told me that he had been hospitalised for anorexia during his time at university. Everything he did, he felt he did badly. In the army he was promoted to sergeant and honourably discharged; nonetheless, he left convinced that he had somehow hindered the war effort. He later attributed his self-doubt to his father constantly telling him he was worthless. “If something like that is repeated to you as a child, you end up believing it yourself,” he reflected. In the wake of the personal crisis produced by Selma’s departure came two consequential first encounters. He went into psychoanalysis and he went into computing. Eniac, one of the world’s first electronic digital computers, circa 1945. Photograph: Corbis/Getty In those days, a computer, like a psyche, was an interior. “You didn’t go to the computer,” Weizenbaum said in a 2010 documentary. “Instead, you went inside of it.” The war had provided the impetus for building gigantic machines that could mechanise the hard work of mathematical calculation. Computers helped crack Nazi encryption and find the best angles for aiming artillery. The postwar consolidation of the military-industrial complex, in the early days of the cold war, drew large sums of US government money into developing the technology. By the late 1940s, the fundamentals of the modern computer were in place. But it still wasn’t easy to get one. So one of Weizenbaum’s professors resolved to build his own. He assembled a small team of students and invited Weizenbaum to join. Constructing the computer, Weizenbaum grew happy and purposeful. “I was full of life and enthusiastic about my work,” he remembered. Here were the forces of abstraction that he first encountered in middle-school algebra. Like algebra, a computer modelled, and thereby simplified, reality – yet it could do so with such fidelity that one could easily forget that it was only a representation. Software also imparted a sense of mastery. “The programmer has a kind of power over a stage incomparably larger than that of a theatre director,” he later said in the 2007 documentary Rebel at Work. “Bigger than that of Shakespeare.” About this time, Weizenbaum met a schoolteacher named Ruth Manes. In 1952, they married and moved into a small apartment near the university. She “couldn’t have been further from him culturally”, their daughter Miriam told me. She wasn’t a Jewish socialist like his first wife – her family was from the deep south. Their marriage represented “a reach for normalcy and a settled life” on his part, Miriam said. His political passions cooled. By the early 1960s, Weizenbaum was working as a programmer for General Electric in Silicon Valley. He and Ruth were raising three daughters and would soon have a fourth. At GE, he built a computer for the Navy that launched missiles and a computer for Bank of America that processed cheques. “It never occurred to me at the time that I was cooperating in a technological venture which had certain social side effects which I might come to regret,” he later said. In 1963, the prestigious Massachusetts Institute of Technology called. Would he like to join the faculty as a visiting associate professor? “That was like offering a young boy the chance to work in a toy factory that makes toy trains,” Weizenbaum remembered. The computer that Weizenbaum had helped build in Detroit was an ogre, occupying an entire lecture hall and exhaling enough heat to keep the library warm in winter. Interacting with it involved a set of highly structured rituals: you wrote out a program by hand, encoded it as a pattern of holes on punch cards, and then ran the cards through the computer. This was standard operating procedure in the technology’s early days, making programming fiddly and laborious. MIT’s computer scientists sought an alternative. In 1963, with a $2.2m grant from the Pentagon, the university launched Project MAC – an acronym with many meanings, including “machine-aided cognition”. The plan was to create a computer system that was more accessible and responsible to individual needs. To that end, the computer scientists perfected a technology called “time-sharing”, which enabled the kind of computing we take for granted today. Rather than loading up a pile of punch cards and returning the next day to see the result, you could type in a command and get an immediate response. Moreover, multiple people could use a single mainframe simultaneously from individual terminals, which made the machines seem more personal. With time-sharing came a new type of software. The programs that ran on MIT’s system included those for sending messages from one user to another (a precursor of email), editing text (early word processing) and searching a database with 15,000 journal articles (a primitive JSTOR). Time-sharing also changed how people wrote programs. The technology made it possible “to interact with the computer conversationally,” Weizenbaum later recalled. Software development could unfold as a dialogue between programmer and machine: you try a bit of code, see what comes back, then try a little more. Weizenbaum wanted to go further. What if you could converse with a computer in a so-called natural language, like English? This was the question that guided the creation of Eliza, the success of which made his name at the university and helped him secure tenure in 1967. It also brought Weizenbaum into the orbit of MIT’s Artificial Intelligence Project, which had been set up in 1958 by John McCarthy and Marvin Minsky. McCarthy had coined the phrase “artificial intelligence” a few years earlier when he needed a title for an academic workshop. The phrase was neutral enough to avoid overlap with existing areas of research like cybernetics, amorphous enough to attract cross-disciplinary contributions, and audacious enough to convey his radicalism (or, if you like, arrogance) about what machines were capable of. This radicalism was affirmed in the original workshop proposal. “Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it,” it asserted. Marvin Minsky in the early 1980s. Photograph: RGB Ventures/SuperStock/Alamy Minsky was bullish and provocative; one of his favourite gambits was to declare the human brain nothing but a “meat machine” whose functions could be reproduced, or even surpassed, by human-made machines. Weizenbaum disliked him from the start. It wasn’t his faith in the capabilities of technology that bothered Weizenbaum; he himself had seen computers progress immensely by the mid-1960s. Rather, Weizenbaum’s trouble with Minsky, and with the AI community as a whole, came down to a fundamental disagreement about the nature of the human condition. In Weizenbaum’s 1967 follow-up to his first article about Eliza, he argued that no computer could ever fully understand a human being. Then he went one step further: no human being could ever fully understand another human being. Everyone is formed by a unique collection of life experiences that we carry around with us, he argued, and this inheritance places limits on our ability to comprehend one another. We can use language to communicate, but the same words conjure different associations for different people – and some things can’t be communicated at all. “There is an ultimate privacy about each of us that absolutely precludes full communication of any of our ideas to the universe outside ourselves,” Weizenbaum wrote. This was a very different perspective than that of Minsky or McCarthy. It clearly bore the influence of psychoanalysis. Here was the mind not as a meat machine but as a psyche – something with depth and strangeness. If we are often opaque to one another and even to ourselves, what hope is there for a computer to know us? Yet, as Eliza illustrated, it was surprisingly easy to trick people into feeling that a computer did know them – and into seeing that computer as human. Even in his original 1966 article, Weizenbaum had worried about the consequences of this phenomenon, warning that it might lead people to regard computers as possessing powers of “judgment” that are “deserving of credibility”. “A certain danger lurks there,” he wrote. In the mid-1960s, this was as far as he was willing to go. He pointed to a danger, but didn’t dwell on it. He was, after all, a depressed kid who had escaped the Holocaust, who always felt like an impostor, but who had found prestige and self-worth in the high temple of technology. It can be hard to admit that something you are good at, something you enjoy, is bad for the world – and even harder to act on that knowledge. For Weizenbaum, it would take a war to know what to do next. On 4 March 1969, MIT students staged a one-day “research stoppage” to protest the Vietnam war and their university’s role in it. People braved the snow and cold to pile into Kresge Auditorium in the heart of campus for a series of talks and panels that had begun the night before. Noam Chomsky spoke, as did the anti-war senator George McGovern. Student activism had been growing at MIT, but this was the largest demonstration to date, and it received extensive coverage in the national press. “The feeling in 1969 was that scientists were complicit in a great evil, and the thrust of 4 March was how to change it,” one of the lead organisers later wrote. Weizenbaum supported the action and became strongly affected by the political dynamism of the time. “It wasn’t until the merger of the civil rights movement, the war in Vietnam, and MIT’s role in weapons development that I became critical,” he later explained in an interview. “And once I started thinking along those lines, I couldn’t stop.” In the last years of his life, he would reflect on his politicisation during the 1960s as a return to the social consciousness of his leftist days in Detroit and his experiences in Nazi Germany: “I stayed true to who I was,” he told the German writer Gunna Wendt. He began to think about the German scientists who had lent their expertise to the Nazi regime. “I had to ask myself: do I want to play that kind of role?” he remembered in 1995. He had two choices. One was to “push all this sort of thinking down”, to repress it. The other was “to look at it seriously”. Looking at it seriously would require examining the close ties between his field and the war machine that was then dropping napalm on Vietnamese children. Defense Secretary Robert McNamara championed the computer as part of his crusade to bring a mathematical mindset to the Pentagon. Data, sourced from the field and analysed with software, helped military planners decide where to put troops and where to drop bombs. A protest against the Vietnam war at the Massachusetts Institute of Technology in November 1969. Photograph: Boston Globe/Getty Images By 1969, MIT was receiving more money from the Pentagon than any other university in the country. Its labs pursued a number of projects designed for Vietnam, such as a system to stabilise helicopters in order to make it easier for a machine-gunner to obliterate targets in the jungle below. Project MAC – under whose auspices Weizenbaum had created Eliza – had been funded since its inception by the Pentagon. As Weizenbaum wrestled with this complicity, he found that his colleagues, for the most part, didn’t care about the purposes to which their research might be put. If we don’t do it, they told him, somebody else will. Or: scientists don’t make policy, leave that to the politicians. Weizenbaum was again reminded of the scientists in Nazi Germany who insisted that their work had nothing to do with politics. Consumed by a sense of responsibility, Weizenbaum dedicated himself to the anti-war movement. “He got so radicalised that he didn’t really do much computer research at that point,” his daughter Pm told me. Instead, he joined street demonstrations and met anti-war students. Where possible, he used his status at MIT to undermine the university’s opposition to student activism. After students occupied the president’s office in 1970, Weizenbaum served on the disciplinary committee. According to his daughter Miriam, he insisted on a strict adherence to due process, thereby dragging out the proceedings as long as possible so that students could graduate with their degrees. It was during this period that certain unresolved questions about Eliza began to bother him more acutely. Why had people reacted so enthusiastically and so delusionally to the chatbot, especially those experts who should know better? Some psychiatrists had hailed Eliza as the first step toward automated psychotherapy; some computer scientists had celebrated it as a solution to the problem of writing software that understood language. Weizenbaum became convinced that these responses were “symptomatic of deeper problems” – problems that were linked in some way to the war in Vietnam. And if he wasn’t able to figure out what they were, he wouldn’t be able to keep going professionally. In 1976, Weizenbaum published his magnum opus: Computer Power and Human Reason: From Judgment to Calculation. “The book has overwhelmed me, like being crashed over by the sea,” read a blurb from the libertarian activist Karl Hess. The book is indeed overwhelming. It is a chaotic barrage of often brilliant thoughts about computers. A glimpse at the index reveals the range of Weizenbaum’s interlocutors: not only colleagues like Minsky and McCarthy but the political philosopher Hannah Arendt, the critical theorist Max Horkheimer, and the experimental playwright Eugène Ionesco. He had begun work on the book after completing a fellowship at Stanford University, in California, where he enjoyed no responsibilities, a big office and lots of stimulating discussions with literary critics, philosophers and psychiatrists. With Computer Power and Human Reason, he wasn’t so much renouncing computer science as trying to break it open and let alternative traditions come pouring in. The book has two major arguments. First: there is a difference between man and machine. Second: there are certain tasks which computers ought not be made to do, independent of whether computers can be made to do them. The book’s subtitle – From Judgment to Calculation – offers a clue as to how these two statements fit together. For Weizenbaum, judgment involves choices that are guided by values. These values are acquired through the course of our life experience and are necessarily qualitative: they cannot be captured in code. Calculation, by contrast, is quantitative. It uses a technical calculus to arrive at a decision. Computers are only capable of calculation, not judgment. This is because they are not human, which is to say, they do not have a human history – they were not born to mothers, they did not have a childhood, they do not inhabit human bodies or possess a human psyche with a human unconscious – and so do not have the basis from which to form values. And that would be fine, if we confined computers to tasks that only required calculation. But thanks in large part to a successful ideological campaign waged by what he called the “artificial intelligentsia”, people increasingly saw humans and computers as interchangeable. As a result, computers had been given authority over matters in which they had no competence. (It would be a “monstrous obscenity”, Weizenbaum wrote, to let a computer perform the functions of a judge in a legal setting or a psychiatrist in a clinical one.) Seeing humans and computers as interchangeable also meant that humans had begun to conceive of themselves as computers, and so to act like them. They mechanised their rational faculties by abandoning judgment for calculation, mirroring the machine in whose reflection they saw themselves. This had especially destructive policy consequences. Powerful figures in government and business could outsource decisions to computer systems as a way to perpetuate certain practices while absolving themselves of responsibility. Just as the bomber pilot “is not responsible for burned children because he never sees their village”, Weizenbaum wrote, software afforded generals and executives a comparable degree of psychological distance from the suffering they caused. Letting computers make more decisions also shrank the range of possible decisions that could be made. Bound by an algorithmic logic, software lacked the flexibility and the freedom of human judgment. This helps explain the conservative impulse at the heart of computation. Historically, the computer arrived “just in time”, Weizenbaum wrote. But in time for what? “In time to save – and save very nearly intact, indeed, to entrench and stabilise – social and political structures that otherwise might have been either radically renovated or allowed to totter under the demands that were sure to be made on them.” Computers became mainstream in the 1960s, growing deep roots within American institutions just as those institutions faced grave challenges on multiple fronts. The civil rights movement, the anti-war movement and the New Left are just a few of the channels through which the era’s anti-establishment energies found expression. Protesters frequently targeted information technology, not only because of its role in the Vietnam war but also due to its association with the imprisoning forces of capitalism. In 1970, activists at the University of Wisconsin destroyed a mainframe during a building occupation; the same year, protesters almost blew one up with napalm at New York University. This was the atmosphere in which Computer Power and Human Reason appeared. Computation had become intensely politicised. There was still an open question as to the path that it should take. On one side stood those who “believe there are limits to what computers ought to be put to do,” Weizenbaum writes in the book’s introduction. On the other were those who “believe computers can, should, and will do everything” – the artificial intelligentsia. Marx once described his work Capital as “the most terrible missile that has yet been hurled at the heads of the bourgeoisie”. Computer Power and Human Reason seemed to strike the artificial intelligentsia with similar force. McCarthy, the original AI guru, seethed: “Moralistic and incoherent”, a work of “new left sloganeering”, he wrote in a review. Benjamin Kuipers from MIT’s AI Lab – a PhD student of Minsky’s – complained of Weizenbaum’s “harsh and sometimes shrill accusations against the artificial intelligence research community”. Weizenbaum threw himself into the fray: he wrote a point-by-point reply to McCarthy’s review, which led to a response from the Yale AI scientist Roger C Schank – to which Weizenbaum also replied. He clearly relished the combat. In the spring of 1977, the controversy spilled on to the front page of the New York Times. “Can machines think? Should they? The computer world is in the midst of a fundamental dispute over those questions,” wrote the journalist Lee Dembart. Weizenbaum gave an interview from his MIT office: “I have pronounced heresy and I am a heretic.” Computer Power and Human Reason caused such a stir because its author came from the world of computer science. But another factor was the besieged state of AI itself. By the mid-1970s, a combination of budget-tightening and mounting frustration within government circles about the field failing to live up to its hype had produced the first “AI winter”. Researchers now struggled to get funding. The elevated temperature of their response to Weizenbaum was likely due at least in part to the perception that he was kicking them when they were down. AI wasn’t the only area of computation being critically reappraised in these years. Congress had been recently contemplating ways to regulate “electronic data processing” by governments and businesses in order to protect people’s privacy and to mitigate the potential harms of computerised decision-making. (The watered-down Privacy Act was passed in 1974.) Between radicals attacking computer centers on campus and Capitol Hill looking closely at data regulation, the first “techlash” had arrived. It was good timing for Weizenbaum. Weizenbaum in Germany in 2005. Photograph: DPA archive/Alamy Computer Power and Human Reason gave him a national reputation. He was delighted. “Recognition was so important to him,” his daughter Miriam told me. As the “house pessimist of the MIT lab” (the Boston Globe), he became a go-to source for journalists writing about AI and computers, one who could always be relied upon for a memorable quote. But the doubts and anxieties that had plagued him since childhood never left. “I remember him saying that he felt like a fraud,” Miriam told me. “He didn’t think he was as smart as people thought he was. He never felt like he was good enough.” As the excitement around the book died down, these feelings grew overwhelming. His daughter Pm told me that Weizenbaum attempted suicide in the early 1980s. He was hospitalised at one point; a psychiatrist diagnosed him with narcissistic personality disorder. The sharp swings between grandiosity and dejection took their toll on his loved ones. “He was a very damaged person and there was only so much he could absorb of love and family,” Pm said. In 1988, he retired from MIT. “I think he ended up feeling pretty alienated,” Miriam told me. In the early 1990s, his second wife, Ruth, left him; in 1996, he returned to Berlin, the city he had fled 60 years earlier. “Once he moved back to Germany, he seemed much more content and engaged with life,” Pm said. He found life easier there. As his fame faded in the US, it increased in Germany. He became a popular speaker, filling lecture halls and giving interviews in German. The later Weizenbaum was increasingly pessimistic about the future, much more so than he had been in the 1970s. Climate change terrified him. Still, he held out hope for the possibility of radical change. As he put it in a January 2008 article for Süddeutsche Zeitung: “The belief that science and technology will save the Earth from the effects of climate breakdown is misleading. Nothing will save our children and grandchildren from an Earthly hell. Unless: we organise resistance against the greed of global capitalism.” Two months later, on 5 March 2008, Weizenbaum died of stomach cancer. He was 85. By the time Weizenbaum died, AI had a bad reputation. The term had become synonymous with failure. The ambitions of McCarthy, formulated at the height of the American century, were gradually extinguished in the subsequent decades. Getting computers to perform tasks associated with intelligence, like converting speech to text, or translating from one language to another, turned out to be much harder than anticipated. Today, the situation looks rather different. We have software that can do speech recognition and language translation quite well. We also have software that can identify faces and describe the objects that appear in a photograph. This is the basis of the new AI boom that has taken place since Weizenbaum’s death. Its most recent iteration is centred on “generative AI” applications like ChatGPT, which can synthesise text, audio and images with increasing sophistication. At a technical level, the set of techniques that we call AI are not the same ones that Weizenbaum had in mind when he commenced his critique of the field a half-century ago. Contemporary AI relies on “neural networks”, which is a data-processing architecture that is loosely inspired by the human brain. Neural networks had largely fallen out of fashion in AI circles by the time Computer Power and Human Reason came out, and would not undergo a serious revival until several years after Weizenbaum’s death. But Weizenbaum was always less concerned by AI as a technology than by AI as an ideology – that is, in the belief that a computer can and should be made to do everything that a human being can do. This ideology is alive and well. It may even be stronger than it was in Weizenbaum’s day. Certain of Weizenbaum’s nightmares have come true: so-called risk assessment instruments are being used by judges across the US to make crucial decisions about bail, sentencing, parole and probation, while AI-powered chatbots are routinely touted as an automated alternative to seeing a human therapist. The consequences may have been about as grotesque as he expected. According to reports earlier this year, a Belgian father of two killed himself after spending weeks talking with an AI avatar named … Eliza. The chat logs that his widow shared with the Brussels-based newspaper La Libre show Eliza actively encouraging the man to kill himself. A humanoid robot interacting with visitors at the AI for Good summit in Geneva earlier this month. Photograph: Johannes Simon/Getty On the other hand, Weizenbaum would probably be heartened to learn that AI’s potential for destructiveness is now a matter of immense concern. It preoccupies not only policymakers – the EU is finalising the world’s first comprehensive AI regulation, while the Biden administration has rolled out a number of initiatives around “responsible” AI – but AI practitioners themselves. Broadly, there are two schools of thought today about the dangers of AI. The first – influenced by Weizenbaum – focuses on the risks that exist now. For instance, experts such as the linguist Emily M Bender draw attention to how large language models of the kind that sit beneath ChatGPT can echo regressive viewpoints, like racism and sexism, because they are trained on data drawn from the internet. Such models should be understood as a kind of “parrot”, she and her co-authors write in an influential 2021 paper, “haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine.” The second school of thought prefers to think in speculative terms. Its adherents are less interested in the harms that are already here than in the ones that may someday arise – in particular the “existential risk” of an AI that becomes “superintelligent” and wipes out the human race. Here the reigning metaphor is not a parrot but Skynet, the genocidal computer system from the Terminator films. This perspective enjoys the ardent support of several tech billionaires, including Elon Musk, who have financed a network of like-minded thinktanks, grants and scholarships. It has also attracted criticism from members of the first school, who observe that such doomsaying is useful for the industry because it diverts attention away from the real, current problems that its products are responsible for. If you “project everything into the far future,” notes Meredith Whittaker, you leave “the status quo untouched”. Weizenbaum, ever attentive to the ways in which fantasies about computers can serve powerful interests, would probably agree. But there is nonetheless a thread of existential risk thinking that has some overlap with his own: the idea of AI as alien. “A superintelligent machine would be as alien to humans as human thought processes are to cockroaches,” argues the philosopher Nick Bostrom, while the writer Eliezer Yudkowsky likens advanced AI to “an entire alien civilisation”. Vanishing point: the rise of the invisible computer Read more Weizenbaum would add the following caveat: AI is already alien, even without being “superintelligent”. Humans and computers belong to separate and incommensurable realms. There is no way of narrowing the distance between them, as the existential risk crowd hopes to do through “AI alignment”, a set of practices for “aligning” AI with human goals and values to prevent it from becoming Skynet. For Weizenbaum, we cannot humanise AI because AI is irreducibly non-human. What you can do, however, is not make computers do (or mean) too much. We should never “substitute a computer system for a human function that involves interpersonal respect, understanding and love”, he wrote in Computer Power and Human Reason. Living well with computers would mean putting them in their proper place: as aides to calculation, never judgment. Weizenbaum never ruled out the possibility that intelligence could someday develop in a computer. But if it did, he told the writer Daniel Crevier in 1991, it would “be at least as different as the intelligence of a dolphin is to that of a human being”. There is a possible future hiding here that is neither an echo chamber filled with racist parrots nor the Hollywood dystopia of Skynet. It is a future in which we form a relationship with AI as we would with another species: awkwardly, across great distances, but with the potential for some rewarding moments. Dolphins would make bad judges and terrible shrinks. But they might make for interesting friends

1 0

A chatbot? No, it’s a doctor.
by Fabio Alemagna July 24, 2023

July 24, 2023

Da https://www.bloomberg.com/news/articles/2023-07-21/meet-your-new-ai-chatbot… For Chinese who do almost everything on their smartphones, seeing a doctor online doesn’t sound like a novel idea. But the way technology is headed in the country, they’ll soon wonder if they’re talking to a real doctor — or one that’s powered by so-called generative AI tools similar to ChatGPT. Chinese online healthcare provider Medlinker in May unveiled an AI doctor dubbed MedGPT, claiming it’s capable of diagnosing some of the most common diseases with the same degree of accuracy and consistency of a human doctor. In fact, AI has already been widely applied in online consultations offered by some of China’s leading Internet healthcare services — but it largely serves an auxiliary role. Patients wanting to chat with a doctor online sometimes find themselves talking to chatbots first, which collect basic information from patients and redirect them to the suitable human specialist. What sets MedGPT apart, according to Medlinker, is that it could handle the entire process on its own, from diagnosis to prescribing tests and medication — replacing human doctors. Medlinker crammed tens of billions of medical records and academic journals into MedGPT, with the chatbot powered by ChatGPT-style services developed by domestic and foreign firms. A team of over 100 human doctors then trained the chatbot to assess a patient and take appropriate actions, such as ordering medical tests, prescribing medicines or offering dietary guidance. MedGPT is a very inquisitive doctor: The chatbot engages patients in multiple rounds of questioning to get as much information as possible before arriving at a conclusion. But how does it stack up against human counterparts? Attendants photograph the stage as Sam Altman, chief executive officer of OpenAI, speaks during an event in Seoul last month. Photographer: SeongJoon Cho/Bloomberg A month after MedGPT’s unveiling, Medlinker ran a trial pitting it against 10 senior human specialists, consulting more than 100 patients with issues from cardiovascular problems to kidney disease. MedGPT and the physicians got similar scores, and a judging panel of seven doctors credited the chatbot for being comprehensive in its questioning and refraining from giving diagnoses too early. Still, they also noted shortcomings: Some of the tests MedGPT ordered were repetitive, and recommended treatments could be excessive. “MedGPT still has lots of problems, but I think the strides it has made is a milestone,” said Ren Jingyi, a cardiologist at a top Beijing hospital, one of the seven experts — and only one of two to rate the bot higher than its human counterparts. MedGPT isn’t yet in commercial use, pending government approval — and guidelines for the real-world application of such generative AI tools for healthcare. The company said it is looking for partnership with companies and medical institutions to improve the AI doctor’s accuracy, train it to diagnose more diseases and run a bigger trial later this year. It also hopes to work with medical experts to establish standards for AI-based healthcare services. It could be a matter of when, rather than if, AI doctors will be able to independently treat humans. But whether they’ll replace some human doctors or be support tools remains to be seen.

3 3

Re: [nexa] The Santiago Boys
by Giuseppe Attardi July 24, 2023

July 24, 2023

Noi invitammo Fernardo Flores a Pisa negli anni ’80 a raccontarci la sua esperienza di ministro e le sue ricerche successive su Language/Action: https://en.wikipedia.org/wiki/Language/action_perspective — Beppe > On 23 Jul 2023, at 06:47, nexa-request(a)server-nexa.polito.it wrote: > > From: Alberto Cammozzo <ac+nexa(a)zeromx.net <mailto:ac+nexa@zeromx.net>> > To: nexa(a)server-nexa.polito.it <mailto:nexa@server-nexa.polito.it> > Subject: Re: [nexa] The Santiago Boys > Message-ID: <d850a417-24b7-1cc9-6a53-aebb6dd7c476(a)zeromx.net <mailto:d850a417-24b7-1cc9-6a53-aebb6dd7c476@zeromx.net>> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Non ho ascoltato ancora il podcast e spero anche io che Morozov intenda > pubblicare sotto altra forma, ma le vicende di Cybersyn e Beer sono > abbastanza documentate da alcuni anni*: > * > > Eden Medina. /Designing Freedom, Regulating a Nation: Socialist > Cybernetics in Allende's Chile/ , in Journal of Latin American > Studies, vol. 38, n. 38, Cambridge University Press, 2006, pp. > 571-606, DOI:10.1017/S0022216X06001179. > Disponibile su sci-hub > <https://sci-hub.se/https://www.cambridge.org/core/journals/journal-of-latin…> > > Eden Medina, /Cybernetic Revolutionaries: Technology and Politics in > Allende’s Chile/ (Cambridge, MA: MIT Press, 2011) > > Nella collezione di Philosophy Kitchen 18 (marzo 2023) presentata da > poco in questa lista si parla di Beer e Cybersyn in due articoli: > > Robin Asby - /On The Framing of Systems and Cybernetic Models/ > <https://ojs.unito.it/index.php/philosophykitchen/article/view/7836> > > Paolo Capriati - /Autopoiesi dei sistemi politici: il caso Cybersyn > </https://ojs.unito.it/index.php/philosophykitchen/article/view/7837/>/ > > Nel 2008 sul NYT è comparso un articolo ben informato (togliere js per > accedere): > > Alexei Barrionuevo, March 28, 2008, /Before ’73 Coup, Chile Tried to > Find the Right Software for Socialism/ > <https://www.nytimes.com/2008/03/28/world/americas/28cybersyn.html> > > ciao, > > Alberto

1 0