January 2019 - nexa - server-nexa.polito.it

Autonomous weapons and the new laws of war -- A technology that may prove hard to restrain

by Alberto Cammozzo

<https://www.economist.com/briefing/2019/01/19/autonomous-weapons-and-the-ne…> [...] The challenge that modern armed forces, and armsmakers like IAI, are working on is the ability to pick the target out from a field of non-targets. There are two technological developments that make the challenge a timely one. One is that computers are far more powerful than they used to be and, thanks to “machine learning”, getting much more sophisticated in their ability to distinguish between objects. If an iPhone can welcome your face but reject your sibling’s, why shouldn’t a missile be able to distinguish a tank from a school bus? The change is that autonomy in the non-killing aspects of military life is spreading like wildfire. Drones, driverless trucks and crewless submarines are all being used for various purposes, most of them entirely non-lethal. At the British Army’s “Autonomous Warrior” exercise in December 2018, on the wet and windswept training grounds of Salisbury Plain in southern England, military officers showed off autonomous vehicles and aircraft designed to watch enemy lines, evacuate wounded soldiers and deliver supplies over the perilous “last mile” up to the front line. “Think C-3PO,” says one officer, “not the Terminator.” Autonomous vehicles do not have to become autonomous weapons, even when capable of deadly force. The Reaper drones with which America assassinates enemies are under firm human control when it comes to acts of violence, even though they can fly autonomously. Satellite remote control, though, involves a time delay which would matter more were the drones being shot at in an all-out war. Co-operation may be better with humans out of the loop, too. The Pentagon’s out-there-thinking department, DARPA, is working on autonomous attack swarms more like a murmuration of starlings than a formation of fighter-bombers. What human operators could co-ordinate such dynamics? This is not just an issue for the future. One of the advantages that MDBA, a European missile-maker, boasts for its air-to-ground Brimstones is that they can “self-sort” based on firing order. If different planes launch volleys of Brimstones into the same “kill box”, where they are free to do their worst, the missiles will keep tabs on each other to reduce the chance that two strike the same target. Cost is also a factor in armies where trained personnel are pricey. “The thing about robots is that they don’t have pensions,” says General Sir Richard Barrons, one of Britain’s most senior commanders until 2016. Nor do they have dependents. The loss of a robot is measured in money and capability, not human potential. If keeping a human in the loop was merely a matter of spending more, it might be deemed worthwhile regardless. But human control creates vulnerabilities. It means that you must pump a lot of encrypted data back and forth. What if the necessary data links are attacked physically—for example with anti-satellite weapons—jammed electronically or subverted through cyberwarfare? Future wars are likely to be fought in what America’s armed forces call “contested electromagnetic environments”. The Royal Air Force is confident that encrypted data links would survive such environments. But air forces have an interest in making sure there are still jobs for pilots; this may leave them prey to unconscious bias. The vulnerability of communication links to interference is an argument for greater autonomy. But autonomous systems can be interfered with, too. The sensors for weapons like Brimstone need to be a lot more fly than those required by, say, self-driving cars, not just because battlefields are chaotic, but also because the other side will be trying to disorient them. Just as some activists use asymmetric make-up to try to confuse face-recognition systems, so military targets will try to distort the signatures which autonomous weapons seek to discern. Paul Scharre, author of “Army of None: Autonomous Weapons and the Future of War”, warns that the neural networks used in machine learning are intrinsically vulnerable to spoofing. Judgment day New capabilities, reduced costs, resistance to countermeasures and the possibility of new export markets are all encouraging R&D in autonomous weapons. To nip this in the bud, the Campaign to Stop Killer Robots is calling for a pre-emptive ban on “fully autonomous” weapons. The trouble is that there is little agreement on where the line is crossed. Switzerland, for instance, says that autonomous weapons are those able to act “in partial or full replacement of a human in the use of force, notably in the targeting cycle”, thus encompassing Harop and Brimstone, among many others. Britain, by contrast, says autonomous weapons are only those “capable of understanding higher level intent and direction”. That excludes everything in today’s arsenals, or for that matter on today’s drawing boards. Partly in order to sort these things out, in 2017 the UN’s Convention on Certain Conventional Weapons formalised its earlier discussions of the issues by creating a group of governmental experts (GGE) to study the finer points of autonomy. As well as trying to develop a common understanding of what weapons should be considered fully autonomous, it is considering both a blanket ban and other options for dealing with the humanitarian and security challenges that they create. Most states involved in the convention’s discussions agree on the importance of human control. But they differ on what this actually means. In a paper for Article 36, an advocacy group named after a provision of the Geneva conventions that calls for legal reviews on new methods of warfare, Heather Roff and Richard Moyes argue that “a human simply pressing a ‘fire’ button in response to indications from a computer, without cognitive clarity or awareness” is not really in control. “Meaningful control”, they say, requires an understanding of the context in which the weapon is being used as well as capacity for timely and reasoned intervention. It also requires accountability. Lieutenant-colonel Richard Craig, who leads the British Army HQ’s research on autonomous systems, agrees that context is crucial. In some contexts it might be right to vet every target. In others it is sufficient to understand the threat and act accordingly. For example a Phalanx system, he says, “wouldn’t be in fully autonomous mode unless there was a high threat. Meaningful human control is to turn it on into that mode, and then to turn it off”. This means that future robot warplanes, such as those being explored by the French-led nEUROn programme and Britain’s Taranis, both of which are experimenting with automatic target recognition, present the biggest challenge. Long-legged as they are, they may encounter a wide range of target environments that could be hard to anticipate. They could be in or out of meaningful human control depending on where they end up, the competence and experience of the operators, what is likely to step into their path and, potentially, changes to their algorithms made through on-board machine learning. A field day for ethicists; a nightmare for the would-be treaty-makers. The two dozen states that want a legally binding ban on fully autonomous weapons are mostly military minnows like Djibouti and Peru, but some members, such as Austria, have diplomatic sway. None of them has the sort of arms industry that stands to profit from autonomous weapons. They ground their argument in part on International Humanitarian Law (IHL), a corpus built around the rules of war laid down in the Hague and Geneva conventions. This demands that armies distinguish between combatants and civilians, refrain from attacks where the risk to civilians outweighs the military advantage, use no more force than is proportional to the objective and avoid unnecessary suffering. When it comes to making distinctions, Vincent Boulanin and Maaike Verbruggen, experts at SIPRI, note that existing target-recognition systems, for all their recent improvement, remain “rudimentary”, often vulnerable to bad weather or cluttered backgrounds. Those that detect humans are “very crude”. And this is before wily enemies try to dupe the robots into attacking the wrong things. Necessity and proportionality, which requires weighing human lives against military aims, are even more difficult. “However sophisticated new machines may be, that is beyond their scope,” says Major Kathleen McKendrick of the British army. An army that uses autonomous weapons needs to be set up so as to be able to make proportionality decisions before anything is fired. Salvation? More broadly, IHL is shaped by the “Martens clause”, originally adopted in the Hague convention of 1899. This says that new weapons must comply with “the principles of humanity” and “dictates of public conscience”. Bonnie Docherty of Human Rights Watch, the NGO which co-ordinates the anti-robot campaign, argues that, “As autonomous machines, fully autonomous weapons could not appreciate the value of human life and the significance of its loss...They would thus fail to respect human dignity.” A strong argument, but hardly legally watertight; other philosophies are available. As for the dictates of public conscience, research and history show that they are more flexible than a humanitarian would wish. Leaving aside law and ethics, autonomous weapons could pose new destabilising risks. Automatic systems can interact in seemingly unpredictable ways, as when trading algorithms cause “flash crashes” on stockmarkets. Mr Scharre raises the possibility of a flash war caused by “a cascade of escalating engagements”. “If we are open to the idea that humans make bad decisions”, says Peter Roberts, director of military sciences at the Royal United Services Institute, a think-tank, “we should also be open to the idea that AI systems will make bad decisions—just faster.” Beyond the core group advocating a ban there is a range of opinions. China has indicated that it supports a ban in principle; but on use, not development. France and Germany oppose a ban, for now; but they want states to agree a code of conduct with wriggle room “for national interpretations”. India, which chaired the GGE, is reserving its position. It is eager to avoid a repeat of nuclear history, in which technological have-nots were locked out of game-changing weaponry by a discriminatory treaty. At the far end of the spectrum a group of states, including America, Britain and Russia, explicitly opposes the ban. These countries insist that existing international law provides a sufficient check on all future systems—not least through Article 36 reviews, which they say should be taken more seriously rather than ducked, as some countries do today. They argue that the law should not be governed by the shortcomings of current systems when it comes to, say, discrimination. Some even argue that autonomous weapons might make war more humane. Human warriors break the IHL rules. Properly programmed robots might be unable to. Samsung’s SGR-A1 sentry gun, which used to be deployed in the DMZ, could recognise hands being thrown to the air and weapons to the ground as signs of surrender that meant do not shoot. All sorts of similar context-sensitive IHL-based restraint might be written into its descendants’ programming. But how long until an embattled army decided to loosen such tethers and let slip the robodogs of war? Which brings back one of the biggest problems that advocates of bans and controls have to face. Arms control requires verification, and this will always be a vexed issue when it comes to autonomy. “The difference between an MQ-9 Reaper and an autonomous version is software, not hardware,” says Michael Horowitz of the University of Pennsylvania. “It would be extremely hard to verify using traditional arms-control techniques.” The urge to restrict the technology before it is widely fielded, and used, is understandable. If granting weapons ever more autonomy turns out, in practice, to yield a military advantage, and if developed countries see themselves in wars of national survival, rather than the wars of choice they have waged recently, past practice suggests that today’s legal and ethical restraints may fall away. States are likely to sacrifice human control for self-preservation, says General Barrons. “You can send your children to fight this war and do terrible things, or you can send machines and hang on to your children.” Other people’s children are other people’s concern.

Jan. 19, 2019

Re: [nexa] "Japan seeks to ban global tech giants from using content contained in emails and other data"

by Stefano Quintarelli

quella roba li' c'era anche in una PDL di qualcuno in italia... On 19/01/2019 12:12, J.C. DE MARTIN wrote: > *Japan seeks to ban global tech giants from using content contained in > emails and other data* > > The communications ministry plans to prohibit global information > technology giants from using email content and other communications data > without users’ consent, informed sources said Friday. > > [...] > > continua qui: > https://www.japantimes.co.jp/news/2019/01/19/national/japan-ban-global-tech… > > _______________________________________________ > nexa mailing list > nexa(a)server-nexa.polito.it > https://server-nexa.polito.it/cgi-bin/mailman/listinfo/nexa > -- Sent with Disk Term on my TRS-80 Model 1 reserve your meeting with me at http://cal.quintarelli.it

Jan. 19, 2019

"Japan seeks to ban global tech giants from using content contained in emails and other data"

by J.C. DE MARTIN

*Japan seeks to ban global tech giants from using content contained in emails and other data* The communications ministry plans to prohibit global information technology giants from using email content and other communications data without users’ consent, informed sources said Friday. [...] continua qui: https://www.japantimes.co.jp/news/2019/01/19/national/japan-ban-global-tech…

Jan. 19, 2019

Call for abstracts // Utopias for a digital society - essay competition

by Francesco Ruggiero

Dear all, I would like to share the following call for abstracts with you. The essay competition twentyforty – utopias for a digital society <http://hiig.de/twentyforty> is a project of the Humboldt Institute for Internet and Society (HIIG) <https://www.hiig.de/> and the EU HUB within the Network of Centers <https://networkofcenters.net/>. They invite researchers from around the globe to develop thought-provoking utopias for a digital society. In a first step, They ask for short abstracts of 1,000 words. Ten authors will be invited to Berlin for a one-week writing sprint (full expenses covered) and the essays will be published in an edited volume. Feel free to share it within your network. Francesco -- Francesco Ruggiero Communication Manager Nexa Center for Internet & Society Politecnico di Torino - DAUIN Corso Duca degli Abruzzi, 24 - 10129 Torino web: http://nexa.polito.it mail: francesco.ruggiero(a)polito.it tel: 011 090 7219

Jan. 18, 2019

Re: [nexa] Abolish Silicon Valley

by armeo＠mail.nexacenter.org

concordo con Giacomo sul segreto industriale Raf Il 17/01/2019 19:07, Giacomo ha scritto: > On January 17, 2019 4:26:52 PM UTC, "armeo(a)mail.nexacenter.org" <meo(a)polito.it> wrote: >> non è necessario, e non servirebbe, abolire la Silicon Valley. >> >> Per contrastare il "capitalism’s desire for endless accumulation" basta >> abolire il "diritto della proprietà intellettuale" e il brevetto, in >> particolare, lo strumento fondamentale del nuovo capitalismo. > Io sono assolutamente d'accordo purché venga vietato al contempo il segreto industriale. > > Altrimenti rischiamo di passare dalla padella alla brace. > >> A questo proposito, mi permetto di invitare chi conoscesse qualche >> brevetto recente utile, a inviarne il testo al nostro magnifico "nexa >> server" per discuterne collegialmente il livello di innovazione proposto. > Sarebbe interessante in effetti. > > > Giacomo

Jan. 17, 2019

Re: [nexa] Abolish Silicon Valley

by Giacomo

On January 17, 2019 4:26:52 PM UTC, "armeo(a)mail.nexacenter.org" <meo(a)polito.it> wrote: >non è necessario, e non servirebbe, abolire la Silicon Valley. > >Per contrastare il "capitalism’s desire for endless accumulation" basta >abolire il "diritto della proprietà intellettuale" e il brevetto, in >particolare, lo strumento fondamentale del nuovo capitalismo. Io sono assolutamente d'accordo purché venga vietato al contempo il segreto industriale. Altrimenti rischiamo di passare dalla padella alla brace. >A questo proposito, mi permetto di invitare chi conoscesse qualche >brevetto recente utile, a inviarne il testo al nostro magnifico "nexa >server" per discuterne collegialmente il livello di innovazione proposto. Sarebbe interessante in effetti. Giacomo

Jan. 17, 2019

Re: [nexa] Abolish Silicon Valley

by armeo＠mail.nexacenter.org

non è necessario, e non servirebbe, abolire la Silicon Valley. Per contrastare il "capitalism’s desire for endless accumulation" basta abolire il "diritto della proprietà intellettuale" e il brevetto, in particolare, lo strumento fondamentale del nuovo capitalismo. Al fine di divertirsi e di comprendere che il brevetto non ha più alcun senso, in conseguenza dell'esplosione di tutte le conoscenze, si dia lettura a dieci brevetti scelti a caso su alcune decine di milioni di brevetti oggi validi. A questo proposito, mi permetto di invitare chi conoscesse qualche brevetto recente utile, a inviarne il testo al nostro magnifico "nexa server" per discuterne collegialmente il livello di innovazione proposto. Raf Il 16/01/2019 19:38, Giacomo ha scritto: > Undergirding this state of affairs is a set of intellectual property rights explicitly designed to favour corporations. This system — the flip side of globalisation — is propagated by various trade agreements and global institutions at the behest of the nation states who benefit from it the most. It’s no accident that Silicon Valley is a uniquely American phenomenon; not only does it owe its success to the United States’ exceptionally high defence spending — the source of its research funding and foundational technological breakthroughs — that very military might is itself what implicitly secures the intellectual property regime. > > Seen in that light, tech’s recent development begins to look rather different. Far from launching a new era of global prosperity, it has facilitated the further concentration of wealth and power. By virtue of their position as digital middlemen, Silicon Valley companies are able to extract vast amounts of capital from all over the world. The most salient example is Apple: recently crowned the world’s most valuable company, Apple rakes in enormous quarterly profits even as the Chinese workers who actually assemble its products are driven to suicide. > > ... > > The Silicon Valley model of technological development is structurally flawed. It can’t simply be tweaked in a more socially beneficial direction, because it was never intended to be useful for all of society in the first place. At its core, it was always a class project, meant to advance the interests of capital. The founders and investors and engineers who dutifully keep the engines running may not deliberately be reinforcing class divides, but functionally, they are carrying out technological development in a way that enables capitalism’s desire for endless accumulation. > > > https://tribunemag.co.uk/2019/01/abolish-silicon-valley > _______________________________________________ > nexa mailing list > nexa(a)server-nexa.polito.it > https://server-nexa.polito.it/cgi-bin/mailman/listinfo/nexa

Jan. 17, 2019

Re: [nexa] Conferenza CPDP 2019 | 30 gennaio 2019, Bruxelles

by Francesco Ruggiero

Gentilissimi, vi segnalo un altro appuntamento della conferenza CPDP 2019 <https://www.cpdpconferences.org/> a cui parteciperà *Stefano Leucci* (Fellow Centro Nexa) nel ruolo di "Chair": *Data Protection Impact Assessments: legal and practical challenges <https://www.cpdpconferences.org/cpdp-panels/data-protection-impact-assessme…>.* Buona giornata, Francesco Il giorno gio 17 gen 2019 alle ore 13:01 Francesco Ruggiero < francesco.ruggiero(a)polito.it> ha scritto: > Carissimi, > > vi segnalo che il Centro Nexa parteciperà alla conferenza CDPD 2019 > <https://www.cpdpconferences.org/>in qualità di Conference Partner e di > organizzatore del panel "*How adequacy mechanism works: Progress in EU's > governance of cross-border data flows?".* > > Il panel, co-organizzato con l'Università degli Studi di Torino, avrà > luogo venerdì *30 gennaio* alle ore 16, nell'*Area 42 Grand*, Rue des > Palais 46, Brussels. > Sarà presieduto da *Ugo Pagallo* (Nexa Center and University of Turin) e > moderato da *Bojana Bellamy* (Centre for Information Policy Leadership); > interverranno *Jos Dumortier *(TimeLex), *Kristina Irion* (University of > Amsterdam), *Ralf Sauer *(European Commission), *Fumio Shimpo* (Keio > University). > > Maggiori informazioni sono disponibili alla pagina: > https://nexa.polito.it/cpdp19 > > Programma completo: > https://www.cpdpconferences.org/ > > > -- > Francesco Ruggiero > Communication Manager > > Nexa Center for Internet & Society > Politecnico di Torino - DAUIN > Corso Duca degli Abruzzi, 24 - 10129 Torino > > web: http://nexa.polito.it > mail: francesco.ruggiero(a)polito.it > tel: 011 090 7219 > -- Francesco Ruggiero Communication Manager Nexa Center for Internet & Society Politecnico di Torino - DAUIN Corso Duca degli Abruzzi, 24 - 10129 Torino web: http://nexa.polito.it mail: francesco.ruggiero(a)polito.it tel: 011 090 7219

Jan. 17, 2019

Conferenza CDPD 2019 | 30 gennaio 2019, Bruxelles

by Francesco Ruggiero

Carissimi, vi segnalo che il Centro Nexa parteciperà alla conferenza CDPD 2019 <https://www.cpdpconferences.org/>in qualità di Conference Partner e di organizzatore del panel "*How adequacy mechanism works: Progress in EU's governance of cross-border data flows?".* Il panel, co-organizzato con l'Università degli Studi di Torino, avrà luogo venerdì *30 gennaio* alle ore 16, nell'*Area 42 Grand*, Rue des Palais 46, Brussels. Sarà presieduto da *Ugo Pagallo* (Nexa Center and University of Turin) e moderato da *Bojana Bellamy* (Centre for Information Policy Leadership); interverranno *Jos Dumortier *(TimeLex), *Kristina Irion* (University of Amsterdam), *Ralf Sauer *(European Commission), *Fumio Shimpo* (Keio University). Maggiori informazioni sono disponibili alla pagina: https://nexa.polito.it/cpdp19 Programma completo: https://www.cpdpconferences.org/ -- Francesco Ruggiero Communication Manager Nexa Center for Internet & Society Politecnico di Torino - DAUIN Corso Duca degli Abruzzi, 24 - 10129 Torino web: http://nexa.polito.it mail: francesco.ruggiero(a)polito.it tel: 011 090 7219

Jan. 17, 2019

Abolish Silicon Valley

by Giacomo

Undergirding this state of affairs is a set of intellectual property rights explicitly designed to favour corporations. This system — the flip side of globalisation — is propagated by various trade agreements and global institutions at the behest of the nation states who benefit from it the most. It’s no accident that Silicon Valley is a uniquely American phenomenon; not only does it owe its success to the United States’ exceptionally high defence spending — the source of its research funding and foundational technological breakthroughs — that very military might is itself what implicitly secures the intellectual property regime. Seen in that light, tech’s recent development begins to look rather different. Far from launching a new era of global prosperity, it has facilitated the further concentration of wealth and power. By virtue of their position as digital middlemen, Silicon Valley companies are able to extract vast amounts of capital from all over the world. The most salient example is Apple: recently crowned the world’s most valuable company, Apple rakes in enormous quarterly profits even as the Chinese workers who actually assemble its products are driven to suicide. ... The Silicon Valley model of technological development is structurally flawed. It can’t simply be tweaked in a more socially beneficial direction, because it was never intended to be useful for all of society in the first place. At its core, it was always a class project, meant to advance the interests of capital. The founders and investors and engineers who dutifully keep the engines running may not deliberately be reinforcing class divides, but functionally, they are carrying out technological development in a way that enables capitalism’s desire for endless accumulation. https://tribunemag.co.uk/2019/01/abolish-silicon-valley

Jan. 16, 2019