<https://www.forbes.com/sites/forbestechcouncil/2022/04/11/supply-chain-attac...> I recently listened to a great podcast by Google's cybersecurity experts on the topic of secure artificial intelligence (AI). The range of issues raised seemed, to me, very interesting and relevant. But because the podcast was limited in time, some of the extremely important topics were covered only too briefly. That's why I decided to take a closer look at some of the topics raised in the podcast—and this has become the article you're reading right now. AI Risks The topic of AI risks, especially when related to security, is currently gaining more popularity. For example, back in 2020, when my company collected information on the most critical security-related AI incidents of the past decade, we were only able to mention six. However, more recently, it was difficult for us to determine the most significant ones for the list of the 10 most notorious AI incidents in 2021. I won't go into detail on each incident now, but I will mention the facial recognition bypass incident in which a man attempted to steal over $2.5 million in state funds, and the Zillow case, which resulted in a $6 billion drop in the company's valuation. As a result, Zillow had to lay off 25% of its workforce and sell 7,000 homes with a total asset value in excess of the $2.8 billion previously acquired. The reason for this is that routine machine learning models can lead to extremely serious consequences if mismanaged. Although the last case is not directly related to security and was caused more by an algorithm error, it nonetheless demonstrates one very important idea: The losses associated with incidents in AI can be very large-scale. The point is that AI applications, by their very nature, will be responsible for making smarter decisions than traditional software. As a rule, the "smarter" the system is, the more important and critical decisions are entrusted to this system—more important decisions involve greater responsibility, and, as a result, greater consequences can occur if the decisions are wrong. Just imagine the consequences of attacks in cyber-physical worlds with autonomous cars, smart cities and a variety of systems based on computer vision automation. If you think such attacks are implausible, I hasten to disappoint you, as all the systems mentioned above have already been attacked as part of many research projects. Such situations occurring in reality is only a matter of time. In fact, cybercriminals just have to find an attractive victim and use case with a good "attack-market fit." And it seems that one of the attack-market fit examples can happen quite soon. Supply Chain Attacks On AI One of the interesting topics mentioned in the podcast was supply chain attacks, which are an incredibly big threat right now. We mainly see examples of them related to traditional software, where an open-source library that is implemented in several or even millions of places is vulnerable—which can potentially lead to the collapse of almost any enterprise around the world, as happened recently with Log4j. But very little information about similar cases related to AI is known to a wide audience. The podcast, however, mentioned one example of a supply chain attack on AI called poisoning. And this, alas, is a completely realistic situation. Someone can poison a dataset in such a way that when it is used to train a system, the system will be poisoned and make wrong decisions. Such a situation could put at risk every company that deals with users' data, such as internet platforms and social networks. AI algorithms can be retrained to make the wrong decisions, and this presents a great opportunity for third parties: Scammers can use this to make AI more biased and then blame the AI companies for it. However, there are other variants of supply chain attacks that were not mentioned in the podcast, such as AI trojans and AI backdoors. AI trojans are methods of modifying a trained AI model so that it will only perform certain actions when given certain inputs. Let's say you have affected the image classification model in such a way that a certain type of weapon is not detected by the AI model that manages their transfer and storage. In terms of carrying out such an attack, someone can hack into the model hosting server and upload Trojan models. This is a very realistic scenario because most AI solutions use pre-trained models, and there is a good chance that this model will be ubiquitous. AI backdoors are almost the same, but the difference is that they are implemented at the model training stage by the authors of the model. For example, some facial recognition vendors may train a model to recognize a particular face as a "universal" key, then this system will be sent out and implemented at hundreds of facilities. These backdoors are very difficult to identify because AI is usually based on deep neural networks, and we still struggle to understand how they work in normal situations, let alone adversarial ones. There needs to be completely new ways of identifying the presence of such backdoors and better methods of dealing with them. While nothing is being done, know that in the hands of attackers, these exploits present huge opportunities that can fatally harm your company. Solutions What can you do right now? Of course, if your core business functions are dependent on AI decisions or you develop AI, then it's critical to enforce security measures in each step of your AI development—from the design and development process itself to testing and operations. For companies that consume AI solutions, it's essential to start with asset inventory and understand where you use AI, who is responsible for it, whether deployed AI systems are based on open-source models with vulnerabilities and whether the vendors are taking care of AI security.