How To Fool AI Into Seeing Something That Isn’t There [...] Today, neural nets are quite good at recognizing faces and spoken words—not to mention objects, animals, letters, and words. But they do make mistakes—sometimes egregious mistakes <http://www.telegraph.co.uk/technology/google/11710136/Google-Photos-assigns-...>. “No machine learning system is perfect,” says Kurakin. And in some cases, you can actually fool these systems into seeing or hearing things that aren’t really there. As Kurakin explains, you can subtly alter an image so that a neural network will think it includes something it doesn’t, an these alterations may be imperceptible to the human eye—a handful of pixels added here and another there. You could change several pixels in a photo of an elephant, he says, and fool a neural net into thinking it’s a car. Researchers like Kurakin call these “adversarial examples.” And they too are security holes. With their new paper [1], Kurakin, Bengio, and Goodfellow show that this can be a problem even when a neural network is used to recognize data pulled straight from a camera or some other sensor. Imagine a face recognition system that uses a neural network to control access to a top-secret facility. You could fool it into thinking you’re someone who you’re not, Kurakin says, simply by drawing some dots on your face. Goodfellow says this same type of attack could apply to almost any form of machine learning, including not only neural networks but things like decision trees <https://en.wikipedia.org/wiki/Decision_tree_learning> and support vector machines <https://en.wikipedia.org/wiki/Support_vector_machine>—machine learning methods that have been popular for more than a decade, helping computer systems make predictions based on data. In fact, he believes that similar attacks are already practiced in the real world. Financial firms, he suspects, are probably using them to fool trading systems used by competitors. “They could make a few trades designed to fool their competitors into dumping a stock at a lower price than its true value,” he says. “And then they could buy the stock up at that low price.” In their paper, Kurakin and Goodfellow fool neural nets by printing an adversarial image on a piece of a paper and showing the paper to a camera. But they believe that subtler attacks could work as well, such as the previous dots-on-the-face example. “We don’t know for sure we could do that in the real world, but our research suggests that it’s possible,” Goodfellow says. “We showed that we can fool a camera, and we think there are all sorts of avenues of attack, including fooling a face recognition system with markings that wouldn’t be visible to a human.” [...] <https://www.wired.com/2016/07/fool-ai-seeing-something-isnt/> [1] <https://arxiv.org/pdf/1607.02533v1.pdf>