#040 - Adversarial Examples (Dr. Nicholas Carlini, Dr. Wieland Brendel, Florian Tramèr)
Machine Learning Street Talk (MLST)
Shownotes Transcript
Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear. there's good reason to believe neural networks look at very different features than we would have expected. As articulated in the 2019 "features not bugs" paper Adversarial examples can be directly attributed to the presence of non-robust features: features derived from patterns in the data distribution that are highly predictive, yet brittle and incomprehensible to humans.
Adversarial examples don't just affect deep learning models. A cottage industry has sprung up around Threat Modeling in AI and ML Systems and their dependencies. Joining us this evening are some of currently leading researchers in adversarial examples;
Florian Tramèr - A fifth year PhD student in Computer Science at Stanford University
https://twitter.com/florian_tramer
Dr. Wieland Brendel - Machine Learning Researcher at the University of Tübingen & Co-Founder of layer7.ai
https://medium.com/@wielandbr
https://twitter.com/wielandbr
Dr. Nicholas Carlini - Research scientist at Google Brain working in that exciting space between machine learning and computer security.
https://nicholas.carlini.com/
We really hope you enjoy the conversation, remember to subscribe!
Yannic Intro [00:00:00]
Tim Intro [00:04:07]
Threat Taxonomy [00:09:00]
Main show intro [00:11:30]
Whats wrong with Neural Networks? [00:14:52]
The role of memorization [00:19:51]
Anthropomorphization of models [00:22:42]
Whats the harm really though / focusing on actual ML security risks [00:27:03]
Shortcut learning / OOD generalization [00:36:18]
Human generalization [00:40:11]
An existential problem in DL getting the models to learn what we want? [00:41:39]
Defenses to adversarial examples [00:47:15]
What if we had all the data and the labels? Still problems? [00:54:28]
Defenses are easily broken [01:00:24]
Self deception in academia [01:06:46]
ML Security [01:28:15]