I heard about this one at a talk on Monday at our Washington DC CTO Roundtable on machine learning.
I had read about a kind of smackdown sport where machine learning gurus set to work trying to break the algorithms of their adversaries.
When I asked the speaker about it, he said, “Oh yeah, adversarial machine learning”.
Well, that was it, and here’s the Wikipedia article on it (flawed though Wikipedia seems to find the article).
Per this article, “AML” as we might call it has been with us for some time, mainly in the form of the fight between spammers and spam-filter developers.
You know:
- Spam filters add the phrase “penis enlargement” to their algorithm. Any email with “penis enlargement” in it gets flagged.
- Spammers start spelling it “penis enl@rgement”
- Rinse and repeat
Since the spammers just have change some generated text and the spam filters have to change and train a changed algorithm, guess who’s more supple?
The Roundtable speaker alleged that there was a sticker you could put on a stop sign that could fool a self-driving car algorithm into thinking it was a “Yield” sign. Think of the fun you could have with that if you were intent on getting self-driving cars to hurt people…