Category Archives: IT Futures

Hack of the Week: Adversarial Machine Learning

I heard about this one at a talk on Monday at our Washington DC CTO Roundtable on machine learning.

I had read about a kind of smackdown sport where machine learning gurus set to work trying to break the algorithms of their adversaries.

When I asked the speaker about it, he said, “Oh yeah, adversarial machine learning”.

Well, that was it, and here’s the Wikipedia article on it (flawed though Wikipedia seems to find the article).

Per this article, “AML” as we might call it has been with us for some time, mainly in the form of the fight between spammers and spam-filter developers.

You know:

  1. Spam filters add the phrase “penis enlargement” to their algorithm. Any email with “penis enlargement” in it gets flagged.
  2. Spammers start spelling it “penis enl@rgement”
  3. Rinse and repeat

Since the spammers just have change some generated text and the spam filters have to change and train a changed algorithm, guess who’s more supple?

The Roundtable speaker alleged that there was a sticker you could put on a stop sign that could fool a self-driving car algorithm into thinking it was a “Yield” sign. Think of the fun you could have with that if you were intent on getting self-driving cars to hurt people…

Two (Possible? Only Possible?) Failure Modes for AI

I’m an old AI guy — back from the ’70’s and ’80’s — and am often blown away by what “deep learning” and statistical approaches are accomplishing nowadays.  I never would have predicted an AI like Watson that could win at Jeopardy.  Or Go.

But some things are still the same, and when AIs fail, they fail in the same couple of ways.

I’m not talking about them turning the universe into a paperclip factory and eliminating us because we get in the way, a la “SuperIntelligence”  Maybe they’ll kill us someday, but that still seems a long ways off.

Today’s AIs disappoint us in one of two ways.

  1. Explanation.  Sadly, almost no humans will trust an AI’s conclusions without some account of how it reached those conclusions.  And most AI’s can’t account for their conclusions, especially the modern AI’s that are based on statistical weights and neural-style nets.  “I reached the conclusion that your cancer will respond to Treatment Cocktail A because I increased the weights on Nodes 1120-3388-692A-QRST and VVTX-8338-QQ94-AAAA from 10 to 30.”  Yeah, right.
  2. Turing Gulf.  This phenomenon, also called the “uncanny valley”, has a nice explanation here.  It’s often used to talk about an AI that’s kind of creepy because it’s near-human, but it can also be used for an AI that’s almost good enough but peeves you when it fails to make the grade.  Imagine an AI that needs to be “90”, where 90 is “90% accurate” or whatever.  And the AI is only “89” (again, whatever that means).  That AI is useless for the task because it will only peeve and frustrate its users.

Are those the only possible ways AIs can fail?  Welcome your comments.

The Magic Invulnerability of blockchain

OK, so last time I gave my impressions of how a coin transaction might be enacted in blockchain.

But now we get to the magic.  How can Bob and Alice be confident that someone won’t abuse their transfer of coin?

Let’s consider the case where Bob himself wants to spend Alice’s bitcoin twice.

He might have a couple of strategies for doing so, but they all involve, in one way or another, fudging the first transaction.  For example:

Erase the first transaction.  Bob has everything he needs to create a “spend” of Alice’s coin: her signed transaction giving the coin to his public key, and his private key.  If he can get erase transaction 1 (after his payee from tx 1 has re-transacted the bitcoin, of course) it would seem he could cover his tracks.

Blockchain makes it very hard to erase a transaction (or change it in any way) because all the transactions are hashed and linked.  Hashed means that there is a unique code generated by the transaction that can’t be duplicated.  And linked means that all the transactions ever are all linked together — and hashed — so that the only way you could fake a change to an old transaction would be to re-do all the transactions since.

And, hard as that might be, blockchain makes it harder by requiring a certain level of effort in the hashing.  Not just any hash result will do: the hash has to be smaller than a certain fraction of the maximum hash value.  Since each time you hash you get a different answer you have to keep hashing until you get an answer below the threshold.  Which means you have to do a lot of computational work.

But who even checks on this stuff?  In a centralized world where there’s a trusted central database, they do the checking.  In a decentralized world, who’s to even notice that a bunch of transactions have been faked or re-done?

The miners, that’s who.  What coin miners do is build valid blocks for the blockchain out of raw transactions.    Since there’s a large number of miners mining, the chances of all of them collaborating on a fraud are low (although I guess it could happen??)  And because they’re doing all this work to prop up the blockchain, they are rewarded by getting a small fee for most or all transactions that they block-ify, which is how miners end up with new coin.

I think I’ve got it, and I think I’ve explained it somewhat clearly, which probably means I’ve missed something profound and simple.  Can people who know more about this tell me where I messed up?

Reading about blockchain, moving my lips

I used to be technical.  I used to know technical stuff.

I still know quite a bit, but what I say about myself nowadays is that “I can follow a technical argument.”  That’s certainly not cutting-edge, nor is it one step removed.  It’s a different kind of skill.

As a techie, I was bound by what Buckminster Fuller or Ayn Rand would call “right and wrong.”  If my stuff — my hypothesis, my code, — “worked”, it was correct.  Not just, necessarily, or even beautiful, but it would do what I said it would do.

“Following a tech argument” is a different thing; I’m assuming that the speaker is speaking correctly, is saying what’s right, and seeing if I agree with the implications.

That’s how I’m approaching blockchain.  I’ve never been a cryptographic heavyweight, and I’m not even an “Alice and Bob” heavyweight.  Alice and Bob, for those who might not know, are personoids who engage in cryptographically motivated behavior.  For example: “Alice wants to send Bob a private message.”  “Bob wants to pay Alice in Bitcoin.”  Cryptographic heavyweights toss around Alice-and-Bob arguments in a way that I can mostly follow.  If I move my lips.

So I’ve been certain I could understand how blockchain — the “distributed ledger” underlying Bitcoin and other cryptocurrencies — worked.  Just one little problem: I didn’t actually understand.

So today I set out to understand how blockchain works.

I started with a Google search for “blockchain for dummies”.

First hit: a youtube video that tried to “explain” blockchain with an analogy.   #fail.  I couldn’t even understand the tenor and vehicle of the metaphor, let alone anything about how blockchain worked except, perhaps, that it was distributed and anonymous.  Well, I knew that already.

Next up?  “WTF Is the Blockchain?  A Guide for Total Beginners“.  #fail.  It kind of explains the various parts of the cryptocurrency ecosystem, but doesn’t explain the main thing: how are transactions in the blockchain verifiable, non-repudiatable, anonymous, and unfalsifiable?  And, just for grins, why are there miners?

I tried a few more, and ended up, as perhaps I should have begun, with the Developer’s guide.  Here we have geeks explaining to geeks how something works and how you might use it in making something of your own.

And here’s what I get out of all this:

  1. If Alice wants to pay Bob, she uses Bob’s public key (well, a hash of it) to create a transaction that authorizes whoever own Bob’s private key to spend <x> coin.  This transaction is “broadcast” to the coin community (presumably by Alice’s software) and Bob’s software presumably notes that such a transaction is created and lets Bob know.
  2. When Bob wants to spend the coin that Alice has given him, he creates a transaction which uses Alice’s transaction ID and a “signature script” saying what he wants to do with the coin.  The signature script is certified by Bob’s private key (and other mumbo-jumbo).  Bob then broadcasts his transaction to the network.

How do you keep Bob from spending Alice’s coin twice?   Next posting…

Google Cardboard and New York Times

Like many, I got a “free” Google Cardboard device with my Sunday New York Times yesterday.

“Free” because it was sponsored by — my wife tells me — a mini-car company who just happens to have a free VR experience or two you can run with the Cardboard.  Like they say, if you’re not the customer you’re the product.

In any case, great gift.  I immediately set it up and downloaded the NYT VR app and the Syrian refugee VR experience to my Android phone.

Good thing I’m a tech lover.  It was hard to find the VR app and it took a long time to download the experience.

Then we had to scrounge around for headphones (I don’t usually use them with my phone), fit the whole thing into the Cardboard, and disable the screen saver on my phone.

When I was at Intuit years ago we used to estimate that every extra click in a web app lost 20% of your audience.  So: 80%, 64%, 51%.

Setting up the experiment with the Syrian experience was like that.  Each step would have buffaloed a non-tech user or someone who didn’t have a clear vision of the endgame.

When I did get it set up it was pretty cool.  I didn’t try the roller-coaster app because I didn’t know it existed until I tuned into my CTO Club listserv later in the day.

And I couldn’t think of what to do about the sad fact that both my wife and I have pretty close-set eyes and the spacing of the Cardboard meant there was a big gap between how the VR was meant to look and how it looked to us.  Not as bad as watching a 3d movie without the glasses, but bad enough that we wouldn’t recommend it to a friend.  Hurts their NPS.