This is brilliant. And disturbing. If we are all wiped out by an AI in the future, this is going to be why – not because an AI has applied a moral judgement to us, just because wiping us out represents the most efficient path to the goal we've programmed it with. Imagine, for example, an AI with the goal "prevent the largest possible number of humans from dying". An efficient path to this goal would be to entirely wipe out the human race. Sure, it kills billions, but it efficiently prevents a potentially infinite number of humans from dying.