Our Last Innovation? Technologists Weigh Risks and Rewards of AI

For Musk and many dedicated AI researchers, it is this exponential improvement that’s frightening, not humanity’s dethronement as board game champions. What if machine learning was unleashed not so that an AI could perform a narrow task like playing board games or piloting a car, but for general problem solving? An AI that did this much better than we could would be a superintelligence, and with an intellect superior to ours it could then improve itself further — in ways we could never have imagined.

“I’m really quite close to… the cutting edge in AI, and it scares the hell out of me,”

No one knows what a superintelligence would do, but plenty of computer scientists, philosophers, neurology researchers and other experts focused exclusively on AI fear that it would do us harm, whether deliberately or through misinterpreting a command. It could meddle in human affairs.

For example, in 2016, lowly Twitter bots may have had a role in determining the U.S. presidential election. What if instead of bots, it had been the smartest intellect the Earth had ever seen with its own agenda?

AI Safety Researcher Allison Duettmann, with the Palo Alto-based Foresight Institute, says that the invention of advanced AI is inevitable despite these risks — and in large part because of them.

“We are moving forward so quickly [with AI] because… if you have more intelligent systems, you have a multiplying factor in every other domain,” she explains. “Given that certain players or agents are already developing AI… every other player has an immediate incentive to develop artificial general intelligence as well, because of the big advantages it would confer on whoever gets that right.”

Duettmann conducted a workshop at SXSW called “AI Philosophy: Why it’s Hard & State of the Art.” In it, she outlined cyber security concerns for AI, the risks of an AI arms race, the need for “social coordination” among those developing AI to ensure safety, and the challenges of instilling AIs with an ethical code. Given the potentially huge military, economic, social and scientific payoffs, it will be impossible to stop states and corporations from developing AI, she said, so it becomes crucial to make sure they do it responsibly.

In what is a rare stance for a Silicon Valley titan, Musk, in his own talk, said that this task will require government oversight. “This is a case where you have a very serious danger to the public, and therefore there needs to be a public body that has insight and then oversight to confirm that everyone is developing AI safely,” he said.

If an AI is truly intelligent, and even has a conscious mind, then we will be faced with the challenge of adapting our society to it. In a perfect future world, Musk said, “…there’s a benign AI and… we’re able to achieve a symbiosis with that AI.”

He added that this may take the form of an uplink between computers and their operators, possibly a neural one. “We’re already a cyborg in the sense that your phone and your computer are kind of extensions of you,” he said.

David Chalmers, co-director of NYU’s Center for Mind, Brain and Consciousness, was one of several experts to address human/AI interaction at a SXSW panel called “Can We Create Consciousness in a Machine?” Like Duettmann, he is a cautious optimist about the potential of AI, and believes that a human-like consciousness in a machine is possible, but we need to be ready for its implications.

“I believe that once we have those machines we will rapidly believe they are conscious,” he said. “Whether that’s correct or not is another question.”