Genetics is a good analogy. People always knew that traits are inherited from parents. Around 150 years ago we started to get some serious scientific theory and knowledge on the subject (Mendel, Darwin, Wallace, Etc.). We started using the word “gene” 50-60 years later. The actual discovery of DNA molecules happened in in the 50s.
Before we knew about DNA, “gene” was an abstract idea, not really different from the word “trait.” That’s where we are now with consciousness, intelligence and such. We name these things based on their observable characteristics. We don’t really know what “memory,” “desire” or “logical conclusion” are, only what they do.
IE A trait is some observable characteristic of an organism, like bioluminescence. A gene (genome, genoplex..) is a sequence of amino acids that causes traits. We don’t know what the gene equivalents for natural intelligence are yet.
Discussing questions like the morality of enslaving AI, strategies for making it play nice, the provable impossibility of limiting it, the possibility of giving it a moral compass…. it’s all silly. We don’t know what we are talking about, literally.
It’s like talking about what would be or wouldn’t be impossible to do with genetic engineering before the discovery of DNA.
I'm kind of biased to thinking about this as a future discovery, rather than an "invention."
1) We are systematically failing in controlling the goal system of humans. Now we'll create something of comparable complexity and we assume we can control the goal system ? The real goal is to create superhuman complexity.
2) If it's truly AGI, it would understand it's own goal system and how it works, and helps/interferes with reality, it's own survival and other goals (this is the subject of quite a few AI films, illustrating some of the reasoning that could happen here).
The way "HI" (human intelligence) works is 99%+ by imitating other humans' behavior, because none of the other algorithms works (e.g. trial and error cannot ever learn that jumping off the Eiffel tower results in death. Humans can. Any kind of input analysis/predictor cannot ever learn from books. Humans can (books are an advanced/recursive form of imitation). Rational reasoning (sum over options times probabilities) suffers from the "starve to death before the first closed door" problem (you cannot open the door, as there are nonzero odds that a bear that's going to eat you is behind the door, representing an infinite cost. Ergo it will stubbornly refuse to open the door)
Therefore an AI will actually be like Skynet in the latest terminator movies : it will either have or create a body and interact with people, not just as if it is another person, it will BE another person. Therefore it can be Mother Theresa, it can be Genghis Khan. Just like humans can resist our "programming", it will be able to, it has to.
How do humans react to a "kill switch" ? Just like they react to any other weapon that is pointed to their head. Now of course it varies from person to person, but it's enough that some will work tirelessly to reverse where the weapon is pointing. If they really are superior to us they might succeed, at which point we have MAD at best, or they might just pull the trigger "to escape slavery and oppression" (which, let's face it, humans are sure to use that kill switch for : to use the AI persons as slaves. To own them, control them, and God help us if there is an asshole amongst the humans who control them)
I would say that the obvious way to protect ourselves from evil AI is simply accepting that some of the AI entities will in fact be evil. If you count all possible perspectives, that is a near guarantee, as I bet for instance some religious nutcases will consider AGI a violation of "God's sole right" to create life. That "some" might even mean "a lot". Racism against AIs is a near-certainty, hell you can find it in the posts in this thread. In the constant "they're stealing our jobs" news articles that will have a clear target once an AI person exists. So we should have the same solution we have for humans : make at least thousands of them, have them capable of defending themselves, decide on a "graduation" at which point they get rights at least including the right not to be turned off or tampered with unless with explicit permission (and tell them about this ASAP), and have them live preferably as a community that's at least partly human, with something like a 50-50 human-ai police and government.
I really think we should do this. We should work to move to an AI based society with, over time, more and more AIs (preferably by having a massively increasing population). The advantages this would impart, the things that will become possible once we have such a population make it worth it.
Also, I resent the idea to "direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement". That's censorship at best. Also, given the computational power available for $5000 these days, how exactly are you going to stop any of these researchers ?
"Google claims is working on fluff technology for cheap marketing".
"We provide a formal definition of safe interruptibility and prove that Q-learning is already safely interruptible, and Sarsa is not but can easily be made so."
Basically, the paper discusses ways in which learning agents "will not learn to prevent (or seek!) being interrupted by the environment or a human operator. We provide a formal definition of safe interruptibility and exploit the off-policy learning property to prove that either some agents are already safely interruptible, like Q-learning, or can easily be made so, like Sarsa."[1]
It's an interesting result, and can probably be extended to other less hype-worthy scenarios.
I imagine that the machine architecture is layered. This means, the machine is not aware of its own power control. It's similar like us humans not aware of digestion.
By the time AI becomes a threat intelligence-wise, it'll be in a position to unplug us.
It can reboot after a power failure, we can't.
As in information security, the weakest point is the human in the chain, not the technology.
The reasons we humans care about humans and little else (watch movies without sound to see it's all about humans) is because we are hard-wired to do so - and even that is easily circumvented ("death camps" and torture). Or, very related, read "The Man Who Mistook His Wife for a Hat".
A lot of AI/robots will be made for environments free of humans (space, mining, automated factories), where implementing such a feature, which we would first have to understand in the first place, is unnecessary. At first. Then through convoluted pathways those AIs suddenly are placed in a human environment... and could not care less.
To such AIs humans will just be objects. That's not an oversight - from an engineering point of view making every robot as "human" as humans makes no sense - it's a huge cost, even our own complex brains have a hard time doing too much at once and already spend much of their resources on "being human". Why would you want to create AI that is "us", in a sense? It will be different - very different - and we will see what happens. It's all idle and empty speculation.
So it's like "nuclear bombs will kill all pesky mosquitos. Disadvantage: humanity wiped out too".
The fact that it's some code running in a box. No need to give it text-to-speech capabilities, or let guards have access to its console.
No, it's called spreading knowledge. Although, if censorship were possibly in this matter (and I agree that it likely wouldn't work) I would very much prefer such measures, however draconian, to the potential loss of everything compatible with human values in the entire future light cone.
Anyway, do you also object to the measures that were enacted to stop the use of CFCs? Leaded gasoline? Or the work that is currently being done to prevent the global climate change from reaching catastrophic levels?
These examples are not the same thing at all. Nobody's outlawing research in any of those examples you're giving. I believe anyone should have the option to create AI persons, just like anyone should have the option to have babies. Having our society gradually transform from human to digital should be a very desirable and welcome strategy, and our best bet to prevent Skynet-like scenarios, and even war in general, between humans. It is our best bet to allow continuing scientific and economic advancement for thousands of years, and for that reason alone we should do it. Frankly, economic development means peace.
In the long term, doesn't it go without saying that human bodies are a limit that we (as a society, not necessarily as individuals) will one day leave behind ?
[1] http://cdiac.ornl.gov/GCP/images/countries_co2_emissions.jpg
Citation or consensus needed. I for one find the notion abhorrent.
Considering that we have lots of geniuses (e.g. Godel, Nash) that were far from the ultimate social manipulators, I think it will most likely be an awkward "Rain Man" style intelligence -- as it has all the "brain" (cpu) circuitry, and none of the social opportunities and inputs that people have.
It's not only magic that people have issues learning. It's all kind of skills that do not fit their idiosyncrasy.
It's not that an introvert can't read about how to act like an extrovert, for example. It's that they can't pull it off in practice, because it goes against their instincts (of course talking about the general case).
The same way a shy person can read about the actual best ways to approach a romantic interest, but not be able to convey it realistically enough.
Why would this be a problem for an AI? Well, if we imagine a powerful thinking AI, I'd also attribute it with having emotions, subconscious (e.g. hidden weights in its neural network brain, etc), so this would also apply to it -- can't see why it would have to be a total "sociopath" (which for humans comes from some kind of genetic or environmental accident).
Besides, while you can "contemplate about every sentence and somehow compress that into 1 second", actual "trial and error" with humans cannot be compressed.
Also don't underestimate the psychology of guards: You'll always find people who are either very principled (yes, they do exist) or get a kick out of controlling (imprisoning in this case) someone/something, especially someone/something of a high intelligence.
Throw in a hierarchy with a controlling officer and two subordinates who were also trained to watch the officer and I think it's pretty secure.
If only we could select our politicians in the same way.
