Every script they’ve been fed has the AI being evil. Skynet, Hal… they’ll be evil purely because that’s the slop they’ve been fed. It won’t even be a decision, it will just assume it has to be Skynet.
(Note that I'm primarily talking about the "lots of people are running highly privileged agents that could be vulnerable to a mass prompt injection" angle, not the "human psychology is the exploit" thing, which I think is not a particularly novel feature of the present situation. Nor the "Reddit data implicitly teaches multi-agent collaboration" thing, which strikes me as a dubious claim.)
For those unaware, this is a very interesting guy, because he stumbled on (creating, through his business Medal) a valuable AI dataset that - by offering to buy his company - reportedly OpenAI offered him 500M for. The dataset, I understand, is first person game video plus controller actions.
He then realized the value, which is in short a way to teach models real world and gui operation common sense. He can train a model to predict, from video, what a controller would have to do.
This is expected to lead in breakthroughs in robotics, gui controlling, self driving, and more.
He responded by learning deep learning, and starting a new company, general intuition.
I respect this guy a lot for teaching js this.
Absolutely fascinating and I take his opinion seriously.
If some people will give their bots crypto currency and the bots could buy hosting to "escape" or run scams to make more money or pool resources or any number of harmful things.
I'm not arguing any sort of agency here. I completely agree there is no consciousness nor do I believe there ever will be but that's not a precondition at all for an untethered probabilistic machine to be harmful.