Stuart Russell calls for new approach for AI, a ‘civilization-ending’ technology

Stuart Russell calls for new approach for AI, a ‘civilization-ending’ technology(news.berkeley.edu)

2 points by rdhyee 3 years ago | 3 comments

13years 3 years ago |

The last “civilization-ending technology” – atomic energy – has been the subject of intense governance and extreme care on behalf of its engineers, Russell said. Even less charged technology fields, like aviation, are meticulously regulated. AI should be, too, he said.

It is hard to conceive there could be successful regulation of this space. Atomic energy has a very high barrier to entry due to the physics involved.

AI is already available at consumer level and with the predicted power increase of computing power, if we reach AGI it will likely be in the hands of everyone.

And the best we can do is only a high level conceptual plan of "alignment" for which nobody can actually define that has no real science behind it as of yet. Additionally, I would if there is anything almost provable, it is the fallacy of the alignment theory is itself a paradox, which I describe in more detail here.

https://dakara.substack.com/p/ai-singularity-the-hubris-trap

smoldesu 3 years ago |

> If we pursue [our current approach], then we will eventually lose control over the machines.

What could he possibly mean by this? How does our status quo threaten to make uncontrollable computers?

> AI should instead be designed to further human interests, to recognize it doesn’t know what those interests are, and to seek evidence to identify and act upon those interests

Or just... don't give it control over things with perilous disaster-cases.

> It also needs to be developed in a “well-founded” way, with a rigorous understanding of all the components and how they work together. That will allow us to predict how these systems will behave, he said.

> “I just don’t know any other way to achieve enough confidence in the behavior of these systems,” Russell said.

This is so unbelievably wishy-washy. You could copy the last 2 sentences there and tell me it was a quote about UNIX error handling in the 80s and I wouldn't bat an eye. If you want to be alarmist and call something "civilization-ending", you should at least have evidence to justify how it might do that. If this is our industry-leading rumination on the state of AI, maybe we do have reason to worry.

nh23423fefe 3 years ago | |

It's all incoherent nonsense to me.

"Using enormous compute to process enormous amounts of data to produce an enormous (but compressed representation of a) function, should be illegal. We should instead fail at designing and hand coding these functions so we understand them but never actualize them."

I mean the argument boils down to "inference is bad because I hate the structure we find in all our data, instead lets impose fake ideas and never succeed"