If Anyone Builds IT, Everyone Dies They make excellent points, such as the damage possible if AI can spend money. Oh, by the way, Google is building a system to allow AI to spend money. https://www.youtube.com/watch?v=8s6nGMcyr7k Read it. Weep. I'm not sure if the authors have met humanity. https://www.amazon.com/Anyone-Builds-Everyone-Dies-Superhuman/dp/0316595640 |
From my PoV, our "shortish-term profits are the ONLY metric that matters" global techno-capitalist system is vastly more dangerous than AI.
If a person wants a reputation as smart and able to make accurate predictions, why would he focus most of his public statments on a prediction that if he is right, no one will find out definitively that he was right because everyone will be dead?
Bigger picture - I see AI as just another bale of straw on the camel's back. Sure, it's possible that AI will end up looking like the fatal bale. But that could come from so much money being invested in AI that we get a global financial meltdown when the AI bubble bursts, leading to a nuclear WWIII.
What gets me about 'rationalism' is how ahistorical it is. No reference to Korzybski or Teilhard de Chardin. There never was the movie Colossus the Forbin Project or the 1960s speculation that progress in computing would lead to accelerated progress in computing by an equation like
dx 2
-- = x
dt
Pointing a search engine at site:lesswrong.com finds many references to either thinker.