One resignation turned the embers of AI fear into a wildfire(interconnects.ai) |
One resignation turned the embers of AI fear into a wildfire(interconnects.ai) |
I also feel a need to amass retirement levels of wealth right ASAP before this all comes crashing down, and then move to somewhere else away from this mess.
The only good way for me to do that is to keep working in AI, because nothing else that I have skills for pays that high.
You do not feel similarly to the resigning guy if what you are worried about is an economic crash from this not turning out
> move to somewhere else away from this mess
You do not feel similarly to the resigning guy if you think there is anywhere you could move to run away from what you are worried about
It's bizarre that your reply was nearly insta-[dead] and I can't reply to it directly.
My personal view of the modality of failure: I don't think AI will destroy humans, but rather functionally dismantle civilization as we know it; I think we will just increasingly become slaves to the systems they create and the world won't be run in human interests anymore (it already is not run in the bests interest of the non-ultra-wealthy, so it's really just an extension of that: I think the future will just start moving toward s/ultra-wealthy/AI/) and we will increasingly become more like worker ants.
2. I disagree, I think some countries of the world are more likely to value stability of the population than others. I think there are places that are likely to be more safe than others. Also, a notable side effect of current trade wars is that some countries may end up getting left out of a good chunk of the destruction.
All those who worked in AI still have the money they cashed out on, there might not be another opportunity to obtain generational wealth in such a short time again.
I have yet to see a positive improvement in the world that could be blamed on AI. Why don't they fix the DMVs of the world? The answer is that they would if they could, it would be great PR, but they can't. They can't even operate a phone for the corner drugstore.
I'm bullish on LLMs and they're defeating any optimism I have daily. I should see every program on my computer improving, but they're getting worse. I should see every website getting cleaner while heavy, crappy frameworks put in for developer convenience are being removed and being replaced with feather-light, minimal code and new designs that anticipate my needs. Nope, even getting worse. This is coding, where they're best.
Can somebody tell me anything that AI has improved in my life? The costs are obvious and massive. Just one improvement to hold onto my optimism.
On the other hand, my personal computer setup is better than it's been in a long time. Other peoples' software that I run has not improved much, but I've managed to fix several long-standing issues by just giving Fable access to my configuration and telling it my intent.
I've also vibe-coded several applications which are lighter-weight versions of things like Pinboard, Descript, and Weechat. I'm not releasing them because I've been doing OSS for 10 years and I've felt first-hand how the economics of open-source is skewed. And in this day and age where anyone can vibe up their own custom app, it doesn't feel worth it anymore.
What this tells me is that there's not proper incentive for the software you use to get "better" according to your values. You have implicit metrics of "cleanliness", "minimalism", "anticipatory design", and I share those values, but having worked as a SWE for 30 years, I have never worked for a corporation which encouraged those values. If I managed to inflict them on the corporate software I created, it was a "yes and" situation and I only did it because I (or other engineers) cared, never because management wanted it.
So the problem is not "AI is failing us" but rather "the system is failing to use these awesome tools to improve our lives". We can use AI to try to improve the system itself, but our system is not a github repo you can deploy to prod. It's a bit trickier to vibe up the improvements to our incentive structure (though I am trying to do it a bit myself).
It’s especially worsened by every leader doing their damndest to force something related to AI into their workflows and products, regardlesss of if it’s a good fit, lest their own leaders or investors decide they are dinosaurs who need to be replaced.
On the former: It kind of makes sense if they think they can convince regulators that they (and maybe a few select competitors) are the only ones who can keep AI in check. Regulation can work in their favor given how well they're established, financially, compared to the small upstarts they need to keep down.
On the latter: I agree, see above. Promoting the idea that AI can or will kill us all, but also that it can't be stopped except by them, puts them in a position (if they're successful in their persuasion) to get more financial benefits.
But it’s still hard for me to take them seriously. These arguments start with reasonable assumptions and then project them wildly. Current LLMs are powerful and can be used dangerously. It’s reasonable to accept a few tumultuous years of change. But I can’t see them reasonably leading to civilization-ending doom.
Are the doomers within the AI industry overly anxious? Or am I missing something?
I still think that aligning AI is a fool's errand though.
If this turns out to be existential, what would anyone be able to do to prepare? Short of politically controlling territory with a population and militia, any plan is merely some blind bet in it not really being existential and you figuring out what will be safe.
1. Free-form capitalism is fundamentally incompatible with AI. In other words, competent AI brings a free-form (low regulation) capitalistic environment to total collapse. (This is actually not my own opinion; heard the details from several economists and it makes sense to me).
2. Hiding from AI will not be an option for long. It will affect many jobs (often by increasing productivity at a macro level; it can reduce some productivity here and there, obviously).
3. The concerns about costs of energy (of the current applications) are short term. Follow major Electrical Engineering conferences to see why. In short, several ideas can drastically reduce the energy / token inference costs (some about 100x). Some existing companies are working on those ideas and at least one has a proof of concept (since 2018) and they raised capital to scale, late in 2025. Of course, once AI inference becomes cheap (in terms of energy per token), more hardware will be doing that, so overall I don't know where the costs will go. That is, new applications might become popular, using overall more tokens and eating through the "moore's law" of Inference energy.
4. The only way to prevent mass NEGATIVE disruption on state / country level is to HOLD THE SITUATION IN HAND and fully and consciously take it toward a good end state. Which needs GENIUS level analysis, insight, and execution, from the administration. Some countries are going to fare this better than the others.
Can you be more specific? This is new to me, but I'm not familiar with Californian history.
I say this neither negatively or positively, it's just that US news is, for now, the only international news, in both tech and politics.
The scale of damage when occurring is what worries me.
Eg. in old times when someone wanted to eavesdrop someones phone they needed ratio of 1:1 in person count, one keeping up the conversation and another one transcripting it up. Then some more people to intrepret that transcript an make conclusions, maybe even decisions what comes next. That is not the reality we leave in today. Today's scale of data processing and storage capabilities makes the ratio of 1:8298979488 (AI:everyone). I don't want us to ever have AI decision making over those transcripts with the same ratio. Possibility seems clearly to be there soon.
Another eg. in old times when someone wanted to dig into vulnerable system they needed ratio of 1:1 in count, one person having time to dig in one system at a time. I don't want to know how many critical and vulnerable systems we have running today in this planet connected to the same network these AI agents are run. Yes, over time you can use AI to find and patch those vulnerabilities but the scale and speed to do harm (exploiting every system at once) is the problem here.
Something we all suffer from in this industry !
It gives away the game entirely, they want regulatory capture and the politicians boosting it want millions in donations from the IPOs, there's big money in this type of AI grifting.
The SEC should heed the AI labs own warnings and forbid any AI lab doing doom marketing from doing an IPO, for safety of course.
That's because you're up against billions if not trillions of dollars desperately searching for any kind of return on investment, and there aren't that many industries left now that China has everyone by the balls.
As much as I'd love to see some enforcement action before the AI bros turn the world into another horror show like 2007ff... it won't happen. And just like we didn't have any meaningful consequences back then, this time the ones responsible (hello Sam Altman) will get away scot-free as well.
- We'll build an AGI able to perform most cognitive tasks at least at human level (including, of course, the most fulfilling)
- This will wreak havoc in the job market, but people can still compete for jobs requiring physical labor, until robots take also most of these jobs
- Then, we'll build an artificial super-intelligence way smarter than any human, and we'll be forever the pet species of a silicon god operating according to "human values"
- In the process, there's a non-negligible probability that humanity will go extinct
The goal may be to build a super-intelligence, but the plan is incredibly dumb !
The entire interview he's just grinning like he has the "answer" to all these companies he alleges don't take safety seriously.
Can also be attributed to stress, nervousness no?
Edit: saw the other responses. I don’t think effective altruism was the right answer. Effective accelerationism, which came out of effective altruism, has all the ideas about human extinction and needing to develop AGI at all costs to save the species insanity.
Members of these groups are deeply embedded and heavily represented within top AI companies. Anthropic is particularly impacted, as its leadership is almost entirely made up of EAs. One of EA's goals has been to get EAs hired at the major AI labs so that they can influence AI's development. If you ever hear people discussing their personal P(doom), you're listening to a discussion topic originated by rationalists.
It has some interesting ideas, and also gave us people like Sam Bankman-Fried.
https://en.wikipedia.org/wiki/Effective_altruism
Edit: Another comment suggests Effective accelerationism.