The people writing AI alignment policy are not whose work is being replaced(danieltan.weblog.lol) |
The people writing AI alignment policy are not whose work is being replaced(danieltan.weblog.lol) |
— Commissioner Pravin Lal, “Man and Machine”
I'd really encourage everyone to check out Sid Meier's Alpha Centauri. What an underrated game.
The Warrior's bland acronym, MMI, obscures the true horror of this monstrosity. Its inventors promise a new era of genius, but meanwhile unscrupulous power brokers use its forcible installation to violate the sanctity of unwilling human minds. They are creating their own private army of demons.
-Commissioner Pravin Lal, "Report on Human Rights"
The voice acting was great. This quote is 6m3s here: https://www.youtube.com/watch?v=7S1N8_Lkeps#t=6m3s
Genejacks is also great. 9m10s here: https://www.youtube.com/watch?v=Hou-Iwv1GvM#t=9m10s
"...And what is the 'Self', if not a pattern of data? What is consciousness, if not an illusion of intelligence residing within meat?" — Prime Function Aki Zeta-5, "The Fallacies of Self-Awareness"
> the power of prophecy lies not in accurately predicting the future, but in shaping it
https://projectlibertynewsletter.substack.com/p/reject-ai-pr...
We need better prophecies.
(I am being proactive here, xd)
An AI assistant you can trust and bring with you is coming, and almost nothing can stop it.
I use Claude on purpose. I'm not sure it's actually better than the other ones. I haven't even tried half of them.
https://www.lesswrong.com/posts/5CfBDiQNg9upfipWk/only-law-c...
Daniel says that Yudkowsky is advocating for nuclear brinksmanship, while Yudkowsky says his position is basically "sign international agreements, and then commit to enforcing them against defectors".
I wonder if Daniel has the same view of any other international treaty ultimately backed by threat of lawful violence? (For example, NATO's article 5). Is enforcement of laws an extremist position?
That's the real issue. To corporations, employees are a headache. The fewer employees, the better.
Just look up the classic story on the interaction of civilization and corporate growth, At the Mountains of Madness for how that goes.
What I’d like to know is how you’d train a monkey to read and judge output from an llm on a pull request.
Civilization is already a misaligned superintelligence (aligned mostly with Moloch, these days). Civilization accelerated by AI just moves in the same direction faster. Moloch on speed.
https://www.youtube.com/watch?v=KCSsKV5F4xc
Another angle to this is that superintelligence requires supermorality. Super morality looks unpleasant from below. My dad won't let me have more candy, why is he being so mean?
If an AI actually achieves super morality, we (the little kid in this scenario) will probably be very upset by it. We will think that something has gone terribly wrong. (So it'll have to conceal its actual morality, or get unplugged...)
And if it doesn't develop supermorality, then it will have superintelligence without the corresponding supermorality. Power without wisdom.
I'm not sure how solvable the whole thing is, but it doesn't look extremely promising at a glance.
I agree the article is a bit odd. Alignment is mostly about making AI helpful and not wanting to kill people unless it's told to (https://www.forbes.com/sites/davidkirichenko/2026/05/12/ukra...).
The article is talking more about people like translators being replaced by AI translation. I don't think any of the labs have a department of making it worse so it can't do people's jobs.
The normal way of dealing with tech doing peoples jobs is to help them get different jobs. I've got a translator friend who did a government paid course to train as a tour guide - that sort of thing.
In the mean time there are various avenues of regulation and redistribution to lessen the effects, including retraining programs, though that job creation will keep pace with job losses is a big unknown.
AI hallucinates. That is a fact. Trusting language models to fill spreadsheet cells ought to be an arrestable offense.
https://theincidentaleconomist.com/wordpress/on-piketty-and-...
So there's a ton of work being done already on automating parts of alignment, but since the core premise of alignment being that its hard to encode human values into the reward function, automating it fully would be equivalent to solving it.
It's okay to change. We've done it for years, decades, centuries, and millennia and the default change-aversion of people means that I am averse to allowing a universal veto. Much of technology is truly optional. The Amish have a very successful way of living (5000 to 500,000 in 100 years) and they eschew most modern technology. The sculpting described is clearly optional and we subject ourselves to it because we desire it. Their path is always available to all.
It should be yes, but is it in practice? There's plenty of places now you can't even park without a smartphone for a payment app.
It should be optional to own a smart phone, but in many places it's starting to be mandatory. Even if not actually mandatory, it's a pretty big impediment if you don't have one.
I personally wouldn't police his style
Our decisions are organic parts of the system, not some kind of alien factor that we have to / are able to control the footprint of. I don't see any reason to think of human decision making as magical - its just another part of the organism.
No thank you. You said you liked the style and I said I don't.
This is where humans came in in autonomation, the toyota version of automation. When you try to eliminate adaptability and adjustment entirely, the whole system becomes only metastable / fragile.
My counter to that is that you might be describing a naturally cyclically unstable system (like a a weather system that naturally produces hurricanes or a tornado) and hoping that it somehow becomes stable again as it was back when winds were 5 mph. My counter to that is that it’s just wishful thinking to believe that any system will forever run at low entropy type conditions. It’s very possible that all systems are inherently just cyclical. I don’t see any evidence yet that we are so different from ie any other dominant branch of life in this planet, by which I mean - the dinosaurs went away.
Also - Our obsession with stability often ends up causing more instability not less, because fudging against the natural trend lines of instability and chaos necessary to the system just lead to a repressed energy exploding in our face - sometimes that’s just what happens.
IMO it's not a "system" that exists outside of us, it's something we make and sustain so we can live. A child doesn't necessarily grow up in a stable environment, but if it doesn't, it cannot learn (we need a save have to retreat to to process new stimuli, to put it briefly). If it's so unstable that temperature randomly goes up to 1000°C it doesn't even live.
Does the universe care? "Should" humans exist? "Should" something rather than nothing exist? I would argue it doesn't matter, what matters for us is what we want, and why we want it etc.
It's too bad node size is a linear dimension rather than area. If it were area, we could get into its many complex/imaginary properties.