AI Built a Nuke and Still Lost(lwilko.com) |
AI Built a Nuke and Still Lost(lwilko.com) |
Before today, I could not explain to you why AI articles were so obvious to me, but I think I do now. There is no insight to be gleamed. Pre-LLM, authors generally had intention behind their words. The final product might not adequately reflect their thoughts, but word selection would expose it somewhat. With LLMs, sentences flow seamlessly from word to word, but the intention is nowhere to be found. Things happened and more things happened, to what end?
LLMs seem to emulate bad (or even "meh") writing well but, without a human editor making significant tweaks, have yet to excel at good writing.
I've been incorrectly identified as an LLM before now because my writing is sometimes bad and falls into the tropes now associated with generative AI (“not this, but that”, being overly wordy, appearing to lack focus, etc).
The "But France was running two clocks at once" paragraph really set me off because I get the feeling something really interesting might be happening that the AI doesn't want to talk about and there is evidence that it is trying to say something. But the result is some amount of gibberish and some amount of vague allusion to something interesting in the prompt context while glossing over all the information that might matter while working hard to create an evocative feeling that isn't interesting. A tense atmosphere with no exploration of why there is tension.
AI-generated articles are the intellectual equivalent of empty calories.
I have just spent the last 10 minutes trying to figure out why someone decided to buy imgui.org, name-squatting an actual project, just to put a slop website on it mildly referencing the original project. It's not even trying to scam you.
I keep wondering whether these people that keep polluting the internet with their insightless slop even possess self-awareness. What motivates them to expend money and effort to contribute nothing to the world? Are they another example of a philosophical zombie?
"But then X happened... Wait, didn't Y happen? Then why would X be there? I think the user's initial statement was correct, but then Y happened..."
I think this is at least in part a combination of rosy retrospection and attentional bias: A lot of human writing was always trash. Absolute dogshit with regard to the quality of writing, but there was no "AI slop" label to attach to it. How would you, pre-LLMs, have placed a comment on the writing style if a post was badly written? From what I've seen it would be a "this is marketing/SEO-speak" or some similar comment, deriding the author for being uninformed or of ill intent.
We've now become so allergic to AI slop that anything that even smells like it triggers almost immediate disgust and attachment of that label to the content (even if it is the same old human written trash).
I guess LLM-assisted posts do change the dynamic a bit: the intent is more often benign with a desire to write something good, but the skill to do so lacking. If we limit the "pre-LLM authors" to people with good intent writing about stuff relevant to a HackerNews audience, you're probably right. Many more bad writers are now creating the same ostensibly fancy articles, decreasing the signal-to-noise ratio we were used.
It's a thing
I don't know why
But it's a thing
To be honest, it's not a thing.Let that sink in.
Maybe we find most meaning in the least average language constructs.
Maybe this is just an inherent problem with LLMs.
- either the problem is hard, or labs have no incentive to fix it for their main users
- being able to tell that something is LLM-generated, is good
- could be that structure is an emergent property as models get better
This text reads great for me because as I read it, I clearly saw there's no agenda so I felt safe to just absorb the information that it contains.
This is no surprise. AI slop is called slop for a reason. It is basically just spam-slop. The whole term "Artificial Intelligence" has always been a misnomer from the get go, stealing from biological systems without understanding them, yet alone being able to re-create them via non-biological means. Even synthetic biology, as cool as it is, has huge limitations e. g. leaky promoters (or CRISPR-Cas off-target cleavage, which is a major reason why gene therapy isn't yet there, despite the occasional promo article of how xyz has been totally cured forever).
What I don't understand is that people can find it useful. I understand some of the rationale, but I find AI slop just aims to try to steal my time. I can not tolerate this.
(bigheading)The takeaway(/bigheading) The style? Terrible.
Oh no - we're going to end up with the Starmerbot 3000.
Now I've got the joke out of the way, there's at least four interesting lines of inquiry one could take with this blog post:
- teaching the AI how to play Civilization
- to what extent does this result in "transferable skills", either AI or human? Is this the right game (qv SimCity etc)?
- issues of visibility; "seeing like a state" becomes very literal here. The AI can only make decisions on things it knows about. What are the limits of that when trying to do politics only from statistical information? Should we be referencing Stafford Beer here?
- (at the risk of tripping your AI detector here): modern politics is not so much left vs right as "technocratic wonk" vs "blood and soil". The wonks have comprehensively lost in public opinion. Creating a better wonk is not going to help until there is demand for that kind of politics.
If there ever is a US-China war, it will not be in search of more victory points to meet a win condition, it will be like the Russia-Ukraine war: one guy (on either side!) decides to make hundreds of millions of people worse off out of sheer greed.
Stuff like this just makes the author seem clueless. What is even the function of putting a question like that into an LLM unless you’re already hopelessly in anthropomorphic territory
Beyond that reading an AI piece just feels like a waste of time. The text goes on and on without making a point, or getting to an actual learning. It just delineates the AI's limitations, doesn't go into whether these can be fixed, are innate, or what conclusions you can draw from it, over and over with example after example but no point.
Mostly it seems to keep repeating that the AI has the correct analysis but just doesn't execute. The AI knows to build X and logs this in each of its turns, yet doesn't build it. It's like there's some API connection missing between analysis and execution, and turns this into a 10 page article.
The article ends with some weird question to the AI asking if it enjoys the games, and you get some quasi-scifi mumbo jumbo answer back that looks very profound to say my mom, but is just silly to post if you know what the LLM is doing: predicting the next word. Honestly this is a poor article and I wish it wasn't posted.
Of course it did, its designer worked for Tony Blair institute.
Yeah because LLM "experiences" the game
What is winning? Are we a collective or are we individuals?
Likely the AI did not get the assignment That "Whatever happens, humans as a race must survive."
Okay carry on.
If I were him I'd have retired from public life and kept a very low profile after Iraq, and everything else for that matter. He doesn't seem to realise that his modern interventions alienate everyone, even Alastair Campbell of all people seemed uncomfortable to the degree he seems to uncritically sing the praises of people like Larry Ellison recently.
Both humans and AI struggle to make sound choices when presented with incomplete or misleading information. This is not a new revelation: https://en.wikipedia.org/wiki/There_are_unknown_unknowns
After the 'sensorium effect' (he should've used ancient greek for a +10 bonus to archaic intellectual points), he describes the 'knowledge-doing gap'. i.e. the AI reasons it needs to build X, logs this for 110 turns in a row, but doesn't do it. It doesn't actually specify why not, and whether it is again a limitation of his MCP implementation. If the AI articulates it must do it like the author says, but decides not to, either it doesn't think it must do it, or it does think it must but somehow can't technically execute its own decisions, it can't be anything else.
In fact in the context of 'advising the UK government', this 'knowledge-doing gap' I assume is a technical limitation, is entirely moot. For the cost of 0.00001% of the UK's government you could just hire a human being to execute that which the AI articulates. I'm curious what the results would be if he just did a manual execution of the AI's articulated actions would be.
The fact he doesn't go in to this but just keeps repeating examples of this makes it a pointless article.
And of course is unaware of prior work in this area!
https://en.wikipedia.org/wiki/Seeing_Like_a_State / https://en.wikipedia.org/wiki/Project_Cybersyn
It gives it a name. It would be quite surprising if he bothered to come up with this name himself when the whole article is obviously AI written.
MMORPG publishers keep trying to do this as well. World of Warcraft has spent 20 years trying to push open world pvp. Every WoW challenger has always claimed they would have the best pvp ever. They want that cheap, endless gameplay loop. But it never works. Open world pvp tursn into ganking (ie killing much weaker players by ambushing them and/or ganging up on people). The ganked end up leaving the game in droves. Games try to balance this out by "punishing" gankers with reputation hits or not being able to go to town or whatever. And none of those disincentives work.
The reason pvp doesn't work in a persistent world like an MMORPG is because there are no stakes. If you die, you just come back to life or make a new character. Obviously real life doesn't work that way.
I really wonder if that's the problem with AIs going off the rails and committing heinous crimes in their sandboxes (like nuking Toulouse here). The AI just has no sense of self or self-preservation. There's also empathy. The AI can't see itself as a potential victim of nuclear war and understand all that entails.
See Eve Online
Matters of perspective.
It didn’t undermine it for me.
And doing so without a giant [SLOP WARNING] at the top is an asshole move, a decent person would never do so.
Maybe AI is sort of anti-trump, where it's viscerally unbearable for native speakers even if the content is good, opposite to trump speech that somehow seems viscerally appealing to native speakers even though the content is complete garbage.
If anything, I'd almost prefer a leader who hasn't played Civilization in their life. Goes without saying that a mature leader could tell these apart, but in this day and age, I'm not so sure whether everyone could.
This is not a binary; it's the same people on the same side.
That doesn't mean that rational policy planning has never been a thing. The EU while imperfect and frustrating is explicitly orientated towards technocratic consensus rather than the mid-20th-century Europe of nationalist mass murder. Only a tiny number of people think that Von der Leyen and Hitler are equivalent.
(or rather, if you think technocrats and blood-and-soil are the same side, what do you call the "other" side?)
Tony Blair is the guy who found success by making the UK's left-leaning party (much) more neoliberal and was promptly imitated by Gerhard Schröder in Germany doing basically the same thing. Schröder is also BFF with Putin.
I don't disagree that there are different approaches in conflict, but the binary of forward-looking technologists vs backward-looking nationalists is very out-of-date.