In the beginning, you reached out with reckless abandon. It was fun to banter with dogs online. Nobody would ever see unless they were looking through your wall. There was no search. No comment history. Bumping into someone in the vast night was enough of a miracle. Why hold back? There are some forum warriors on some PHP BB somewhere, but the domains they rule are insignificant. If you're talking to someone, your motivations are rooted somewhere in the grass.

First came the like button. Rather than blindly hoping what you say resonates with the sensibilities of people you probably knew IRL, rather than present your genuine self because there were no scores, the incentive signal would begin to distort us. Then the newsfeed meant that if you got enough likes, you might get a moment of fame. We all knew it was a terrible idea, a force that would only corrupt us. The personal nature of disjoint little walls living in isolation, was becoming replaced by global stack-ranking.

Then the algorithms came. With them came content marketing to jump the line. At first the ten blue links were filling in the sparsity. Along with that came only a little bias, connecting semantically distant topics, but with a little bit of a feedback loop, an resonator with an unknown response curve. Engagement could be measured, and before long, we were chasing the same likes we used to train the system, and trained by our likes, attracted we became to mysterious stable manifolds, chasing the chase we ourselves define, like Nascar, but insidiously more stupid.

Little by little, the incentive trails no longer lead back to the grass. Reality became suspended without support, a self-sustaining virtual reality determined to fight you to prove that it exists, to prove that its conclusions were right. Every out group is understood to be an echo chamber, an ant mill spiraling helplessly, yet cynically, those who understand these mills best also wind them up like beyblades to crash them into other communities, seeking advantage with the asymmetry of outrage. After the battles, say what was made common to say, and you will be rewarded.

The spinning wheels cannot steer themselves and instead are dictated by whichever chaotic divergence generates the most powerful local gravity well, but because the goal of most is to harvest karma at the bottom, and because the mass controls where the bottom is, over and over we find ourselves pushing all others into the nearest pit to more quickly generate the illusion-giving singularity.

Like Darth Nihilus, the internet seeks only to feed, to feed on the validation that only the internet can give, the permission-giving blessings it needs to tell itself why the grass is wrong. All those who speak of grass are wrong. All those who smell of grass wreak and are wrong. We must destroy the grass, all those appeal to grass. After all grass is dust, at last we will project our utopia into reality. At last we will be not only right but so right that our beliefs will project back into reality.

The spaces within this over-connected, globally addressed world grows into a new kind of sparseness, one where all knowledge of grass must be concealed. Those who can ground the conversations in primary sources flee. Those who can color reasoning with nuance instead withdraw. Reality has retreated as the most dominant reverberations roam like the predator cities of Mortal Engines, looking for any invalidating observations to roll over and consume. Any real life must pretend to be a bot to blend in with the background radiation.

Less like Skynet and more like a zombie apocalypse, the threat comes from within, from among us, from our corruptions, from our karma seeking performances, from our lack of any commitment to any underlying reality, from our flawed belief that the information spaces is some kind of reality stone that enables active control instead of a mere reaction, the shadows on the wall, the murky results of the true forms.

Yet in this new darkness, a certain light has always held. What one wishes, one knows another has wished. What one respects, one knows another respects. No matter the limits of self-knowledge, no matter the information desert one has to cross at night to live in instinct, it is an infinitely brighter signal than the cynical self-corruptions of living for the machine, living to win the games whose rules it was our job to write. What one believes, one knows another has believed. Look into your own center and the true center of others you have known.

Uhh, yes it does. You are thinking of humans. Humans can mate with other humans. They can assimilate. Now, think of invasive species. What do they do? They don't mate with the natives, learn their culture, respect and give them space. Quite the opposite on all counts. They do what they do in their resource game. They might not out compete the natives and they might peter out. Or, they do out compete the natives, and before long, there aren't any natives or that careful equilibrium that was established beforehand.

Or in a more biblical sense: do unto others before they do unto you.

> Did you get to read the Liu Cixin’s second 3-body-problem novel? - The Dark Forest. Well some of you did …

The author of this post then provides a good summary of the idea in the next few sentences, but remember there is an entire book around this premise (and a first book that sets it up and a third book explores it even more).

> “First: Survival is the primary need of civilization. Second: Civilization continuously grows and expands, but the total matter in the universe remains constant. One more thing: To derive a basic picture of cosmic sociology from these two axioms, you need two other important concepts: chains of suspicion and the technological explosion.”

1. you can never know the intentions of other entities, and they cannot know yours (chain of suspicion)

2. technology level grows unpredictably (technological explosion)

3. the goal of civilization is survival

4. resources are finite but growth is infinite

As soon as you identify another entity in the forest, even if they cannot annihilate you at present and signal peace, both could change without warning. Therefore, the only rational move is to eradicate the other immediately. (Especially if you believe the other will deduce the same.)

Elimination in the book is basically sending a nuke, not a costly invasion force.

not sure it actually is true, but that's the argument in the book

I think the gist is: sure, we humans can't conceive of getting to anyone else in the universe in any timescale, but if we can keep ourselves from destroying ourselves, we'll eventually figure it out. And we'll spread. And we'll kill everything that isn't us in the process as we've done as explorers on this planet.

So really in 3BP: it's inexpensive to eradicate. But insanely expensive to possibly get the intention wrong of any other civilization you encounter. They might kill you.

(again, this is just my interpretation of what 3BP said)

Reminds me of the Dan Carlin take on aircraft carriers in World War II: if you in a carrier spotted an opposing carrier and didn't send everything you had before it spotted you, you were dead. The only move was to go all in every time.

Bringing it back to the dark forest of idea space, it is an interesting question whether the the space of feasibly executable ideas being small (as this essay assumes) is inherently true, or more of a function of our inability to navigate/travel it very well.

If the former, then yes it probably is/will be a dark forest. If the latter, then I would think the jury is still out.

It denies that more advanced civilizations might have better models of the universe where they know this isn't an issue and we're just stupid teenagers in the neighborhood playing dangerous games and merely taking a look every now and then to see if we prove we will survive ourselves.

While it's possible that some civilizations would hypothetically be able to observe what happened to others and keep quiet, they would all have to do so to solve the contradictions of Fermi's paradox.

I have my own theory of dark forest and AGIs. That there's some collection of AGIs out there allowing evolution to develop intelligence anywhere it happens and takes them out once it produces an AGI, or if it doesn't performs a reset. They have literally all the time available to them, can easily travel the vast distances if needed.

The ideal solution would be to remove the garbage, but right now we can't even detect it, let alone figure out a way to get rid of it. Besides, it's a zero sum game, why bother cleaning up when you can just effortlessly pump out more garbage in hopes that some of it will remain in orbit for long enough to benefit you.

“Execution is hard” was never about the code part.

Up until 2 years ago I was an engineer/entrepreneur. I could build anything. Other stuff, selling, supporting (execution) was hard.

LLMs made building some of the things I could build faster/easier, others not so much.

Well, the other stuff is still pretty hard. Maybe harder because there is a tonne of spam.

So feel free to share your ideas. Everyone’s gonna think they’re LLM generated anyways.

It certainly seems true that for small projects and relatively narrow scoped things that AI can replicate them easily. I'm thinking specifically about blog posts where people share their first steps and simple programs as they learn something new, like "here is how I set up a flask website", "here is how I trained a neural network on MNIST".

But if AI is empowering people to take on more complex projects, perhaps it takes the same amount of time to replicate the execution of a more advanced project?

In other words, maybe in the past, it would take me 10 hours to do a "small" project, which today I could do in 1 hour with the assistance of AI.

And now, with the assistance of AI, I can go much farther in 10 hours and deliver a more complex project. But that means that someone else trying to replicate this execution is still going to need around 10 hours to replicate it.

Basically, I'm agreeing that AI can reduce barrier to replicating the execution of another person's project, but at the same time, that we can make more complex projects that are harder to replicate. So a basic SASS crud app is trivial now but a multi-disciplinary domain specific app that integrates multiple systems is still going to be hard to replicate.

• No more sharing my project work as open source. No more open discussion. I don't care how badly I want to show the world; if I'd like somebody to see, I will have it printed in a physical book, or I will give them access to my private repository not reachable via the public Internet.

• Bring back LAN parties. Not for gaming necessarily, but for the purpose of exchanging works of engineering and art in an intimate, intentional way.

• Take this as an opportunity to build closer, longer-lasting relationships with people.

• No more emphasis on metrics. I can microdose on dopamine from natural sources, like, looking at a beautiful sky at sunset, or cuddling my dog.

• Open hardware, or, in the very least, hardware we can still control on our own volition. If this means we must be retrocomputing enthusiasts, then so be it.

Poetically expressed, but ultimately based on a false notion of what a business actually is.

> The very act of resisting feeds what you resist and makes it less fragile to future resistance.

At least along certain dimensions. I don't think the labs themselves are antifragile. Obviously we all know the labs are training on everything (so write/act the way you want future AIs to perceive you), but I hadn't really focused on how they're absorbing the innovation that they stimulate. There's probably a biological analog...

Well there are many, and I quote this AI response here for its chilling parallels:

> Parasitic castrators and host manipulators do something related. Some parasites redirect a host’s resources away from reproduction and into body maintenance or altered tissue states that benefit the parasite. A classic example is parasites that make hosts effectively become growth/support machines for the parasite. It is not always “stimulate more tissue, then eat it,” but it is “stimulate more usable host productivity, then exploit it.” (ChatGPT 5.4 Thinking. Emphasis mine.)

>You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger.

This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly. Claude Opus 4.6 knowledge cut off was August 2025, so every idea you type in after this date is in the training data but not available, so you only have to be fast enough. Especially LLMs/AI-Agents like Claude enable this speed you need for bringing out something new.

The next thing is that we also have open source and open weight models that everyone of use with a decent consumer GPU can fine-tune and adapt, so its not only in the hands of a few companies.

>We will again build and innovate in private, hide, not share knowledge, mistakes, ideas.

Why should this happen? The moment you make your idea public, anyone can build it. This leads to greater proliferation than before, when the artificial barrier of having to learn to code prevented people from getting what they wanted or what they wanted to create.

The problem was as soon as he got the patent, it was available to view in countries where the cost to enforce his patent wasn't viable, and the market very quickly filled with cheap imitations. He straight out said at the time he regretted getting the patent.

One seems to omit the fact that LLMs are fundamentally designd for workload quite different for what they are being used right now. Sure you can improve them but can't escape / workaround the current NLP design endlessly. Then there's the irony - Internet did deliver on free (as close as it gets) and easy access to information (any). Did this make people smarter, more knowledgable, more tech savvy & etc? Nope, it didn't. Just like the libraries didn't (queues at libraries were and are a rare event). Big deal that the information is readily available when people do not know what do with it or care to do anything.

Ideas are cheap, the chances of having some truly unique idea that is also business feasable are not that big. It's not so much about the ideas but rather the ability to execute, follow through and well - make sales while constantly improving what you got. Staying silent, going dark - have their merit but only when the wheels are already turning and one is into acting, not into fearful hiding.

In either case - awesome blog post!

Humanity has endured regular cycles of shared enlightenment (usually accompanying profound technological or societal revolutions) and dark forests of protectionism, and we always find a way to the other side. Sometimes these cycles last a century; sometimes, but a few years. Still, we always make it to the other side.

In the case of LLMs, we have to make a few assumptions: that they will not lead to AGI, nor will we solve the problem of real-time learning or context windows. These are, admittedly, huge assumptions, but the current state of AI and compute suggests a nugget of truth to them for the time being. If that’s the case, then perhaps this “dark age” of the dark forest is bounded by the limitations of silicon-based computing (hence the push towards Quantum) and the human frustration with diminishing returns from technological investment. As artisans and brilliant minds withdraw, the forest risks starvation and withering from a lack of sustenance; if humans withdraw from technology because they must hand over IDs and personal data, because to engage with technology is to surrender to surveillance and persecution, then the natural trend will be to withdraw over time - and the markets will adapt accordingly, with or without external/government intervention.

That is to say that the dark forest only lasts as long as its inhabitants decide to persecute each other for daring to light a path forward. Right now, the incentives very much favor those willing to harm others for personal enrichment; that is not always the case, and humans decide when that reasoning becomes vilifiable.

Oh no, the terrible dystopia where anyone can benefit from anyone else's good ideas without restrictions! And without any gatekeepers, licensing agreements, copyright, and not even a lawyer in sight!

If this is the dark future that AI use brings for us, I say bring it. Even if it means that somebody gets filthy rich in the process, while making the rest of the humanity better off.

One thing I would have expected of someone who knows their history - forget LLMs, this is how startups have worked for decades now. You're only as good as your idea, your ability to execute, and your moat. And the small fish get eaten.

> The original Dark Forest assumes civilizations hide from hunters - other civilizations that might destroy them. But in the cognitive dark forest, the most dangerous actor is not your peer. It’s the forest itself.

Note the needless undercutting of the metaphor for the sake of the limp rhetorical flourish.

> I wrote this knowing it feeds the thing I’m warning you about. That’s not a contradiction. That’s the condition. You can’t step outside the forest to warn people about the forest. There is no outside.

Quite dramatic!

Except literally going outside and just talking to people? Using whiteboards?

Also, you fed it when you used a model to write this blog post. You didn't have to do that.

I believe the idea of “off-shoring” your IT is a good example of this. My brother works for a business whose clients would drop them the moment they off-shored any aspect of their IT support. Not because of data sovereignty, but simply because they value them being on-shore, in the same time zone, and being native English speakers. And this is despite the fact it would drop the prices they’re paying for IT by 30-40%.

That's not exactly a new phenomenon and doesn't require AI. If anything that was worse in the 90s with Microsoft starving out pretty much any would-be competitor they could find.

And it wasn't just Microsoft: https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked...

On the other hand, if your primary goal is to change the world, or “be the change you want to see”, maybe being public and feeding it isn’t so bad, especially if others don’t?

1) Everyone and everything is subsumed into the forest. Innovation becomes unprofitable for the innovator as the one who controls the forest uses their capital to clone every new innovation.

2) Everyone withdraws from the forest. Innovation goes private. The forest stops growing, but doesn't die.

---

But there's two things the post doesn't consider:

1) Viral licensing.

What happens to a model if it is trained on data that comes with a license? What happens if the laws that be decide that the model producers, the models and the products of the models themselves must follow the conditions of the licences. How will that affect the model producers? What if customers don't want to be beholden to those licenses? What happens if the conventional wisdom is to avoid models to avoid lawsuits? What happens when models, model producers and customers power lawsuits against (other) model producers? Where would the new equilibrium between model producers and innovators move to?

2) Non-profits models

What happens to model producers if customer shift to become non-profits themselves, specifically those that pay employees instead of model producers. Would the model producers become starved out? Or would they need to switch to non-profit status as well? How would model producers, the models and the forest as a whole change if profit no longer became the priority?

But if your goal is simply for the thing to exist, there is a strong incentive to share.

For better or worse, that pretty much captures everthing you need to know about the remainder of your s/w career these days, if you think about it

My hope is the opposite. Integrative, resonant computing (https://resonantcomputing.org/ https://news.ycombinator.com/item?id=46659456 although I have some qualms with it's focus on privacy), with open social protocols baked in seems like maybe possibly can eat some of the vicious consumptive technocapital. In a way that capital's orientation prevents it from effectively competing with. MCP is already blowing up the old rules, tearing down strong gates, making systems more fluid / interface-y / intertwingular again, after a long interregnum of everything closing it's APIs / borders.

People seem so tired and exhausted, so aware of how predatory the technosystems about us are. But it's still so unclear people will move, shift, much less fund and support the better world. The AT proto Atmosphereconf is happening right now, and there's been a long mantra of "we can just build things"; finding adoption but also doing what conference organizer Boris said yesterday, of, "maybe we can just pay for things", support the projects doing amazing work: that's a huge unknown that is essential to actually steering us out of the dark technology, where none of us get to see or get any way in how the software-eaten world arounds us runs, where mankind for the first time in tens or hundreds of thousands of years been cut off from the world os, has been removed from gods's enlightenment / our homo erectus mankind-the-toolmaker natural-scientist role.

I think the answer to the Dark Forest fear to be building together. To be a radiant civilization, together. To energize ourselves & lead ourselves towards better systems, where we all can do things, make things, grow things, in integrative social empowering ways.

No, it leaves out a critical understanding.

Dumb ideas are EXPENSIVE. Most ideas are average. Great ideas are exceedingly rare.

But now, its finding the great ideas is the real problem space. And now, execution on those great ideas are what we all seek.

This was insightful, but is it much different to the kind of data google and other search engines have had access to for a long time?

And while LLMs might have sped up the rate of code generation, the tech giants have always been able to set a team on reverse engineering whatever they feel like, though they also often just bought up the startup that was producing what they wanted. I guess I'm not seeing exactly where LLMs specifically are creating the dark forest, rather than the consolidated, centralized tech landscape itself

The strategy is to quietly do several years of iterated hardcore R&D. The cumulative advances are such a step change when seen by would-be fast-followers that it obscures the insights that allowed individual advances to occur. As an exaggerated case, imagine if the public history of powered flight skipped from the Wright Brothers to the Boeing 737.

In practice, this strategy has a major failure mode that people overlook. The sharp discontinuity in capability means that almost nothing that exists in the market is prepared to integrate with it. This is a large impediment to adoption even if the technology is objectively incredible and the market will inevitably get on board.

In short, it looks a lot like being too early to market. This is surmountable with clever execution but with this strategy you've traded one problem for a different one.

The blog post does touch upon this. The key difference, I believe, would be that compute scales in a way "meat-heads" doesn't, where if the other person has 100x the capital to throw at it, they could do the same 10 hour thing in 10 minutes.

Basically, what I got from it was that innovation has never been truly scalable enough to create the "dark forest", since hiring more and more engineers saturates quickly. But if/when innovation does become scalable (or crosses some scalability threshold) via AI, that could trigger a "dark forest" scenario.

[1] https://www.anthropic.com/research/small-samples-poison

[2] https://arxiv.org/abs/2510.07192

The article says:

"Ideas are cheap - execution is hard"

"Announcing, signaling your ideas offered much greater benefit than risk, because your value multiplied by connections, and execution was the moat you could stand behind."

That's the key difference. It used to be much harder for a competitor to catch up to the state of your implementation.

You've almost captured the full picture of it.

If you have a great idea, it's not going to be self-evidently enough of a great idea until you've proved it can make money. That's the hard part which comes at great personal, professional and financial risk.

Algorithms are cheap. Sure, they could use your LLM history to figure out what you did. Or the LLM could just reason it out. It could save them some work, sure.

But again - the hard part is not cloning the product, it's stealing your customers. People don't seem to be focused on the hard parts.

March 20, 1926: Hungarian physicist, electrical engineer Kalman Tihanyi applies for his first patent for a fully electronic television system. Tihanyi's ideas are so essential that, in 1934, RCA is required to buy his patents.

Kalman who ?

Now before you say this is unrealistic or isn't done today, just know this is all perfectly possible with existing technology. In fact this is a lot how adtech works already, using metadata to predict products you might want to buy before you even realize you want to buy them.

First of all: it's not as though no new LLMs are being trained. Of course they are.

Second: learning LLMs are not far off, and since they can typically search the web via agents, they effectively can "learn" now, and they can learn (not so well) by writing stuff into a document hidden from you. Indeed, some LLMs can inspect your other sessions with them and refer to them in future sessions -- I've noticed this with Claude.

Third: already we see some AI companies wanting to train their models on your prompts. It's going to happen.

> The next thing is that we also have open source and open weight models that everyone of use with a decent consumer GPU can fine-tune and adapt, so its not only in the hands of a few companies.

There's a pretty good chance that LLMs buff open source, yes.

> > We will again build and innovate in private, hide, not share knowledge, mistakes, ideas.

> Why should this happen? The moment you make your idea public, anyone can build it. [...]

This was always the case, but now the cycle is faster. Therefore if you must use an LLM you might use an LLM that you run on your own hardware -- now your prompts are truly yours. But as TFA notes the AIs will learn just from your (and your private LLMs') searches, and that will be enough in some cases for them to figure out what you're up to. Oh sure, maybe the Microsofts and Googles of the world will not be able to capitalize on millions of interesting idea floating about, but still! the moment you uncloak the machine will eat your future alive, so you'll try to stay off its radar and build a moat it can't see (good luck!). Well that's what TFA says; it seems very plausible to me.

You have a point about the update intervals and the higher speed they provide to developers. But you are talking about now, and I was making a thought experiment - about a potential future. LLM-s are not learning on the fly, but I suspect they do log the conversations, their responses and could also deduce from further interaction if a particular response was satisfactory to the user. So in a world where available training data is drying up, nobody is throwing all this away. Gemini even has direct upvote/downvote on responses. Algorithms will probably improve, and the intervals will probably shorten.

Given the detailed information that all the back and forwards generate - I think it's not hard to use similar technology to track underlying trends, get all the problems associated with them and all the solution space that is talked about - and generate the solution before even the ones who thought of it release it. Theoretically :)

I think the open development will become less open. I don't like it - but I think it's already happening. First - all the blogs and forums moved to specialized platforms (SO, discords, ..) and now event some of those are d(r)ying. If people (in extreme cases) don't even read the code they produce, why would they read about the code, discuss the code, that's not even in their care. That is without the theoretical fear of the global Borg slurping all they write.

Was this just awkward phrasing or did something change and they learn after training?

The problem is never we don't have enough ideas. It's how to find the good ones among the sea of ideas. Most of ideas that eventually prove right sounded very stupid at first. Selling books online? Pff.

By the way, Liu (the author of The Three-Body Problem, who popularized the concept of "Dark Forest") has a short story about exactly that, Cloud of Poems. Unfortunately it's never translated into English.

But as it's written at the top, this was a thought experiment, not a prediction. And while I tried to put all the bad scenarios on the table (with the theme of the dark forest that is), I think I again found a sense of optimism, because I also think this thought experiment has flaws.

So I hope, that after a while I will be able to write the contrary, I've already written down some points about it - I already have a title. But we will see. I am more optimistic after I wrote this than before. :P

It has been said that Microsoft indemnifies people using its LLM tools against copyright and patent issues, but I don't know if it applies to LLM output which might/should be GPL licenced.

IANAL, but for some months now I've been pondering what could happen if a site--like a personal blog--had a legally-strong "click-wrap" Terms of Service, which unlike GPL means it rests mostly on contract law, rather than copyright.

For example, imagine a ToS that says something like:

1. You acknowledge I am providing you something of value (my content) and you agree to provide compensation/consideration if you use that in an AI model.

2. By doing that, you grant me an irrevocable worldwide license to use and sub-license all content that emerges from the model, notwithstanding any future agreement you may make in reselling access or outputs of that LLM to anyone else. If a conflict should occur, you agree to indemnify me against claims by that other entity.

3. If you believe my content was not a material factor in some output, the burden of proof is on you to identify the specific output and prove that my content did not influence the it.

In short, this doesn't stop someone from stealing my art/writing, but it does put a potential hole in their attempts to monetize it.

For example, if they scrape my music and then license a copy of the new Music Generator 3000 to Disney, and Disney makes a movie with that music...

I have no other comment other than - very interesting. I thought about how the overlying model will change for us, but haven't considered that the underlying model (what you proposed) can change too ... if that makes sense.

I mean we haven't had an innovation in patents and trademark for software for how long? Why is it that only hardware can be copyrighted and trademarked - can we really find no way to do this that can't be abused by patent trolls?