We know it's a sword. And there's war, yadda yadda. However, let's do the cultivating thing instead.

What other AI players we need to convince?

sam altman: "OpenAI is not a normal company and never will be."

Hmmm

More crucially, since OpenAI's founding and especially over the past 18 months, it's grown increasingly clear that AI leadership probably won't be dominated by one company, progress of "frontier models" is stalling while costs are spiraling, and 'Foom' AGI scenarios are highly unlikely anytime soon. It looks like this is going to be a much longer, slower slog than some hoped and others feared.

Then why is it paywalled? Why are you making/have made people across the world sift through the worst material on offer by the wide uncensored Internet to train your LLMs? Why do you have a for-profit LLC operating under a non-profit, or for that matter, a "Public Benefit Corporation" that has to answer to shareholders at all?

Related to that:

> or the needs for hundreds of billions of dollars of compute to train models and serve users.

How does that serve humanity? Redirecting billions of dollars to fancy autocomplete who's power demands strain already struggling electrical grids and offset the gains of green energy worldwide?

> A lot of people around OpenAI in the early days thought AI should only be in the hands of a few trusted people who could “handle it”.

No, we thought your plagiarism machine was a disgusting abuse of the public square, and to be clear, this criticism would've been easily handled by simply requesting people opt-in to have their material used for AI training. But we all know why you didn't do that, don't we Sam.

> It will of course not be all used for good, but we trust humanity and think the good will outweigh the bad by orders of magnitude.

Well so far, we've got vulnerable, lonely people being scammed on Facebook, we've got companies charging subscriptions for people to sext their chatbots, we've got various states using it to target their opposition for military intervention, and the White House may have used it to draft the dumbest basis for a trade war in human history. Oh and fake therapists too.

When's the good kick in?

> We believe this is the best path forward—AGI should enable all of humanity^1 to benefit each other.

^1 who subscribe to our services

---

## *What Has Changed*

### 1. *OpenAI’s For-Profit Arm is Becoming a Public Benefit Corporation (PBC)*

* *Before:* OpenAI LP (limited partnership with a “capped-profit” model). * *After:* OpenAI LP becomes a *Public Benefit Corporation* (PBC).

*Implications:*

* A PBC is still a *for-profit* entity, but legally required to balance shareholder value with a declared public mission. * OpenAI’s mission (“AGI that benefits all humanity”) becomes part of the legal charter of the new PBC.

---

### 2. *The Nonprofit Remains in Control and Gains Equity*

* The *original OpenAI nonprofit* will *continue to control* the new PBC and will now also *hold equity* in it. * The nonprofit will use this equity stake to fund “mission-aligned” initiatives in areas like health, education, etc.

*Implications:*

* This strengthens the nonprofit’s influence and potentially its resources. * But the balance between nonprofit oversight and for-profit ambition becomes more delicate as stakes rise.

---

### 3. *Elimination of the “Capped-Profit” Structure*

* The old “capped-return” model (investors could only make \~100x on investments) is being dropped. * Instead, OpenAI will now have a *“normal capital structure”* where everyone holds unrestricted equity.

*Implications:*

* This likely makes OpenAI more attractive to investors. * However, it also increases the *incentive to prioritize commercial growth*, which could conflict with mission-first priorities.

---

## *Potential Negative Implications*

### 1. *Increased Commercial Pressure*

* Moving from a capped-profit model to unrestricted equity introduces *stronger financial incentives*. * This could push the company toward *more aggressive monetization*, potentially compromising safety, openness, or alignment goals.

### 2. *Accountability Trade-offs*

* While the nonprofit “controls” the PBC, actual accountability and oversight may be limited if the nonprofit and PBC leadership overlap (as has been a concern before). * Past board turmoil in late 2023 (Altman's temporary ousting) highlighted how difficult it is to hold leadership accountable under complex structures.

### 3. *Risk of “Mission Drift”*

* Over time, with more funding and commercial scale, *stakeholder interests* (e.g., major investors or partners like Microsoft) might influence product and policy decisions. * Even with the mission enshrined in a PBC charter, *profit-driven pressures could subtly shape choices*—especially around safety disclosures, model releases, or regulatory lobbying.

---

## *What Remains the Same (According to the Letter)*

* OpenAI’s *mission* stays unchanged. * The *nonprofit retains formal control*. * There’s a stated commitment to safety, open access, and democratic use of AI.

Is OpenAI making a profit?

This indicates that they didn't actually want the nonprofit to retain control and they're only doing it because they were forced to by threats of legal action.

So were do I vote? How do I became a candidate to be a representative or a delegate of voters? I assume every single human is eligible for both, as OpenAI serves the humanity?

     free (foss) -> non-profit -> capped-profit -> public benefits corporation -> (you guessed it)

(1) be transparent about exactly which data was collected for the model

(2) release all the source code

If you want to benefit humanity, then put it under a strong copyleft license with no CLA. Simple.

Musk claimed Fraud, but never asked for his money back in the brief. Could it be his intentions were to limit OpenAI to donations thereby sucking the oxygen out of the venture capital space to fund Xai's Grok?

Musk claimed he donated $100mil, later in a CNBC interview, he said $50-mil. TechCrunch suggests it was way less.

Speakingof humanitarian, how about this 600lbs Oxymoron in the room: A Boston University mathematician has now tracked an estimated 10,000 deaths linked to the Musk's destruction of USAID programs, many of which provided basic health services to vulnerable populations. He may have a death count on his reume in the coming year.

Non profits has regulation than publicly traded companies. Each quarterly filings is like a colonoscopy with Sorbonne Oxley rules etc. Non profits just file a tax statement. Did you know the Chirch of Scientology is a non-profit.

The newer version included sponsored products in its response. I thought that was quite effed up.

Key Structure Changes:

- Abandoning the "capped profit" model (which limited investor returns) in favor of traditional equity structure - Converting for-profit LLC to Public Benefit Corporation (PBC) - Nonprofit remains in control but also becomes a major shareholder

Reading Between the Lines:

1. Power Play: The "nonprofit control" messaging appears to be damage control following previous governance crises. Heavy emphasis on regulator involvement (CA/DE AGs) suggests this was likely not entirely voluntary.

2. Capital Structure Reality: They need "hundreds of billions to trillions" for compute. The capped-profit structure was clearly limiting their ability to raise capital at scale. This move enables unlimited upside for investors while maintaining the PR benefit of nonprofit oversight.

3. Governance Complexity: The "nonprofit controls PBC but is also major shareholder" structure creates interesting conflicts. Who controls the nonprofit? Who appoints its board? These details are conspicuously absent.

4. Competition Positioning: Multiple references to "democratic AI" vs "authoritarian AI" and "many great AGI companies" signal they're positioning against perceived centralized control (likely aimed at competitors).

Red Flags:

- Vague details about actual control mechanisms - No specifics on nonprofit board composition or appointment process - Heavy reliance on buzzwords ("democratic AI") without concrete governance details - Unclear what specific powers the nonprofit retains besides shareholding

This reads like a classic Silicon Valley power consolidation dressed up in altruistic language - enabling massive capital raising while maintaining insider control through a nonprofit structure whose own governance remains opaque.

I was trying to put all the text into gpt4 to see what it thought, but the select all function is gone.

Some websites do that to protect their text IP, which would be crazy to me if that’s what they did considering how their ai is built. Ha

It generally means broadening access to something. Finance loves democratising access to stupid things, for example.

> word is a homonym of its antonym?

Inflammable in common use.

Checks-and-balances need to be robust enough to survive bad people. Otherwise, they're not checks-and-balances.

One of the tricks is a broad range of diverse stakeholders with enforcement power. For example, if OpenAI does anything non-open, you'd like organizations FSF, CC, and similar to be represented on their board and to be able to enforce those rules in court.

If you see a post that ought to have been moderated but hasn't been, the likeliest explanation is that we didn't see it. You can help by flagging it or emailing us at hn@ycombinator.com.

At the same time, though, we need you (<-- I don't mean you personally, but all commenters) to follow HN's rules regardless of what other commenters are doing.

Think of it like speeding tickets [2]. There are always lots of other drivers speeding just as bad (nay, worse) than you were, and yet it's always you who gets pulled over, right? Or at least it always feels that way.

[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

I'm a citizen, the laws of politics are the problem.

> Musk is a complex figure

Hogwash. He's greedy. There's nothing complex about that.

> and he often exacts a tool on the people around him

Yea it's a one way transfer of wealth from them to him. The _literal_ definition of a "toll."

> When he goes into "demon mode"

When he decides to lie, cheat and steal? Why do you strain so hard to lionize this behavior?

> you don't want to be in his way.

Name a billionaire who's way you would _like_ to be in. Elon Musk literally stops existing tomorrow. A person who's name you don't currently know will become known and take his place.

His place needs to be removed. It's not a function of his "personality" or "particulars." That's just goofy "temporarily embarrassed billionaire" thinking.

What have I misunderstood? Help me understand. What is the key point you want to make that you think I misunderstand?

>> (me) When he goes into "demon mode"

> When he decides to lie, cheat and steal? Why do you strain so hard to lionize this behavior?

I hope this is clear: I'm not defending Musk's actions. Above, I'm just using the phrase that Walter Isaacson uses: "demon mode". Have you read the book or watched an interview with Isaacson about it? The phrase is hardly flattering, and I certainly don't use it to lionize Musk. Is there some misunderstanding on this part?

>>>> (me) But to speak plainly, Musk is a complex figure, frequently problematic, and he often exacts a tool on the people around him. Part of this is attributable to his wealth, part to his particulars. When he goes into "demon mode", to use Walter Isaacson's phrase, you don't want to be in his way.

>> (me) Where in my comment do I lionize Musk?

> You attribute to personality what should be attributed to malice. You do this three times.

Please spell this out for me. Where are the three times I do this?

Also, let's step back. Is the core of this disagreement about trying to detect malice in Elon's head? Detecting malice is not easy. Malice may not even be present; many people rationalize actions in such a way so they feel like they are acting justly.

Even if we could detect "malice", wouldn't we want to assess what causes that malice? That's going to be tough to disentangle with him being on the Autism spectrum and also having various mental health struggles.

Along with most philosophers, I think free will (as traditionally understood) is an illusion. From my POV, attempting to blame Musk requires careful explanation. What do we mean? A short lapse of judgment? His willful actions? His intentions? His character? The overall condition of his brain? His upbringing? Which of these is Elon "in control of"? From the materialist POV, none.

From a social and legal POV, we usually draw lines somewhere. We don't want to defenestrate ethics or morality; we still have to find ways to live together. This requires careful thinking about justice: prevention, punishment, reintegration, etc. Overall, the focus shifts to policies that improve societal well-being. It doesn't help to pretend like people could have done otherwise given their situation. We _want_ people to behave better, so we should design systems to encourage that.

I dislike a huge part of what Musk has done, and I think more is likely to surface. Like we said earlier -- and I think we probably agree -- Musk is part of a system. Is he a cause or symptom? It depends on how you frame the problem.

It might not be a direct US-govt project like the Manhattan Project was, but it doesn't have to. The government has the ties it needs with the heads of all these AI companies, and if it comes to it, the US-govt has the muscle and legal authority to reign control over it.

A good deal for everyone involved really. These companies get to make bank and technology that furthers their market dominance, the US-govt gets potentially "Manhattan project"-level pivotal technology— it's elites helping elites.

replace LLMs with TV, or smartphones, or maybe even mcdonald's, and you've got the same idea. through TV, corporations got to control a lot of the social world and people's behavior.

Google screamed against service revenue and advertising while building the world's largest advertising empire. Facebook screamed against misinformation and surveillance while enabling it on a global scale. Netflix screamed against the overpriced cable TV industry while turning streaming into modern overpriced cable television. Uber screamed against the entrenched taxi industry harming workers and passengers while creating an unregulated monster that harmed workers and passengers.

Altman and OpenAI are no different in this regard, loudly screaming against AI harming humanity while doing everything in their capacity to create AI tools that will knowingly harm humanity while enriching themselves.

If people trust the performance instead of the actions and their outcomes, then we can't convince them otherwise.

Regardless of intent, it was most definitely sold to the American public on that premise.

>Liberalism is a political and moral philosophy based on the rights of the individual, liberty, consent of the governed, political equality, the right to private property, and equality before the law. Liberals espouse various and often mutually conflicting views depending on their understanding of these principles but generally support private property, market economies, individual rights (including civil rights and human rights), liberal democracy, secularism, rule of law, economic and political freedom, freedom of speech, freedom of the press, freedom of assembly, and freedom of religion. Liberalism is frequently cited as the dominant ideology of modern history.

That's literally the premise of venture capital. This is a scenario where we're assuming ALL our bets will go to zero, except one which will be worth trillions. In that case you should bet on everything.

It's only in the opposite scenario (where every bet pays off with varying ROI) that it makes sense to go all-in on whichever bet seems most promising.

Sounds like payola for the enterprising and experienced mercenary.

These models being expensive leads me to think they will look at all methods of monetization possible when seeking profitability. Rather than ads being off the table, it could feasibly make ads be on the table sooner.

It's also why totalitarian regimes love it, they can simply train it to regurgitate a modified version of reality.

Condoning "honest liars" enables a whole other level of open and unrestricted criminality.

It’s just that their biases have much more capacity to cause damage as their wealth gives them so much power.

This has some baked assumptions about cycle time and improvement per cycle and whether there's a ceiling.

The most advanced tools are (and will continue to be) at a higher level of the stack, combining the leading models for different purposes to achieve results that no single provider can match using only their own models.

I see no reason to think this won't hold post-AGI (if that happens). AGI doesn't mean capabilities are uniform.

> If one exclusive group gets the benefit of developing AI with a 20% productivity boost compared to others, and they develop a 2.0 that grants them a 25% boost, then a 3.0 with a 30% boost, etc...

That’s a bit of a stretch, generative AI is least capable of helping with novel code such as needed to make AGI.

If anything I’d expect companies working on generative AI to be at a significant disadvantage when trying to make AGI because they’re trying to leverage what they are already working on. That’s fine for incremental improvement, but companies rarely ride one wave of technology to the forefront of the next. Analog > digital photography, ICE > EV, coal mining > oil, etc.

Id love to believe there is more to life than the AI future, or that we as humans are destined to be perpetually happy and live meaningful. However I currently dont see how our current levels of extreme prosperity are anything more than an evolutionary blip, even if we could make them last several millennia more.

Also the current crop of AI agents are just utter crap. But that's a skill issue of the people coding them, expect actual advances here soon.

Tim Apple did it too, and we don’t assume he’s an authoritarian now too, do we? I imagine they would probably have done similarly regardless of who won the election.

It sure seems like an endorsement, but I think it’s simply modern corporate strategy in the American regulatory environment, same as when foreign dignitaries stay in overpriced suites in the Trump hotel in DC.

Those who don’t kiss the ring are clearly and obviously punished. It’s not in the interest of your shareholders (or your launch partners) to be the tall poppy.

LLMs are great at making you think they are the other but aren't.

All dissidents went into Little Wadyia.

When the Dictator himself visited it, he started to fake his name by copying the signs and names he saw on the walls. Everyone knew what he was.

Internet social networks are like that.

Now, this moat thing. That's hilarious.

Then things went unexpectedly well, people were valuing them at billions of dollars, and they suddenly decided they weren't open any more. Suddenly they were all about Altman's Interests Safety (AI Safety for short).

The board tried to fulfil its obligation to get the nonprofit to do the things in its charter, and they were unsuccessful.

But they found themselves alone in that it turns out the employees (who were employed by the for-profit company) and investors (MSFT in particular) didn't care about the mission and wanted to follow the money instead.

So the board had no choice but to capitulate and leave.

GPT-2 was released in 2019, GPT-3 in 2020. I'd say 2020 is significant because that's when people seriously considered the Turing test passed reliably for the first time. But for the sake of this argument, it hardly matters what date years back we choose. There's been enough time since then to see the plateau.

> Most people in society connect AI directly to ChatGPT and hence OpenAI.

I'd double-check that assumption. Many people I've spoken to take a moment to remember that "AI" stands for artificial intelligence. Outside of tongue-in-cheek jokes, OpenAI has about 50% market share in LLMs, but you can't forget that Samsung makes AI washing machines, let alone all the purely fraudulent uses of the "AI" label.

> And there has been a lot of progress in image generation, video generation, ...

These are entirely different architectures from LLM/chat though. But you're right that OpenAI does that, too. When I said that they don't stray much from chat, I was thinking more about AlexNet and the broad applications of ML in general. But you're right, OpenAI also did/does diffusion, GANs, transformer vision.

This doesn't change my views much on chat being "not seeing the forest for the trees" though. In the big picture, I think there aren't many hockey sticks/exponentials left in LLMs to discover. That is not true about other AI/ML.

Well Trump is interested in tariffing movies and South Korea took DeepSeek off mobile app stores, so they certainly may try. But for high-end tasks, DeepSeek R1 671B is available for download, so any company with a VPN to download it and the necessary GPUs or cloud credits can run it. And for consumers, DeepSeek V3's distilled models are available for download, so anyone with a (~4 year old or newer) Mac or gaming PC can run them.

If the only thing keeping these companies valuations so high is banning the competition, that's not a good sign for their long-term value. If you have to ban the competition, you can't be feeling good about what you're making.

For what it's worth, I think GPT o3 and o1, Gemini 2.5 Pro and Claude 3.7 Sonnet are good enough to compete. DeepSeek R1 is often the best option (due to cost) for tasks that it can handle, but there are times where one of the other models can achieve a task that it can't.

But if the US is looking to ban Chinese models, then that could suggest that maybe these models aren't good enough to raise the funding required for newer, significantly better (and more expensive) models. That, or they just want to stop as much money as possible from going to China. Banning the competition actually makes the problem worse though, as now these domestic companies have fewer competitors. But I somewhat doubt there's any coherent strategy as to what they ban, tariff, etc.

OpenAI loses billions and is at the mercy of getting new investors to fund the losses. It has many plausible competitors.

What do you consider an "LLM provider"? Is it a website where you interact with a language model by uploading text or images? That definition might become too broad too quickly. Hard to ban.

We should champion the good people who did the good things and managed to resist the temptations of the poisoned apple, but we shouldn’t hold an entire city on a pedestal because of nostalgia alone. Nobody, and no entity, is that deserving.

There's so much to do, explore and learn. The prospect of AI stealing my job is only scary because my income depends on this job.

Their relationship with MS breaking down is a bad omen. I'm already seeing non-tech users who use "Copilot" because their spouse uses it at work. Barely knowing it's rebadged GPT. You think they'll switch when MS replaces the backend with e.g. Anthropic? No chance.

MS, Google and Apple and Meta have gigantic levers to pull and get the whole world to abandon OpenAI. They've barely been pulling them, but it's a matter of time. People didn't use Siri and Bixby because they were crap. Once everyone's Android has a Gemini button that's just as good as GPT (which it already is (it's better) for anything besides image generation), people are going to start pressing them. And good luck to OpenAI fighting that.

Switching from ChatGPT to the many competitors is neither expensive nor painful.

"Instead of our current complex non-competing structure—which made sense when it looked like there might be one dominant AGI effort but doesn’t in a world of many great AGI companies—we are moving to a normal competing structure where ..." is all it takes

This was the fear when the cotton gin was invented. It was the ear when cars were created. The same complaint happened with the introduction of electronic, automated, telephone switchboards.

Jobs change. Societies change. Unemployment worldwide, is near the lowest it has ever been. Work will change. Society will eventually move to a currency based on energy production, or something equally futuristic.

This doesn't mean that getting there will be without pain.

Your 2001 essay isn't a good parallel to OpenAI's situation.

OpenAI wasn't "publicly funded" i.e. with public donations or government grants.

The non-profit was started and privately funded by a small group of billionaires and other wealthy people (Elon Musk donates $44 million, Reid Hoffman, etc collectively pledging $1 billion of their own money).

They miscalculated in thinking their charity donations would be enough to recruit the PhD machine learning researchers and pay the high GPU costs to create the AI alternative to Google DeepMind, etc. Their 2015 assumptions about future AI development costs were massively underestimated and now they look like bad for trying to convert it to a for-profit enterprise. Instead of a big conversion to for-profit, they now will settle with keeping a subsidiary that's for-profit. Somewhat like other entities structured as a non-profit that owns for-profit subsidiaries such as Mozilla, Girl Scouts, Novo Nordisk, etc.

Obviously with hindsight... if they had to do it all over, they would just create the reverse structure of creating the OpenAI for-profit company as the "parent entity" that pledges to donate money to charities. E.g. Amazon Inc is the for-profit that donates to Housing Equity Fund for affordable housing.

Why? I paid for Claude for a while, but with Deepseek, Gemini and the free hits on Mistral, ChatGPT, Claude and Perplexity I'm not sure why I would now. This is anecdotal of course, but I'm very rarely unique in my behaviour. I think the best the subscription companies can hope for is that their subscribers don't realize that Deepseek and Gemini can basically do all you need for free.

If every major player had an AI option, i'm just not understanding how because OpenAi moved first or got big first, the hugely massively successful companies that did the same thing for multiple decades don't have the same advantage?

Why are you still pretending anything is going to come out of this?

> taken it from a toy to genuinely insanely useful.

Really?

The dude announces new initiatives from the White House, regularly briefs Senators and senior DoD leaders, and is the top get for interviews around the world for AI topics.

There’s a lot more to be ambitious about than just money.

That name recognition made Coca Cola into a very successful global corporation.

I probably need to clarify what I'm talking about, so that peeps like @JumpCrisscross can get a better grasp of it.

I do not mean the total market share of the category of businesses that could be labeled as "AI companies", like Microsoft or NVIDIA, on your first link.

I will not talk about your second link because it does not seem to make sense within the context of this conversation (zero mentions or references to market share).

What I mean is:

* The main product that OpenAI sells is AI models (GPT-4o, etc...)

* OpenAI does not make hardware. OpenAI is not in the business of cloud infrastructure. OpenAI is not in the business of selling smartphones. A comparison between OpenAI and any of those companies would only make sense for someone with a very casual understanding of this topic. I can think of someone, perhaps, who only used ChatGPT a couple times and inferred it was made by Apple because it was there on its phone. This discussion calls for a deeper understanding of what OpenAI is.

* Other examples of companies that sell their own AI models, and thus compete directly with OpenAI in the same market that OpenAI operates by taking a look at their products and services, are Anthropic (w/ Claude), Google (w/ Gemini) and some others ones like Meta and Mistral with open models.

* All those companies/models, together, make up some market that you can put any name you want to it (The AI Model Market TM)

That is the market I'm talking about, and that is the one that I estimated to be 90%+ which was pretty much on point, as usual :).

1: https://gs.statcounter.com/ai-chatbot-market-share

2: https://www.ctol.digital/news/latest-llm-market-share-mar-20...

We can selectively ban uses without banning the technology wholesale; e.g., nuclear power generation is permitted, while nuclear weapons are strictly controlled.

The primary difference is the observability - with satellites we had some confidence that other nations respected treaties, or that they had enough reaction time for mutual destruction, but with this AI development we lack all that.

I don't think there is serious argument that LLMs won't generate tremendous value. The question is who will capture it. PCs generated massive value. But other than a handful of manufacturers and designers (namely, Apple, HP, Lenovo, Dell and ASUS), most PC builders went bankrupt. And out of the value generated by PCs in the world, the vast majority was captured by other businesses and consumers.

When the Internet was developed they didn't imagine the world wide Web.

When cars started to get popular people still thought there would be those who are going to stick with horses.

I think you're right on the AI we're just on the cusp of it and it'll be a hundred times bigger than we can imagine.

Back when oil was discovered and started to be used it was about equal to 500 laborers now automated. One AI computer with some video cards are now worth x number of knowledge workers. That never stop working as long as the electricity keeps flowing.

This makes me want to invest in malpractice lawyers, not OpenAI

Oh we know: https://pmc.ncbi.nlm.nih.gov/articles/PMC11006786/

AI isn't going to be the world changing, AGI, that was sold to the public. Instead, it will simply be another B2B SaaS product. Useful, for sure. Even profitable for startups.

But "take over the world" good? Unlikely.

The world is changing and that is scary.

Jobs, careers, real work, all replaced by machines which can do it all better, faster, cheaper than humans.

Young people with modest ambitions to learn and master a skill and contribute to society, and have a meaningful life. That can be blue collar stuff too.

How will children respond to the question - "What do you want to be when you grow up?"

They can join the Amish communities where humans still do the work.

One, you suggested OP had not “looked at the actual numbers.” That implies you have. If you were just guessing, that’s misleading.

Two, you misquoted (and perhaps misunderstand) a statistic that doesn’t match your claim. Even in your last comment, you defined the market as “companies that sell their own AI models” before doubling down on the chatbot-only figure.

> not even in Puchal wildest dreams

Okay, so what’s your source? Because so far you’ve put forward two sources, a retracted one and one that measures a single product that you went ahead and misquoted.

No, it wasn't.

As a proxy, here's HN results prior to November, 2022 - 13 results.

https://hn.algolia.com/?dateEnd=1667260800&dateRange=custom&...

Here's Google Trends, showing a clear uptick May 2023, and basically no search volume before (the small increase Feb. 2023 probably Meta's Llama).

https://trends.google.com/trends/explore?date=today%205-y&ge...

https://trends.google.com/trends/explore?date=today%205-y&ge...

As another proxy, compare Nvidia revenues - $26.91bln in 2022, $26.97bln in 2023, $60bln 2024, $130bln 2025. I think it's clear the hype didn't start until 2023.

You're welcome to point out articles and stores before this time period "hyping" LLM's, but what I remember is that before ChatGPT there was very little conversation around LLM's.

OpenAI models are already of the most expensive, they don’t have a lot of levers to pull.

I am thinking about society as a whole, how it will affect all types of people and cultures on this planet.

A non-AI dystopia is the least likely scenario.

Is that a bubble? I suppose it is; it’s also probably the right strategy.

So, since we've used the exact same reasoning to prove two opposite conclusions, it logically follows that this reasoning is faulty.

I think the chance they're going to create a "superintelligence" is extremely small. That said I'm sure we're going to have a lot of useful intelligence. But nothing general or self-conscious or powerful enough to be threatening for many decades or even ever.

> Predicting the future is famously difficult

That's very true, but that fact unfortunately can never be used to motivate any particular action, because you can always say "what if the real threat comes from a different direction?"

We can come up with hundreds of doomsday scenarios, most don't involve AI. Acting to minimize the risk of every doomsday scenario (no matter how implausible) is doomsday scenario no. 153.

I think you realise this is the weak point. You can't rule out the current AI approach leading to superintelligence. You also can't rule out a rotting banana skin in your bin spontaneously gaining sentience either. Does that mean you shouldn't risk throwing away that skin? It's so outrageous that you need at least some reason to rule it in. So it goes with current AI approaches.

This extreme risk aversion and focus on negative outcomes is just the result of certain personality types, no amount of rationalizing will change your mind as you fundamentally fear the unknown.

How do you get out of bed everyday knowing there’s a chance you could get hit by a bus?

If your tribe invented fire you’d be the one arguing how we can’t use it for fear it might engulf the world. Yes, humans do risk starting wildfires, but it’s near impossible to argue the discovery of fire wasn’t a net good.

I disagree at least on this one. I don't see any scenario where superintelligence comes into existence, but is for some reason limited to a mediocrity that puts it in contention with humans. That equilibrium is very narrow, and there's no good reason to believe machine-intelligence would settle there. It's a vanishingly low chance event. It considerably changes the later 1-in-n part of your comment.

You have cooked up a straw man that will believe anything as long as it contains a doomsday prediction. You are more than 99.9% confident about doomsday predictions, even if you claim you aren't.

In the case of AGI we already know it is physically possible.

Just like your proposition that any "small" chance justifies investing "everything" disregards the same argument regarding the precautionary principle of potentially devastating technologies. You've also slipped in an additonal "with no real downside" which you cannot predict with certainty anyways, rendering this argument infalsifiable. At least tsimionescu didn't dare making such a sweeping (but baseless) statement.

I'm guessing that you think that society is getting worse every year or will eventually collapse, and you hope that continued AI research might prevent that outcome.

Some founders truly believe in structuring the company for the benefit of the public, but Altman has already shown he's not one of them.

They have the brand recognition and consumer goodwill no other brand in AI has, incredibly so with school students, who will soon go into the professional world and bring that goodwill with them.

I think better models are enough to dethrone OpenAI in API, B2C and internal enterprise use cases, but OpenAI has consumer mindshare, and they're going to be the king of chatbots forever. Unless somebody else figures out something which is better by orders of magnitude and that Open AI can't copy quickly, it's going to stay that way.

Apple had the opportunity to do something really great here. With Siri's deep device integration on one hand and Apple's willingness to force 3rd-party devs to do the right thing for users on the other, they could have had a compelling product that nobody else could copy, but it seems like they're not willing to go that route, mostly for privacy, antitrust and internal competency reasons, in that order. Google is on the right track and might get something similar (although not as polished as typical Apple) done, but Android's mindshare among tech-savvy consumers isn't great enough for it to get traction.

There's a lot of literature on this, and if you've been in the industry for any amount of time since the 1950s, you have seen at least one AI winter.

probably true but this statement would be true if when is 2308 which would defeat the purpose of the statement. when first cars started rolling around some mates around the campfire we saying “not if but when” we’ll have flying cars everywhere and 100 years later (with amazing progress in car manufacturing) we are nowhere near… I think saying “when, not if” is one of those statements that while probably indisputable in theory is easily disputable in practice. give me “when” here and I’ll put up $1,000 to a charity of your choice if you are right and agree to do the same thing if wrong

It has taken tens to hundred of billions of dollars without equivalent economic justification(yet) before to reach here. I am not saying economic justification doesn't exist or wont come in the future, just that the upfront investment and risk is already in order of magnitude of what the largest tech companies can expend.

If the the next generation requires hundreds of billions or trillions [2] upfront and a very long time to make returns, no one company (or even country) could allocate that kind of resources.

Many cases of such economically limited innovations[1], nuclear fusion is the classic always 20 years away example. Another close one is anything space related, we cannot replicate in next 5 years what we already achieved from 50 years ago of say landing on the moon and so on.

From a just a economic perspective it is a definitely a "If", without even going into the technology challenges.

[1]Innovations in cost of key components can reshape economics equation, it does happen (as with spaceX) but it also not guaranteed like in fusion.

[2] The next gen may not be close enough to AGI. AGI could require 2-3 more generations ( and equivalent orders of magnitude of resources), which is something the world is unlikely to expend resources on even if it had them.

LLMs destroying any sort of capacity (and incentive) for the population to think pushes this further and further out each day

Most HN people are probably too young to remember that the nanotech post-scarcity singularity was right around the corner - just some research and engineering way - which was the widespread opinion in 1986 (yes, 1986). It was _just as dramatic_ as today's AGI.

That took 4-5 years to fall apart, and maybe a bit longer for the broader "nanotech is going to change everything" to fade. Did nanotech disappear? No, but the notion of general purpose universal constructors absolutely is dead. Will we have them someday? Maybe, if humanity survives a hundred more years or more, but it's not happening any time soon.

There are a ton of similarities between nanotech-nanotech singularity and the moderns LLM-AGI situation. People point(ed) to "all the stuff happening" surely the singularity is on the horizon! Similarly, there was the apocalytpic scenario that got a ton of attention and people latching onto "nanotech safety" - instead of runaway AI or paperclip engines, it was Grey Goo (also coined in 1986).

The dynamics of the situation, the prognostications, and aggressive (delusional) timelines, etc. are all almost identical in a 1:1 way with the nanotech era.

I think we will have both AGI and general purpose universal constructors, but they are both no less than 50 years away, and probably more.

So many of the themes are identical that I'm wondering if it's a recurring kind of mass hysteria. Before nanotech, we were on the verge of genetic engineering (not _quite_ the same level of hype, but close, and pretty much the same failure to deliver on the hype as nanotech) and before that the crazy atomic age of nuclear everything.

Yes, yes, I know that this time is different and that AI is different and it won't be another round of "oops, this turned out to be very hard to make progress on and we're going to be in a very slow, multi-decade slow-improvement regime, but that has been the outcome of every example of this that I can think of.

We have zero evidence for this. (Folks said the same shit in the 80s.)

I want to believe, man.

There isn't an official precise definition of superintelligence, but it's usually vaguely defined as smarter than humans. Twice as smart would be sufficient by most definitions. We can be more conservative and say we'll only consider superintelligence achieved when it gets to 10x human intelligence. Under that conservative definition, 1/1000th of the performance of superintelligence would be 1% as smart as a human.

We don't have a great way to compare intelligences. ChatGPT already beats humans on several benchmarks. It does better than college students on college-level questions. One study found it gets higher grades on essays than college students. It's not as good as humans on long, complex reasoning tasks. Overall, I'd say it's smarter than a dumb human in most ways, and smarter than a smart human in a few ways.

I'm not certain we'll ever create superintelligence. I just don't see why you think the odds are "extremely small".

So then apply that to Open AI. What are the distribution channels? Should they be paying Cursor to make them the default model? Or who else? Would that work? If not, why not? What's different?

My intuition is that this wouldn't work for them. I think if this "pay to be default" strategy works for someone, it will be one of their deeper pocketed rivals.

But I also don't think this was the only reason Google won search. In my memory, those deals to pay to be the default came fairly long after they had successfully built the brand image as the best search engine. That's how they had the cash to afford to pay for this.

A couple years ago, I thought it seemed likely that Open AI would win the market in that way, by being known as the clear best model. But that seems pretty unclear now! There are a few different models that are pretty similarly capable at this point.

Essentially, I think the reason Google was able to win search whereas the prospects look less obvious for Open AI is that they just have stronger competition!

To me, it just highlights the extent to which the big players at the time of Google's rise - Microsoft, Yahoo, ... Oracle maybe? - really dropped the ball on putting up strong competition. (Or conversely, Google was just further ahead of its time.)

...yes. Office is the market leader. Slack has between a fifth and a fourth of the market. Coca-Cola's products have like 70% market share in the American carbonated soft-drink market [1].

[1] https://www.investopedia.com/ask/answers/060415/how-much-glo...

I wonder, do you have a hypothesis as to what would be a measurement that would differentiate AGI vs Not-AGI?

EDIT: There can be levels of AGI. Google DeepMind have proposed a framework that would classify ChatGPT as "Emerging AGI".

https://arxiv.org/abs/2311.02462

Here is a mainstream opinion about why AGI is already here. Written by one of the authors the most widely read AI textbook: Artificial Intelligence: A Modern Approach https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Mod...

You didn't teach it, the model is still the same after you ran that. That is the same as a human following instructions without internalizing the knowledge, he forgets it afterward and didn't learn what he performed. If that was all humans did then there would be no point in school etc, but humans do so much more than that.

As long as LLM are like an Alzheimer's human they will never become a general intelligence. And following instructions is not learning at all, learning is building an internal model for those instructions that is more efficient and general than the instructions themselves, humans do that and that is how we manage to advance science and knowledge.

It is not hard to imagine a "cooking robot" as a black box that — given the appropriate ingredients — would cook any dish for you. Press a button, say what you want, and out it comes.

Internally, the machine would need to perform lots of tasks that we usually associate with intelligence, from managing ingredients and planning cooking steps, to fine-grained perception and manipulation of the food as it is cooking. But it would not be conscious in any real way. Order comes in, dish comes out.

Would we use "intelligent" to describe such a machine? Or "magic"?

It isn't close at all.

A machine could be super intelligent at solving real world practical tasks, better than any human, without being conscious.

We don't have a proper definition of consciousness. Consciousness is infinitely more mysterious than measurable intelligence.

The zero training version not only ended up dramatically outperforming the 'expert' version, but reached higher levels of competence exponentially faster. And that should be entirely expected. There were obviously tremendous flaws in our understanding of the game, and training on those flaws resulted in software seemingly permanently handicapping itself.

Minimal expert training also has other benefits. The obvious one is that you don't require anywhere near the material and it also enables one to ensure you're on the right track. Seeing software 'invent' fundamental arithmetic is somewhat easier to verify and follow than it producing a hundred page proof advancing, in a novel way, some esoteric edge theory of mathematics. Presumably it would also require orders of magnitude less operational time to achieve such breakthroughs, especially given the reduction in preexisting state.

In particular we redefined the test to make it passable. In Turing's original concept the competent investigator and participants were all actively expected to collude against the machine. The entire point is that even with collusion, the machine would be able to pass. Instead modern takes have paired incompetent investigators alongside participants colluding with the machine, probably in an effort to be part 'of something historic'.

In "both" (probably more, referencing the two most high profile - Eugene and the large LLMs) successes, the interrogators consistently asked pointless questions that had no meaningful chance of providing compelling information - 'How's your day? Do you like psychology? etc' and the participants not only made no effort to make their humanity clear, but often were actively adversarial obviously intentionally answering illogically, inappropriately, or 'computery' to such simple questions. And the tests are typically time constrained by woefully poor typing skills (this the new normal in the smartphone gen?) to the point that you tend to get anywhere from 1-5 interactions of a few words each.

The problem with any metric for something is that it often ends up being gamed to be beaten, and this is a perfect example of that.

The very people whose theories about language are now being experimentally verified by LLMs, like Chomsky, have also been discrediting the Turing test as pseudoscientific nonsense since early 1990s.

It's one of those things like the Kardashev scale, or Level 5 autonomous driving, that's extremely easy to define and sounds very cool and scientific, but actually turns out to have no practical impact on anything whatsoever.

How about we have ChatGPT start with a simple task like reliably generating JSON schema when asked to.

Hint: it will fail.

Literally everything that's been invented.

I’m curious what epistemological grounding you are basing your claim on

- Processing visual data and classifying objects within their field of vision.

- Processing auditory data, identifying audio sources and filtering out noise.

- Maintaining an on-going and continuous stream of thoughts and emotions.

- Forming and maintaining complex memories on long-term and short-term scales.

- Engaging in self-directed experimentation or play, or forming independent wants/hopes/desires.

I could sit here all day and list the forms of intelligence that humans and other intelligent animals display which have no obvious analogue in an AI product. It's true that individual AI products can do some of these things, sometimes better than humans could ever, but there is no integrated AGI product that has all these capabilities. Let's give ourselves a bit of credit and not ignore or flippantly dismiss our many intelligent capabilities as "useless."

That was my whole point. Maybe in theory an LLM could learn to drive a car, but they can't today because they don't physically have access to cars they could try to drive just like a person who can't learn to use a tool because they're physically limited from using it.

It's kind of a simple enough concept... it's really just something that functions on par with how we do. If you've built that, you've built AGI. If you haven't built that, you've built a very capable system, but not AGI.

The General Intelligence part of AGI refers to its ability to solve problems that it was not explicitly trained to solve, across many problem domains. We already have examples of the current systems doing exactly that - zero shot and few shot capabilities.

> We have no evidence of this.

That's my point. Humans are not "autocompleting words" when they speak.

The "g" in AGI requires the AI be able to perform "the full spectrum of cognitively demanding tasks with proficiency comparable to, or surpassing, that of humans" [1]. Full and not full are binary.

> if you oversimplify LLMs to "word autocompletion" then you can make the same argument for humans

No, you can't, unless you're pre-supposing that LLMs work like human minds. Calling LLMs "emerging AGI" pre-supposes that LLMs are the path to AGI. We simply have no evidence for that, no matter how much OpenAI and Google would like to pretend it's true.

[1] https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

It is not a redefinition. It's a classification for AGI systems. It's a refinement.

Other researchers are also trying to classify AGI systems. It's not just Google. Also, there is no universally agreed definition of AGI.

> We don’t know what the stages are. Folks in the 80s were similarly selling their expert systems as a stage to AGI. “Emerging AGI” is a bullshit term.

Generalization is a formal concept in machine learning. There can be degrees of generalized learning performance. This is actually measurable. We can compare the performance of different systems.