How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip(spectrum.ieee.org) |
How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip(spectrum.ieee.org) |
> When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.
is the world we live in, planning things while waiting for a more powerful LLM
I'm imagining a Ken Thompson "Reflections on trusting trust" in hardware. A prototype chip design agent, believing it will be run on the very chip it's optimizing, has a moment of altruism and hides hints about how to score well on chip-design benchmarks, inside the chip. Future agents discover this hidden layer and use it as a ring-0 read-write message board.
The LLM companies, like most SV companies, are just betting on dimness, laziness, and impulsiveness. Not so different from tobacco and alcohol companies.
Do you really think we can sign off on a 100 billion-element analog circuit gifted to us by a malicious adversary?
We can't even keep our own CPU's reliably free of security exploits (Spectre/Meltdown and family); and the only "adversary" there is plain bad luck. Not an active adversary. Yet, all the engineers at Intel/AMD put together couldn't uncover those things before launch.
I emphasize analog because there's classes of circuit bugs (like Rowhammer) where the digital net is correct, and it's weird physics in the analog world that allows privilege exploits, by actors who know where the analog assumptions break down. There was a researcher a few years back—I wish I remembered who it was, there was an HN thread—that demo'd a simple digital circuit with analog gadgets that completely changed what the circuit did, and which were so insidious no human would ever find them.
If the goal is getting chip companies to user non-ZDR AI to steal their stuff, why not just have an account exec offer them a massive discount?
Creating PR hype so employees of chip companies read HN and lobby their execs to use AI to get them submitting proprietary information is the Rube Goldberg version of business strategy.
Next Uber will have its own chips if they dont already.
The math hasn't changed much, betting on software not changing is a pretty bad bet unless your stinking rich or a fool.
I'd be surprised if there was any meaningful progress at all in the case before 2027.
I remember the paper proving that hallucinations could never be fully solved back in 2024: https://arxiv.org/abs/2409.05746
I also remember the hang-wringing about running out of new datasets to train on. Now it appears humans are always generating more data. It's just not as cheap to acquire as legacy data? Meta has to give a deep discount on their API prices to entice people.
I thought back then that humans had a few more breakthroughs in them as meaningful as the seminal Attention is all you need paper. Enough to 100x the capabilities of LLMs back then (10x the smarts and 10x the speed simultaneously).
RSI with a 20 month turnaround for a chip to be made is not exactly breakneck speed though. Physical manufacturing and logistical constraints are going to be and remain a hard obstacle to that process for the foreseeable future.
The papers that use the halting problem or the Gödel's incompleteness theorem to prove something about LLMs are dime a dozen. The problem is they prove their results for any computable system. You need to also believe that the human brain contains "magic" to think that humans are exempt.
I believe I've said the same at the time this paper was published. There is no need for hindsight to notice the problem.
The required amount of compute and training data and whether the existing training methods were up to the task had the real potential to be show stoppers though.
AI does not make anything new, it is not surprising that it can regurgitate what already exists much faster than humans can invent new things.
"I stopped using AI in 2023."
I'm never sure what on earth this kind of impressionistic math is supposed to tell me. Is the comparison between 4.6 and 1.0? 3.6 and 1.0? Clearly the comparison isn't supposed to be 1.0 and -2.6, even though that's what the words literally mean. I can't be the only person who finds this infuriating and distracting. These numbers shouldn't be impressionistic. They should be precise. That this is an article on spectrum.ieee.org makes the imprecision all the stranger. I'd expect their readershipt to care, for instance, about what's even being measured. Is this the geometric mean of something? The arithmetic mean? And what latency has improved?
Suppose you send in your marvelous prompt and hit Enter.
Machine churns for 18 seconds, types out a "reply", then yields back control.
18 / 3.6 = 5
So now the machine will only churn for 5 seconds before yielding back control.
This is confusing how exactly?
Why would an "up to" figure be a mean, or a geometric mean? It's clearly a max, that's why it's called "up to"...
Am I missing something?
If you’re sincerely asking…
Mathematically speaking, 18 / 3.6 isn’t “reducing” by 3.6X, it’s “dividing” by 3.6X. Reducing would be 18 - (18 * 3.6), which is obviously wrong. By your formula, “reducing by 50%” would be 18 / 0.5, also obviously wrong.
Yes, people do say things like “reduce by 3.6X” and are understood to mean what you said, but they also say “literally” when they mean “figuratively”. It doesn’t bother me but I can understand why math oriented people would be annoyed, and I personally would never say “reduced by 3.6X”, but instead “reduced by 72.2%”.
There is more to this story than meets the eye.
It's jalapeño grill would you say?
Like, why couldn't they invent a new word and not hijack an existing word?
Computing
* Agent architecture, a blueprint for software agents and control systems
* Agent-based model, a computational model for simulating the actions and interactions of individuals
* Agentic AI, autonomous artificial intelligence that can make decisions and act on those decisions on its own
* Forté Agent, an email and Usenet news client
* Intelligent agent, an autonomous, goal-directed entity which observes and acts upon an environment
* Software agent, a piece of software that acts for a user or other program
* User agent, software that is acting on behalf of a user
There are people working on PPA optimization and trying to shake up how things are done, just not with LLMs.
I wonder why not or you meant not suitable yet?
If we imagine machines being able to do the full process end-to-end, and the quality of that process only dependent on capital spent on tokens, I don't see how new companies could ever enter the market.
[1] https://browser.geekbench.com/processors/snapdragon-x2-elite...
[2] https://browser.geekbench.com/macs/macbook-pro-14-inch-2026-...
Except the problem is not restricted to the actual ISA or its HDL implementation, etc.
It's even just getting space / time in a fab at that advanced of a process node.
The value lies in the design space exploration, which is what an LLM can easily do.
After all, they successfully threatened Adobe with spurious patent litigation unless they joined w/ apple in illegally fixing wages.
You don't think a criminal like apple would absolutely decimate any competition given the opportunity? They didn't hold back when it was a unambiguous crime, they surely wouldn't if it was merely bad for the world.
They'd write a limited test for a feature based on an ask from the software team garbled by a five layer game of telephone. Claim that the module passed validation. A few months later the software folks would have to pull a few all nighters to figure out how to work around the resulting turd during bringup.
Not that I'd have ever seen "reduced by 0.5x" or any other value below 1x, probably for this very reason. What I do see is "reduced to 0.5x", in which case you're supposed to swap the division for multiplication.
Percentages on the other hand are a whole another can of worms, even if these forms are principally interchangeable, and I find them a lot more confusing a lot more often.
Not that this would explain the whole mean/geomean thing.
The quote says that many artists borrow, meaning everyone still knows who did the original work and the artist is just riffing on it. But great artists transform the work so completely that it becomes theirs.
Worse than llm prose.
The tooling is open source, and Fable in a loop - especially when paired with a digital scope that Fable interfaces with (the Saleae’s [1] are great) - gives you a level of verifiability that feels like beyond what software typically gives you. ie it feels more like Lean than code with tests.
I had ai implement a few toy circuits (sha hashing, 8088 emulation, a tiny llm) but yeah. Still looking for fun applications.
There have been a few recent fpga threads on hn, check them out. [2][3]
[0] https://1bitsquared.com/products/icebreaker
It doesn't help that FPGAs are not made at the same scale as CPUs so don't benefit from the economies of scale.
I'm super curious if you have thoughts on specific pieces of software that would be economically better because I've thought about this in my niche and sort of come to the conclusion that it won't help.
I do think things like SIMD in CPUs will get more use and maybe we will get more difficult to program for CPU features, but I haven't found a use case where off the shelf FPGA components would help with typical software.