Moore’s Law is dead – Long live the chiplet(semiwiki.com) |
Moore’s Law is dead – Long live the chiplet(semiwiki.com) |
https://en.wikipedia.org/wiki/Chiplet
This seems to be about the third reason listed:
> Known good die (KGD): chiplets can be tested before assembly, improving the yield of the final device
Problem:
> In general, a killer defect is defined as a defect that is 20% the size of the fabrication node. For
> example, a defect that is less than 9nm may be acceptable for the 45nm fabrication node, but a defect
> larger than 2.8nm would be defined as a “killer” defect for the 14nm fabrication node. For the 5nm
> fabrication node, a defect measuring only 1nm could be a killer.
>
> This is one of the primary reasons that it has become increasingly difficult to yield large monolithic
> ICs (as measured in die area) when using leading edge fabrication process technology
https://www.nextplatform.com/2021/06/09/amd-on-why-chiplets-...
https://cdn.wccftech.com/wp-content/uploads/2018/11/AMD-EPYC...
Because a vacuum pretty eliminates dust - with no air, dust just falls towards either the ground (if it is uncharged), or towards a positive or negatively charged surface (if the dust particle itself is charged).
Until 2005 or so, shrinking transistors automatically increased speed and reduced power consumption. When that ran out of steam, the industry went to multi core and massive parallelism with GPUs.
Until recently each shrink also lowered the cost per transistor, but that seems to have run out also and has something to do with why Intel was stuck at 14nm for so long and why new GPU prices are so insane despite a collapse in demand and resolution of the supply chain crisis for high end chips.
Chiplets at best are neutral with regard to cost. If manufacturing overhead is low, two chiplets give you twice the transistors at twice the cost. The industry did not pursue chiplets with a lot of vigor until now because it was a less competitive approach to scaling than shrinking transistors until now.
> However, as it has in the past, the semiconductor ecosystem is adapting and as Chiplet technology builds traction, we will very likely see a period of accelerating innovation and new market opportunities opening as we move forward.
The whole premise is that chip innovation (and overall computing power) is continuing to accelerate, even though "Moore's Law as we've known it" has ended
Especially in the startup space I saw companies building software with the hypothesis “users need the latest device for our product and they will get faster anyway so we don’t need optimize our code. Instead we deliver features on max speed skipping optimizations and wait until our users upgraded to newer devices during the coming 2-4 years”.
I don’t even blame the companies for doing this. The benefit to cost ratio of using idk C++ for everything is just too bad.
It was basically a monopolistic warning from one of the founders not to release new products too fast so the company wouldn’t burn out. It worked. But it’s not some scientific or physical law and I hate when people refer to it as such.
According to the article, Moore's Law predates the founding of Intel by several years.
Not about transistor size of performance.
So it was an economic insight, more than anything else.
Moving to chiplets doesn’t change transistor density. This is a packaging feature and not a fabrication feature. This is done for manufacturing cost reduction and yield improvements.
Transistor density doubling every 18 months for a similar cost.
https://t7m8e9c8.rocketcdn.me/wp-content/uploads/2020/09/pre...
https://en.wikipedia.org/wiki/Moore%27s_law#cite_note-Moore_...
By those terms, Moore's law is totally extinct.
Folk haven't noticed however, because the "leading edge" logic manufacturers have 60% gross margins. The vast majority of their costs are in design, distribution and overhead.
Price rises of 30% to 100% have disguised that the cost of manufacturing the silicon is an order of magnitude more than a decade ago.
Granted, the above numbers are not the actual inflation adjusted wafer cost for leading edge nodes.
But, $16,000 for a 300mm wafer is extraordinary.
I am no silicon engineer but I suspect a chip that fully takes advantage of the third dimension would be something like a sponge full of built in channels for the working fluid to remove heat.
First however I suspect you will see chiplets arranged vertically like heatsink fins and the whole cpu would effectively be the water block, basically a vlsi version of the cray 3
What you're describing is called 3D stacking, and it works.
It's just extraordinarily complex to resolve the intra-die latency issues, and many others, when going vertical. Hence expensive.
3D is also used in AMD's 3D cache.
But 3D logic on logic is more complicated because of heat issues. AFAIK there's no workable technical solution for that yet.
If you look into flash memory which aren't limited by heat, they have dozens of functional layers already and then we also stack those silicon wafers.
3d stacking is another innovation that can help.
But I think within a decade or two there will be a move away from silicon-only transistors to something like memristors or some type of optical or optoelectronic system that hasn't even been invented yet. This will provide some iterations with again radical parallel interconnect and quite possibly single thread speedups.
Off-topic but I wonder how much cheaper mobile phones would be if the manufacturers did not have to come up with a hardware design update every year or so? What if mobile phones were built to last longer, which would reduce the cost per phone due to high volume? Of course, I am not suggesting this is good for business but as a mere thought experiment.
> With $100 million in funding, the IDEAS and POSH programs ... aim to combat the growing complexity and cost of designing chips, now approaching $500 million for a bleeding-edge SoC. Essentially, POSH aims to create an open-source library of silicon blocks, and IDEAS hopes to spawn a variety of open-source and commercial tools to automate testing of those blocks and knitting them into SoCs and printed circuit boards. If successful, the programs “will change the economics of the industry,” enabling companies to design in relatively low-volume chips that would be prohibitive today.
2017 vision, slide #22, https://www.darpa.mil/attachments/eri_design_proposers_day.p...
My DARPA dream
$ git clone https://github.com/darpa/idea
$ git clone https://github.com/darpa/posh
$ cd posh
$ make soc42
UCSD OpenRoad, https://theopenroadproject.org/ & https://vlsicad.ucsd.edu/Publications/Conferences/378/c378.p...
> OpenROAD is a front-runner in open-source semiconductor design automation tools and know-how. Our project reduces barriers of access and tool costs to democratize system and product innovation in silicon. The OpenROAD tool and flow provide autonomous, no-human-in-the-loop, 24-hour RTL-GDSII capability to support low-overhead design exploration and implementation through tapeout. We welcome a diverse community of designers, researchers, enthusiasts and entrepreneurs who use and contribute to OpenROAD to make a far-reaching impact.
https://semiengineering.com/will-open-source-eda-work/
> All the big EDA providers, as well as leading chip companies, are active contributors to ERI projects. In fact, Cadence, Synopsys, Mentor, NXP, Intel, IBM, Intel, Qualcomm, Arm, Nvidia, Analog Photonics, SRI International and Applied Materials all have contributed speakers and engineers or materials to ERI effort ... the key to getting industry players to accept open-source EDA is whether it makes the design process more efficient without breaking anything—and whether it is possible to extract decades worth of design experience from libraries of millions of existing designs and use that to spot errors in real time in existing designs.
Shocking
That business plan is no longer functional for Intel. Translated, Moore's business plan was to shrink die size, and speed up clocks, so much as to obsolete their previous offering with an 18 month half life.
It just doesn't work that way anymore, and hasn't for quite some time.
Engelbart's Scaling Observation[0], on the other hand, remains quite interesting, and from what I see remains in force. Genetics in particular is still pulling exponential gains out of the luminiferous ether.
So everyone that works in other domains, without access to libraries written by the GPU druids, largely ignores their existence.
Consumer machines vary wildly in their GPU capabilities, especially VRAM. So how do you know that your nice accelerated algorithm is going to work if the user has an old GPU? And what do you do if it doesn’t work? Run on the CPU? Tell the user their machine is too weak?
Here the advantage of GPUs (performance) is also the biggest disadvantage: a gigantic range of performance profiles. At least with CPUs the oldest CPU is only going to be a small integer factor slower than a new one in single thread.
What unites gamers and machine learning is an expectation that the user has a reasonably recent and capable GPU. But these are small, self-selecting populations.
On the server side the issue is cost. GPUs are expensive, and usually not necessary, so nobody is going to write code that requires one without a good reason.
That cost about the same as a single mid-range video card. (Nvidia RTX 3070)
Why on earth would you add a requirement to your software/workflow that doubles your cost, and is just about impossible to find in stock?
However they failed to implement with good results the methods that they had hoped to work, while the others, i.e. TSMC and Samsung had much more realistic roadmaps, which added EUV at the right moment.
Intel was not stalled by waiting for EUV, on the contrary they were not prepared for the transition that was necessary when EUV was eventually ready.
https://www.theverge.com/2022/10/4/23385652/pat-gelsinger-in...
If progress continues then they will need some other expensive machine. Either that or they’ll try to stretch the life of EUV the same way Intel tried to delay EUV with extreme multiple patterning.
It seems to me though that the ASML machines ought to get some competition from something more like a free electron laser.
That doesn't take into account yields. One chip with twice the transistors is physically larger than two chips with half as many, and more likely to have a defect during production.
https://www.cpubenchmark.net/singleThread.html
It's amazing how often this is parroted. Anyone with a passing familiarity with the numbers knows this is actually not true at all.
Better caching, branch prediction, plus vast amounts of SRAM. There's been a slow & steady increase in the vast variety of single threaded workloads grouped together by "instructions per clock."
Both at the peak of the voltage frequency curve for workstations & overclocking, the apex of the optimization curve for data centre, and especially at the bare minimum for mobile devices with idle workloads.
Yes, it's a small fraction of the old days. It's still double in 10 years.
And as anyone who's migrated from an Intel Mac to Apple Silicon knows, "merely doubling" is a LOT.
4.5 years later and Intel is bragging that their upcoming topline CPU will run 6GHz stock. I suppose one could call this a plateau compared to the good old days of the 80s and 90s, but it’s definitely still progress.
In late 2000, Intel promised that Pentium 4s will hit 10GHz by 2005 – on a presumed 130W power budget –, after the last 5 years saw clock speeds increase from 150MHz to 1.4GHz for the P6 architecture (at a stable 30-40W power budget), and other vendors saw similar increases.
Over 20 years later, we're barely scratching the 6GHz barrier with an opportunistic turbo mode that isn't guaranteed to kick in, if your cooling isn't up to the task of dissipating a record-breaking 250W of peak power consumption.
Pentium 4 HT 3.8F, November 2004, 3.8GHz, 115W TDP
Core i9-13900KF, October 2022, 3.0GHz, 125W TDP
Of course, the latter does give you 8 performance cores and 16 efficiency cores so performance-per-watt has clearly improved; and it has 'turbo boost'. But in terms of sustained single-core performance? It's clear Intel's attention has been elsewhere. Such as on the laptop market, where power efficiency is king.
and fwiw ive had a 5ghz+ overclock on every cpu ive bought in the last ten years with a corsair 240mm aio, going back to the 3570k
It's at least enough that we have to take it into account:
We run our workloads across multiple Intel cpu generations and to be able to optimize utilization we have a "speedup factor" which is currently up to 1.7 for the latest generation we've tuned it for. And the base 1.0 performance is from Ivy Bridge, launched 2013.
No one is scared of employees gaming. Employees can’t install applications themselves on their laptops at most place.
Today’s iGPUs are fast enough comfortable run plenty of games.
I have work provided high-end POS Dell Precision engineering laptop. It has an Nvidia discrete GPU, but I don’t think I’ve ever actually needed its power, and I’d gladly trade it for a laptop without…
[1] https://www.howtogeek.com/676963/why-desktop-linux-still-mat...
> Furthermore, much of the purported single thread performance is taken from a small set of benchmark tests, and so chip makers just optimize them for those tests
"Single thread" is a notoriously difficult benchmark to quantify. Instruction queue depth, floating vs integer, branching vs linear, there are so many variables.
Passmark is fine. Workload simulation is state of the art.
This is the problem, GPUs are still a very specialised skill.
But how we write the code also matters of course.
Furthermore, competency also costs money. I’m not saying this to be mean to people abusing REST calls.
It's just designed with O(N^3) architectures, "servers on servers" and "callbacks on callbacks", etc. because, seemingly, new features are made by very jnr developers -- who glue these idiot-proof frameworks together ---- trading needed-knowhow for polynomial behaviour.
Exponential growth is exponential growth.
They'll exceed the GDP of the United States in a few cycles.
The question becomes, when costs hit the ceiling that Apple is willing to pay, will TSMC still be able to front up with the good stuff and deliver the growth.
The current pace of development is such that that they are plugging stuff in and hoping it works in LOTS of different ways.
A small army of electrical engineers work night shifts at TSMC as well, something Intel could never pull off given American labour market demand for EEs.
I suspect you're right, when costs have to come down timelines will blow out to something more reasonable.
I wasn't talking about Web apps.
If you aren't doing signal processing, physics or something that would benefit from simd, then the gp is correct, a gpu won't do much for you.
For instance a frightening amount of CPU is spent in financial messaging systems on validating UTF-8, parsing XML and JSON, converting numbers written in decimal digits to binary and things like that. You'd think these are "embarrassingly serial" problems but with clever coding and advanced SIMD instructions such as AVX-512 they can be accelerated for throughput, latency, and economy.
The benefits of the GPU are great enough that you might do more "work" but get the job done faster because it can be done in parallel.
For instance the algorithms used by the old A.I. ("expert systems") parallelize better than you might think (though not as well as the Japanese hoped they would in the 1980s) despite being super-branchy. Currently fashionable neural networks (called "connectionist" back in the day) require only predicated branching (which side of the ReLU are you on?) but spend a lot of calculations on parts of the network which might not be meaningful for the current inference. It depends on the details, but you might be better doing many more operations if you can do them in parallel.
Given that GPUs are out there and that so many people are working on them I think the range of what you can do with them is going to increase, though I think few people will be writing application logic on them directly, but they will increasingly use libraries and frameworks. For instance, see
And still requires specific skills to use, and is constrained to Python, C++ and Java based languages.
GPUs need to be exposed like SIMD, something that the language runtime takes care of, even if not perfect, better than not using them at all.
Not just Intel either. AMD has joined the game as of Zen 4, and NVIDIA's been playing it with their GPUs forever as well.
Zen 4 desktop CPUs appear to have (as expected) virtually unchanged single core performance, and maybe 5% reduced multi-core performance, on CPU-bound workloads by reducing the power limit to cut total power consumption -- by over a 100W reduction in the case of the new 7950X! Granted, Intel's been doing that forever -- reign in Alder Lake and its power consumption also comes way down, again for barely a performance hit in CPU-bound multi-core tasks.
-----
Enthusiast grade CPUs and GPUs are basically sold in the equivalent of a TV's retail "demo mode" now -- where a TV has max brightness, contrast and saturation that you'd NEVER use, but is intended to just grab a customer's attention as they walk by. Being pushed so far outside of their efficiency sweet spot just to get that extra 5% to "win benchmarks", when outside of specific use cases (and even if you actually need that 5%!) you're just consuming 50-100% more electricity for utterly marginal gains.
What a waste of resources! All so children (or people who still act like them) can feel better about a purchase as they argue on the internet over nothing worth arguing about.
You absolutely can reign it back in to sanity, and get better performance per watt over the previous gen, and still be noticeably faster over the previous gen.
-- -----
With AMD, apparently we're going to see BIOS updates from board manufacturers to make doing that simple. Set it to a lower power limit in a few keystrokes, still have something blazing fast (faster than previous gen), but use 145W instead of 250W. Or go even lower, still be a bit faster than previous gen while using around 88W on a 7950X instead of the 118W a 5950X did.
Intel -- who has been redlining their CPUs for years now -- even noted Raptor Lake's efficiency at lower power levels. Again, cut power consumption by 40-50%, for only a tiny performance hit. They actually made entire slides for their recent presentation highlighting this!
NVIDIA no different, and has been for years. Ampere stock voltages were well outside their sweet spot. Undervolt, cut power consumption by 20-25% and have performance UNCHANGED.
-- -----
Sure, there's more efficient stuff. Take last generation's 8-core Ryzen 7 PRO 5750GE. About 80% of the performance of an Intel Core i5-12600K, but only uses 38W flat out instead of 145W.
CPUMark single threaded score for the 13900K ~4800
So obviously a lot has changed and improved.
That was my impression from games at the time, that they were coded with an expectation that clock speeds would keep going up in the future. But they didn't and the games probably run just as bad now as they did before.
This isn't about thermals. This is about power consumption.
I'm not suggesting reigning in a CPU's power limits because it's "too hot".
I'm suggesting getting 95% of the performance for 59% of the power consumption. Because it's not worth spending 72% more on electricity for 5% increased performance. Again, even the manufacturers themselves know this and are admitting it. Look at this slide from Intel: https://cdn.arstechnica.net/wp-content/uploads/2022/09/13th-... Purported identical performance of previous gen at 25% of the power consumption. They KNOW the default configuration setting (which you can change in a few keystrokes) is total garbage in terms of power efficiency.
-----
I guarantee you server CPUs aren't going to be configured to be this idiotic out of the box. Because in the datacenter, perf/W matters and drives purchasing decisions.
