Meta buys 600k H100s to train LLaMa3(twitter.com) |
Meta buys 600k H100s to train LLaMa3(twitter.com) |
[0]: https://news.ycombinator.com/item?id=39044911 [1]: https://news.ycombinator.com/item?id=39046193
> “We’re so short on GPUs the less people use our products the better… We’d love it if they use it less because we don’t have enough GPUs” — Sam Altman
> Meta who has the 2nd most H100 GPUs in the world is increasingly leveraging GPU supply as a recruiting strategy.
https://www.datagravity.dev/p/2023-year-in-review-the-great-...
Though as a wise man [1] once said: "The more you buy, the more you save".
[1] Jensen Huang, NVidia's CEO, says this quite often at conferences, its become a meme at this point, one example https://twitter.com/NVIDIAAI/status/917681161391841280
lol, that's an investment!
Meta's annual revenues alone exceed $100B+ at this point, assuming these purchases are staggered over 2-3 years, it is basically adding a 2-3% overhead to their expenses on an annualized basis.
At the end of the day, the bottleneck is TSMC, on which all GPUs are produced right now, even the ones from Intel.