1) Reviews are super fast. Sometimes I get the first review 5 minutes after submitting a PR. Today I had a PR merged with two reviews within 42 minutes. This is incredible work by the whole team. It keeps contributors motivated. The pace is intoxicating. On other projects, I've sometimes been waiting months for a review.
2) Georgi motivates people by giving them responsibility. There's no gatekeeping as with other projects. He happily delegates. You do good work? It's appreciated, and you get the freedom you need to make an impact. That feels awesome.
I've rarely become addicted to an open-source project so quickly.
I know there's a lot of discussion about AI overwhelming OS maintainers and I can see that. It does seem like llama.cpp is successfully riding that dragon right now though.
Of course, llama.cpp also gets lots of AI slop PRs from first-time contributors, but from what I've seen, maintainers simply ignore the bad submissions and prioritize the good contributions. I think that's probably one of the best ways to handle the slop overwhelm.
- The projects started without HF/nvidia involvement and were massively successful BECAUSE folks want to run models on their own devices.
- With highly capable agents, "hard to implement" should be less of a barrier. The inference market should in some sense become more efficient, as agents should make it easier to transition between software and hardware solutions. Sure there might be less training data for integration platforms, but if we've learned anything in the past few weeks, it's that agents can be remarkably persistent.
This is like a company buying your local public pool, and you trying to convince everyone that things won't slowly turn to crap.
What we should be seeing is competing businesses trying to undercut Nvidia/HF and offer a better experience. China has alternatives to HF; it's just that in the US, Nvidia is the only business that can get off their ass to sponsor infrastructure like that. America's poor underdogs like Intel and Apple simply can't be expected to make investments into real competition anymore. Nor can our startups be expected to find market traction when incumbents like HF are so deeply ingrained in the ecosystem.
Fact is, the pool was already crap. The rest of the industry had close to a decade to formulate any type of response, and America's vanguard took that time to rest on their laurels and formally forfeit to Team Green. Nvidia's played the "Luigi Wins By Doing Absolutely Nothing" card for the entire datacenter market: https://youtu.be/m6PxRwgjzZw
X threw a fit.
Hugging Face has been acquired by NVIDIA
It is quite exciting to be a part of this journey! NVIDIA has been an active supporter of the llama.cpp project. For more than a year now, their engineers have actively contributed to the codebase, collaborated with the community and provisioned hardware for development and testing purposes. The local AI ecosystem has largely benefited from our joint efforts.
Going forward, llama.cpp/ggml will stick to its founding principles. One of the most important qualities of the project is to be hardware-agnostic. Therefore the development and support of all backends will continue to be done as usual - driven and shaped by the community. Open to everyone who is willing to participate. The existence of such an independent software platform is crucial for the rapid adoption of AI locally and for bringing it closer to the user.
Now, with such a significant partner as NVIDIA supporting us, in addition to the wider Hugging Face team, I believe that our long-term goals will be easier to achieve. We will remain focused on building one of the most exciting projects based on the most exciting technology of our lifetime and making it truly available to everyone in the most accessible and efficient way possible.The service XCancel is stopped until further notice.
We are seeking legal advice and won't share more details for now.
Thank you for the trust you have put in these two years of XCancel."
Can we take this as an official statement that Nvidia supports local models?
Why would Nvidia increase GPU efficiency for local models? Surely they'll operate like athletes and only establish a new record from time to time when necessary.
The first reply on the X thread is by Gerardo Delgado, who is "Sr. Director of Product - Local AI, Creators, Developers @NVIDIA" according to his profile. In his post, he says "My team's goal at NVIDIA is to grow Local AI."
Hedging against a data center bubble burst or at least massive pullback
Trivial for them to cut margin on desktop cards and clean up/prop up falling data center sales serving the hungry gamer, blockchain, local AI that's been sitting on its hands the last year or two
Nvidia is the PC ecosystem's best answer to Apple
Sorry… what does this mean? I thought most people considered Apple irrelevant because of its low market share.