This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept.

The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.

I strongly recommend the authors rewrite the readme by hand. It’s kind of a snif test for how much care someone put into this project.

One of the 2 dependencies, httpx, isn't really maintained anymore. Pydantic picked it up as httpx2: https://pydantic.dev/docs/httpx2

Funny, everything you pruned away is the reason I’m deploying LiteLLM in our platform. Having a reliable way to track token spend per customer across different services is important to us, and LiteLLM handles this well

I like the API, and since this is open source I will copy the design with attribution: I have been organically hacking little bits of LLM client code for most providers in Common Lisp for about 3 1/2 years and it is time to clean up all my old code. I probably need to do the same sort of refactoring for my search API wrappers.

Thanks for the cool project.

It would be great if there were a plugin/extension infrastructure. For example, to write the cache and costs however and whenever I see fit

I imagine many people code their own LLM client after getting fed up with the bad options out there. It’s very easy with ai coding tools.

I’m biased but I think mine is coded to a higher standard than litelm. https://github.com/s-banach/langchaint

First off, cool project! It's always great to see derivatives that question the efficiency of the established product.

I think the main thing the readme is missing is the core benefits. Reducing LOC and dependencies is cool, but it would be great to understand if this provides some additional benefits like lower latency or memory requirements.

Cool project, I had to implement the same for my experimental AI Agent: https://github.com/quantized-ai/luca-py

LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs.

Btw, you should update httpx to httpx2, and I think it's not much effort to remove openai's SDK compatibility.

I'd love to have a provider-agnostic LLM router (almost) dependency free (aside from httpx2).

ROTFL, I had to laugh so hard!

"Without the Bloat"

vs.

"litellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC"

Seriously anybody considering 100k+ LOC not a bloat? You made my day!

Let's just say the author's and my definition of bloat is not the same. Full disclosure, I'm the guy who reimplemented etcher (over 400Mb) in a mere 300Kb, Capstone (over 1Mb) in only 66Kb and who compressed LPC charactersheets (over 700Mb) into 4Mb. That's my interpretation of "non-bloated".

Also Strongly recommend renaming to avoid confusion

the smaller surface is nice. i'd still keep auth and spend caps outside the proxy though, because once every app shares one key the blast radius gets ugly fast.

Drop the Lite, it's cleaner, just use the godamn LLM directly.

Immediate first impression is that this tagline should go. If the project brings something valuable to the table it doesn't need to shit on other permissively licensed open source projects to make a case for itself

most software like this will be dematerialized, democratized, and demonetized - companies building in the infra band being increasingly disintermediated

Make a docker image rootless please, thanks

This is a 30 minute project with a frontier LLM. I don’t see why anyone would use anyone else’s router. Techniques are valuable today. Libraries are not.

token counting is useful though. There's also other ways to reduce size, ie a plugin API.

This readme, when it talks about all the different AI endpoints it can use, reminds me of something.

I'm not an AI bro, but I've dabbled. It's kind of remarkable that all the different providers speak the same "openai compatibile" https endpoints. In other realms of software development, real interoperability like that can be kind of rare. Even if people support conceptually the same API, everybody always puts their unique incompatible spin on it. In the dabbling that I've done, big incompatibilities seem rare.

How does it compare to Bifrost?

I have the same complaint about oh-my-pi's readme. The tone is obnoxious. It's somehow jaded and matter-of-fact at the same time. Like it was written by that one guy at work who never misses a chance to brag about how clever he is.

Agree. The LLM'isms are offputting.

This readme is better than most readmes. However they came to making it, it's clearly working

Throwing my support for this. Do not use LLMs to write things humans should write.

Really? I mean, yeah, it's probably written by an LLM, but it's hardly the worst that I've seen. Looks way more straight forward than the modern README featuring a ton of badges, emojis, confusing out-of-context screenshots, "trust me bro" installation instructions, vague elevator pitches, "used by netflix, nasa, disney, good morning america, alex jones, the church of scientology", and other verbiage to create the illusion that the author won't immediately get bored and abandon their glorified dissertation piece. They all scream "give me your github stars" whereas this one doesn't. But I still get what you mean when it comes to the particular 'isms.