Lessons from building a small-scale AI application

Lessons from building a small-scale AI application(thelis.org)

8 points by rdli 1 year ago | 6 comments

"There’s a plethora of developer libraries that claim to make AI development faster and easier. These libraries introduce new abstractions designed to improve your productivity. I tried many of them, and I found that the abstractions work well only if I stayed on a narrow path (the quick starts generally worked great!)."

This tracks my experience with developer libraries. Most of them are very brittle abstractions.

jackmpcollins 1 year ago | |

I believe we'll see two main types of AI/LLM libraries/frameworks emerge like has happened for ORMs (at least in Python). - The "Sqlalchemy" level: just bridge the gap between code and LLM - The "Django" level: additionally handle how data is indexed for RAG, standard patterns of connecting the pieces, possibly GUI, support for plugins.

It seems langchain and llamaindex are aiming to be the latter, and I'm building https://magentic.dev to be the former. I'd be interested to get your take on whether these abstractions would allow you stray from the narrow path while still being helpful!

rdli 1 year ago | | |

I took a brief look (~5 minutes). My $0.02 is that it's not clear what problem you're trying to solve. I get what some of the features do (e.g., templated prompts) but it would be v helpful to have an example of how you actually use magentic, versus the non-magentic way. It feels like a lot of syntactic sugar, if I'm being honest (not a bad thing, but something you might want to be clear about, if that's the case.)

rdli 1 year ago | |

(author here) I didn't put this in my post, but one of my favorite moments was when I read some of the LlamaIndex source code which pointed to the GitHub commit where they copied the code verbatim from LangChain. (LangChain is MIT-licensed, so it's OK, but I still thought it was funny!)

BrandiATMuhkuh 1 year ago | |

Same here. I feel like they are not used/tested in a real world scenario.

Given the space moves so quickly, the frameworks always lack massively behind what they try to abstract.

efriis 1 year ago |

Really like your writeup! I work on LangChain and would love to set up some time to chat if you're game. My email is in my profile