Show HN: Semantic grep with local embeddings(github.com) |
Show HN: Semantic grep with local embeddings(github.com) |
sam0x17 20 days ago:
Didn't want to bury the lead, but I've done a bunch of work with this myself. It goes fine as long as you give it both the textual representation and the ability to walk along the AST. You give it the raw source code, and then also give it the ability to ask a language server to move a cursor that walks along the AST, and then every time it makes a change you update the cursor location accordingly. You basically have a cursor in the text and a cursor in the AST and you keep them in sync so the LLM can't mess it up. If I ever have time I'll release something but right now just experimenting locally with it for my rust stuff On the topic of LLMs understanding ASTs, they are also quite good at this. I've done a bunch of applications where you tell an LLM a novel grammar it's never seen before _in the system prompt_ and that plus a few translation examples is usually all it takes for it to learn fairly complex grammars. Combine that with a feedback loop between the LLM and a compiler for the grammar where you don't let it produce invalid sentences and when it does you just feed it back the compiler error, and you get a pretty robust system that can translate user input into valid sentences in an arbitrary grammar.
In general, a drum I beat regularly is that during development the code spends most of its time incorrect in one way or another. Syntax errors, doesn't type check, missing function implementations, still working out the types and their relationships, etc. Any developer tooling that only works on valid code immediately loses a lot of its value.
I seem to have gotten 'lucky' and it split an emoji just right.
---
For anyone curious: this is great for large, disjointed, and/or poorly documented code bases. If you kept yours tight and files smaller than ~600 lines, it is almost always better to nudge llm's into reading whole files.
Really? My thinking is more that human devs are way too likely to sink time into powerful but complex tools that may end up being a yak shave with minimal/no benefit in the end. "too lazy to use" doesn't seem like a common problem from what I've seen.
Not that the speed of an agent being able to experiment with this kind of thing isn't a benefit... but not how I would have thought to pose it.
For example under the "Why CK?" section, "For teams" is of no substance compared to "For developers"
~/c/l/web % ck --sem 'error handling'
ℹ Semantic search: top 10 results, threshold ≥0.6
⠹ Searching with semantic mode...
My thinking is that for large codebase, sorting embedding matches maybe more efficient than reading all files and hence there is no point to put semantic search behind a --semantic flag
I added the VSCode plugin but it didn’t seem to help, likewise searching around yesterday I didn’t see anything surprisingly.
Looks like you have to build an index. When should it be rebuilt? Any support for automatic rebuilds?
It uses whatever arbitrary embedding model you want to point it at and backs it with a qdrant vector db. Roo's documents point you toward free cloud services for this, but I found those to be dreadfully slow.
Fortunately, it takes about 20 minutes to spin up a qdrant docker container and install ollama locally. I've found the nomic text embed model is fast enough for the task even running on CPU. You'll have an initial spin up as it embeds existing codebase data then it's basically real-time as changes are made.
FWIW, I've found that the indexing is worth the effort to set up. The models are generally better about finding what they need without completely blowing up their context windows when it's available.
versus https://github.com/BeaconBay/ck#:~:text=yes%2C%20completely%...
I did look into the core features and I gotta say, that looked quite cool. It's like Google search, but for the codebase. What does it take to support other languages?
My idea was to make a tool that just does a quick and simple embedding on each file, and uses that to provide a semantic alternative that is much closer to grep in nature, but allows an AI tool like Claude Code to run it from the command line - with some parameters.
Arguably could be MCP, but in my experience setting up a server for a basic tool like this is a whole lot of hassle.
I'm fairly confident that this is a useful tool for CC as it started using it while I was coding it, and even when buggy, was more than willing to work around the issues for the benefit of having semantic search!
Put the title aside, the tool, if it works as described, is pretty insane
(Submitted title was "Semantic grep for Claude Code (RUST) (local embeddings)")
Apart from anything else it appears to be very misleading as Rust (ironically) according to the documentation is not one of the languages supported.
(Submitted title was "Semantic grep for Claude Code (RUST) (local embeddings)")
(Submitted title was "Semantic grep for Claude Code (RUST) (local embeddings)")
Have they no shame?
Seems like CLI tools are all the rage these days
CC in my case likes it so much, it started using it to debug the repo rather than grep and suggesting its own additions
“We have a new grep semantic hybrid tool installed called ck - check it out using ck --help and take it for a spin”
If you're getting useful results from hybrid mode that's very interesting to me since well-constructed grep that claude executes don't really look like they'd work great for semantic search to me! But intuition is often wrong on this stuff.
I am very curious your thoughts on speed. I'd rather any tools claude invokes be as fast as possible so it can get feedback immediately and execute again.
i do a lot of context management with hooks for all sorts of tool calls.
Went to the github repo and was expecting a section about Claude Code and best practices on how to set this up with Claude Code. Very curious to hear how that might work, especially with what you've found compared to Claude Code's love of grep.
A write up on this would be great!
I have a lot of PreToolUse hooks that injects guideline messages whenever certain tools are called or bash commands run. My hooks also prune older versions of those out of context. All of the transcripts are in ~/.claude/projects/ in jsonl format and are hot-editable.