Show HN: Opty – A Zig-based HDC that reduces token use by up to 90%

Show HN: Opty – A Zig-based HDC that reduces token use by up to 90%(github.com)

4 points by bojo 81 days ago | 4 comments

I kept seeing people all over social media talking about how they were making custom local-LLM systems to help reduce the token load injected into their context window.

On the side my recent project has me looking at Hyperdimensional Distributed Memory. I couldn't help but wonder if I could make an MCP server to improve token usage. Sure enough, using a combination of HDC + TOOL format, I was able to get opty's own self audit down by 93% in token usage.

Still experimenting with large codebases but feel pretty good about how this should drive overall token usage down.

Happy to hear any feedback.

guld 81 days ago |

Interesting. Can anyone provide personal insights or benchmarks on how effective TOON compared to e.g., JSON or Markdown is (Codex, Claude, ...)?

verdverm 81 days ago | |

Ideas like this are bad ones. Words matter, you should put effort into them, minimization is not the primary optimization, don't let something like this MitM and change your hard work for the worse.

The reason people do custom is to craft very good instructions and tools, something a machine is not capable of

bojo 81 days ago | | |

Perhaps? I just used it to analyze one of my 96k Zig codebases using Claude Code and here is (part of) what came back. (I snipped out the deeper analysis above as it exposes my private project - but it was all correct).

  Head-to-Head

  ┌──────────────┬─────────┬─────────────┬────────────┐
  │    Metric    │  Opty   │ Traditional │   Ratio    │
  ├──────────────┼─────────┼─────────────┼────────────┤
  │ Input tokens │ ~13,500 │ ~39,408     │ 2.9x fewer │
  ├──────────────┼─────────┼─────────────┼────────────┤
  │ Tool calls   │ 21      │ 61          │ 2.9x fewer │
  ├──────────────┼─────────┼─────────────┼────────────┤
  │ Round trips  │ 5       │ 9           │ 1.8x fewer │
  └──────────────┴─────────┴─────────────┴────────────┘

I had it run a separate analysis using traditional vs. opty and count the actual tool calls and input token counts. My prompt was basically, "do a full analysis of this entire codebase."