I’ve been exploring how recommendation systems are actually implemented in production, beyond just training models.

A common pattern I kept seeing is to split the problem into two stages:

1. Retrieve a small set of relevant candidates

2. Re-rank them using a model

Instead of doing brute-force inference across all items, I built a small prototype around this idea.

The flow looks like this:

- Store embeddings in a vector database (ChromaDB)

- Retrieve the Top-K most similar items/users based on vector similarity

- Run a TensorFlow.js model to re-rank the candidates

The goal is to reduce the search space before applying inference, which seems necessary when latency and scale matter.

What I found interesting is that once you move to this approach, a lot of the complexity shifts from the model itself to the retrieval layer:

- choosing K

- filtering candidates

- embedding quality

- latency vs recall trade-offs

Curious how others approach this in real systems:

- How do you decide on K?

- Do you rely purely on vector similarity or add heuristics?

- How do you handle re-ranking at scale?

Project: https://github.com/ftonato/recommendation-system-chromadb-tf...