Show HN: Lantern – a PostgreSQL vector database for building AI applications

Show HN: Lantern – a PostgreSQL vector database for building AI applications(docs.lantern.dev)

196 points by ngalstyan4 2 years ago | 43 comments

We are excited to share Lantern! Lantern is a PostgreSQL vector database extension for building AI applications. Install and use our extension here: https://github.com/lanterndata/lantern

We have the most complete feature set of all the PostgreSQL vector database extensions. Our database is built on top of usearch — a state of the art implementation of HNSW, the most scalable and performant algorithm for handling vector search.

There’s three key metrics we track. CREATE INDEX time, SELECT throughput, and SELECT latency. We match or outperform pgvector and pg_embedding (Neon) on all of these metrics.

** Here’s what we support today **

- Creating an AI application end to end without leaving your database (example: https://github.com/ezra-varady/lanterndb-semantic-image-sear...)

- Embedding generation for popular use cases (CLIP model, Hugging Face models, custom model)

- Interoperability with pgvector's data type, so anyone using pgvector can switch to Lantern

- Parallel index creation capabilities -- Support for creating the index outside of the database and inside another instance allows you to create an index without interrupting database workflows.

** Here’s what’s coming soon **

- Cloud-hosted version of Lantern

- Templates and guides for building applications for different industries

- Tools for generating embeddings (support for third party model API's, more local models)

- Support for version control and A/B test embeddings

- Autotuned index type that will choose appropriate index creation parameters

- 1 byte and 2 byte vector elements, and up to 8000 dimensional vectors support

** Why we started Lantern today **

There's dozens of vector databases on the market, but no enterprise option built on top of PostgreSQL. We think it's super important to build on top of PostgreSQL

- Developers know how to use PostgreSQL.

- Companies already store their data on PostgreSQL.

- Standalone vector databases have to rebuild all of what PostgreSQL has built for the past 30-years, including all of the optimizations on how to best store and access data.

We are open source and excited to have community contributors! Looking forward to hearing your feedback!

bryan0 2 years ago |

> Switch from pgvector, get FREE AirPods Pro.

> Book some time here, and we will help switch you over for FREE and get you a pair of FREE AirPods Pro

This just comes off as sketchy to me. If the tech is good it will stand on its own.

gregwebs 2 years ago | |

Although it doesn’t meet the definition of bribery it’s a similar concept. It’s trying to influence someone’s decision by personal gain rather than just what’s best for the company. DataDog used to do this (maybe they still do) where if you signed up for a trial they said you would get a 1 in 8 chance of getting an iPhone.

swyx 2 years ago | | |

for like the last 8 years they have promised a shirt if you send your first log. i think its ok if its low value (<$20). when its airpods pro level (>$200) then it seems desperate. that said, this is basically paying for some real time with serious customers. not the worst idea, just doesnt scale but who cares

rawrawrawrr 2 years ago | |

This is fine, its a standard enterprise marketing technique. Watch a webinar, get a gift card.

siva7 2 years ago | |

That’s like saying if the product is good it will stand on its own. So far the theory. As most founders here know it’s not so easy.

carlossouza 2 years ago |

I'm using pgvector in production, mainly in a table with 500k-1M rows.

My main use case is to return search results with pagination: page 1 from 1-50, page 2 from 51-100, page 3 from 101-150, etc. (Think LIMIT and OFFSET).

After a lot of experimentation and help from pgvector's team, we discovered that, for this specific use case, IVFFLAT index is much faster than HNSW.

I looked at your documentation and only saw HNSW, no IVFFLAT.

What would be Lantern's performance for this specific use case?

Thx!

ngalstyan4 2 years ago | |

This sounds like a very useful feature, and we will prioritize this.

You’re correct that IVFFLAT would be faster for your use case. However, IVFFLAT’s shortcoming is bad recall, which means less relevant results for your application. We believe that our HNSW implementation (or other indexes) can handle use cases like yours.

We currently handle a similar use-case by rerunning our index searches with exponentially increasing LIMITs and dropping the results which are not needed. Could you send us an email at support@lantern.dev? We can generate the numbers by this weekend, and get back to you with concrete results.

By the way – not sure if you saw in our blog post, if you’re using pgvector in production and switch to Lantern, we’ll help you every single step of the way. It’s very quick, and we’ll also send you some free AirPods Pro at the end of it!

simonw 2 years ago |

"There's three key metrics we track. CREATE INDEX time, SELECT throughput, and SELECT latency."

There's a fourth metric that I'm really interested in: assuming it's possible, how long does it take to update the index with just one or two updated or inserted vectors?

Is the expectation with this (and the other) tools that I'll do a full index rebuild every X minutes/hours, or do some of them support ongoing partial updates as data is inserted and updated?

Just had a thought: maybe I could handle this case by maintaining an index of every existing vector, then tracking rows that have been created since that index itself.

Then I could run an indexed search that returns the top X results + their distance scores, then separately do a brute-force calculation of scores just for the small number of rows that I know aren't in the index - and then combine those together.

Would that work OK?

Even if the index doesn't return the scores directly, if it gives me the top 20 I could re-calculate distance scores against those 20 plus the X records that have been inserted since the index was creation and return my own results based on that.

diqi 2 years ago | |

> There's a fourth metric that I'm really interested in: assuming it's possible, how long does it take to update the index with just one or two updated or inserted vectors?

Here’s a chart for INSERT latency (sorry about the formatting): https://docs.lantern.dev/graphs/insert.png

At the moment, we underperform Neon wrt this metric, but a better implementation is coming soon that will address this.

> Is the expectation with this (and the other) tools that I'll do a full index rebuild every X minutes/hours, or do some of them support ongoing partial updates as data is inserted and updated?

The HNSW algorithm updates the index after every insert. So all existing HNSW options (Lantern, pgvector, Neon, …) already support this.

With pgvector IVFFlat, you expect the performance to degrade over time, and you will need to re-index. This is because IVFflat’s index quality heavily depends on the centroids chosen at index creation time. HNSW does not have this limitation.

In both cases, you might want to do a full-index build to tune your hyperparameters.

We’re working on this in a few ways. One is automatic hyperparameter tuning. Another is supporting external index creation that would offload this to another server. Does this answer your question?

simonw 2 years ago | | |

That's really useful, thanks.

ezra-varady 2 years ago |

Hey everyone for those interested I built an updated version of the lanterndb semantic search application that should be a bit nicer. An instance is running at

http://170.187.170.169/

And code can be found at

https://github.com/ezra-varady/react-semantic-search

fakedang 2 years ago |

This might be a noob question but what does Lantern have that a normal Postgres dB with pgvector does not? I think Supabase already has a Postgres as a service product with the pgvector extension too.

Second:

>Creating an AI application end to end without leaving your database (example: https://github.com/ezra-varady/lanterndb-semantic-image-sear...)

What does "without leaving your database" mean in this context?

ashvardanian 2 years ago |

Epic result, and thank you for mentioning USearch! Would be happy to further optimize it for your needs!

I also love your specialized CI! Pgvector probably doesn’t report performance changes between releases, or does it? Was it easy to implement? Do you run the whole eval on GitHub?

diqi 2 years ago | |

Thanks!

I don’t believe pgvector reports performance changes between releases.

At the moment, we run the benchmarking on Github CI, but we plan to move this to an external machine, since the results are unstable on Github machines. We’re planning to extend benchmarking across other repos and versions.

howon92 2 years ago |

Congrats on the launch!!! I suggest you highlight "why lantern is better than pgvector" at the top of your page. The first thing that came to my mind after reading this was "why should I use this instead of pgvector?"

jeungsp 2 years ago | |

This is a great idea. We will have lots of content in the days ahead to talk about why people should make the switch.

Do you use pgvector now? Would love to switch you over.

mattashii 2 years ago |

How do you do cleanup of the index during VACUUM?

And, do you have recall-vs-qps graphs like those on https://ann-benchmarks.com/ ? Those are generally more apples-to-apples, as 100k rows isn't exactly a reputable benchmark.

diqi 2 years ago | |

We don’t do cleanup of the index during VACUUM yet. That said, it’s coming very soon. We’re built on top of Usearch, which supports deletes. We plan to work with the Usearch team to port the post performant deletes to Lantern, and thereby support VACUUM.

With respect to recall vs QPS, we went ahead and generated this plot, hope this is helpful? http://docs.lantern.dev/graphs/recall-tps.png

You're right, 100k rows isn’t a reputable benchmark. We wanted to launch very quickly, and have benchmarking for larger datasets coming soon. Benchmarking is baked into our CI/CD, we take it very seriously!

skeptrune 2 years ago |

Looks like the AirPods bit was removed. Are those still on offer? :/

loxias 2 years ago |

Curious about how it scales. Plenty of solutions look good with small amounts of data, but completely fall apart past a point. Would be interesting to see the latency numbers for 1M, 10M...

raoufchebri 2 years ago |

How do you handle conflict with pgvector's hnsw if you want to install both extensions ?

CREATE INDEX semantic_image ON image_table USING hnsw (v dist_cos_ops) WITH (M=5, ef=30, ef_construction=30, dims=512);

ngalstyan4 2 years ago | |

Our index access method will be called lantern_hnsw if pgvector or any other provider has already taken the hnsw access method name.

btw, we did not create our own vector type and just use size-enforced real[] arrays to represent embeddings. However, you can use our index with pgvector's vector type. So, if you already have a table with pgvector's vector column type, you can start using Lantern by just creating an index on the same column.

jw903 2 years ago |

Impressive performance. In your experience, is there a range of vector dimension for faster search results?

ngalstyan4 2 years ago | |

We have not run microbenchmarks to see what dimension ranges perform best but those are coming soon! Below is an anecdotal answer:

We run our ci/cd benchmarks on 128dim sift vectors. We have some demos using clip embeddings (512dim) and baai/bge 768 dimensional embeddings.

Generally, smaller vectors allow higher throughput and result in smaller indexes. But the effect on performance is small. Once we merge the PR implementing vector element casts to 1 and 2 byte floats, the effect of this on throughput should be even smaller.

dalberto 2 years ago |

Any plans to support sparse vectors?

jeungsp 2 years ago | |

We’re built on top of Usearch, which will very soon support sparse vectors. We’re working with them to make sure it also works in Lantern.

Can you tell me more about your use case?

dalberto 2 years ago | | |

For hybrid search

swalsh 2 years ago |

I like how easy postgresql vector is to use, but scaling up seems to get pretty expensive when comapred to something like qdrant.

jeungsp 2 years ago | |

Nothing fundamentally stops a postgres implementation being equally performant as something like Qdrant.

Fundamentally, an index’s performance is based on the hardware and the algorithm and the quality of implementation. Any optimizations Qdrant can make, we can also make.

We will be benchmarking ourselves against all of the other standalone database options as well and we’ll be working to try and outperform them. Excited to share those once we have them.

We are curious – what are you most concerned about? SELECT time? INDEX size? Latency? Throughput?

swalsh 2 years ago | | |

I never spoke of performance, clearly postgres can match performance. It just seems more expensive in terms of dollars to do so.

justanotheratom 2 years ago |

can I use this in Supabase?

diqi 2 years ago | |

Unless Supabase decides to integrate Lantern (currently they integrate pgvector) you unfortunately cannot use Lantern with Supabase.

That said, we will offer Lantern Cloud, our own hosted postgres offering (very soon. Happy to keep you in the loop. If you’re interested, please feel free to join the waitlist here: https://forms.gle/PouJxAWiSa63udJW8

radus 2 years ago | |

Not a Supabase user, but to my knowledge -- no. Except for a set of blessed extentions, Supabase only supports extensions that are written in "trusted languages" (ie. supported by pg_tle), and Lantern is written in C, which is currently not supported.

diqi 2 years ago | | |

pgvector is written in C and is supported by Supabase. There's nothing inherent preventing Supabase from supporting Lantern.

consoomer 2 years ago |

Not going to lie.. the more I use Postgres the more I realize my entire application is Postgres. Soon you'll be doing entire CRUD endpoints and sending emails from Postgres...

Wait, PostgREST already does...

builds entire SaaS with Postgres

_boffin_ 2 years ago | |

Postgres is love. Postgres is life.

codetrotter 2 years ago | |

PGaaS :D

SELECT abstract, introduction, figure1, clip_text(abstract) AS abstract_ai, clip_text(introduction) AS introduction_ai, clip_image(figure1) AS figure1_ai INTO papers_augmented FROM papers; SELECT abstract, introduction FROM papers_augmented ORDER BY abstract_ai <-> clip_text("The Foundation of the General Theory of Relativity") LIMIT 10;