Long story short: We (Dataherald) just open-sourced our entire codebase, including the core engine, the clients that interact with it and the backend application layer for authentication and RBAC. You can now use the full solution to build text-to-SQL into your product.

The Problem: modern LLMs write syntactically correct SQL, but they struggle with real-world relational data. This is because real world data and schema is messy, natural language can often be ambiguous and LLMs are not trained on your specific dataset.

Solution: The core NL-to-SQL engine in Dataherald is an LLM based agent which uses Chain of Thought (CoT) reasoning and a number of different tools to generate high accuracy SQL from a given user prompt. The engine achieves this by:

- Collecting context at configuration from the database and sources such as data dictionaries and unstructured documents which are stored in a data store or a vector DB and injected if relevant

- Allowing users to upload sample NL <> SQL pairs (golden SQL) which can be used in few shot prompting or to fine-tune an NL-to-SQL LLM for that specific dataset

- Executing the SQL against the DB to get a few sample rows and recover from errors

- Using an evaluator to assign a confidence score to the generated SQL

The repo includes four services https://github.com/Dataherald/dataherald/tree/main/services:

1- Engine: The core service which includes the LLM agent, vector stores and DB connectors.

2- Admin Console: a NextJS front-end for configuring the engine and observability.

3- Enterprise Backend: Wraps the core engine, adding authentication, caching, and APIs for the frontend.

4- Slackbot: Integrate Dataherald directly into your Slack workflow for on-the-fly data exploration.

Would love to hear from the community on building natural language interfaces to relational data. Anyone live in production without a human in the loop? Thoughts on how to improve performance without spending weeks on model training?

user:	thedynamicduo
created:	January 12, 2024
karma:	11