Charts built for Chat(dbtcharts.com) |
Charts built for Chat(dbtcharts.com) |
very disappointed that their domain specific language is just yaml. conditionals and variable binding become insane war crimes when yaml comes to town. you have a friggin llm, do better
and when we computer folks see the word language its implied that its a computing on computers language not a friggin data format.
edit: the yaml to avoid complexity that should live in the sql side can back fire, some folks I was working with last fall were evaluating if a yaml based OLAP tool would work for them, and there were some pretty gnarly gotchas from a yaml based approach. Secondarily, theres a real case to be made that having the data linkages not visible in the charting layer means that groups of related plots with different axes wont have the right data linkage without forcing a lot more ETL for what should be a quick plot if the data already fits in memory.
theres actually a very important reason you want it to be an actual embedded dsl or tiny programming language!
The reason why llms can code at all is the hugeeeee amount of RL based on the loop of 1 "write code", 2 get compile time or runtime errors,3 fix it and iterate. Data file formats dont have that feedback loop so models will fall off the rails faster. Writing code that fits a latent adhoc schema just wont work as well, or will require burning a lot more context.
from that perspective, it could just be an EDSL little library in the host language, or it could be a friggin little custom language with an interpreter and good error messages.
That lack of flexibility is actually a feature for one of the bigger problems we're solving: lineage, or knowing where a number came from. If the chart layer can't transform data, the logic stays in SQL, where you can version, test and audit it.
It's also a choice about who this is for. The data community already works in SQL, Jinja and YAML every day, so there's no new language to learn.
Looking forward to the yaml confusion when helm/dbt both see their default "charts/" directory in the same repo, or the dbt chart is set as a config map so it can be updated without rolling a new version of the full app... helm_argo is already a nightmare
rant aside, this does look super useful, and I do have CUE to help with the yamhell, but I may still prefer js/ts options so I can dynamically change the chart (like user clicking a dropdown for a different set of data). Sounds cool until the "dynamic" part of the chart shows it's limitations in crafting your ideal UX
I'll definitely be taking this for a spin, gets at that unbundling and "I need a quick chart" situations
I discovered CUE after trying to do too much in yaml, after adding imports I had to step back and ask what I am I doing. It's super hacky, a real language is much better long-term in reliability and maintainability.
DevOps seems to enjoy the pain, unsure why we continue to use messy tools. CUE is actually nice and fun to use before any comparisions to existing config formats.
Gosh I'm so "behind" it's painful. /s
> dct validate <file.yml>
Or any of the other commands that read the file will quickly fail with any syntax or SQL issues. It’s not just an open schema.
to be clear: youre taking this feedback wonderfully well and i appreciate that maturity :).
im just cranky with the current opportunity market and mucking with attacking a classical markov chain conjecture while i hurry up & wait to hear back from places XOR slowly build some stuff in the harness space that i hope can hit the equivalent of sublime text tier but for harnesses (is there even a market for a prosumer/pro tier harness, im not sure!)
DBTCharts says you can serve charts locally, but it seems they want you to use their hosting service in production. Whereas, Malloyyo / Publisher are free to use anywhere. If you like college football checkout how I visualize drive data in [2], made with Malloyyo.
[1] - https://github.com/malloydata/malloyyo [2] - https://mrtimo.github.io/cfb-games/games-2026.html?%24SEASON... [3] - https://github.com/malloydata/publisher [4] - https://community.credibledata.com/hackernews
Publisher runs the model: you define the what, the engine figures out how. You write down what counts as a sale, how revenue is calculated, who sees which rows. Reshaping the data out of the form your systems store it in, deciding what to precompute, serving whichever agent or dashboard or API is asking -- that's the Publisher's job.
A chart spec has to get its numbers from somewhere. If "revenue" is defined inline in the chart's SQL, the definition problem moved, it didn't get solved. Define it once in a model and the chart just renders it.
Engineering detail if anyone wants it: https://www.credibledata.com/blog/posts/inside-the-ai-analyt...
"Unbundling BI" is absolutely where things are headed now that more people have agents, coding agents, agent computers to help with work.
I'm seeing this all up and down knowledge work tools. I've started treating email as a BI problem -- ETL it from Gmail and and create many different views into it and reports from it.
I just wrote up some thoughts on that here: https://housecat.com/blog/making-gmail-data-fast-for-humans-...
A natural followup is how to better visualize this data in chat. The DBT table component looks like it could help https://docs.dbtcharts.com/charts/tables/
> Make charts of this with dbt Charts. Start with: uv tool install dbt-charts && dct skills intro
>
> build a dashboard of my hiring inboxes: list of candidate name / email / locale, application quality, response age
The one-shot dashboard is surprisingly good.
>"Unbundling BI" is absolutely where things are headed now that more people have agents, coding agents, agent computers to help with work.
Agreed. Coupled with bento[0] for slide decks, I think this type of project is a very welcome development, helping us move away from walled gardens and proprietary software suites.
The rocket either lands or doesn't, safely, as expected. Green and Red lights are abstracted data plots.
The context should be the focus.
"Rocket must land at less than 0.2/mps" and pump the data into that context filter, more - reduce velocity, less - green light.
Because your eyes are enormous bandwidth channels hooked right up to your brain. So high that we can take things like writing which is just a small part of your fov and a set of squiggles that roughly corresponds to sounds you’ve heard and effortlessly comprehend it.
> The rocket either lands or doesn't, safely, as expected.
Crash vs land is a terrible amount of data to have if you want to fix it for next time. Did it accelerate suddenly? Is there a lurch? Did all the accelerometers have a small jump at the same time? Visualising data in different ways can really help show patterns.
Who knows, with AI this may be the future. Visualizations were created to help us understand data. Florence Nightingale published them "to affect thro' the Eyes what we fail to convey to the public through their word-proof ears."
Often a formula better represents a situation. But I'm not sure how much impact they'd have in spreading that understanding outside of a formula-minded audience.
e.g. when highlighting a quarterly revenue result, there is a big difference between missing the target by 1% or 50% (and of course the absolute number behind it is also important).
And on a larger scale, trajectory is also something that is easily visualized in graphs, and hard to boil down to a binary marker.
There certainly is a lot of "forced synesthesia", and more looking at charts than necessary out there, especially when decisions are then made more from the gut than based on the data one is claiming to base the decisions off, but a culture of binary-based decision making is just as bad of an idea.
Sure the data is correct, but it's obviously way easier for me to read it as a properly rendered table. Then you want to sort, group, aggregate the data. Visualize it many different ways if you will :)
If you've ever used Aruba Central, this methodology is everywhere, absolute dogshit networking tool but there is a graph for absolutely everything. There's a VLANs overtime graph which provides information that absolutely 0 network engineers in the world would ever need to do their job.
When making dashboards with claude or other agents, a lot of free-form artifacts are created that makes it hard to audit and scale. dbt Charts is a simple YAML dialect that declares and renders a chart (think markdown but for dashboards). Along with dbt its Apache 2.0 and launched today.
We hope this language + AI help make the BI space more open with dashboards as concise auditable code. Would love any thoughts and feedback.
We already have variables and links (which work with variables)[1]. Variables create interactive menus that update urls, queries and charts. If you have a variable `region` and you set it the url will add a get param ?region=<value>
Each chart also has a basic `link:` property can use to link to the existing or other urls with the variables set. For example
type: bar x: revenue y: region link: '?region={{ region }}'
The result is a big range of flexibility, for example [2] here's an example of Conways Game of Life inside of dbt Charts.
[1] variables example - https://play.dbtcharts.com/?example=variables%2Fall-input-ty.... [2] Conways game of life - https://play.dbtcharts.com/?example=variables%2Fgame-of-life...
Dave built Chart.io and been seeing how much they’ve poured into this from his decade of experience for gnarly data.
You don't really want to give a LLM client code execution environment like Observable Framework does. dbt charts appear to validate the entire yaml input including the SQL being sent to the data source.
So I'm getting: a data vis layer that can be dynamically defined and rendered via LLM/MCP client without the headache of sandboxing a JS Runtime environment.
Maybe this is targeted to pure SQL folks that can get lost in UI details.
PS: one issue that I have with observable framework is authentication and authorization , it requires some system to be built on top of it to handle authn.
i'll compare and look at folding this into setoku for app generation [1]. right now apps are just html blobs your claude authors + a mechanism for populating them with live data. definitely hard to audit but very flexible for operators to claude together internal apps. anyway, the charts look decent but really depend on the model that's making them and don't really follow any sort of style guide (example: https://demo.setoku.com/apps/a7a1240ae0bc202c5eefa1cc). Your lib could bring some consistency and make global styling possible.
[0]: https://vega.github.io/vega-lite/
[1]: https://setoku.com
So yes, a nice and logical development of dbt, but hardly as innovative as the blog post wants to sound. Nevertheless, I think it's a good idea that will be popular in certain circles. Hiring a professional data designer is a good idea - data visualization is very easy to get wrong.
One of the key claims is actually that it's based on _old_ established visualization patterns from before BI tools were popular, and therefore results in consistently nice charts. The other key claim is just that it's simple and maintainable.
Sometimes tools & products can be useful as just well executed points in the known design space. You can already do all this with AI, but it's perhaps a little bit less nice and less maintainable.
[I work at dbt/Fivetran]
AI can generate reusable artifacts including definitions of the data sources, why not? Most BI dashboards are just XML or JSON (or YAML) files and include definitions of data sources (directly or via semantic models). I struggle to see how dbtCharts is different. Yes, your YAML schema is clean and nice, but that's because you're in the early stages :) Once you go through feature bloat, your YAML format will become much more complex.
AI can generate an XML/JSON/YAML definition of a BI report according to a spec and link the data source in whatever form it should be referenced in the file. For instance, here is a skills file for defining data sources (semantic models) in AI-generated Power BI dashboards: https://github.com/microsoft/skills-for-fabric/blob/main/plu...
This is obviously a space we are very much interested in, so it is definitely nice to see approaches that attack the same problem. I haven't played with dbtcharts in depth yet but it looks very similar to DaC in principle, and also in the actual spec. I believe the industry definitely needs solutions like this to help get out of the legacy BI tools as Bİ is one of the biggest bottlenecks for AI adoption in large orgs.
Excited to see further competition in the space, nice launch!
On the standardized charting language, that would be the dream. All agents giving the same spec when they want to build a chart or dashboard, and then having different renderers of that. That is a steep goal. Such a language would need to be simple but still extremely versatile. I'm not sure that combination exists yet. Our language is simple and fairly versatile, but not as versatile as vega-lite itself, or JavaScript code even. Maybe we can evolve in that direction though
I have to dig in a bit more to what is possible in https://docs.dbtcharts.com/charts/extensibility/ and https://docs.dbtcharts.com/charts/interactions/ so far, and how feasible it is to actually use the end result in a customised website.
Also similar: https://github.com/microsoft/flint-chart
This is also likely to compete with features already provided by the warehouse such as https://docs.databricks.com/aws/en/dashboards/manage/visuali...
Is this list growing over time or shrinking or remains the mostly the same?
If the list is growing, is it growing evenly across all teams , or some teams stand out? Can I narrow down to a few teams and compare?
What about overall health of the inventory - how many instances list security or billing optimization findings, how many instances are on the version that would be out of support soon and would need to be upgraded?
A dashboard can answer all those questions immediately and make issues visible, as opposed to a long list in a form of a spreadsheet.
on edit: So a dashboard is just one of those things were you see all the things you can do or interact with assembled together and from which you can navigate into any particular tool or view of data. Probably there would be meaningful data views on top. Like Number of Purchases versus people who started a purchase. And you could then drill down into the data that this chart represented to see how long purchases took, repeat customers, when people leave process or purchase etc. etc.
The dashboard is the entry point for how you will navigate your business data.
People are already used to getting *exactly* what they want from an agent. e.g. in our agent (https://www.definite.app/) we often see people take pictures of a sketch on a piece of paper or a screenshot from a few other tools (Stripe + Excel).
Our agent has templates to start from, but ultimately writes a react app to give the user what they want. It'd be hard to get that experience in a framework like this.
So, are those other pieces going to be part of dbtTran Cloud?
> There’s governance, access controls, interactivity, and connectivity to semantic layers
I'd add to that list as well: KPIs, dropdowns, text boxes, theming, shared data sources.
If you're familiar with Vega-Lite [1], I like to explain dbt Charts as the Vega-Lite but for BI, in that it is a language that specifies all of the components of BI.
> are those other pieces going to be part of dbtTran Cloud?
Certainly there are some features that are more conducive to being offered as a managed service. However, we've built this enterprise BI tool backwards in that we've started less with commercialization in mind, but rather laying the groundwork in the language and OSS project, so that we need not reinvent the wheel over and over again.
I'm hand-waving here about a future that doens't yet exist. But in theory dbt Charts could be extended in the front end to allow for not just SQL or SL metric support, but any query language. Same for the backend, we dbt Charts can render to png and html today, but other formats are also possible!
https://github.com/preset-io/headless-bi-blog-post-examples/...
But this is mostly just a dump of internal Superset data structures.
The raw HTML/SVG or base libraries approach may well win out, but it does make it quite hard or impossible for humans to follow along and verify for instance where the numbers on a chart came from.
I think in a future where AI's doing all that verifying (or we just trust it), the AI might still prefer to use a DSL like ours because the abstraction maintains consistency, lowers maintenance, and saves a lot of tokens.
But the most helpful bits of a structured DSL are for sure still for humans. The structured format ensures things are readable and testable. Ours also enables a generative UI, which for now at least is still a much faster way to make visual edits while working with an AI, vs always through it.
All of my ai-driven plans use a tool I built to rasterize all text based visualizations. And I make the model present the rendered version to a clean room agent for brief-back. They go in circles until the diagram is good. Then, I instruct planning tools to always look at both the text and rendered version of all diagrams. I have found it to be surprisingly powerful.
AI slop has a "smell" meaning there's an abstracted data filter that results in rose or poop output. Shakespeare variable says the name doesn't matter.
I've been mulling extending dbt Charts to support ggplot, and will likely reach out to the team about this in the next few weeks.
The difference is that Vega-Lite, ggsql, ggplot2 are designed to create a single chart. Yes, you can use facet_wrap() to great a series of charts based on a particular variable, but arguably that's still one chart!
What's missing that is table stakes for a BI dashboard: KPIs, text boxes, dropdowns, data sources, governance, and how the charts are arranged on the page and in relation to one another.
In the next few months we hope to ship a JSON schema that represents a dashboard in the same way that a VegaLite does for a single chart.
Does that make sense?
I guess at the moment GGSQL itself wants to be directly connected to the source database, whereas you've got a split between queries and the charts which display the results.
It would be really cool to have SQL "all the way down" to the chart definitions as well :)
What drew me to the dbt project back in 2020 and what ultimately led me get a job at dbt Labs was this notion that when compared to software engineering teams, analytics teams had been vastly unserved by their tools.
In 2016 (and largely to this day), BI teams work in point-and-click interfaces without Software Development Lifecycle mainstays like source control, testing, and deployment environments.
There are BI engineers who do work like software engineers, but the gap b/w those who work like SWEs and those who don't is wide.
dbt Charts (like dbt before it did for data engineers) aims to empower data analysts to work with stakeholders in a more efficient and sustainable manner than was previously possible. The means to this end are: a succinct YAML DSL spec for defining charts, a powerful CLI that lets you validate, compile, and render the code into a dashboard spec the YAML.
Like data analysts being able to do PRs for changes to a dashboard is still rather unheard of especially if those PRs are in repos with other analytics code. Many BI vendors do ship some form of diffing and environment promotion, but virtually always these features fall short of what SWEs use every day.