Hey HN – Stefan and Elijah here from DAGWorks (http://dagworks.io/, YC W23).

If you don’t remember us from our previous HN launch (https://news.ycombinator.com/item?id=35056903), we’re the authors of Hamilton (https://github.com/dagworks-inc/hamilton), an open-source library for building self-documenting, modular dataflows in python that works for data, ML, LLM pipelines, & even web-workflows.

We’ve been developing this UI for a while and we’re excited to say we open-sourced it! It comes out of the box with the following capabilities, and only requires a single line code change to get:

1. Execution + metadata capture, e.g. automatic code profiling

2. Data/artifact observability, e.g. summary statistics over dataframes, pydantic objects, etc...

3. Lineage & provenance of data, e.g. quickly see what is upstream & downstream of code/data.

4. Asset/transform catalog, e.g. search & find if feature transforms/metrics/datasets/models exist and where they’re used.

While the UI currently only self-populates for Hamilton dataflows, we’re looking to expand to other frameworks (we’d love your feedback!).

Check out the following video for an overview: https://www.youtube.com/watch?v=0VIVSeN7Ij8, as well as the documentation: https://hamilton.dagworks.io/en/latest/concepts/ui/.

We’re looking for feedback/adopters – feel free to reach out if you have any questions!

user:	alm1
created:	July 9, 2018
karma:	28