Show HN: Visualize Airflow's schedule by exporting future DAG runs to GCalendar

Show HN: Visualize Airflow's schedule by exporting future DAG runs to GCalendar(github.com)

37 points by domenp 5 years ago | 10 comments

jka 5 years ago |

This looks very neat!

Just a thought - one that could involve implementation challenges and/or require too much effort to be worthwhile:

It might be possible to generate iCalendar-format schedules, which could widen compatibility and ease up the access permissions concerns you mention in the readme.

Python has a decent icalendar[1] library - and updates to existing entries be handled by ensuring that the ID for each schedule object is stable when updates are distributed[2].

[1] - https://icalendar.readthedocs.io/en/latest/usage.html#exampl...

[2] - https://en.wikipedia.org/wiki/ICalendar#Distributing_updates

X-Cubed 5 years ago | |

iCalendar is a pretty nice format to use and well supported, but the trouble that you then need a website to host the iCalendar file. It adds more yak-shaving to the process.

jka 5 years ago | | |

Yep, true; how you get the iCalendar file to the user could affect the requirements and experience a lot.

Web could work well for pull-based retrieval, and email's probably a good push option when you have a defined set of recipients.

shcheklein 5 years ago |

Brilliant! Each item could include a meeting link to debug the failed part together :) Perfect for the remote-first world.

domenp 5 years ago |

Believe it could be also relatively easy to adapt it to other scheduling solutions. As long as schedule expression and job/DAG run history and can be provided that is.

Gys 5 years ago |

Apache Airflow DAG = Directed Acyclic Graph

https://airflow.apache.org/docs/stable/concepts.html

In case you were also wondering what this item is about and came to the comments first.

encoderer 5 years ago |

This is great work, I love how it's basically using gcal as a rendering engine.

We are building something similar for Cronitor that will work for any kind of scheduled job. If anybody is interested in beta testing or hearing more I would love to chat.

treypitt 5 years ago |

Hello, thanks for open sourcing this terrific project. I like the idea but am a little confused about usage. How do you recommend monitoring existing pipelines, with an additional pipeline or by modifying the existing one?

domenp 5 years ago | |

I run it as a (additional) DAG on Airflow. Take a look at the example script (example.py).

alpineidyll3 5 years ago |

Airflow is _the worst_