Orchestrating Data Pipelines at Lyft: Comparing Flyte and Airflow

Orchestrating Data Pipelines at Lyft: Comparing Flyte and Airflow(eng.lyft.com)

15 points by kumare3 4 years ago | 2 comments

karog 4 years ago |

Great comparison between Flyte and Airflow! Is there a way to incrementally adopt Flyte and use Flyte downstream from an Airflow dag?

kumare3 4 years ago | |

Thank you. So at Lyft, there exists an AirflowFlyteOperator. It was designed to interop with Airflow. I know the community is working on open sourcing it. Keep a lookout for the same

karog 4 years ago |

Great comparison between Flyte and Airflow! Is there a way to incrementally adopt Flyte and use Flyte downstream from an Airflow dag?

kumare3 4 years ago | |

Thank you. So at Lyft, there exists an AirflowFlyteOperator. It was designed to interop with Airflow. I know the community is working on open sourcing it. Keep a lookout for the same