Scaling Data Pipelines on Kubernetes

Scaling Data Pipelines on Kubernetes(airbyte.com)

3 points by davinchia 4 years ago | 2 comments

streetcat1 4 years ago |

why didn't you use grpc?

Also, a standard kubernetes client (e.g. the go client), uses cache on the client side for queries, and does not hit the api server.

davinchia 4 years ago | |

Since this is executed from Java code, we did not want to have to bundle kubectl (I'm assuming that's what you are referring to) with the Jar. We also did not want to run an exec command.

Are you asking why not GRPC between the various job pods?