How to track costs in multi-tenant Amazon EKS clusters

How to track costs in multi-tenant Amazon EKS clusters(aws.amazon.com)

64 points by AjayTripathy 5 years ago | 6 comments

jakozaur 5 years ago |

Another way to do that is to buy commercial SaaS vendor monitor AWS costs: CloudHealth or Cloudability.

webb 5 years ago | |

There would be a number of differences with these products. First, Kubecost does not egress any data from your infrastructure, it's based on open source technologies (e.g. Prometheus), and because it is build specifically for Kubernetes it takes a pretty different approach for determining the cost of a tenant, workload, etc.

Disclaimer: I'm a founder of Kubecost.

simonebrunozzi 5 years ago | | |

Congrats on being the founder.

>The default installation of Kubecost includes an optimized Prometheus server that only contains metrics that are useful to Kubecost. This optimized version retains 70-90% fewer metrics than a standard Prometheus deployment. You can also use an existing Prometheus installation.

I really like this approach. What do customers say about it?

gbrayut 5 years ago |

Is there any way to setup a sharding solution for Prometheus? Like instances per AZ or that only scrape a certain set of nodes? On our larger clusters we had issues even with the optimized Prometheus instance in kubecost, and it's starting to look like metrics collection/processing may be a big driving force on our cluster scaling designs.

AjayTripathy 5 years ago | |

There's definitely ways to do this by configuring your scrape configs to ignore sets of nodes. Curious though when you last tried kubecost out? We've built out some caching mechanisms in the product over the last month or so that should dramatically reduce load / memory consumption on prometheus. If you reach out on our slack kubecost.slack.com we can discuss more about expected Prometheus resource consumption.