After a while, the more mature Linux engineers start going the other way. Ripping out as much as possible. Stripping down to the leanest build they can, for performance but also to reduce attack surface and overall complexity.

Very similar dynamic with k8s. Early days are often about scooping up every CNCF project like you're on a shopping spree. Eventually people get to shipping slim clusters running and 30mb containers with alpine or nix. Using it essentially as open source clustering for Linux.

If you have direct access to Etcd (which may not be possible in a managed cloud version of Kubernetes?), putting a watch on / might scale better.

(As an aside, with the Go client API you have to jump through some hoops to even deserialize objects whose kinds' schemas are not already registered. You have to use the special "unstructured" deserializer. The Go SDK often has to deal with unknown types, e.g. for diffing, and all of the serializer/codec/conversion layers in the SDK seem incredibly overengineered for something that could have just assumed a simple nested map structure and then layered validation and parsing on top; the smell of Java programmers is pretty strong.)

Is this a question of Kubernetes just sticking everything into "standard" datastructures instead of using a database?

Getting this to perform well required several optimizations at both the Go and Postgres levels. On the Go side, we use prioritized work queues, event de-duplication, and even switched to Rust for efficient JSON diffs. For Postgres, we leverage materialized views and trigger-based optimistic locking

In fact, recently it is preparing to integrate Cyphernetes as a new search method. I believe this will be a new start!

re. intended purpose: Initially I started writing this to help tackle bigger problems - stuff you'd normally use multiple nested kubectl commands or write a lot of code for interacting with api-server.

Over time, I developed the shell environment around it and it became a daily driver for me as well. Indeed, there's a threshold where writing Cyphernetes becomes more economical than using kubectl but for doing most of the simple day to day stuff writing Cypher is too verbose.

The Cyphernetes shell has an early-stage feature that allows a syntax like you suggested - there's a tiny "macros" feature that lets you define custom procedures of one or more queries (currently shell only, not supported in the web client yet).

Macros are prefixed by ":" and you could define something like:

:pod condition

MATCH (pods:Pod) WHERE $condition

RETURN p.metadata.name, p.status.phase; // and whatever other fields you'd like

Then use it like this: > :pod .metadata.name=~"foo%"

So it gives you a tiny way to customize how you do this day-to-day stuff. Ships out-of-the-box with common stuff you do with kubectl like :getpo, :getdeploy, :createdeploy, :expose and so on - definitely a feature that could be developed further to make this more of a daily driver.

      go run github.com/avitaltamir/cyphernetes/cmd/cyphernetes@v0.14.0 --help

Disclaimer: I am the creator of Karpor.

The example on the homepage is literally "give me deployments with more than 2 replicas with pods that are not Running, and give me the IP address of the service they're serving"...

Any idea how to do that with kubectl | jq? Their solution seems elegant to me.

  $ kubectl logs -n foo $(kubectl get pod -n foo | awk '/Running/{print $1}')

I don't see how their queries can be more efficient than hitting the kube-apiserver multiple times, unless they have something that lives clusterside observing lifecycle events for all CRDs and answering queries with only one round-trip instead of multiple.

Or maybe they're selling "no nested kubectl queries" as an experience feature, saying that a query language is more ergonomic than bash command redirection. My brain has been warped into the shape of the shell, for better or for worse, so it's not a selling point for me.