The most important thing that I found out while doing my research is it's not the fastest bytes that win. Well, that does matter, but it's not that important. It's reducing performance variance [1]. While my project wasn't optimizing for speed, it was optimizing for zero-copy de/serialization, but this often ends up as a solution for high-speed transfers. SBE, Flattbuffers, Cap'n Proto all had their places, but I ended up not using any of those and just hand-rolling something similar to what SBE would do. If this was a $DAYJOB project I'd probably end up doing something with SBE.

[1]: https://speice.io/2019/07/high-performance-systems.html

TCP has no concept of a message, you need to build it on top of TCP.

Which HTTP does, but it doesn't have a standard concept of a message format, you need to build it top of HTTP.

Which gRPC does, but it closes the connection after a successful exchange.

Which is avoided by streaming gRPC - makes sense if you know you'll be talking more over this channel.

None of those protocols are really comparable, and most of the differences boil down to the serialisation protocol (binary/proto or textual, like JSON).

- Suddenly you want to enable TLS between one or more components, meaning you need to wrap the socket inside a TLS channel.

- You discover that your client behaves poorly when servers go offline, so you add your own logic for keepalives/pings.

- At some point you want to add metrics to all of this, so you decide to manually add Prometheus metrics to the client/server.

- Later on you want to attach OAuth2 tokens to requests as well, so that you can do credential passing.

- In order to get more insight in your setup, you decide that you want to use this in combination with OpenTracing/Jaeger.

Once all of those features are added to your Redis-like protocol, you discover that you've basically reinvented gRPC... poorly.

For fullstack dev, I've been immensely happy with grpc/protobuf over http because of the type safety I get communicating between Golang and Typescript. This eliminates a whole class of bugs but is only a serialization benefit.

Generally: use the thing with the best tooling!

While GRPC has its place it also comes with headaches like pretty lousy generated interfaces, horrible debuggability, and unpredictable scaling.

https://zeromq.org/

Atom is an easy, Redis Streams-based RPC that also emphasizes docker containerization of microservices. We support plug-ang-play serialization with msgpack and Apache Arrow currently supported and more on the roadmap. You can also send raw binary if you please.

Another nice thing about Redis is that if you're running microservices on the same instance you can connect to redis through a linux socket on tmpfs and bypass the TCP stack to get even better performance.

Anyway the problem with comparing rpc protocols is that you need to do a per case benchmark if you really care about performance. Pretty much every decent solution will be better than another equally good solution depending on the use case.

Years ago, one of my customers told me he used to work doing high frequency trading in a bank, and the bank had several tailor made solutions for data serialization and RPC made for very specific cases, and they were just better than any generic solution.

The main benefit was we could suddenly reuse all the codegenerated routers/docs/authentication from the HTTP ecosystem. It significantly simplified/standardised our IPC layer and reduced the "weirdness" in the codebase.

* AMQP 1.0 - can also be used for RPC without a broker in between client and server. See https://qpid.apache.org/proton/

* Aeron - low latency, UDP based, see https://github.com/real-logic/aeron

This is just as bad as front-end web dev!

If the channel is marked as persisting, I put the message to a hashset with current nanosecond as the timestamp and just send a ping instead.

It is pretty fast with asyncio redis client clocking at an avg 15k rpc/s on a single core with uvloop on a lowly i7

[0] https://github.com/smfrpc/smf

1. Having a complex build system that is aware of gRPC

2. Massive migraines

Unfortunately build systems integrations, IDE integrations, and generated code is unanimously awful for gRPC.

Every company I've seen use grpc has unfortunately adopted the practice of basically manually running protoc and committing the generated code into repo - sometimes modifying the code manually to make it import successfully (Python).

I hope Bazel evolves into a state where it's usable by the average engineer and has first class support for gRPC.