gRPC: Internet-scale RPC framework is now 1.0(cloudplatform.googleblog.com) |
gRPC: Internet-scale RPC framework is now 1.0(cloudplatform.googleblog.com) |
Performance. gRPC is basically the most recent version of stubby, and at the kind of scale we use stubby, it achieves shockingly good rpc performance - call latency is orders of magnitude better than any form of http-rpc. This transforms the way you build applications, because you stop caring about the costs of rpcs, and start wanting to split your application into pieces separated by rpc boundaries so that you can run lots of copies of each piece.
I cannot sufficiently explain how critical this is to the way we build applications that scale.
Performance benefit comes from the fact that schema is defined on each side (generally server / server) so you only send the information bytes. With a good RPC system you can also access specific fields of your structure without unpacking (or very fast unpacking, depends on what RPC system you're using).
gRPC uses Google's protocol buffers. https://developers.google.com/protocol-buffers
Comparable systems use other IDLs, e.g Facebook uses Apache Thrift.
Versioning is easier because your fields are defined explicitly, so you can ignore clients sending an old field to no ill effect (again, you can access individual fields without unpacking). Whereas with json you need to deserialize first. Also if your schema changes when using json without protobufs, you may experience either the client or the server making the wrong assumptions about input data. Whereas there is no ambiguity with protobufs; future changes to proto messages add fields and old fields can just be marked deprecated.
RPC is preferable to JSON for server-to-server communication, but client-server still often uses json just because it's often easier for your client app to interpret json. Systems like gRPC allow servers to emit json as well: https://developers.google.com/protocol-buffers/docs/proto3#j...
I've heard of performance-oriented web apps using protobufs on both client and server.
CBOR -- Concise Binary Object Notation -- http://cbor.io/ -- is all the performance of a binary protocol, with semantics basically identical to JSON.
I appreciate some of the things protobuf does to help you version, but I also do not appreciate the protobuf compiler as a dependency and a hurdle for contributors, or for wire debugging. CBOR has libraries in every major (and most minor) languages and works without a fuss with tiny overhead, both at runtime and in dependency size. It's pretty pleasant to work with.
I'm sure it's possible to create the same thing with HTTP/JSON (and swagger?), but I find it to be more work.
* streaming in any direction (client->server, server->client, both). Not something easily done with a simple HTTP/JSON endpoint, all handled for you
* Arbitrary cancellation from the client or the server
If performance isn't a big deal, JSON-RPC over HTTP or something like it is fine. If it's critical, something like gRPC makes more sense.
At the moment as long as you want to use the endpoint directly from a browser. gRPC uses some HTTP features that are not [yet] available from within browser JS APIs.
We have had to force keep alive and even forcefully turn it off in some cases :(.
Mostly. I can't speak to protobuf's serialization format personally, but similar binary serialization like Thrift's TCompactProtocol or TDenseProtocol outperform JSON and you get schemas for "free*" (as in, in your producer and consumer's glue code by virtue of codegen, and not as an afterthought).
The IDL and codegen is a big part of gRPC and Thrift. The rest is opinionatedness and less cognitive effort -- the format is non-human-readable anyway so less propensity for bikeshedding about cosmetic stuff in JSON.
So if you are looking to load testing (or integration testing) for gRPC based apps check https://goreplay.org
https://github.com/grpc/grpc/issues/282 https://github.com/eleme/thriftpy
def GetFeature(self, request, context):
However, according to the style guide, camelcase is preferred, and the compiler is supposed to generate language-native names with the correct case [1].
One thing the terrible years of SOAP and WSDL should have taught people is that generated stubs are awful to use if they go against the grain of the host language.
[1] https://developers.google.com/protocol-buffers/docs/style
The library for Python for one seems very unpythonic to me, starting with all those CamelCase method names. It does not seem possible to use asyncio or any other non-blocking solution on the server.
Finally, gRPC obviously only handles the transport: Are there any other useful related Python packages out there for validation etc.?
No plain-old-C?
That's a shame.
e.g.: Will it have things like monitoring (we've handled x calls to this API in the last hour, the average API call took y milliseconds). Clustering? (a client connects to a list of grpc servers, if one server goes down, the client will automatically connect to the next on the list)? And load balancing?
If not, are there existing third party tools to implement these, or is the expectation that the community will create these?
At Square, we created a custom balancer (https://godoc.org/google.golang.org/grpc#Balancer) which not only handles updates from the service discovery system in order to manage the pool of connections, but also handles which connection to use per-call (so we can do targeting of specific capabilities, datacenters, etc.)
for example, if you visit: https://api.github.com/repos/grpc/grpc/issues
you'll get some json back. what happens is the browser resolves "api.github.com" (via DNS) to an IP address, and then opens a socket connection to that address. then, in accordance with the HTTP protocol it executes a "GET" operation (a notion particular to HTTP), and in response the github server will talk to a database or cache and respond with data appropriate to request.
because this sequence of events happens across network boundaries (ie, your browser is talking to something outside your computer), it's often referred to as a "remote" procedure call.
Also the github api is more RESTful than RPC.
Oh lol, again
The performance is good, and it's nice to have proto files with messages and services, which acts both as documentation and a way to generate client and server code. Protobuf is much faster, produces less garbage and is easier to work with than JSON/jackson. The generated stubs are very good and it's easy to switch between blocking and asynchronous requests, which still only require a single tcp/ip connection.
We've had two performance problems with it:
1. Connections can die in a somewhat unexpected way. This turned out to be caused by HTTP/2.0 which only allows 1 billion streams over a single connection. Maybe not a common issue, but it hurt us because we had a few processes reaching this limit at the same time, breaking our redundancy. It's easy work around it, and I believe the grpc-java team has plan for a fix that would make this invisible to a single channel.
2. Mixing small/low-latency requests with large/slow requests caused very unstable latency for the low-latency requests. Our current work-around is to start two grpc servers (still within the same java process and sharing the same resources). The difference is huge with 99p going from 22ms to 2.4ms just by using two different ports. Our old code with JSON over HTTP/1.1 implemented using jackson and netty didn't suffer this unstability in latency, so I suspect grpc is doing too much work inside a netty worker or something. I haven't yet tested with grpc-java 1.0, which I see has gotten a few optimization.
Still, these have been minor issues, and we're happy so far. The grpc-java team is doing a good job taking care of things, both with code and communication.
Hilarious. People called this issue out as an obvious flaw when HTTP/2.0 was first proposed, got ignored, and here the issue is.
For those unfamiliar:
HTTP/2.0 uses an unsigned 31-bit integer to identity individual streams over a connection.
Server-initiated streams must use even identifiers.
Client-initiated streams must use odd identifiers.
Identifiers are not reclaimed once a stream is closed. Once you've initiated (2^31)/2 streams, you've exhausted the identifier pool and there's nothing you can do other than close the connection.
For comparison, SSH channels use a 32-bit arbitrary channel identifier, specified by the initiating party, creating an identifier tuple of (peer, channel). Channel identifiers can be re-used after an existing channel with that identifier is closed.
As a result, SSH doesn't have this problem, or the need to divide the identifier space into even/odd (server/client) channel space.
Some of the language bindings (Ruby) started off feeling experimental quality when we began the project, but overall it's been a huge win for us versus HTTP+JSON. I'm sure a non-zero portion of the benefit has been using protobufs at all, but gRPC gives us a great way to generate clients for every language we use without worrying.
Every RPC system needs this, or you end up with hacks like HTTP long polling.
My least favourite feature is that it is tied to HTTP2. I'm not sure what you're supposed to do if you are running on a microcontroller.
The tying to HTTP/2 and especially the way it is done is also not my cup of tea. E.g. if it wouldn't have chosen to use HTTP trailers (which are mostly unsupported) it could be implemented with a lot more HTTP libraries. It's also sad that it doesn't run in current browsers because of the lack of trailer support as well as streaming responses there. With putting a little bit more thoughts in it (maybe choosing multiple content types/body encodings) this could have been supported - at least for normal request/response communication without streaming.
Regarding microcontrollers: It should be possible to implement HTTP/2 and gRPC also on microcontrollers, but imho it will neither be easy nor necessarily a good choice. Implementing HTTP/2 with multiplexing will need quite a lot of RAM on a constrained device, especially with the default values for flow control windows and header compression. You can lower these through SETTINGS frames, but that might kill interoperability with HTTP/2 libraries that don't expect remotes to lower the settings or to reset connections while settings have not been fully negotiated.
I've seen backwards incompatible changes made by core go team members and core gRPC maintainers. Where the API is statically consistent but actually behaves in completely different broken ways. One of these was big enough that I said screw it and am moving that application away from gRPC.
I've seen multiple issues where the library you generate against ends up being incompatible with the library you link against at build time. They finally added a version check as part of the build/run step to prevent this from causing silent runtime errors.
Maybe in a year gRPC will actually be stable, maybe it has been over the last three months. I don't really know but I gave up, am moving my applications off of it and actively pushing for coworkers to do the same.
If you look at the example code, you'll see that RPCs aren't modeled exactly as local function calls. You're right that that wouldn't work very well. The libraries for all or most languages let you make RPCs asynchronously, without blocking the thread. And all of them provide with ways to write and read RPC metadata (headers and trailers).
As an ex-Googler, using gRPC feels just like using Stubby: the interfaces are the same, the serialization code is the same, the only thing different is the networking code and transparent hooks into other systems.
- Assistance with schema evolution and versioning is one of the best parts about using Protobuf in an API. It is really like the best parts of XML and XML Schema (validation, documentation, interoperability) without any of the bloat.
- Working with code generation can be a pain to get working initially, but is very friendly when actually using real objects in code. There is no need to think about any representation on-the-wire... everything 'just works'. There is no need to ensure you don't accidentally serialize fields in the wrong order, or worry about encodings, etc.
Also, there is a binary decoder included with protoc that can print a debug decoding of any protobuf binary message, including integer tags for different fields. Wouldn't you have pretty much the same problems with dissection and debugging on-the-wire in CBOR?
It is really quite pleasant to use Protobuf for an API, I can see why Google is opinionated in including it as the only option with gRPC.
I don't find the schema validation powerful enough. You still have to write correct code to migrate semantics. Avoiding incorrect reuse of field names is... nice, but also the most trivial of the problems.
(I do like schemas in theory. It's possible I haven't worked with good enough tooling around protobufs to really experience joy resulting from those schemas. The protos themselves certainly aren't guaranteed to be self-documenting, in some of my experiences.)
I don't find code generation results to be smooth in many languages. At $company, we switched through no less than three different final-stage code generators in search of a smooth experience. Not all of this was misplaced perfectionism: in some cases, it was driven by the sheer absurd method count verbosity in one code generator basically making it impossible to ship an application. (You've perhaps heard of the method count limits in early android? Bam.)
I don't think the debug decoding tools for protobuf are directly comparable to CBOR's interoperability with JSON. CBOR and JSON are literally interchangable. That means I can flip a bit in my program config and start using one instead of the other. Config files? Flip the bit. Network traffic not meant for humans? Flip the bit. Need to debug on the network? Flip the bit. Want to pipe it into other tools like `jq`? No problem. There's a whole ecosystem around JSON, and I like that ecosystem. Working with CBOR, I can have the best of both worlds.
Sometimes opinionated is good and powerful and really makes things streamlined in a way that contributes to the overall utility of the system. I don't think this is one of them. Almost every major feature of gRPC I'm interested in -- multiplexing, deadlines and cancellations, standard definitions for bidi streaming, etc -- has nothing to do with protobufs.
natch, I used "I" heavily in this comment, because I realize these are heavily subjective statements, and not everyone shares these opinions :)
Most of the projects that we're integrating gRPC into are existing codebases that have their own build tools that are (mostly) in isolation. JSON schemas have been agreed on beforehand, and there are separate client implementations in different languages that basically exists as independent units.
By adding protobufs to this process, the "JSON schemas that have been agreed on" become protobuf definitions - which is _fantastic_ for development teams, because they have a single spec to work from, and there is no ambiguity (or, significantly less).
The challenge comes when we are trying to generate gRPC clients in Go, Ruby and Python for the same profobuf file - in order to do it in an automated fashion without a 'monorepo', we need to create a build system that pulls this protobuf from a central place and generates the client, which doesn't exist right now.
It's not a huge challenge to ensure services can communicate at all - as you said, protobufs have thought of this and have an extensive amount of decoupling built in. When we're working on adding new features however, we need to have a place to keep the "gold master" of protobufs, and grab it for all of our projects to build+deploy at once, which is where the above becomes challenging.
Not an un-solvable problem, and different languages have different tooling for this. We've settled on placing the proto definitions in the "server" side (most of our interactions are fairly well modeled by client/server), and then updating the clients as-necessary, as we can deploy server changes without needing to update the clients immediately.
That said; If you've got a set of shared proto definitions, you should probably either go to a monorepo, or share the shared bits with a git submodule. Doesn't prevent you from needing to follow those conventions, but does make it far easier to debug when things changed.
[1] https://developers.google.com/protocol-buffers/docs/proto3#u...
Ideally this system would also allow the package owner to release updates within an existing version if they wish. For example, backward-compatible changes to the service interface can be released while keeping the major version 2. In this way, clients automatically consume safe updates, while incompatible or risky changes can be given a major version bump (e.g. to 3). Consumers who want to pin the interface to a specific version like 2.5.1 could do so, in some build systems, though dependencies this specific are rarely useful or a good idea. In my experience it's best for the contract between producer and consumer to be explicitly versioned at the "major version" level, and only implicitly versioned (meaning updates are automatic) at the minor version level.
So, if we ever close a connection from having reached a billion streams we are in a very very good position.
Since HTTP2 is a client->server protocol, the server can close whenever and the client can just open another one.