The CLI shows key-value store features.
I don't know if this software has real world savings in performance. I don't think I would ever use this software.
such an awful format, I wish people would stop using it
The format is not that bad.
The binding/libraries OTOH are often awful and they often require unnecessary full-message deserialization
Why don't you make up your own bullshit words instead of randomly picking stuff from other places? That's not even multiplexing.
It would be nice if the next serialization format either is truly original or just solves problems that somehow none of the many existing schemes do.
How many serialization formats are there? See: https://en.wikipedia.org/wiki/Comparison_of_data-serializati... (which is NOT a complete list).
you couldn't be more wrong. what happens when you lose the schema, or never had access to it in the first place? think from the point of view of reverse engineering
I'm not sure that's a helpful way to look at things.
There are tradeoffs. Can you elaborate more about what aspects of a TLV encoding you find problematic? Is it decoding speed? The need to copy the encoded value into a native value in order to make use of it? Something else?
Now, you may disagree that tolerating unknown fields is a features (as many people do), but one must understand the context where protobuf has been designed, namely the situation where it takes time to roll out new versions of binaries that process the data (either in API calls or on stored files) and thus the ability to design a schema evolution with backward and forward compatibility is worth a few more cycles during encoding.
Not all users have that need and hence there exist other formats, but I wouldn't dismiss the protobuf encoding as flatly "wrong" just because you don't have the requirements it has been designed for.
See ASN.1's extensibility rules. If you mark a record type (SEQUENCE, in ASN.1 parlance) as extensible, then when you later add fields (members, in ASN.1 parlance) the encoding rules have to make it possible to skip/ignore those fields when decoded by software implementing the pre-extension type. PER/OER will include a length for all the extension fields for this purpose, but it can be one length for all the extension fields in each round of extensions rather than one per (which would only save on the type "tag" in TLV).
> the decoder can skip decoding fields
This is mainly true for on-line decoders that produce `{path, leaf value}` tuples _and_ which take paths or path filters as arguments.
> Now, you may disagree that tolerating unknown fields is a features (as many people do), but one must understand the context where protobuf has been designed, namely the situation where it takes time to roll out new versions of binaries that process the data (either in API calls or on stored files) and thus the ability to design a schema evolution with backward and forward compatibility is worth a few more cycles during encoding.
This is the sort of thing I mean when I complain about the ignorant reinvention of the wheel that we all seem to engage in. It's natural and easy to do that, but it's not really a good thing.
Extensibility, versioning, and many many other issues in serialization are largely not new, and have been well-known and addressed for decades. ASN.1, for example, had no extensibility functionality in the early 1980s, but the only encoding rules at the time (BER/DER/CER), being TLV encodings, naturally supported extensibility ("skipping over unknown fields"). Later formal support for extensibility was added to ASN.1 so as to support non-TLV encodings.
ASN.1 also has elaborate support for "typed holes", which is what is referred to as "references" in [0].
ASN.1 gets a lot of hate, mainly for
a) its syntax being ugly (true) and hard to parse (true-ish)
b) the standard being non-free (it used to be non-free, but it's free now, in PDF form anyways)
c) lack of tooling (which is not true anymore).
And every time someone re-invents ASN.1 they miss important features that they were unaware of.
Meanwhile ASN.1 is pluggable as to encoding rules, and it's easy enough to extend the syntax too. So ASN.1 covers XML and JSON even. There's no other syntax/standard that one can say that for!
Next time anyone invents a new syntax and/or encoding rules, do please carefully look at what's come before.
[0] https://en.wikipedia.org/wiki/Comparison_of_data-serialization_formatsOne of the design requirements was simplicity and ease of implementation and for all the love in the world I can muster for ASN.1 I must admit it's far from simple.
IIRC complete and open implementations of ASN.1 were/are rare and the matrix of covered features didn't quite overlap between languages.
And if you lose your own schema, then you probably have more serious underlying problems.
Do I want protobufs replacing JSON? No. I want flatbufs augmenting JSON (and XML, and ...).
I maintain an ASN.1 compiler and run-time that supports transliteration of DER to JSON. This is possible because the syntax/schema language/IDL and the encoding rules are separable. This is how you get the best of both worlds. You can use optimized binary encoding rules for interchange but convert to/from JSON/XML/whatever as needed for inspection.
Ideally it is capnproto instead of protobuf, but in general the same applies.
It is a _transport_ protocol. You can still write your config or data in any format you like.
> IIRC complete and open implementations of ASN.1 were/are rare and the matrix of covered features didn't quite overlap between languages.
When protobufs was designed there were zero protobufs implementations.