How (not) to sign a JSON object (2019)(latacora.com) |
How (not) to sign a JSON object (2019)(latacora.com) |
> Add the tag and the exact string you signed to the object, validate the signature and then validate that the JSON object is the same as the one you got.
In cryptographic practice, redundant information usually spells disaster, because inevitably, someone will use the copy that wasn't verified.
But let's dig into it. If I understand this correctly, the suggestion is to have something like this:
{
"object": "with",
"some": "properties",
"signatureInfo": {
"signedString": "{\"object\":\"with\",\"some\":\"properties\"}",
"signature": "... base64-encoded-signature ..."
}
}
1. Parse the outer object as JSON, extract and remove signatureInfo.
2. Verify the signature.
3. Parse the signedString as JSON.
4. Verify that the object you got in step 1 equal to object you got in step 3 using a some kind of deep equality.
First of all, this is error prone, and as underspecified as JSON is, there are potential exploits if the comparison isn't done carefully. But even worse, if you think about it, the outer JSON is entirely useless, since you need to parse the inner JSON anyway -- so why not just use it directly?
It seems to me that this suggestion is strictly worse than just sending the inner part:
{
"signedString": "{\"object\":\"with\",\"some\":\"properties\"}",
"signature": "... base64-encoded-signature ..."
}
The redundancy is absolutely a recipe for disaster, but so is the part where you have brownfield consumers that you can't break and know that they also don't care about message security.
Unfortunately, it's an all too common brownfield to find yourself stepping into, which is why it is such a too common ask for "inline JSON signatures" (or other document languages like XML) that don't change the outer shape of the JSON document to break backwards compatibility with dumber consumers.
Also, unfortunately the most correct answer in cryptographic practice is also often the hardest to sell to those consumers (or to business people prioritizing changes to them): break those consumers and force them to care about security so that a rising tide lifts all boats.
If you only expect one signature I would recommend you wrap the signed content instead of treating it as a sibling. And even if you have multiple, maybe have signatures be siblings but put them all in the same wrapper. This means the recipient has to know signatures exist but honestly tough shit. If you’re adding sigs you’re going to end up expecting them and that’s a fact not an opinion. You don’t want any tools that ignore the signature and make decisions without validating them first. That’s a Confused Deputy attack waiting to happen.
Also in XML you have to canonicalize the document first, so that any formatting changes don’t invalidate the signature. So a couple other parts of what you said are true but there are solutions, even if annoying ones.
I don't think the protocol still injects the signature into event structures, but this weird "unsigned" field is still there looking at the source json for a message I sent today, but it's possible it's removed after processing and Fluffychat is just removing it.
I know it's not the most elegant thing ever, but if it needs to be JSON at the post-signing level, why not just something like `["75cj8hgmRg+v8AQq3OvTDaf8pEWEOelNHP2x99yiu3Y","{\"foo\":\"bar\"}"]`, in other words, encode the JSON being signed as a string. This would then ensure that, even if the "outer" JSON is parsed and re-encoded, the string is unmodified. It'll even survive weird parsing and re-encoding, which the regex replacement option might not (unless it's tolerant of whitespace changes).
(or, for the extra paranoid: encode the latter to base64 first and then as a string, yielding something like `["75cj8hgmRg+v8AQq3OvTDaf8pEWEOelNHP2x99yiu3Y","eyJmb28iOiJiYXIifQ"]` --- this way, it doesn't look like JSON anymore, for any parsers that try to be too smart)
If the outer needs to be an object (as opposed to array), this is also trivially adapted, of course: `{"hmac":"75cj8hgmRg+v8AQq3OvTDaf8pEWEOelNHP2x99yiu3Y","json":"{\"foo\":\"bar\"}"}`.
console.log(JSON.stringify({ x: 5, y: 6 }));
console.log(JSON.stringify({ y: 6, x: 5 }));For example most JSON parsers default to interpreting numbers from JSON as floats or ints. but in the canonical format you would have to force all parsers to interpret them as exact decimal values. then determine how to encode them (is one hundred "100" or "1e2") etc.
Is "I want the server/validating side to be safe even against server-side attackers with read-only permissions" not a good reason? Because that's one thing that asymmetric signatures provide out of the box compared to MACs.
Saying there's "sure there's lots of ways to serialize, but these specific rules get you the same octet and you sign that" is key to sanity in such situations.
For all of ASN.1's many sins, they got that part absolutely right.
https://docs.aws.amazon.com/amazonswf/latest/developerguide/...
Is the improvement COZE has over COSE that the body is default human readable, whereas COSE it's in some machine format that needs a reader util?
Cose also picked its name while thinking of JOSE. Cose is binary oriented, and attempts to be as similar to JOSE as possible.
Coze is a first principles reimagining of signing JSON.
As an exercise, we've played around with creating a binary format, (which we're calling Booze, Binary Oriented cOZE) but since Coze is already much more space efficient, it's not as beneficial. It may become more relevant with post quantum, as currently post quantum systems are much larger than ECDSA. The only other advantage would be removing the JSON semantics, but that's at the cost of implementing a binary format. The human readability aspect is paramount for our application, and we feel it's generally better practice.
How not to sign a JSON object - https://news.ycombinator.com/item?id=20516489 - July 2019 (151 comments)
Does this imply that the application doesn't trust the transport/presentation layers?
Not all signed content is meant to be confidential. Or two-party confidential. Think about tokens. You have a refresh token that’s private between you and the destination, but you hand out session tokens to your users so they can talk to the destination directly. Or via another server that doesn’t have a cache coherency with the source.
The documents are transmitted via a relaying party, as we don't have support for the protocol the recipient requires.
Similar cases could pop up in JSON-land, I imagine.
Not just "OK". It's the only sane way to do it.
> Anyone who cares about validating the signature can, and anyone who cares that the JSON object has a particular structure doesn’t break (because the blob is still JSON and it still has the data it’s supposed to have in all the familiar places).
As author mentions, you can compromise by having "hmac", "json" and "user" (for routing purposes only), but this will increase overall size. This is approach 2 in the blog.
Would it be guaranteed to survive even standard parsing?
It wouldn’t surprise me at all, for example, if there are json parsers out there that, on reading, map “\u0009" and “\t" to the same string, so that they can only round-trip one of those strings. Similarly, there’s the pair of “\uabcd” and “\uABCD”. There probably are others.
Then, if you couple this with sending data through a proxy (maybe invisible to the developers), which may or may not alter that text representation, you end up with a mess. If you base64 encode the JSON, you now lose any benefit you might gain from those intermediate proxies, as they can’t read the payload…
I'll take stringified json-in-json 90% of the time, thanks. if you're using JSON you're already choosing an inefficient, human-oriented language anyway, a small bit more overhead doesn't hurt.
(obviously neither of these are good options, just defer your parsing so you retain the exact byte sequences while checking, and then parse the substring. you shouldn't be parsing before checking anyway. but when you can't trust people to do that...)
https://datatracker.ietf.org/doc/html/rfc8259
Says:
> JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences.
So, different signature makes sense. But it should not be an issue as long as both software are calculating/validating the signature on the string and not json.
I agree that human readability can make things somewhat easier. The part about a binary format for the signatures confuses me a little - the signatures are going to be big no matter what. If you care about minimizing size would it make sense to just find a more efficient way to encode the signatures rather than change the entire message format?
When compared to encoding a value directly in binary, base64 has about a 25% overhead (6 bits /8 bits, 3/4). As far as the concern about using better encoding, base64 is just about as good as it gets while being maximally compatible. If using base 128 (7 bit ASCII), there's too many incompatible special characters for a human readable format. The full 8/8 bits, extended ASCII, isn't generally possible as systems use UTF-8 which begins using multiple bytes. (I've done a lot of work in this area, including a patent on base conversion. See also convert.zamicol.com) An advantage of a binary format is that there is minimal encoding overhead for binary values (escaping/padding is typically the only overhead, so usually around 99% efficient compared to base64's 75%.)
This isn't too much of a concern when signatures are small as encoding inefficiency is small compared to the payload's overall size, but if signatures are in the kilobytes or even megabytes, that extra 25% becomes meaningful for some hyper-efficient applications, like high cost blockchains. Our thought is using post quantum is already much more massive than existing elliptic curve, so any future applications of post quantum are going to have to deal with much larger signatures anyways. The signatures can also be stored on disk using binary or compressed which also makes it not a concern.
The original data take ≥25% less space than the base64-encoded data, but the base64-encoded data take ≥33⅓% more space than the original data. The overhead is about 33⅓%.