This article is full of mistakes and misleading claims:

1) It's claiming SHA1 insecurity is theoretical, while SHAttered from 2017 was specifically a pratical proof of concept. The only reason Git wasn't affected, is because they didn't bother bruteforcing a git-blob prefix.

2) It's claiming collision attacks don't matter, only second-preimage attacks do. This is incorrect, collision attacks are enough for code-smuggling problems, when two repositories are on the same git commit (verified by the full commit hash), yet contain different code in their git checkout.

3) The Linus quote "The real security is in distribution" is arguing that "git's content-addressed system should not be used to address content". It's arguing that, in case of curl|sh, you shouldn't use a sha256sum-gate to pin the content to something you've reviewed, you should instead ensure curl is fetching from an https server.

Linus Torvalds in 2007:

> but the point is the SHA-1, as far as Git is concerned, isn't even a security feature. It's purely a consistency check. The security parts are elsewhere, so a lot of people assume that since Git uses SHA-1 and SHA-1 is used for cryptographically secure stuff, they think that, Okay, it's a huge security feature. It has nothing at all to do with security, it's just the best hash you can get. ... [1]

[1] https://www.youtube.com/watch?v=4XpnKHJAok8&t=56m20s

So Torvalds used SHA-1 purely because he needed a hash function with no other property than identifying content.

One of my favorite fun facts about Fossil SCM (another source control by the devs of sqlite) is that they patched their use of SHA1 6 days after the shattered attack was published:

"Both Fossil and Git started out using only SHA1 hashes. But when the SHAttered attack against SHA1 was published on 2017-02-23, the need to migrate to a stronger hash algorithm was recognized. Fossil added the ability to use SHA3-256 as an alternative on 2017-03-01 (six days after the SHAttered attack was first published). SHA3-256 is now the default for all new repositories and check-ins in Fossil, though older check-ins that occurred prior to SHAttered can still use their original SHA1 hash. Hence, no repositories had to be rebuilt and no hyperlinks were broken."

https://fossil-scm.org/home/doc/trunk/www/hundredandone.md

To me it's so interesting watching in realtime Git is still battling with this decision and for Fossil it was just another week of development.

That whole page is fun to read. Another fun fact somewhere else in the docs is that Fossil uses a grow-only set to store commits. They came up with this scheme some years before it was formalized by CRDTs!

I don't understand why Git is not making the SHA-1 and SHA-256 modes far more compatible with each other.

SHA1-hashed objects should be able to refer to SHA-256-hashed objects, although this seems somewhat pointless.

But SHA-256-hashed objects should also be able to refer to SHA1-hashed objects, with a major caveat: if those objects themselves are part of a collision pair, then there is a genuine problem. But this is avoidable! Suppose that Linux decided to migrate to SHA-256. The upstream project could choose a pair of dates, say January 1 2027 and March 1 2027. Up to the first date, maintainers would be welcome to submit hashes of objects that are not yet in the repo but that they think they might submit later on, and, on that date, the upstream tree would finalize the list of these objects and reference it in the repo (with a new mechanism for this purpose). Effective the second date, the repo would start publishing SHA-256 commits and would never again accept a SHA1-hashed object that was not in the repo at the cutoff date or referenced as part of the Jan 1 block.

And now it would be impossible to get a new SHA1 collision in to the repo.

The only new git features needed would be:

a) actual compatibility so that a SHA-256-hashed object could reference a SHA1-hashed object

b) a new object type that's a list of allowed SHA1 hashes (or probably a tree of them) that is itself hashed with SHA-256 and a mechanism to link to one of these from a commit

c) a policy mechanism to set a repo to only allow SHA1-hashed-objects that a reachable from a preconfigured SHA-256-hashed commit

So they’ve been talking about this for many years, planning, and finally announce when they’re going to switch the default.

So this is the right time to post that everything they’re doing is wrong? Did you engage in all the discussions about it and how best to handle it? Whether SHA-256 was the best solution?

I don’t see anywhere that it talks about alternate proposals or why they might have been better. Why the particular suggestions here were rejected.

This seems like a bunch of Monday morning quarterbacking.

I thought this would be a snark but it's an extremely well put together argument against the "Hashmageddon".

If you're replacing the weakness of SHA-1 just by going to another algorithm, you better be prepared to go to the next one when sha256 collisions happen, and it doesn't sound like git's design would be easy to modify for this type of crypto agility.

I do like their proposal for using signatures to establish trust and allow swapping sha256 for whatever comes next.

This means, if you migrate your repo, every single commit message that contains text like: "please see commit <sha1>" will now be broken.

This will be a train wreck. I hope they don't release before adding compatibility modes to keep the existing sha1's around in the database.

>it will be an incomprehensibly expensive and ultimately valueless and avoidable global nightmare.

thought "costly" in the title and "incomprehensibly expensive" in the subheader meant this piece would discuss how much less performant sha-256 is on modern machines, but didn't see anything. isn't there hardware acceleration? how much worse is it?

From what I'd read, SHA256 in git is showing every sign of being another IPv6. In particular:

- It's implemented in a non-backwards-compatible way

- The benefits over the older model are a bit nebulous

- There's a large amount of tooling that needs to catch up, and little sign that there is movement there

Make git init use SHA-256 if git is invoked as git3, SHA1 if invoked as git2.

Plain git init could fail with a diagnostic: informing to use one of the two aliases or an option.

What people don't want is making git repos SHA-256 by accident and finding out later that they made repos not compatible with older git.

Yes every repo is either one or the other but you fix that by rehashing the entire repo. Everyone can do this independently. It's entirely possible to maintain to identical repos in SHA1 and SHA256 mode but for the most part I suspect once updated people will simply pull down the new repo and use git 3.0 as a required version.

As migrations go, it's reading as simple to me. You'll just have to backpoint the commit signatures. I must assume there's a backwards compatible reference for them in git 3, right?

Or drop them and reference the old structure in a dire pinch.

I positively don't care about the collision issue.

If you need to certify the authenticity of some code, and you've decided that a Git hash of any kind is going to be your certificate, you have a problem between keyboard and chair which is not fixable by stronger hashes in Git.

I don't want instability and churn in tooling.

Can someone more cyber-pilled than me explain what the actual risk with Git hashes being susceptible to collision attacks is? Obviously accidental collisions are problematic, but to my understanding the probability of that is still approximately zero.

Best I can tell, all a forced collision would do is let someone who already has control of a repo modify the history in a far from plausibly deniable way. Which in practical terms, they already could do simply by replacing the whole thing, because who's out here using git hashes as a security tool? Every pinning I've ever seen has been to tags (which can be modified at will), or hashes of the actual payload (which doesn't need to be the same as what git uses).

schacon: Really like the "Independent Tree Hash Headers" idea. How difficult would this be to get this functionality into git? Would it cause any breaking changes with older versions? Have you discussed this with any git devs to see if they are open to adding it?

> We can go through years of this SHA-1 to SHA-256 migration and then quantum computers break 256 and we're back in the same stupid boat again.

SHA-256 is considered quantum safe by the NIST and is left out of PQC migration guidance entirely.

Ugh, I didn't know that SHA-1 submodules wouldn't be supported in SHA-256 repos. That changes the transition from painless to a major dumpster fire. Having to maintain converted forks, and use different hashes from upstream is going to be a mess.

The post's argument that hash collisions are irrelevant in practice is not convincing at all. Basically they amount to:

1. Collisions aren't as bad as preimage attacks

2. Even if you made a file-with-malicious-hash, how would you get people to pull it?

3. Other attacks are a bigger problem (social engineering)

(2) is laughable in a world with github. It's common for unknown people to submit pull requests to code bases, and for those changes to be reviewed and merged. For example, as part of reviewing pull requests, I have `git fetch`'d proposed changes to my local machine to check behavior on some additional test cases. "If you fetch it you're fucked" is unacceptable as a security boundary.

(1) and (3) are just tu-quoque arguments about other attacks being worse. The relevant question isn't how bad other attacks are, it's how bad this attack is.

The fundamental problem with collisions is that software often assumes they can't happen (or is not tested against them). Thus collisions can trigger bugs, or otherwise cause surprising behavior. For example, webkit figured the colliding PDFs demonstrating a sha1 collision would be excellent for unit tests, so they merged the PDFs into their SVN repo... which completely fucked it [1]. I don't know the exact internals of git so I can't comment on how you would get surprising things to happen, but "oops the file you merged was different than the file you reviewed" and "oops the repository got corrupted" seem entirely plausible.

[1]: https://www.reddit.com/r/programming/comments/5vyhy2/webkit_...

A lot of replies here seem to be asserting that this "isn't that hard" without addressing the thing that makes it most hard: submodule compatibility and the breadth of tooling

I run a small git/jj forge and for us it's already painful dealing with this. Can't imagine how GitHub is going to handle it.

The xz backdoor is basically his point in practice — that was a maintainer-trust compromise, not a hash collision.

The first reason the author lists for why this will be bad is only an "issue" on Git hosts that don't allow repo creation on push (which is brain dead of GitHub). Any other host, you push your new repo, and it will see the hashing algorithm, and receive the contents accordingly.

Submodules is a legitimate argument against this, though I don't know how widely this feature is actually used, and similar to the arguments in favor of switching the default branch from master to main, this is simply a setting which can be changed.

I do like the idea of commits having both hashes, and am surprised that idea has not been explored further.

Generally though, I think the author's strongest argument is simply that the change isn't strictly "needed", and all the other issues presented aren't the strongest arguments against change.

Thanks for writing this. I'd only been loosely following it and I hadn't realised how bad this is going to be. I have repos with tens of submodules and it's going to be a nightmare if any of them switch to sha256 in place. Not to mention I won't be able to use any new projects unless I rebuild my repo and all the submodules therein.

I thought the master to main thing was bad enough but this is going to suck. And just like the master rename it achieves basically nothing.

What is it about these projects that attracts people who just want to change things for the sake of it? Real engineering means coming up with a solution for backwards compatibility. This is just irresponsible and, frankly, a fuck you to everyone who will be affected by this.

This seems like Y2K fud.

The alternative to making sha256 the default is to leave sha1 the default. Nobody changes to sha256. sha1 is broken in 10 years. Suddenly everyone has to switch all at once on the same day because it is a critical security issue, but github never implemented sha256 because they didn't have to. This would be a major problem.

This is very very easy to fix if you run into it.

1. Adopt git 3.0 if you can with sha256.

2. If you can't use sha256, set the config to put things back to sha1. Wherever you need to do this you probably already set dozens of ENV vars or settings, just add a new one.

Or write a 15 page analysis about how the above is so hard people will probably just find it catastrophic to even think about.

That's impressive. I suppose they had a more flexible architecture to make that change so fast.

Is there any writeup on why it was easy for them and not for git?

I mean, there are two things here. One is how difficult it is to have a different hashing mechanism. Brian and other heroes in the Git core group have done amazing work to make this _technically_ possible on a repo level. To test some of my theories, I trivially implemented MD5 and an insanely dumb and easily breakable hash backend. It's not _hard_ to change the mechanism now. It's about the community.

Fossil isn't difficult to change not because it's technically harder for Git but because Git has a community and ecosystem that Fossil does not. The cost is not in the individual project for Git, the cost is because there is _so much_ in Git and this bifurcates everything.

It's easier to make world breaking changes when the world is really small. If git could magically just get everything and everyone to cut over and use git 3.0 in a magic instant, it wouldn't be having this problem.

Emily's talk does a pretty good job of summarizing the issues with intermixing the hashes: https://youtu.be/eJJp0RE7cd4

Hm but the date is stored inside of the commit. The only way we can know that a commit's date is authentic... is through its hash. If I can forge commits with any SHA1 hash at will, I can make a repository whose head commit has the same SHA1 as the one in torvalds: /linux but where any commit was replaced by a malicious commit with the same SHA1 and a fake date. You have no way to detect that my repo is inauthentic other than through a deep history comparison. The whole idea behind a merkle tree is that just checking the hash of the top is sufficient to know the identity of the whole tree.

I don't know what the solution is, but I'm inclined to believe that any repo with a single SHA1 commit is as weak as a repo with all SHA1 commits.

(2 counter) is impractical because all nodes of git will not replace objects if it thinks it already has it. So any attack has to assume this is the first time the node fetched, which is difficult before trust is established, which is difficult. This is part of the argument Linus originally outlined for this vector, which is that it only works for _very recent_ objects.

(1/3 counter) is not what I argued. I argued from the worst-case position that collision and preimages were theoretically cheap and fast. Even in that case, I feel my arguments hold.

The main issue here is that you assume you can replace an existing object with a replaced one, which you cannot. Not only that, but in all known cases, the sha1dc variant of SHA1 that Git uses will even _tell_ you that someone tried to do this, which singles out the source quickly.

couldn't github reject a push that contains an existing hash in the repo?

It's easy for _one person_ to fix. It's not easy for the entire git ecosystem as a whole. GitHub, large internal corporate git repos, CI/CD systems, projects with submodules, etc. The second half the article explains all of this.

It’s already broken, but even though it’s broken it’s hard to generate git collisions because of the repo metadata. It’s easy to generate (for instance) standalone PDFs with identical hashes, but doing this with git in a useful way is much harder.

That said, it’s still a good idea to migrate to a more robust hashing algorithm. Defense in depth, etc. Just because it’s a difficult migration doesn’t mean it shouldn’t be done.

> sha1 is broken in 10 years. Suddenly everyone has to switch all at once on the same day because it is a critical security issue

If you read the OP article, the entire point he's making is that this would never happen, because a hash algorithm being "broken" doesn't matter in practice, because true supply chain security has nothing to do with file hashes.

> Adopt git 3.0 if you can with sha256.

Who is "you" in the context of a distributed version control system? I think this is not just the plural you, but the unbounded you -- it's all people who not just interact with your project now, but who you hope may interact with it in the future. The question is what the cost is of committing a near-infinite population to this migration, not the cost of doing a single `brew update` on your personal machine, no?

You can certainly do this, as I said, this is Google's backup plan. But defaults matter. People will start running this and getting repos that are uselessly incompatible with other repos, tools, libraries and server instances. Having it as an option is one thing. Making it a default will cause a lot of pain for people who don't want to care about this.

For the record, Y2K was not fud. It was very real, in a long list of datetime problems that are to come. Further datetime problems are coming at scheduled dates.