Mold: A Massively Parallel Linker(arxiv.org) |
Mold: A Massively Parallel Linker(arxiv.org) |
see https://lwn.net/Articles/276782/
GNU gold is deprecated and will probably be removed soon. ( https://www.phoronix.com/news/GNU-Gold-Linker-Deprecated )
But back in the late 2000s, the blog posts were a combination of algorithm optimization class and software history since no one seemed to step up to cleanup the technical debt that GNU ld had accumulated over the decades.
I read how mold implements parallel input file parsing a while ago. It is overly complicated. wild's model seems superior.
---
"Table 4." in the paper feels misleading to me. I have never measured such high speedups. In addition, on debug builds (especially --gdb-index builds), mold's edge over lld is smaller.
On ICF, some unknown fraction of mold's speedup is due to "doesn't do the symbol bootkeeping". The symbol tables are different in mold but identical in lld.
(https://x.com/rui314/status/1560277403787997185)
> my strategy is to increase the value of commons by, say, 100 and get 1 as a return, so bear with me. If you want to use mold for free, you can just do that. This is my open-source business strategy.
Wishing the best for Mold and Rui
https://github.com/wild-linker/wild
(Benchmarks in the README.)
Edit to add: I think David's talk on Wild [0] is an interesting summary of what a linker is and why its hard to make incremental
Told me Australia in two words
Seems that some optimizations went to mold 2.42 and others are not yet released. Wild benchmarked mold 2.40 or 2.41
Mold is already very fast, however it doesn't do incremental linking and the author has stated that they don't intend to.> A new linker's early adopters knowingly accept such differences, which is one reason why drastic improvements tend to come from new linkers rather than existing ones
This is true of many things, not just linkers. I think this is encouraging for people to (re)try things that have already been done, because newcomers are allowed to take risks that established things can't take.
I use mold a lot for code embedding in my lisp. I needed extra null PHT entries in the ELF so I could easily patch in custom segments into the interpreter after the link was done.
GNU pretty much ignored the feature request and LLVM probably never even saw the issue, but Rui shipped the --spare-program-headers feature pretty much immediately after I requested it and it immediately restored my sanity.
https://www.matheusmoreira.com/articles/self-contained-lone-...
Current version of my patcher supports ld and lld, but it has to move the PHT to the end of the file in order to append the new segments. Mold is still the only way to avoid the ugly hole where the PHT used to be.
Mine is also much faster than mold/wild, because I do much less work, and I have all the objects already in memory. Similar to tinycc. Only when someone needs a very unusual unsupported linker feature, I have to fallback to an external linker, which is ~100x slower. The less work it has to do, the faster. I dont appreciate the unix habit to use intermediate files all along, which needs additional readers and writers.
Are you sure? All three servers I regularly use, as well as all of the mirrors that I found with a quick search, are down. If there are other servers still working then can you please give a hint about how I could find them?
Before I created lld, linking was much slower than it is today, and people somehow seemed to accept that as a given. People would often argue that since linking was slow, we needed to reduce the amount of linking work by implementing incremental linking. I thought that was too pessimistic. Why don't we just make the usual full link much faster, ideally almost as fast as a cp command, so that we don't need incremental linking nearly as much?
In other words, I wanted to solve the slow link problem head-on rather than work around linker slowness. And now we can link multi-gigabyte binaries in a second or two, which makes incremental linking much less urgent than it was ten years ago. That's pretty much what I wanted to achieve.
Am I satisfied with the speed? No. Could incremental linking speed it up even more? Maybe. Would it still be useful? Sure. It may well be worth doing. But people don't need it nearly as badly as they did ten years ago. I wanted to solve the larger problem first.
Says who? You make it sound like there's some authority out there assigning these numbers. It's not like GNU had to go out there and beg them for it. They just did it, and the world just had to cope.
Also, I'm building lone directly on top of the Linux kernel. That absolutely makes me the "operating system", in the GNU and POSIX sense. Lone/Linux, if you will. The idea is to take this far enough to boot Linux into a custom lisp userspace.