Rejuvenating Autoconf(lwn.net) |
Rejuvenating Autoconf(lwn.net) |
Are there even any notable commercial projects that use autotools?
Autoconf solves problems that existed in the 90s unix market which are just not very interesting or relevant today. For a few decades now "portable" has meant linux, windows and mac.
A lot of effort is spent on generating shell scripts for the lowest common denominator and this is both useless (nobody cares about 90s unix, everybody can install bash) and insufficient (bourne shell is not native on windows).
Perhaps energy would be better spend on migrating everybody towards newer build systems or just plain makefiles. A lot of projects have migrated already, does anyone have statistics about what percentage of a distro like Debian needs autotools?
Autoconf often does a good job in having some “—with-libX11=some-path” but sometimes people just do weird stuff with their packages.
But maybe I’m just butthurt from trying to package stuff to nixos and having too many “smart build tools” fail.
Did you even read the article? The work is supported by Bloomberg for their internal use with Python, a clear commercial usage. https://lwn.net/Articles/835687/
Bloomberg has about 10 G$/y in revenue. https://en.wikipedia.org/wiki/Bloomberg_LP
That’s a great way to stifle any kind of innovation. Some newcomers may still appear that are worthwhile.
You're absolutely correct that Autotools is no longer synonymous with portability. If it ever was. Not catering for Windows or iOS means it's missing out on the lion's share of the market. There's a massive misplaced priority in catering for 90s Unix platforms which have marginal use by hobbyists and zero commercial use, above current open source and proprietary platforms. And this doesn't just extend to operating systems, it also extends to language support. C++17, C++20, Rust, Go, Swift, you name it. It's not kept up. Likewise build systems. It's make-only. And what about language and platform feature tests and support for stuff like threading, locking etc. It's non existent other than some third-party support which is quite limited.
It's for all these reasons, as well as other well documented problems, that I moved all my personal projects over to CMake, as well as work projects, and other open source projects. I needed to support Windows. I needed to support building libraries on multiple platforms. I needed threading support. I needed C++14, and C++17 support.
For some open source projects I maintain both Autotools and CMake support. I've had several instances of people trying to build on e.g. Solaris and having problems. I always ask "Were you using the Autotools build, or CMake?" They tell me they used the Autotools and I suggest retrying with CMake. It always works. In short: portability to other Unix systems is inferior today compared with CMake. That's supposed to be its main advantage...
Agreed about the shell. There is no point to M4 and it could be removed compatibly by replacement with shell functions. This was true a decade ago. It was even true two decades ago.
With regards to migration. The time, money and effort required to upgrade the Autotools would be better spent on converting the affected project(s) to use CMake or some other more modern build tool. The functionality gap is simply too big to overcome at this point, and the contributor base for CMake is large. The FSF have only themselves to blame for their contribution and development practices hindering forward progress, as well as a lack of concern for the real-world needs for software development. The turnaround time for submitting a CMake change is a few days to get it change reviewed and integrated, and a few weeks to be in a release.
I've converted some large and complex projects to use CMake over the years, including Xerces-C, Xalan-C, libtiff and others. All in all, that time was well spent, and I'd rather have done this than waste time updating Autoconf/make/libtool.
Regarding Debian. Current unstable Build-Depends is 1999 (CMake), 1544 (Autotools), 395 (Meson). So I think we're a good way past the tipping point now.
Full disclosure: I've done the FSF copyright assignment for Autoconf and Automake and submitted some major features like C99 support. I'm also a CMake maintainer and I've submitted many different changes there. Primarily lots of Find module support including the Boost support, but also some lower-level bits and pieces as well. The community surrounding CMake is very active and also very friendly to submissions from new people. Kitware really deserve appreciation and thanks for their fostering and growing the community and committing resources to it including timely and friendly code review.
These days pretty much everything build-depends indirectly on autotools, via debhelper → dh-autoreconf → autoconf.
These numbers are probably more meaningful:
$ wget -q https://deb.debian.org/debian/dists/unstable/main/Contents-source.gz
$ zgrep -E $'(^|/)configure[.](in|ac)\t' Contents-source.gz | cut -f 2 | tr ',' '\n' | sort -u | wc -l
5562
$ zgrep -E $'(^|/)CMakeLists[.]txt\t' Contents-source.gz | cut -f 2 | tr ',' '\n' | sort -u | wc -l
2455
$ zgrep -E $'(^|/)meson[.]build\t' Contents-source.gz | cut -f 2 | tr ',' '\n' | sort -u | wc -l
466Autoconf has been using shell functions since 2008 (support added by me :)). M4 is used to properly handle dependencies between tests and to store information that is used later when generating files.
CMake requires a C++11 compiler and quite a few C libraries like libuv, so you can't even compile it on platforms like HP-UX.
Last time I checked, the CMake docs say that adding find module support is responsibility of the user. Which is a main reason why pkgconf-based dependency resolution works much better with other build systems. If you compile package abc on, say, Ubuntu, and it needs package xyz you do not need to bother about how it is found. You just install xyz.
What I want to say, CMake makes things way more complex and solves problems which exist only on Windows. Otherwise, it does not add much, and is a burden. Why should developers on Linux carry that burden?
Do you mean the market for free software?
I guess the GNU/Linux developers are not that keen to work for free to support commercial, nonfree platforms.....
Guix does support, for example, Rust and other languages relevant for Linux pretty well, and in a much better defined way than CMake.
“At the bottom of the food chain, so to speak, is libtool, which tries to hide the fact that there is no standardized way to build a shared library in Unix. …the source code for devel/libtool weighs in at 414,740 lines. Half that line count is test cases… the tests elaborately explore the functionality of the complex solution for a problem that should not exist in the first place. Even more maddening is that 31,085 of those lines are in a single unreadably ugly shell script called configure. The idea is that the configure script performs approximately 200 automated tests, so that the user is not burdened with configuring libtool manually. This is a horribly bad idea, already much criticized back in the 1980s when it appeared, as it allows source code to pretend to be portable behind the veneer of the configure script, rather than actually having the quality of portability to begin with. It is a travesty that the configure idea survived. …
“The configure scripts back then were written by hand and did things like figure out if this was most like a BSD- or a SysV-style Unix, and then copied one or the other Makefile and maybe also a .h file into place. Later the configure scripts became more ambitious, and … rather than standardize Unix to eliminate the need for them, somebody wrote a program, autoconf, to write the configure scripts.
“Today's Unix/Posix-like operating systems, even including IBM's z/OS mainframe version, as seen with 1980 eyes are identical; yet the 31,085 lines of configure for libtool still check if <sys/stat.h> and <stdlib.h> exist, even though the Unixen, which lacked them, had neither sufficient memory to execute libtool nor disks big enough for its 16-MB source code.”
For most of this year, @elwoz and I have been working on a pretty ambitious project:
* to make a fresh release of GNU Autoconf, a crucial #FLOSS build tool that hadn't had a new release since 2012
* to get paid for that
* to help put Autoconf on a more sustainable footing, so it doesn't have to get rescued again a little while down the road
https://gitlab.kitware.com/cmake/cmake/-/issues/17137
Part of it is HP saying they'd have a C++11 compiler and then not, some is modern GCC being a problem to compile as well.
C++11 and libuv have vastly improved the situation in CMake's codebase itself and letting HPE hold everyone else hostage for that would not have been worth it (and it takes a lot for us to drop platforms). IRIX was dropped because its owner killed it off and we're looking at dropping support for macOS 10.9 and older (in premade binaries; feel free to compile yourself) because we don't actually have tests running on those older releases.
https://gitlab.kitware.com/cmake/cmake/-/issues/20825
Full disclosure: CMake developer here.
None of this consideration is really concerning free software or proprietary software specifically. It's simply looking at cross-platform capability. If you need to develop cross-platform software of either sort, it's not typical to be able to ignore Windows. It's ~90% of the market. Similar considerations for other markets like smartphones.
None of that is reflective of my personal preferences, simply stating the requirements which drive the build system considerations of many projects. The GNU Autotools are unsuitable for the vast majority of current systems in existence, and that means selecting it for its portability is nowadays often not a possibility because their choice to not support common platforms makes it untenable.