Ask HN: What executable format is objectively the best?

4 points by thrwawy283 4 years ago | 5 comments

I'm no great software engineer, but I'm curious if there are any strong opinions on which executable format is the best. Formats like PE or ELF or (a.out?) or others?

Typically if you run a particular operating system you're not going to see several types of executable formats. The question wouldn't come up.

There have to be others with odd advantages, like an executable format that lends itself toward streaming-in the parts of the binary you're executing (exec as you torrent, lol). Or perhaps 1 that stores different parts of itself in a SQL database (read-only, strings, debugging info, etc). Someone has to have created an executable format out of sqlite databases. I remember "Universal" binaries on Mac archived several architectures' binaries together.

In the interest of a discussion I haven't seen: What is the best executable format and why? What are some creative ones?

mikewarot 4 years ago |

I think the .COM file was one of the most elegant ways to go. Load the code into a segment at address 100, and start at that address. No parsing, just load and go. It would be interesting to see 32 and 64 bit equivalents.

In the present day, the best, and most creative, format I'm aware of is Actually Portable Executable[1], which is a Windows Portable Execution (PE) file with some added hacks to make a single executable work in a wide variety of places.

[1] - https://justine.lol/ape.html

jepler 4 years ago |

I don't know that there could be a single "best" executable format. Here are some thoughts, as a person who knows some about Linux and little to nothing about Windows or Mac. As sibling states, formats have largely stagnated but in Linux there's continuous progress on debug information. Another interesting area for research would seem to be executable/object file formats for fast linking or for later patching/updating. Of course, projects like gold and mold (and maybe the clang linker, lld?) show that substantial performance improvements can be had without changing formats.

kstenerud 4 years ago |

All of these formats evolved out of necessity up until the 90s, and innovation largely stagnated after that since there's little to be gained in this area anymore.

colburnmh 4 years ago |

The one that allows you to run your application on the computer you are using.

archi42 4 years ago |

tl;dr: I think it's not a good question. IMHO the best format is the one your tools already know, and (if you're building tools) which can be expanded for your future needs.

Anecdata: At an old job, we worked with various formats and I recall that people in the company were more happy with ELF than with other formats; a sentiment I shared. I can think of two major reasons (my POV, others from that company might beg to differ): 1. If you already can parse ELF for five platforms, adding the 6th is quite easy and 2. a new niche format means not only parsing a few bytes of poorly documented header info (which is usually easy) so you know where to start disassembling, but if you need the debug information (which we did) you need to parse & understand their custom format as well (and map the info to how you keep debug info in memory and process it).

Now ELF might be pretty static, expect maybe adding some sections for LTO, but DWARF did and does change much more. DWARF v5 was nearing release when I changed careers. Since debug info describes the underlying program, I would identify these main drivers for change (though there are more, and other people have other emphasis): 1. Represent programs written in new languages or for new architectures, for which the current present representation is not well suited 2. better represent the program so tools can be improved, especially when optimizations are enabled 3. keep the size of the debug information at manageable levels and finally 4. - and this should not be underestimated - make sure that a consumer can easily load the relevant debug info while ignoring things it doesn't need (having 100MB of size-optimized debug info on disk is one thing, but keeping the same debug info in memory and optimizing the data structures for actually using it could mean using a GB or two; the largest file I've ever seen contained 8GB of DWARFv1 or v2 [the compiler failed to properly optimize the size]). And as I already said, if you can already parse ELF/DWARF for some target (Linux on x86 or ARM) you can also parse it for more obscure targets (e.g. some RTOS on a PIC). Worst case you need add a few compiler-specific extension or handle some quirks.

Maybe some day the issues with ELF/DWARF stack up and become too much and someone suggests a new/better format that fixes these while keeping the good stuff; maybe it even replaces ELF.