How can the Linux kernel compile itself?

How can the Linux kernel compile itself?(stackoverflow.com)

34 points by dchester195 12 years ago | 31 comments

lutusp 12 years ago |

> How can the linux kernel compile itself?

Short answer: it can't. Each software system relies for its own existence on a predecessor, usually one of smaller scope. Sort of like biology and evolution.

When I wrote Apple Writer in 1978, there were no high-level development tools available, just a primitive assembler built into the Apple II BIOS. By making direct hand entries, I used the BIOS assembler to write a better, faster assembler that supported alphabetic labels. Then I used that assembler to write a better one, and eventually I had a development tool suitable for writing applications.

Linux development has relied on a similar bootstrap process, on a larger scale.

FedRegister 12 years ago | |

Just out of curiosity, have you ever written a book about the process of bootstrapping an Apple II development environment and the little discoveries you made? I would read that in a heartbeat.

lutusp 12 years ago | | |

> Just out of curiosity, have you ever written a book about the process of bootstrapping an Apple II development environment and the little discoveries you made? I would read that in a heartbeat.

Thanks for this suggestion! I write a lot of articles, but I haven't written much about that period, possibly because other people were writing it for me. But I think it might be interesting to write about that time, how we created software before we had access to high-level languages.

In the meantime, here's an old piece I wrote in those days, even though not quite what you have in mind:

http://www.atariarchives.org/deli/cottage_computer_programmi...

chris_wot 12 years ago |

If he thinks that's weird, then he ought to read Ken Thompson's article about trusting compilers.

http://cm.bell-labs.com/who/ken/trust.html

IgorPartola 12 years ago | |

While I love the story, it's really amusing to watch how it gets re-discovered on HN every few years, blows everyone's minds, and then gets quoted all over the place, even where it is only mildly relevant. This might be our "does glass flow" meme.

chris_wot 12 years ago | | |

Seemed relevant to me. The question asked about compilers and the kernel is what compiled the compiler. Ken Thompsons' talk related to hiding code by adding code to a compiler then compile the compiler, remove the code then continue to compile the compiler with the modified version that doesn't have the source code - your modification remains.

Point being: both relate to compiling compilers.

vowelless 12 years ago | |

A somewhat related albeit primarily entertaining idea is that behind the Quine: http://en.wikipedia.org/wiki/Quine_(computing)

dllthomas 12 years ago | |

Obligatory DDC link: http://www.dwheeler.com/trusting-trust

tcoppi 12 years ago |

This is why I think more people should expose themselves to hardware design, CPU design, and compiler writing, if only at an abstract/academic level. I've worked with a lot of entry-level programmers, and even some that were not so entry-level, that wouldn't have a clue how to answer this question, which is quite depressing.

dublinben 12 years ago | |

I'm not a programmer at all, and I found this answer quite obvious. Where else does new binary software come from, if not from a computer running older software?

lutusp 12 years ago | | |

It shouldn't be such a mystery, because it follows a biological model -- small, simple organisms produce more complex descendants by way of natural selection, all fueled by proteins. The same process creates complex software systems out of simple predecessors, all fueled by coffee.

Someone 12 years ago | | |

If you think through, it is not _that_ obvious; it can't be turtles all the way down. So, where did the first binary software come from?

The answer is that somebody punched in a program from a computer's console, probably somewhere in the early 1960s, but possibly earlier. That program was used to make it easier to enter other programs, etc.

Given that copying working code is so simple, chances are that most of the currently running hardware can be traced back to a few persons entering a boot loader by hand.

That might even be one person. For example, Windows NT was initially developed for Intel's i860 (http://en.wikipedia.org/wiki/Windows_NT#Development), so chances are that it was bootstrapped from a Unix machine. Because of that, it may trace back to the same initial boot loader as, for example, Mac OS X.

mkohlmyr 12 years ago |

Link to Ken Thompson, link to Quines and that one dude who always goes 'When I wrote Apple Writer...' - all in the first few comments.. thread complete?

niuzeta 12 years ago | |

i'm still waiting for some way to tangentially connect this issue with NSA backdooring.

Robin_Message 12 years ago | | |

I guess you found one.

XorNot 12 years ago |

This does make me wonder about something though: is there a crashplan for a compiler? Can a very minimal gcc be written which would be capable of bootstrapping the entire gcc compiler?

It seems like it would be relevant if a new architecture was developed at least.

FedRegister 12 years ago | |

If a new architecture was developed you could bring up the whole stack by cross-{assembling,compiling} from an already supported target. By doing so you could test the output using a simulator (or actual hardware) and then when the assembler/compiler is stable enough you could then cross-assemble/compile the assembler/compiler and then be self-hosting on the new architecture.

PeterisP 12 years ago | |

For any practical purposes, it is best to simply use the full complex software stack on another computer to generate binary code meant for that new architecture.

If we'd really need to bootstrap a compiler from scratch, then a possibility would be to hand-code a Forth system, and then find/modify/make an assembler written in forth; then write a compiler for some small language (scheme?) in that assembler, and then from scheme you can easily get to some C compiler that can compile the full GCC.

seclorum 12 years ago | |

>Can a very minimal gcc be written which would be capable of bootstrapping the entire gcc compiler?

Yes. In fact, this is how gcc is built - in stages. First stage: build a compiler that can build the compiler. Recompile the compiler with the newly built compiler. Do it again. Repeat until completion. ;)

OnACoffeeBreak 12 years ago |

Another interesting thing to remember is that an embedded operating system (Android phone OS and software, for example) is likely being built on a PC set up with a cross-compile environment and not directly on the embedded hardware.

username42 12 years ago |

The bootloader can compile linux before booting : http://bellard.org/tcc/tccboot.html

gnarbarian 12 years ago |

http://en.wikipedia.org/wiki/Self-hosting