Introducing Kirk

41 points by IanChiles 12 years ago | 46 comments

blueblob 12 years ago |

I take quite a big issue with the statement "The lack of information and documentation on developing OSes is, in my opinion, a pretty big problem"

There are entire forums for this, ie osdev http://wiki.osdev.org/Expanded_Main_Page

There are entire books for this http://www.amazon.com/s/ref=sr_ex_n_1?rh=n%3A283155%2Cn%3A5%... amazon lists about 57,000 results for operating systems books

You will get out of LFS what you put into it. It tells you how to download the source, extract and install it. When you have the source, nothing is preventing you from looking at it.

I also take issue with "how can we hope to continue developing and innovating on the OS level when it’s practically impossible to figure out what anything does or how it works."

There are general concepts that you will learn from getting a book on os development (including the kernel) and writing a minimal kernel yourself will give you insights into the work that goes into a full kernel such as the linux one. The concepts, not the specifics are what you need to learn, and you do this through theory, you pick up the rest from implementation (but starting with a kernel, half of the os is already developed fof you).

IanChiles 12 years ago | |

Absolutely agree with you. I struggled with wording this for the longest time - and when I talk about OS development, I'm referring much more to the userland development - package managers, core utilities, shells, init systems, and the like, than I'm referring to kernels. Sorry for the confusion.

blueblob 12 years ago | | |

I think some of the less explored portions of LFS contain well-documented reasoning for implementing some of these things yourself. For example there is a single solution (out of the many ways to solve this problem) to the package manager dilemma in the hints section of LFS

http://www.linuxfromscratch.org/hints/downloads/files/packag...

which explains reasoning for the package manager and how to implement it yourself (in a much less copy-compile-execute manner than LFS itself).

Implementation is a great way to learn but generally people will survey the literature first to understand why things are implemented the way that they are. Understanding what all package managers have in common will allow you to distill it down to the simplest factor and implement that.

A lot of the core utilities and why they are expected should be covered in the standard (POSIX in this case).

I guess what I am trying to say is that implementation is a useful way to learn but it requires you to know the details whereas understanding the literature first will give you a better idea of what to implement.

Good luck with your project.

runjake 12 years ago |

  For as long as I can remember, I’ve wanted to create my
  very own operating system, one which I have complete and 
  total control over. Because I have neither the time, nor 
  the skill to write my own kernel, this means that I’d 
  eventually land up creating a OS based on the Linux 
  Kernel.

Non sequitur. If you want complete and total control, you're going to have to learn the ins and outs of a kernel.

I applaud your decision to go forth with this, but is it really worthy of an HN self-post given that Kirk does not exist and is just a blog post at this point?

IanChiles 12 years ago | |

For me, at this moment, it's simply not a feasible goal to learn the ins and outs of a kernel in any reasonable amount of time. And Kirk doesn't just exist as a blog post, I've done a relatively large amount of work on it, but am just now starting to write about it by request of a friend.

runjake 12 years ago | | |

If I were going to go back and learn the ins and outs of an OS again, I'd still go back and start learning from the bootloader to the kernel and then progress from there. The kernel is the OS for all intents and purposes. It sounds more like you want to learn how to create your own Linux distribution.

Don't fret, we've all been there and done this ourselves :). Some people catch the bug and create stuff like Gentoo and Arch.

Patrick_Devine 12 years ago | | |

I think it's a worthwhile endeavour to stick a distro together, however I agree with a lot of the other commenters here. If you really, truly want to learn about how an OS works, you're going to need to get your hands dirty and learn about the kernel. There are so many tradeoffs made on the kernel side of things which impact userspace, and knowing about them impacts the decisions you're going to make in terms how how you configure software and which software you're going to use.

If you can afford it and have a week to spare, I'd really recommend taking the Linux Foundation's one week crash course on the kernel. It talks about how the scheduler works, gives you insight into modules, how processes and threads work and hooks the kernel libraries give you for semaphores, mutexes, etc. It's expensive, but it's well worth the money.

http://training.linuxfoundation.org/linux-courses/developmen...

antocv 12 years ago |

Kirk does not exist, this is a blog post. Just a reminder to everyone like me who went looking for it.

And the blog post is about the same dust that was relevant 10 years ago and sparked the creation of LFS. Nothing new to see here.

fusiongyro 12 years ago |

> an OS without limits, built to be open from the ground up - with as open of a development process as possible.

I see the benefit of documenting each step. The result will be a pretty great tutorial, or at least a very interesting series of blog posts. But what is closed about the development of the current distros?

I hope that Ian is hoping to build a very different kind of distro, breaking the mold. NEXTSTEP was fundamentally a Unix, but everything about it was synthesized to create a totally new organism. Same with OS X. This kind of experimentation hasn't been attempted very much with Linux distros, which mostly amount to different installers and package managers. Etoile comes to mind, and apparently CoreOS is discarding much of the mold, but a completely new, holistically built system would be quite interesting to see. I suspect it hasn't been attempted much because the sheer amount of work is overwhelming. Writing a new shell is a lot of work. Writing a different init, shell, filesystem layout, and window manager, all unified around some new concepts, would be a lot of work. If that's what he's aiming to do, the result will be really interesting to use and learn from.

IanChiles 12 years ago | |

If there's any confusion, that's absolutely what I plan on doing - but I firmly believe that if it's kept simple/lightweight that it's more than doable. The GUI portions slightly worry me, but they're not happening until much later down the line

norswap 12 years ago | |

> Same with OS X

OS X is BSD with a fresh coat of paint. It really depends if you want to consider the whole experience (which I consider less stellar than most people would care to admit) or the internals (which are the same old unix stuff).

mwfunk 12 years ago | | |

Mach and IOKit and and a gazillion other things are hardly "the same old Unix stuff", if you're talking about the kernel. If you're talking about everything above that level, read Amit Singh's book if you really think there are just splashes of superficial changes at and above the POSIX layer. If you have and you still feel this way, I don't know what to tell you. I'm not saying it's all great, but there's a LOT there.

jff 12 years ago | | |

It's not a BSD kernel. It has the BSD userland, though.

examancer 12 years ago |

I like the concept, but wish this announcement would have come with at least one or two concrete steps taken towards the stated goal. Maybe your experience getting a basic shell working or something similar. At this point I have no reason to believe anything will come of this. Was maybe a bit too early for an "announcement".

v0land 12 years ago |

Dreams: "I’ve wanted to create my very own operating system, one which I have complete and total control over."

Reality: Another Debian clone out there.

IanChiles 12 years ago | |

I assure you that I won't do that. If I wanted to make a Debian clone I wouldn't be doing this

olalonde 12 years ago |

> the development process of Kirk be accessible enough for anyone interested to jump right in and contribute

Would be a good idea to host it on Github.

IanChiles 12 years ago | |

Absolutely 100% will be, don't worry

jff 12 years ago |

I want to echo the recommendations of others: learning about kernels and writing your own would be a lot more valuable.

I'm speaking from experience when I say that trying to provide a userland sucks. You've said elsewhere on this page that you want to write your own shell and such; the problem is that people don't want IanSH, they want bash or zsh or whatever. If you write IanMACS, they'll ask how to run emacs. And I hope the libraries you write can compile Firefox.

Or if you just want to write init replacements, modify how configuration is managed... remember that Ubuntu (and Gentoo, but nobody cares about Gentoo anymore) did a ton of this and everyone I've talked to hates it.

So here's my concerns:

- If you intend to write an entirely new userland, you're going to spend a lot of time replicating existing work, only to have people complain about it.

- If you intend to do a new init, improve configuration management, change how logging works, whatever, your life will be consumed with modifying every program to use your stuff.

If you want to learn about operating systems, I'll echo the others and say write your own kernel and a minimal userland.

Skoofoo 12 years ago |

The open source desktop is kept alive mostly through donated spare time. This time is limited. We would do well to maximize the impact of time spent creating and maintaining open source desktop software.

Right now the standard language to build Linux software is C, which is great for performance, but not great for development speed. There are ports of things like GTK+ to other languages, but ports are less supported and usually don't take full advantage of their language's unique flavor.

To maximize impact per unit of time spent developing open source software, and also perhaps to entice more people into contributing to open source software, I believe it would be a good idea to build a desktop environment/widget toolkit with an opinionated API in Ruby. Anyone who has experience writing Ruby on Rails code with an automated red-green-refactor cycle likely could testify that it is pleasant and fast. Why not bring that sort of experience to building desktop software?

IanChiles 12 years ago | |

I considered Ruby and have worked heavily with it in the past, but it's simply too slow/heavy for this type of programming.

I can't believe I left this out of the post, but all/most new code will be written in Go - I'll explain why more clearly in my next post

clarkm 12 years ago |

If you're interested in how a simple version of unix works, I highly recommend checking out Xv6, a teaching OS based on UNIX version 6:

https://pdos.csail.mit.edu/6.828/2012/xv6/book-rev7.pdf

ivanbrussik 12 years ago |

I applaud his efforts - but how is it so many people find main stream Linux distros that non-configurable that they need to create their own?

I mean, even distros like Ubuntu are highly configurable. You can kill off as many packages as you want and even compile your own kernel if you really want to.

diaz 12 years ago |

I completly understand the feeling and it's something that I wanted to do for a long time. I thought of starting with LFS but never got much time to play with it.

I'll definetly follow your posts about this.

I don't get why so many negative comments here. There were many times that I really wanted to understand and configure all my machine just for the sake of it.

Understanding and learning, it's simple. That's the main reason I use Arch, but even with it I don't usually go really under it to understand what program calls what and what is the function of program foobar, why it really is needed or how many switches and power can it give.

This is great :).

agumonkey 12 years ago |

I'm curious about the result, I share his idea about LFS being a little bit too 'automatic' and the feeling of blindness that resulted. The joy of booting a system build from source was there, but didn't last long, in fact I quickly hit very weird network issues such as elinks seeing the network but not wget, reinforcing the notion that I there was a need for a different kind of information about the system.

drill_sarge 12 years ago |

I don't understand. You want to do a non-GNU/Linux OS or what? Or "just another GNU/Linux distro"?

jff 12 years ago | |

It sounds more like Kirk/Linux.

IanChiles 12 years ago | | |

Which is exactly what it'll be. I really hope to keep the userland portable - so Kirk/FreeBSD or Kirk/Darwin could happen in the distant future

donniezazen 12 years ago |

Arch Linux strikes perfect balance between usability and learning. I am looking forward to what you have to say.

internalnet 12 years ago |

How in the hell is this kind of vaporware garbage on the front page?

Shut up and code.

For as long as I can remember, I’ve wanted to create my very own operating system, one which I have complete and total control over. Because I have neither the time, nor the skill to write my own kernel, this means that I’d eventually land up creating a OS based on the Linux Kernel.