Writing a Unix clone in about a month(drewdevault.com) |
Writing a Unix clone in about a month(drewdevault.com) |
Source: UNIX: A History and a Memoir Paperback – October 18, 2019 by Brian W Kernighan (Author)
[1] SELinux is unmanageable; just turn it off if it gets in your way:
Right. Almost nothing does.
You see, it's https://en.m.wikipedia.org/wiki/Turtles_all_the_way_down
https://www.youtube.com/watch?v=wqI7MrtxPnk
By the way the CHM oral history video series is full of gems.
Sometimes small and simple is good.
I'll have to check because my memory is failing me atm.
Would love any resources that goes in more details, if any HN-er or the author himself knows of some!
As a baseline, I support developers using whatever license they would like, and targeting whatever operating systems, indeed, writing whatever code they would like in the process.
That doesn't make this specific policy a good idea. Even FSF, generally considered the most extreme (or, if you prefer, principled) exponents of the Free Software philosophy, support Windows and POSIX. They may grumble and call it Woe32, but Stallman has said some cogent things about how the fight for a world free of proprietary software is more readily advanced by making sure that Free Software projects run on proprietary systems.
They do at least license the library code under MPL, so merely using Hare doesn't lock you into a license. But I wonder about the longevity of a language where the attitude toward 95+% of the desktop is "unsupported, don't ask questions on our forums, we don't want you here".
Ironically, a Google search for "harelang repo" has as the first hit an unofficial macOs port, and the actual SourceHut repo doesn't show up in the first page of results.
Languages either snowball or fizzle out. I'm typing this on a Mac, but I could pick up a Linux machine right now if I were of a mind to. But why would I invest in learning a language which imposes a purity test on developers, when even the FSF doesn't? A great deal of open source and free software gets written on Macs, and in fact, more than you might think on Windows as well.
From where I sit, what differentiates Hare from Odin and Zig, is just this attitude of purity and exclusion. I wish you all happy hacking, of course, and success. But I'm pessimistic about the latter.
Example of “creating something impressive in X days” requires a lot of experience and talent that is built over years.
GPLv3 license.
That answered my initial surprise of clicking on the ISO and getting a 60MB download.
For comparison, Linux 0.01 was a 71k download, but contained only the kernel source.
Though this limitation will limit its adoption in this multicore age I think:
From the FAQ https://harelang.org/documentation/faq.html
....
Can I use multithreading in Hare?
Probably not.
We prefer to encourage the use of event loops (see unix::poll or hare-ev) for multiplexing I/O operations, or multiprocessing with shared memory if you need to use CPU resources in parallel.
It is, strictly speaking, possible to create threads in a Hare program. You can link to libc and use pthreads, or you can use the clone(2) syscall directly. Operating systems implemented in Hare, such as Helios, often implement multi-threading.
However, the upstream standard library does not make reentrancy guarantees, so you are solely responsible for not shooting your foot off.
This is actually pretty powerful. I personally prefer it for most purposes, because it restricts the possibility of data races to only the shared memory regions. It's a little like an "unsafe block" of memory with respect to data races.
Well, that not only rules out multi-threading, but also usage in interrupts. Quite a limitation for a "systems programming language" methinks.
> google/syzkalleR
> Fuschia / Zircon syscalls: https://fuchsia.dev/fuchsia-src/reference/syscalls
"Memory Sealing "Mseal" System Call Merged for Linux 6.10" https://news.ycombinator.com/context?id=40474551
"I called it Linux originally as a working name. That was just because "Linus" and the X has to be there--it's UNIX, it's like, a law--and what happened was that I initially thought that I can't call it "Linux" publicly because it's just too egotistical. That was before I had a big ego."
Seems a bit like Python’s philosophy of not introducing too much optimizations to prevent the runtime complexity from spiraling out of control.
Plus it is a heavy dependency which means projects like writing a self-hosting OS in a month are much less realistic to achieve when your compiler relies on LLVM.
And not the least, the code generation is pretty slow. If your languages cares greatly about compile speed, which it should, this is a bummer.
So yeah, for many projects avoiding LLVM might be a good idea.
Like PC bootstrap, basic kernel action loops, process forking, yada yada
https://www.amazon.com/Advanced-Programming-UNIX-Environment...
It is about using all Unix APIs from user space, including signals and processes.
(I am not sure what to recommend if you want to implement signals in the kernel, maybe https://pdos.csail.mit.edu/6.828/2012/xv6.html )
---
It's honestly a breath of fresh air to simply read a book that explains clearly how Unix works, with self-contained examples, and which is comprehensive and organized. (If you don't know C, that can be a barrier, but that's also a barrier reading blog posts)
I don't believe the equivalent information is anywhere on the web. (I have a lot of Unix trivia on my blog, which people still read, but it's not the same)
IMO there are some things for which it's really inefficient to use blog posts or Google or LLMs, and if you want to understand Unix signals that's probably one of them.
(This book isn't "cheap" even used, but IMO it survives with a high price precisely because the information is valuable. You get what you pay for, etc. And for a working programmer it is cheap, relatively speaking.)
If all syscalls are async (a design principle of many modern OSes) then that aspect is solved. And if there is a reliable channel-like system for IPC (also a design principle of many modern OSes) then you can implement not only signals but also more sophisticated async inter-process communication/procedure calls.
See the original comment [0] for slighlty more spellt out ideas on better designs for those three-and-a-half concepts.
You have SIGSTOP/SIGCONT/SIGKILL, which don't even really signal the process, they just do process control (suspend, resume, kill).
You have simple async messages (SIGHUP, SIGUSR1, SIGUSR2, SIGTTIN, SIGTTOU, etc) that get abused for reloading configuration/etc (with hacky workarounds like nohup for daemonization) or other stuff (gunicorn for example uses the latter 2 for scaling up and down dynamically). There's also in this category bizarrely specific things like SIGWINCH.
You also have SIGILL, SIGSEGV, SIGFPE, etc for illegal instructions, segmentation violations, FP exceptions, etc.
And also things that might not even be good to have as async things in the first place (SIGSYS).
---
As an aside, it's not the only approach and there's definitely tradeoffs with the other approaches.
Windows has events, SEH (access violations, other exceptions), handler routines (CTRL+C/CTRL+BREAK/shutdown,etc), and IOCPs (async I/O), callbacks, and probably some other things I'm forgetting at the moment.
Plan 9 has notes which are strings... which lets you send arbitrary data to another process which is neat, but it using the same mechanism for process control imo has the same drawbacks as *nix except now they're strings instead of a single well-defined number.
If you're including all that other stuff, it's probably fair to include all of the subsequent development of notification mechanisms on the UNIX side of the fence as well; e.g., poll(2), various SVR4 IPC primitives, event ports in illumos, kqueue in FreeBSD, epoll and eventually io_uring in Linux.
It goes into the problems with Unix signals, and then explains why Linux's attempt to solve them, signalfd, doesn't work well.
How does Windows handle this? There's still signals, but I believe/was under the impression that signals in Windows are an add-on to make the POSIX subsystem work, so maybe it isn't as broken (for example, I think it doesn't coalesce signals).
https://pubs.opengroup.org/onlinepubs/009604499/functions/bs...
It's important to remember that code in a signal handler must be re-enterant. "Nonreentrant functions are generally unsafe to call from a signal handler."
On the other hand, that is hardly the only thing from the FAQ that raises one's eyebrows:
> we have no package manager and encourage less code reuse as a shared value
> qbe generates slower code compared to LLVM, with the performance ranging from 25% to 75% the runtime performance of comparable LLVM-generated code
> Can I use multithreading in Hare? Probably not.
> So I need to implement hash tables myself? Indeed. Hash tables are a common data structure that many Hare programs will need to implement from scratch.
As it stands, this is definitely not a language designed for mass adoption. Which is fine, and at least they're upfront about it.
While I understand your concerns, I disagree with your the idea of “imposition”. Someone doing something for free doesn’t owe anyone to do it in a particular way (as long as it’s not malevolent). You’re free to express your opinion, but if the developer has already established his guidelines, criticisms like this is not constructive.
Not every band has to hit the Billboard charts to be worth listening to.
Supporting an OS the devs don’t use is a big ask.
This is not true and a naive statement. There are quite few languages which are not popular across the board but have a very firm niche in which they thrive and fulfill critical roles.
So I get it. Especially if it is to be a more niche or pet project but then again I don't buy the ideological reason. I am a really big proponent of free software and their stance just doesn't make any sense. I agree with you here. But then again they can do whatever they want.
I believe Apple could probably get away with keeping Swift proprietary, or only supporting Apple platforms. But they don't. I have no inside-track information on why that is, but I suspect the reason is fairly simple: developers wouldn't like it.
I understand that you don't like it, but how do you come to regard a statement like this as "arbitrary?" It's exclusive, for sure. "Purity test" is one way to characterize it. But do you really think that statements like this are just the product of individual caprice? That it's not someone's attempt at a principled intervention, but just an "attitude?"
The Hares are saying they require that, which I totally understand and respect.
They just don't want to maintain Mac/Windows ports themselves. If somebody else is interested, they can maintain a port. Like that macOS one that you've already found.
I put the English string in the catalog, updated a number of tests, run the tests on the local system, pushed the change to staging cluster, fix unanticipated test failures, push the change to production, contact the translators to have the string translated to a number of languages, and have documentation updated.
Buy anyway there's no "then vs now" when you are really comparing "prototype" to "deliver to users". It took Unix decades to get those strings translated.
https://www.ticalc.org/archives/files/fileinfo/463/46387.htm...
Sadly defunct. I guess the real OS was the syscalls we made along the way.
I am not a programmer today, but I can still wrap most of my head around many low level concepts. I can't, however, write anything resembling a modern web page. Nor can I understand how any larger JS application works.
https://en.wikipedia.org/wiki/Not_Another_Completely_Heurist...
The best we have nowadays is using compute shaders for the same purpose.
Just like when using a TMS34010 with its C SDK.
Knowing how to write a kernel (which incidentally I am doing for the second time) doesn't mean you have years to dedicate to reverse engineer something as complicated as a GPU
https://multicians.org/unix.html
"Brian Kernighan takes credit for the idea, but adds that 'no one can remember' the origin of the final spelling Unix."
https://en.wikipedia.org/wiki/Unix
Also note that Prof. Kernighan has written a historical memoir about Unix:
Mr Stevens replied to something I wrote back in the day. I can't recall if it was a Usenet post or email, but I was over the moon!
A lot of the mechanisms are very OS specific but I do think they're good comparisons to have with signals as well.
Seems like a good decision.
The lesson seems to be: it's a race to the bottom on price. The lesson seems to be: get lucky and have your competitor just happen to be on a trip the day IBM knocks at their door. The lesson seems to be: have a parent who sells your stuff direct the board. The lesson seems to be: take advantage of decades of now-non-existant anti-trust atmosphere to make the world's biggest computer company seek outside OS. DOS itself? I struggle to think of anything remarkable at all. Maybe the availability of very cheap BASIC on-ramps for enthusiasts.
The business computing world still, to this day, largely runs on Windows, and Windows NT was built on the foundations of DOS: it bootstraps from a DOS filesystem, as UEFI still does in 2024, and it could be installed from DOS. It implements an API designed on DOS for a DOS GUI and to this day supports DOS-compatible filenames.
All the core system folders in Windows 11 still have DOS-compatible names, from `SYSTEM32` to `SYSWOW64`.
DOS itself was emulated by DR-DOS, FreeDOS, PTS-DOS, and other OSes.
> it's a race to the bottom on price.
Always was, still is. Why do you think Linux does so well? It's not technical merit!
> have your competitor just happen to be on a trip the day IBM knocks at their door.
Absolutely cast-iron lie, and you should be ashamed of yourself for repeating it.
AFAIK Windows NT was mainly influenced by VMS (which Dave Cutler worked on before NT). The DOS-isms were mainly coming in via the Win95 side and for backward compatibility reasons, but I bet everybody on the NT team hated those requirements ;)
> Absolutely cast-iron lie, and you should be ashamed of yourself for repeating it.
Not the parent, but it's at best a good urban legend and not much different from "Gary Kildall was not interested". Do you have any first-person accounts that paint a different picture?
Normal windows apps doesn't have a full POSIX subsystem running under them. The libc signal() call is a wrapper around structured exceptions. It is limited to only a couple well-known signals. MSVCRT does a bunch of stuff to provide a emulation for Unix-style C programs: https://learn.microsoft.com/en-us/cpp/c-runtime-library/refe...
In contrast to Unix signals, structured exceptions can give you quite a bit more information about what exactly happened like the process state, register context etc. You can set the handler to be called before or after the OS stack unwinding happens.
I have obviously read this up before and just didn't remember :-(
So the whole part of your message about "even the FSF saying that free software should run on proprietary system" works when you want to criticize Hare, but not when looking at Apple proprietary software, right?
A language is just another piece of software, I don't see why you should apply different rules to a programming language than, e.g. to a serializing system like Protobuf. And I don't think Google actively supports swift-protobuf (https://github.com/apple/swift-protobuf).
Hare upstream just says "we are not interested in supporting non-free OSes, but we won't prevent you from doing it". It's your choice to not use Hare because of this, but it's their choice to not support macOS.
> That doesn't make this specific policy a good idea.
Additionally one should get the TCP/IP and UNIX streams books from the same collection.
Although, I did a mistake, I was thinking about all Richard Stevens books for networking, that go beyond plain TCP, UDP, IP.
https://en.wikipedia.org/wiki/W._Richard_Stevens
Unfortunelly given their CS focus, they are kind of on the expensive side, I read most of them via libraries, or eventually getting my own copies.
Some things (“imprecise traps”) sometimes blur the difference between the two categories, but they usually admit little in the way of useful handling. (“Some of the code that’s been executing somewhere around this point caused a bus error, now figure out what to do about it.”)
It had three parents: OS/2, DOS and VMS. However, MS could use code from 2 of them but not from VMS. I've blogged about this more than once:
https://liam-on-linux.livejournal.com/67492.html
https://liam-on-linux.livejournal.com/54464.html
> The DOS-isms were mainly coming in via the Win95 side
Nope. Not true, and you have the timeline backwards.
NT was released in 1993, 2 years before Win95, and only the 2nd version of NT, 3.5, supported VFAT long file names.
NT did not support Win95B's FAT32 until its 5th release, Windows 2000.
> backward compatibility reasons, but I bet everybody on the NT team hated those requirements
No, I don't think so. NT could be installed on top of DOS, via the WINNT.EXE setup program. (Something I urged in OS/2 communities, but they didn't understand the need or usefulness.)
https://networkencyclopedia.com/winnt-exe/
NT could dual-boot with DOS, even in the same partition in early versions. It could also dual-boot with Win9x.
This level of interop was hugely important and useful and really helped the new OS gain adoption. It was not some reluctant bolt-on.
> a good urban legend
No, it isn't. It's a horrid calumny against a good and brilliant man.
> not much different from "Gary Kildall was not interested".
Also utter nonsense.
> Do you have any first-person accounts that paint a different picture?
TL;DR version.
Dr Kildall's wife, Barbara McEwen, was DR's lawyer. She negotiated with clients and suppliers, not the CEO who was a programmer.
IBM wanted an NDA which DR was unwilling to do. She said no. Remember DR was the industry giant in microcomputer OSes at this time, and IBM didn't have an offering at all.
Kildall was flying to visit an important client; this wasn't some accidental joyride.
This lie about Kildall literally drove him to drink and his early death. Stop repeating it. It's not funny or clever. It's an evil, vindictive lie.
Tom Rolander was the other passenger in the plane. Is his testimony good enough?
Listen to him describe the flight he was on.
https://youtu.be/bLVbSjDq0DE?si=Ig9KksWWiJG3KDFn&t=1025
A much longer interview:
https://www.mercurynews.com/2008/12/18/cassidy-theres-more-t...
Video interview:
https://www.youtube.com/watch?v=VREZ6Zx_usc
Transcript:
http://archive.computerhistory.org/resources/access/text/201...
The thing is, the computer industry is now old enough it has a lot of folklore and legend: stuff that "everyone knows" and repeats.
But many of the people involved are still alive and you can just ask them.
And there are some really nasty people in this industry -- such as Bill Gates, or Larry Ellison -- who tell lies about others and to others, and then some of those lies catch on and everyone repeats them.
These lies that people share destroy lives. Don't repeat stuff you heard. Just Google it. It's easy to find the truth.