The Rule of Silence (2006)(linfo.org) |
The Rule of Silence (2006)(linfo.org) |
Even if nothing else, this rule/principle/whatever is my favourite by name :)
https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
- Contrary to popular belief, Unix is user friendly. It just happens to be very selective about who it decides to make friends with.
If structured data was embraced we would have developed appropriate tooling to interact with it in the way that we prefer.
This runs very deep in unix and a lot of people are too "brainwashed" to think of other ways. Instead they develop other exotic ways of dealing with the problem.
Oh you don't like that output? Easy! pipe that crap into sed then awk then perl then cut then wc and you're golden!
When you get tot that point you have to understand that you have chosen to ignore the fact that the data you are dealing with must be represented in something much closer to a relational database than lines of ASCII text.
Logging is another area you see the consequences of this. A log is not a line of text. Repeat after me a log entry is not a damn line of text.
"Oh but isn't it neat you can pipe it to grep?" NO! No it's not neat, maybe it was neat 20 years ago. Today I want that damn data in a structure. Then you can still print it out in one line and pipe it to grep all you want.
Another area that you see the unfortunate side effects of this philosophy is with the mess of file-based software configuration.
Yes I get it, you like your SSH session and Emacs/Vim blah blah but that's short-sighted.
I want my software configuration stored in a database not in a bunch of fragile files with made up syntax that are always one typo or syntax error away from being potentially silently ignored.
The fetish for easily-editable ASCII files and escaping from structure is holding us back. Structured data does not automatically imply hidden and inaccessible, that's a matter of developing appropriate tooling.
All settings in a database and not text files (the registry); a command-line that pipes data, not text (PowerShell). Tailored UIs to change settings, not magic invocations and obsure text file syntaxes.
I guess most developers on HN are also aware of the downsides of this philosophy. If not, try configuring IIS.
Imagine if the default wasn't bash, but something like Ruby + pipes (or some other terse language).
What is the argument for shell scripts not working on typed objects? How much time has been lost, how many bugs have been created because every single interaction between shell scripts has to include its own parser. How many versions of " get file created timestamp from ls" do we need?
Something Windows does get right is the clipboard. You place a thing on the clipboard, and when you paste, the receiving program can decide on the best representation. This is why copy-pasting images has worked so magically.
I could see an alternative system where such a mechanism exists for shell programs.
Microsoft forgot (and still forget) the documentation for Windows - if you want a nice A-Z reference for the bootloader, or kernel, or shell or IIS's configuration file or half the command-line tools, you're usually out of luck. The official place is often an inaccessible, badly-written knowledgebase article written from a task-based, usually GUI-based perspective.
It never mattered that they'd implemented a more coherent system full of better ideas, because the only way they'd tell you about it is through the GUI.
The MSDN CDs from 20 years ago were really good for a complete programmer's reference, but 1) I'm not sure how well they kept that up and 2) I could never find anything as comprehensive for sysadmins.
Every time I come around a German installation of IIS or SQL Server I'm cringing. Googling the right solution and then trying to figure out how they've translated this option is something I can't stand.
Maybe the downsides are because of the execution?
PowerShell' object-oriented nature is great btw!
The amount of tooling that surrounds text is vast and has evolved over decades. You cannot replace that with a single database and call it better.
I can place the majority of my config files in git and version them. I can easily perform a full text search on any log regardless of its syntax. I can effortlessly relocate and distribute logs to different filesystems.
> I want my software configuration stored in a database not in a bunch of fragile files with made up syntax that are always one typo or syntax error away from being potentially silently ignored.
So you would like your configuration stored as a bunch of made up data structures? Databases do not make you immune from typos and syntax errors, ask anyone who has ever written a delete without a where clause.
And what happens when the giant, all-knowing database your system depends on has a bug or vulnerability? When something on my linux box breaks I can drop to a recovery mode or lower runlevel and fix it with a kernel and a shell as dependencies.
I think you would be a lot happier with a configuration management system (puppet, ansible et al) and some log infrastructure without having to completely redo the unix philosophy and the years of experience and fine-tuning that comes with it.
Or the amount of cruft.
> git
Any structured data can still be serialized and diff'd, but it isn't always the clearest. Where is the contrast here?
> made up data structures?
so standardize the non-text format
> Databases do not make you immune from typos
Depends on the constraints. There are few on text files, possibly excluding sudoers.
If you aren't sticking to good practice you can just as easily rm a text file.
> has a bug or vulnerability?
What happens when the kernel has a bug or vulnerability? There are quite a few mature db systems. Plus, all text files depend on the file-system, which is why you store root on something stable like ext (still depends on hdd drivers though, unless you have some some ramfs on boot).
> years of experience and fine-tuning
Can you describe specifically what that experience is, and what the "fine-tuning" is?
The fetish for easily-editable ASCII files and escaping
from structure is holding us back. Structured data does not
automatically imply hidden and inaccessible, that's a
matter of developing appropriate tooling.
In short, your configuration database scheme is like an executable, whereas text config files are like source code (literally, in some cases). I'd much rather work with source code, as it remains human readable while at the same time amenable to a variety of text manipulation tools. Databases and APIs are more difficult to worth with, especially from the perspective of enterprise configuration management.
Edit: See also http://catb.org/jargon/html/S/SMOP.html.
What kind of tooling might work with ad-hoc structured data and still getting all the tools to talk with each other like in Unix? How would it work without having to write input parsing rules, data processing rules, and output formatting/filtering/composing rules for each tool?
I suspect that the reason it's not very popular to pass around structured data is that it's damn difficult to make various tools understand arbitrary data streams. Conversely, the power of text is that the tools understand text, only text, and do not understand the context, i.e. the user decides what the text means and how it needs to be processed. Then the tools become generic and the user can apply them in any number of contexts.
There's an awful lot of JSON that gets passed around. That seems a reasonable compromise between readable text and some sort of structure.
How so? I'm one of those people that like my SSH session, and in my case vim and "blah blah blah". I've contributed to countless open source software packages that you likely use with this method, and so have tons of other developers. Nothing is broken here, things are working great for everyone who reads the manual and follows it.
> I want my software configuration stored in a database not in a bunch of fragile files with made up syntax that are always one typo or syntax error away from being potentially silently ignored.
apache, postfix, haproxy, even vim are certainly not prone to silently ignore anything, just to name a few.
> The fetish for easily-editable ASCII files and escaping from structure is holding us back.
Holding us back from what?
I am both a developer and an administrator, and I've had all the fun with solaris/aix configurations that are often not stored in plain text that I care to have. If you also have this experience, and still feel the way you do, then I'd love to hear more. Otherwise, your rant comes off as "your way is hard, and I don't want to learn it!"
Look at all the available structures available for that plain text you speak of... XML, JSON, YAML, the list goes on. You are free to use one of those, then you have that structure you crave. There are plenty of areas that could use revolution, but UNIX-like configuration files are not one of them. There is no problem here. If you are making typos or mis-configuring your software, then you have a problem of your own creation.
As I mentioned in another comment in my view the problem is that something like the configuration is shared both by the humans and the computers. Because of this we settle on something that is not optimal for either group.
We end up with something that is hostile to both the humans and the computers, just in different ways.
In fact the argument of people like you for ASCII config files exactly demonstrates my point. You are fighting for your human-convenience against the machines.
> Holding us back from what?
By embracing and acknowledging that the humans and computers are not meant to share a language we free ourselves from this push-pull tension between human vs machine convenience.
We can develop formats and tooling that respects its human audience, that doesn't punish the human for making small superficial syntax or typo errors and so on.
And we can finally step the hell out of the way of computers and let them use what is suitable for them.
And at that point you could still have your SSH session and Vim/Emacs and blah blah blah and you could still view and interact with stuff as plaintext if you wanted to.
> apache, postfix, haproxy, even vim are certainly not prone to silently ignore anything, just to name a few.
It's not always a matter of silently ignoring something but due to the nature of the task it is certainly very easy to shoot yourself in the foot doing something that isn't technically an error but wasn't your intention.
For example you can silently break your cron jobs by leaving windows-newlines in them.
Perfect example of humans and computers sharing a language that is hostile to the human.
BAD BAD human! You stupid human why do you use bad Windows invisible characters? Use good linux invisible characters instead that are more tasty for your almighty lord Linux.
The systemd journal works how you describe, and it is very painful to interact with. I'll take plaintext logfiles any day of the week.
It's fine if you want to interact with the log in ways that have been designed into it. But:
- it's harder to work out what you can delete to free up space in an emergency
- it's harder to get logrotate to do what you want
- it's harder to use "tac" to search through the log from the bottom up
> I want my software configuration stored in a database
So now you can't put comments in your config, you can't (as easily) deploy config with puppet, or in RPMs. You can't easily diff separate configs.
All the things that you mention can in theory be fixed over time.
The stuff I'm talking about is not for the next 6 months. It's not very meaningful to compare it against the current tools and landscape.
I can almost imagine a similar conversation in the past.
Someone saying "MAYBE ONE DAY WE CAN FLY!" and everyone's like "BUT OUR HORSES CAN ONLY JUMP 2 METERS HIGH! It would never work."
I understand your comment from a pragmatic point of view but none of those problems are big or important enough that we couldn't fix them in other ways.
Throwing away a rich structured piece of data and trading that for a dumb line of characters that needs to be re-parsed just so that it's easier to use logrotate and tac with them and so on is a losing trade.
Reminding me of one of the little niceties of Gobolinux.
You have a pristine pr version config copy sitting in the main dir, and a package-wide settings dir. It also provides a command (implemented as a shell script, as is most of Gobolinux tools) that gets run upon installing a new package version.
If said command detects a difference between existing and new config files it gives you various options. You can have it retain all the old files, replace them with the new files, or even bring up a merged file that give you the new lines as comments next to the old ones.
You are forgetting a crucial point: plain text is very well defined. Actually, it was already defined when the first Unix tools were being written. Using plain text means that you can use grep to search the logs of your program, even if your program was written yesterday and grep was written 40 years ago.
Structured data? In which format? Who will define the UNIQUE format to be used from now on for every tool? The same people who chose Javascript as the web programming language?
Do you realize that choosing plain text prevented any IE6-like monstrosity from happening?
Everyone can just boot into emacs :P.
UNIX philosophy is, as written in the article, is based on programs that do one thing well and work with other programs. The second part, "work with other programs" is the one that encourages (but not requires) simple, text based I/O.
If A, B and C write programs and independently design some custom, structured, binary I/O the chance of them being compatible is nil. If they output text, the UNIX glue of pipes and text conversions makes them cooperate quickly and efficiently. Not elegant? Sure. But working well in no time.
I guess our industry is meant to run in circles, only changing the type of brackets on each loop (from parens to curly on this iteration).
It's not always the best way to approach a problem, but it's not meant to be. It's duct tape. You use it where it's good enough.
People ran away screaming from it :)
I want to be able to look at your file format using tools that haven't been specialized to the task. Is that so wrong?
Second, you are right about structured data and all. The only thing is that it's either impossible or extremely hard to achieve. Many have tried, all of them failed. Windows now has a mix of registry, file and database configs which is a nightmare and is much worse than any unix. AIX has smitty and other config management solutions which are a bitch to work with if you want something non-trivial. Solaris is heading this direction (actually it's heading to the grave but it's another story) and it's also not nice. There are a lot of other OSes and programs which tried to do it but failed.
This is much like with democracy: it's a terrible form of ruling, too bad we have nothing better. This is exactly what's up with unix configs and data formats. It is possible to make some fancy format and tools which will achieve it's goal for like 80% of the time. But it will cause huge amount of pain in the ass in the rest 20% and this is where it will be ignored and you'll end up with a mix of two words which is worse than one.
I remember getting fairly excited when Apple OSX first came out and quite a few of the configuration files were XML-based. Finally, a consistent format, but it wasn't pervasive enough. Even Apple couldn't see fit to break with the past.
I've even contemplated rewriting some core utils as an experiment to spit out XML (because I didn't know about typed objects at the time), but I lack the skillset.
I know we can't (and maybe we shouldn't) change Unix and its derivatives. There's too much invested in the way things work to pull the rug out. But, when a new OS comes along that wants to do something interesting, I hope the authors will take a look at designing the interface between programs rather than just spitting out or consuming whatever semi-structured ball of text that felt right at the time.
Wouldn't it be neat, for example, if instead of 'ls' spitting out lines of text, which sometimes fit the terminal window, sometimes not, which contain a date field which is in the local user's locale format, which is in a certain column which is dependent on the switches passed to the command, instead you get structured, typed information, ISO-formatted date and time, etc. On the presentation side, you can make it look like any old listing from ls if you like, rather than mashing the data and presentation layer together. I'd like to imagine such a system would be more robust than one where we could never change the column the date was in for fear of breaking a billion scripts.
It's the shortest path from thinking "I need to persist this crap" to getting something working. Write the bytes to a file, sprinkle some separators, read and parse it back.
This is true. But when your needs aren't that complex, basic textual output sure is nice.
> The fetish for easily-editable ASCII files and escaping from structure is holding us back. Structured data does not automatically imply hidden and inaccessible, that's a matter of developing appropriate tooling.
Good plan. I'll set up a schema by which people can exchange data, and wse'll get it standardized. Given the complexity of the relationships involved - and the fact that I really don't know how my data will be used downstream of me - I'd better make it something really robust and extensible. Maybe some kind of markup language?
Then we can ensure that everyone follows the same standard. We can write a compatibility layer that wraps the existing text-only commands and transforms the data into this new extensible markup language (what to call it thought? MLX?). Then anyone who has the basic text tools can download the wrappers, learn the schema , and start processing output.
Then again, I could just do that grep | cut. The only thing I have to learn is the shape of the thing I'm grepping for, and the way to use cut - the basics take a few seconds, and no additional tooling is required. Best of all, chances are high that it'll work the same way 20 years from now (though likely with expanded options not current available).
There's a lot to be said for having simple tools that accept simple input and produce simple output.
This doesn't mean it's the only approach - databases and structured data absolutely have their places in modern CLI tooling - but that has no bearing on the value of an ASCII pipeline.
Pipes can transfer arbitrary data, so it's just the tools that you don't like, not the underlying mechanism.
I totally agree that Unix sux. We need a better philosophy. But you eventually have to come up with something that actually works in practice. I am still waiting.
I remember my excitement at the idea that things like CP/M and MSDOS running on personal computers were going to free us all from the tyranny of mainframe computers running things like Unix. We all know how that turned out. Everyone eventually just gave up and started emulating the top of a desk.
So Unix is good at messing with unstructured text? Good. Get back to me when you have something better that actually works.
Now, it's true that we should recognize where text is really inadequate, especially when indexing and searching is needed. Webpages, for example, should not be plain text.
I think the problem resides in programmers not being able to properly use and understand how a database works. Databases and their engines are black boxes, so it's normal if fewer developers want to to use DB like you say. Meanwhile dictionaries and B-tree are not very sophisticated algorithms, yet I see almost no programmers using them consciously. The less a programmer know about the tools he has in his hands, the less he will get benefits from it and thus he will start using easier things.
So really my thought is that the tools are not accessible enough. The concepts of file and database are so distant that it's completely impossible to work with both, but to me it should.
It may be that given proper tooling for database-driven configuration it could be visible and accessible, but the fact is, I haven't seen timing that pulled that off.
You missed the whole point about signal vs noise.
When I'm ALWAYS presented with a blob of something to decipher, it requires a context switch.
Nothing IS something, and it's a structured something.
I think the rule makes sense within the specific constraints *nix programs are usually expected to work in (two output channels with no structure except the one informally defined by the program and the convention that the output should be human- and machine-readable at the same time) but I don't see it as a general rule if better ways to filter the output are available.
This is exactly my experience, too.
Some programs get this even worse: They spam you with lots of useless information, yet when something goes wrong, you don't get the information you need. Instead, you have to rerun it to increase the verbosity even more.
So ... if I have to rerun and having a hard time trying to reproduce the issue anyway, why did it spam me in the first place?
As an example: I love curl for piping the data to stdout per default, but I'm frequently annoyed by the progress bars I didn't ask for, especially if a script involves multiple curl commands.
FFmpeg is also quite bad here, but at least you can use -hide_banner and/or -loglevel to alias the problem away and mostly forget about it.
At its core, it is about putting humans before computers. Engelbart coined HCI as Human Computer Interface, not CHI. This philosophy steered my product designs ever since I read that as a teenager.
Granted, with too many options it could quickly get confusing (should this message go to stdout or stdinfo; is that message more informational or more debugging?), but I think that it could be managed.
Similarly, I think that Unix fell down by relying too much on unstructured text (in the sense that the structure isn't enforced, not in the sense that it's altogether absent): because of this, every single tool rolls its own format, and even very similar formats may have subtle incompatibilities.
I'd love to see a successor OS which builds on the lessons of Unix, Plan 9 and other conceptually brilliant OSes, but I fear the world will never see another successful operating system.
Going back to the man page point: I will granted you that not everyone does check what a program does before running it. Sadly in those instances there's little you can do to protect them from themselves. It's similar to how you cannot protect people from blindly copying and pasting code from the internet. If someone is willing to run a command "blind" then the usefulness of the output is the least of their worries.
They could at least link to tAoUP[1].
[1] http://www.catb.org/esr/writings/taoup/html/ch01s06.html
User kagamine has read you comment.
User kagamine has processed your comment.
User kamamine came up with a barely witty response.
User kagamine has included the barely witty response for your amusement and irritation.
User kagamine hopes to not be banned for this trivial response.
Follow me on TwitFace for more like this and pictures of food.
This is the world we live in, the Rule of Silence seems like a unique and valuable asset to Unix in this day and age.
Acks are important.
It's not infuriating, it's just different.
http://www.catb.org/~esr/writings/taoup/html/ch01s06.html#id...
However the business and product developer in me wonders how I apply this to building more complex systems. Normally this involves building multiple functionalities. Does the philosophy say I shouldn't build "systems" that are complex and do multiple things? Or does it talk about how these should be implemented, as co-operating processes?
Reality: no unit exists outside the context of it's ecosystem.
Ergo, the API to that unit, and the degree to which it integrates with the other pieces is paramount.
The 'do one thing and do it well' ideal implies almost a kind of 'unit sovereignty' that in many cases does not exist.
In large, complex systems, 'units' can only view as parts of a greater whole.
'What they do' is almost less important than 'how well they fit'.
http://xahlee.info/UnixResource_dir/_fastfood_dir/fastfood.h...
I hadn't heard of it until now. Thanks for the good read!
Take an example. Build small components that can be reused. Its like SOA way before SOA came into use and makes perfect sense. Now criticizing that is much more difficult and will require technical depth than just dimissive comments about 'unix philosophy'.
In this case this is the first I've heard of a 'philosophy' of silence and it is often not golden. From a technical perspective its important for users to get feedback and not generic unhelpful error messages or commands disappearing. Fortunately on Linux logging is usually quite good and most experienced users can pinpoint errors quite quickly but options like -v, --v, -vv, -vvv far from helping often increase technical load.
Generally I think engineers need to fight the temptation to show off the importance and complexity of their software by spitting out all the unnecessary details and logs.
* good: easy to parse the result, easy to chain.
* bad: no progress report, annoying with long duration commands.
You expect it to finish at any time. It continuing is the surprise. A marker for its progression is thus, IMO well within the range of respecting this philosophy.
I suspect the reason is that for most signals, kill can't determine if the signal was acted upon or not. But for KILL and TERM it could wait a few milliseconds and then print if the process is still alive or not.
Edit: those who are saying that I can write a wrapper script is missing the point. The point of computers is to be useful to their users, not to follow some philosophy people invented almost 50 years ago. Like if someone is bothered by kill showing messages, they can write a wrapper script (kill > /dev/null how hard is that?) or beg the developers to add a -q option to kill (like grep has) or write a new tool for sending signals.
Also, the program is called KILL so one could be forgiven for assuming it's main purpose is to KILL things...
Well, powershell did solve that problem...
Of course some websites had to do it in an even dumber way than the law asks for. Like slashdot: http://i.imgur.com/5Fp0nmo.png
This is what greets the French every time slashdot decides to forget you agreed to let them put cookies on your computer and you need to click continue before you can get to the actual website.
The law actually made it worse for the people it's supposed to protect (those who might refuse cookies for privacy?) because those warning then will stick around like glue if they can't give you a cookie to remember you accepted their existence.
It would have been nice if this was permitted to be violated only by GUIs. Ask about the first-time *nix experiences before the GUI-embracing era and one of the few things they were noticing was the continuous text-spitting. For instance, that was happening on boot and OS loading sequence (and still happens, now only being hidden by default with splash-screens), a lot of reporting about all the things that were performed successfully. It's funny in this regard seeing Unix' Rule of Silence being respected more... outside Unix, where is just common sense, with no need to be formulated as rule.
To be fair, this has been fixed a long time ago. At least Vim (which is the Vi installed on most systems) shows the following message on startup:
~ VIM - Vi IMproved
~
~ version 7.4.1829
~ by Bram Moolenaar et al.
~ [...]
~ Vim is open source and freely distributable
~
~ Help poor children in Uganda!
~ type :help iccf<Enter> for information
~
~ type :q<Enter> to exit
~ type :help<Enter> or <F1> for on-line help
~ type :help version7<Enter> for version info
$ man foo
*scroll to the end with the EXAMPLES section*
Wow, I haven't seen such a blunt and unhelpful RFTM comment for a while. This comment is inappropriate in so many ways:
1) The unix systems have an inconsistent documentation mix of man pages, info pages, "-h", "-help", "--help", HTML docs, separate manuals (e.g. Debian Administrator's Handbook) and so on.
2) "man foo" leads to: "No manual entry for foo"
3) "man vi", as well as "man vim" both lead to a manpage that has no EXAMPLES section at all (see https://www.freebsd.org/cgi/man.cgi?query=vi, https://www.freebsd.org/cgi/man.cgi?query=vim)
4) The Vi(m) manpages explain only the command line arguments, not the editor commands. The latter are available by typing ":help" in the editor.
eg(){
MAN_KEEP_FORMATTING=1 man "$@" 2>/dev/null \
| sed --quiet --expression='/^E\(\x08.\)X\(\x08.\)\?A\(\x08.\)\?M\(\x08.\)\?P\(\x08.\)\?L\(\x08.\)\?E/{:a;p;n;/^[^ ]/q;ba}' \
| ${MANPAGER:-${PAGER:-pager -s}}
}
Usage:
$ eg tar
EXAMPLES
Create archive.tar from files foo and bar.
tar -cf archive.tar foo bar
List all files in archive.tar verbosely.
tar -tvf archive.tar
Extract all files from archive.tar.
tar -xf archive.tar
$The interesting thing here is that in order to make this argument people forget that at least some new users know the "Press F1 for help" dictum, because that particular part of Common User Access was drummed into them from the start of their encounter with computers. The new users press F1 in vim (not vi) and how to exit is the second and third items on the screen.
(Press F1 in actual vi, and, in some terminals at least, it very informatively inserts the letter "P" into the document. (-:)
I am getting really comfortable with unix and i can say that the transition on a mac is not bad! I started using it with only rudimentary knowledge about how to navigate the shell, but you really don't need it for most parts. When you start digging deeper you explore more and more of the unix-backend until it's like a second face to the computer. SSHing into a server is no inconvenience anymore etc.
I think this is the reason for developing simple, stupid GUI programs that help beginner do beginner stuff. They will (sooner or later) obtain knowledge of the terminal, but i think it's critical that the first steps are not too challenging.
Note: i am speaking about developers. I don't think non-developers need to know how to navigate the terminal. It just doesn't provide any value for them.
explains how to exit vi.
This is what made Unix last. Text and keyboards are the universal computing interface that has survived since the 1970s.
There's no particular reason why /etc/passwd couldn't be:
((root nil 0 0 root /root /bin/bash)
(daemon nil 1 1 daemon /usr/sbin /usr/sbin/nologin) …)
An unfortunate turn of phrase. It's ambiguous. Do you mean last as in "endure"? Or do you mean last as in "last place"?
The worse thing about the unix philisophy implemented as a unix shell environment is that programs (often) have only one interface, and that's used both for interactive use and as a programming API. This means, for example, that when we realize git version N has terrible default behavior given some arugments, we can't fix the behavior of those arguments t in git version N+1 because we would break it's API.
And yes of course - structured in/out, sane encoding handling etc. is just missing.
Unix makes it trivial to write a wrapper script around `kill` that does exactly what you want -- that's the entire point.
No, it is exactly the point.
Unix gives you Lego pieces to build things from.
A bunch of small Lego pieces you can join however you like is way more useful than a giant chunk of Lego that someone has glued together into whatever lump they happened to need that day.
When you ask why your Lego bucket didn't ship with pieces preglued into exactly the combination you want today, people absolutely will reply saying you have all the pieces you need and you can just join them yourself.
Because that is the entire point of Lego.
If that wasn't what you wanted, perhaps consider a different kind of toy?
That way, everybody wins. Both those who don't like Lego playing (me) and those who do (you).
> If that wasn't what you wanted, perhaps consider a different kind of toy?
I'd switch from GNU/Linux to something with a more sane design in a heartbeat! Unfortunately for me, the only free operating system began life as a Unix-clone. Thanks to an historical artifact more than a testament to the greatness of the "Unix philosophy".
Fortunately for me, there are a lot of people who have understood that thinking of an operating system as a jumbled bag of Lego bricks doesn't lead to good system design and are doing something about it (systemd and many more projects).
By the way, I haven't down-voted you, but I think I understand why others did. Before calling stupid a pillar of a (programming) philosophy with decades of useful outcomes, you should stop and ask yourself whether it's just a matter of taste on your side or, even more likely, of ignorance.
It wouldn't make sense:
1) the TERM signal can be trapped by the process, and "kill" has no right to assume that the process is supposed to quit;
2) the KILL signal is managed by the kernel and it just works (if it doesn't, then your kernel is buggy and you have more serious problems); even if "ps" shows the process as still alive after a "kill -9", you can assume it'd dead process walking.
I think this is the source of the problem. kill really is just for sending signals.
In the "Unix philosophy" there's no standard way to end processes, normally you send it SIGTERM and hope that it catches that, cleans up and exits. If it doesn't it could be a bug in the program, or maybe the program uses SIGTERM with slightly different semantics (eg Celery will wait for its children's tasks to finish, and does the "normal thing" of exiting as fast as possible when it gets two SIGTERMS); in any case I think the idea is that it's a weird, non-standard situation and the user needs to explicitly send SIGKILL (kill -9), since doing so might be dangerous.
This is the essence of 'small, single purpose programs that work well with each other' rule.
Windows Server 2012 Command-line reference:
* https://technet.microsoft.com/en-gb/library/cc754340(v=ws.11...
Windows XP A-Z Command-line reference, that is actually named that:
* https://technet.microsoft.com/en-gb/library/bb490890.aspx
Doco for the wdsutil and md commands, picked at random:
* https://technet.microsoft.com/en-gb/library/cc771206(v=ws.11...
* https://technet.microsoft.com/en-gb/library/cc754711(v=ws.11...
bcdedit command references:
* https://technet.microsoft.com/en-gb/library/cc731662(v=ws.11...
* https://msdn.microsoft.com/en-gb/library/ff542205(VS.85).asp...
And so forth. There's a huge amount of doco, including reference doco.
[0] Storing it server-side, per IP address, is obviously impractical.
The boot-up splash screen hardly even counts as "GUI" anyway.
I agree with you here, it's easy to write perfectly valid configurations that don't do what you intended. But throwing it all out seems like the baby going out with the bathwater to me.
In all seriousness, what if it was XML all around? I hate writing XML by hand, but that's part of the problem you are describing(ie human editing of raw config). XML parses very nicely, so difficulty of coding tools to speak XML is almost non-existent.
All in all, it's just a big change you are proposing. Us UNIX people are incredibly change-averse. We keep getting burned :)
Well that's exactly my point. I don't propose that we deploy it tomorrow but I want to see people do it and think about it and talk about it so our grand children can have better computing. Instead of sticking to the way of our fathers for all eternity.
XML as you identified has the same problem.
You need a human-dedicated interface that captures the intent of the human and you then convert that to something that the computer likes and tell them here ... this is what the human has ordered.
When we share the same raw input file with the computers that's when we set ourselves up for trouble.
Do I have the ultimate solution? No. Can I still complain about it being a problem? Yes.
One "solution" is a graphical rich interface with things like auto completion and validation and all that so it's much more capable of capturing the true intent of the humans. Basically the same way we interact with other web sites.
Imagine your bank told you to append a new text line to the end of "transactions.txt" file if you wanted to transfer money.
Now I put solutions in quotation marks because I know a GUI has other practical limitations and problems but you get the idea.
My point is as humans eventually we have to learn to graduate from sharing a rudimentary text language with the computer for the sake of short-term convenience.
Would it not be easier to keep this format, and then add some integration into your favorite text editor that prevents you from accidentally editing the INI keys, or warns if you create a syntax error, etc? You could even make a generic INI gui editor that replaces appropriate values with checkboxes, sliders, etc.
I guess I don't see much wrong with the formats for most config files, but if you insist that there is a problem, and the problems you've outlined are caused only when editing them, then why not change the editor and preserve the format?
(That cron problem does sound bad though)
You run a command, look at the output, now you know what the next command in the pipeline will see, and can add adjustments as needed.
Powershell etc seems to be more programmer oriented in that one keep thinking in terms of variables and structures that gets passed around.
And this seems to be the curse of recent years. More and more code is written by programmers for programmers. Likely because everyone has their head in the cloud farms and only the "front end" people has to deal with the users.
UNIX came when you risked having admins and users sitting terminal to terminal in the same university room. Thus things got made that allowed said users to deal with things on their own without having to nag the admins all the time.
In current, unstructured text reality, each program has to have its own (usually buggy, half-assed) shotgun parser, and it has to introduce semantic meaning back to the data all by itself. And then it destroys all that meaning by outputting its own data in unstructured text.
It works somewhat ok until some updated tool changes its output, or you try and move your script to a different *nix system.
I do like the idea of a standard parser to avoid the ad-hoc implementation of parsers into everything.
Your last comment gives a hint at the real problem, which is people using command line hackery in 'permanent' solutions. It's duct tape. You don't build important system components out of duct tape. Well, you shouldn't, anyway.
That's actually a great idea. Better yet, it's actually viable now, unlike many proposals for "fixing" unix.
Yes, basically. Someone in power should just pick any format - modified JSON (without the integers are really IEEE754 floats stupidity), s-expressions, whatever - and make standard deserializers part of the system library.
There's no authority to complain to about all this...
Structured text formats come and go: S-expressions, SGML, XML, JSON, "modified JSON", etc etc etc...
Fortunately the Unix tools work the same with any byte stream, pretty much, so they've survived gloriously since the 1970s.
That's kind of the whole deal with "worse is better."
Interestingly, it would only take the addition of a pdopen() to something like FreeBSD's pdkill() to get enough for a tool that opened a process descriptor, sent a signal to the process, and determined (with poll() and pdwait4()) whether the process then exited; with the open-descriptor-style interface guaranteeing that it was always referencing the same process.
If you've taken the time to understand why generations of professional have done that way, and if you you have something else to propose, than the discussion is welcome and the result will be interesting in every case.
Otherwise, it's just a waste of time.
Most people on HN are developers and not as fluent as system engineers in shells and UNIX tools. The need for minimalist, "silent" UIs comes with fluency. This is also true for network engineers and database administrators.
No, it comes from wanting to reuse the tool!
Would you like your browser to open plenty of pop-ups with messages like "Congratulations! The font XYZ was successfully loaded from C:\foo\bar" or "That's incredible! I looked for a JPEG on server www.qqq.com ... and it was there!"? No, you wouldn't. And I wouldn't like "mv" to babble every time it renames a file on the hard disk.
So what? Is desktop computing all that matters? Networking, scientific computing, mobile telephone, don't they matter?
Unix was never meant for looking magic to clueless users; it was meant for programmers, i.e. for people who use the computer and not just some programs written by other people. And, in its realm, it has been brilliant! Not perfect, but very good!
By the way, here's a link to learn something more: https://blogs.msdn.microsoft.com/wsl/
I.e. your "decades of useful outcomes" argument doesn't favor the point you are making because the opposite philosophy (a more verbose userland) has many times more "useful outcomes". Therefore, according to your own logic (which I don't subscribe to because I think popularity is a useless measure of sound design) you "should stop and ask yourself whether it's just a matter of taste on your side or, even more likely, of ignorance."
... except that they could and did. People were not lacking in imagination. A science fiction writer imagined the Internet-connected smart 'phone in 1976, and put it into a science fiction novel (calling it the "mini-sec") titled Imperial Earth. A decade before that, an engineer imagined windows, hypertext, the mouse, word processing, graphics, and video conferencing, and gave what was later called The Mother of All Demos in an ACM/IEEE conference in 1968.
Jos Backus asked for the JSON option in 2013.
Programs that use the input and output to carry data around can not show prompts to the user.
Basically there is no universal structural language that's ergonomic and nice for all uses, so it makes sense that the Unix hackers of yore preferred to create tiny custom languages for everything. It's also because "worse is better".
This is wrong. If you just type "vim file", you don't get any usage info, not even in the status line. See also: https://news.ycombinator.com/item?id=13165795
pkill vi
I believe that the registry, in a way, caused PowerShell.
PowerShell works the way it does because Windows is structured data all the way down. A text-based shell a-la bash would not be very useful for Windows sysadmins. If you want to do Windows automation (e.g. on a cluster of windows servers or whatnot), you need to process and manipulate structured data (server settings, user permissions, AD groups, whatever). Hence, a shell and scripting language for doing just that.
If Microsoft hadn't moved from .ini files to the registry between Windows 3.1 and 95, I don't think PowerShell would've had the same design goals as it does now.
The keys 'mode', 'number of links', 'owner', 'group', 'size', 'last modified' would be repeated over and over again, stacked vertically.
Mode would remain an arbitrarily formatted string, or (worse) be broken into its own object for every attribute represented by the string.
A reasonably populated directory would fill multiple screens with cruft.
The formatting of the timestamp in the last modified field would still be an arbitrary string.
Comparing two files would require piping through another utility to extract just the appropriate fields and display them unadorned.
Sure, it might be moderately easier to consume in another program since you can simply iterate over a list and reference specific keys, but it's not really that hard to iterate over a list of lines and extract a specific field by number.
ls -al | awk '{print $5,$9}'
ls -al | jq '.[] | [.size,.name|tostring] | join(" ")'All right, maybe you do have a common tool that implements a query language so that you can filter out certain paths and objects from the JSON data into a thinner data set with known formatting expected by the next command in the pipeline. Then you need to write that command and you need more instructions, possibly again in another language, to describe what you actually want to do with the data now that you know where to find it.
At this point you typically write a separate script file to do this because it's easier to express in full-blown programming language what you want to do with the tree of hashes and lists and values. On the other hand, programs for lines of text are quite short and fit on the command line.
I don't see an immediate value in structured data, and especially none that would outweight loss in general applicability and usability in comparison to text based data processing.
Don't get me wrong: I would love to see a good prototype or sketch of how such a thing would work, and then try to imagine how I might be able to apply it to similar things for which I use Unix command line today. But I'm sceptical of "how" and also quite sceptical of "why".
> "What happens when the kernel has a bug or vulnerability? There are quite a few mature db systems. Plus, all text files depend on the file-system, which is why you store root on something stable like ext (still depends on hdd drivers though, unless you have some some ramfs on boot)."
I'm not sure I get your point. What do kernel bugs have to do with text vs binary formats? Or the argument for or against centralised databases? Databases still need to store files on persistent storage so if your text files are compromised from a kernel vulnerability or instability in the file system then your binary database files will also be.
> What do kernel bugs have to do with text vs binary formats?
you said "what if the db has a bug or vulnerability", my point is you have to rely on something, even the kernel. The difference is how stable these things are, and databases can be very stable.
> if your text files are compromised from a kernel vulnerability
not all kernel vulnerabilities will put the db at risk, it depends on the exposure to parts of the kernel. You can restrict the type and "fanciness" of the file-system a database will use if you know you don't need those additions, in the same way you use a stable fs for system files. You need a basic set of binaries one way or the other to access this data.
DBs still have files, they can be rm'ed. DBs also have other delete commands like 'DELETE FROM x'
My point is it's just as easy to "accidentally" delete data in a database as it is in text files.
> you said "what if the db has a bug or vulnerability", my point is you have to rely on something, even the kernel. The difference is how stable these things are, and databases can be very stable.
Someone else said that. I think the whole stability point is moot.
> not all kernel vulnerabilities will put the db at risk, it depends on the exposure to parts of the kernel. You can restrict the type and "fanciness" of the file-system a database will use if you know you don't need those additions, in the same way you use a stable fs for system files. You need a basic set of binaries one way or the other to access this data.
If a software vulnerability exposes text files like /etc/passwd then it can expose the database disk files in exactly the same way. Having a database format won't magically stop files from being read remotely.
It's also worth mentioning that most of the time it's not kernel vulnerabilities you need to be worried about (not that I'm saying they're not bad); any bug in software (e.g. Wordpress vulnerability) that allows an attacker to specify the source file to be read would put both your database config concept and the existing UNIX config layout at risk.
Whoever accepts patches to base UNIX system libraries would be a good start.
> Fortunately the Unix tools work the same with any byte stream, pretty much, so they've survived gloriously since the 1970s.
So would a structured text format, except you wouldn't be discarding and then recreating semantic information at every junction of a pipe.
That can be you! Get to work, or pay someone else to get to work if the skills required aren't in your set.
examples ()
{
man $1 | less +/^EXAMPLES
}
$ examples su
EXAMPLES
su -m man -c catman
Starts a shell as user man, and runs the command catman. You will
be asked for man's password unless your real UID is 0. Note that
the -m option is required since user “man” does not have a valid
shell by default. In this example, -c is passed to the shell of
the user “man”, and is not interpreted as an argument to su.
su -m man -c 'catman /usr/share/man /usr/local/man'
Same as above, but the target command consists of more than a
single word and hence is quoted for use with the -c option being
passed to the shell. (Most shells expect the argument to -c to be
a single word).
su -m -c staff man -c 'catman /usr/share/man /usr/local/man'
Same as above, but the target command is run with the resource
limits of the login class “staff”. Note: in this example, the
first -c option applies to su while the second is an argument to
the shell being invoked.
su -l foo
Simulate a login for user foo.
su - foo
Same as above.
su - Simulate a login for root.Underlining is the only formatting I care about and that works without MAN_KEEP_FORMATTING on FreeBSD.
I don't have colorization enabled in man pages on my laptop, and also I don't have bolding enabled either. I like it this way.
>your solution does not respect the user’s pager preference; the user might prefer to read man pages in “w3m”, for instance.
The pager preference of the user in this case is `less`. I know because the user happens to be myself :^)
Man pages are OK when you're first learning how to use something; but if you're already familiar with a command and just need to remind yourself of a the specific sequence of options to achieve a desired result, they're not the most convenient.
I think it's useful to have a tool that fulfills the latter purpose without worrying about the former.
The FreeBSD, TrueOS, and related worlds put the "using" doco into what are often called "handbooks" or "guides".
* NetBSD Guide: https://netbsd.org/docs/guide/en/
* FreeBSD Handbook: https://freebsd.org/doc/handbook/book.html
* DragonFlyBSD Handbook: https://www.dragonflybsd.org/docs/handbook/
* TrueOS User Guide: https://www.trueos.org/handbook/trueos.html
* PC-BSD User Guide: http://web.pcbsd.org/doc-archive/10.1.2/html/pcbsd.html (viewable off-line directly in both PDF and HTML forms in /usr/local/share/pcbsd/doc/)
Some parts of the Linux world do the same. upstart had the Upstart Cookbook for example:
* http://upstart.ubuntu.com/cookbook/
The Linux Documentation Project was supposed to contain a wealth of this stuff, but large parts of it are seemingly moribund, and incomplete after decades or woefully outdated. Wikibooks tried to take up the slack with an "anyone can edit" Guide to Unix and a Linux Guide:
* https://en.wikibooks.org/wiki/Guide_to_Unix
* https://en.wikibooks.org/wiki/Linux_Guide
If you want examples and doco that works from the basis of what you usually want to do, then these handbooks and guides are the places to go, not reference manuals.
User-submitted and voted-upon examples for commands.
(hadn't seen tldr, looks great, I'll check it out)
The canonical example of the complete failure and total friction is in the simple case of obtaining a file from a web server and sticking it on disk. This is the steps:
1. Try various built in cmdlets. Eventually find one that works.
2. It's a big file and gets entirely buffered in RAM and takes the machine out. You don't get the privilege of finding this out until you try dragging a 20GiB vhdx image from a build server to plonk on your SCVMM platform. That conks it completely.
3. So you think you'll be clever and just use curl. Oh no there's a curl alias!
4. Every damn machine gets sent a reconfig to remove this stupid alias.
5. Then when you do get there you find that the curl output pipe just pukes multibyte garbage out. 20GiB of it that took a long time to come across the wire.
6. You tell curl to save the file directly. Might as well have used cmd.exe by now!
So 8 Googlings and 3 hours later you got a fucking file. Every time I use powershell that is my life.
man iwr
man iwr -param outfile
Not to excuse bad examples on the internet (there are lots of people who fail to grasp PowerShell and still try to write articles and how-tos), but PowerShell having built-in documentation for commands and parameters is actually fairly easy to figure out from within the shell. Admittedly, once you learn the basic half dozen or so cmdlets you tend to use all the time.
You're free to ask me (or on SO, I tend to answer PowerShell questions there) if you're having trouble. Figuring above out took literally just 20 seconds. Really. I'm kinda sick of people cursing and blaming tools just because they're different from what they're used to. If I did the same and complained about bash and Unix (which I rarely find the need or time to learn) I'd be tarred and feathered in seconds ...
It's just that once you learn how to use a tool you don't have to think about how to approach a problem anymore, you just do it. And then, when learning a completely different tool you have to learn again. Surprise.
The problem with Powershell is that it tried to have its cake and eat it too - it wants the lightweight IDE of Unix-philosophy tools, but the detailed structure of C# objects. The problem is that the former is tightly tied to the simple common api of raw text, and the latter is inherently dependent on an intellisense-oriented IDE and static typing that heavily hints the names and parameters of useful actions.
Powershell manages to combine the worst of both worlds - objects means more complex APIs, but without the powerful IDE guiding you around those APIs.
It's also been really neat to see how well Powershell lends itself to extending other applications.
It actually is really good, but I do find it interesting that the first and last loser of the desktop wars got there a good long time ago.
[1] https://en.wikipedia.org/wiki/Workplace_Shell , https://en.wikipedia.org/wiki/Object_REXX
To clarify: The registry is a database for OS or application stuff (caches, settings, etc.). It's not meant to be user-editable and outside arcane trouble-shooting stuff you're unlikely to ever have to venture in there.
"Use the COMMENT statement to add a comment about a table, view, materialized view, or column into the data dictionary."
https://docs.oracle.com/cd/B19306_01/server.102/b14200/state...
https://www.postgresql.org/docs/current/static/sql-comment.h...
I think that's kind of a stretch since regedit has been a part of windows as long as I can remember.
... is exactly the reason to have useful comments in there.
Also, if you ^C in Vi, you get the message:
Type :quit<Enter> to exit VimRemember what the difference between $VISUAL and $EDITOR was intended to be. There's a whole range of places in Unix where there was, and even still is, a distinction between a line editor and a full screen editor. Consider, for just one example, the ~v and ~e commands in BSD Mail.
Now enjoy the discussion at https://news.ycombinator.com/item?id=13113556 .
It tends to annoy me, but it's easy enough to change :)
Though I do agree with your overarching point that some of the formats/outputs could do with a more consistent structure. Perhaps something like YAML would strike a good balance between structure and conciseness/readability...
git blame -L ${LINE},+1 --porcelain ${FILE} | sed -n '/^author / {s/^author //; p}'
(name (author (first (git blame file :line line))))
It's already error-prone (as anyone who's ever incorrectly edited /etc/passwd knows).
Ultimately, structured data (which is pretty much all data) should be edited with structure editors. Good text formats make it easy to write such structure editors.
That particular example is fairly straightforward (at a simple level, passwd files aren't complex), but being able to express arbitrary nested structure would make various things a lot more straightforward. Line breaks and some sort of tab/colon/what have you work fine if everything has at most two levels of hierarchy, but it starts being painful after that.
Missing matched parens are a bit of a specious argument, since many of the random formats for files are fairly strict about what they parse, and the ones that matter (e.g. passwd, sudoers, crontab) are conventionally edited through tools that check the syntax before committing.
Also, jq, jshon and similar tools are a thing.
----
Programmers tend to carry over the structure of the program as the structure for its documentation. But this structure is not necessarily good for explaining how to use the program; it may be irrelevant and confusing for a user.
Instead, the right way to structure documentation is according to the concepts and questions that a user will have in mind when reading it. This principle applies at every level, from the lowest (ordering sentences in a paragraph) to the highest (ordering of chapter topics within the manual). Sometimes this structure of ideas matches the structure of the implementation of the software being documented--but often they are different. An important part of learning to write good documentation is to learn to notice when you have unthinkingly structured the documentation like the implementation, stop yourself, and look for better alternatives.
[…]
In general, a GNU manual should serve both as tutorial and reference. It should be set up for convenient access to each topic through Info, and for reading straight through (appendixes aside). A GNU manual should give a good introduction to a beginner reading through from the start, and should also provide all the details that hackers want. […]
That is not as hard as it first sounds. Arrange each chapter as a logical breakdown of its topic, but order the sections, and write their text, so that reading the chapter straight through makes sense. Do likewise when structuring the book into chapters, and when structuring a section into paragraphs. The watchword is, at each point, address the most fundamental and important issue raised by the preceding text.
https://www.gnu.org/prep/standards/standards.html#GNU-Manual...
EDIT: Some comments here are slowly making me like the idea of a standard structured binary more and more.
curl -svo /dev/null http://example.com/file 2>&1 | grep Last-Modified | cut -d ' ' -f 3-
Wouldn't it be nice if your shell could do this?
curl(http://example.com/file).response_headers.Last-Modified.subtract(date().now).days
curl -j example.com|select headers.Last-Modified|time before now|time to days
At some point you have to admit that what's meant for the computer is not always byte by byte the same as what's meant for the human.
We try to shove these two together and we screw up both of them.
Empower the computer to be the best that it can be by taking the human out.
Empower and respect the human by giving him/her their own representation.
The "I just want to read the bytes that are on disk" philosophy is inherently limiting and broken when the audience are two very different things (humans vs computers).
My argument is that instead of fighting that we must embrace it.
I agree with you, the other extreme - treat binary files as sort of opaque black boxes that you can only access with specialized tools beloniging to your application - is even worse. But I don't see why we can't reach some middle ground: have a well-documented, open, binary format that encodes a very generic data structure (maybe a graph or a SQLite-style database) and a simple schema language that lets you annotate that data structure. Then you can develop generic tools to view/edit the binary format even though every application can use it in their own way.
What if pipes worked the same way? What if we added stdout++, stderr++, and stdin++, and when you write to stdout/err++, you can say which format you're writing to, and you can write as many formats as you like. And then you can query stdin++ for which formats are available, and read whichever you like. And if stdin++ is empty, you could even automatically offer it with a single "text" format, that is just stdin(legacy).
The appeal of the Unix text-based approach is a kind of "worse is better". It is so simple and easy, compared to Powershell. The clipboard idea seems like it has a similarly low barrier to entry, and is even kind of backwards-compatible. It seems like something you could add gradually to existing tools, which would solve the marketplace-like chicken-and-egg problem.
You could even start to add new bash syntax, e.g. `structify my.log || filter ip_addr || sort response_size`. (Too bad that `||` already means something else....) Someone should write a thesis about this! :-)
Let's say you're MS writing word for the Amiga.
They provide a datatypes description for doc files. this gives ability to read and write the format, and fingerprint it (not based on extension).
Now any program, old or new, that wants to read or write doc files can do. It's just there.
... at which point you basically have a scripting language, so you could just as well use an existing one (e.g. Ruby).
In PowerShell if the result of a command is just an object the object is pretty-printed to the console in practice ends up looking pretty much like what a Unix command would have given you.
Compare the output of "df -h" vs the PowerShell equivalent "gdr -psprovider filesystem", for example. One provides the data in dense (easy to follow) rows, while the other spaces it out across the whole screen, leaving large gaps of empty space around some columns while also cutting off data in others. The difference is especially noticeable of you have network shares with long paths.
PowerShell is probably nice for scripting, but I wouldn't want to have it as my shell.
So... no real improvement, then?
Typed objects can make it harder to pipe commands together. How do you grep a tree when tree is an actual data structure and grep expects a list of items as input? You would need to have converters. Either specific converter between tree and list, or a generic one: tree->text->list.
>Something Windows does get right is the clipboard.
It useful, but the actual implementation is pretty bad. Opaque, prone to security issues, holds only single item, cannot be automated.
To be fair, untyped objects also require converters, but at every boundary. That is, instead of having some pipes of the form `program -> mutually agreeable data structure -> program` and some pipes of the form `program -> unacceptable data structure -> parser -> program` (as happens with a typed language), you are guaranteed by a text-based interface always to have pipes of the form `program -> deparser -> text -> parser -> program`.
For example, if you grep the output ls -l for a file named "1", you'll also get files with 1 in their timestamp. In text land, you have to edit the ls command to get simpler output. In structured land, you could edit your filter: ls -l | grep name~=1
You could imagine various structured data filter tools that could be built that wouldn't require modifying the input.
Though in this example, you can easily use awk to select the column, wouldn't it be nice to not have to worry about things like escape characters when parsing text?
ls -> repeated FileDescriptor files; | repeated string names;
Where FileDescriptor is whatever it needs to be, has all the info ls -l does. You have a heirarchy of outputs: if the next takes FileDescriptors, you give it FileDescriptors, if it doesn't you give it strings.
What would go to stdout goes through a toString filter.
There is one possible complication, though. The two would need to be reconciled in the same way by everyone who wants to write a shell tool. Given that even fairly simple standards (RSS, HTML, etc.) cause lots of failures to comply, what are the odds of near-universal compliance in a larger and more diverse ecosystem like shell utilities?
Aside: that's what the stat command is for. My big concern with types is how would you make sure that the output of a command will always have the right types? Otherwise you'll have runtime type errors which would be just as bad as runtime parsing errors.
Parsing errors in script shells can easily go unnoticed (until you realize your data is corrupted).
none? 'ls' is for humans. 'stat' is for scripts.
Knowing all the edge cases, exceptions and places where reality breaks down is the problem.
Where software should indeed work with the principle of least surprise, Microsoft have patented the principle of most inconvenient surprise.
Also SO has one small comment about this which didn't exist when I discovered it. I had to use windbg to dump the CLR heap and find out that there were a crap ton of buffers...
And that's every day using powershell. It's even worse if you trip over someone else's cmdlets which aren't aware of the correct harry potter style incantations to issue that don't cause the universe to implode.
The whole thing is a joke. A bad one.
I'm doing this on windows 2008 server.
You can probably install a more recent version of WinRM on that machine, though (or by now probably the open-source version of PowerShell).
systemctl status wibble.service : Displays human-readable information, with colours, line drawings, and variable-length free-form text; that is exceedingly hard for a program to parse reliably.
Contrast with
systemctl show wibble.service : Outputs machine-readable information, in systemd's favourite form of a Windows INI file.
But.
- People above were just complaining about having to use sed to tweak output. I don't see why they would prefer a built-in filter to an external one. The external filter is far more flexible, and if that isn't enough, you can replace it.
- I'm generally not a fan of applications that tailor their output to what they think the human wants. Unless there is a deeply compelling reason, I want to see the same output on pts/0 as something down the pipeline. The reason for this is that picking up environmental hints to serve as implicit configuration is hacky, subject to error and and can later be the cause of really difficult-to-find bugs.
Perhaps I'm just irredeemably brainwashed. If you like a typed command line, Redmond has your back. For me, wanting types is a signal that I should start considering whatever little shell hack I'm working on complex enough to take it to a language that wasn't designed for ad hoc interactive use.
And at the same time, I really, very much do not want my command line to look like C#.
I get that a lot of folks these days are mostly GUI users who maybe type a little at git or run test suites from the command line, and not much else. I get why things like Powershell are appealing to such folks[1]. But when the command line is your primary interface, strong typing and OO hoop-jumping is huge waste of cognitive energy.
I do feel that Unix, to a first approximation, got the balance somewhere close to right. Loose coupling with lot of shared commonalities instead of a rigid type system and nonexplicit magic works really well for me, and if tighter coupling is a good idea, then I'll build it.
[1] Why they want to radically change the command line instead of using their favorite language to do systems stuff from the comfort of wherever they spend most of their time, I do not get.
You don't see why someone would prefer 'select' to scraping with grep or awk?
Get-PSDrive | Where Provider -like '*FileSystem'I find the idea baffling that basically having to have the whole syntax tree memorized instead of the shell providing it for me is somehow less of a "waste of cognitive energy".