Ask HN: How do I become a shell god? It seems many things can be done with a one-liner or short shell script, but I'm not fluent enough to recognize those patterns yet. I do know a few basic commands, but much of *nix seems like a bag of undocumented tricks; and man pages are hieroglyphic IMO. Is there a cohesive path to learning all the tricks? All of them! |
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
From there, you have a better background for reading the documentation of shell implementations, like Bash.
Also study Awk:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/a...
Those who don't know Awk do comical things with cockamamie combinations of cut, sed, grep, echo ... and clumsy loops written in shell.
Other than shell syntax and semantics, it's a matter of knowing a large number of utilities.
Learn the shell chainsaw: `xargs`. `xargs -L` vs `xargs` and `xargs -I` (examples will say use `{}` but I'm partial to `xargs -I @` because that works with Fish and Bash, plus it's a single character.
Try to write small loops and pipes on the command line. I do things like this all the time:
for i in foo*.txt; do mv $i OLD-$i; done
for i in foo*.txt; do mv "$i" "OLD-$i"; done
Also, shell isn't the end. I move to python when shell scripts get cumbersome. python can easily parse arguments (argparse), handle filenames with weird characters in them (os.path), walk trees of files (os.walk) and more.
As of yours loop, such thing, when renaming files in bash, you can do `mv {,OLD-}foo*.txt`
Great practice would be to write report scripts which aggregate multiple data sources into single array of predefined objects which are finally sent into `stdout`, `/var/spool/user` or anywhere else.
I don't think so. Doesn't this break when the glob matches more than one?
Then, whenever you have some everyday problem (frequently some "batch" or "bulk" job operating on many files), consider using the shell instead some 3rd party tools.
If you really are into it, consider working with suitable files. For instance, drop office files (MSO or LibreOffice/OpenDocument) and instead switch to plain text files (Markdown or Latex). They are much easier to process in the command line. Also consider using git instead of dropbox and friends, because it is CLI-first. There is a universe waiting to be explored: There are CLI clients for almost everything (chatting, mail, web, etc.).
Files were the default datastore, and text streams the default inter-process interface.
Although the world is changing, a surprising amount of it still operates through those tools.
A good way to understand the tooling is to do one of the Linux from scratch or similar projects, perhaps look at swapping out some tools for self-written alternatives, and scratching your own itch building projects that solve problems you have yourself.
Most people who reach a degree of proficiency in shell also have an innate understanding of the kernel's general organization, the nature of filesystems, the boot process, database types, are capable of programming in multiple languages, comoprehend network programming and the challenges of locking and distributed state, and so on. You can't learn all of it in one go, so just leverage that interest and enthusiasm and devote the time.
- read the manual (see other posts) - customize it (and persevere) - make it fun
The second is really helpful. A lot of my shell commands/utilities [0] are simply better-for-me ergonomics on commonly available commands. These include things like `fzf` wrappers and more sophisticated aliases.
I specifically found the making of `fzf` based utilities to be a source of major learnings. It requires you to know quite a bit of shell mechanics to do things nicely. It's very similar to building a UI with APIs. You invariably learn the surrounding concepts (HTTP, auth for APIs, content types, caching, etc.)
I created a tutorial on Linux Terminal Tools a while ago where I covered many topics but not in too much detail. Perhaps going down the rabbit holes on these slides might help: https://ketancmaheshwari.github.io/pdfs/LPT_LISA.pdf
That said, I challenge the usefulness of deep shell knowledge.
If you're planning on writing 300 lines shell scripts, you're most likely doing it wrong, you should be using a full featured, modern programming language.
Worth noting that shell is a programing environment (not language) so is highly dependent on the tools you are using or gluing together to accomplish something.
If you think that python and bash are equally equipped, think again.
https://www.oreilly.com/library/view/unix-power-tools/059600...
Honestly? If you can think of anything that needs to be automated, then write a shell script to automate it!
I’d suggest taking any task that you would do in excel, and do it on the command line.
Classic Shell Scripting by Arnold Robbins
Unix Power Tools 3rd edition
Two additional worthwhile books:
The Unix Programming Environment by Kernighan and Pike
Mastering Regular Expressions by Friedl.
All 4 are classics.
Also try tldr.sh website in addition to man pages. They are easier to grok.
Good starting point