Hm so when I am writing web servers, I typically have a bash script that simply does curl on a bunch of URLs. I just run it by hand and make sure it stops on the last failure with 'set -o errexit'. I divide it up into bash functions so I can run the set of tests I want, and then string them all together with a top level function.
It's ultra-minimal, but that is the point. My job is to deliver working software (relatively quickly), not necessarily beautiful tests.
And I actually should say I do "system tests" in bash, not necessarily unit tests. But I START with system tests. My workflow is now like this:
- write SYSTEM tests in bash first. This helps me plan what I want to get working.
- write the code (It's like TDD but with system tests)
- sometimes there is some complicated logic I need to test. Then I switch to unit tests in Python, JS, R, C++, etc.
- when I fix bugs later, I tend to reproduce it with a system test. This makes bug fixing easy. Then I write a unit test to zero in on the broken part, and sure it never comes back.
I probably save 70% lines of test code, while delivering the same level of quality. A system test can be fast and can substitute for a lot of unit tests, and it helps you not ossify your code structure too early.
I don't have an example of that online, but you can see 880 lines of shell in the root dir here: https://github.com/google/rappor
Granted, this is more like scientific / batch code than web / backend stuff, but I assure you I follow the same strategy in the latter case. It's basically using Unix as leverage.
One thing I would notice about the scripts is how the last line is "$@". That runs the function $0 with the args $1 ... $9. This is a very handy trick that helps keep everything modular and independently executable.
I use shell scripts for all these things:
- building code (in C, JS, Python, or any language)
- testing code (running unit tests in multiple languages, and also system tests, constructing test data)
- deploying code
- sys admin; setting up servers (e.g. setup.sh in the repo), backup, managing log files
- building docs; making static web sites
- building config files and data files (e.g. lists of URLs)
My assertion is that, combined, these things take up MORE time than actually writing code! If you can speed them up, you will be more productive at "coding" overall.
In addition, using the same tool (bash) for all of them makes you more productive (once you have climbed the admittedly substantial learning curve). I'm not a fan of all these language-specific build tools now. I don't even use make that much; my first choice is bash!
Here is one concrete example of shell minimalism, which helps you get things done faster. Think about the common situation of having a {star}_test.py files in many directories, and you want to run them all together as a test suite.
I am embarrassed to say that at one point I spent 2 or 3 days researching and implementing some kind of Python test runner (this was over 10 years ago). I think people do it now with "nose" or something.
Here is my preferred solution:
find . -name {star}_test.py | sh -x -e
Or if you need parallelism, something like:
find . -name {star}_test.py | xargs --verbose -n 1 -P 10 -- sh -c
({star} should be asterisk; HN is not letting me escape those)
That's it. It doesn't necessarily give you pretty output, but it's what you need to deliver working software. Add one line to a bash script and be done with it, rather than getting distracted by yak shaving.
The shell scripts may not look that impressive until you think about what they actually save -- what common alternatives are. One line CAN actually save you 2 or 3 days of time; this is not the only time I've experienced this.
The overall point is that I used to spend a lot of time on tooling and automation (yak shaving), because I wanted a certain level of code quality and iteration speed. But now I do most of those things with a small, sharp tool -- bash -- and I find it saves a lot of time. I get in the zone and don't get distracted from the main task.