Ask HN: How do you manage manual testing? We're just getting down and dirty with a final round of testing before our initial private beta release. Part of this includes manually testing the system, mostly in the form of scenario tests for our most important use cases. How do you manage this process? e.g. create a test script (txt/html/xls?) and run through it with several different OS/Browser etc. and then record the results |
We had plain text test plans, which worked well enough. We didn't do as good of a job of keeping track of finished plans as we should have.
You need to very tightly specify what you are testing for. Manual testing is incredibly valuable to test the things that programs can't pick up, and also very wasteful if you are testing the simple stuff like that.
Also, look for what's scriptable around the manual test point. For instance, if you are verifying the 4 different kinds of order views, have a script which generates those orders on staging, and fires up a browser window w/ 4 tabs to those orders (already logged in, etc). Manual testing doesn't have to mean manual setup & tear down.
An xls or google spreadsheet is the best bet for tracking, at least for a one-off smaller scale testing environment. At my job, we did eventually get around to writing a "Manual Test Manager" rails app to help us track it, but I left before it got much traction.
The other suggestion here of selenium is useful, but perhaps not what you're looking for. It's very useful to record whole click-throughs of your app, but you still need to verify stuff like CSS & layout, wording of text, that labels match up with the right fields, etc. Selenium just doesn't do all that very well (too much fuzzy human intelligence).
If you plan on making this repeatable, divide the test cases into "smoke tests" and "full tests". (Smoke Test == "is it spewing smoke" ie. is anything obviously broken). That gives you the ability to run through a 20 minute manual test suite right before the weekly deploy, and do a full manual test of edited areas, etc.
Also, don't slack on your automated tests. Computers are just awesome at unit tests.