Unix famously comes with a lot of documentation, but it's still famously hard to learn. Why?

As a teacher and cognitive scientist, the reason is interesting: Unix documentation, starting with its man pages, is presented in a format that makes it easy for experts to look up details, not for beginners to learn concepts or build skills. O'Reilly, Sam's, APress and others have built large businesses selling second-best solutions to this problem.

This is a teaching anti-pattern that emerges when domain experts -- but teaching novices -- try to write a curriculum from scratch. They don't realize that teaching is a separate skill from the material to be taught, and that a great deal of effort is needed to break a domain down into right-sized, self-sufficient chunks for the reader to consume. Instead, they produce big volumes of unorganized knowledge (man pages) that reflects their expert-level understanding, that the reader tries to sift through and, as often as not, gives up on in frustration. Even when you pay experts to write beginner-friendly books, you get separate books on bash, awk, sed, grep, regexes, files, processes, packages and so on. These put the onus on the reader to remember all the details and put them all together into something useful.

Having dumped all this information on you, I always get the feeling that the gurus are patting you on the head and saying, "Well, learning is hard. You have to work at it. Be sure you practice enough and you'll figure it all out eventually." Learning is not supposed to be easy, and you do have to practice, but too often that attitude serves as an excuse to stop looking for better teaching methods.

So I'd like to propose one here.

The interactive piece takes place on the user's machine. They clone a git repository that contains lots of text files that they can manipulate with complex commands, according to lessons and exercises delivered via a web app. The web app helps the user remember using a technique I call graduated spaced repetition, which extends the basic SuperMemo algorithm to test concepts, not just facts, with progressively more elaborate questions.

All of this should be relatively simple to write. The main differentiator will be good instructional design which -- sorry if I sound like a jerk here -- I have not seen much in evidence in the 20 years I've spent learning this material.

Which brings me to my questions for you:

1. I know a lot of people have already tried to do this. Can you point to any existing learning tools that have really knocked it out of the park?

2. What text processing tasks do you consider absolutely essential to Unix command line proficiency?

3. What kinds of text files (Natural language text, CSV, logfiles, etc) would it be useful to practice on to get there? Where can I find some?

If you want to talk about building the training set or the extensions I've made to the SuperMemo protocol, or if you just want to spitball ideas, you can email me at $MY_HN_USERNAME@gmail.com