I like the concept of this; we've had a similar problem in the past and tried to build an in-house solution which kind of worked; but we ended up wasting a bunch of time on it.

Your site could really do with some code samples though - I get that you're trying to make the API look simple, but unless you know curl, it's not particularly clear how you actually hook this up into a real world unit tests in different languages?