Pathod: A pathological HTTP daemon for testing and torturing client software

Pathod: A pathological HTTP daemon for testing and torturing client software(cortesi.github.com)

54 points by cesther 13 years ago | 14 comments

cortesi 13 years ago |

Drat - this story is just a tad premature. A brand new release of pathod will be out next week, including a publicly accessible pathod instance, pathoc (pathod's evil client-side equivalent), and a huge range of other improvements and bugfixes.

cesther 13 years ago | |

apols

cortesi 13 years ago | | |

No, thanks for posting! Just upvote again when you see the release notice next week. :)

cafeconleche 13 years ago |

This is a similar tool as http://httpbin.org/ which was written by Kenneth Reitz to test python request http://python-requests.org/. I like that this is written in tornado thanks for sharing.

cortesi 13 years ago | |

One difference is that pathod (and pathoc, the client-side equivalent) works hard to make it possible to violate the HTTP protocol specs at will, in pretty much arbitrary ways. The next version of pathod also moves away from Tornado to let it take more direct control of comms.

cafeconleche 13 years ago | | |

Ok the client sounds really interesting to me. I am goung to play with it a bit.

blatherard 13 years ago |

bane is a similar tool that pathod-interested users might be interested in.

https://github.com/danielwellman/bane

From the readme: "If you are building an application, you may depend on third-party servers or web services for your data. Most of the time these services are reliable, but at some point they will behave in an unusual manner - such as connecting but never respond, sending data very slowly, or sending an unexpected response. To ensure your application survives these scenarios, you should test your application against these bad behaviors. Bane helps you recreate these scenarios by standing in for your third-party servers and responding in several nefarious ways."

suresk 13 years ago |

Really cool!

A year or two ago, I was working on some automated deployment tools that basically called a bunch of Rest APIs. As I discovered how unreliable the API calls could be at times, I ended up writing something kind of similar (although very application-specific to this) to be used as part of my unit tests for it - it helped immensely.

One thing that would be nice is if I could pre-define reactions to certain requests - ie, some sort of way to specify that when you get a POST at uri /add with a content-type of application/json, I want a 500 error (or maybe I want a 500 error 5% of the time) - rather than having to specify what I want in the actual request. Is this something your client library will make possible?

cortesi 13 years ago | |

This is already covered. Have a look at the anchor feature for pathod. For instance, starting pathod like this:

pathod -a /foo=200:b@100

Will result in an HTTP 200 response with an body of 100 random bytes if you hit the path /foo.

enginous 13 years ago |

I like the idea, but does the syntax have to be that terse? It's quite expressive but less readable than code (or JSON/YAML), and I'm not sure why that's useful in this context.

cortesi 13 years ago | |

Yes, this is something I've pondered too. The language is ugly for a number of reasons - not least the constraint of avoiding syntactically significant characters for both URLs (because of pathod) and shells (because of pathoc).

At the moment, my stance is that the syntax needs to be as terse as possible so that it can comfortably be specified in a URL, or in a snippet in a unit test. In the current master, there's a variant syntax that lets you use newlines (or arbitrary whitespace) wherever the more compact syntax uses colons. In time, I could see an argument for also expanding the single-letter abbreviations in this mode to make a more verbose syntax. There's also no reason why we couldn't add a way of describing requests by manipulating an object, which then compiles down to the request language.

Anyway, this is all still evolving - if you have ideas for keeping the language compact but making it less like line-noise, drop me a line.

cypherpunks01 13 years ago |

What's the practical usage scenario for this? Would it be only used by http client library developers, or is there a broader audience?

cortesi 13 years ago | |

I started work on pathod to improve the testing of mitmproxy (http://mitmproxy.org). It's since become a very capable Swiss army knife with many creative uses. I now routinely use it in pen tests (it's great for exploit delivery), and pathoc has become quite a capable tool for simple fuzzing and stress testing. I'm working on the next release docs as we speak, and I'll try to document a range of different use cases there.

Hoff 13 years ago | |

Or you could hand examples of your favorite rogue web bot a whole pile of confusing chunder.