Dead simple web crawling (at scale) with Zillabyte

Dead simple web crawling (at scale) with Zillabyte(blog.zillabyte.com)

28 points by jakequist 11 years ago | 4 comments

gamegoblin 11 years ago |

So just two hours ago the other post by OP (https://news.ycombinator.com/item?id=8143000) was flagged to death. Presumably because quickly after posting, two eerily generic and positive comments showed up (one from a new account).

That being said, as far as the service goes, some comments:

1. The popup "join our newsletter" after a minute was a little annoying.

2. Without guarantees on capacity and utilization, pricing by "machine hour" is useless to me. I'd prefer something more concrete.

kumarski 11 years ago |

Why hasn't someone built this before?

Maybe I'm missing something. I figured you could do this already using a few mashable API's.

sjarvie 11 years ago | |

There have been efforts. Looking at the product, I see that the component builds on an open source project.

Doesn't passing data through multiple APIs raise concerns of data polution/cost overhead? At least this uses a flat,pay as you go cost.

markitos144 11 years ago |

useful tool