Show HN: Scaling PhantomJS With Ghost Town

Show HN: Scaling PhantomJS With Ghost Town(blog.buzzvil.com)

32 points by tkazec 12 years ago | 11 comments

m0th87 12 years ago |

We developed the same thing: https://github.com/dailymuse/phantom-cluster

It was used for prerender [1] for some time, before being changed so that clustering happened at the HTTP worker level.

Scaling PhantomJS is no walk in the park. There's a lot of duct tape to get it to communicate with node.js, and it's easy for a memory leak to occur.

[1] https://github.com/prerender/prerender

gcb0 12 years ago | |

you are all doing it wrong... we use phantomjs on its own. and run the test code in it. so its analogous to loading the target page, and then running your code/test in a browser dev console. you take care of concurrency and retries in your preferred shell. this also allows things like integration with Makefiles, etc.

using it inside node is just masochism.

codereflection 12 years ago |

Can you please qualify the "neglected projects" statement? Looking at the commits on both projects show that they are not neglected.

phantomjs: https://github.com/ariya/phantomjs/commits/master

phantomjs-node: https://github.com/sgentle/phantomjs-node/commits/master

tkazec 12 years ago | |

PhantomJS: Extremely popular (10k+ stars!) with a comparative trickle of commits. The major flaws might be excusable, but the number of small issues and unmerged pull requests is not.

phantomjs-node: Popular but badly structured and documented. Many small ignored issues.

FootballMuse 12 years ago | |

At least for PhantomJS, there are 236 outstanding prs/issues since their last release Jan-26 with only 19 commits mostly dealing with Travis.

Also the development update Twitter stream @PhantomJS is not very active.

mootothemax 12 years ago |

Has anyone had any luck with PhantomJS and web fonts?

I'd love to use PhantomJS for taking screenshots of designs supplied by third-parties to us, and changing the CSS involved by hand is not a realistic option.

The issue itself appears to be nearly 2 years old:

https://github.com/ariya/phantomjs/issues/10592

And PhantomJS's core developers appear to be putting off the fix until version 2 is released, whenever that may be:

https://github.com/ariya/phantomjs/pull/12108

Any ideas? I'm currently investigating using SlimerJS, which uses the Gecko engine, but that involves its own different set of challenges.

damian2000 12 years ago |

Interested to know if Selenium web driver integration was looked into? You can use Selenium Grid for example to run distributed jobs with a Phantom JS backend... https://github.com/detro/ghostdriver

dsl 12 years ago |

> a system to reliably render images at scale

I'm curious what exactly "render" means in your context? Phantom.js is headless, so it would be futile to be simply rendering GIFs and JPGs within the browser I assume?

martin-adams 12 years ago | |

Imagine it like a web page to PNG converter. This doesn't need a desktop/display attached to do that. That's what "render" means in a headless context.

dsl 12 years ago | | |

Ah. I get it. Brain did not parse "render" as the output side, but rather the input, so I was trying to figure out why they used phantom.js as an elaborate image format converter.

qhoc 12 years ago |

I wrote my own doing something similar with leveraging the casperjs layer instead. I was just too lazy to deal with phantomjs directly.