Ask HN: How to wget pages with javascript?

4 points by baconface 15 years ago | 8 comments

I need to wget a url, allow the javascript to execute, and capture the result in a file. I've tried wget, curl, httrack,plugins like lget, and links(2) and none of them execute JS.

I want something like $: firefox <url> -s <file>

Does anyone know how to accomplish this behavior without using JS?

dstein 15 years ago |

You can write a NodeJS script using the JSDom module to do this.

baconface 15 years ago | |

Thanks for the suggestion. Any idea how to accomplish this task without using JS?

I'm surprised chromium, konqueror and firefox don't offer this functionality for offline browsing.

dstein 15 years ago | | |

Selenium (http://seleniumhq.org/) might work but it's not a simple command line script.

a3camero 15 years ago |

iMacro? Rhino?

profitbaron 15 years ago |

Personally, I'd use http://www.phantomjs.org/ and create a script which loads the page like a browser. Once you've done that you can get it to take screenshot or use JS to pull data off the page.

baconface 15 years ago | |

Thanks. I really want to get this done with the command line and it looks like Phantom JS will do the trick.

ddemchuk 15 years ago |

perfect use case for celerity http://celerity.rubyforge.org/