15:[[1564035,{"by":"askjob","descendants":3,"id":1564035,"kids":[1564133,1564149,1564166],"score":3,"text":"After I saw three \"Who's hiring\" threads, I recognize that startup companies are mostly looking for web developer that relates on back-end front-end development, RoR, Scala, Hadoop, Javascript, CSS, LAMP stack.

It totally makes sense since most of the startup companies are building their websites now.

I was just wondering if there is anyone hiring for a non web developer position which only requires knowledges in basic CS (algorithm, memory, etc) and basic programming in C, Python, Java, C++, etc.","time":1280598682,"title":"Ask HN: Is there any non-web-development jobs out there?","type":"story","url":""}],[1495941,{"by":"kimfuh","descendants":3,"id":1495941,"kids":[1496240,1496322,1496163],"score":1,"text":"IBM. Apple. Microsoft. Google. The i-line of products. Facebook. Twitter(?). What do you think will be next?","time":1278546210,"title":"What's going to be the next \"really big\" thing?","type":"story","url":""}],[1400194,{"by":"haihai","descendants":7,"id":1400194,"kids":[1400297,1400252,1400294,1410092,1400449],"score":7,"text":"I want to crawl a site, like foxnews.com and find all the URLs that match a pattern.

A pattern like:

http://www.foxnews.com/\\w+/\\d+/\\d+/\\d+/.*/<p>That would find all URLs like:

http://www.foxnews.com/world/2010/06/02/report-natalee-holloway-suspect-sought-murder-peru/

I know I could do this myself. I also know it's a seemingly easy problem, that is actually quite hairy.

I'm hoping there's an open source crawler that I can point to a start page and say \"Find all the URLs that match this pattern.\".

I know there are dozens of crawlers out there. I just don't know if there's one or two that are really great. I'm really hoping there's a modern/simple/fast one that would be good for this purpose.

Does such a thing exist? If not, are there any great documents detailing the common problems and how to solve them?

Thank you.","time":1275531759,"title":"Ask HN: Is there a great open source crawler?","type":"story","url":""}]]

user:	Ledio
created:	June 3, 2010
karma:	9