Ask HN: Why are web directories dead?

5 points by Noxchi 9 years ago | 8 comments

I am in the process of looking for a job. I have been keeping a text file of the sites to search for on a regular basis (because linkedin, indeed, craigslist, plus numerous niche sites all have different jobs.) I again randomly stumbled upon a good site (a venture backed startup that has $62mm funding) that provided me with a pleasant experience, that I didn't know existed before. (I actually found it going through a github list of sites effected by the cloudflare breach.)

I wondered how many other job search sites there are that I don't know about, and this immediately made me think of web directories. Gee, wouldn't it be great if all job search sites were in one list that someone maintained?

I went to DMOZ (the "open source" directory) and typed 'job.' Went to the appropriate section and a lot of the results were outdated including missing the $62mm startup that's #3 in Google. I applied to be an editor after carefully putting together an application, and was denied exactly 4 minutes after -- with no reason why. DMOZ basically seems rotten to the core and outdated as it's used by a lot of self promoters.

My question is, why isn't there a good web directory out there? Internet advertising is many billions of dollars large... unnecessarily. If there was a directory, people would have an easier time finding sites they like.

cauterized 9 years ago |

They died because the web is so large that it's somewhere between cost-prohibitive and impossible to index directory-style manually (which is how directories used to be maintained). And nobody's come up with a way to automate it that's been effective and not subject to gaming by spammers.

Additionally, categorization is useful for quickly answering only a tiny subset of query types.

sebst 9 years ago |

AOL is going to shut down DMOZ on short notice by end of February. Maybe that's the reason for your denial.

DMOZ had a history of being very strict with applications of both, editors and sites and in the end was primarily used to improve ranking.

In my opinion, the future of web directories, if there is any, is a combination of strict moderation, wiki-like collaboration and automated processes. Some questions will arrive from personalization and real-time content (think of news, weather, twitter/blogs...)

tugberkk 9 years ago |

I believe web directories will come back, just like people are going for lightweight web pages (no css, html only). It is easy to use, and there are still some people using directories whether it is web or not.

The only hard part is to be objective about listing pages, or just list those who you see fit.

benologist 9 years ago |

Web directories used human-classification, search engines became much better and almost real-time.

jfoster 9 years ago | |

The interesting thing is, search engines are not so good at this. I run a niche site that is generally what people want when searching that category, but it's a very slow climb to the top of Google, to the point where several users have contacted to say that the site was what they were looking for, but complained that it wasn't on the first page of Google when they were looking. It's climbing the rankings and starting to get on the first page, but it's taken 2 years.

Google doesn't really seem to know what's relevant when I search for "job search". The first page of results is okay, but the rest has lots of odd results. (Eg. Job search pages of particular companies when there are surely more broad job search sites that I'd rather see)

greglindahl 9 years ago |

Web directories are dead because humans now navigate the web using search engines. The blekko search engine was a combination of a human-curated web directory and search engine. It worked well in some categories, not as well in others.

id122015 9 years ago |

Both web directories and search engines can be fooloed by people.

Even the jobs posted on jobs sites can be fake. Understand first how companies employ people, through friends of a friend, etc.. and it will be much easier to find what you want.

crispytx 9 years ago |

Maybe you should bring web directories back.