Theres a lot of members of the mathematical & computer science online communities who have spent time discussing and thinking about this sort of problem.

Timothy Gowers for example http://gowers.wordpress.com/2012/02/02/abstract-thoughts-abo...

The problem with all the semi active (or at least working demo) proposals is that they wind up looking like stackoverflow or the like. This is probably not ideal, especially for a system that might be intended to help evaluate a scientists professional work. Not every good scientist is active on the internet beyond posting their papers and replying to email.