Maybe dang can replace the link to the actual announcement and remove the blog spam.

I did the same for a few months earlier this year. It was theoretically a decent sum of money here (3rd world SE Asia) if you could do the minimum amount of hours they require.

From the announcement linked to below: (https://news.ycombinator.com/item?id=10601232)

Ratings from evaluators do not determine individual site rankings, but are used help us understand our experiments. The evaluators base their ratings on guidelines we give them; the guidelines reflect what Google thinks search users want

Google has been upfront about greater use of learning AI (as opposed to hand-tuned algorithms like PageRank) to index and rank their search results. In order to return a good result, a learning AI system must first be trained on what a good result looks like. Since search returns results to humans, humans have to provide the guidance on what is a good or bad result.

Often an AI can be trained by a corpus, like training a spell correcter by feeding it the text of a dictionary. But the data that Google works is the entire web, which changes all the time, so they need constant feedback on what is good or bad.

Google also personalizes results now, so the AI needs to learn not just what is a good result, but what is a good result for that particular person, given everything else the AI knows about that person. That would require inputs from a wide variety of people.

I think it's proof that google has done an amazing sales job.

Their ML algos really come down to codified, scaled human intuition and decisions. However, to avoid having to answer for them (google has the ability to make or break many internet companies), they repeatedly go on and on about ML and pretend it's just math, as if there is a first principles answer to eg which candle company should rank highest passed down from Gauss.

The reason this is valuable to google is if they admit it is just collective human judgement, governments are much more likely to demand input.

I suspect this is also a fail safe which allows them to use learning algorithms that give less than well understood results that still perform well (think neural networks and deep learning).

That approach is prone to creating a positive feedback loop through an order bias.

Click through rate isn't necessarily indicative of quality.

I'm sure they measure that, also.