Ask HN: Musical archive needs your help - Safety ML filter for 200K+ photos

4 points by panosfilianos 3 years ago | 3 comments

Hello everyone,

A friend is working on one of the largest musical historical archives as an archivist. His team has been tasked with going through the personal archive of a donor.

The archive contains hundreds of thousands of pictures which are downloaded with random filenames from standard browsing from that donor.

The issue is that the archive includes useful pictures mixed together with adult pictures with no way to distinguish between them besides manual review. They have been working on this for weeks and they have a ton more pictures to go through.

Is there any software I can help them put together to make the distinguish automatically with a good accuracy? Even sorting (possibly adult/ possibly safe) is good enough. I am SWE myself, so I can create something fast in many langugaes (especially Python and JS).

The archive pertains to one of the greatest opera singers of the 20th century and is the largest to date, so your help here will be meaningful.

Thanks a lot!

jpoesen 3 years ago |

No experience with any of this, but a quick search turns up stuff like https://deepai.org/machine-learning-model/nsfw-detector, which looks affordable and straightforward to implement.

And here's bunch more: https://rapidapi.com/collection/nudity-detection-image-moder...

panosfilianos 3 years ago | |

Thanks! I was wondering what kind of open-source models could exist for this.

sinuhe69 3 years ago |

You can use CLIP to build a database to search (offline) for keywords like “nude” or “naked”. Specifically, I use clip-anytorch with the ViT-B/16 pertrained model and find the result very good. Just go to pypi and the corespondent GitHub. They have examples and demo for a quick start. It can run on CPU, too, albeit a bit slow.