Genshin Impact Fine-tuning CLIP for anime search

Genshin Impact Fine-tuning CLIP for anime search(colab.research.google.com)

32 points by alexcg1 3 years ago | 3 comments

Wonder how this stacks up with ResNet-50 family classifiers trained through TensorFloe or ML.Net

alexcg1 3 years ago | |

This is more related to search tasks, where we encode text and image pairs to use text to search image. ResNet can also be served a backbone for search tasks: content-based image search/reverse image search/search image with image. You need to remove the ResNet50 classification head.

On the other hand, Tensorflow or MLNet are machine learning frameworks, to achieve the task you can choose whatever you want to build the model components.

Khelavaster 3 years ago | | |

Sets of ResNets can do search indexing. Picture similarity can be inferred by vector cosine of the two sets of ResNets evaluations between two pictures.

[Run a numerical optimizer on weights for the different ResNets' outputs for best results ]