Instead of going the tags + categories route, I used the vertex api multimodalembedding@001 model. How it works When a pin is to be saved, fow's server sends a client-side embed token, with permissions to call the vertex-api directly (through the impersonated access token flow), this returns a f32 512 dim vector. I store this in a usearch vector db on my $15 VPS. Usearch doesn't have runtime disk persistence, so I had to write a WAL wrapper around it. What you can do The perks of multi-modality is pretty fun: you can currently search for web design patterns via images and text, in combination with other filters - colors, font(s), domain. Some examples to try: https://fontofweb.com/search/pins?q=elegant+serif+blog+with+... single fonts, pairings, or 2+ combos, e.g https://fontofweb.com/search/pins?family_id=109 https://fontofweb.com/search/pins?q=&family_id=109&family_id... color search/sorting (done in perceptual CIELAB space not RGB) https://fontofweb.com/search/pins?q=&color=%23ff0000 domain search (filter by site, e.g. https://fontofweb.com/search/pins?domain=apple.com, https://fontofweb.com/search/pins?domain=blender.org) Appreciate your thoughts, scathe me, praise me, don't hold back, thanks for you time :) |