I’ve been ruminating on the postulation of a universal signature for every entity across sensory complexes (per sense organ reality, vision, touch, mind) which translates to the problem of entities represented in binary needing to be related across modalities as in “butterfly” vs a picture of a butterfly vs the audio of butterfly vs the thought pointing to one of those or other.

I was wondering if there was a universal signal that can be used as the identity and then based on that signal one could measure the distance to any other signal based on the principle relation of not(other). That is to say the identity would be precisely not all else for any X. Said another way, every thing is because it is exactly not everything else.

So thinking as first principles as possible I wondered if it were possible to represent everything as some frequency? A Fourier transform analog for every “time slice” of a thing? This is where it gets slightly slippery.

So the idea was trying to build relationship and identity and labeling from a simple rule set of things arising out of relation of not being other things.

In my mind I saw nodes on a graph forming in higher dimensions as half way points for any comparison. Comparisons create new nodes and implicitly have a distance metric to all other things. It made sense in my mind that there was an algorithmic annealing to new nodes in a “low density higher energetic state” allowing them to move faster in this universal emergent ontology/spatial space; eventually getting more dense and slower as it gets cold.

So the system implicitly also has a snapshot of events or interactions based on that where every comparison has a “tick” that encodes a particular density relation for some set of nodes it’s in association with.

The idea that cemented it all together was to treat each node like an address:chord. Similar to chording keys like a-b-c in some ux programs, but also exactly like chords in music too.

The idea being that when multiple “things” are dialed in at same time it becomes its own emergent label by proximity and association of those things being triggered to new information coming in classified as a distance to not(signal).

I didn’t really realize how close this idea was to what encoders/decoders seem to be doing although I do know I’m trying to think myself towards a universal solution that doesn’t require special encoders for every media type. Hence the Fourier transform path.

Know anything like this or am I spitting idiocy?

You're actually kinda hitting the nail on the head. _Generally_, the word2vec woman + king = queen thing was cute but not very real.

People rarely have to get down to the real true metal on the embeddings models, and they're not what people think they are from their memory of word2vec. Ex. there's actually one vector emitted _per token_, the final vector is the mean. And cosine distance for similarity is the only metric anyone is training for.

In summary, there's ~no reason to think a visualization trying to show multiple vectors will ever be meaningful. Even just starting from "they have way way way more dimensions than we can represent visually" is enough to rule it out

Mini LM v2, foundation of most vector dbs, is 384 dims.

n.b. dear reader, if you've heard of that: you should be using v3! V3 is for asymmetric search, aka query => result docs. V2 is for symmetric search, aka chunk of text => similarly worded chunks of texts. It's very very funny how few people read the docs, in this case, the sentence transformers site.

If I gave you a live GPU shader that let you arbitrarily position any of say a few million words with simple Cartesian coordinates, what would you do with it? Whole words expressed as Individual letters - not symbols, representations, or abstractions. Just letters arranged in a specific order to form words.

Hey there! Sincerely cool stuff, I’m glad it’s fun for you.

It’s actually quite approachable to play with, and some of the comments about, “wut?” may be best answered by a little more experimentation on the user’s side, haha. I think the content itself is tricky, which may trip people up.

Something I’ve seen before that may be interesting is doing something with the definitions of words. ATM, you’re using a source list of words and using the embedded vectors to visualize. But what if you visualized not just the words themselves, but the ordered list of words that make up the definition(s) of that word visible in some spatial relationship. This would look interesting because around (connected to?) each word is its meaning in this case; changing the definition (the context use of the word) would also change the definition… and also change the connected word nodes in the graph. I envision ordered lines and colored words in this style.

If you end up doing something like that, start with like.. a “sentence player”. At the moment you show the words at once. What would it look like to “animate” the appearance of the words and their relationships by definition?

Anyway. Thanks for getting this far, haha. This is a really fascinating project and I’m glad you shared it. Please do tell if any of this is close or far off from something you might be interested in!

T SNE Not PCA. With t SNE parameters that can be configured

I think the PCA dimensionality reduction is non deterministic, but I say this with really low confidence

Similarly, words like “I” and “am” appear in a slightly different dense section of the map by default.

I think that's going to be a geodesic in a hyper-dimensional manifold. There was an article here about 'wordlets' on a hyper-sphere and a piece on time and LLM and the relating manifold. Visualising LLM topology (multi-dimensional topological manifolds) is a very rich area for exploration. I'm waiting for someone to use PHATE to do the dimension reduction, it's used in neuroscience to reduce dimensionality providing information not visible using PCA, t-SNE, LDA or UMAP.

In the bottom left "?" button it says it performs PCA down to 3 dimensions. That's going to lose a ton of information, rendering the space mostly useless.

I wonder what patterns we could create from the word glyphs that produce meaningful patterns. You said “edge of the galaxy” based on the position of the words. I wonder what else you’d come up with different embeddings and organizations.