Ask HN: How to visualize all human thought

1 points by poorcedural 289 days ago | 3 comments

One of you AI researchers with almost all data and cheap compute has already thought of this. What would it look like to visualize all human knowledge?

1. Tag all content with a date (probably almost done). In long passages, estimate a date based on published + written rate. 2. Once every sentence or phrase has a date, create a few blockchains. One for short thoughts (128 tokens), and more chains for longer thoughts (3072+ tokens). 3. Run an LLM embeddings cosine similarity or some better metric for each tokenized idea with a threshold of say 75%, tokenized by natural punctuation if not found. 4. Only NEWish content gets stored in the block, chained if novel. Again, short and long thoughts recorded. Coordinates from the vectors. 5a. I remember reading once that working through a PhD was like pricking the inside of a balloon, I love that visualization. Every novel idea pricks the inside of a growing sphere of knowledge over time. 5b. Has anyone ever tried to map human knowledge in three dimensions? 5c. Alternatively, a tree or root system growing over time. Branching from the vectors.

sunscream89 289 days ago |

I’m pretty sure human thought is nothing like you describe.

1) human knowledge is “holographic” not like a block chain. There are no dates, and the ability to compare and relate is much different than block chain serialization.

2) much of what we know is WRONG! Even if only a little. Things like “big bang” and “states of mind” and “dark matter” and so much more are cloudy imaginations that blur into so called science, yet you would include them as “knowledge” when they are fantasy suturing our ignorance and confusion.

3) you have more than three dimensions to work with. Any vector attribute is a dimensionality when graphing so color, size, line thicknesses, solidity, etc may be useful considerations while “graphing”. The mind is more like this, a hyperdimensional holographic interrelationships.

4) in your idea you would have one representation, in the human mind there are many representations even those contradicting each other. Think about those who both believe in God and Science, or humans and peaceful coexistence (we are natural competitors who covet what we do not have), or even reconcile how words and deeds do not align.

Much of our “knowledge” is artificial and prescribed, still more is usefulness not necessarily accuracy, still more relative to those who accept whatever it is as a principle of something else unrelated.

Not even science can cut through the ambiguity and contradiction that is human thought or “knowledge”.

Have fun with it though!

sunscream89 289 days ago | |

I thought of something, what you (OP) describe may be an over-the-top ambition.

Consider if you did a “knowledge graph” of one single sentence, or on a paragraph, or short story, or whole story.

That would be something, one (or more) graph representations of a starting point.

That way you could compare and contrast two or more such graphs to see differences in the composition of expressed ideas.

That would be something!

BillyTheMage 289 days ago |

This seems to relate to my quest as a metamage "to identify and assemble the Epistemological Primes into the Knowledgecore and use it to defeat the Omnarch". I'm taking a somewhat opposite approach to a PhD, embracing what I've seen someone else refer to as "high-leverage generalism" but at the metaphorical doctorate level.

All this stuff about tokens and vectors is quite different from my approach. I'm always interested to see how other metamages approach this issue.

I think a hypergraph structure is the only thing capable of properly representing knowledge. Knowledge is not just a bunch of nodes with 1:1 connections; there's complex structure in the nodes.