TokenTown: A visual way to understand how LLMs work(laurentiugabriel.github.io) |
TokenTown: A visual way to understand how LLMs work(laurentiugabriel.github.io) |
Proceeds to literally loop around 6 times.
The "Output Plaza" doesn't seem very outputtish - cars/embeddings seem to loop around back into the input.
Not sure why the Attention Plaza is occasionally shooting stuff down the storm drains/whatever in the center!
The sampler as centrifuge is a strange visualization!
So I guess the takeaway is that LLM are related to cars? Or maybe highways. Was this designed by Schmidhuber?
I made it because I kept explaining tokenization to friends and realized there wasn't a simple, interactive tool that focused on learning rather than documentation.
I'd love feedback on:
Features that would make it more useful UI improvements Bugs or edge cases you've found
Hope it's useful!
*I asked a LLM to make it