Show HN: Single file transformers implementation for learning(gist.github.com) |
Show HN: Single file transformers implementation for learning(gist.github.com) |
The focus was on being able to demonstrate training, inference and attention, all in one file;
This can be run on a GPU thanks to cupy, a kernel needn't be written for this whole thing to run. I definitely think that more people can mess around with different attention mechanisms and models and try training models out on their computers. That is the post.