As a side note, I checked out gadersd's implementation of llama2 before. [1] I was surprised to see how clean the code is. [2]

[1]: https://github.com/Gadersd/llama2-burn/

[2]: https://github.com/Gadersd/llama2-burn/blob/main/src/model.r...

Who cares, is it faster?