Guide to the Mamba architecture that claims to be a replacement for Transformers

Guide to the Mamba architecture that claims to be a replacement for Transformers(blog.oxen.ai)

5 points by gregschoeninger 2 years ago | 2 comments

Been diving deep into the Mamba paper and put together my notes here:

https://blog.oxen.ai/mamba-linear-time-sequence-modeling-wit...

Took me awhile to wrap my head around some of the terminology, so hopefully this helps anyone else trying to grok it.

andy99 2 years ago | |

You may also be interested in https://github.com/rbitr/llm.f90/tree/master/ssm it's my inference only implementation of mamba which ends up being much simpler than the training code in the original repo