Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels

Flash-MSA: Accelerating Million-Token Training with Sparse Attention Kernels(nanduruganesh.github.io)

41 points by rawsh 50 days ago | 5 comments

kamranjon 50 days ago |

I’ve actually been really interested in Minimax M3 - seems like it flew under the radar but size wise might actually be runnable for local inference with a footprint somewhere between Deepseek V4 flash and pro.

Has anyone used the new Minimax M3 model? I’m curious how it compares with Deepseek V4 and GLM 5.2 and other larger open weights models.

throwa356262 49 days ago | |

I have used it a little bit (0.5B tokens) for agentic tasks and coding. It is pretty nice and a serious step up from M2.7. I prefer it to GLM 5.2 for simpler tasks because of the cheaper token plan.

[Update: their cheapest token plan has been removed, I guess its back to GLM now]

Right now M3 is not far behind DS4, but I belive DS4 will improve much more with each round of training. It simply has a bigger brain, it just needs to fill it with more information.

villgax 50 days ago |

World’s first?

Such lazy, much farming

https://github.com/fla-org/native-sparse-attention?utm_sourc...

kamranjon 50 days ago | |

The Minimax paper was published in June 2026 coinciding with the Minimax M3 release - I’m not sure how the repo you posted here could have been an implementation of Minimax sparse attention when it was updated over a year ago?

throwa356262 49 days ago |

Speaking of minimax, has anyone used their video/music models? The demo page looks cool but I never heard of anyone actually using them in anything.