Follow live the open training of a 535B (23B activated) LLM | Dark Hacker News

Dark Hacker News

new|best|ask|show|jobs

Built with the HackerNews API. Developed by LJT.AI

Note: The HackerNews API provides up to 500 stories per category

Follow live the open training of a 535B (23B activated) LLM | Dark Hacker News

Follow live the open training of a 535B (23B activated) LLM(twitter.com)

1 points by ggcr 11 days ago | 1 comment

ggcr 11 days ago |

Announcement tweet by Stanford prof Percy Liang and Marin: https://x.com/percyliang/status/2090918065634684997 https://xcancel.com/percyliang/status/2090918065634684997

It will include Pre-training and Mid-training. ETA 3 months.

Live wandb dashboard: https://wandb.ai/marin-community/marin_moe/reports/535B-A23B...

GitHub issue tracking progress: https://github.com/marin-community/marin/issues/8435

23T tokens (not deduped) directly in S3 for anyone to see: s3://marin-us-east-02a/marin/datakit/store_4d2e363d

Intermediate checkpoints also in S3