Writing an LLM from scratch, part 13 – attention heads are dumb | Dark Hacker News

Dark Hacker News

new|best|ask|show|jobs

Built with the HackerNews API. Developed by LJT.AI

Note: The HackerNews API provides up to 500 stories per category

Writing an LLM from scratch, part 13 – attention heads are dumb(gilesthomas.com)

351 points by gpjt 160 days ago | 67 comments

Writing an LLM from scratch, part 13 – attention heads are dumb | Dark Hacker News