So, an LLM scoring over 80 on MMLU is no longer newsworthy on Hacker News? We've indeed come a long way.

Or perhaps it's just the skepticism surrounding such "claims". That's why I reckon more chatbots that don't rely on an API, similar to Inflection (or Microsoft's Copilot ?), should be directly integrated into the LMSYS Leaderboard. Isn't this what they did with Gemini/Bard?

Hope they'll release API access soon so I can add it to https://double.bot

They've been talking about an API since June and it's now 9 months later with no updates on API access.