Andecdote #1: I subscribe to poe.com so that I can pay one fee and have access to many models. I am still trialing a bunch of prompts that help me prepare for class. I am normally okay with Claude 3.5's responses, but sometimes I am unhappy with the output and decide it's worth the extra cost to use GPT-4o. So there is some differentiation between them, and there may still be room for both companies to exist. If I was forced to choose, I'd probably be okay with the cheaper Claude 3.5. Maybe Marc has a point and we'll see some companies fail if they can't differentiate themselves.

Anecdote #2: With image generation models (not LLMs but they are generative AI), there are some models I have learned to avoid with certain prompts. Dall-E 3 won't draw a good picture of Mickey Mouse no matter what- they probably kept Disney properties out of its training set to avoid legal disputes. But Ideogram will make a great picture of Mickey Mouse- so Ideogram has at least one area of differentiation over Dall-E 3, and they might survive the commoditization with that as long as they can sell that difference to people.