GPT5 is worse than 4.1-mini for text and worse than Sonnet 4 for coding

10 points by hitradostava 281 days ago | 16 comments

It seems that OpenAI have got the PR machine working amazingly. The Cursor CEO said it's the best, as did Simon Willison (https://simonwillison.net/2025/Aug/7/gpt-5/).

But I've found it terrible. For coding (in Cursor), it's slow, fails with tool calls often (no MCP just stock Cursor tools) and stored some new application state in globalThis - something that no model has ever attempted to do in over a year of very heavy Cursor / Claude Code use).

For a summarization/insights API that I work on, it was way worse than gpt-4.1-mini. I tried both mini and full gpt5, with different reasoning settings. It didn't follow instructions, and output was worse across all my evals, even after heavy prompt adjustment. I did a lot of sampling and the results were objectively bad.

Am I the only one? Has anyone seen actual real-world benefits of GPT-5 vs other models?

cellis 281 days ago |

I have not found it to be better than Sonnet. I wrote this on another thread:

Claude Code certainly not as easy to engineer with, though it is less expensive. For instance the @feature isn’t as robust as cursors ime. Also no shift+enter is quite a pain. Linting doesn’t “just work”, cursor with Claude 4.0 max is really thorough, I think even better than GPT-5. Not that Sonnet is better but that whatever “ensemble” of models cursor uses with sonnet seems to both adhere and tool call better than with GPT-5. GPT-5 often says what it will do and then says “say go and I’ll go” or says “you should run command x”, but doesn’t just DO it. Also for bug fixes in difficult codebases nothing beats Gemini 2.5 pro

blurbleblurble 280 days ago |

Really struggling with GPT-5 in all the clients I've used (currently using claude code router). This is the default openrouter endpoint. It's opaque, slow, ridiculously terse, botches basic tool calls and edits, pauses constantly even when I ask it not to. It does seem "sharp" at analytical tasks but it's just clunky to work with.

I'll try with cursor CLI but am hesitant to ditch claude code.

softwaredoug 281 days ago |

I feel like they should have let GPT 5 overlap in experimental mode for a month or so. It took a while to get the kinks out of GPT-4 until people trusted it. Just switching it on is really hurting their brand.

The fact they didn’t do this makes me think their finances are in very bad shape.

hitradostava 281 days ago | |

I agree, I just don't understand how the team at Cursor can say this:

“GPT-5 is the smartest coding model we've used. Our team has found GPT-5 to be remarkably intelligent, easy to steer, and even to have a personality we haven’t seen in any other model. It not only catches tricky, deeply-hidden bugs but can also run long, multi-turn background agents to see complex tasks through to the finish—the kinds of problems that used to leave other models stuck. It’s become our daily driver for everything from scoping and planning PRs to completing end-to-end builds.”

The cynic in me thinks that Cursor had to give positive PR in order to secure better pricing...

8thcross 281 days ago |

I tried it with cursor-agent, their cli - and it generated better code than expected. YMMV. It was more thoughtful and strategic than the other frontier models.

hitradostava 281 days ago | |

Planning was ok for me, much slower than Sonnet, but comparable. But some of the code it produces is just terrible. Maybe the routing layer sends some code-generation tasks to a much smaller model- but then I don't get why it's so slow!

The only thing that seems better to me is the parallel tool calling.

canerdogan 281 days ago |

GPT-5 isn’t really a brand-new model in the way people think. From what I’ve seen, the goal was more about reducing costs and unifying the interface than releasing a totally different architecture. Under the hood it is still routing to models we already know, just picking what it thinks will give the “best” result for the request.

That can be fine for a lot of general use cases, but if you’re working in specific domains like coding agents or high-precision summarization, that routing can actually make results worse compared to sticking with a model you know performs well for your workload.

hitradostava 281 days ago | |

Thats not what OpenAI are claiming. They are claiming that there are two new flagship models and a router that routes between them.

"GPT‑5 is a unified system with a smart, efficient model that answers most questions, a deeper reasoning model (GPT‑5 thinking) for harder problems, and a real‑time router that quickly decides which to use"

cranberryturkey 281 days ago |

it solved a huge bug i've been struggling with.

hitradostava 281 days ago | |

Had Sonnet 4 not been able to?

cranberryturkey 281 days ago | | |

No, it kept going in circles....spent like 3 weeks trying to fix it. Got access to gpt5 yesterday and all major bugs are resolved.

revskill 281 days ago | | |

Sure.

gaws 281 days ago | |

What was the bug?

tim_angus 281 days ago |

And yet the media keeps using the term "exponential improvement"...