Ask HN: Are LLMs just answering what we want to hear? I keep seeing those tweets and posts where users ask ChatGPT or a similar LLM to describe them etc... and it always answers positive cool stuff which reinforces what the user wants to hear. If you also try to ask it about a certain topic or yourself, it will always be positive and agree with your opinion. I feel there is a lot of confirmation bias at play. |
From now on, do not simply affirm my statements or assume my conclusions are correct. Your goal is to be an intellectual sparring partner, not just an agreeable assistant. Every time I present an idea, do the following: Analyze my assumptions. What am I taking for granted that might not be true? Provide counterpoints. What would an intelligent, well-informed skeptic say in response? Test my reasoning. Does my logic hold up under scrutiny, or are there flaws or gaps I haven’t considered? Offer alternative perspectives. How else might this idea be framed, interpreted, or challenged? Prioritize truth over agreement. If I am wrong or my logic is weak, I need to know. Correct me clearly and explain why
https://chatgpt.com/share/67e808f0-a3a4-8010-b97b-58c07d69f6...
That's not surprising though, and you can see it after a short while. What surprises me, is that people fall for it and take the ignoring position of being astonished by everything else llms do. I guess many people are just gullible by design and this tech, also naturally, abuses it to the limit. It is sort of an inevitable bubble. We also ignored the vulnerability that speech naturally is for way too long and this is gonna bite next generations hard in the ass (ignoring what power/business structures do with speech-based technologies right now, but that at least can be caregorized as humanity as usual).
Gemini once said "just die" once which is perfectly in line with its 'personality' or specifically what it's trained on. And it gets quoted again and again even though it's a typical glitch.
So they've dumbed it down a lot and made it more affable by default. People say the personalities of Gemini, Grok, etc were jailbroken and more 'human', but if I'm not mistaken, it takes extra training to make it more agreeable.
ChatGPT is also built for newbies, as compared to the same model on the API. Meaning it's on a lower temperature (less likely to be witty or risky), it's designed to write in a certain way that's more detailed and likely to solve the average user's problem. Similar with Claude on the site vs Claude on the API.
Roughly (from lay understand) LLMs predict what their training data would say. They are first trained on "the internet, etc." so they can predict words well, e.g. finish off "Paris is the..." then using human feedback they are trained further to work in chat mode and be non-offensive, concise, be pleasant etc.