Stop Telling Me to Ask an LLM(blog.yaelwrites.com) |
Stop Telling Me to Ask an LLM(blog.yaelwrites.com) |
It makes sense to hate and despise that answer.
And yet, I'm not 100% sure I've never used it myself. I will have to watch out for that.
Huh, what if I don't want to spend time answering a low-effort question? I will have a look if the default answers/approaches don't work.
The more busy you are, the more valuable your time... the more expensive context switching is. When you are known as the person with the answers, your day is at least partially structured around getting people to leave you tf alone so that you can actually concentrate on getting your own work done.
There's a really toxic expectation that people who are senior should stop what they are doing XX times per day to help other people figure out their issues. Usually there's zero consideration given to how much each one of these interruptions takes away from them. Resentment builds cumulatively.
Before LLMs, this conversation usually went like this:
"What should I do?"
"What do you think you should do?"
"X"
"Do X"
There's only so many times that can play out before you really want everyone to just fro.
Anyhow, you should try hard not to "hate and despise" LLMs. Life is too short to invent paranoid reasons not to use the best tools available. That's another instinct you learn as an experienced dev.
Well the people who keep bugging me verifiably do not, so that's tough.
On the off chance they do, they either spectacularly self-sabotage, or treat the response like they do a typical message box popup. So I'll be asked to essentially read the same thing out aloud, only for them to go "ok-ok". It's beyond insulting.
I'll 100% keep telling people to ask an LLM when I suspect this shit. They do NOT respect my time and attention, and have robustly demonstrated so. But then these are the same people who cannot internalize the idea behind nohello.net either (gotta remind them every few weeks/months), and have demonstrated this kind of helplessness even before LLMs, so it's clearly a deeper issue, likely cultural.
It seems your peers might have a similarly low opinion of you, or at least I'd definitely feature that as one of the options.
People are really really tired.
Because of not just Claude, but also "the recession" "the strait of hormuz closed" "we've never recovered our economy from COVID" "everyone works from home now / the company is forcing us to all come back in" "FAANG had 10000 layoffs" "the global warming" "the <panic about XYZ>", our employers are making us work much harder, with a subtle but palpable panic in their emails, with WAY less promises of any kind of job security, companies that never had layoffs for decades are now doing them regularly, our githubs are flooded with people pointing robots at our issues to generate tepid pull requests, and at our pull requests to generate tepid reviews, and look shit is just crazy now.
So I think the whole "how would you approach this interesting problem..." thing is, for now, at least for me it feels a little bit on hold. Like oh that problem. How to scale? how to horizontally shard PostgreSQL? sure, real problems. But geez whatever we're building, it will be replaced in three months anyway. That's a hard problem you have there! I remember when I used to have problems like that, and my solutions sucked anyway and it was replaced with a node.js app two years later. Whatever advice I have, Claude is going to have 98% of it plus another 10% that I didn't even have.
This is all bad. So I think your post is possibly extremely useful. Maybe we should, for people we know and trust as humans in the real world, actually take the time and approach an issue as though we didn't have the Matrix to approximate it for us. I'm going to think about this and consider it.
> I already did.
They repeat multiple times in the article that asking Claude was something they already did. So this isn’t an anti-LLM article.
This seems to be a communication problem. The other party either doesn’t know that they’ve put a lot of effort into researching this already, or their trying to give a gentle let-down instead of saying they don’t have time for this.
For the first case, the solution is to explain what you did to reach this point. People are more interested in helping those who have already tried helping themselves.
The second case is more of a social situation with an infinite number of explanations. Some times you have to read the room and realize that someone may not be interested in having those conversations with you. Some times it’s only in the moment (we all have bad days where we want to be left alone) but other times it’s a signal that they’re not interested in discussing this topic with you or maybe even anyone else.
It's also helpful to the problem-solving and learning processes. For the expert, knowing what you've tried and how it didn't work refines the set of potential problems. For you, it's a free opportunity to get feedback on your methods from someone with domain expertise.
To me, such screams “I’m too lazy to do anything more than ask a LLM. I’ve tried nothing and I’m all out of ideas”
Show me you put a modicum effort and aren’t just looking to be spoon-fed the content from the first Google result that would have been found.
Same solution as case one though - don't be afraid to say "Claude said X but that doesn't sound right".
And having to do this on the corporate environment saps the energy and time of people that could be doing something productive by wasting their time answering a clueless person that asked an LLM about something they don't understand, got the answer they wanted (but that isn't real), and now are asking multiple people to prove it can't be done.
Here's an example, a PM decided they wanted to build a metrics framework, to track team success, with high level metrics. They asked claude to build such high level metrics (out of nowhere, these metrics don't exist), it happily produced hallucinated code that said it was collecting the metrics and the PR opened a pull request. Now we have to go there, review, find out is all bullshit and explain to the person that what they're trying to build doesn't exist.
So now we have to fight misinformation even on the clock.
She’s a journalist, and, from what I can see, a pretty good one. Someone that is fairly used to being able to talk to very senior people.
I suspect most journalists get told to intercourse themselves, from time to time, but it seems “ask Claude” is a new way of saying it, and, whether it’s meant, or not, a subtle insult. The kind that people like her especially resent. The intimation that she didn’t do her homework. It puts her on her back foot, and I think it’s meant to.
Us nerds can relate. How often are we told that we didn’t do something basic, with the onus on us, to prove we did? For a newb, that’s understandable, but it’s a real slap, for experienced pros.
It kind of feels like it though. We can be anti-LLM even though it's smart or helpful or whatever. It's reduced so many interesting conversations to this type of boring redirect to just "Ask AI"
Unpopular answer that the author seems to be dismissing: Maybe the thing that 30 years has taught this guy is that the LLMs can answer the question better than he can. Or that he can't give a substantive answer without doing research into it with an LLM.
>LMGTFY
I mostly saw LMGTFY used when the question was the sort of thing that a person would have to research but that google results had a high chance of getting with "I'm feeling lucky".
If you've already done a bunch of research, and already asked the LLMs, when someone says "Honestly, ask Claude", you should be able to come back with what results you got to your question and what you need clarification on.
I've been doing programming and sys admin for 40 years. When I run a coworkers question through the AI tooling and talk through the answer with them, it's because my 40 years of experience tells me that's the next step.
Compare:
— What's the best way of doing X?
— Ask Claude.
vs:
— I thought about this and found there are options A, B, and C of doing X, I like A more but C is the fastest; what do you think?
I believe a normal senior engineer won't suggest to talk to Claude in this case.
I was talking to a very tired friend of mine and she described renting a small tiller from Home Depot. They didn't know how to set the choke and so flooded the engine. They spent hours troubleshooting, no one at Home Depot knew anything about the equipment they were taking money to rent. They eventually figured it out and were tilling their backyard garden well into the late night.
I told her next time just ask ChatGPT. I would have done it myself if I'd been there. She hadn't even thought of it. But a few pictures and descriptions would have gotten them rolling in half an hour tops.
That is, the author asked for
the thing 30 years had taught him that a search engine couldn't
I have forgotten that thingDo you need to know if X is faster than Y? Do both. Measure. Sometimes the answer requires actual research.
Maybe you need a real Subject Matter Expert because it turns out that nobody ever published something on the internet about something so that an LLM could soak up the real world information.
Before the internet, we would consult with books. After the internet, it seemed faster to search and find answers in things like blog posts, (paid) articles, and CDs. Wikipedia and Stack Overflow are great resources. Maybe you need an answer from Hacker News - ask HN.
By relying on LLMs more than these other sources and allowing LLMs to write articles and posts to these other sources, we lose subject matter experts.
Add to that companies like Microsoft and Meta and others laying off and offering retirement packages to get rid of institutional knowledge as fast as possible, and we are headed towards a gigantic crash of knowledge.
LLM's are good at learning from whatever humans have posted online. But with the agentic workflows getting more popular, more and more problems those AI agents figure out are not posted online, and the next time another agent running into the same problems they would have to figure it out from the scratch again. It'd be nice if there's a mechanism these agents would share the lessons they learn with each other, which could save a lot of trials and errors and wasted tokens. Humans share knowledge online. AI agents should be able to do so too. The moltbook thing from half year ago could have this potential, but too bad it's flooded by spams.
Of course, to make this AI knowledge sharing truly work, there may need to be a peer-review mechanism to ensure the knowledge being shared is truthful, reliable, non-trivial etc. That can probably be all worked out if somebody (or AI agent) really put effort into it.
These should be opt out. Maybe have a skill that is similar and is opt out
junior developers on my team are often asking questions about our code base without even attempting to explore or self direct. “ask claude to look at <subsystem> and explain how its designed the key files and dependencies so that i can better understand it” is unsurprisingly effective and far cheaper than a couple of hours of opex
Doesn’t that mean his answer was that he, with all of his years of experience, would ask Claude?
"What did an LLM say about it?"
"What did the docs say?"
These are all better follow-ups than telling someone to do a specific thing.
15 minutes later a different coworker responded pointing out that everything Claude said was wrong.
I asked on slack for a reason! It's not like i don't know how to ask Claude.
Maybe the people in your life are just trying to be helpful instead of effectively saying "Go away."
But hey, sure, it's AI so you should definitely hate on them. FUCK AI!!1!! AMIRIGHT?!?!
How do people get jobs at jar opening factories by deferring jar opening to someone capable? These are your collages who have full time jobs, not monks at some monastery, go do that if that's what you want.
Its now a polite way of saying "I dont want to work on this project" without having to go through the effort of thinking hard enough about the problem to put the "go away" price on it (or even worse having to DO the work I dont want to do).
Then, after I've put in all this work, the developer just replies with a copy-paste of what the LLM thinks about my comment. I have no idea if the developer read or understood my point. I have no idea if he agrees or not. It doesn't just seem disrespectful to the effort I put into the review, it also leaves me in a difficult position as a tech lead because I have no idea if the person who is ultimately responsible for this code understands the code, my feedback on the code, or the changes that the LLM made to address my feedback. If you're responsible for a feature, I want to be able to feel like you're thinking critically about how that feature works. Right now, I just feel like you're blindly doing everything that I tell you. It also feels like I'm shouting into the void. We're colleagues, we should be able to have a conversation about technical subjects!
The +/-2000 line MR was bad when humans wrote it. It’s way worse when the human didn’t even write or read it.
And just vomiting automated CodeRabbit talking points back and forth at each other feels equally harmful.
Are we really tolerating turning ourselves into LLM rubber stamps?
Or even, even worse is when you get a PR from a co-worker, you spend a lot of time explaining why that's bad idea, only for the person to say "Sorry, my openclaw/etc posted that. I'll close it." Or even the opposite, you tell a co-worker: "Hey, it was a great idea to change X." and he says "X? What is X?... Oh that must have been my agent."
— Ask Claude.
Good advice obviously if it’s not being followed already but also likely over-simplifying the problem. Also a normal person on the receiving end would probe a bit about what has already been tried. Which to be fair makes the whole thing a bit weird and does sound more like she’s being brushed off.
Without knowing what/how they asked, it's difficult but I would be tempted to suspect this was actually a way to say "please stop asking me questions"
I don't think anyone in any industry regardless of seniority would redirect you back to AI assuming you're having a genuine conversation.
"Ask the LLM" is not at all a valid answer in a professional context where part of your job is to educate the less experienced, no matter how little effort is put in the question.
However, search has become enshittified, and I don't think LLM chatbots have achieved a similar level of certainty, so it may just be the end of an era for me.
“Why would you even want to do that?”
I dunno, man, it’s the job I was given. Do you know or not? No. No one does.
I take a week to figure it out, I come back to the forum to post my response, and it’s automatically closed as off-topic.
*sigh*
that's the point where i would want to start firing people. not because i want to replace them with AI, but because if they use AI to answer without contributing their own thoughts they make themselves replaceable.
(HTTP) http://linuxmafia.com/faq/Essays/smart-questions.html
I don’t need Claude to give a second opinion on something that Claude already ran through. It’s fucking useless and it’s a huge blind spot.
This shit happens when everyone in an organization is expected to use Claude to answer questions and do the initial investigation / analysis. Yeah, I did that. Now let’s get to work as humans and provide our expertise, and not turn it into an AI circle jerk.
It worked for a while. There is a GitHub action that you can configure to fail if the PR is too large.
But then we started a new project and I didn’t add it right away since it’s a small team and I figured we could use the honor system.
Since then there have been lots of massive PRs but there’s not much willingness to go back to enforcing the rule because it might slow us down…
It’s frustrating.
I have one project where there must be hundreds of pages of design proposals I have not read and will never read, because the author really likes having Claude generate complete design proposals based on incomplete understanding. So every week or two he sends me a new one, I spend 30 seconds skimming it, and then I tab back to Slack to ask him to explain.
I don't like working this way, but you know, I don't like doing rollouts either. It's certainly better than being a human rubberstamp.
Perhaps you could explain how this is different from rubber stamping, if it’s just 30 seconds of reading.
Does the conversation you have reveal what they actually want?
And what about the 2000 line change? Does that get stamped after someone talks about the change but without deeply reading it?
Do you know if there’s a way to incentivize them to lean toward doing the right thing first? Are the company and stakeholder goals and objectives in line with them making progress toward higher quality engineering? That is, are these juniors protected from randoms asking them to circumvent good process, etc?