Yes, and: the most important time to use your brain isn't in the loop.

That _is_ valuable, and will remain so, but the temptation to turn it off is there because the loop is good and getting better at what it does. Once you're in the loop, with the rare high-value exception of catching total mistakes nad redirecting, you're mostly choosing between similar-yet-reasonable options. This isn't so much right-vs-wrong as relativistic optimization. Letting the loop do the work will get you a mediocre result quickly, and that's usually fine.

The most important time to use your brain is _before the prompt_. Once you engage with your LLM and agent, you start biasing yourself, and its reasonable suggestions constrain your visibility into other options, other worlds.

First, use your brain. Then write the prompt.

It doesn't help that this is being forced from above. I agree you can't turn off your brain, but any time I argue that AI isn't the right tool for something management gets angry. There is a constant push to fully automate things, regardless of what those things are.

Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.

I think a lot of people are not able to reason to this point and then adjust behaviour accordingly. They probably reason to the point when pressed for it, but cannot resist the low effort and low resistance route of asking AI, and then probably make up reasons why in their case it's fine (typical behaviour to address cognitive dissonance).

Then again, for many people their job security doesn't depend on what work they do, it depends on how well they are embedded in the organisation. Before AI you also already had plenty of people doing their job at a questionable level, yet never seem to get fired.

Even pre-LLM: I've found a lot of colleagues (mystifyingly to me, both seniors and juniors) who really want to be able to turn their brain off when doing investigations into complicated cross-system behavior (feedback loops abound), and they don't like hearing that it can't be collapsed into a runbook. If a drinking bird (https://en.wikipedia.org/wiki/Drinking_bird) on an Enter key could do it, I could rig something up so I wouldn't need you!

You are a photographer from the olden age. Taking a photo back then took forever. You are standing on a hill trying to get the perfect picture of a sunset. You measure and sort and calculate angles and light. A tourist stands next to you with an high-speed iphone auto-settings camera. They spin around and get thousands of photos from different angles while you are thinking and measuring your one shot. That’s what AI does. It gives you many shots at the same problem. The thinking guy with the one approach will likely lose a photo competition to one of the accidentally better shots of the person who tried a lot more. Make and filter now may be a better approach than overthink and try only once.

Airline pilots and train conductors are basically meat proxies 95% of the time and they are still employed. Still necessary for when things really go wrong.

The version of "meat proxy" I've heard is the person who forwards a machine-answer to someone else who didn't ask for a machine answer, and could just as easily made one themselves.

Calling someone a "meat proxy" for being the human in the loop is wild.

"What reason is there for the company to employ the meat proxy?" Maybe some form of liability, accountability...

New concept: the AI patsy

Thats not the problem of LLM/AI.

The problem is, that digital work, the new frontier of employment for a lot of epeople, got and will continue to get A LOT cheaper, faster and more accessable.

The PO who always complains about someone? They will just vibe stuff. It might even be shit but it will work good enough until LLMs/AI/AGI/ASI is good enough or better anyway than an avg developer.

Man i have seen so so much shitty code even in software companies.

You know when your HR software takes seconds to load? Yeah an LLM can do that today too

I would like to argue opposite.

Developers like to "turn their brain off" during "super hard debuging" sessions. Mental flow state is a well documented as pleasurable experience. It is like a drug!

Auditing and reviewing autogenerated commits, is hard gruelling mental labour, compared to that. You do not even get a credit as an author! You can not get into groove to get your fix!

     Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee

I would argue that there's still an important role for human employees as compliance ablative armor.

The human's purpose is to keep an eye on the LLM, occasionally intervene/interfere with its work, and most importantly, incur social/legal liability for anything that the LLM does wrong.

Kind of like the backup drivers used by self-driving cars.

Agents are still a bit to eager to please, so they are still missing cases when the better answer is to push back, change course or refactor.

But when the human suggests changing the course, the LLMs will often agree the new direction is better and start heading there...

Turning your brain off is usually a defense mechanism to protect yourself against something bad. dissociate during trauma etc. lighter dissociation points to the interaction trending toward badness, not goodness. Seems like the mind and body is telling us something that we shouldn't ignore

I think this is the dissonance that has been making me feel insecure / depressed around the state of AI coding. On the one hand I am way more productive. But on the other hand it is almost requiring more of my brain than before, because I am having to determine what is good output and what is slop. But that's not the part of my brain that I particularly like engaging — I like understanding something deeply. But in this case, that is just not possible. I like to think that as the human-in-the-loop, I am supplying the requisite _taste_ to the design and overall structure of the program. But at what point does my taste get taken over by the LLM as models improve?

Another problem is that when something breaks then I am very limited in my ability to reason through the breakage. It's necessary to go back to the LLM and ask it why it broke. At which point, I'm able to follow the reasoning — but not in any durable manner where I'll be able to remember it. To me, it's sort of like the difference between memorizing a passage and having a ready reference at hand at all time. We had this same problem with information before — now it's come for our reasoning.

That's the smallest dan luu post I've seen.

There are a lot of programmers who are working with chatbots and watching YouTube simultaneously for the first time, simply matching the rest of the economy for the nature of work.

If you can do your job while watching TV your brain is turned off.

> Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee.

If this is true, turning off the brain... is completely rational. Why burn out the brain for something happening either way? It's not like being fully engaged is going to result in anything that much better for the employee. In a world where code is this disposable, memorizing a codebase that won't recognizably exist in 3 years is also stupid.

> What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee.

Company is pleased! Cost savings!

Alas, the joy doesn't last long as pretty soon after the company realizes that after an LLM replaces almost all their employees, the much easier task of replacing the entire company begins, because the company itself has just become a much thinner meat proxy for getting an LLM to do whatever it is the customer relies on the company for.

Customer is pleased! Cost savings!

Alas, the joy doesn't last long as whatever savings they have from replacing the company with their own LLM loop is offset by whatever it is they do to earn income being similarly impacted.

All of the moats degrade together though they may fully give out at slightly different speeds.

Maybe, but the parts of my brain I'm still using are a lot easier to use.

I was dreaming to be promoted to a cushy management role where I could talk bs for a living.

I managed to attain 'hands-on team lead' and later an ill-fated 'bootstrapped cryptocurrency CTO' role... Nobody wanted to pay me to talk as a career. Now they do!

So I'm very happy about that.

I feel contempt for all the people who had it easy and 'chose' to be engineers voluntarily.

I still don't get who is this guy and why anyone should care.

It's an interesting idea. I'm not sure I agree.

Here's the thing I'm finding - LLMs allow me to do more coding when my brain isn't firing on all six cylinders. It means I get more code written because I don't have to be "in the zone" to exhaustively hunt down every piece of API and dependency and have my flow continuously broken up by missing information. The code written in that state is lower quality, but I can patch up five-cylinder-brain code faster than I can synthesize code from a blank page outside of my flow state.

When throughput is an issue, you have to think judiciously. I'm definitely not saying you shouldn't think but you have to apply human oversight effectively. For me that means front-loading through dialectic planning, and maximizing guardrails.

There's some software that really doesn't require my brain. I gave Astra an ssh key for root on a little Pi Zero with a microphone attached and asked it to make recordings of a siren going off in our neighbourhood, based on a couple of sirens. This would have been a great deal of work a couple of years ago, now it's a few tokens, and honestly, it doesn't really matter if it's shitty code, I just want the timestamps.

Then there's the software that I write for a living, and almost my entire day is now spent making hard decisions. /grilling gives me difficult technical design decisions endlessly and I am forever doing product management on features from agents. My brain is back in CTO mode, which isn't what I'd planned for myself after I'd left that particular cursed path, but fine. I am doing the opposite of turning my brain off, because I no longer can just chill out and mindlessly write code.

Both these states can exist simultaneously. But my brain work has shifted into something different, while getting no less intense.

trends like these remind me very much of how true some of the predictions in Douglas Hofstadter's "Anti-Intellectualism in American Life" were: https://ia802903.us.archive.org/6/items/richard-hofstadter-a...

he was writing in reaction to the McCarthy era which, arguably, is very similar to kind of bombastic nativism we're seeing now from our political elites. his argument essentially being that there are cultural and institutionalized forces that are driving people to, well, turn their brains off - specifically to no longer be skeptical of what they consume, to not want to refine and improve their toolbox of analysis, to not want to understand how things work for the sake of knowing

and that these acts were deliberate - a populace is so much easier to control when it's complacent and unknowing, who believes, unskeptically, that Oceania has always been at war with Eastasia, unquestioning about who was writing the PR missives that feed the news cycle (see also: https://www.commondreams.org/opinion/what-is-copaganda), how verbatim they are, and what the real intent of the authors were

like, I get that you don't want to put in effort for a job you don't like. but that effort also makes you better at your work. it makes it easier on you, it lets you create less incidents, it empowers you to be more agent in your own life, to, who knows, have the guts to start a consulting gig because your skill-sets actually are better than the average

there's so many benefits from just being more diligent, longterm, proactive benefits. but that's not everybody's extrinsic motivator and it's so much easier to just be Skinner-boxed by social media companies into doomscrolling, I suppose

The obligatory XKCD, panel 2: https://xkcd.com/568/

Suppose I do something myself. I reap all the benefits, but it takes a lot of time, I may not have the expertise to do a good job, and I get all the blame if something goes wrong. Suppose I go to the park and find one of several guys who would do it for free. I get all the of benefits, he might do an excellent job, but, because there's no contract, I still take all the blame.

Suppose I actually hire a professional to do it. I receive fewer of the benefits. However, he's much more likely to do a good job, and the legal system provides a variety of ways to hold him accountable for the work.

Suppose I hire a professional, but then insist that he use legions of guys from the park to do the actual work, since that's going to be some much faster and cheaper per quantum of work. I also tell him that if he's unwilling to do that, I'll find someone who will. I also tell him that, since he has so much more unoccupied time now that the original work is in the hands of all those other dudes, he gets to supervise still more work done by other swarms of guys from the park.

At this point it should be clear that I'm not interested in quality work, but in having someone poised to absorb blame. I want a patsy.

Now replace "guys at the park" with LLMs, and replace "free" with "nearly free, but with no warranty."

> Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.

Some would tell you that you need to find a different job with a better boss, but I'd say that this is quickly becoming something very hard to escape from. Investors want line to go up, and they all talk to each other about this "AI" thing that lets them cut staff and keep productivity, so naturally we're all getting it crammed down our throats.

My company is doing this, too. It’s the reason I’m submitting my resignation notice next week.

Same. My team lead sits near me. He keeps checking if I'm using claude, or when we talk he pushes me to use claude.

It's have gotten to the point that I wanted to work on something manually, so I booked a room in the office. Just to not be at my desk, because I'm afraid my claude usage will be reflected in my performance review.

I don't buy this analogy and I think it takes a depressingly reductive view of creativity. In both cases, the photographers had to take the time to go to a place, see a view they thought was worth capturing, operated their equipment to get a shot, and then had enough discernment to recognize which of their final products were worth something. Both the traditional photographer and the modern photographer can create images of value in different ways.

If you want to shoehorn AI into this analogy at all then no one is even bothering going hiking at this point, let alone seeing anything worth capturing. It would just vomit up endless permutations of statistically average images that look like the kind of pictures people take of landscapes.

This analogy seems strained to me.

The credulous LLM users Dan Luu is complaining about don't take the equivalent of thousands of photos of the same sunset. They take a couple photos at most, say "LGTM", and move on to taking a photo of something else, even if their couple of photos of the sunset aren't great.

As someone who will often compare different things, I think doing that usually takes time. I think LLMs can help reduce the time (especially if it's programming, not necessarily so much in other areas), but I'm not seeing many people use LLMs to try a large number of independent approaches and compare them.

The tourist has another advantage over your photographer: experience. They have thousands of photographs to look at and thus get a better scene of what makes things good. That is if you took both to the same spot and gave them one photograph (the phone's memory is almost full) the tourist is likely to get the better shot because they have a better idea of when to pull the trigger having seen all the possibilities.

I think this analogy works with respect to stuff like coding, engineering, etc, where the AI is capable of validating the success of its own efforts and where the result just has to "work", regardless of its elegance or aesthetic quality.

I also think the analogy works if we're talking about average photographs, or even very good photographs, that are conventionally well-composed, balanced, etc, but lack the special spark that we associate with true art.

But if the photographer on the hill is a real artist, it's hard for me to imagine a current frontier LLM winning the contest. I suppose it may stumble upon a winning photo sooner than a roomful of monkeys would, but I believe the number of attempts would still be so vast that it would be more efficient for the human to make one single attempt at taking a photo, rather than dealing with prompts and sorting through the millions of AI-generated options.

EDIT: I also think it's very easy to underestimate the size of the nearly limitless set of possible "solutions" to the photograph, and also easy to overlook the constant feedback loop in the artist's mind as he considers each option and looks through the viewfinder, while the LLM is shooting blind the entire time.

> The thinking guy with the one approach will likely lose a photo competition to one of the accidentally better shots of the person who tried a lot more.

If that were the case then photo competitions would have been upended by now by tourists who accidentally took masterpieces with their phones.

I can take as many thousands of pictures as I want but, without taking the time to critically review them to see what works and what doesn't, I will rarely produce a prize-worthy picture. And then there's the risk that I'll discard it anyway because I don't have the eye to separate the prize-winning picture from the Instagram shot all my contacts are publishing.

Even in the late 19th century "bracketing" was already a photographic technique where a photographer would take several shots of the same subject from various angles and with various settings. Digital cameras may make this faster and cheaper, but the one who employs thoughtful practices in their photography is far more likely to win that competition than the one who just (to paraphrase your example) spins to win.

Maybe so, but for the good shots by the thinking guy, I suspect he would derive longer term emotional gratification from how well they came out. Effort and investment adds to the experience.

Making things easy is good as long as it is not too easy, then it's Spotify muzak.

>That’s what AI does.

Except it sometimes shows you a photo of something completely different than what you're looking at for or alters the landscape to "improve" it. Maybe that's not a good analogy.

You are a god. You step on the puny human and create a thousand sunsets. Maybe the photographer wasn't needed in the first place.

This ignores the reality that many people are, unfortunately, taking that one iPhone picture, sending it to everyone in their contacts list as if it’s art, and then moving on to the next slop shoot.

Once upon a time, in Switzerland, there was a "chronometry contest" putting the best watchmakers of Switzerland against each other to make the most accurate mechanical watch and have it certified by the observatory of Neuchâtel (observatories were at the time the entities in charge of giving the most accurate time, based on reference star observations).

It worked for a time, and manufactures employed experienced, legendary even, watchmakers and specialized "watch setters" to create and tune to the best possible extent a few experimental watch movements. They would be sold as collection pieces, the firm would gain boasting rights in its advertising, and the experience informed the design and tuning of more mainline movements.

By the end of the history of (the revival* of) this concours, the winner was always Tissot, a second-rate company. What hides behind Tissot is ETA, a huge watch movement manufacturer for all of the Swatch group (dozens of brands, from Omega to the worst no-name stuff you can find).

What they were doing to win was just to pick the best watch movements out of the millions of basic, bland, uninteresting "workhorse" ones they were producing each year. Just by chance, there was always one combination of the hundreds of components, each with their manufacturing tolerances and variations, that would fit all together to make an exceptionally accurate movement.

In the end, yes, Tissot won repeatedly. But the concours is dead. And Tissot is still a second-rate brand making second rate watches. Nobody is interested by their watch movements, no collector, no historian, no customer.

And how do you get a person able to nail "that one shot" when the moment is fleeting? Your "tourist" is unable to do it, because he never put any deliberate practice in photography.

Incidentally, analog film photography is having a huge revival, and Instax is still Fujifilm's best seller. Maybe there is something missing from "I did an award-winning photo but I have have no clue how, lol".

Is there anybody here who wants to have their portrait taken by the "tourist"? Is there anyone here who would prompt an AI to generate their portrait, apart from Ghibli-style remixes (which are funny five minutes, and stale a month later)?

That's an interesting analogy, because there's still a spectrum. Photography's big advancement was the move to digital. Prior to that, you had to know what an F stop is, different developer chemicals, manual focus. These days, yes, an iPhone is actually quite good at taking pictures. Apple invested billions for that to happen, it wasn't an accident. SLR cameras are still there though, you still have wedding photographers. You don't have the mall photographer for once a year photoshoots so much anymore, however. Everyone just takes pictures everywhere as they need. Software is the same now. I need a program to do a thing, I'll just have AI whip one up instead of looking for software that does something that but has 100 more features I don't need. Gone are the days where customers would come to the mall to get photos done and buy software. (Though that world's been gone for a while.) Software is now only a means to an end, so better find the customers that have needs and embed with them instead of having them look for Photoshop.

Or a different way around, Asimov's The Machine That Won The War, 1961: https://xpressenglish.com/our-stories/machine-that-won-the-w...