What I have found super useful actually is having models make a interactive 3d viewer in which I use move / highlight / paint (soft body painting directly onto the geometry for issues and different colors mean different failures). This gives a much better way to communicate the physical relationships and positions that are hard to get across in a labeled screenshot.

Its for sure still a lot of manual work so the "seeing-eye dog" description definitely holds. But I have found that after a couple of examples with the extra context the model gets much better at handling the problem and becomes useful.

He realises the current woe of things but doesn't realise he's been arguing with bot farms and teams of people hired just for this reason - to sow doom, arguments and engagement.

Around 10 years ago I noticed this happening on trending topics of Twitter - it wasn't that the opponents were stupid because they disagreed, it was that they were simultaneously intelligent and stupid in the way they spoke in a way that I realised I'd never seen in genuine people, making me realise it was probably different people or bots under one account.

If I realised it 10 years ago it was probably happening for at least 15. He wasn't better than these people he was falling into their trap

LLMs generally only debug systems by reading the code with whatever information you give them in a prompt. The image in the article is meta-prompt - the prompt is whatever comes from the vision model the AI happens to use to 'understand' the red circle annotation. That won't work. To successfully debug what's going on it will need much better state information. Has the 'shelf' been explained to is? Is the contrast and lack of shadows in the image messing up the vision model? Why isn't the 'lid' in the image? And so on.

LLMs are clever but they're not magical. Treat them like a naive junior dev. Give them enough data about the state of something to understand it properly.

"Spot the bug in this code" when the code can be looked at and pattern-matched against bugginess is something they seem really good at.

Some parts of debugging, like "Here is this logfile, what do you think is going on?" are also surprisingly good.

It's that thing kind of in the middle -- I know it when I see it honestly is the best way I can put it into words. An example from recently, I'm receiving some bad data on a network message parser. Immediately I don't know whether it's a my-side or their-side thing, but I know if I try and just vaguely describe the behaviour to the LLM it will start churning tokens.

My current approach to problems like this is -- I need to tell the LLM what it needs to do to give itself the data it needs to solve the problem. My first reaction now isn't "It's not working, there's a bug, it's not doing X". It's "Okay, this isn't quite working properly; I need you to add some debug logging around X, Y and Z so we can figure this out". That tends to avoid spirals and get me out of the situation much more quickly.

The seeing eye dog analogy is pretty apt actually. I would love to see some transcripts from the author if they are able.

Edit to add: I think the 'thing' I'm alluding to might be -- if I have trouble expressing the buggy behaviour clearly in words, then I know it's probably going to be a fair few back-and-forths with the LLM to get something; the harder I find it to concisely describe, the more risk that it'll fall into a pit. Doubly so if I offer up a hypothesis which turns out to be wrong.

I've been using a few different "smart" LLM to work on an analysis, parsing, search and correlation tool that ultimately deals with a 5.5GB on disk (with indexes) mariadb database that has its origin as a federal government department's 905,000 row plain text CSV file.

There are a ridiculous number of data entry errors and just plain weird fuckups in the data origin that don't seem they will be ameliorated any time soon, so automating the drudge work of cleaning it up and rectifying it into something usable is a textbook case for this. Very pleased with the results so far.

Also.. uh.. isn't this.. good? I thought AI was to steal all our jobs.

___

Beside that, kinda weird self-description.

Isn't the computer executing your commands and you're just filling in where it cannot do that?

Being that dog implies that the computer is in the driver seat.

I mean it's supposed to be a joke I guess, but I read it as one that leaks internal metadata which seems to be incorrectly calibrated.

I call this "thanoscoding" for two reasons:

1. "Fine, I'll do it myself"

2. In the past I found I have to "snap away" the mess the LLM made in order to start afresh from a known good state (generally with git reset). But that was 1-2 generations ago when it comes to models. GPT6 Astra probably does things right the first time, 90% of the time.

Coding was never the reward. Acting as a translator for a machine is far worse than allowing the machine to solve the mundane parts and leave you with bigger building blocks to play with.

After 25 years I have to confess I hated being a software engineer. It felt like grinding in a video game.

Now I can finally create and innovate at the speed of thought, and I'm very grateful to have this technology now.

Astra is already good at taking screenshots and acting on it (part of the agi claims).

You are doing those junior engineers a disservice with this message.

I'm usually trying to go in the other direction with them - you don't need to be able to reproduce it yourself or see the log files to narrow down or find the cause of an issue.

Logs/etc are just additional tool that make it easier.

Lost track of how many juniors I've seen throw their hands up and say they can't progress because they don't have logs/clean repro

For LLMs, this is likely to be disproportionately effective as well: especially because they don't really build up a persistent view of the codebase, they're generally re-reading it each session, and they tend to be surprisingly good at predicting the behaviour of code.

(That said, knowing where and how to gather more evidence to make things clearer is a pretty core skill in troubleshooting, so it's generally good advice anyhow)

This is exactly how I debug things. When I first started working I was forbidden from using the debugger “you won’t have a debugger in the field”. Being able to read the code, spot the bug and tie it to the symptoms; that’s a skill. A damned important skill.

Giant corpos hold every sliver of your so called freedom, you dont innovate, you repeat what others created before you. You use a tool that shackles your thoughts and creativity in a never before seen way, what you perceive is an illusion of liberty that is no present. Without others you are nothing and they can take it away any seconds, you are an addict, not an innovator.

Serious question - what do you do for fun? I find fishing with friends enjoyable, and using my hands tidying up the old place I bought. It’s not innovating software I’ll never use but I’ll never tire of writing the same old ASP that delivers my clients the results they’re after.

Yes, when I think of OpenAI and Anthrophic and Google and Meta and any AI labs and their intentions, i cry a single tear for how these great instituitions are working so hard to bring freedom for humanity.