Seriously? If so, which model produces actually human sounding output?
I guess my thought / question is: this is your code. Why are you submitting the code to the git repo like that, with all the wrong text? I get making mistakes here or there; but ... everywhere? Enough that it's causing huge problems you can't quickly correct?
And ... aren't you lucky you still *have* the context in your mind? What if you hadn't caught it, or someone else had done that and you inherited that code with those wrong comments that you let through the PR and into the main branch?
1. It writes SO MUCH code, in minutes, that theres no way I as a human can review it.
2. I am not very motivated to read/review this code anyway: it was cheaply written, by some _thing_ that is not going to improve from my feedback.
3. If you DO review it conscientiously , it becomes a never ending thing: you keep finding issue upon issue.
4. If you report the issues for fixes, the fixes normally fix that immediate issue, and typically add another parallel path/another option/another 100-odd lines of code, instead of a structural fix.
5. If the code base is large, and if you let AI write a meaningful amount of code in it, its no longer _your_ code. You lose the depth+width of understanding needed to reason thru things mentally, cause you no longer KNOW enough about the code.
6. If you do review seriously, and either fix things yourself, or have the ai/harness fix the issues for you, the rounds of fixes take so long, if you look back you realize it would have been better to just do it yourself in the first place.
My personal opinion there are really only two choices:
A. If you want to build FAST, using an agent, just let the agent write tests, validations, extensively, have it keep running them (it likes to call them gates), and just let it loose. Give up on the idea that its your code, and that you understand it. The bottom line becomes: does it work, and WHEN it breaks in weird ways, just use the agent to find and fix the issue (probably breaking something else in the process).
B. limit AI use for non trivial, prod-quality projects to limited research, very tiny targeted changes, write most of the code yourself still, and review every line. You won't get much speedup, maybe 20-30%, but it will still be YOUR code, and you will still be able to reason about it.
I used to. I've stopped in recent months, only because I just can't keep up with the pace it's churning out the code. If I was to read it, it would take multiples longer to develop anything, maybe into orders of magnitude longer. Occasionally I'll dip in just to get a sense of things, especially if it's struggling with something, or at the opposite end, if it's completely trivial. But I'm hardly reading anything now.
According to my enterprise architect I shouldn't be reviewing code, it's a waste of time in this new reality.
I'm still doing it because it's going to be me answering that 3 AM call. But I don't know for how long I'll be allowed to swim upstream like that.
[1] Not sure if I read that or came up with it.
The absolute least worst outcome is they chew up your token budget. But what tends to happen, and this is much more serious, is they poison future work and make further modification of the codebase more burdensome and error prone.
Agreed.
Probably better to add whatever instructions it takes so that the agent doesn't write comments, at all, ever. If you need comments to understand the agent's code, the necessary information should already be in a conversation somewhere; and you should summarize it yourself, because the comment will be for your own benefit. Otherwise you are letting past-agent steer future-agent more or less at random.
Seeing LLM comments in other peoples code is very upsetting because theres just so much meaningless noise.
```
## Comments
Use comments extremely sparingly. Most comments should be at the request of the user. When something warrants a comment, keep it to one or two lines: what the code does and why it's necessary. No background narrative, no replaying the investigation or failure mode, nothing a test name or the commit message already says. Applies to specs too. If a comment needs a paragraph, make the code clearer instead.
```
The comments Claude was leaving got absolutely out of control. Just lines and lines of LLM drivel that was barely intelligible and not remotely relevant to what a code comment should be used for.
It deleted the comments, and when I asked why it did that even though I expressedly asked it not to, it responded that me prompting it in the first place explicit permission. I have no idea if that's the actual reason or just some post-hoc explanation.
But I genuinely don't think it's possible to just have these things be completely, 100%, indpenedent and also solve deep problems that need to also be understood by people in a people-based organization context.
Just be clear, it can't know, and by asking you're just making it roleplay as someone excusing themselves.
It's very unlikely that the choice to remove the comment was driven by an internal monologue based on learned criteria that it can refer to. The sampler most likely picked tokens to remove it while writing the patch, because that's what the statistics modelled, and that's it.
AI can't read your mind, and it will read what you (or anyone else) wrote.
I feel like I'm losing my mind, what the fuck is wrong with all of you?
They spent 3 years pretending that their magic robot made them 10x harder/better/faster/stronger than the lazy peons rubbing shoulders with them and now the bill has come due.
I started practicing this for the stuff I'm interested in ... and the result is that I generate a lot of ideas. 99% of them are garbage.
Before AI, a conversation with colleagues would help me weed out the garbage, and possibly give me a new perspective ("here's something surprising about what you just said!")
Now, AI plays a helpful role in helping me weed out the garbage for the subset of ideas that I can test empirically -- and sometimes it does give me a new perspective, but with significantly more noise than my colleagues, who would take me right to the point that would be useful for me to see.
I've never felt like I needed a second brain ... because if I had such a thing, I think I would be keeping the 99% garbage around -- instead I just let biology GC it.
> When people think AI did the creative work, task meaning and effort decline
https://www.brookings.edu/articles/when-people-think-ai-did-...
that being said, i got a big kick out of feeding my daily notes into my local model. i had some more diary-like entries that i would never share with anyone, but it was extremely funny to have my local model take a 3000 word recap of what i did in a day and summarize it like "writer reflects on their job: they worry about their performance. meals: the writer ate oatmeal, and a sandwich. family: the author contemplates their fathers absence growing up."
ive developed a really large corpus of written work over the years and am hoping to fine tune a model using my diary as the training set. i think this would yield hilarious results. keeping it all local though, i drop my passwords in there at times and i dont want that leaked.
Yes! And that's because you can't really delegate ownership/authorship to AI (at today's capability levels). Coding agents might have been RL'd out the wazoo to write code to solve well defined problems, but given a situation the onus of making sense of the ambiguity and identifying the problem -- and even crafting a skeleton for an acceptable solution keeping in mind tradeoffs -- needs to be done by a human taking ownership.
I wrote about this recently from a slightly different angle [1], but the core idea is the same -- which is why it is hard to take over an idea mooted by AI. Before even writing/implementing, the AI has internally resolved ambiguity with several decisions which are arbitrary, and match neither your mental model nor the actual ground reality (which you need to spend time understanding in order to productively interpret).
[1] https://woventhought.substack.com/p/ai-assistants-need-adapt...
Might be a good candidate for a pi extension.
I love you guys, but number 1 is numero uno, and if the chips are down and I'm working for free, I'm going to be working on myself.
It used to be a mess, but with OKF (Open Knowledge Format), things are more manageable and navigatable (for me as well).
Consider AI vault as a scratch pad that AIs use, not as something you put your own thought into it.
Having an AI write notes would only produce text, but the actual notes are mostly a side effect — an important one, but not the most important one in note taking.
I think hard about related notes, and peek at them when linking, sometimes finding interesting one-off connections. This makes me think further and sometimes expand the topics. Automated linking would not make me think, it wouldn't make me learn. It would just produce yet another pile of information that's already available online anyway.
The reason to not add it is that the quality of summarization will likely only in prove in the future, so it’s better to do it on demand whenever needed.
For the Original title, I wish they added the qualifier "generative" for AI. You may well use e.g. embeddings to semantically search through your notes, and there are only gains in that.
Having an AI write notes for you is however of course completely pointless. The act of writing notes is 80% of the reward.
The exception is transcriptions and summaries of transcriptions for things like lectures; I find those useful even when I know writing them myself would be better.
Of course the problem with this for me (I'm a state employee) is that said transcript is a public record.
It's so hard to finish an idea that is not yours and is just suggested by AI.
LLMs don't have sense experience, so it doesn't have real reasons to prefer one option over another when nothing has been specified in the prompt. It can only guess based on what's most represented in its training. "Ideas" are only being simulated by an RNG; the AI can "suggest" (output) them, but they don't "have" the idea either.
If you couldn't make the decision yourself in the first place, pulling a slip of paper out of the hat only helps in the cases where the choice really didn't matter at all. Otherwise you'll look at it, frown, and still feel stuck.
As Claude said to me when I said the same thing - it's not really its idea you had to ask the right questions. I find this true in some ways - asking the right question and finding out what the AI's have hidden inside them is like panning for gold imho.
It used to be that we only recognized a tiny fraction of the actual ideas involved, because most of them were intermingled with execution. We would sit there and bang out code at single-digit WPM while trying to keep block diagrams of process memory (or UML diagrams, or whatever else) in our heads, and then run the compiler and/or the tests and realize we'd gone wrong anyway.
Now we have lightning-fast "execution" devoid of real insight (just making choices more or less arbitrarily as they come up) because the LLM can do more and more without human intervention, but for some tasks it goes off the rails because the insight actually is necessary.
1. Those who kind of semi-obsessively write down every little thought, semi-obsessively catalogue everything, etc.
2. Those who use the idea of "writing helps you think", where the second brain functions in support of this
Personally I find 2 quite valuable, and although I could "think about programming all the time" (or one of my other interests), I find it helpful to have f.ex. exact quotes from some book at hand (and various other Obsidian-related things)
I keep obsidian vaults mostly to augment recall, and often all it takes is a small cue to reveal that thing I couldn’t remember. I’ve found LLMs helpful for translating my question into search criteria - “agentic search” is magic.
One, it forces you to do a crude approximation of critical thinking. It allows you to examine and organize your thoughts, either physically or digitally. It isn't the same as being able to sit and break something down in your mind, but for those who struggle with that, a "second brain" allows them to reap some of those benefits.
Two, it allows you to turnover things quickly in your head. By training yourself to immediately dump out an idea into an organizational system, you don't have to hold so many things in your head. Furthermore, words are immutable and the thoughts in your head aren't.
But for me, the good thoughts usually rise to the top. And there are a lot of things I'm thinking about that are so vague, it's just a feeling. And trying to write those down robs those thoughts of the long term, "on the backburner" mental processing they need to become actual ideas. They're thoughts that are still in the embryonic phase. I can't write down the qualia of them, and when I try to I move them into a "second brain" I lose it.
I think organizational systems are work dependent. Running a business lends itself well to an organizational system. Where it's good to try a lot of shallow things. But something like physics, for example, requires playing with ideas in your mind over a long period of time until they become solid enough to chase more seriously.
I think programming probably straddles that line and it depends on what you're doing at any given time.
Eventually I’m looking at a whole page of good ideas, and some pattern jumps out at me. I attribute many of my best ideas to this process. Writing lets you work beyond the limits of your own brain, especially the constraints of working memory.
I think that is the primary value in a “second brain” - externalizing your thoughts and reasoning makes them stronger - though the productivity community may use the term to mean some other thing.
Where it works better is as a place to store more careful thoughts, and as a place to force you to express those thoughts in writing.
That is, it works best if you don't try to use it as a literal second brain, don't rely on it doing any thinking for you.
All that being said, confession time: using obsidian for 6 months as second brain among other things, i don't believe it has really helped me think other than as being a place where I also create more polished notes where I must crystallize ideas.
What it has done is enable my idea hoarding/belief there's gold in them there old thoughts, so I spend too much time checking if I already had a thought rather than just thinking. So the verdict is still out for me.
Note taking app, personal wiki, etc, broadly any central thing you store information in solely for yourself that's searchable. It's an artsy term that doesn't seem to add much of anything but confusion.
While it doesn't necessarily have anything to do with AI, the trend of glue AI onto everything certainly didn't help.
For you guys' 5 min of entertainment: https://youtu.be/6NukGtwJb7Y?t=256
I have a lot of interesting notes and I can also plug them to AI to get some reminders, insights etc.
But I agree with your general message - thinking is still superior
Yes, but writing also helps you lens your thoughts in a completely different way. This is why so many people journal, or blog, or vlog, even though only a handful of people might actually audit their output. Obsidian sort of abdicates a lot of the value of journaling, but I think there's still some (although much less, perhaps 5% as much) value in the actual act of directing the journaling, even if the rest of the thinking is outsourced to the clanker.
How do you take notes, generally speaking ?
Reverse engineering can take several forms for me: it started with writing parts of systems from scratch, and now (due to academic training) it oftens looks like "thinking about how to explain the system to a group of colleagues", which is one degree of separation from writing the system from scratch, but is much faster - and seems to work well when working with LM agents on code (because you have to confront your ignorance)
part of the training in academia is how to know when you don't understood the thing, so this ends up working well for me now.
One trick: you can force yourself to understand something better by making a presentation and imagining you're going to be grilled by someone smarter than you. This is kind of the feeling of pressure that academic paper reviews give (in an ideal world) and it translates when you think about explaining something to someone else.
If you haven't tried Obsidian at all I think you should give it a shot and write notes without following any system in particular. If you've tried it and you still prefer paper notes and the occasional apple note, then perhaps Obsidian isn't necessary for the way you take and organize notes.
My handwriting has degraded from consistent keyboard use over the last 30 years. Obisidan has replaced my paper notebooks, mostly because of this. However, it is handy to be able to mesh those notes with hyperlinks, LLM output snippets, quotes from other sources, embedded images, etc. Doing all of those other things would be laborious to impossible with paper.
What I also really like about notetaking apps is that I can organize my notes hierarchically, move things around and make cross references.
I still regularly sketch ideas on paper, but once I have something that's worth keeping, I write it down in Obsidian.
I'm also skeptical about this "Second Brain" thing. People make videos on YT about how they have been enlightened by Obsidian and then try to sell you their online course. To me it seems like notetaking for its own sake. You could call it "idea maxxing" :)
We're all becoming mental hoarders and discussing our piles of though junk in our digital storage units.
IMO it's way better to do what you think is good, whether that's Obsidian, notebooks, or something else
The new model can only guess/hallucinate when ordered to give reasons.
It's like having an actor play a role while wearing a hat. Then the next actor comes, puts on the same hat, and we're asking them to explain why "they" did something while there was someone else playing the role.
LLMs inserting code comments makes zero sense by definition.
That’s why they’re always doing the “it’s not X, it’s Y” kind of patterns. It’s reinforcement.
Not completely, because anyone who reviews the code in the future does not have access to your original prompt, so theoretically leaving a comment that explains the "why" portion of the prompt would be useful. LLMs rarely do that, though.
Catching code desyncing from it is a valuable place to reconsider assumptions and maybe even invariants.
My bet is 3 or 4 years before the rocket really takes off, and then we start seeing capabilities improve incredibly quickly.
I don’t want a vibe-coded, mass-produced diabetes tracking app, or banking tool, or tax management software. I don’t want a vibe-coded power grid analysis software.
I don’t care if a person’s little, local scripting thing is vibe-coded. I don’t care if an artist’s 0->1 game code is vibe-coded; but that’s not what a lot of the things we’re talking about here really are.
That’s not “disposable prototyping” that’s whole versions of the codebase written with barely a human in sight.
LLM written PR descriptions and comments are a sight to behold. I am not sure how stuff can be written so cryptically. It seems that LLMs just make up what ever terminology so that it can cram as much details into a single sentence as possible! I generally just paste it to chatgpt and ask it to decrypt it.
The problem here is that after a while it is impossible to detect when such potential abstractions is overlooked in the generated code. Because it has become hard to reason about the existing code.
Not all work is green field.
So all of that happens before the manual review and usually catches a lot of the 3-5 line comments it inevitably adds.
E999: human generated comment :-)
It usually only takes one of those in the context for the model to reorient its behavior for future edits.
And if you’re not reading your own work, is someone else during the PR? Are you reading other people’s PRs?
How do you know what your product does and how it does it?
Mostly it proves as a valid approach, barring the bugs the model can introduce to value-demonstrating benchmarks which can of course skew the evidence on hypotheses, and thus code trajectory the model opts to go with.
The problem I see with this is really that I am not anymore under the control but I am not sure I see other alternative. I am becoming more and more like a system observer with surface-level understanding of the system rather than the engineer with zoomed-in level of understanding of how the code actually behaves. Perhaps we're transitioning into a QA roles present.
I find myself having to use Claude to untangle its own mess piece by piece, a as it charged full steam ahead with the design we made together. However, not everything can be foreseen in a design, unless you go full waterfall. Sometimes I find myself in front of a mountain of bad abstractions stemming from a subtle oversight in the design. Before AI coding, I would find details while coding and catch them in time before they became sinful abominations
You may have to write your own linter for that specifically.
And that’s just the stuff where the house of cards failed quickly.
You’re using a non-deterministic algorithm to generate output. If you want deterministic rules applied to it, you have to use deterministic systems to do it.