This is far from horrible.
If I wrote the code, writing a commit message is trivial.
A readable summary for the ones who may not understand code - your developer will never write that.
This is amazing. Humans should only need to read commit messages, never write them.
Put another way, you asking GPT for stuff that it learned from Stack Overflow: good. Using it to post to Stack Overflow: bad.
As programmers we learn that adding a comment like:
// The limit is 16
const SOME_LIMIT = 16
So what's a good commit message for changing this limit? Ideally we want to describe why we've changed it but this information isn't always available so even when we're avoiding redundant comments we often use redundant commit messages like "increased SOME_LIMIT" to make browsing through history easier for others.
As we do not need to provide this information (it is already in the code), it seems like a reasonable idea for an AI to help us provide it.
In contrast, commit messages often stand alone: If you browse the history, you only see the messages, but now a large number of them; if a commit changes more than one file, the commit message has to sum up the changes from all files.
In all those contexts, a simple, high-level description of what has changed can be enormously helpful.
I struggle to imagine situation in which this is the case. Surely, even in the worst case of you being told to make a particular change with no explanation given, you can at least drop a "increased from 5 at a request of ${name of your boss}", or "increased from 5, see ticket #${ticket number}" in a comment, and/or a commit message.
You think? For some programmers writing commit messages is like ... i don't know because i'm not one of them... some kind of torture?. I bet the kind of person who likes this service would otherwise put in blank commit messages or at best ticket IDs.
If ChatGPT could change that to something like "disable current limits" or "disable safety checks" or whatever that might be marginally better.
Maybe prefixing them all with gpt: would help
It's just the same thing as with comments and "self-documenting code". The code tells you what (and if written carefully, it may be even somewhat effective at it). It can't tell you why. Neither can a GPT-3 summary of it.
I agree with you, but I'm assuming this could just send a diff and that context would be small enough to not leak.
Then again, if GPT can keep track of all the diffs...
As pointed out by other comments, the commit message should be telling you facts about the change that are not evident from the change itself. GPT-3 can't tell readers why the change happened.
Taking a step back and thinking about what I have actually done often helps me to find misconceptions, the worst bugs of them all.
Automating this away would be like learning a foreign language by pasting book exercises into a translation app... you may get good grades, but does it help your understanding if you didn't put in the effort yourself?
https://mcmansionhell.com/post/618938984050147328/coronagrif...
I think the same phenomenon is at play here. Everybody sharing their own silly parrot tricks: it's the least interesting topic in the world right now.
- in Demo 1 tool wrote "Switch to colored output..." while in the diff we can see that colored output was already present;
- in Demo 3 tool wrote "Add installation options and demo link to README", while in the actuall diff we only see a link being added, no changes to installation options.
Props to the author for being honest and not cherry-picking the examples.
… and those who tag you as a reviewer on +8,298, -1,943 commits/PRs with the commit message "JIRA-PROJ-84138".
At my workplaces, we've told people who do this to break up their larger commit into smaller ones before reviewing. If they haven't done that initially, well, their life is going to get harder for a few days.
This happens in environments where it takes hours for CI to let your change pass, making small commits prohibitively expensive in terms of time and infrastructure.
(And yes, I know the answer is: make it so CI that's part of review takes minutes, not hours.)
I even wrote an IntelliJ IDEA plugin 9 years ago [2]. Half as a joke, half to learn about IDEA plugin development. I'm puzzled by seing so many people actually using it. Last month the HTTP link became invalid, and soon after someone opened a PR with a fix. I really hope noone actually uses those commit messages on shared repositories.
[1] https://whatthecommit.com/
[2] https://darekkay.com/blog/what-the-commit-plugin-for-intelli...
A lot of the commit messages were typical and sort of redundant but this one stood out to me https://github.com/zurawiki/gptcommit/commit/82294555e7269e6...
"Add github token to address GH Workflow rate limits"
This is a good commit message, it describes a problem and a solution. I'd be very impressed if the GPTCommit tool wrote this and knew why the github token was being added.
2. If GPT-3 can write commit messages even close to as clear as you can, you're doing something wrong.
But my main though is that IDK about using this for anything closed source. Feed openai's API your codebase, one commit at a time. Even if they promise not to train on your prompt history today, ToS could change. Seems fine if you run it locally though.
Would also be cool to generate commit messages while viewing history, it could really do a good job of orienting you. I'm imagining "human commit msg | gpt commit msg" so you can look at both. It's a little simplistic right now, kinda just describes the diff, but GPT-3.2 could rock.
I think this is the core of my argument, yeah. If a _reader_ needs better than they can run GPT themselves. But the _writer_ using it is worse than useless, it's actively harmful.
Like semi-jokingly asking it to "improve" some code thinking it'd come up with some non obvious control flow... then instead having it immediately point out a subtle bug along the lines of "the code sets flag A instead of setting flag B on <insert line>" flag B wasn't even unused, so it's not like a simple unused variable heuristic would have caught that.
Sure, but you are leaving out the point of the original reply -- the GPT-written commit messages are not trustworthy. They will look convincing, but they are likely to have errors.
I've experienced messages barely better than this in products that were under no immediate threat, and let me tell you this: having to figure out why some changes were made, three years earlier, in a bunch of badly-described commits whose author already left for another job, with no old documentation hinting at the purpose of the changes - this is one of the few things in this job that make me want to shout someone's ear off.
Ex of a recent change I saw, but anonymized a bit.
PROJ-12345: Added preview flag to video player
PROJ-12345 in Jira:
When a preview of a video is playing in the persistent player, preroll ads will display on app launch.
Bug: #12345
Closes: #12345
See https://git.wiki.kernel.org/index.php/CommitMessageConventio...
(I'd want a better commit message than those, though. But they might just be examples for the sake of discussion.)
I tried to start getting people to follow a rule of "if it's a cosmetic/stylistic change, so long as it passes CI, +1". (Nowadays I work in a language what has a decent auto-formatter, and CI just runs & enforces that…) There's a whole slew of similar changes that fall under that umbrella, if you can have the test for it. (I.e., if I can encode my review into a program that CI runs … then yay! For PRs that meet that, if CI is happy I'm happy.)
There are times new work does result in larger commits, like, a few hundred lines. But I've had some 20k delta PRs dropped on me and it's like, let's be honest, the review will be shallow, at best.
ChatGPT can't correctly guess what the author's intent was, because that information is not contained in the code (exception: if the code includes a comment explaining the intent).
I mean, a human brain is arguably also a purely functional transformation adding zero information.
And yes, technically they could do it in comments, which would allow GPT-3 to process it. Except, I think it's highly unlikely for a coder using ChatGPT to write commit messages for them to actually author useful comments on their own. If they write any at all, they'll probably use ChatGPT (or Copilot) for that too.
Elsewhere in this thread someone was wondering if sending a change diff to a fly-by-night third party SaaS could be leaking company IP. They were thinking too small.
But cynicism aside - giving the model access to all that contextual info would definitely increase the chances it would generate useful commit summaries. It would also increase chances it would generate much more convincing bullshit, full of just the right phrases to confuse your programmers, PMs and architects alike.
I've never seen a 1:100 code to commit message lines ratio so far, but I've seen a few 2-3 line changes with a paragraph or two long explanations. I cherish those. Same if the explanation is in a comment. In fact, if I spot something like this, I tend to praise it publicly on the team chat.
I had one case where a single weird line added much earlier messed up a seemingly unrelated piece of code I've been developing. It took me a while to figure out that something is emitting compiler flags that, with surgical precision, prevent the very thing I was attempting - and then find it nested deep in the build configuration. At that point I wanted to strangle the person who put it there - but a paragraph of commentary attached to that line, plus some extra context in the commit message, made me change my reaction to "oh. OH. I see the point now.", and I ended up commending the author instead.
I’ve definitely also added multiple paragraphs of in-code comments to an otherwise-single-character change, where it’s an ongoing consideration rather than something that can reasonably be left in a commit message alone. Then my commit messages gets to be brief, directing you to read the added comment instead.