Just generally though: what we're seeing a ton of these days is people writing something and then passing it to an LLM with a request to improve it somehow, e.g. by fixing grammar, tightening the style, etc. In such cases, the answer to your question is that the "prompt" is (1) a first draft, and (2) an instruction to edit it.
It's clear, though, that the LLMs leave far more imprints on the text than most people realize, and that although they may have asked the LLM to restrict its edits to "just" X or Y, the actual changes to the text will often go beyond that.
How this will evolve over time is anyone's guess, of course.
>Have you tried putting known human writing into pangram? I have. I've gotten 100% AI with multiple samples of my own human writing. It has also given me 50% on things I know were 100% AI written (from my prompts).
https://news.ycombinator.com/item?id=48326698
>Pangram is basically a made-up number. / I've tried it on large docs I've written well before the AI times, and that are nowhere available on the Internet (so it can't be a corpus issue) - and it is happily classifying me as 60%-80% AI.
https://news.ycombinator.com/item?id=48378226
Two of my own thoughts:
Unfortunate there's an incentive to pay to sign up to protect oneself against false accusations.
An earlier claim in this thread stated 100% from the same tool, but another commenter claims 76%, so apparently the tool is even susceptible to that failure mode.
I guess the idea is AI gives you back time so you could now do the 20% but you still really can't because you have to still think about it even if the code is generated? Not even sure after reading all that text what the idea is .
I was always jealous of the 20% off concept, because there's so many jobs and places where I'd use that time to solve things nobody wants to "fund" within my org, sometimes there's some really dumb bug somewhere, or easy to solve for internal tooling need (I'm sure Google has had this resolved many a time internally) that could be met if I could even have two hours on a Friday to work on anything.
only a few companies like google had that imo. most companies cannot afford that.
Aside: different LLMs sound different too! ChatGPT is the worst offender for LLM-sounding writing and needs the most smoothing, but Claude (web) actually sounds like a humanities major from the get-go.
Where some people are getting into trouble, at least in the HN context, is underestimating the impact that this has on their text. There's a big perception gap between the author's view ("fixed up the grammar a bit") and the reader's view ("this sounds entirely like an AI wrote it") in many cases. So many, in fact, that I feel I can say something about it. I'm no authority on any of this and don't want to sound like one, but this is such a common pattern at the moment that I feel confident reporting it. How it will change over time, I have no idea.
(I also don't want to sound anti-LLM - we rely on these tools heavily, they're amazing, they've already improved HN, and they show every sign of high potential to improve it further. The bottleneck isn't the LLMs, it's how quickly we can figure out how to use (and test) them. We just don't use them to process any text that we put on HN itself.)
You shouldn't crucify people based on this alone, but if it reads like AI, quacks like AI, and is detected as AI, it's probably AI.
Since early 2023 or so, when the detectors were widely reported (off platform) as unreliable?
>and frequently comes from people with some vested interest in filling up the internet with slop
I'm sure sometimes.
At least once, has come from someone who recently philosophized about whether to call out the specifics of AI writing in the first place given the potential for aiding labs in their training missions https://news.ycombinator.com/item?id=48326913
>I've done systematic testing
You in the field?
>but if it reads like AI
Actually didn't to me, and I'd like to think my detector's no worse than the average for a commenter here... perhaps as we'd all :)