Why do LLM pelicans look like this?(arcfu.com) |
Why do LLM pelicans look like this?(arcfu.com) |
1) Chollet stated that RL:Pre-training is 9:1.
2) Google said argon was trained on SVGs
3) Wang said that Muse Wonder was not trained on 3D models
One of the reasons image AI slop has been so prevalent is that the slop is the base case (as derived from tons of RLHF because the median user has no taste) and it is becoming increasingly difficult to steer those models even when the prompt is more specified.
Lost count of the number of times a requirement is given in terms of how another product is doing it. "make chips like gmail uses for tags", "let's copy anthropics LLM chat UI", "we should try and make it look like X competitor".
"Looking the same as everyone else" seems to be a bonus in their mind
People weren't happy with boring answers, so they added randomness to the output. The randomness led to hallucination, so they made it less random. Now you want it to give you a unique artistic impression for every prompt? It's a spell checker, not an art student.
Perhaps the prompt should include what the scene should contain.
For example, a lot has been written about how all corporate logos look the same now, https://news.ycombinator.com/item?id=32040506 , and also about how so much interior design and architecture has converged to the same look. In the 70s there may have been tons of styles people derided as ugly, but at least there was colorful diversity. Now so many higher end (and "aspirational high end") interior design has the same minimalist, gray/white/tan, "look you can't see my refrigerator or dishwasher" look. I was recently looking for an apartment and all of the "luxury" apartments I looked at (obviously take that adjective with a heap of salt) had nearly the exact same layout and style.
You'd probably need to ask thousands of people (or some children) before you heard "a pelican on the moon" or "a pelican in neo-tokyo"
That's why I'm not an artist.
I don't see why you couldn't do the same in real projects.
If the design looks too generic, you can just ask for a change, no?
You have to be in the loop to inject taste at some point. This typically requires a lot of testing and feedback.
I don't know if AI will ever develop taste, or if it even should. We have enough human creativity and creating is fun.
Now within our GUI/TUI to the llm (which we all use now) could just add a couple of extra steps to simulate that diversity.
Are we making an image? If yes, and is this the first prompt in the convo? neat! List 5 elements that would make up this image. Now for each of those 5 elements, make a list of 10 possible choices. Now randomly select from each one. Now include those elements on top of the original prompt.
I got claude to generate 10 of these as a prompt transformation, and i think they would be suitably random to postpone the diversity collapse for a while.
I'm not pretending the problem goes away, but these kinds of creativity varying steps I think will become hidden steps in our LLM systems very soon.
The same thing happens when doing any group work in a classroom or workplace. One of the facilitators job is to add in randomness and guides to remind people to make different choices rather than the obvious ones.
I don't think people care much for what they say.
The article faults LLMs for not being more creative (riding in the city in the rain, in East Asia, in outer space, etc.). But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
The article talks about "the LLM diversity collapse problem," which makes it sound like LLMs are getting less diverse over time, but the linked benchmarks suggest that newer, more capable models are _more_ creative.
Out of curiosity, I tried a couple of prompt variations in Opus 5.5. "Be creative" did a beach scene but added fish to the bike basket and a lighthouse in the background - not too creative. "Pick a unique and creative variation of this concept" resulted in a "midnight circus act" of a pelican riding a penny-farthing on a tightrope while balancing a fishing pole with fish (and, this time, two lighthouses - the tightrope is implausibly strung between them).
It also affects writing, if you ask for a story with a minmal prompt something very similar comes out of all of them. Often fixated on clocks.
There was a weirdness going on with the first Midjourney models that I miss.
But LLM's are structured to do exactly the opposite. They discover specific commonalities and tropes and most-frequently-sufficient wells or ley lines that they gravitate towards from whatever their starting point it. You can nudge them away from the most generically cliche tropes by being more detailed in your prompt, but they'll still just find the nearest most right cliche from there.
It's what they do and the same quality that makes them tiring and predictable for creative output makes them very useful for things like business communication or coding where landing on such cliches is ideal.
To get more variety out, it can be useful to use Perchance-like text-randomization techniques to shuffle combinatoric variations on a prompt before sending it to the model. But that technique is best made available through some kind of special purpose harness for the creative task, not basic chat or direct curl calls to the API.
If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.
I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"
(I didn't read the linked LLM creativity paper to see if it covers any of this yet)
That said, I think it’s a fun idea for some training, and would be pretty cheap to do - you’d need some sort of judge as to what a viable beam is.
The Hyères photo has the bike going in the opposite direction and from above. It's also tricky because it's a focal point for the eye, but not the center of the photo. The other elements of the frame direct your eye, but that visual behavior is not obvious to an AI. Lastly the image must capture the unquantifiable "decisive moment" that made Cartier-Bresson so famous.
So far I haven't seen an AI that really gets this prompt. Adding specificity will improve the results to a degree, but I haven't seen an AI that really gets what makes a Cartier-Bresson photo.
> But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic
I believe the claim is that without a detailed specification, you see very similar outcomes. Every pelican is going left to right, every picture has similar sky background, every picture seems to be drawn at the same time of the day (position of the sun), every picture has a few clouds... I don't have data on this, but I imagine the same question being asked to a group of people would result in more varied outcomes.
Therefore asking a group of people would result in more varied outcomes of something visual, but from the textual training data the direction is statistically the most likely so is what is output, that sky is the most likely, time is the most likely, cloud coverage, etc. You're prompting with as little as possible so it fills in the blanks with what as likely as possible. When you ask Alex from the North Pole they're going to fill in the blank with a different kind of weather than Bob from the Equator.
That does mean that an LLM itself is in essence not as creative as a human, but then it is a tool to be used by a human doing more than oneshotting "Generate an SVG of a pelican riding a bicycle" and expecting creativity from the tool rather than the user. If the user is lacking in creativity they can prompt an LLM to get wacky scenarios to change it up a little.
“Generate an SVG of a pelican riding a bicycle. Be creative”
And it replied:
“I’ll draw a seaside pelican on a vintage bicycle, with a fish-filled basket and a scarf catching the breeze.”
So it came up with exactly the same fish basket and seaside idea. No lighthouse though. Maybe these ideas are just cliche though.
The curious thing is that trying to prompt it by steering towards some specific goal indirectly gives much worse results than just stating the goal itself.
same goes for pelicans. the canonical pelican riding a bicycle is riding on a sunny day on an asphalt road going through some lawns or praires, left to right.
but ask it to tell a joke about something specific, turns out it knows five jokes about those specific things.
you could say the LLMs are quite aware of the zeitgeist they're in, by construction (their RL frying automated feedback loops burn the zeitgeist right into the weights.)
All going in the same direction? With a sun? With a full mouth (pelicans dont look like that, but a cartoon one might - why no reference to realism). Most of them have clouds.
There are helmets (makes sense) but just as many scarves.
A shocking number have chains.
If we asked 1000 people to do this same exercise would we have this level of convergence? If they were split between men and women would we see this many chains? (https://www.messynessychic.com/2016/04/20/making-bicycles-fr... )
Would they even look like bikes?
Is any one going to draw a gray pelican? Does it matter?
The question is designed to be as generic as possible, and were converging on the most generic and safe answer ... the question is WHY (more so when humans would have much higher diversity, even if wrong). We're asking the generic question because no one wants to repeat the "German solders from 1940's" fiasco: https://archive.is/5l66U (archive link to skip pay wall NYT).
This is what "alignment" looks like? Are we reaching some "ground truth", some platonic form around pelicans on bikes? Are we just back feeding the best output into the training data and this is the convergence we ended up at? Is someone who LOVES pelicans deeply offended that all of them have full mouths and it's just not anatomically accurate and we are all missing it?
I realize that some of this seems over the top, but there are foundational conversations to be had around this without the conversation devolving into someone accusing someone else of having an "-ism" style bias.
True. You see a lot of the same criticisms when someone asks for a one-shot game or 3D demo and the model responds with something bland and generic. That's exactly what I prefer, it's my job to specify the details. Extrapolation may be the ultimate sign of intelligence, and it might be needed when using AI to tackle open math problems where humans have failed, but it's not usually what I want if I'm doing anything creative or artistic.
Argon is pretty good at getting the distinction right from what I've seen. Ask for something generic, you get something generically functional without a lot of ornamentation. Follow up with requests for specific enhancements, and the model complies effectively and tastefully. If anything, it overcooks the pelicans when given the standard open-ended prompt.
Do a Google image search for pelican, you get a diverse set of sub species of pelicans and different poses and angles for them. Go on a writing prompt forum, you get many diverse ideas from the same simple prompt. Hell, ask your friends at a dinner party to describe a pelican riding a bike and see what happens.
Now, there are a lot of images of bikes from that side angle facing right on the web. Why? Because that's how they're staged on e-commerce websites. Does this say something about AI?
I don’t want creative code. I want boring code following generally agreed upon best practices.
If you need more creativity it can come from the user or the harness.
>If you need more creativity it can come from the user or the harness.
Not fully possible. You can randomize the prompt all you want but you'll always get repetitive patterns and same choices in the same generation. This failure mode makes the model learn a very rigid mapping between this input idea and that single output idea (at best! at worst it's multiple input ideas -> single output idea).
I was thinking just today about how, if I could hire my competition's lead designer/programmers, have them design a system for me cleanroom, that I'd effectively know everything there is to know about my competition's limitations. This is trivial to do with LLMs.
"Make me a thing that does X" is going to come out implemented basically the same way most of the time, as a kind of stack-overflow, well commented, internship project. (or, if undirected enough, as a collection of those based on a bog-standard architecture)
Same-y enough that you could train an adversary on it? Or hack it afterward? I am beginning to think so!
So perhaps the first part is how the GP leans the limitations of the competitor, by seeing what they overdo in the second system? The GP never said they’d get - nor that they wanted - a functioning program?
However, quite how the LLMs clairvoyantly know the trauma of the competitors architecture is beyond me :)
This is a feature, not a bug.
If I ask for a pelican riding a bicycle, I am expecting a relatively boring image of a pelican riding a bicycle, not a pelican flying through a thunderstorm while on a bicycle. That's just straight up not what I asked for. I didn't ask it to be creative, when I imagine myself riding a bicycle, I'm not flying! If I added "be creative and unique, you can add or change anything as long as there is still a pelican on a bicycle", then sure.
Suppose I want an SVG of a laptop. It should probably look like a Macbook Air or something, neutral colors. Easily recognizable as a laptop. If it chose to be creative (unprompted) and gave me a futuristic or retro laptop, or one that's super wacky, there's a high chance of it looking bad, and a high chance I say "no that's not what i wanted". So during the training step, it probably learns to just follow instructions and be less creative unless asked.
Sounds to me like LLMs are just getting better at following instructions and doing what they're told. Simple prompt, simple output.
But step back, someone did ask it, perhaps that person is you. That person has reasons. Whatever those reasons are should be reflected in the image. Of course those reasons need to expressed in the prompt, but the bug here is the prompt doesn't give the details. If you can't figure out details (beyond "be creative") then give up. If I'm illustrating a Harry Potter book and just want to show far out magic things that better should a very different style from illustrating a science fiction novel where pelicans have been genetically engineers such that they now do ride bicycles - likely the anatomy changes mean they can't fly...
This is important. I recently asked AI to take an outline and turn it into slides. I got some good slides out of the outline except that once in a while it decided to generate an image which looked cool, had the correct corporate colors - but otherwise had nothing to do with either any product we make or my presentation. I deleted the images but maybe I should have refined them to something meaningful?
Much of that is a pre-AI bias: A majority of bike-pictures are from an angle which reveals the bike's gears to the audience, which are usually on the rider's right-hand side.
https://www.youtube.com/watch?v=Ys8-a0yD-MM
https://pubmed.ncbi.nlm.nih.gov/26561966/
https://pubmed.ncbi.nlm.nih.gov/1509704/
https://www.newneuromarketing.com/right-to-left-isn-t-right-...
We’ve been trying to tell this on every pelican thread. All models were recreating the same composition. The sun, the road, the green, the scarf, going right. Even the style. They’re not obvious associations one would make given the prompt.
Why does an SVG pelican imply that it should look polygonal?
I ran into this issue recently when asking Claude to design a logo for my van. It made a polygonal SVG. I asked it to make it more photorealistic and it looked way better, still an SVG though.
After all if you ask 100 people to draw a horse, it is quite likely it will be a lot more similar to a "here's a stock horse photograph" than a "here's what picasso would draw when asked to draw a horse"
"Write an svg with graphical instructions on how to make a pelican riding a bicycle in Origami using any number of A4 paper sheets."
All models struggle a lot. Most current models all get a "basic airplane that really floats" pretty well.
The reason why LLMs cannot be purely* pre-trained on LLM outputs is because LLM output diversity is low.
This implies two things:
1) There is going be no functional difference between hypothetical Qwen 4.5 and Fable 5.5
2) LLMs are not AGI
- https://arxiv.org/abs/2609.37891
- https://arxiv.org/pdf/2508.10975
- https://www.datologyai.com/blog/eating-the-rainbow-a-primer-...
As well as recent model releases that used large fractions of synthetic data (Arcee Trinity):
- https://arxiv.org/pdf/2602.17004v1
It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens.
There have also been some interesting results about models that use the same data converging to very similar behavior, but I can't find the specific reference at the moment. That one tracks intuitively with the notion that the model is building a latent representation of everything it sees.
Upon a quick glance of the data sections of those papers, every data set you have there is human data augmented by LLMs. Also not to mention the implicit data curation and generation by humans.
Can a model be trained purely on LLM-Outputs and access to tools such as a Turing complete environment?
No one paraphrased the laws of gravitation and handed it to Newton...
These look surprisingly similar and not necessarily something that directly follows from the prompt.
It's unsurprising and perhaps even desirable that different LLMs return similar results to an identical prompt.
If you want variation, prompt for it.
If I'm asking simply and directly for a pelican riding a bicycle, then please honor my intent and gimme that pelican. I don't need it on the moon. I do not need a top hat. I just want what I asked for.
Fulfilling a simple request simply seems like a feature, not a bug.
Snapping out of the "boring and repetitive" rut here is on us to prompt more creatively. Maybe that's the new "garbage in, garbage out".
"Make me a PB&J" should result in what a reasonable person would consider a peanut butter and jelly sandwich. I don't want to interact with an unpredictable system that "thought it would be fun" to deep-fry it.
"Well, you didn't say NOT to deep-fry it!"
tho who know to refine, say NO. & do more with less even in the age of too cheap to meter tokens.
that has always been the work that produced every delightful product.
The pelican from the post can be changed by prompting, as other commenters show. But I found it pretty hard to guide Opus away from the above.
Other models' results look different, but also show style convergence.
You can also see in "creative" game output (Astra loves 3d), or text output.
It's not all bad - this shows we should treat AI as a tool and assume any creativity (or lack of it) as coming from the human.
This nicely redefines slop as work you slapped together without much care, irrespective of what the tool was.
Is every kid a Picasso? No, but there definitely is creative diversity among these kids who all have relatively similar shared backgrounds (neighborhood, social class, religion, etc)
It's the reason that if you ask a random child to draw a house, you'll end up with the classic “house with a pitched roof”: a square plus a triangular roof, one door, two windows, and sometimes a chimney.
If instead you ask to draw some fancy interestingly looking house you'll get more variety - now the task is not to have the question giver recognize the house in their drawing, but to show how they imagine fancy or interesting house.
For most kids there isn't enough time to draw a better house - they want to go play. They also are well aware that there is only so much time in class and the better house wouldn't be done and in turn look even worse.
And with ever-more LLM usage both in labs and the real world, the task of finding new/“creative” out-of-average-distribution material will only continue to increase in difficulty. Meaning if this continues, in the future, all output becomes the equivalent of the color brown?
Now the images look like polished, generic things you would see in an airplane magazine, which is what most people want, I suppose (?)
Variety is not randomness though. You can't fix this by playing with samplers without probing the model state.
That's called greedy decoding and is never used in practice. Everybody's using random sampling, so it's the range of most likely results. This issue is all about that range being way smaller than the training data implies.
In the UK and Europe, the wing is "flying" to the right.
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?
There are some things that are true (I hope), so using those concepts, approximated by a human text filter, as seed seems reasonable. Hopefully the diversity in sampling methodologies and opinions about what makes a meaty token pays some dividends down the line for model diversity.
