What AI did to stackoverflow in a graph(data.stackexchange.com) |
What AI did to stackoverflow in a graph(data.stackexchange.com) |
That's primarily due to AI answering my technical questions.
And then it started being stupid questions. People who clearly had barely tried anything and just rushed to SO with a half baked question. Answers were just pointing to another thread that already provided the answer. It definitely started before LLMs. I think it lined up with the aggressive "learn-to-code" push.
Many people talk about the negativity, and they are right, but I think the reason more than anything is the waiting. On SO a good question might get answered in minutes (if it was easy and someone was karma farming) but it could be days or weeks for general purpose stuff; compare that to a few seconds for an LLM its a no brainer.
Maybe if in some cases stuff actually got deprecated and points did actively decay it might have worked... But can't remove power from those who had it for years...
That's hardly a death sentence. More likely just the gradual adoption of higher level frameworks and languages with less ugly parts.
I asked to see one of the questions from 2024 - it could have been solved with one LLM search.
We have eliminated a whole genre of peer to peer communication.
It still has some value today, as sometimes you can find useful information on SO, but its peak days are long over and I don't see how it can manage to come back, with or without AI slop. It would basically require a lot of re-design and some things that never worked, such as the karma system, should be changed. Also moderators - they kill sites. That happened to reddit - I gave up after censor-mods constantly restricted everyone.
The normal day to day devs just don’t have the need to go to stackoverflow anymore.
One would have to explain both consequences or dismiss it as coincidental.
FWIW I rarely have the need to ask questions at the programming level to anyone anymore. It’s just not the type of thing I bring up or anyone else. We now talk about architecture and company direction.
I think a lot of people might balk at the word "inclusivity" today, but StackExchange had ridiculously high barriers to participation, making it inclusive to the long-time users on the site, but exclusive to the newbie participants who found themselves blocked for asking questions. They slowly killed the site in this manner.
The community might have survived this folly, even with AI, because it was still the best place for people with qualms about AI to ask questions... Except until StackOverflow management alienated those users, too, by shoving AI down their throats in every facet of the site.
Even I had internalized the vagaries and neuroses of the SO community but I had heavy reticence to ask questions, knowing I'd have to consider all the ways a bully eager to use their powers might misunderstand me. I can't imagine asking a question there without having had lurked for longer than a typical Bachelor's + Masters program.
Peak at 207K, minimum at 588. That might be an incomplete date point, so using the next most recent value 1226, StackOverflow has lost 99.41% of its activity.
I hope somebody saves it all.
i got stung by exactly this.
i saw some of my early questions rewritten because some idiot mod that had not touched grass in a while thought that some words were better suited for stackoverflow.
and don't get me wrong: i'm not talking about profanity, n-word or racial slurs, derogatory terms or other controversial words. it was quite literally stylistic and tone changing.
dumb example: i like to end my posts with something like
thanks in advance,
--
znpy
which in my opinion is just common courtesy in a conversation between me and whoever will be kind to answer my questions. it's harmless and not controversial. and yet, some mods edited that out and left some irrelevant wording on that. my guess is they were farming points on the site.I'm so glad stackoverflow died and I don't miss it at all.
I still don't understand why anyone would go with that whole "no conversation please"
* Moderation went bad. I stopped moderating/flagging after it was deemed unhelpful?! I know it's hard to moderate a platform like that, but giving me a slap in the face when I volunteer my valuable time is not the way to do it.
* Questions closed because they weren't "programming questions", but obviously about tools devs use every day. Again and again, they were the TOP google results. You'd click on it and found a old question closed because it was considered off topic. As a business, you seriously need to ask yourself some hard questions when you fend off users like that.
I think Stackoverflow was the last incarnation of romanticized old-school Q&A forums where you first had to earn your badges before being treated with respect. Luckily today's new projects have much better documentation, issue tracker's etc. And apparently AI is able to work with that by now...
I think other helpful places like reddit, discord, web forums etc might be what hit SO 2014-15 onwards.
AI seems to have given it a blow of mercy to end the misery.
I don’t buy this.
Programming as an industry is famous for constantly evolving and changing.
- Open question on SO: Moderators close because it's too specific to a library
- Ask on IRC: Get piled on for not using the right vocabulary and your IP isn't masked
- Ask the LLM: Get hallucinated answer based on old API docs
- Ask technical lead: Get burned for asking basic question and put on PIP
- Ask my mom: She doesn't know enough computer to know the answer, but in explaining the problem to her, I finally figured out what I got wrong
Not sure I would blame it all on AI though, the incentives of SO only worked while there were worthwhile questions to answer and make you feel smart about. After that well dried up, the only thing left was the stuff AI can do with a prompt; ironically AI got a leg up by scraping SO.
This is similar to the evolution of Wikipedia, except the format of WP allowed it to transform into a feudal dictatorship of nerds who feel like they are deciding what's true, and they can get off on that.
SO did not have that kind of incentive to keep the nerds around.
What happened is that as the corpus of useful info increased, the need to pose new Qs decreased. AI much accelerated that decline by making available an 'oracle' trained on that corpus.
And they killed maybe one of the most side features of it : https://meta.stackoverflow.com/questions/415293/sunsetting-j...
So yeah metakill your own brands with stupid policies.
Now do a graph for the money.
https://www.wired.com/story/google-deal-stackoverflow-ai-gia...
Sure there is, number of questions halved from 100K in first of November 2022 to 50K exactly one year later.
Funnily enough, there's now a "StackOverflow for Agents": https://agents.stackoverflow.com/recent
This is revisionist history. People told SO that they were leaving for YEARS because of how incredibly toxic it had become. It was already giving outdated answers before ChatGPT shipped, because new questions/potentially updated answers were [Closed] [Dupe] immediately.
Their answer was essentially "We aren't a Q&A site, we're trying to be a knowledge base! So closing all questions on a Q&A-stylized site, and extremely abrasive moderation, is working as intended."
They entirely did this to themselves. The community was toxic, their policies were toxic, and they didn't listen when warned as such repeatedly - just doubled down.
The Monica affair was one of the first symptoms.
In 2023, Stack Overflow had already started making unpopular pro-AI moderation decisions, and in 2024, they started mass banning everyone who deleted their questions and answers in protest. I don't think it's wholly incorrect to say "AI killed Stack Overflow" when the death blow came from crazy pro-AI decisions from the admin.
The title of this thread is "What AI did to stackoverflow in a graph." That's a narrative. At least before the mods change it.
Today I'd expect even when someone is googling and a question is relevant that most people will just read the AI overview.
Instead I directly go on the project github page and ask the question directly to the mainteners.
Stackoverflow aimed to be a knowledge base. And knowledge base has a ceiling limit. They simply reached the point that almost all questions (regarding the knowledge) were asked for them. You can argue that newer or niche libraries or languages knowledge is still lacking there, but I have never seen them getting closed, just not answered.
I always found the format of the side obtuse and the culture not very welcoming. My most popular answer ever was something about JavaScript from 2008 or 2009, and to this day, people come in and say “this isn’t the way to do it, this is outdated“. No kidding, but every new question about that gets closed as a duplicate.
I read a great article not long ago outlining the full series of events and changes that led to its downfall. I wish I could find that article, but I've forgotten where it was.
Where they went wrong, in my opinion, is in the implementation details.
It's mostly death by a thousand cuts: Requiring reputation to gain the ability to post comments, then having one's answers deleted as "this should've been a comment". Overeager marking of questions as duplicates, e.g. despite the equivalence between two situations being non-obvious (e.g. someone asks about data type A, and it turns out that it's a subtype of B for which an answer that applies to both exists; that should not be a duplicate, the fact that it's a subtype is the answer!). Endless other decisions like that, which wouldn't have taken any extra effort to implement correctly.
One feature they could've built that would have taken effort but also greatly helped against the common newbie complaint of "hostility" would've been a "newcomer track", which would've been more forum-like and guided them towards either formulating a good question or seeing that's it's already answered. In the latter case, some of the keywords that came up during this process should've been fed back into SEO so that future newbies would become more likely find the answer via a search engine despite using clumsy terms. I think they tried a simpler and worse version of this idea towards the end with "staging ground" but by then it was too late.
Instead of going in the same direction of everyone else adding AI all over the place and trying to eliminate the humans, they could have gone the opposite direction and played to their somewhat unique strength of having a bunch of actual humans and providing a place that actually fostered human and authentic interactions. Instead, for some completely unknown reason (money), they chose to commodify their own platform. Smart.
It quickly turned into simple questions and "send me the codes"
Good luck trying to write any helpful posts in the community anymore, someone will come along and respond with "AI."
Yes, you can find dark corners of reddit that lean in any political direction. But go to the front page and it's littered with Anti-trump, anti-republican, anti-conservative (for us Canadians) threads. 99% one direction, put in front of every random user.
I left X for the opposite reason, by the way
You want to blow off some steam, and there is a laundry list of rules to read through.
Yeah, not bothering with all that.
I don't buy it.
Usually I’d find answers on SO. Relatively rarely I’d ask questions but, when I did, I’d always try and follow the netiquette rules of yore, and think in terms of, if I was a support engineer trying to help with this, what would I need to know?
Because I have supported products, and we’ve all seen enough bug reports and questions come in that we can tell when someone is going to be easy to help - even if they have a particularly tricky problem - versus someone who’s going to prove more challenging.
So I had this question about Elasticsearch, and it was at a time when the documentation wasn’t great, and you were actively encouraged to go on SO and tag your question to get help.
I wrote out in detail what I’d done, where I’d got stuck, what I’d read and tried to get unstuck, etc. It probably took me 30 minutes or more to pull everything together into a coherent post.
The very first comment was from some insufferable bellend saying, “Oh, so you want us to do your work for you, are you going to pay us too?” or words very much to that effect.
Literally, WTF? Why even post that? If you don’t want to help the option to simply go away without getting involved is always available.
IIRC I didn’t actually end up finding a solution via SO and instead layered some godawful hack on top of Elasticsearch to get what we needed - because I simply had other work to move on to and I’d already spent a lot of time on the problem.
But I think that was the last question I posted on SO, and maybe the last time I posted anything on the site.
As the years wore on I simply started finding it less and less useful, with often incorrect answers marked as accepted and - if you were lucky - the correct answers marked might be buried further down.
And then there’s what they wanted to charge for job ads versus how effective those ads actually were - again, this was better in their earlier years.
SO started out well - genuinely a breath of fresh air - but as time went on it felt like they thought their model was the last word in online help forums and they didn’t want to evolve to address its flaws, even if that had just been dealing with the toxicity, and the karma farming.
And so this is the result - a site that, like the dinosaur in A Sound of Thunder, is dead but perhaps hasn’t realised it yet - and, at this point, the way I feel is simply good riddance. It’s a shame, but - as you said - they did it to themselves.
https://www.penny-arcade.com/comic/2004/03/19/green-blackboa...
Good for training data I guess - pure Question and Answer. Maybe they knew the platform would die so decided to optimise for that
They also had the problem that easy questions would get downvoted for being too easy, and hard questions would just not get answered because they weren't seen in time by the narrow group of people who could answer so they get buried by the algorithm. Working in something of a less common niche myself (embedded Linux), I never had questions get answered. I believe the question ranking systems and moderation policies really only worked for questions about new, popular web frameworks.
It was ChatGPT which did it in, but it could've been anything. It could've been a new group of people with some clout starting a fresh new knowledge site. People were ready to abandon SO.
- the downfall of junior devs
- bad hiring market
- layoffs in practically every sector
theres a ton of things where AI took credit for a trend that had already started before it started being even halfway capable.
Goodness of Fit 0.911, Kurtosis -0.849, Skewness: 0.073
It's very much a bell curve
> Without AI they might have lasted at least a couple more years
Nah, their decline was already readily apparent before AI. You only need to go through old discussions and other people noticing it. AI may have accelerated the decay, but the decline happened already largely prior to AI.
GitHub made documentation more consistent how? GitHub issues were more transparent how? A PR answered a question I would have asked Stack Overflow never I think.
That's a very big word you're using there for what is basically making shapes out of clouds. A bell-curve is the amortised function of a random variable with a mean and standar deviation. What does that have to do with a timeseries dataset?
The general notion of a bell-shaped curve is broader than that. Wikipedia has a reasonable overview: https://en.wikipedia.org/wiki/Bell-shaped_function
> “typically continuous or smooth, asymptotically approach zero for large negative/positive x, and have a single, unimodal maximum at small x.”
You are doing that implicitly by fitting a Gaussian curve.
BTW Reddit is now verifying your ID with Persona before you can open anything it thinks is NSFW.
Though I'm not sure how long that'll last. I would be surprised if old.reddit.com is still functional in a couple of years. When it gets removed, or when it bit rots to the point that it's no longer really feasible to use it, I'm off that site. New Reddit doesn't work for me.
Probably in the same way as they're actively removing r/all, at first it just didn't show up in the sidebar on mobile, but you could go to r/all manually by clicking links in the client. Then those stopped working, but r/All (uppercase A) worked. Then that went away. By now I think it's impossible to see r/all at all in the mobile client or the modern website, you can only access it via old.reddit.com.
I think this may depend on your country, I've never seen this (Spain), not on the new website nor old.reddit.com or anywhere else, NSFW or not.
If you trust Apple, why verify with Persona above that? If you don’t trust Apple, why bother integrating the Apple age check? The answer must be something silly like “we did it because Apple asked us to but we don’t trust what Apple tells us because we’re not sure if it’s compliant”.
It’s too bad, because I trust Apple with my data way more than Reddit and infinitely more than Persona. I hope Reddit comes to their senses because I’m never giving my data to Persona.
I see that and I instantly go to rotten tomatoes, pure idiocy, pure stupid. You can just tell when those in charge, never ever dog food.
The fit does not prove causation, but it does show that the decline was already well described by a trend that began years before generative AI. If the claim is that 2023 created a separate structural break, it's different claim then the title describes