Saying goodbye to Agile(lewiscampbell.tech) |
Saying goodbye to Agile(lewiscampbell.tech) |
It's been workable for me. We can change the requirements as late as you like, because I'm getting paid by the hour. Scaled up to a company, that translates to not giving a fixed price for anything.
If you need to give a fixed price, either be experienced enough to know by how much your agreement can change and factor that in, or turn it down. You should also turn down demands for a fixed price on novel solutions you can't have experience on.
Engineering (even in computing) has a formal basis and practice. Project management does not. Systems thinking and industrial organizational psychology does, but rarely do you see it applied like bullshit such as agile (and in environments that do - it works spectacularly).
Out with the voodoo, and in with the scientific method, I say.
Replace "LLM" with "compiler", "specs" with "code" and "correct code" with "correct machine code" and we are back to square one.
I feel this in my core. There has been a period of the last 5-6 years where folks have stopped writing specs entirely and it has driven me nuts. Everything is a story or some other abstract requirement. The absence of a specs has made software worse and less predictable in my opinion. All hail the specification!
the teams that behave the closest to what the Agile manifesto seems to define as agility had three things in common, two of them were inside the team:
1. emotionally mature team members
2. competent team members that were able to deliver and knew their strenght and acknolwedge their unknowns
and the one item outside the team:
3. Trust and respect for them from the business leadership
Of course having these 3 things makes any SDLC work
Agile is stronger than ever, spending time on every small detail in a waterfall approach makes you burn human time on work that most probably could have been perfectly fine with a default approach.
Context matter but I fail to see people spending months on planning out a system before building anything.
Closing feedback loops. That’s the whole thing. WE Deming would have recognised agile (little a) as a PDCA system and approved.
- Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan
The problem is that the Agile industry mostly didn't follow the Agile manifesto and ended up with monstruosity like SCRUM, which is all about processes over people.
Daily Standups, Retrospective, Backlog Grooming = PROCESS
This crap should all be replaced with async written communication (for quality of life and recording), but each team should ultimately be free to decide.
The Agile manifesto was all about freedom and it was turned into a jail.
It's only a promise of a method.
If you think it's waterfall again: Wrong.
Everyone who phantasizes about "just"™ writing the perfect spec will be in for a rude awakening.
The spec will change over time and your initial version will turn out to be very wrong.
So many times I have found myself writing the end-user documentation (even after writing tests for the code), and realized that the design should change.
This is the kind of post that makes me log in to hn to give a vote.
Hell, half the devices in your life probably run some hacked together crap that was built by people who barely knew how to program and eschewed version control for USB sticks.
I really hate discussions of "software" as if the software in an F-35, the software presenting data on a webpage, and the software making a child's toy blink and speak are all the same thing. Only in a very abstract sense are they similar.
Let me find my: Requirements Specification Requirements Analysis ...
The circle will turn once again when people re-realise that by tue time you've written what should happen in enough detail, you've written the software, and English isn't that great at avoiding ambiguity.
The same is true of software. At first you try to make software, and you do, and it's kinda easy. Then you try to make more advanced software, and it seems much harder than it should be, as what you think will work doesn't. You spend a lot of time changing your design to make things work, which ends up not being exactly as you thought it should at first. Finally, after you master software development, things get easier and work like you expect.
In both cases, when you are ignorant, you do the wrong thing, and it works despite your ignorance, because you're doing an easy thing in the most straightforward way. But then you get cocky and try things that aren't as easy, and suddenly the straightforward way doesn't work anymore, because complex things never work the way you expect. Finally, after you've screwed up doing the thing enough, you remember what not to do, and now you can do it without the mistakes. But you're just not-screwing-up the things you already screwed up once before. You'll still screw up new things, because you haven't learned them yet. And you'll screw up again when you forget a past screw-up.
What separates the woodworker from the software engineer is, the woodworker doesn't make a lot of different things, and doesn't use a lot of different ways to do it. The software engineer is constantly doing new things, in different ways. So the software engineer is perpetually rising to their level of ignorance, while the woodworker stays mostly within their level of competence.
This is why there is no system in the universe that will be better than any other at software development. Agile, Waterfall, or anything else, doesn't matter. As long as you keep doing new things, you'll never not be screwing up. But stick to one thing and master it, and it doesn't matter how you do it.
From "scrum masters" to "planing poker" it's all very silly.
I understand how we got here, where many experienced programmers, managers, and bloggers only know capital-A Agile as the watered down version sold via certifications, crummy medium posts, and atlassian flavored kanban boards. But that isn't agile.
I can’t even with the pitch into spec driven development as some sort of high watermark of software methodology.
Agile was always aiming to solve the wrong problem (that code is the bottleneck) but it turned out to be a massive lie exposed by LLMs.
It’s always the poor specs, terrible analysis and release constraints that kill projects.
No, it aimed to solve the "out specs are bad and we need to iterate faster" problem.
"a massive lie exposed by LLMs"
No. LLMs add no insight about the problem and they expose nothing. They just help to engage this well-known problem with another tool.
But that is fundamentally what agile is about. It's not about coding faster, it's the recognition that the specs are incomplete or wrong because fundamentally, a lot of customers cannot tell you what the want until they see it. That's why "build something simple and iterate on it" works. Regardless of how good your spec is, once the coding is done the customer is going to realise that that's not what they actually wanted.
Agile is about working code instead of hundreds of pages of spec nobody reads.
So most of the problems are related to business people and not the development teams? Who would have guessed?
I bet some jerk is going to organize a multi agent scrum process at some point and burn some tokens on this nonsense.
Say you're building a boat. Boats require not only lots of skills in woodworking, but a whole 'nother skill of design, to get a boat that does what you want on the water. It is always time-consuming, expensive, and hard.
And there's two basic ways to build it: with plans, and without plans. Without plans, you have to design it yourself, then try to build it, then make mistakes, maybe even to the point you have to start from scratch. Time-consuming, expensive, hard. But start with plans that have already been built, and you benefit from somebody else's time, money, expertise and toil. The boat is built faster with less effort and fewer mistakes. And instead of needing master craftsmen, you only need journeymen who can follow orders.
What that means is that Agile and agile are not the same thing. Most companies practice Agile, very few are agile.
It means, that you need capable engineers, who are willing to talk to users and build understanding, rather than just being Jira task rabbits. It means, that you might not even need all the intermediaries, each having their own personal agenda and incentives, that engineers usually have to put up with. It also means, that certain layers in an organization have to trust their employees a little more.
It means, that what you build it based on user feedback, rather than what someone tells the engineers, what the user feedback supposedly is, without having a clear idea, what the users actually want. I have seen this first hand. When in the beginning of the product I as an engineer interacted with the users and found out their issues with the platform I was developing, later on the organization added hierarchy layers, shielding or preventing engineers from talking to the users, to build their own job moat, and patronization going on in terms of "Your time is better spent developing the actual features.". In the end the product ended up drifting far from what is good for the actual users, and instead people talked more to B2B customers, than the employees at those other businesses and what they actually need or want.
With each additional layer between the user and the developer (which is also people who want to be paid btw.), the business is inflicting significant cost and increases the risk to steer off course.
In that: if it fails, it is only considered evidence that you were not doing it enough.
The solution can never be at fault, it's your execution, or your devotion to the process (in this case) that was faulty.
It's also true for Cloud providers; that they're not suited for certain tasks is no longer considered an engineering trade-off, it's that you architected your solution wrong, and the answer is to buy even more into how the platform works.
If your microservices become slow or difficult to debug, it's never that fatter services could have been preferable, it's that we didn't go hard-enough into microservices.
If Austerity is not working as an economic model; the answer isn't to invest in growth, it's to cut even more corners.
I feel like I see it all the time.
If a team adopts agile (in any variation) and doesn’t like it, the Agile defenders will appear and argue that the team wasn’t actually doing agile. Agile is defined as the process that works, so if it didn’t work it couldn’t have been agile. If only you read The Agile Manifesto you would understand!
Regardless of success or failure you can say to what degree this is true, and to me this is really that only part of "agile" that is worth locking in.
"We're going to do agile by following this rigid process" is an oxymoron.
What compels you to believe it isn't?
I mean, read the Agile Manifesto. All it does is basically define a set of values and principles. Things like "customer comes first" or "we welcome changes in requirements" or "software must be delivered frequently".
What leads you to believe Agile implies a fixed set of precise, rigid rules?
Can you show a reference of where it is defined like this?
The problem mostly arises when processes are shoe-horned under the guise of 'Agile' in setups where they might not be the best fit by so-called process experts under pressure from management which does not know any better. The authors of Agile Manifesto have frequently said the concept of Agile has been badly twisted.
The manifesto is a minimal set of principles but every real world Agile shop I’ve interacted with has subscribed to a set of processes that everyone in tech would recognize as “Agile”.
The manifesto has become a safe retreat that agile fans bring out whenever someone has criticisms about real-world agile; Whenever someone has a complaint about Agile as implemented in the real world, someone will show up and try to defend it by pointing out that The Agile Manifesto doesn’t contain the specific thing they dislike.
The Agile industry moved beyond The Agile Manifesto almost as soon as it was popularized. We can’t keep returning to it as some safe home base that shields Agile from any criticism.
I'm not religious is any traditional sense, but I'd argue that it's not always the hallmark of a bad dynamic when a system always asks of you to do inner work when failures happen in contact with the real world. Sometimes that's a healthier mode than the alternative -- externalizing the blame, and blaming the system (or the god).
I suspect there a very abstract game theoretic conversation that could be had about this :)
Yes, and that's because God, spirituality and religion make fuzzy truth claims and can be used to argue for and justify anything. God can be used as the excuse to start a genocide and the inspiration to stop it, spirituality can be the way for wounded people to work with their trauma and the vehicle for people without scruples to sell horoscopes or some shit, religion (the same religion) was used to justify and uphold slavery and to fight for its end.
They are containers for our politics, our lifestyle, for who we are and for who we hope to be.
The Agile manifesto is a series of statements in the form "we like X more than Y." It doesn't say anything. To make it mean anything you have to project onto it a framework of interpretation that exists independently of the "sacred text" itself.
So yeah, they are similar, and that's because Agile, sociologically, works like a religion.
If someone handed you a plan for making a jet engine and you messed around with the instructions ... why would you expect it to work? If you have a bug because there are not enough tests ... you write more tests don't you? Why would a method be forgiving when compilers and reality itself aren't?
BINGO. Managers and execs want (or get sold on) "agile" but only want it to affect the structure and processes for the very lowest-level workers. They don't want to change the organization or what they do, and odds are those are really bad and won't let most systems like this, agile or otherwise, function properly.
(The big secret is there's no framework like this that "works" for fixing broken organizations; there are [rare!] well-led well-managed organizations where damn near any halfway-reasonable system they choose will work, so if they decide to do Scrum or whatever in places like that it'll work just fine—and then there's everyone else)
You can never use enough tokens.
With agile, at least no one was charging you for it. Like sure, there’s a cost to the process. But there wasn’t direct agile.com profiting from you.
Meanwhile agentic workflows every solution to the problem is giving more money to the ai companies.
Model is bad? Made more expensive model. Still bad? Here’s an infrastructure that reads huge text files again and again making you consume tokens. Still bad? Here’s a way to easily spin up multiple agents at once so you can delegate work. Still bad? Here’s a new service that will automatically review code. Still bad? Maybe a biggger more expensive model will help.
A concept older than agentic software development is bad workmen blaming his tools.
I mean, if you can't possibly hammer a nail then is it reasonable to blame the hammer?
Or not doing it properly. And I understand the suspicion, I really do; but in hindsight, if you honestly tried to review how an organisation was operating, would you sincerely be able to say that it was adhering to a certain agile methodology/framework/mindset/strategy/whatever?
I have so far not see an organisation that would be following scrum, as it is described in the scrum guide; or kanban, as it is described in the kanban guide. I have seen or heard about various organisations that use these words, but they have little resemblance to what was actually proposed. So I can't really say if agile (or any of its particular variants) work or not. I have not seen honest experiments properly run.
If that's true, wouldn't it point at the process being impossible to implement?
It is a myth. There exists a version of Agile that could be implemented, and it would be the true Agile. The pure, honest experiment that would just work, because Agile cannot fail, you can only fail to Agile.
It signals to me that the process doesn't work in reality. You are better off doing something else.
I think you are purposely omitting the fact that those failures have root causes that come from violating key Agile principles.
Things like "we started a project without a big design upfront and we accepted all of the product owner's suggestions, but whe were overworked and ran out of time, and the product owner refused to accommodate requests, and when we finally released the product owner complained the deliverable failed to meet the requirements and expectations".
This scenario is very mundane and showcases a set of failures that are clearly "not doing enough Agile" because it violates basically half of them.
> The solution can never be at fault, it's your execution, or your devotion to the process (in this case) that was faulty.
Agile is a set of principles focused on the process and its execution. What compels you to talk about Agile and pretend that processes and execution are anything other than the core topic?
If your stakeholders change requirements but don't change allocated resources and fail to stay up to date in the daily affairs to monitor and adapt, how is this anything other than glaring failures to adhere to basic Agile principles?
Agile doesn't have that, there is no functional equivelant of "the cake should be moist and rise evenly". What does "Agile" adoption look like? Faster delivery? Happier Developers? More revenue? Fewer bugs? This is never defined up front and they shift depending on the person being asked. This means you can never actually determine if someone "left out an essential ingredient".
The irony is that Agiles own favoured development practice (TDD) cannot be applied to Agile itself. There is no acceptance test for the process, you can't iterate on something that isn't measured and has no defined outcome.
/r/ididnthaveeggs works because everyone agrees on what the dish should have been.
These discussions are always fascinating in a sort of baffling way to me because I've only had great experiences with what I call agile. Like, you bring it into the team and within months everyone is gushing about how much better life is now. Yet threads like this one are full of people reporting awful experiences.
Apparently whatever it is they're doing involves a lot of meetings and little actual flexibility? The deeply unexpected thing about that, to me, is, if they hate some parts of the process, why are they keeping them? Every team and every business is different and you have to iterate to arrive at whatever will work best for you. That's possibly the one most important point, IMO. Dropping the things that don't work is a key part of that!
Eric Brechner of Microsoft (of all possible places...) gives a great talk on his team's approach, and I've had good experiences using it as a starting point: https://www.youtube.com/watch?v=CD0y-aU1sXo
But again, every team is different. Even the greatest possible theoretical approach is only a starting point.
And like with Switzerland vs North Korea, I guess the key thing is how much ownership of the process those subjected to it have?
Why are you assuming that they are given a choice? In my experience, whenever a team is trying "agile" in some way but hate it AND are given the choice, they drop it ASAP and are 100% convinced that they are better off without it. Those that hate it and don't stop doing it, are doing so because they are forced to.
If it isn’t presented as a theory that might be proven wrong, or an idea that might not work, that’s when my alarm starts going off.
Another signal: trying stuff we already tried that didn’t work, usually with an unconvincing reason why it’s different this time.
You can try the same thing under different contexts and it will probably yield different results (at least in any context that is organizational / social)
So the problem with that is that it's an oversimplification in an attempt to sound smart and insightful and can't be used as a general principle to reason as to whether or not you need to double down to see results.
Sometimes its justified. Like "This is only satisfied when x, y and z are correct"
But then you get
"We will do x and y as a compromise but not z"
And then you have to explain that, the compromise is actually worse.
This reminds a lot of this: "I'm going to try this extremely difficult pastry recipe at home, but I'll use margarin insted of butter because <idiot reason> and a teasponn of stevia instead of the prescribed 200 g of sugar for <another idiot reason>."
Seen this multiple times
The problem is agile as in the original manifesto was an ethos, not a process.
Everything since the manifesto, called agile, has tried to wrap an ethos up as a process, playing lip service forgetting the ethos.
High performing teams are already doing agile, following the ethos without attempting to be agile. High performing teams made to do agile become average teams and low performing teams made to do agile can become average teams.
This is also my observation. I compare it to McDonalds vs a star restaurant. Put the top chef of a star restaurant in McDonalds and he will perform average. Put a McDonalds member in the star restaurant and he will perform badly.
The amount of process needs to be tweaked to your team. Ideally, you can give your star players more freedom.
> The solution can never be at fault, it's your execution, or your devotion to the process (in this case) that was faulty.
This isn't some religious premise, it's the lesson of bitter experience. It's like how when two trains crash into each other the inspectors start by looking for which one went through a danger signal, rather than questioning whether signalling systems work.
Organizations are all different IMO (even if slightly), and you gotta try things and move on.
Is it the fault of Agile if it doesn't work? I don't know, I'm more interested in finding what works.
If your thing sometimes harms people, for whatever reason, your thing isn't safe enough, or easy enough to understand how to do safely.
In my experience, when it fails there's always someone to tell you that you were just doing Agile wrong and they've got a different brand of Agile and a training course to sell you
And when the final product isn't fit for purpose, what do they say when their decision becomes visible?
the off-the-shelf solution is never at fault. It's your execution. You architect your solution wrong. You didn't configure it right. You just didn't adopt it fully enough. The answer is always to dig deeper into the solution and leverage more of its features.
The problem is that the off-the-shelf solution doesn't even have the right feature set needed for the job in the first place.
The do-more-with-less cancer that is infecting so many companies, where the tumor is Continuous Growth.
It's definitely not a magic bullet and I suppose the only reason it's had the staying power it has had is because unlike other project management philosophies, it has an extremely profitable "Agile Seminary" ecosystem.
That’s because it’s a common trait in ideologies. It predates Agile by a couple of millennia. We could add to your examples things like "if it failed, it means you are not pious enough; make more sacrifices", or "if the offensive fails, it means that you are not committed enough; bring more men and more artillery", or "if <whatever totalitarian idea> fails, that’s because people don’t believe enough; purge them". There are many, many examples in History.
20 years ago, this was the meme about XML.
More seriously, this was also the answer about Communism.
It's led to misery every time - but it's always because they just "did it wrong"
This is a cult tactic, for what it's worth
Good way to ensure devotion to a process rather than devotion to a desirable outcome. Seems distinctly cult-like.
For reference, here's all the Agile you need, it's 4 sentences:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
The real problem is that capital-A Agile is not agile at all, but exactly the opposite: A fat process that enforces following a plan (regular, rigid meeting structure), creating comprehensive documentation (user stories, specs, mocks, task board) and contract negotiation (estimation meetings, planning poker). It's a bastardization of the original idea, born by process first people who tried to copy the methods of successful teams without understanding them.
It was really telling at a smaller company that was trying to behave like a big company. I asked a coworker (who had great metrics) what the secret was for dealing with the middle-management-heavy and quite dysfunctional environment. He told me how he did it. Paraphrased: "It's easy. During each sprint, I work on the next sprint's work. Once it's complete I'll know how to make sure things match the work that's already been done and that way its always a bullseye and on time - because the work is already done.". Agile at that company was a joke to the people who got things done, and was a weapon used against people who didn't realise it in time. It sure generated a lot of metrics and stats though. I used to joke amongst coworkers that the company produced metrics, not products.
At my current contract we use "SAFe", "scaled agile framework" which basically revolves around quarterly plannings, but above that is a long term planning of course. (energy industry, scale of hundreds of engineers)
So this sprint shows what you delivered 2 sprints ago, next sprint will be the work you just finished.
Lewis is right that most of these principles were described before the manifesto, but I can vouch for the near-impossibility in many contexts of convincing anyone who wasn't a coder (and a lot of coders too) why these might be sensible defaults.
For every person burned by a subsequent maladaptive formalization of these principles, there was someone horribly scarred before the agile manifesto by being forced to go through a doomed waterfall process.
Ask anyone with 30 years in the industry whether "agile", for all its problems, was a force for good or bad, and the answer will be an emphatic Good!
If nothing else, it gave us ammunition to argue against the impossibility of delivering a fixed thing in a fixed amount of time - which was the universal view from senior stakeholders of what competent software delivery looked like.
This was really all that Agile was ever trying to avoid -- the tyranny of imaginedf omniscience. The bad old way (which I did labor under in the '90s) set up a Gant chart of dependent requirement up front, during a "design phase" which completely de-valued learnings and insights gained along the way as a software system was constructed during the "implementation phase". It was the best we had till then, but many software projects were failing due to their inability to adapt to unforeseen design flaws or to the feedback of stakeholders (once the software finally got into their hands).
I don't know why the ceremonies became ossified and sacred. I guess every movement must confront the danger of settling for form over substance. I do know one thing. You can't build an amazon dot com, a Facebook, or a Grand Theft Auto in a 1-million token context session with an LLM. I'm sure you can do it with many such sessions, but it won't be an LLM that ties it all together properly (again - too much context). And I say this as an enthusiastic user of agentic programming.
This is just a confusing and confused article.
Agile just finally embraced that specs are incomplete and can even be wrong because the writer of the spec does not yet really know or understand what they want. So they need working software to show the spec in action and then we can iterate on the results.
We are still doing that and will be doing it in the foreseeable future. Agile is very much alive and here to stay.
It is not something invented by the Agile proponents.
They have proposed a much more specific variant of iterative development, which at least as I have seen it implemented in any company which claimed to implement it, was really bad in comparison with the right ways of organizing development work, which I have seen elsewhere.
Any high quality product must be designed starting from a good written specification. Obviously, almost always the initial specification must pass through one or more update cycles, after experience is gathered through the implementation. This has always been universally used, not just by Agile practitioners.
There have always existed bad managers, who wrongly believed that a development process can always be linear and who did not include in their timelines the necessity for loops, but that was just bad management, so if Agile proponents pointed to such cases, those were just strawmen, not the best existing practices.
I agree, but what you describe is agile, not Agile (capital A).
Agile (capital A) is Scrum (capital S) where you have Backlog Grooming (patent pending) where the team clears any ambiguity to define a spec (ticket).
Deviating from said spec is seen as Scope Creep (gasp) and might lead to complaints during Sprint Review (trademark).
So yes, agile prefers working software over detailed spec. But typical manifestations of Agile (capital A) are exactly the opposite.
The manifesto says "Stop trying to micromanage your programmers."
It's written vaguely and politely but its spirit is the opposite of mandating daily meetings ("processes"), having trained coaches, or any metrics like story points ("tools").
As the explanation says:
> Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
Then I read Steve Yegge's Good Agile, Bad Agile. It basically says, Agile is just a Kanban queue. And I think I got it, and I think that's working very well. At least from the project management side.
There are IMHO three management angles to look at any engineering project - product, project and architecture. If you are building a house, you need a blueprint to tell where to put what concrete, you need a render (or paper model) that you show to a customer, and you need a BOM and a timeline to make the investors happy. The software is not different. But that's also where there are misunderstandings in what Agile is - the product management, project management and engineering all have different ideas what kind of "plan" is needed.
So in the case of software, specs are like the house's blueprint. In some cases, specs might be useful prototype, in some cases not. It's just not the type of plan that the project or product management cares about.
Regarding the project management angle, for me Agile today is clearly Kanban, and almost everything else is wrong or not required. I often make an analogy with computers. In the 50s and 60s, people tried to plan the work that the computer calculates by creating some scheduling algorithms that plan ahead the use of resources, avoid conflicts and such. Eventually, we found out that simple dispatch queues work the best, don't estimate at all how long the task will take. Just give it a priority, a time slice, and let it run. And I think the same applies for the SW development. And I think it's time that project management people take note from computer scientists - they already know.
Doesn't mean SW development time cannot be estimated if you need to, it's just not a very efficient to do so (it takes extra time, depending on how good estimate you want).
For 20 years, I have seen it working and not working, and the reasons are a lot. It can be affected of level of expertise, quality of documentation, pressure from management, engagement of the clients, etc.
Simple example of failing, and how one of my team overcome it. There is no specification. Option 1: team complains that the specification is bad, and this makes the code quality bad. Option 2: the team pro-actively prepared the specifications, gave them to the client for approval. Writing the specification was, a kind of, added flexibility, that was introduced in the sprints.
Another example, why should the sprints be fixed at 2 weeks. Sometimes, people try to finish for two weeks and they produce bad quality code, because they are time pressured. Be flexible and make them 3 weeks, if the sprint includes things like, preparing specifications, or if the sprint includes pauses for bug fixing. :)
So it is not the Agile that makes the project successful, it is the people. Agile just help for tracking where you are , and what you need to do ;)
Now with AI, you can use Agile again, there are agentic frameworks that support it and they give good results, in my opinion. If the people use it wisely, think what they do, and try to do things better, it will work. Of people are lazy, don't know what they are doing, don't have expertise on software development, it will fail :)
Oh that was it you're right. We have those documents but they are full of lies. Yet everyone can read it and believe it to be true in the way they want it to be.
That's not agile.
> A project where all the work comes directly from tickets with no overarching, agreed-upon document on what the end goal is supposed to be sounds hellish.
Maybe this is why so many people can't even try to do agile. It sounds bad. But it works great.
In a fake agile project or org, the source of truth is a made up document written by the PO or PM and only remotely related to what the actual user says. Devs are kept away from the user by their higher ups, who seek job guarantees.
Well I think this just proves we can slap "agile" onto anything. The people before agile actually wrote things with more substance than the manifesto.
The agile projects you worked on sound wonderful, and I would align "writing specs" with what you describe, at least in terms of the design doc.
The tagline from the handbook: "Agile started with a manifesto. It ended with Jira."
Handbook: https://agile.flights/docs/introduction/why-flights/
So one of the main points in this massive, 700 word Treatise (which I do hope you will find the time to read) was that nothing Agile practitioners slapped their label onto was actually novel.
Why re-invent agile, when agile itself was just a reinvention by "the kids" (your words, not mine) of things people in the 1970s already knew?
One might as well go straight to the 1970s directly.
Is anyone actually doing true waterfall development any more? How would that even work with the amount of open source software in use? The world is fundamentally different now than it was 25 years ago.
Stuff like SOLID and Design Patterns etc. are such good ideas that they've been incorporated directly into the design of modern languages and frameworks. It's natural that someone would pick up Design Patterns today and think it's all pointless. That's because the book was written in 1994 and it wasn't pointless to say it back then.
I guess this is why history tends to repeat itself. Many people can't internalise why something is bad unless they've experienced it themselves. Many more don't even read about it in the first place. Scary to think.
I've found that Finance, and the Tax Office of any government, rarely care about your Agile processes. They have their yearly cycles, and C-Level will always want to follow _those_ cycles.
Then, for those that have schoolgoing kids or work with people that have schoolgoing kids (aka: everyone everywhere): there is the school vacations cycles.
These too rarely care about your scrum rituals or PI planning. This means that your calendar is not a reflection of reality: July/August barely exists. Same goes for November or December. And at the same time December is full of actual deadlines due to end-of-year financial cycles.
And finally: the complexity of the work itself rarely lends itself to the linear timelines people expect.
Rarely have I met a Product-person, or a SCRUM person, that actually understands this. And can account for it in their Agile way of working.
End result: a continuous stream of disappointment. What fun times we live in.
e.g.
- 45 minute "standups" (!?)
- PI "planning" that consisting of deadlines and glorified multiplayer MS Paint
- Rigid adherence to ceremonies or processes that add zero value
- Retros that focus on complaints and venting with no actionable outcomes
- etc etc
Every time I've introduced Agile to a team or project that was new to it I was always met with skepticism. But 6 months down the line noone on the team/project wanted to go back to the "old" way of working. I don't even really care about any text book definitions. These are the only things we try to stick to:
- Short, daily standups
- Planning based on risk reduction
- Estimates based on complexity (ties in with risk reduction)
- Actionable retro items
- User demos every sprint (makes it easier to pivot - users rarely know what they want)
Of course. But you shouldn't run retros that are focused on it.
With the years, I've come to think about it as a sing and dance designed to make the project managers, PMs and sales feel like the actually impactful ICs considered them.
There's something really absurd about making programmers sit down and say it's a 5 or 8 effort, then punish them for being "wrong". All it achieves is reduce velocity at best, with the illusion that it's for the greater good.
a) engineer spends more time trying to guess which direction the wind will blog b) engineer starts sandbagging estimates c) engineer changes nothing. looks bad next time the imaginary goal isn't met. "bob needs help estimating".
So now you write specs, and then an LLM, which is known to be overly compliant, will handle the implementation. If you don't see the issues with this workflow, you for sure have learned nothing about how the software development process evolved.
Put your hand up if you are ever programming with poor specs?
Put your hand up if you have a better idea of what really was wanted after the first cut?
And what I really dislike is those that try to design a Swiss Army knife from day one when they haven’t a clue. Jump immediately into over complexity.
Yes, that would be the first paper I linked to in the article, "Managing the Development of Large Software Systems" (Royce, 1970).
The first diagram is the classic waterfall diagram, used there for illustrative purposes as an example of what not to do.
Highly recommend it to people - it's short but a real breathe of fresh air. Mostly still applicable today.
It is difficult to take the author seriously after his claims that the Agile Manifesto is only "platitudes" and "near devoid of meaning"...
I like a saying from the LEAN manufacturing culture - "The Process is the Expert" but that comes with a caveat, each and every team member is a Process Engineer!
But if agile is criticized... only worse alternatives are given, if at all. Here, spec-driven development is inferior, as in most cases the goal is only vaguely known. Cyclical development is not some hollow mantra, it is how life works. All the rituals around it were just to faciliate more communication. A lot of people in this field just hate that, they want their tickets and to be left alone.
Now that implementation cycles are even shorter, there is even less manual need for coding, agile methodologies will be actually more prevalent.
> All of these things were later claimed as Agile innovations
Are there some references that demonstrate that? [EDIT: that the signatories thought they were their own innovations]
And if so, is that a bad thing? Ideas are repeatedly rediscovered. This article isn't called "Saying goodbye to Royce, Bell and Thayer", and I'm wondering why not.
For example, https://www.infoworld.com/article/2334751/a-brief-history-of...
It's as if people believed that all the microcomputing software of the 1970s and 1980s, from VisiCalc to Zork to the Macintosh, was done by waterfall design.
In all seriousness, this pattern is probably hard to avoid in any reasonably complex entity/environment. If any such situation would be solved in a global solution (aka silver bullet), it would be used by everyone. As this seems not possible, any framework like Agile, Communism, … can only be a guidance to be applied locally, and broken internally and by external factors in many ways
On a lighter note . .
The world of overly-complex CCS (component content systems) like DITA has made this "Agile flavor of treadmill[1]" into the entire business, greased with liberal squirts of FOMO and "Industry Standards".
It rhymes with capital A Agile in many ways, although in the case of DITA specifically I'd posit that the underlying assumption of the spec is a vast category error: that natural language has formal types.
[1] i.e., "you aren't doing it properly" . . . and with every change in technology the DITA / XML priesthood claimed to hold the keys to unlock it. SEO? Information Typing. Web Content? XML/XAL pipelines. Big Data? Content granularity. LLMs? Information typing and schema will "help" llms and not just be an unholy clog in the guts of vector embedding operations. And yet, the years go by, and all of it has worked and continues to work fine without switching the world to DITA (and a writing universe of strict validation based on speculative assumptions).
Isn't that the biggest issue here, though? I think all of us can agree on the four sentences you wrote, but this only works in a team of professionals with shared goals (and alignment on them!), each individually competent and motivated.
That is the case for a small founder team and maybe a while after that if you're lucky, but IME the more people join a company, the more the alignment and median expertise lessen. At some point, you need to introduce control mechanisms and additional communication tools to rake in the outliers.
I don't really have a better answer, though…
It's not a system of management and it won't work if the way you're managing sucks. Nothing targeted at a similar "level" as the various "agile" systems will either, though.
But at one point you need not one team, but a hundred.
Counterpoint: I learned a variant of agile in exactly this type of environment, long before any of this was publicized. Which is another point: agile wasn't something new, certainly not at the time of the manifesto, which was a compromise document. But not even before the manifesto. XP, arguably the first agile methodology, very clearly and deliberately stated that this is nothing new, just a distillation of things that experience has shown to work well.
Anyway, at my next job I introduced agile (small-a-agile) to a team that was anything but skilled. In fact, that team was where the leftovers of that particular development organization had been shunted (public company, very difficult to get rid of people). When I arrived, the team was as non-functional as the software it was responsible for. Well...
We rocked.
And all the team member improved dramatically in skill during my tenure there. Including myself.
We did not do Agile. No scrum, no standups, no sprints, none of that BS. We were agile. We focused on the technical practices. Test first. Red-green-commit. To trunk, obviously. Because if it's green why on earth would you not? Do the simplest thing that could possibly work. We had a design for a database and then never found a need to put it in...so we didn't.
It took a while for the other parts of the org to adapt to this. The answer to the common question "well, when can you deploy?" was always "now". Well after a quick look that the tests were, in fact, green. So they stopped asking. The tests were rarely not green, and when it did happened there was usually a quick "Oops, I'm sorry" and they went green again a couple of minutes later. Our ops team got bored very quickly. Put jar on box. Start. Forget about it.
What made the experience scientifically interesting is that we had a control group: the main team, much larger, working on the "important" software with all the "good" engineers started with a new project about the same time we did.
They did Agile. Capital-A. Scrum, sprints, standups.
They did not deliver and in fact the project had to be completely reset about two years in. My team-lead (we were co-lead, I did mostly internal/technical, he external/managerial) then got to take over that team as I left for Apple.
TFA, incidentally, is just about as good summary of misunderstandings of agile as I've seen.
When playing piano, the condition you are measured by is acoustic harmonies in the air, not finger movements. The only reasonable advice is either practice more or give up. If you are tone-deaf, it's not reasonable to expect you will learn to play the piano.
"Oh, feature no 32 is going to take months and we realised that users can just...."
"No"
It's a farce.
I have worked at an org where team members were not allowed to create tickets because that was the scrum master's job and the product owner had to approve all tickets etc. Who can even think that is a good idea??
Not sure what the solution is. There might not be any.
this is 100% backwards for anything safety-critical or that needs to be maintained past a butterfly's lifetime. this is what encourages yolo-driven-development instead of considering what actually should be done, and this is why agile or Agile or whatever formalization or bastardization of it can not be considered software engineering, but merely code monkeying.
The others I get, but only after having already spent years in software. I guess like many things you have to see the other way before you can appreciate the better way.
It's got loops and infinity markers, AND iconography representing humans!
At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.
The real claim of the agile industry is that they know what can work and that you can find it using agile. That also has real value - if true.
Depends. There are companies [1] making loads of money out of it. Charging for certification and imposing the idea that either you are certified, or you are going to fail. They are even eating the lunch of PMI, as PMI (PMBoK) is turning into an Agile manual. Where I work is being expended literally millions per year in Agile.
Charging people for Agile via his company ThoughtWorks (which sold for 785M) is how Neville Roy Singham made the money to fund far left groups in the US from his base in China.
Because self-determination was the defining component of every agile deployment I've ever been involved in or personally driven. I don't think I can really picture what something calling itself agile yet lacking this component would even look like, honestly -- hoping you will forgive me this failure of imagination.
Isnt that in itself "agile"? And I specifically dont mean following a religous ceremony plan etc but recognizing that a part of their process isnt working and then changing it. To me thats the entire point of actual agile. You try a process, it doesnt work, you analyze, and adapt.
That's not true for the way I understand agile. The way I understand it, the testable outcome is whether the principles of the agile manifesto are satisfied
For example, is your highest priority to satisfy the customer through early and continuous delivery of valuable software? If not then you're not agile.
If and when I see actually-doing-agile fail, I'll change my mind. But so far I've seen a direct correlation between the extent to which an organisation was actually-doing-agile and the effectiveness of that organisation, across a wide range of industries/environments/countries.
Push back on that. Agile says other things are more important.
All AI has shown us is that we should probably write specs before we code. That was true before AI, but LLMs have just shone a light on this, and it will remain true if agentic coding falls out of fashion again.
Never once did I advocate waterfall...
Read the Royce paper, seriously. It's short, he's a much better writer than I am, and if nothing else it's a fascinating looking at the old state of the art.
Which they almost never have.
Which means that any true agile cannot be industrial. And therefore any industrial agile cannot be true to the principles of the Agile Manifesto.
Yeah, spending the amount of words in this thread trying to diagnose or complain about this simple problem in abstract strokes seems silly and frankly confounds me when considering the amount of time people wish to waste discussing the problem.
As with political parties, bad gentrification in cities, and all the rest, once money and consultants turn things into an industry you're pretty much fucked.
People should just immediately stop taking people with conflicting interests at face value when they talk. Stick to concrete details when you talk about stuff, avoid industry terms, don't let them turn things into abstract and general discussions. It only feeds the trolls (consultants) when you even complain generally about it.
Fight it with your day-to-day actions, not so much with your words. And then let it die in silence, it will die faster (I'm referring to any tech topic captured by consultants and monied interests).
I still doubt that your average person would see any real health benefits from eating only meat, even if they ate enough organ meats to actually get all their vitamins and minerals (the lack of fiber is a particular concern) but I guess you're right that there's a version of an "all meat died" that doesn't promptly kill you through malnutrition, you got me there.
Since then I've been on teams with any number of pathologies. From developers it is sometimes the desired to be special - those ones who want to work on their bit of the code and not let anyone else touch it. From managers it's things like forcing the way stories are split so that they're always too large and can never fit into a sprint - because they think that everything must be a "user visible change". Management types also sit in retrospectives and use them to crap on everyone. Product managers demand features which they don't know will really interest customers and are inflexible about them - they want "everything" just in case and that delays the work and deletes any chance of a feedback loop.
The good agile feeling came from being able to have control and be successful. When it's messed up, you're out of control and cannot avert disasters. Whatever method you want to call it, I think we need to feel we're in control enough to succeed.
There are a million possible reasons for failure, but here is a very easy one: It doesn't matter how good you feel about the development process, if the company has the wrong objective. You will still end up being frustrated, and failing. Of course this will have all sorts of pathological and uncomfortable ramifications.
So while it is easy to say, "just act this way and you'll have success". You're not actually appreciating all the hidden elements that allow any hope of acting that way. You've been lucky enough to be in situations where it happened to work (ie. the rain dance made rain), but that does not mean it's actually representative, or that the prescription actually captures the critical information needed to ensure success for other people. Instead, you've described a rain dance.
#1 IMO is that if the company you're in is non-agile in its general attitude, which is influenced by its own customers, then everything is geared against you.
That isn't to say that something like Kanban might not be usable or better than no plan at all but certainly scrum is not some universal solution.
That's where it becomes "impossible to implement"—you can't impose it as a cookie-cutter solution driven and controlled by management, and get much good out of it, yet that's the usual way it manifests in the wild. But that's not so different from anything else management might push in its place.
"Agile" is a very vague and shapeless idea which is hard to design an experiment for; but I would settle for clean experiments with well-defined methodologies/frameworks/strategies/whatever. Specifically, for scrum or kanban. Whenever people talk about these two, they seem to misunderstand them more often than not.
Whatever you do instead, you will also cargo-cult to some degree and fail equally as badly at.
For all the "You're doing it wrong!" I've seen in industry with respect to agile, I've also felt that every team I've been part of that did some version of it, seemed to function OK. I always found the "Agile Manifesto" a completely silly nothing-burger, but always understood the core tenet of 'agile' to be "employ tighter feedback loops", which... is sort of mostly how it plays out in practice??
The shape of what Agile meant in each of those teams was very different from one another. It would be disingenuous to say "the ones that succeeded were truer to Agile".
If Agile can be summarized as "employ tighter feedback loops", the whole Agile thing was beyond useless. A single sentence, as useful a tenet as it may be, does not a philosophy make. And this idea was not even new by the time the Agile manifesto came out (as explained in the linked blog post).
Not just that, Royce's original paper that coined the term "waterfall" in 1970[1] can be summarized as "employ tighter feedback loops" compared to top-down design (figures 2-4 in the paper).
A suitable comparison would be to be faced with a nailgun and proceeding to criticize it on the grounds it doesn't have a handle, it doesn't pull nails, and it requires electricity to run.
While you complain about those detailed those using nailguns are an order of magnitude more productive at the same task, and can still carry a hammer in their toolbelt.
- [1] Get 20% off your Hammer Master™ certificate with referral code THUMBPAIN
My thoughts when PE forced Agile on my employer were dismissed as "you're the technical expert, we're the process experts".
As someone without decision power, you read words of empowerment but your reality is a different one, and you're left resolving that dissonance on your own (quietly, otherwise you get pushed aside).
As always, the problem isn't the process, the problem is the people. There's whole industries out there set up to sell A Process, so they come in and try to force something like this on you. They want to stay in business, so they need to make sure they have something to sell.
That's the dysfunction - a company that is forcing this laborious process on you, rather than giving teams the autonomy to figure out how they best work.
Agile works best as a toolbox of practices you can adopt, mix, and match to solve whatever problems you have. Do you need to work to a fixed schedule, or provide delivery estimates? You should probably have a way to regularly estimate your work. Are you struggling to actually ship and do things? Maybe it would be useful to plan things on a smaller, more frequent cadence.
I would go as far as to claim the problem is middle management types, who feel pressured to adopt buzzwords and want to micromanage things to cultivate an image of control and progress to justify their role.
It's the same type that brags about scrum but don't even bother to show in standup meetings.
That would clearly be a problem that falls well inside the domain "you are not doing enough Agile".
A key principle of Agile is literally "Business people and developers must work together daily throughout the project."
If a team suffers from that disconnect, it's failing Agile.
More to the point, whatever they are doing is not working, and Agile would fix it.
I actually think OP's criticisms apply mostly to Scrum. Scrum is well defined but its adherents' wont hear a critical word said about it. "You just werent doing it right" even when you were doing it precisely as described.
I don't really agree. The set of principles are quite straight forward. It's things like delivering software frequently, accommodating new requirements, continuously looking into improving processes, business types and developers working together, etc.
Then you have concrete executions like scrum vs kanban. Agile doesn't specify one or the other. Retrospective meetings are popular, but aren't specified by Agile per se.
How it's been implemented at every company. They are always a fixed set of precise, rigid rules.
That's it really.
> I'm not smart enough to NOT keep it simple
Yeah, sometimes I feel that most of my "amazing architecture skills" is not understanding what 90% of that stuff is supposed to do or why, and hey, maybe we can just do without it?
For reference: what we did was replace an existing system, which was running over a hundred processes on about a half dozen boxes. We replaced it with a jar.
The jar was around 1000x faster, 100x more reliable, 10x less code while handling around 10x more of the domain.
Not at the start.
I guess if the point is that agile doesn't work for incompetent teams because teams become much more competent through agile, then I'll concede the point.
And if this tenet is all Agile is, then it contained zero new ideas or contributions.
The point is still unaddressed, isn't it?
Well often the real world forces it upon you. As in customer will switch invoicing system on September 20th, integrations have to be ready by then.
We have a lot of this, and hard cut-off is very frequent. If we ain't got all those deliverables implemented by then we will lose customers.
If the org focuses on the customers one can work together to find a way.
When there's a time constraint one has to be able to winnow out the real must-haves from everything else.
That's OK, the latter is not incompatible with the former. Agile vs waterfall is orthogonal with having to commit to deadlines to deliver features.
Then you have a choice - find something to cut out or accept a later date. This is a mode of thinking that I find non developers have difficulty accepting. They want it all and they want it now and their modus operandi is to keep pretending that it's possible and suggesting that if they shout and stamp a bit that it will somehow rescue the situation.
This is crucial to get surfaced early, along with how painful it is to actually move said date if possible.