Coconut Headphones: Why Agile Has Failed(mikehadlow.blogspot.co.uk) |
Coconut Headphones: Why Agile Has Failed(mikehadlow.blogspot.co.uk) |
I'm a consultant. Officially my title is Senior Test Engineer and usually my job is to go to a company where shit has already hit the fan or will hit in a short time.
The reality is that the company has a bunch of developers who are mediocre at best (and maybe one or two good ones) and managers who do not understand the situation. And of course the budget is already been eaten.
They want to ship their product and make some money with that so they can continue to live their life. If I go there and say hey, you need better developers, more rapid prototyping and managers who are technical enough, nothing will change. Nothing will change because there aren't "rock star" developers avaialble, there isn't time to find new managers and definetly no motivation to change everything right now.
Sure in some sense I might be correct to say so, but my goal is not to show my superiority, but to improve the situation.
But if I teach them a little Scrum, help them setup some CI and so on, they almost always perform better.
And then this statement "Because creating good software is so much about technical decisions and so little about management process, I believe that there is very little place for non-technical managers in any software development organisation."
No. Good software comes from understanding the needs of your customers and meetings those needs. Shitty developers have and will create awesome products just because they know what the users want and need. "Agile" helps even the shitties companies to meet the needs of their customers.
You're talking about very abnormal situations. Companies that are so technically incompetent they have to hire you.
And then saying that we should listen because in your abnormal experience of dealing with teams so dysfunctional they can't even ship bad software that agile at least gets them going a little bit.
I suspect almost any change would get them going a little bit as the change itself is the trigger.
EDIT: The more I think about it, the more your comment reminds me of the people defending the other fads of years gone past. Workflow diagrams, OOP-design (as a be-all-and-end-all process), UML, Worflow Engines, RAD tools. Your comment reminds me of the defences of them. These things worked because those desperate teams need a light to guide them, not because of the tool itself.
And with each of those tools, it turned out the tool itself is mainly counter-productive to functional teams. I think we're realising Agile is another of those false prophets.
Company has an average team, maybe even bit worse + 1 good person. The managers are not that great either. Everything is average.
The team has been doing something, nobody really knows what, nobody really knows at what pace. Nobody knows what they should be doing.
Occasionally the company releases something which sells a bit and customers give some feedback. Mostly not that great.
When you tech them a bit of Scrum, help them get builds every night and help managers to try out the product once in a while (remeber, not all software is a website, some are airport weather observation systems). And help the company to get some of their customers to try out the software once in a while even though it is not "ready".
Suddenly the results of the team is visible to interest groups and interest groups needs are visible to developers, managers can get some idea of the velocity and everybody can have a better idea where we are, where we should go and when we might get there. Then iterate and improve.
What we have done here is that we improved the quality of the company with really little cost with our existing talent pool.
Yes, this is really common.
Agile helps force groups like this, who have been doing things a certain way for a long time, to re-think how they manage requirements and release schedules. Most likely, the developers at the bottom level have been screaming about this for years, but middle management doesn't hear them because they're stuck in meetings all day having to explain why all their projects are over budget because change management costs are through the roof. Along comes Agile as a solution to all of this.
This isn't a technical problem, it's a management problem. So yeah, a lot of these companies hire consultants to come in and tell them how/what to do because consultants have no skin in the game and aren't involved in the typical petty management squabbles.
The improvement isn't really specific to bringing in Agile, but rather to giving the team a clear focus, and way to measure their improvement in deliverables. There are many ways to do that, it's just that Scrum happens to be a relatively straightforward and lightweight way to introduce that in a way that is compatible with many of these teams.
Unlike a lot of those bad approaches, though, I don't think Scrum is necessarily a hindrance to a properly functioning team. There are a lot of good ideas in Scrum that are designed to get obstacles out of the way, IMO.
Agile has to start with management adoption. And not Fake-Agile™ either. I've seen a PowerPoint slide a few times now that lists "Iteration 0 - Envisioning" as a bullet point. It's like RUP got painted with Agile terminology. And everyone in the room nods their heads knowingly because it's what they've been doing all along, only now they're Agile.
Depends what you class as "good software". My colleague left some code which "worked" and "met the needs" (at the time). I made a small change to the database it interacted with and it fell over. I had to go in a debug it, but it was so badly written I had to completely refactor it just to debug it properly.
So her software met the needs, my rewrite of it met the needs, and has been far more reliable and maintainable. I would say that is better software.
But it does not follow that doing good software is a technical challenge. It does not matter how fast your algorithm is if nobody wants the results that it spits out.
Defining good software is really hard task, but in general what I mean by it is that good software increases the well-being of its users. Yes, one aspect of that is the technical quality of the implementation.
The Moon Lander software on the Apollo spacecraft probably had several defects, was unmaintainable, but it got the job done several times.
MS-DOS had several defects, was more maintainable than the above software sure, but it transformed MS.
Sure, it is better when the software is maintainable, more flexible, etc, and I don't advocate against it. What I do advocate against is overthinking all of this and not shipping.
1. Good managers to manage new or mediocre engineers. 2. Good engineers to control new or mediocre managers.
No system will completely mitigate bad managers or bad engineers, and if they're both good the right system will manifest itself.
The main audience is shifting toward the shitties. It's the circle of life for cultural movements. Nothing wrong with Agile. It's just not my thing anymore.
I've known some really good developers who go belly up in Agile systems because they can't cope with the tight deadlines, having to ship something in two weeks, or people constantly breathing down their necks.
Over the years, I feel like Agile has been used to deliver mediocre products, and in turn, turned good developers into shitty developers simply because the "business" wants to ship something a lot sooner than it should be. It's a downward spiral since all they care about is something getting built, not the inherit quality of said product.
Hence, its because of Agile there are shitty developers and shitty products being shipped, not the other way around.
But that isn't "agile" it's just some good practices with a label du jour. Good software companies were doing nightly builds twenty years ago (when doing such builds was a major technical accomplishment). The agile cargo cult against which the writer is ranting is far larger, more insidious, and counterproductive than that. E.g. Our project is infested with "release train engineers", "coaches", "rally consultants" and other highly paid morons who actively damage the project while claiming credit for any improvements we manage to accomplish behind their backs.
That said, the thing I dislike about this new "Agile has failed" meme is that it discounts how much the movement has changed the status quo. Sure it make sense now that small iterative releases, continuous integration, automated tests, customer collaboration, etc will produce better software. And yes there were teams that did this before Agile became popular, but it certainly wasn't the norm.
There used to be no standard way to write/run unit tests. The class of software called continuous integration server did not exist. Gant charts were completely normal planning devises.
I've worked in environments that literally believed that a senior developer could use uml to design the entire system down to the method level and then it would be a trivial matter for a junior developer to "fill it in". That sounds insane now, but it happened throughout the software industry.
Agile didn't fail, it worked so well it became the norm, and now the term is being co-opted by opportunists. That's sad, but let's not let that detract from the very real success story of the Agile software movement.
The way to reconcile his view and yours is to have parallel engineering and product management structures. There are non-technical product managers, but there are no non-technical engineering managers. Every team has engineers and a product manager, but neither reports to the other.
That way you get people dedicated to understanding needs without putting them in charge of things they aren't qualified to understand.
Again, it's possible to do, but it takes talent that understand both the business needs and how to manage a development organization (note: this is not the same as knowing how to code.) That's rare to find through a corporate HR process.
I definitely think the consultants get a good chunk of the blame. But as I explain in detail elsewhere [1], I think that happened because executives, the consultants' customers, were mainly interested in buying BS. Not consciously, but when they were offered a choice between hard Agile and easy Agile, they bought easy Agile.
It's sad, because in the 2001-2005 timeframe, there were a lot of great people doing a lot of great stuff. There are still some doing great stuff today. But yeah, among most of the people I talk to that are "doing Agile" (as if there were such a thing), Agile is just putting new labels on the same old power dynamics.
And it's those power dynamics that are the problem, and no matter what methods you supposedly adopt, unless you change those, the system will return to making powerful executives and managers feel safe and in control. At the expense of productivity, quality, value delivery, and a whole lot else.
[1] http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...
Agile is part of a more disturbing trend I've noticed [0] of companies striving very hard to turn software into a literal sausage-making factory [1] and to make software engineers just another cog in the machine or a replaceable part to fatten the bottom line with a lower salary. This is provably the aim of some of the top companies given news on no-hire agreements. [10].
[0] with Java being the favored "currency" of programming languages being the other disturbing trend-- it's much easier to replace a Java programmer than any other for a reason
[1] you know what they say about how sausages are made
[10] http://gizmodo.com/apple-guilty-of-price-fixing-730018979
In some ways I don't blame people. Industrial approaches to organizing people provided a major leap forward for humankind. And they work well with primate power dynamics; modern corporate structures are basically feudalism in suits. It's natural that people would just want to take the top-down, command-and-control structures and replicate them in the new thing they're doing.
But they just don't work well. They don't even work well for industry anymore; there's a reason that Toyota, which has a very different management philosophy, wiped the floor with the US auto companies, which stayed stuck in the early 20th century.
To be fair, Agile started out to be 100% the opposite of that sausage-factory approach. I know a lot of the early players, and they sincerely had a very different vision. It makes me sad to see their work used as just another stick to beat developers. Meet the new boss, same as the old boss.
Agile works fine for teams that embrace it. It's going to work totally different from team to team. The real point is to find the process that works best for your team to deliver software that meets the customer requirements and budget. Agile for a team of 1 or 3 is not the same as agile for a team of 10 or 100.
Agile tends to fail in two places and they are both communication related. First, developers give terrible estimates because of pride. They don't think through the problem, they don't consider complexity, and they want to look awesome so they ALWAYS over promise and under deliver. That makes them look bad and destroys confidence in the project because it's dishonest.
Second, managers will take estimates and turn them into deadlines because that is kind of their job and they are doing the best they can with the bad information that developers give them (see bad estimates above). A good PM needs to really push their team for real esteems. Poor estimates lead to poor communication because when things go badly, nobody wants to send up the signal flare for help or tell the boss that the project is not going to hit the deadline. This is often compounded when the client decides to firedrill a feature or bug fix mid sprint, and the manager doesn't push back and say that it is going to push everything else back.
The best estimates are the ones that are the most honest, not the shortest.
Between the bad estimates and the poor communication that comes out of it, there are plenty of times that "Agile" goes wrong, but it's not agile's fault. It's your fault. It's your team's fault. It doesn't work if you aren't willing to continuously tweak and reevaluate the process until it fits your situation. That means doing retrospectives and making improvements based on them.
Continuous improvement makes agile work. A stagnant process is doomed to failure as requirements and resources change over time.
One of the best things for me about teams that were working well is that everybody was in charge. Everybody felt responsible for the outcome. Everybody cared. Everybody knew they could make things happen, and that differences of view were resolved through collaboration and experimentation, not power.
You can see that explicitly in the structure of Extreme Programming, a major Agile process. There were developers and there was a product manager (called "customer"), and neither controlled the other. Indeed, people created an XP bill of rights that described the balance of powers:
http://agile.dzone.com/articles/worth-repeating-xp-bills
You can see that working in the large at places like Spotify, where teams are cross-functional. People do have managers, but they aren't on the same team, and technical people report to technical managers, not generic businesspeople. Those managers aren't "in charge" in the typical sense. They mentor and support the people working directly on teams. They only really manage when things go wrong.
And I think that's what the Agile community was going for early on. It's a shame that fell by the wayside.
It got hell when the company hired very smart and capable guy who turned out to be very lazy. Nobody is in charge in that case means also that it takes too long time until someone in charge finds out about the situation.
When he does get to concrete points, one of them is "Short feedback loops to measurable outcomes create good software." And yet "two week iterations" he calls "agile nonsense".
The overal tone is kind of "technical macho" to me.. like, Real Developers don't need management and if you slipped then it's because your programmers suck and you should just hire better ones.
The sentence should read "The core problem is that bad managers will usually fail, or at best be counter productive, whatever the methodology".
Non-technical agile management uses points to measure the work that gets done because they don't know any better and they "need" to measure something.
The technical architecture starts to go downhill over this never ending treadmill of the prioritized backlog with the non-technical task masters whipping their developers to get more points done.
Sure, we can educate the non-technical management over keeping a low standard deviation of points. We can bargain to get technical cleanup time (marked as chores). We may even have a debt cleanup week out of the month.
However, the spirit of the engineering endeavor is lost to the marketers, or their henchmen, running the project. It's too bad because subpar products and subpar code are the result.
I just continue to write software the way I always have. Kind of disciplined version of cowboy coding I would say. Sometimes I have design documents - when I feel it helps. Other times the problem might be less clear, built a prototype, and iterate from there. It really depends on the problem you are trying to solve and the time frame you need to do it in.
I get software working, usually in a decent time frame. Is that not what agile is supposed to be about?
[1] Assuming your codebase isn't already a monstrosity. If it is... well, then nothing works.
Spot on. Whenever I go into one of my own anti-"Agile" rants, among my main points is that formal organizational process is simply unnecessary with talented, motivated development teams. In my long experience across many start-ups (which tend to get the aforementioned sorts of teams), if you just put a bunch of really smart people (devs) in a room with a project to do, Good Things happen. They know what needs to be done. They know how to do it. Any kind of formal development methodology just gets in the way. Management in those environments (and I've done that) is about care and feeding and listening and gaining consensus. It's not about religion or process. When I first read the Manifesto, not long after it came out, my reaction was YES! But in no time the Formal Methodologists came out of the woodwork and hijacked the whole thing, turning an attractive philosophy into just another management fad, one with nearly as much religious orthodoxy as what it replaced. Agile, with a capital "A", can't die quickly enough.
That's why, at least with Scrum, the agile manager, if he or she is part of the same team, reports to the same manager as the development team. The agile manager's job is to keep the agile process running smoothly, removing impediments, etc.
Of course many individual teams fail at agile but those teams would probably fail no matter what approach they were using.
This isn't the fault of the agile manifesto. It's the fault of consultants who used Agile as a buzzword. The same consultants who never understood agile or cared to; producing apps riddled with bugs.
Agile was heading to the abyss the day it was co-opted into a marketing buzzword.
Then support queries came in for the features we developed previously. At first a couple of team members would split off and work on the existing features while the rest of us carried on working on the new hotness. As we increased the number of (quite diverse) features, the more diverse the support queries got.
Now we're basically no longer a team, but a group of individuals working in different areas, who happen to be in the same stand-up each morning.
I am actively looking to fix this problem. I'm sure this must have been discussed already but I can't find it! Any suggestions?
I've been involved with a few inception meetings. Two of them had terrible consequences for the life of the project. I got into a disagreement with a high level company executive in one and an "inception master" consultant in the other. They had no accountability for their "guidance" and they nearly killed the projects from the start.
Well said.
Besides the objections concerning throwing out the baby with the bathwater when it comes to Agile, I also object to the notion that non-technical managers cannot manage a software project.
True, 90% of all non-technical managers do not have the knowledge necessary to manage software development, but very little of that actually includes hard technical knowledge. As a tech manager with 25+ years of development experience, most of my work involves management skills, not technical skills. The technical insight needed can be taught to any smart non-technical manager.
Also, managing a software project is not about being "in charge" and micromanaging, but mostly about serving, protecting and coaching a team. People in the "no management" camp mistake management for hierarchy. Being the manager is a team role, just like being a back-end developer, and interaction designer etcetera.
None of this is about Agile or management, it's about two sides, old school hierarchical "programmers-are-codemonkeys" management and tunnel vision "we-don't-need-no-stinking-suits" developers, trying very hard not to understand each other.
And using Agile as a stick to beat each other with.
If Agile is dead, it's because it's been brutally murdered by two factions unwilling to face their own shortcomings.
I like the Cargo Cult analogy, and the author paints a fairly clear picture of what misapplied Agile tends to look like. Just not clear on what it should look like.
Well now you know why agile was just iterated waterfall in most organisations...
Success comes from trusting your people, having a clear goal the team believes in, and a team of the right size to accomplish said goal. Every successful team I've been a part of had all three. When even one is missing, there is much dysfunction.
The agile rituals are there to try to make those things easier, but they are just a way in: If you meet the important principles, you do not need them. Just look at the Valve method: No management, no ritual, but a hiring process that attempts to just get the kind of people that focus on those principles like a laser.
@sbellware we should bury agile, mourn the dead, and get on with establishing something that is designed to be resistant to being so easily undermined
https://twitter.com/sbellware/status/443397344436817920
Which makes me wonder, is it even possible to create a "thing" (idea, movement, methodology, system, whatever) that is resistant to being undermined? I don't think so. It seems like a natural cost to attaching a name to something.
This is why I try not to talk about "Agile" these days, but rather just to try to discuss the specific principles that have proven valuable for me.
Enforcing standards doesn't make it proof against being undermined, but it does make it harder. There's still a tradeoff between being popular and being great that's hard to resolve. Especially since a popular, non-great thing is more likely to get lots of money and attention.
Learn about all good and important methodologies, and take what fits your character and team. Any religious treatment of any method is not natural.
My current project feels more sluggish. Long planning meetings where just one or two people decide on the story points. Somehow we never really get all our stories done in our sprints. This isn't a big problem, but it makes you wonder how we plan this, and why we plan this.
The main differences I can tell between the two: this project we have a lot more documentation. All on wiki, but on the previous project, be barely had anything, and just asked people what the idea was and then did it. But my current project is at a bank, so it makes sense they want more planning and documentation, and less just figuring it out as we go.
At another job, our Scrum wasn't really Scrum. We had standups, but that was it. No sprints, no poker, no scrum board, no burn down charts. But what this article makes me realize is that that project may actually have been more agile. Very few formal planning meetings, lots of informal communication and programmers just doing what they're good at and calling the shots.
When you raise this at retros, what sort of response do you get? I believe you can measure the health of a Scrum team by a) what issues get raised in a retro and b) do they get addressed?
The iterative aspect of Scrum is also very much about iteratively improving process.
I got involved with Extreme Programming in 2000. Loved it. Best thing since sliced bread, yadda yadda. I was completely spoiled for other kinds of work.
So when that contract ended, I went looking for other opportunities to do XP. But guess what? In 2001, there weren't any. So I started teaching people how to do it. Bam! I'm a consultant.
Several lean years later (I don't mean Lean, I mean ramen), I'm figuring out this consulting thing. I've got a network, I've got a business entity, people actually call me, and oh, oh, and I make a real damn difference.
Then Agile starts getting really popular. Certification starts picking up. Scrum's the new hotness, XP's too "unrealistic." I start noticing some of my friends in the biz are dropping out, going back to start companies or lead teams or something real. But I stick with it. I'm thinking, "Sure, there's some bottom feeders creeping in, but Agile's still based on a core of people who really care about doing good work. Besides, if we all leave, what will keep Agile on track?"
It gets worse. Now I'm noticing that there are certain clients that simply won't be successful. I can tell in a phone screen. And it's not Scrum's fault, or certification, or anything. It's the clients. They want easy. I start getting picky, turning them down, refusing to do lucrative but ineffective short-term training.
Beck writes XP Explained 2nd edition. People talk about Agile "crossing the chasm." I start working on the 2nd edition XP Pocket Guide with chromatic and it turns into The Art of Agile Development. We try to write it for the early majority—the pragmatics, not the innovators and early adopters that were originally attracted to Agile and are now moving on to other things. It's a big success, still is.
It gets worse. The slapdash implementations of Agile now outnumber the good ones by a huge margin. You can find two-day Scrum training everywhere. Everybody wants to get in on the certification money train. Why? Clients won't send people to anything else. The remaining idealists are either fleeing, founding new brands ("Software Craftsmanship"), or becoming Certified Scrum Trainers.
I write "The Decline and Fall of Agile" [1]. Martin Fowler writes "Flaccid Scrum" [2]. I write "Stumbling through Mediocrity" [3]. At conferences, we early adopters console each other by saying, "The name 'Agile' will go away, but that's just because practices like TDD will just be 'the way you do software.'" I start looking very seriously for other opportunities.
That was six years ago.
...
Believe it or not, things haven't really gotten worse since then. Actually, they've gotten a bit better. See, 2-5 years is about how long a not-really-Agile Agile team can survive before before things shudder to a complete halt. But not-quite-Agile was Actually. So. Much. Better. (I know! Who could believe it?) than what these terribly dysfunctional organizations were doing before that they're interested in making Agile work. So they're finally investing in learning how to do Agile well. Those shallow training sessions and certifications I decried? They opened the door.
And so here we are, 2014. I see these "Agile is dying" threads as a good thing. Because they mean that the word is getting out about Agile-in-name-only. Because every time this comes up, you have a horde of people saying "Yeah! Agile sucks!" But... BUT... there's also a few people who say, "No, you don't understand, I've seen Agile work, and it was glorious." That's amazing. Truly. I've come to believe that no movement survives contact with the masses. After 20 years, to still have people who get it? Who are benefiting? Whose lives are being changed?
That means we have a shot.
And as for me... I found that opportunity, so I get to be even more picky about where I consult. But I continue to fight the good fight. Diana Larsen and I have produced Agile Fluency [4], a way of understanding and talking about the investments needed. We've released it, permissive license, for everyone to use. Use it.
Because Agile has no definition, just a manifesto. It is what the community says it is. It always has been. Speak up.
[1] http://www.jamesshore.com/Blog/The-Decline-and-Fall-of-Agile...
[2] http://martinfowler.com/bliki/FlaccidScrum.html
[3] http://www.jamesshore.com/Blog/Stumbling-Through-Mediocrity....
This.
Speaking as a manager struggling to get agile implemented properly I think this is key. Pseudo agile isn't great but it is way better than the pseudo waterfall (or just hacking - in the pejorative sense) that was in place before.
Organisational change is really hard. When the consequences of that change go against everything that's happened before and when they reach a long way from the place trying to make that change (including outside the organsiation to customers in many cases), it's even harder, but it is slowly happening.
And just out of curiosity, are there any popular Agile books that you would not recommend?
This touches on what I believe to be the real, underlying issue: you (as a company) can't manage yourself out of a hole you've managed yourself in to. If management is defective, it will taint any process no matter how well conceived, because the underlying problem is not that the process is flawed, but that the people enforcing it are.
Edited to add: your (fixed) link is right on the money
> [1] http://mikehadlow.blogspot.co.uk/2014/03/coconut-headphones-...
Did you mean to link something else?
First, if the software project is part of a larger integrated hardware/software project, people above the project manager may be making promises of deliverables without consulting the program manager at all, thus creating externally-imposed deadlines that cannot be changed without rippling through to other teams, who may or may not be in your own company. Of course, the same upper management that pulled deadlines out of their asses is reassuring the customer and other teams that they are Agile, so this won't be a problem.
Second, you have a stubborn customer who wants deliverable deadlines, holds you to them, and views your Agile-based explanations as "excuses". The US federal government is notorious for this.
That said: I think these are not failure modes that can be laid at Agile's feet. They represent a situation that Agile quite explicitly does not even attempt to fix. A key part of the (for lack of a better word) Zen of Agile is that on anything above the smallest of scales, it's impossible to promise both a feature set and a due date.
To an approximation, that's what the whole sprinting thing is all about. It's breaking things down into bite-size pieces that are small and simple enough that you can hit milestones on deadlines with something approaching regularity. But on top of that you've got the overall development arc, and on that scale there are (or should be) no promises made about what's going to be happening on any sprint past the current one. The point of this is to buy the product team flexibility: Either the flexibility to adjust the requirements in response to new information that's discovered during the product lifecycle, or the flexibility to adjust the number of sprints that will be needed to achieve a given feature set in response to new information that's discovered during the product lifecycle.
In short, this is a feature of Agile not a bug. It's nothing more than being realistic about an immutable law of the universe: The more rigid you need to be about deadlines the less rigid you can be about requirements, and vice versa. Product teams have a professional responsibility to be honest about this fact. Customers and managers who aren't comfortable with it are free to restore their sense of certainty by building ample buffer space into the schedule.
That's what everyone has said about every dogma they've ever believed in, ever.
So do you disagree that there are a lot of ways to implement "agile", some of which will be more successful than others? Or do you think that there are some organizations for which agile (even well-implemented) will never be better than methodology X? (where X = ?)
Yes, it's easy to run into No True Scotsman and "ur doin it rong", but we're not going to advance the state by pointing out that it's hard to draw correct meaningful generalizations (we know that already).
Relative estimates are much more reliable than absolute estimates. That's what fibonnaci sizing, etc. is about -- you can gauge effort relative to other effort.
Once you have an actual consensus of the complexity of tasks broken down, you can then apply that to the team velocity and make estimates.
Better is Arlo Belshee's system of getting everything to roughly the same size block. http://arlobelshee.com/planning-with-any-hope-of-accuracy/
This sounds like bullshit. More often than not, bad estimates are a result of built-in bias in the overall estimation system that gets blamed on the people.
For example, I routinely see people underestimating larger tasks when the cost of splitting them is prohibitively high, while the cost of being late with a task is largely imaginary.
It works fine for us, but we have the ideal conditions for it to - business product owners who understand that their role is to provide a prioritised backlog and clarify stories in a timely manner - and nothing else. We have an organisation that accepts that the developers will drop features from a sprint to safeguard reliability and quality. We have teams that have stable core domains so that our estimations are, for the great majority, sufficiently accurate within that core domain.
Every time I see a "Agile is a scam / it's dead Jim / it's a myth" post, it usually involves into someone doing something dysfunctional and then trying to justify it with "But Agile!" in a game of Buzzword Bingo.
I went to help a local government IT department that was trying to implement Scrum to get some clarity on what the hell was going on in their dev teams, and I sat in on sprint retrospectives where all the talking was done by the 'product owner' - who was actually a rebranded business analyst who had no authority to prioritise backlogs. He wouldn't stop talking either, even when I explained to him that the retro was for the team, not him.
To that team, Scrum was a bunch of bullshit. To me, how they were doing it was obviously flawed. That said though, ultimately, that organisation has derived value from their half-assed implementation of it. It's shown them precisely who contributes in their IT team and who is cruising. They've got a few people who claim a monopoly on certain areas and jealously defend them because it makes them feel necessary and as such, safe from being laid off. Hence I had a GIS guy telling me that "You can't expect .NET developers to learn GIS!!" and an ABAP developer saying the exact same thing.
Now that organisation faces the challenge of managing the coasters out - unlike the US of A we don't have at-will employment.
Support is still very costly for us, because the distraction of working on old features slows down new development. To combat that we're putting a lot of effort into fixing the root causes of our support issues and training our support team to handle more and more technical problems.
* 'support queries' are actually bug reports: you're getting something wrong with your automated testing that's letting bugs get out into released code. Ideally you should have acceptance tests that make sure the code is doing what it should before you release it. Check what your team's criteria for 'Done' is - are they cutting corners to get things out the door?
* 'support queries' are actually missing functionality in new features: you're not getting all the requirements for the feature in the sprint where you implement it. You need to make sure the conversation with the product owner covers everything that the feature (user story) should do.
* 'support queries' are actually new feature requests: they should go in to the backlog and be prioritised along with everything else. You shouldn't let people push work into a sprint that hasn't been prioritised by the product owner.
The advantage is that disruption to a sprint is amortised to a constant instead of being a variable. The weekly rotations ensure that people who are writing bugs aren't isolated from the fixing of them - it also maintains morale, being stuck on a bug-fixing team is a bit of a gulag for devs IMO.
From the point of view of a well-run Agile team, there's just work to do. It all gets managed through the same process. One queue, one team.
In my last team, we created a rule that when you finished something, you'd just pull the next card at the top of the stack. You could ask for help as needed, but it was your job to see that card through to completion. That forced us to cross-train. Which was certainly fun. But it also really helped us to make better software. It was very consistent, both on the surface and under the hood.
I think the application of the OP here is "fire the bad/mediocre managers, put that 1 good tech person in charge". Couldn't they do that?
Here there's dev meetups, Second Wednesday, Notts Tuesday and probably more regular events I don't even know about.
I've talked and worked with a lot of developers round here now as well as interviewed with a fair few companies and done their technical tests and talked to their lead developers.
A lot of them write better code than I've seen in the github repos of Californian 'rockstars'. And yes, there's plenty of bad code too, out-dated practices, people asking me if I know VB.Net. And yes one of the companies I worked at (oh so briefly) had terrible code.
No different from the Californian mecca I imagine. Do you think that only good developers go to SV. There's no magic forcefield keeping bad developers out of California. In fact because SV is so desperate you might even argue they probably have a lower average standard.
There's plenty of shipping, successful products coming out of this city and successful IT Teams working for other industries. And bad ones.
"And yes there were teams that did this before Agile became popular, but it certainly wasn't the norm."
And there are now lots of teams not doing it but calling what they do "agile" because they're engaging in cargo cult practices. A whole bunch of stupidity is simply painted with agile terminology (e.g. unfocused daily meetings that are referred to as "standups").
I'm sure "healthcare.gov" was "agile".
I hope to address this next time. Or maybe I should just point it out before then.
For a team-oriented approach to work, you really need a team. A team is a group of people that has different skills but the same goal. They're a group of people that win or lose together. They have to have the same purpose, or it won't hold together. If every team member had different priorities, then something was badly screwed up about how the team was managed.
In the case of the smart but lazy guy, that's where external-to-the-team management structures come into play. E.g., if you're using a management structure like spotify, the lazy guy's manager should have noticed issues during their weekly one-on-ones. If not, other team members would be talking to managers about the deadweight.
I at least try and make my code maintainable.
[1]Or she may not be. Often times when you develop a system or even write a piece of code, you operate under certain constraints that might make it impossible to make it fool-proof. You may also not have the opportunity to come back and refactor it later. Maybe that was the case when she wrote her code. You may have had the time to debug what she wrote and even re-write it from scratch. She may not have had that luxury.
Heck, maybe she wasn't even a professional programmer. Maybe she had just taught herself some basic programming and wanted to apply it to the job to add value to the company and succeeded in doing so, and she had no idea that some douchebag would come in at a later date, break her program and then blame her for being a shitty programmer. The point is: her code worked at the time she wrote it. It was you who broke it. End of story.
What happens when the software could only ever meet the old needs, but is impossible to update to meet the new needs? Was it ever good?
Needs change. The ability for software to be altered to meet evolving needs is not simply a "technical quality of the implementation." It doesn't just have a faster algorithm. It's better software.
See also the odd thing that most people consider themselves better than average, which by definition cannot be true.
I've always wondered with that whether the problem is with the question. What it's really checking is whether someone will admit to being worse than average, which is a very different thing to thinking you're worse than average.
I'm talking about something more binary. Good or bad. Progress or stagnation. The right or wrong side of history.
I'm also talking about power. Who determines engineering practice. The engineers or the marketers?
Evidence of power is in the corporate hierarchy. Are there n tiers of middle management? Can an engineer truly make a difference at a company? Will the engineer be able to have a voice in what the priority is?
As an engineer who aspires to be on the leading edge of the practice, I've become more picky with who I work with. Luckily, "those few" tend to also be picky and seek each other out.
When I first started out, I sought people who were entrepreneurial and innovative.
The relationship most certainly does not need a mediator. Sticking a mediator in there would probably end up destroying the arrangement. If the two camps can't work together without one, a better solution is to figure out who it is that doesn't know how to play well with others and get them replaced.
I know a lot of people would say "Well just hire better people," but that's often not realistic. It takes time to hire people, corporate HR processes are terrible at differentiating good developers from mediocre ones. In the time it takes you to hire people, you still need to ship product.
That actually edges back toward my one serious complaint about Agile development: On the teams I've worked with, there's a bit too much value placed on generalism over specialism in team members' skill sets. Anything that's everybody's responsibility is effectively nobody's responsibility.
I'll add that Agile teams should also have something releasable every iteration (which ideally is every week). When that's true, dates are less of a problem. Instead of managers sweating engineers over when it will be done, managers in an Agile context spend their time arguing with other managers about the business question of whether to release something small and soon or something bigger and later.
In a well-run Agile context, anyhow. If you're not seeing those behaviors, but instead see the traditional drama, a high-pressure single convergence on a fixed date with fixed feature goals, then it's the sort of faux Agile this article is talking about.
This is not to say these structures should be inflexible. There needs to be some flexibility in requirements and dates, and it is the responsibility of the program manager and systems engineer to make sure the project can bend without breaking. There must be limits on it, though, or the project will tear itself apart.
I'd point out that the same problem exists for every other software development methodology I've ever tried. The difference is that when they do slip deadlines, it tends to be a whole lot more surprising (and therefore damaging to the schedules of other teams) because their feedback mechanisms tend to result in poorer-quality progress tracking.
A schedule can only have so much pad. My assertion is that Agile is useless to these projects because it can't handle these things.
> I'd point out that the same problem exists for every other software development methodology I've ever tried. The difference is that when they do slip deadlines, it tends to be a whole lot more surprising (and therefore damaging to the schedules of other teams) because their feedback mechanisms tend to result in poorer-quality progress tracking.
Every project management methodology has these problems, because at some level these problems are political. No methodology is going to overcome that, though some are better than others at dealing with it. The poster I first replied to said, Agile works fine for teams that embrace it. I'm saying such is not the case if you are working in an integrated or "team of teams" environment or if you have an unreasonable customer. Those may not be very common in the pure commercial software world, but they are in the rest of the world that software is trying to eat.
That an emphasis on delivering software, on interactions instead of process, etc. seems normal and no different permeates the industry such that people see it as the status quo. I assure you, that is not how it used to be, and there is a huge difference other than less emphasis on documents.
The major innovation of Agile was that you acknowledged that the various steps of the Waterfall/Spiral happen at the same time - in the Spiral days, that was considered something to be embarrassed about.
I don't doubt, by the way, that you were taught that the Spiral Model was called Waterfall. But you should be aware that this was a case of historical revisionism on the part of whomever taught you.
I also remember being taught that errors caught after the software had been implemented was 100% more costly to fix than errors at the design stage - I think that was the idea behind big(ger) design up front.
It is basically true for waterfall because the feedback loops are broken. Think cooking, for example: if I cook all my meals for a year at once and put them in the freezer, I'll have to do a lot of research and planning. Otherwise, a single mistake could lead to me throwing out up to 1000 meals.
But if I just cook every meal as I go, I can tinker quite a bit, because the cost of failure is limited to one meal. Which is no problem; if I really screw up, I just pull the frozen pizza out of the fridge.
Extreme Programming in particular can be looked at as a set of methods to flatten the cost of change curve. Then if you add the Lean Startup approach on top of that, you end up testing your core hypotheses early on, so even major shifts in business direction end up being pretty inexpensive.
It has a section in the front that describes which pieces to read depending on which role you're in.
The one gap is that it mostly focuses on greenfield projects. It does a good job of highlighting the parts where legacy systems will make agile more difficult, and I understand that spending more time on that would have made the book harder to understand. But we've had to do a lot of learning on our own to try to fill in those gaps for ourselves. With that said, this book does the best job of addressing legacy projects of any that I've read.
If anyone has any recommendations for resources on using agile in legacy systems, I'd love to hear them.
The big piece that we're missing is how to apply agile practices consistently when our velocity varies so significantly because of legacy code. Feathers' book gave us a long-term path to getting out of that issue, but we're struggling with the planning and communication side of things in the meantime.
In my Let's Code JavaScript screencast, I've recently started a three-part special on legacy JavaScript code. It follows my efforts to "do it live" with actual legacy code someone else wrote. You might find it useful. (Full disclosure: subscription required, and parts II and III don't come out until April 4th and May 2nd.) http://www.letscodejavascript.com/v3/episodes/lab/6
Edit: I just realized that you already said you're using Feathers' book, and that the challenge isn't technical, it's planning and communication. Let me try again:
Part of that is just "the way it is." At the risk of sounding snarky, I imagine you could plan fairly effectively ("we have huge error bars because our velocity is unpredictable, so given backlog X, we're confident we'll be done sometime within 3-12 months"). See the "Risk Management" and "Slack" sections of AoAD for that.
I also imagine you could communicate that fairly easily ("given our three month estimate, we think we have a 50/50 shot of being done in six months and we're almost certain of being done in less than 12").
For many teams I work with, the problem isn't planning or communication, the problem is that reality isn't conforming to stakeholders' wishes. And there's no way to make it conform. (Short of kicking the problem down the road by accumulating more debt while you search for a new job.)
So how do you break the news?
One way is to, um, not. At this point, stakeholders are used to delays and bugs. The status quo ("we'll pretend to follow your schedule, and you'll pretend to believe us") can be comforting. It might be useful to stick with that.
Another way is to ramp up the frequency of delivery so much that the error bars are less noticeable. An estimate to deliver in 3-12 business days is less painful than an estimate of 3-12 months.
A third option is to negotiate for scope rather than schedule. "We'll absolutely deliver in three months. It might be tiny. That's because of technical debt." (I'm less fond of this approach than I used to be. Negotiating scope makes stakeholders profoundly uncomfortable in a way that slipping dates doesn't, for some reason. Perhaps because they see it as an attempt to weasel out of producing anything.)
Another approach is to stop estimating entirely. "We're working on Feature X and we'll tell you when it's done."
There's also simply explaining the situation: the cynical approach ("we have a ton of technical debt because of the decisions your predecessor made, isn't it great you're here now") or the enlightened approach ("we've got a technical debt problem, it has effects W, T, and F; we're addressing it with P, D, and Q; you can expect consequences O, M, G; we're preventing it from happening again with techniques S, L, A, C, and K.").
I'll typically use some combination of the above depending on the audience.
(I'm in a weird mood. Sorry if this came across as uncaring. The ideas are serious even if the tone is not.)
We're a small enough team that we all understand the business drivers. The main goal of our agile transition is to improve predictability and clarity. Month-wide error bars on a three-month release don't help those goals much.
Right now we're working to shorten our release cycle and learn how to communicate clearly even when it hurts, on the assumption that shining light on the difficulties will help us find and solve the problems as quickly as possible. But it's definitely a painful transition.
One more point: I agree that negotiating scope with a fixed schedule is difficult. We have done that occasionally. But sales and business people are definitely more willing to accept schedule slippage than scope reduction.
It saves me from playing games that are not in my favor. It also helps me avoid & call out people's bullshit. Life is too short to play someone else's bullshit game.
Good luck being exploited or exploiting other people ;-)
Googling it, the example given is "starting conversations with strangers" and his theory is that people ignore the question they are asked ("how they compare with average") because it's hard/impossible to actually answer, and instead, without realising it, substitute an easy question ("are you good at x") and answer that instead.
Does he talk about other theories? I wonder if there's also an element of what the person has tied up in that skill and what the consequences of being bad at it are?
Certainly it seems unlikely that anyone would want to admit to being bad at their job (kind of tantamount to admitting to be a fraud), or at something which, as with driving, can be actively dangerous if you're bad at it.
It's a bit of a contrived example, but you get the idea. Unless you have a centralized management function, all of this doesn't happen and your glorious pizza button either doesn't work or fails deployment. Usually the onus falls on the product manager to do this, but with so many teams involved it can be a nightmare.
In the kind of organization I describe, teams have business purposes and are cross functional. So in your example, there'd be a pizza-direct-to-desk team. In that case, you don't need centralized management of everything.
Plenty of organizations work this way. Spotify, which has done a number of talks on their organization, works just fine with cross-functional teams focused on individual purposes. I interviewed some team members at YouTube, and they had a similar structure.