That's ironic since the Agile Manifesto was originally written by software developers, in 2001. The problem is that since then - as Dave Thomas put it in his "Agile is Dead (Long Live Agility)" post
"The word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products."
>what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products.
Worse. I once attended an agile community meetup, and felt like I had just stumbled upon a weird cult. It's scary to think that there are so many software developers out there who have to put up with these authoritarian doctrinaires.
As a developer who has been quite unhappy in Agile environments, I don't actually think Agile is to blame, which the article alludes to.
I think the issue with Agile is that software is a field in which is difficult to measure productivity easily. Then Agile comes along and offers the appearance of making it easy to score and and measure team and individual effectiveness. Sure it's "supposed" to be for the team to measure their own effectiveness and empower people to work more productively, but for folks in management roles, the opportunity to 'score' engineers based on Agile metrics is too attractive.
Once it becomes just a way to figure out who's not moving fast enough, or "failing to meet their commitments", of course people will hate it. It gets changed from a tool into a stick.
-- Value individuals and interactions over processes and tools
-- Value working software over comprehensive documentation
-- Value customer collaboration over contract negotiation
-- Value responding to change over following a plan
That doesn't sound like "agile" anywhere I've seen it implemented. The issue is that you need real buy-in from all levels of the company for any process. I've always said that a "bad" process executed consistency and with real buy-in will outperform any "good" process with only lip service commitment.
You know the problem with those principles? They are not a methodology.
Most of the problem described in the article -and everywhere- is precisely that "the theory is good but everybody is doing it wrong".
But that doesn't really help, because it overlooks the fact that the origin of all this is just a pile of generic, hard-to-argue small-time guru ruminations. I mean, it is not necessarily bad advice: "These are valuable things and we think that in particular these ones are more valuable than those other ones". Okay. Probably. They do look good. Let's say they are. Now what? Oh, that's left as an exercise to the reader.
It's funny. Just yesterday I almost had an argument over one of these things. I heard Karen say to the guy on my left that "Agile means less documentation", so I raised my eyebrow. "Agile means less defined documentation, more open, because documentation has no value at all" - she added. I was about to point out that that is not at all what it means, but then she went further and told him that what he needed to do was to "work as a team and become more involved in collaboration". That, right there, is the problem: pretending a bunch of cool sounding expressions can be a substitute for actual things like writing down what they wanted the guy to do.
It seems like the problem might be that people assume agile software development is, in and of itself, supposed to be a methodology. As I understand it, agile software development is meant to be a set of principles describing any methodology that serves those principles. As such, Extreme Programming is an agile methodology, intended to serve those principles when employed well, and Scrum is a pseudo-agile methodology that serves as the halfway point to which one might be able to drag management, but more often ends up being an excuse to measure developer productivity in story points.
There's value in enumerating important principles for consistently pursuing quality work product. That's why the Agile Manifesto has value. The manifesto is just a manifesto, a statement of intent and principle, and not a methodology in and of itself. One doesn't need a published, monetized methodology to engage in agile software development. One simply needs a methodology -- whether unique to one's project, or widely known -- that reinforces those principles to engage in agile software development.
The biggest problem with agile software development as a concept is that it, like any other elaboration of any principles of quality work, is that most people will fail to see the forest for the trees, will marry themselves to some well-marketed thing that pretends to be about principles, and will end up largely doing the same dumb crap they've always done.
The documentation one is the one that completely throws me for a loop, and why I will never be on-board with anything claiming to be Agile. You know what? sometimes the documentation is more important than the working code. We're not all building crappy web properties as an agency, sometimes we are building stuff that is supposed to last longer than any one developer's career.
Sometimes just documenting what the thing is supposed to do and why would solve so many issues, but since people have gotten hooked on capital-A Agile so much has gone by the wayside, or been delegated to non-technical roles.
> You know the problem with those principles? They are not a methodology.
No, that's not the problem with the five values and twelve principles.
In fact, the problem is the desire to have a single, divinely ordained methodology (which both the first value and twelfth principle reject, and that rejection is arguably the central basis of Agile.)
OTOH, while it is good that the Manifesto and Principles don't ordain a methodology a it's bad that they are written in a way which makes it harder to extract actionable direction than is necessary.
The only place I've worked where Agile was like this was, surprisingly, one of New Zealand's major banks. The buy-in for Agile started at the executive level, and they made a call that everything was Agile. Obviously, this would've been an absolute mess for most teams. But my team functioned perfectly. The woman who was our scrum master was originally a project manager in the bank. But with the Agile reshuffle she went over-the-top with her retraining. This was horrible in some areas, where she would follow processes to a T, but she would catch herself, and recognise the importance of those principles. She grew into the role well, and the team was eventually working great!
As a developer become engineering manager become director of product, I agree with you wholeheartedly.
But I think the bigger problem is that often developers and management (at all levels) give very little thought into "delivering value" and a lot of thought into "doing whatever they want".
I can't fault individuals (workers, management, or executives) for trying to do what's best for their careers rather than what's best for the company. As a manager, I think management's primary objectives are to set up their reports for success and to figure out how to align their reports' incentives with the company's incentives in the best possible way.
Management, I have seen, often does a terrible job 1) actually knowing what "value" is, 2) figuring out how to measure that they're actually delivering value, and 3) figuring out how to align delivering value with the worker's own incentives.
If you can't do these things, you have NO chance to do agile effectively. It's easy to go through the motions of Agile, but you will gain nothing if you can't define and measure value (and hardly any companies can -- it's not easy!)
But as you pointed out, even if you can accomplish the prerequisites, it's still INCREDIBLY difficult to attribute an engineer to value. You can execute effectively, but be on a project that turned out to be useless. Making the button green might be easy, but because of previous technical debt, making it blue might be really hard, etc...
Here some reasons I have seen why nobody has a clue about value...
* Often the financials of a company are "need to know" or obfuscated (Y-Axis unannotated)...
* The financials are regarded as irrelevant to engineering (by management, finance AND engineering)
* The estimates of where the value lies is often just a spreadsheet sucked out of the thumb of somebody in marketing who already has a eye on his next job.
* Nobody _ever_ goes back and closes the loop and evaluate the value estimations and the value estimation process, so the company never learns to do better.
* Let a drooling slob of an engineer talk to a customer? You're joking Right?
And yet engineer is all about tradeoffs, we make several per day under the illusion we know which are wise....
TBH I think it's normal for any person to put their own interest on first priority. The key is to hire people who actually share the same interests. I fully agree with your point actually.
I worked in fake agile for a few years. It took me a while to place my finger on why I never really loved everything about it, but I eventually decided that it felt like it was a façade to a management power grab. It gave the ability to micro manage workers to those who don’t understand the problems or even the product. If agile is necessary to be productive then you already have too many people involved. If it isn’t necessary then it introduces a lot of overhead for a little bit of book keeping (which isn’t inherently bad).
I remember a time I worked with a team not using the agile process.
We were agile.
When I worked in teams following the agile process, we were rigid.
Agile shouldn't be a process, mgt see it as such. It becomes a process. Clever engineers hate poorly thought out processes. Agile as a process is a contradiction in itself.
Agile is no process. Let the team find their ways and iterate as often as possible, along with a few other proven principles.
Points becomes estimates for managers, they will see it as such and build huge waterfall projects documents to track everthing, and they will pressure teams to deliver shitty software when they feel the stories are sand bagged.
I think that's pretty close to the truth. Like the Tao, you can point at instances of it, i.e., you can point to a particular team and the processes it is today using, and that is agile. But it is not Agile. On this same team I'm on, I've used different process a year ago, and two years ago, and now, and almost certainly in a year and two years. Those specific processes aren't Agile... but they were each agile, because we're constantly re-evaluating how we're working and what we're doing, and tuning our processes to match the environment we're in. If you take my agile process and just blindly apply it to your team, regardless of what that process is, that's not Agile. The grass of the field does not become a bird by shaping itself into wings.
It's the burgerization of software development, where every engineer can be turned into a generic fungible burger-flipping engineering unit, and the output measured in the number of burgers flipped.
The software architecture is falling apart, the tech debt overshadows the actual debt of a small country, our subject matter expert quit the company over being micromanaged, and the clients are abandoning our buggy software for our competitors? What went wrong, we closed all of those JIRA tickets, surely that must be good for something.
I’ll add this — To a large extent, many management systems are about reducing the failure modes — but I don’t see why a system must downplay creativity and deep thinking about the core business engine (often the software).
Ticket driven development isn't just an agile issue though. Project management has always loved granular tasks and the assumption that at some point they are all worth the same effort.
It took me a while to place my finger on why I never really loved everything about it, but I eventually decided that it felt like it was a façade to a management power grab.
Yup that's exactly it. Or to put it more delicately (than simply a "power grab"): a façade that servers various ulterior purposes -- from making their company seem hip and modern (to the uninformed) to simply being able to say "finally we have some structure!" even though it's very debatable whether new "structure" really brings the benefits that it claims to (and evidently brings many negatives on board with it).
I'll bite. Management always seem to get the crap. To summarize, if engineers knew how to manage things better than management, why don't they? From my experience, teams without management seem to complain about that and end up with bad results. There are more factors at play, and honestly it's very rare that the engineers sees that (or even acknowledge that they don't know all the relevant factors, with all the implications that gives).
Of course, just as with engineers, there are good and bad managers. But it seems pointless to always complain about management being the ones screwing it up all the time.
A space alien with a raygun and a watch knows just enough English to tell you to build a time machine "or else!"
Any attempt to persuade him otherwise just results in him pointing his raygun to his watch. Either he's ignorant, can't or won't understand, or simply doesn't care.
Software management in 2019 is still very much like this. Management has had 40 years to solve this problem, and it looks like it won't be solved in the next 40 either
Management gets the crap because they have the responsibility and push the processes.
It’s as if we were complaining that elected officials get the crap for mismanaging the country. Well, that’s the point of them doing their job. Our goal is not to all become managers (though some do, exactly for these reasons), it shouldn’t be a ”why don’t you do it ?” rhetoric.
Yeah I don't mean to imply that it's managers per se. Management seems suuuuper hard and I wouldn't want the task of assessing engineering productivity.
I just mean that the "metrics" put out by Agile give the illusion of something useful to managers. It's totally understandable that folks in positions of authority would attempt to make use of them. But I think the illusion of objectivity is more damaging to teams than the acceptance of uncertainty.
I'd say that attempting to use the same tool to help a team communicate with one another and to communicate with management or stakeholders who aren't very close to the project (involved daily near the ground level) is essentially doomed. Management will either hate the answers they get or they'll ruin the tool's utility to the team, resulting in worse communication at every level and slower work. Or both.
It's where I think Jira and other do-everything tools that try to be useful to both ICs and management fall down hard, every time I've seen them used. Management wants to see their burndown charts and such. They want projections out months on minimal data. They want to see the performance of people, not teams. Points become BS because everyone knows they're being watched. Totals are missed the first few sprints (of course they are, you're still calibrating WTF a point means for this team and this project!) and management gets pissy, exerts more control, implies ICs are failing at their job, project goes downhill from there. But look, we're Doing Agile! Such professionals we are.
As a project manager, I hate taking a bug tracker and trying to beat it into being a project management/communication/measurement tool. The only reason I always end up doing so is that bug trackers are already in the engineers’ daily workflow so they are usually the source of truth about the project.
I’d love it if developers would be willing to collaborate using a dedicated project management tool and leave the bug tracker for bugs, but the chances of that happening voluntarily are nil. You gotta go where everyone else already is.
> I’d love it if developers would be willing to collaborate using a dedicated project management tool and leave the bug tracker for bugs, but the chances of that happening voluntarily are nil. You gotta go where everyone else already is.
What'd that look like, that's different from what they do now? Trying to imagine what developer input to that would look like. I think a lot of devs already feel like existing PM tools they use for sprint tracking and such are worse than if they pushed more planning into git or somewhere otherwise closer to the code, so getting buy-in might not be so hard if devs got to be a little less heavy in a bunch of not-actually-that-useful-to-them tools on a day to day basis.
Could be something as complex as MS Project or simple as a shared spreadsheet with rows for each project and columns for each milestone or for things like ETA, dependencies we’re blocked on, escalations needed, scope changes, etc. Not sure how that could gel well with a source control tool but I’m all ears!
I think 1/4 of the challenge is what tool to use and 3/4 is convincing and motivating people to be reliable and forthcoming with accurate information. A good project manager demonstrates he or she can add value, unblock things, escalate and communicate appropriately, and not just be the Project Police.
By using things like story points and other metrics intended to be internal to the team, developers have a lot of incentive to not lie on their estimates. On the other hand, management has incentive to push things to be finished quicker.
By letting management having access to those metrics, they're automatically at a disadvantage when negotiating timelines and priorities.
Management has, culturally, become very metrics-oriented. This is the root of the problem under discussion here. See Goodhart's law for a brief statement of why this can't work.
Almost every time I bring this up somewhere, someone pops up to say that metrics are important, key to fair and impersonal management. Without it, everything would suck, and managers couldn't do anything. The problem with that thinking is two-fold:
1. It ignores the fact that, as soon as there's a metric that determines someone's rewards and punishments in a management system (whether it's governmental regulation, corporate middle-management, anarcho-communistic self-management, or anything else), the system of measurement will get gamed, and the most overtly "successful" will end up being the most cleverly sociopathic about it.
2. There is an alternative to strictly quantitative evaluation. The alternative is qualitative evaluation. This requires good judgement, intimate knowledge of the problem domain, good intentions, and the ability to consider context, among other things. Sadly, this is incompatible with bureaucracy, which means you need to either carve a bureaucracy-free zone out of the larger organization to manage a team well or stay out of medium-to-large corporations (and out of small businesses that ape the behaviors of large bureaucracies).
Managers who insist on doing everything quantitatively (and sometimes they're forced to do so by their managers) will not get good results. "Good" is qualitative, not quantitative. Equating "good" with metrics just gives you results that look "good" on paper, but really aren't good.
Probably the biggest benefit of a small organization is the ability to eschew managerial bureaucracy in practice.
> if engineers knew how to manage things better than management, why don't they?
They do: engineering-led companies are often dynamic success stories, then when they get big enough traditional management works down from the top to lower levels and the firm stagnates.
Agile was proposed as a way for engineers (or "self-organizing teams") to manage things better. The complaint is that the engineers' better management process has now been hijacked by management and turned into something else.
Somewhere, there's always an impedance mismatch. Agile can be managed, and managed well - I've seen it done - but above that manager, there's another manager. Somewhere up the chain, someone wants to see progress on a PERT chart or similar, and the person who is managing an agile project doesn't have a good way to answer.
It is vital to have someone close enough to the project that the team trusts them to see their honest opinions about progress and blockers, including non-bullshit pointing of tasks, who is the point person for communicating progress to management, and who is capable of producing reports that are technically bullshit in an is-this-quantifiable-this-way sense but actually do reflect reality usefully to managers. Someone who does that well is worth All The Money.
They're useless if you're giving management direct access to team planning & communication tools, of course.
In some places these are called Engineering Project Managers, in others they are Technical Program Managers. You may think they are worth All The Money but let me assure you they usually make far, far less than the developers :-)
The failing after every step has been when they close the loop. That is, agile is used to score and measure velocity. This velocity is then used to impact future scoring. This nullifies Agile's use for measurement since management is now insisting that x number of points be completed per developer per sprint.
The places I've had it succeed greatly have all been new development. We consistently scored stories. Product owners could then had a budget equal to the team's velocity to purchase from the backlog into the next sprint. What it really looked like was they got to arrange the order that the backlog would be picked up and how much would get done in a sprint was an estimate, but with consistent implementation it became eerily accurate over time. It has been my observation that it takes about 6 sprints to calibrate -- where the team scores consistently and the variances from sprints cancel out.
Another point of emphasis is that velocity is a team score. Every failing effort I've been involved with used it to score individuals which inevitably ends up in scoring the stories higher so that an individual gets more points. It also results in cherry-picking easy points and ignoring tasks and bug fixes which may result in zero. It also (like bug fixes) results in everything being assigned points such that points become a proxy for time. If you're developing a new system and you have a bug fix, the team has already claimed the points for that bug fix. You should not make more points for the bug fix, your velocity should actually go down, giving a more accurate estimate in the future of how much new system is actually produced and not a measure of how much a programmer has worked.
So, measurements should absolutely be available to management. But they shouldn't be used to impact future scoring.
That assumes really good (measured against the norm) management. If your management is "normal" (or worse), giving management access to that just destroys the whole value of the scoring system. (Always remember that fifty percent is worse than the centerline, to say nothing of the still-within-a-standard-deviation "better" that isn't enough better to matter.)
It really sounds like a "not true Scotsman" argument, but I've had Agile fail when the team failed to use Agile. As I wrote above, points inevitably became a proxy for time in one manner or another. Production was pinned to a certain number of points (say 15 per developer per sprint).
I can't throw it all on management either. People who should know better will accept 2 days worth of points for conducting training or attending a workshop.
Largely, though, I would say it's a simple matter of not understanding how Agile works at all. It has just become, "you score points for doing stuff" (in proportion to time, but it's not time, but it is).
Your observation is right on target, and it has been predicted and validated [0]. The people who originally developed agile understood this, but it's one of the kinds of things that doesn't really sink in in many cases.
[0] 'Measuring and Managing Performance in Organizations', Robert Austin, Dorset House, 1996.
But who ensures it isn't? Management who are the ones inherently attracted to having the metric to use? The ones who are able to withstand that pressure and not touch the information are likely the ones would could see the information without using it as a stick.
I guess upper management could enforce the separation on a level lower management could not override, but I would assume such cases are rare.
The solution is to stop blaming the methodology for the failures of management, and just accept that bureaucracy breeds mediocrity. Corporations grow into bureaucracy, inevitably. Choose accordingly when evaluating potential employers. Keep this in mind when running a business.
I am not sure it would even save the situation. Most agile methodologies assume a lot of constants to allow measurements to have any sense at all (for instance scrum assumes teams members are stable, of that knowledge over their work perimeter is good, etc.)
We work in a field where inherently there are unknows, and measuring the unknown is not gonna be that useful, however we turn it.
I was in a company that did Agile well (my first time using it). Team velocity was never used to measure the output of the team, only to better predict how much work we could fit in a sprint. The other big think was that they saw Agile as a framework, each team was free to do a different "implementation" and use different tools. Team velociyy had different units in different team and couldn't be compared.
I worked for an employer that had a different PROCESS.md document in every project repository. At every retrospective, we'd evaluate the process for that project to determine whether it needed updating. It was awesome.
Eventually, things changed, but for a while there it was awesome. Things got done well, in reasonable amounts of time, with a lot of attention to quality work.
I think the beginning of the end was probably around the time when the company (a software development consultancy) started using fine-grained hourly measures of time pre-sold to clients. This started happening around the time investors in certain clients got more involved in trying to dictate company policy, and I'm pretty sure it was all part of an attempt by investors to ensure there were "better metrics" for evaluating their investments.
You can't measure something without it directly affecting the measured thing's state. In this case, it started having detrimental effects on almost everything. Previously happy and productive devs started burning out, mediocre devs became new management favorites, more bugs got into production, projects suffered under less and less reasonable goals (including at one point my task effectively being to solve one of the Hard Problems of software dev -- cache invalidation), and so on.
In the US, management culture seems to have a monomaniacal focus on metrics. (I'd say "simplistic metrics", but the way they use them guarantees they're always simplistic.) This is driven, in part, by bureaucratization, which is in turn driven by a bunch of other things (mostly stemming from politics, directly or indirectly) in our business culture. Many managers would complain that without metrics they can't evaluate team productivity, individual value to the organization, and other important stuff, and they're right. The solution is not to double down on those metrics obsessions, though; it's to replace those managers with managers who are capable of effective, qualitative evaluation, instead of the almost purely quantitative approach that holds the high ground.
> for folks in management roles, the opportunity to 'score' engineers based on Agile metrics is too attractive.
One nice thing about this is that the metric is visible. Then you can optimise your performance against it. Is this necessarily productive, or in the real interests of the company? Maybe, maybe not. But I look at this as a kind of a favour: if your boss is complaining "velocity is too low" well, at least you know what you're being measured by. Put some extra points on those stories. Undercommit. Make your boss happy and get them off your back. You'll be doing the same work either way.
But IMO agile is not at all about measuring productivity, I'm not sure how you'd come to that conclusion?
There is, I admit, an element of it in the sprint planning, but that is only meant to be a tool to figure out how much work can be put into the next sprint. Agile does not care what your absolute value of velocity is.
If however, some misinformed PM is obsessing over velocity then I think that is a very different issues and not really related to the essence of agile.
But using agile does change how developers communicate progress to management. So agile processes can definitely indirectly impact how your productivity is perceived and measured.
> But IMO agile is not at all about measuring productivity, I'm not sure how you'd come to that conclusion?
It wasn't a conclusion it was a summation of what is happening when agile ideas are misused. I've seen it used that way too, it never ends way because nobody can measure a programmer's productivity/effectiveness.
This is absolutely correct. For example, what is the point of a sprint in Agile ? Do what one can finish in a sprint but try and finish "modules" while maintaining quality and carry over rest to next sprint. What does management read? Finish ... modules ... current sprint?
I do think that "Agile" is to blame. In my view the Agile software methodology is like Communism or libertarianism. Whenever you point out the myriad ways that it can fail in practice, someone always jumps out to say, "Well, what you're not describing is not true Agile. True Agile has never been tried!"
This is a great point. If there's a tool that no one can figure out how to use properly or effectively, it's hard to argue that the tool is well designed.
You could say the same about vim though...
Jokes aside, many resources about agile start with explaining how agile is “easy to understand, difficult to master”, and I think that’s true.
The difference is that vim is a specific tool, and if I see someone struggling repeatedly to be effective with vim, I can gently suggest that they try some other tool. But if I see a team repeatedly to implement something that looks like an Agile methodology, what do I tell them?
I suspect that many devotees of agile would speak at this point about getting management buy-in, or reforming the wider organization, but at this point "agile" just becomes shorthand for "reform your entire organization from top to bottom so that your management doesn't suck".
If the team struggles, the org does not need to be changed. However, from my point of view, companies should increase support for their teams when transitioning to agile. Training would be a good start. I have taken a few agile trainings, and they have been among the best I had in my career.
You've hit the nail squarely on the head. The problem with Agile isn't Agile itself, but the fact that humans can't keep themselves from using the data it produces to their own non-Agile ends.
But maybe Agile is the source of problem with Agile. Agile, to me, is like the nitroglycerine of development methodologies. The explosive instability of Agile isn't a problem from outside that corrupts Agile - it arises from the methodology itself, which inherently invites abuse.
We moved to Kanban a few years ago and I think it's been much nicer than the "sprints" we used to do.
There is a board. The board has a prioritized list of things to do. You work on your thing until it is done. Then you pick your next task based on where the priorities currently sit.
It's really not far off from the "sprint" idea. The main difference is it is a constant re-balancing rather than an every X weeks affair.
There is always a list of things to be done and it's usually prioritized and often the highest priority item is what you do next. This isn't unique to software. It isn't even unique to white-collar jobs. Why does that even need a name?
It came from Toyota manufacturing, so right off the bat software is a borrower of the idea.
I love Kanban, it's all about minimizing work in flight, to allow you to focus on the task at hand until it's Done instead of hurrying to meet the sprint deadline.
Kanban goes a little further in my own experience. Here are some things I’ve seen in the wild:
- make sure that developers have some slack time, so that they can cater to their tools and study new things
- never move back an item to a previous column. If
something more important came up, you “cancel” the current task and start the new one. This helps to identify bottlenecks
> never move back an item to a previous column. If something more important came up, you “cancel” the current task and start the new one. This helps to identify bottlenecks
Could you explain this a bit further? I'm imagining three columns, todo / doing / done, and I'm working on a widget when I'm asked to put out a fire. I'd imagine that the fire is moved from todo to doing and that the widget work is moved from doing to todo, but it sounds like you're advocating for something different.
Could you explain more about it or link to resources that might help me understand? Thanks!
You need a mechanism to keep track of your unpredicted interruptions, so that you can improve your process continuously (which is what Kanban is about).
Moving items back makes this harder (although possible). What I saw some teams doing was to mark such items as cancelled, explaining in the tickets why they got cancelled, so that a manager can then see which columns (i.e. steps of a process) are interrupted more often. This could point to a bottleneck in this process.
In the near future, the "rest" of the cancelled ticket would make it back into the backlog as a new ticket entirely. This has the added benefit of keeping your backlog cleaner.
I will try this today with my own personal kanban board. I do find myself with cards moving back and forth a lot. The way I've dealt with "Cancellations" is I have a list called "Waiting" where I stick tasks that are blocked. Sort of like pending review but it could also involve waiting for a resource, a response, etc.
I think if I implement cancellations with details it could help me figure out what processes are broken that caused fires(i.e. this report is suddenly due, overriding everything else. Did I miss a memo? Do we have a solid process for anticipating these sorts of reports in the future? etc.) I could then go back during some planning time to view what needs some brain time.
IMO the two main points of Kanban (and Agile in general) is limit the size of activities and make the backlog visible. The goal is to empower the team to quickly change priorities.
Of course, It's possible to do that without Kanban, but it requires a lot of management skill, and workers hate changing priorities mid-work. And Kanban/Agile also doesn't assure that it will happen.
>This isn't unique to software. It isn't even unique to white-collar jobs. Why does that even need a name?
Kanban originates in Toyota's factory line. The key thing isn't the existence of the list, it's in making people responsible for maintaining and tracking the list rather than consolidating the authority in foremen who direct minions on what to do next. This makes it possible for teams to set priorities collaboratively and move between roles fluidly.
It also establishes a clear order in which things need to be done and an agreed upon subdivision of tasks that make it possible to track the status of work.
the main innovation of Kanban is to limit the number of items being worked on (in a specific category) at one time. For example - You can't pick up new work if testing is behind - go help the testers (somehow) instead.
You think that it is how it's supposed to work, but that's not really how it works. When a bottleneck presents itself everyone gets to work around it as it is not fixable by them! (Or the fixes are deemed either or of scope out to time consuming, so you're directly even prevented trying at it.)
Generally you cannot fix things you do not have control over. Kanban is meant to have a process person fix the bad bottlenecks - not the developers, not the managers - and said person is supposed to have the clout to pull it off. I've never seen Kanban implemented properly.
As for the example - if testers are not keeping up, either you hire more or enhance testing methodology. Throwing developers at it slows down development moving the bottleneck.
(Of course, throwing some developers to provide better testing methodology, testable components or automation would be a proper fix.)
well, any type of process will fail in a dysfunctional organization :). And you can't (well, shouldn't) arbitrarily implement a specific process without thinking through the fit for your organization. Just buying a license to a Kanban tool isn't enough :).
The example you gave is a good one - devs spend their time making improvements to to code or the test automation to make the testing more effective, if testing is the bottleneck to delivering finished features. Throwing more code/features over the wall when that code isn't actually being tested isn't real progress.
In principle, the manager/lead would tweak guidance until the most expensive resource (dev engineers) are the actual bottleneck. I guess of course teams/companies that care less about quality might just tell the testers to test less thoroughly in order to get their work items through the system faster, but at least it's a conscious decision.
I've worked for teams where they literally close every testing ticket once the code is pushed to production regardless whether the testing has been done. 'tested in production' - that's one way to keep up I guess...
It's a great way to have an unfixable mountain of tech debt.
See, Kanban only works when the focus is on quality. It's the main secondary premise of the original Toyota approach of Lean which spawned Kanban. Note that it starts with minimizing waste (in programming, tree breakage or waiting on tests) and secondly minimizing overloads. The philosophy is kaizen, continuous improvement. Sacrificing quality is not an improvement, Toyota itself started by breaking process into quality assured steps to fight waste due to rework.
Otherwise anyone can push undone paper tickets over the wall and that's not the point.
So here's the thing... "Sprints" are not specifically part of "Agile" at all! Look at the Agile Manifesto and see if the word "sprint" appears at all. (Hint: it does not).
So anybody complaining about "sprints" isn't complaining about "Agile", they're complaining about one specific attempt to implement the principles of Agile. Likely that's Scrum, since "sprints" are mainly a Scrum notion. But Scrum is not the only way to do Agile, and it certainly may not be the best. And even within the world of Scrum, there's a lot of variance in terms of exactly how it's implemented.
One of the advantages of moving to Kanban in my previous job was that people was that the code quality improved a lot because they were rushing at the end of sprints.
In reality, there are only things that people are working on, and things that no one is working on. There's no need for a third category of "things that are in the sprint" as long as someone (anyone!) keeps the task queue sorted in priority order.
Sprints are meant to be a tool to measure a team's velocity, defined as average amount of work throughput in X weeks. You don't know the average until a sprint team has worked together for several sprints on similar work.
Sprints can work very well if you need to:
A) Estimate the time it will take to complete a large project
B) Compare productivity of different teams so you can shift or add people
C) Get continual feedback on the work quality and the workflow process
Over time, I've seen sprint teams become very accurate at estimating their work throughput ahead of time. This can be extremely valuable to the business delivering on its goals on time and committing to the right amount of deliverables.
Sprints are less about categorizing the incoming work, but partitioning the outgoing work, to establish a quick feedback loop. This should in theory ensure that risks are tackled early.
The outgoing product is nevertheless fully determined by the incoming work being done. It would make more sense to group a set of tasks into a deliverable than a time period. You have to do that anyway to deliver anything coherent.
There are two arguments against grouping as a deliverable:
1) sprints are (supposed to be) predictable. So you know when to expect something and can plan other activities around that date.
2) (the more important reason IMHO) a sprint enforces early delivery, where features are rough and not very comfortable to use, but it’s already visible how it will look like. That means feedback can be collected and fueled into the next planning. The assumption is that it’s easier for people to talk about hidden assumptions when they interact with the software, than when they write specs
Kanban seems pretty similar to a method I used in college to track tasks, not even software tasks. This leads me to believe that it is intuitive since it more or less naturally made sense to me before ever knowing Kanban was a thing.
Going to look for this or something like it when job searching.
Who prioritizes? On a large team, most tasks require at least some collaboration and discussion. Kanban seems very chaotic for a large team, with priorities constantly shifting. With sprints, everyone on the team is in agreement about the priorities for the next N weeks, and you regroup at the end to iterate.
As a manager, how do I know how long a Kanban task should take? How do I know how long it's been in-progress, or how much is left? (With sprints, the idea is that every task should be defined as completable within N weeks. So it's pretty simple to estimate when deliverables will be done.)
Business is very chaotic. Admittedly, we (where I work) are a very bare-bones team, so we are quicker on our feet than a larger organization.
The main issue we would run into with "sprints" was that the business world-view didn't stay fixed for 2 weeks at a time. There was always something that came up and got bumped in priority. This led to people feeling disappointed that "we blew the sprint".
In reality, "the sprint" doesn't matter. Solving business problems matters. Kanban (Wikipedia says it's just japanese for "billboard" i.e. "big priority board") is just a way of explicitly stating the current priorities so that you can react to them.
> As a manager, how do I know how long a task should take?
You still break down big tasks into small tasks and estimate them. The planning process isn't really any different than with sprints, it's just that you're more free to make trade-offs in real time.
Isn't the idea of the sprint there to protect developers from having the priorities shift under their feet every two minutes? It's easier to get work done if you know you can focus on a task for a few weeks and not be forced to context-switch suddenly.
Under Kanban the way that is protected is by enforcing "don't start something new until your current task is finished."
Obviously life is messy and it doesn't always work out that way, but it tends to be closer to reasonable if you break the size of the tasks down enough. I can reasonably anticipate what I will be doing on a given day. Possibly even the next couple days. I have a good idea what I'll be doing in a week, but any number of things could happen before then.
Another thing to keep in mind is that the changes I'm talking about are "course correction" type changes. I.e. "We have projects X, Y, Z ongoing. Z has suddenly become more important, so we're going to bump it's priority." This is vastly different from a business that is managed incompetently where one day we are trying to sell milkshakes and the next we're in the tuxedo business.
It reminds me of a comment I heard regarding earthquake safe buildings: "They are scarier because everything moves more. But since they move more, they don't fall over"
Wow, I honestly didn't even know that had a name. I worked at a place that did this but I just thought of it as "the only time I've seen Agile actually work".
Sometimes we go a very long time without doing any explicit planning. Things can tend to get a bit stale if we wait too long.
What I meant by "constantly" was probably more "at any time". The truth is that we cannot commit to a list of priorities for two weeks. There will always be something that comes up and pulls someone off track.
The different between Kanban and 2 week sprints is that, rather than viewing that as a failure, that is seen as responding to business priorities, which is a good thing.
Here's what's interesting... of all the people screaming about how much they hate "Agile", pretty much none of them are proposing any alternative. And if they did propose an alternative, how much you do you want to bet that it would look an awful lot like ... wait for it ... the Agile Manifesto?
It's been said before, even in this thread, but "Agile" as it's practiced today is not the "agile" of the Agile Manifesto. The problem is ignorant managers and consultants who don't actually know anything about how software is developed, want a buzzword to slap on to what they're doing to CYA, and will do stupid shit no matter what it winds up being called.
The problem is that the "agile" of the Agile Manifesto is underspecified to the point of uselessness. I'll quote the Manifesto in full:
We are uncovering better ways of developing software by doing it
and helping others do it. Through this work, we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value
the items on the left more.
That's it. That's all the manifesto gives us. In a real sense, any methodology at all is "not agile", since agile values individuals and interactions over processes and tools. But while that might work for a small team, it's a recipe for disaster in any kind of larger organization.
Right. which is why I argue that it is important to quit referring to "Capital 'A' Agile" as though it's an actual methodology in and of itself. Agile really just refers to a family of (loosely) related methodologies which purport to adhere to those principles to varying degrees, and where different methodologies make more sense in different settings. Collapsing all of this discussion under the umbrella of "Agile" elides the point that there actually are different approaches for different kinds of organizations, and different projects.
The entire category of ideas that agile occupies is utter bullshit, and the alternative to bullshit is just common sense. Hire good people, give clear instructions, and leave them alone. The problem with common sense is that you can't make six figures peddling it for a living.
It's the opposite of the Agile Manifesto. These are 3 practical steps to management that apply to any industry where you depend on workers with diverse skillsets, software included. I actually picked them up from "First, Break all the Rules," a general management guide. OTOH, the Agile Manifesto is collection of value judgments that are not only vague to the point of being incomprehensible but also completely ignored by all agile practitioners (i.e. evangelists, coaches, whatever you call them).
I viewed the Agile Manifesto as a sacred text of software development for most of my career. Eventually, I came to understand that this is not a sacred text but a golden calf. It's just a gentleman's agreement between consultants not to interfere with each other's snake oil. It's not a misunderstood document; it simply wasn't intended to be understood.
* Committing to dates and then being accountable for the results
* Spending time planning, rather than "doing"
* Admitting to business leaders that much of their work time is not spent writing lines of code, but dallying in all the interesting byways of the field, favorite websites, pet projects, and random semi-related technology interests
* Sitting in meetings of any sort (sprint planner and retrospective)
* Having spoken, out-loud disagreements and negotiations in order to arrive at a consensus (sprint point estimations)
* Having their mistakes, miscalculations, lack of planning visible to everyone on a big monitor (sprint board / retro)
* Getting out of their chairs to go talk to someone else in order to remove a blocker or understand a dependency (task estimation and completion by end-of-sprint)
* Anyone involved in the process that isn't a coder (Scrum Master, Project Manager)
- Loosely estimating dates, then having management set them in stone
- Spending more time talking about things than it would take to do them ("planning")
- convincing business leaders that engineering is more than just writing code
- sitting in meetings that add little value to the team
- arguing over made up numbers for vague deliverables
- having their mistakes, miscalculations, and other normal aspects of software development enumerated and held against them, as though it was an actual problem
- having "scrum masters" come up with straw men to justify even more process
- reporting to too many non-technical managers
The last point sums most of it up though. In many orgs, Agile is used as an attempt to compensate for lack of technical leadership, as though process can obviate the need. You need some, most, or all of your management and executive team to have technical foundations, depending on how technical your product is. You wouldn't put a football coach in charge of a coal mining operation (forgive the hyperbole). The same principle applies to software.
* Committing to dates and then being accountable for the results
* Spending time planning, rather than "doing"
* Admitting to business leaders that much of their work time is not spent chiselling wood, but dallying in all the interesting byways of the ...
* Sitting in meetings of any sort
* Having spoken, out-loud disagreements and negotiations in order to arrive at a consensus
* Having their mistakes, miscalculations, lack of planning visible to everyone on a big monitor
* Getting out of their chairs to go talk to someone else in order to remove a blocker or understand a dependency (task estimation and completion by end-of-sprint)
* Anyone involved in the process that isn't a tradesman
>Committing to dates and then being accountable for the results
I found a very good way of getting around this. If you need to estimate a task, estimate it honestly in your head (say 2-3 days) and then double the amount.
It almost feels like cheating but with this method I have consistently delivered timely on my responsibilities. This method works because we usually plan way too optimistically.
>Committing to dates and then being accountable for the results
Being forced to commit to dates, as only a soothsayer would dare.
>Spending time planning, rather than "doing"
Because "doing", as you say, helps planning.
Development is not production (which is compilation), it's conception all the way, it's the non-automatable and hard to estimate phase of creating order out of void and chaos, by completing up to a complexity that no one can hold at once in his mind, and precising unambiguously into a formal language, some vague expectations expressed in a natural language, or some creation imagined by the developer himself.
Preventing a developer from "doing" is like preventing a mathematician from interacting with his board.
"Programming is a good medium for expressing poorly understood and sloppily-formulated ideas."
(Marvin Minsky)
"The computer revolution is a revolution in the way we think and the way we express what we think."
"My real goal is to transform things that are hard to understand into things that are easy to understand,
and programming has been one of the tools in doing that."
"I always learn things by writing programs."
"Clarity sometimes need programming because (...) if I write a program that doesn't represent something sensible, I get an error."
(G. J. Sussman, https://www.infoq.com/presentations/Expression-of-Ideas)
"What I cannot create, I do not understand."
(R. Feynmann)
"A number of people are desperately waiting for the software industry to reach some kind of maturity and start resembling traditional industries. They will have to wait forever."
(V. Lextrait, Mindfacturing)
Software developers, like everyone else, hate bad managers.
Good managers don't do "agile". They just manage. They may use techniques that fall under the umbrella of "agile", but in reality, they just do what the situation calls for.
Bad managers followed a course about the latest trendy methodology and pick only the parts that that can be directly applied, ignoring the context completely.
Books describing a process do it on an idealized case. It is like everything is perfect except for the well defined problems that need solving. But "by the book" situation is extremely rare. When it happens, it is the mythical "$methodology done right". In practice managers have to mix and match techniques for a real life situation, where nothing is perfect, but nothing is hopelessly terrible either.
Indeed, but this criticism suffers from the no-true-Scotsman fallacy. Better to just eschew anyone who endorses "agile" no matter how persuasively he claims to practice the one true actually-agile Agile methodology.
> Individuals and interactions over processes and tools
Take a look at the rest of the principles and compare them to that diagram. SAFe is clearly the opposite of agile. It calls itself "Agile", but it's as non-agile as is possible to get.
What we have here is not the no-true-Scotsman fallacy, it's somebody rightfully pointing out that somebody born in Australia, who has never left the country, with no heritage that comes anywhere near the northern hemisphere, is not, in fact, Scottish.
Sure. Yet every single time an organization tries agile, the result is a dysfunctional mess. After this history of pain, "real agile isn't like this!" has just as much legitimacy and persuasive power as the old "real communism has never been tried" line.
I dislike Agile, but let's not operate under the assumption that removing or changing the process is going to stop "the business" from wanting more features, faster. There is no process that will make that go away.
Agile Fails for the following reasons.
1) In reality, most stories/tasks that make it out of the backlog and into sprints are the ones that were inserted by PM's, that support the short term goals placed on the PM. End result, massive technical debt in short order. Even if developers express the 'need' for refactoring, it gets delayed as the PM doesn't get fired for a messy code base, but gets fired for a late deadline.
2) Companies 'turned' to "SCRUM" and imagined/expected that by renaming meetings to "stand-ups" etc.. their productivity would increase significantly. This usually resulted in management blaming their engineering staff for their own follies.
3) Most Biz environments don't lend to a 'perfect' scrum model, as such, many organizations try to 'shoe-horn' practices in situations where it doesn't make sense. For example, an organization may not have the ability to adjust features or timelines, resulting in stories being a convenient way to blame engineers for poor decisions outside of their control (Sales selling products that don't exist and promising it yesterday, or Product demanding features that are unrealistic, etc..).
4) Another buzz word that has little functional value other than to help 'experts' get promotions, who in practice are nothing but glorified PM's with a pinch of snake-oil salesmanship.
Agile Fails for the following reasons. 1) In reality, most stories/tasks that make it out of the backlog are executed on, ones that were inserted by PM's, that support the short term goals presented to the PM.
This is a classic example of the problem we're fighting here. Agile is not a methodology and it does not mandate use of stories, a backlog, scrums, etc. Those things are artifacts of one particular methodology which purports to be an exemplar of "Agile" -- that being Scrum.
I really feel like most people saying "I hate Agile" really mean "I hate Scrum".
Agile is like HR - everyone thinks it's supposed to benefit the developer/employee. It's not. It exists to benefit the company.
Agile was created so that consultants could provide better estimates to their clients in a field that was incredibly new, at a time when methods of evaluation were based on things like construction (think Gantt charts).
Its primary focus is on communication and setting expectations, which is critical when dealing with non-technical stakeholders that think everything should take a few hours. Everything beyond that is a bunch of bullshittery to sell books, conferences, keynotes, courses and certificates.
My favorite response when Product Managers bring up that business needs to know when a feature will be done is "And I would like to know Apple's stock price in 6 months."
I really believe this accurately communicates how difficult it is to estimate software development timelines. You can't estimate unknown work, and unless you're building wordpress blogs, your work in software is unknown.
The idea referenced in the blog post that sprints are "rushing" developers is just nonsense. At least in every sprint process I've been a part of, the developers are the ones assigning effort (points/hours) to each task. The dev team is free to decide how much padding any task deserves to get the job done right -- and some padding is totally legit and expected.
If you have a large task, one that's either too complex to estimate accurately or with too many unknowns or outside dependencies, you should be breaking it up into smaller tasks.
Developers decide how many points they can do in a sprint, AND they decide how many points each task gets. There's no rushing involved here.
Except maybe for the point that humans are notoriously bad at estimating creative work and biased in the shorter side... So taking estimations as commitment is indeed a way to short-hand the one estimating. Thus the rush when you realise your naturally optimistic estimate became a deadline.
I've had one great experience working in an "Agile" team using the Extreme Programming method. I have fond memories of pair programming, starting work by writing tests with the business analysts, and focus on principles : DRY, YAGNI, no hesitation to refactor what needs to be. The driving forces were those principles. My other experiences with SCRUM, on the other hand have either been deeply dysfunctional, or just plain devolving into churning tickets. The driving force were metrics, velocity, charts.
I agree with one of the conclusions of that article (but not the title) : Agile has been taken over by SCRUM, for most people now Agile = SCRUM. However SCRUM is so successful precisely because it fits into the existing hierarchical structures, it gives the impression of control to managers who don't understand what software development is about, and it avoids having to profundly question corporate organization. It's at its core a management technique. If all implementations of SCRUM get it wrong, then SCRUM itself must be wrong.
I think an alterative way of doing things would be to create small teams that incorporate both programmers and business people, without product owner or project managers or hierarchy. The work would be focused on one specific project and driven by principles of craftsmanship. I've yet to try this idea in the real world...
The missing ingredient in 99.9999% of "Agile" shops is developers having hand in the relationship.
That's it. I realized a few weeks ago that I've been lucky to almost always work under other engineers, nearly my whole career. Magically I become a thousand times more persuasive and articulate! (I didn't, I just have an audience with better priors).
One developer-stakeholder relationship to keep in mind is Google-consumer. When was the last time Google had to endure a call from you asking about "velocity?" ...never. And this is because Google has a zillion dollars and will continue on merrily if you die tomorrow. (Yes, Google has meme problems like killing own products, releasing too many things, etc. The point is that you are not an active impediment to their development cycle)
The best way to Do Agile is to have engineers hold >50% voting stock in the company, and for the company to be in a position that it can say FU to any particular client.
Actual agile is not rocket science, nor is it magic---you still have to make tradeoffs, test with users, etc. It's just that you're doing those things from an informed point of view, compared to the alternative where decisionmaking power rests with those incompetent to wield it.
There is of course more to business than engineering. You also want >50% of the voting shares to be able to read an income statement, (and, in small-enough businesses, to understand the value-add of accounting at all!) and whatever metrics keep the lights on (CAC, etc). Yes, to effectively manage complex businesses turns out to be complex!
Devs LOVE Agile, the ideas behind Agile and the agility itself.
What devs hate is cargo cults. When management slaps "agile" sticker on your team while having no clue what Agile is about, it's called deception, misrepresentation, misinformation, and exactly what people usually "hate".
You direct your disappointment towards wrongly placed label, not Agile. Hate your incompetent management instead.
I agree with this sentiment. As the author said, agile was invented by programmers for programmers but somewhere along the way the army of scrum-certified project managers hijacked it and turned it into something we (or at least I) despise.
The worst part of it: the box-checking. The incessant, never-ending, "was this and this and this and this and this ceremony done", all of which provided no value to the product.
I think you captured my post better, so I'm just doing it as a reply:
---
"Agile" is like "xml" (or JSON.) Just following a process, or using a technology, doesn't mean you'll do it right. I'm sure plenty of us have come across hair-brained xml, or JSON, that looks like it was designed by a moron.
The same can be said for "Agile." Just because a group tries to follow it, doesn't mean that they follow it correctly or manage themselves the best way.
I'd also like to point out: All of the quote in the article could equally apply to waterfall, or any other software management model.
One of the problems with "agile" is that no one has a clear idea of what Agile is. Day-to-day, what does an "agile" team do? As a dev, I've encountered many software development methodologies that lay claim to the term "agile". I've hated every single one of them. At what point do I get to say, "Actually, no, it's the idea that's the problem not the implementations"
I hate agile. And I'm a dev. This is not a matter of hating the wrong thing or not knowing what agile is.
I literally hate agile, i think the definition of agile is loaded.
I propose a new form of project management called "Relativity." The definition of relativity based project management is effectiveness. If something is not effective don't do it, if something is effective do it. Wait aren't devs doing this anyway? Is putting a fancy word on top of this obvious concept pure BS? Is this what agile is doing? No it can't be BS. Not when people can take a class for it. Once a class is made for "Relativity based project management" you know it's no longer BS.
One alternative I prefer: not specifying any deliverables or deadlines at all. It’s just that mangers often have a hard time selling that to stakeholders. But if you can be in that process, it’s fantastic.
That is “our product is great, but it’s going to be outdated in 10 years, so we’ll put you 5 devs in a room to try to build the Next Big Thing”.
Yeah, a process like this is applicable to maybe 5% of development projects out there, and is definitely not ripe for abuse by all and sundry in the other 95%.
Demonstrate you're producing valuable work on a regular, fixed cadence. Use whateve process you want to do so.
The hard part is defining value somewhere below "the whole project is done." Harder than it sounds, people get it wrong all the time, astoundingly so, almost to the point where I'm convinced it's some sort of evolutionary defect.
Completely agree. But here is the thing: as a developer I'm probably going to dislike any process that management and stakeholders agree is viable. It's not that I expect to find myself in any kind of non-viable process, but I also don't expect to not dislike it.
That is a great way to do software if you have unlimited money. This can be achieved either by working on something that does not yet have any stakeholders and get income from elsewhere or by working at a huge company that has time to spare.
If you want to create a company that is profitable, you absolutely must have a deadline for shipping something. If you don’t decide one yourself then your bank account will do it for you.
I'm gonna name a new process of walking, but not just walking... effective walking where you avoid obstacles. I'm going to give it a fancy name and call it "Quantum movement." It's different from regular walking as Quantum movement adapts to terrain and avoids obstacles where the priorities of the person who is walking are satisfied.
Agile is the same concept as above. It's different from regular development as it's trying to satisfy the customer.
There was a big push for Agile in my old group at Microsoft (no names here).
It was basically micromanagement. I saw hours-long, daily meetings by PMs and managers who were digging into the "velocity" of individual developers. They would still add and remove work mid-sprint, and generally treated estimates hard commitments rather than, well, estimates. You'd get a talkin' to if they thought you were behind.
Mgt: "We need that Frobicle system done in a week."
Me: "What. This is mid-sprint. You're changing things up? And what the heck is Frobicle?"
Mgt: "Don't care. Our estimate is a week, go do it."
Me: "Repeat: What is a Frobicle? Is there a spec? Someone I can talk to? What color is it? Is it animal, vegetable or mineral?"
Mgt: "A week."
Our mandated daily stand-up was actually just a PM getting the team together in a hallway while he polled us and wrote down our status, grist for those aforesaid meetings. I got our PM to say, "Time for status" instead of "Time for standup" because the meeting was useless to our team. Eventually I just stopped going, and a few other developers did, too. We were doing our own scrums, informally and as we needed them.
Naturally there was still crunch time. I'm actually okay with that, as long as management doesn't try to deny that crunch time is happening, and limits the damage to people and the organization, and doesn't do it continuously. A bit of crunch every six months to a year is fine, while a crunch at the end of every sprint is NOT, and that's where things were headed about the time I left.
Throwing this on agile seems like a stretch. Kind of reminds me of the recent article about Commoditised Excellence. As soon as you slap a name on something, people will use it willy-nilly everywhere and then blame you.
Agile is fine if it is an organic, bottom-up process. And as any of the founding Agile gurus like to profess, this was the original intention.
The problem is that it is usually implemented as a centralized, top-down imposition by management. The net result is usually just a tax on productivity to ever more process, paperwork, status updates, and meetings.
I built a from-the-ground-up agile/scrum process workflow for a team shipping a monolithic enterprise app that had a complex build process. Our productivity sky-rocketed. We went from semi-annual production deliveries to monthly ones.
Management didn't skip a beat -- they pivoted to new and innovative ways to push against the guardrails we built for requirements, approvals, agreements and deadlines.
In this case, neither agile nor Agile were the issue.
> Some of the most frequently-mentioned problems with Agile are: Agile ignores technical debt;
This isn't a problem with Agile. This is a problem with the management in your organization. I worked at an org that implemented Agile into 5 sprints. Sprints 1-4 were focused on the business work - things that deliver value to the organization. Sprint 5 was an "IT" sprint, where the developers owned the priorities and had time to fix things like technical debt. This worked pretty well.
> frameworks like Scrum are just “red tape,
Yes, companies need their people to operate in a predictable way. This isn't the wild west - you need to be accountable to the people you report to. These "red tape" processes are just regular checkpoints with various people in the org to make sure everyone is on the same page.
Waterfall had this too, except the checkpoints were every 1 - 3 months.
I think the bigger problem is that some software developers don't want to be accountable to people. They just want to do their own thing, like Joel Spolsky recommends with his staff at StackOverflow. It can work great, in the right environment. But the vast majority of companies do not operate this way.
> programmers are asked to commit to arbitrary estimates and deadlines
This has been as old as software itself. I've been on a waterfall project which had "things that needed to be done" and those things had zero detail in them. Yet I was asked to come up with a "ballpark estimate" by my PM. When I had zero idea what it was about, who it would interact with, etc.
> never get the time to think thoroughly about the features they’re creating.
So add a Spike and give it a two week estimate so you have the time to do whatever thinking / analysis you need to do to come up with a solid plan. Spikes exist for a reason - not everything on the backlog needs to be a feature.
Devs are such a special kind of people that you should just leave them alone? #sarcasm
I for one like where we are heading with things like Event Storming/Modeling. I admit I care more about solutions than spending time on perfectly describing a problem. Stop the navel gazing. IMHO. Agile tries to solve the Big Issue. Communication.
I've been a developer in both good and bad Scrums. Usually, the difference boils down to one or more of these points:
- Is the product owner any good? (physically present, has mandate, shields the members from company politics, open to story input from the team; like taking care of some tech debt or swapping stories around to fit developer needs, setting requirements that are not too rigid)
- Are the team members any good? (physically present, self-sufficient, experienced enough or with a buddy, communicate open and clearly with the product owner and other team members / disciplines, flexible enough to produce a balanced result within the requirements)
- Is the scrum master any good? (physically present, proactively alert for difficulties in the process or team, encourages interdisciplinary pragmatic solutions, expectation management product owner, balances the needs of the team vs the needs of the product owner, enforces 5 minute standups in the morning and a good but short retrospective after a demo)
- Is the project any good? (realistic budget and MVP, enough room for creativity in development and design, good reason for existence, stakeholders that show up for demo's and stay for the drinks after)
- Is the location any good? (a reasonably creative environment, preferably where stakeholders cannot make surprise visits, with an open floorplan so interdisciplinary communication is encouraged, with all needed materials and enough room on the walls for a scrum board, a burndown chart and to drown them in things like post-its, designs and technical schematics)
- Is the timing any good? (one to two sprints minimum to get the team oiled, not more then 3 days a week - so the other 2 can be spent at a lower tempo, no team members with large attendance gaps or shuffling people in and out, not more then 12-16 sprints because if the project needs more then clean it up, have some downtime, and start a phase 2)
I probably forgot something, but I believe these are the main issues that can really influence the success (and pleasure within) a scrum. One or two issues can be worked around; any more and the project is a drag or even a bust.
Agile was misery for me. There were 5 managers on the call and just two developers. The managers were taking status from the developers (me and another guy). The deadlines were measured in hours and it was just horrible to be on those calls.
I've got mixed feelings about this article. Sure, I hate the idea to be force to meet unrealistic deadlines, but that doesn't come from Agile. It is a condition imposed by markets. Sure, in the long term, good quality code, whatever that is, trumps anything else. But that analysis doesn't account for the fact that those that don't fare well at the beginning...perish!
As in "we'll do it slower than the competition, but in two years they're going to be in a spaghetti code nightmare". Possibly, but YOU WILL BE GONE.
I'm currently part of a team implementing a "next-gen" HMI on an industrial machine. There are too many new technologies, not enough experience and no existing work.
Trying to squeeze this development effort into a Scrum workflow has produced all the problems mentioned in this article. I feel that we're producing a huge pile of crap that will come down on us sooner rather than later.
Our Certified Scrum Master is almost always absent.
Just read the Agile Manifesto. Most likely whatever your company does has no semblance to what the manifesto says meaning it’s not agile other than the name.
I find that every single discussion I ever read about agile misses the point, both on the side of its defenders and its detractors. Agile is about one thing, and one thing only: alignment between stakeholders.
Every single one of the twelve principles is either a truism that applies to any and all situations ("Build projects around motivated individuals (...)", "the team reflects on how to become more effective (...)"), or are observations about where consulting-like relationships go wrong, and simple strategies to mitigate those problems.
If you have an arm's length relationship between yourself and the business stakeholders, where the business stakeholder (knowingly or unknowingly) has a poor understanding of what the solution to their problem looks like, where iteration is cheap, and where partial success is materially better than nothing, then agile could work for you. This is common for consulting, but a lot of product-like projects look like this as well, and agile might actually be a good fit for product teams.
It doesn't work for a lot of environments, though.
In security/safety-sensitive environments, and many hardware-adjacent environments, iteration isn't cheap. Infrastructure work often has long lead times before the first milestone that can be meaningfully delivered to end-users for feedback, and the engineering team is also the stakeholder with the best understanding of the requirements. In cases where the business stakeholders are internal to your company, and they have a really good understanding of the work that needs to be done, the default agile posture is to keep them at arm's length where a better strategy might actually be to fold that business stakeholder into the development team.
The Agile manifesto is fatally flawed, though for rather different reasons than the ones typically given.
Agile was started by software developers, and everything in the manifesto is very strongly tilted to how software developers do things and for bottom-to-top (or at least shallow) organizations devs like. However, every existing software company has much more of a top-to-bottom structure and is run by managers. They even have people who are not managers or developers!
Every 'twisting' of Agile is a completely natural outgrowth of trying to fit Agile to how organizations are actually ran and how people who are not software devs need to do things. That exactly is 'true' Agile, because business as structured today cannot support any other Agile you might conceive of.
That's why Kanban is so much more appreciated - it wasn't conceived in denial of how businesses are currently structured.
P.S. The other option would be to built the business to support Agile and not the other way around. For better or worse, almost no existing company actually works that way and that's not going to change.
snarf21 posted elsewhere in this thread a perfect example. I'd like to refer to his Agile list as some other roles in the organization might see it.
"Value individuals and interactions over processes and tools"
Management - Processes and tools are exactly how we ran our business. It's the only way that scales (and is also lawsuit-proof!).
"Value working software over comprehensive documentation"
QA - How am I going to test that software if I don't have good documentation of what it needs to do and how it works? Documentation is necessary for working software, not something to be balance against working software.
"Value customer collaboration over contract negotiation"
Management - We do need to get paid you know. On that matter, our interests are not exactly aligned with the customer's. Furthermore, having clear expectations and legal protections for both sides is a form of collaboration, a necessary one in fact.
"Value responding to change over following a plan"
Investors - Ah, so you're just following the market and have no intention of initiating change on your own? How is your company going to be the next Apple if you keep up like that?
I'm not necessarily saying the other characters are right - but these perspectives are a given in today's organization and are going to influence the actual process. Indeed, given that managers have more power, these are going to be the defining influences, way over some manifesto.
CPP, CORBA, XMPP, SOA, UML, Agile... Whenever I've seen the developer community embrace a new set of tools (whether hw, sw, process, or a mix), I've then then the marketing departments and consultants follow. After a period of time the term for the new tools is muddied beyond recognition as people who don't truly understand it try to use it in sales pitches and class materials. In my experience the tool is rarely to blame (maybe I'll make an exception with CORBA, ymmv). The 'Agile movement' started with a bunch of developers who were good comparing notes on their processes. They came up with some points of commonality, and some points from one or more of the others they'd each like to adopt. These all revolved around the engineering version of the unofficial USMC motto "Semper Gumby", always flexible. Some call this the agile mindset (ycmv, i.e. your caps may vary).
"If I have an organization with an Agile mindset, and really rock-solid product management, Agile processes and tools will evolve out of that. If you have the Agile mindset and an awesome connection with your customers and are solving their problems, things will evolve in the right way" - Todd Little, CEO Lean Kanban.
Over and over, people like Kent Beck have said that no one set of tools will work for everyone. Teams need to continually adapt, continually learn, continually improve, and do what's needed to enable those things.
I love the agile mindset, and the palette of tools I can choose from to paint a development process for a team that fits the way they work, their needs, and their customer needs.
OTOH, I hate the "Agile this. Agile that. You got to get certified in them all" Pokemon-style consultant sales pitches, and feel sorry for those who get sucked in by them, or who have managers who get sucked in by them.
Any movement that gains traction will be corrupted. The moment companies want to hire people who tick a box, they pull on a supply chain of certification authorities (which includes, most perniciously in my opinion, grad schools) that codify and dumb down any set of good ideas and practices so that it's teachable at industrial scale to mediocrities.
Back in the late Aughties I worked with someone who'd recently graduated from a design program and was interested in becoming a UX expert, the new-hotness of the moment. The person wanted to go nearly straight from undergrad to a Master's program at CMU. I -- being a HCI/UI person myself (and rolling my eyes at the term "UX") -- counseled doing freelance work for a couple years to get some practical experience that would provide grist for the grad school mill. No interest.
I can't say I blame the person too much; big companies want to show their seriousness by hiring people with credentials, the value of the credentials be damned.
As an engineering manager, I prefer to use the word lean instead of agile because even if the agile manifesto is a great manifesto, the word really has been hijacked by people in suits who live in their own world far away from day-to-day engineering work.
Lean is much nicer, it implies it's totally ok to remove process/meetings/things who don't add value
Eating up post-it notes ("issues") and crapping out code - i.e., doing agile - is just not gonna cut it.
When the end goal is a robust, resilient system that customers depend upon to suit their needs and your organization depends on for its business, by doing Agile you're going to end up with something that neither your customers nor business can fully rely upon (once your system or service is past a certain level of complexity).
Two major reasons:
1. A set of agile "Issues" (whether stories or bug reports) do not constitute a necessary or sufficient set of tasks necessary to produce, release, and operate a resilient or complex system.
2. You know the phrase that "Not everything that can be counted counts, and not everything that counts can be counted"? Agile gives you utterly meaningless metrics (e.g., story points) that can be counted, but don't count. What does count depends on the nature of your business or product: Small start up? Probably revenue or users. Large-scale infrastructure system? Maybe average latency or uptime. Unless you can directly correlate "story points" to your bottom line, just skip that clutter in the first place.
Call me old fashioned, but I found that when you bring engineering back into software development, specifically systems engineering (not "engineering" in the sense of "I'm a ruby-on-rails engineer"), products and teams are able to support a high level of confidence and reliability in their work products. This means (yup) requirements engineering and analysis, QA, Independent Validation and Verification, etc etc. It does not mean you have to have full departments for each of these, but at least having someone to "own" each of these areas (esp for small teams). Again, in the end, no amount of stuffing post-it notes down your developers throats will replace old fashioned engineering process and engineering management.
So if you can't use Agile, what is the way to manage software development?
There are many industries with complex product design processes, which need to deliver at a fixed deadline. We expect Apple to deliver new iPhone every fall, and they meet their deadlines without exception. Same thing for many other product manufacturers.
There is no reason to assume software development is very different from other industries regarding product design. Other industries also work with changing and innovating semi-finished products, components, and new materials.
Donald Reinertsen has written some interesting books on managing the design process (not related to software development or agile). It offers a lot of theory and economics behind product design processes. A lot of focus on reducing the size of Work in Progress.
Agile's not the problem. Micromanagement under the guise of agile is the problem. Ideally implemented, Agile sort of allows a development team to micro-manage themselves, but management is usually unwilling to let this happen.
Software developers hate working for the pointed-haired boss too. Agile is just a name for a process which is a meta "tool" for a company to get its work done. We're always going to hate working in an environment where management doesn't understand what goes into the sausage and what's important to do now that won't cause problems later. And we can all agree that hard work and code quality are key but I will push to do THOSE under any methodology. If a company isn't doing them then maybe it's not the process -- maybe it's the company/culture.
All modern Developers are doing agile, whether they believe it or not.
The agile movement of the 90's was in response to waterfall constraints imposed by the tools of the 70's and 80's.
The majority of projects today are no longer handcuffed by these constraints.
For historical context, I would suggest reading up on the Scrum vs XP war, as one camp pursued a commercialization and certification path to agile and the other remained largely principle-based.
When I see people expressing frustration with agile, the cause is typically a byproduct of the commercialization camp.
Agile was designed for small consulting teams (like PwC consulting teams, not remote software development teams).
I am not be surprised that bits and pieces of it are useful in other contexts (e.g. standups -- when they really are quick, standing-up touching base).
I am very much surprised by wholesale fetishization/adoption in other contexts. My GF sees it used by teams that don't even do software development.
Makes me think of that "commoditization of excellence" essay posted here yesterday.
My own take is that agile simply become tool for non developer actors to influence development team in (extremely) unproductive ways. It leads to short term thinking, really strange people practices.
I have only seen it getting exploited. Specially in large orgs, where you are surrounded by non development team members (people higher up are more detached to development). So the roles like scrum master etc etc. become important to them. And quite frankly leads to burnout and toxic environment.
It's because Agile as a theory (in a vacuum) works but in practice, fails to be anything but a time sink for most development teams. Agile was supposed to help teams react to change and allow for elasticity in the development process but in my experience, managers rarely allow pushing deadlines or allowing for cutting features. Agile is basically just used by managers as a way to make sure people's velocity is maximized at all times.
When I was working on an agile project based on scrums, I developed a real dread of the daily standup meeting. You don't always have results to report on a daily basis. On the days that I had nothing much to report, I was implicitly given the feedback that this was _not_ an acceptable answer. I was simply not able to lie and always give a positive progress report. This was one of the main motivations for my quitting that job.
SCRUM is toxic. It is toxic because it dis-empowers the developers who are doing the actual work and hands that power over to the "SCRUM master". The SCRUM master is the ultimate bureaucrat -- and a proxy for Management -- in a heavily bureaucratic methodology. The job of the SCRUM master is to guard and protect the SCRUM process. Often to the detriment of the developers and ultimately the project.
Any process can be abused all the way to unproductivity.
The problem is literally always some empowered moron in charge, without any clue what the work actually looks like, yet with the ability to destroy work.
I’m looking forward to a time when companies don’t insist on running themselves like hierarchical feudal societies... it’s such an immense efficiency drag, but since it’s the dominant dogma in business, the garbage continues.
Most software developers like the principles of Agile development but its become a marketing term that high level management don't understand. For Agile to work, it has to have buy in from the top, hence it often is assumed Agile means quick results for the business if they follow Sprints. Recruiters market places using Agile, but never really understand what it means
The problem with current implementation of agile is it tries to standardize a process and use it across all companies and all project.
In reality, I've found that every developer are unique. And together with unique company cultures and projects, the process need to adapt with those things, and not the reverse.
A scrum may work well in a team / project, while it may not work in other.
> In reality, I've found that every developer are unique. And together with unique company cultures and projects, the process need to adapt with those things, and not the reverse.
Which is pretty much the core of the agile manifesto.
A daily SCRUM standup is often described as answering three questions: "What I did yesterday, what I will do today, what impedes me"
When I described a daily standup to my father and brother, a physician and a university professor, they shuddered a bit. Many people from outside software do. I've heard this described as violent transparency, and if it seems intimidating to people on the outside, that may just mean that we, as software developers, have grown so accustomed to a fundamentally unfair work environment that we no longer question it. Similar to "interviews" that are actually whiteboard exams. Did you know that most job types don't interview this way? Again, my friends who work outside software shudder a bit when they hear about this practice. It sounds awful, because it is kind of awful.
Now, in spite of my stated belief that it is awful, I did explain (or at least tried to) why we do it. Software is highly unpredictable, and difficult to estimate. We pick a task, and work on it, and report on it. This, at least at first, was to me the essence of agile. Waterfall wasn't working. It is difficult and, ultimately, unproductive to try to spec out everything in advance. You have to evolve toward a solution, through repeated iteration and constant communication. Not a bad idea.
Here's the problem - agile turned out to be a particularly insidious version of waterfall. The point of all this "violent transparency" was that we embraced the essential futility of estimates. We make them at as granular a level as we can get away with, but what we are really doing is reporting in every day on how things are going, and embracing the fundamentally uncertain nature of what we do. Well, these standups turned out to be an irresistible opportunity for micromanagement and a daily application of pressure deadlines.
I view agile as the Nitroglycerine of the software development methodologies. At some point, you can't keep saying "it would be safe if they'd just use it more carefully." The inherent instability of nitroglycerin is the flaw. And agile is just too damn unstable. Any dev who things the customer (could be a project manager, a boss, a stakeholder from a different department) is going to "collaborate" if the spec isn't met by the deadline is just shaking a big bottle of nitroglycerine. And at this point, the daily "standup" will be a person patiently waiting for the developer to finish making excuses (which is what these three questions will be interpreted to be), followed by "so are you going to make your new deadline, or will this be pushed back even further).
In many ways, if hard deadlines are involved, I prefer standard old waterfall. The original agile manifesto says "customer collaboration over contract negotiation". Indeed! Sounds great. Let's work together with trust, rather than lawyering up at every occasion.
Here's the thing - if you're committing to a deadline but the customer (who is often internal, with "litigation" being complaints to the bosses), doesn't commit to a spec, well, your customer is protected by a contract, but you, the dev, are completely exposed. Yes, change is common and must be embraced. If you'd like to change the contract, I'm all ears, but that will require - yes - a renegotiation of the contract, including the deadlines.
Otherwise, the agile manifesto, almost seems like it encourages software developers to be pacifists in the middle of a war zone. Look, if a customer is going to hold my feet to the fire on a deadline, I am going to hold the customer to the original spec.
Personally, I got out, found a job in software development where deadlines aren't a big part of what I do. Not sure if everyone can get a job like this, but it seems product oriented companies are more likely to work this way than contracted development projects. Deadlines under conditions of uncertainty are a pretty hellacious source of stress for me, feels like a kind of roulette, and while agile in theory had the promise of making it better, the methodology turned out to be nitroglycerine.
Your point about deadlines gets to the heart of the issue I think. Scrum and agile are flawed in many ways, but generally you can bend the methodology to at least mitigate the issues. The one totally unsolvable existential problem, though, is the impedance mismatch between a team trying to be agile and a customer who's almost invariably not.
The idea that your team can work on an open-ended basis with the customer to add functionality a piece at a time and dynamically adjust expectations as you gradually pin down the requirements and get a better idea of how much work they'll take, is, unfortunately, a fantasy. Somewhere up the chain, someone is going to want to know how much it's going to cost to get the features they want. The very best you can hope for is to estimate (ha ha) how long the milestone/project will take and then attempt to conduct scrum within that time box. But it's farcical and demoralizing because you end up structuring your sprints with what you know you need to get done instead of just what you think you can reasonably get done. The "agile" nature of it fractures the very first time you get behind your guesstimated schedule.
I think the process could work if you win the lottery with a customer who's actually enthusiastic about embracing the uncertainty of agile, or if you work in a type of org that sets its own release schedule. But unfortunately I think those situations are rare and for most of us the fundamental mismatch between agile developers and non-agile stakeholders kill the whole idea.
>we, as software developers, have grown so accustomed to a fundamentally unfair work environment that we no longer question it.
Compared to a physician or a university professor a software developer might have a low level of autonomy (in the sense of free from being micromanaged) but developers do better than the vast majority of employees I suspect. Having to account for your time every day in a standup can get psychologically tiring but there's so many other jobs where it's much worse. I've worked in places where being out of your seat for more than a few minutes (when not on a break) would invite commentary or provoke questions.
(This isn't intended to sound like a "You should be grateful because there is a kid starving in Africa" thing, just trying to provide a perspective why I find it hard to feel hard done when so many more people have it worse than me all around me. This doesn't mean we shouldn't all enjoy more autonomy and agency in our work.)
This is a tough one to answer. Yes, there are harder done people, and I suppose that is enough to end the conversation.
The only part I'd mention here is that you focus on micromanagement and autonomy, which alone, to me, isn't really the source of stress. I've had boring but predictable butt-in-seat jobs with clear instructions. You do your work at a reasonable pace, you will make your deadline.
The source of the stress is being pressured to make firm deadline commitments under conditions of complexity and uncertainty, and the organizational aftermath when you don't make them (ranging from confrontational meetings, to angry emails to your boss, to threats to your employment, to threats to your professional reputation). I really have seen them all.
Can't speak to how much of this goes on in other jobs, I'm sure it does -- then again, I don't constantly hear about a "critical shortage" of workers in most of these other jobs either, which implies to me that people with options may be rationally choosing to avoid software development in favor of other fields.
Typical agile methods squeeze the max out of developers. You optimize estimates (velocity...), break down the task ahead of time and allocate every developer maximum out. Since the next release is always around, dept and a Sprint without full allocation is never there. Essentially, you transfer an engineer into a factory belt labourer.
The part missing from agile is the model for what software is and how it is built. If management don't correctly understand the nature of software development, they will inevitably manage it incorrectly, tie it to the wrong accounting measures, and embed an adversarial perspective to their own development teams.
#It gives management an easy chance to be abusive. Management abuse seems to be the norm with Scrum, not the exception.
#The methodology does not take this problem into account. The common answer to management abuse is that if you are experiencing it, you are not doing Scrum.
One strange thing about the Agile Manifesto that people usually miss is that it's signed at the bottom by a bunch of consultants. Who are these people and why was it important for them to attach their name to this?
Been working Kanban since June, life is better and I think our company is putting out more meaningful features than my last job using agile. We also continuously deploy everyday instead of doing 2 week releases.
kanban is for factories to make more cars etc. the boss likes it, the workers, not so as they're pushed by the rule with less leeway.
agile is for software companies to hopefully build product faster, managers like it, developer not so, as daily standup alone pressures developers to stay on track all the time, e.g. less "free" time at work etc.
both are really designed for the owners, not for developers, so it's absolutely normal when you say software developers hate it.
The point that agile produces tech debt is something that wrings true to me and matches my experience. Any good processes people have implemented to reduce tech debt?
Someone (I think possibly John Cutler) made a good point about this, which is that by now the early and middle adopters of classical manifesto-style "Agile" have absorbed it so thoroughly that it's no longer something deserving of a special term, it's merely the way things are done.
This leaves the late adopters, and the sociopaths whose main interest is extracting money from the late adopters. As a result, it's becoming a safe bet that anyone using the word "Agile" unironically is either going to inflict upon you whim-based development with added Post-It notes, or sell you an enormous slab of process binderware with bonus added two day certification courses. (From which your managers will return having spent the entire course answering e-mails on their laptop, and proceed to inflict upon you whim-based development with added JIRA.)
Whatever you are trying to accomplish as a team, you need some things.
You need to know what you intend to build. Depending on whether you're trying to create a new market or compete in a large field, you might have a clear picture of the end result, or a lot of ideas you need to try out. Where you fall on that spectrum should tell you how far you're allowed to plan in advance. Rarely do you need to change what you're building on a timeline as short as a week, but of course that can happen, too. But given that planning and meetings require time, and rework also requires time, you need to find a balance that works for your team size and the above allowance of planning based on volatility of your goals.
Someone has to be able to make decisions. Often those decisions are high-level - what is the market value of the thing you are building, and how many resources are you willing to allocate. We know in software that estimation is hard and we will get it wrong, so that's always going to be a thorn in the side of the decision-makers, and some of them will always try to offload that responsibility back on us anyway. Still, at some point it is reasonable to relay how well things are going back up the chain.
You need to be able to share plans, decisions, constraints and a swath of other information among team members. Some of it will be for builders. Some will be for decision-makers. Team structure will probably dictate how important leadership and decision-making is compared to maker details. And that structure will hopefully flow from the kind of thing you're building, whether it's just an idea, or something that has to be done a particular way against a particular cost profile to ensure it's worth doing at all.
There's plenty of room in all of this for ambiguity, for conflict, for human errors, for ego. For prioritizing the wrong thing (whether that's the thing to work on, or the thing to measure, or how much of a stink you make about mistakes.)
Given all of the above, what I don't like about Agile is the tendency to put off figuring out how you're going to build things until the last minute, and in the middle of figuring those things out, you change what you're going to build. If you're building out ideas and moving fast and breaking things, then that pain I just described is something you have to accept as one of the costs of being a pioneer. In so many cases I've seen, you're not being a pioneer, so if you spend a bit more time on planning, you'll figure out what to build and how to build it, and then each member of the team will buckle down and focus on the details required to do to the building.
Focus, and less volatility. That's what I want out of the process in play for accomplishing something. I think the decision-makers mostly don't want to be caught out making a bad decision, such as spending resources to build something only to find that the value isn't there - either because of the market, or the unknowns in resource cost.
I've worked in a few Agile shops, and it was horrible in all but one. Perhaps that one was "doing it right".
The thing is that it appears to be extraordinarily hard to do Agile right. I think that any system that is so difficult to implement correctly is an inherently flawed system.
Might it not be that it's extraordinarily hard to manage software development in general? Are you aware of any other methodology that's easier to do right and get the desired results?
Yes, managing software development is generally hard and there is no silver bullet. But I do think that the methods that I commonly encountered before Agile were better in terms of both developer stress and software quality.
Interestingly, in many ways they also weren't all that different from what Agile is intending to bring. They still involved something analogous to "sprints", they still allowed for course correction and pivoting during development, and all of that.
Agile seems to position itself as the opposite of "waterfall", but outside of large corporate environments, I'd never seen a shop that actually used waterfall as described by the Agile community.
If I hate something only when it is done wrong, but it is so easy to do wrong that it is far more often done wrong than not, at what point should I consider just hating the thing because it is so easy to do wrong?
I mean, if you use a process that's apparently fixed enough it's worth making certifications around it, it's quite likely not "agile" in both the dictionary sense of the word and in the sense of the agile manifesto.
The agile manifesto is so broad and vague, it is all things to all people, and therefore useless. As soon as you start transforming it into actionable items, in a non-working team, you immediately start getting capital-A 'Agile' abominations, because there are always capital-R 'Reasons' for why <your company's particular flavour of horrorshow> is having a negative impact on developer productivity. 'Reasons' that neither you, nor your team lead, nor his manager have any power to change, regardless of whether or not you are six-sigma-black-belt-scrum-master-rock-star-certified.
There's only two valuable ideas in agile, that are actually broadly applicable.
1. Do continuous integration.
2. If it doesn't make sense to do something, don't do it.
You don't need to get into philosophical discussions over the manifesto, and the purity of your adherence to agile philosophies, or any of that nonsense.
What you do need to do is to go and deploy Jenkins (Or whatever the new hotness is in the CI world), and don't build features that nobody needs.
As soon as you start transforming it into actionable items, in a non-working team, you immediately start getting capital-A 'Agile' abominations,
I think we should stop referring to those things as "Capital 'A' Agile" ( which isn't even an actual thing to begin with) and refer to them by their actual name: Scrum, Scrumban, SAFE, XP, Crystal, RUP, USDP, AUP, etc. At least we can try to make the point more clear that when we criticize these things, we're criticizing a specific "thing" and not some catch-all term. Given that there is no concrete methodology named "Agile" to actually criticize.
I work in an environment with projects for many different customers, and almost as many different approaches to coordination with the customer, and I'd say the points from the agile manifesto are quite useful to suggest and evaluate the process for a project. Can we get a proper quick feedback loop? Can I talk directly to people that actually know about the details? How does the change process work? How much overhead is there that's not directly relevant to doing whatever it is were asked to do? What kind of decisions do we need to stick to that have been made without/against technical guidance?
It really depends on the customer if that process is something that can't be influenced. Although of course we're as an external party often have more freedom in that regard than a customers direct employees.
Agile works perfectly fine where I works. We just try to adhere to the manifesto. Works much better than anything I have been part of the last 25 years of my career, for sure. We wasted so much time back then.
TL/DR: devs don't hate agile, they hate non-technical enterprise middle-managers and the people who hire them.
If you've ever worked in an enterprise that embeds non-technical managers on technical teams to run the show, you've probably seen some flavor of the following:
The manager starts with good intentions. They try to learn the product, the team, the rough implementation outline, etc. Over time, due to the delta between their understanding and the reality on the ground, their job devolves into:
1. Attending ill-conceived meetings that generate mountains of noise because they demand the impossible, the nonsensical, or the redundant.
2. Communicating this noise to their team through a series of terse bullet points / JIRA items / trello lists / whatever
3. Assigning a senior+ engineer on the team to reconcile their list with reality, and then to do the actual work of translating product requirements into meaningful units of work for the more junior engineers on the team.
4. Scheduling more meetings to reconcile the original list of items with the list of items that now makes nominal sense, producing more noise because people get upset about items they didn't understand in the first place changing, or feeling like they've "made commitments to stakeholders" that they now have to apologize for or backtrack on, etc.
5. Scheduling meetings to handle the "drama" that emerged when manager A and manager B had an ego-fight over some insane laundry list of garbage. "Going to the mat" for their "stakeholders," etc.
6. Scheduling more meetings with engineers to "fix the process"
7. Scheduling more meetings to hire replacements for the engineers who burned out doing both the manager's job and their job while being held accountable to the latter while hobbled by the former.
8. Rinse, repeat. Note that the cycle begins afresh with every new manager, who in my experience have at most a 1-year lifespan on a team because they were either good enough at looking important in meetings and showing up for 70 hours a week that they got promoted to the next shit-show, or they "moved on to other opportunities".
This whole ouroboros of noise gets wrapped up in whatever management terminology is prevalent at the company.
In this situation, the team loses time and energy every time the manager does a thing. They either have to stop working to correct the noise (which if left uncorrected festers and becomes worse) or stop working to attend an unnecessary meeting to make sure it doesn't go off the rails, or whatever.
Agile, which is really just a cover-all in enterprise for
a "continuous planning because we don't ever really know what we're doing" strategy, multiplies this noise many times over because error compounds upon error multiplicatively, and it creates a lot of opportunities for error by creating many decision points.
(Edit: there's a darker version of this, which I've personally witnessed, where no one pushes back on the insanity because the team is very weak. This is how you get teams serving static web pages at 2.5 transactions a minute on giant web service frameworks that cost tens of thousands of dollars a month to "operate", or pick your daily wtf)
It's not just management that gets into cargo culting. And it's not just "top-down" that results in pain in an agile working environment.
Sometimes "bottom up" can be a shitshow too.
I've (tried to) work with developers (special-grade assholes) who were quite far up their own orifice at using the language of "Agile" processes in an adversarial but socially point-scoring way. To beat other people over the head, and even harass people out of their jobs (including management), implying that anyone who isn't doing what they say, or providing continuous, detailed arguments why not, is doing software wrong - while producing mediocre, low-effort, low-results work themselves.
That isn't Agile, but I have seen the language of Agile used to abuse and pressure in a seriously uncool way. So I'm personally a bit cautious when I hear Agile advocacy from individuals, until I learn what they really mean, and how they use it.
Among other things, I listen for whether they have a working empathy gland, and use their tools and knowledge to lift up people they work with, or knock them down.
I've also seen it used to ignore proper handling of PII (GDPR-worthy personal data), via the principle "everyone is equal in the team, so should all have access to production data". Naturally this causes tensions, when some people pay attention to PII issues, and some would rather ignore them, perhaps disagreeing about how important it is, or perhaps just sloppy. (Related, I've observed a really interesting dynamic between privacy advocates and anti-privacy advocates.)
And the same with other quality-control issues when the level of care varies considerably.
At the same time, some people are very happy in their agile teams, and report that the rest of their team seems to be as well; and they are constantly shipping good work. It seems to be working for them.
Between all the things, I have the impression that some people's "Agile" approaches risk a "productivity feel-good illusion", where the perceived productivity over weeks and months feels great, and there's a happy buzz in the team, but long-term business-affecting output is sub-par because the feel good factor and constantly doing useful things misleads people into thinking they are operating at the best they can be.
And yet, it's not like planning everything, or top down direction changes from the business arm, makes for particularly good results in software either.
Since it works really well for some, and is used as a tool of abuse by others, I'm neither for or against "Agile" as such, and instead, prefer to look past the buzzwords used by each company and team, and look at what they actually do each week/month, how the individuals feel (not just "the team" but each member), and what sort of results over time they are producing (measuring strategically useful outputs, in addition to whether they seem to be busy and happy).
I think the principle of developers deciding themselves how to develop is an excellent one, to the extent we can encourage and assist that. But when developers are in conflict with their peers, and using the language of "Agile" as part of conflict or even abuse, it can be quite messed up, and may need an intervention from above.
One of the biggest issues with the scrum (master|lord|bag) model of Agile is that any process grievances magically transform from "this process has issues" to "that person is wrong."
Dogma destroys agile. If the team is coerced into behaviour that doesn't create momentum its counter productive, both in terms of morale and buy in on the end goal.
Yeah not having meetings and process is great for a solo developer working in isolation, but it doesn’t scale to even a single team, let alone a much larger organization. Once you go past a single person working on a product, you need some kind of process to facilitate communication.
Once you move beyond yourself you are gonna have a process, either implicitly or explicitly. I vote that it is best acknowledge there is going to be a process regardless of you liking it or not and be explicit and intentional about its formation.
Once you acknowledge that there is gonna be a process you can either start from a known set of processes (scrum, kanban, waterfall, XP, whatever) and then iterate on it until it suits your environment. The alternative is to somehow reinvent the process wheel from the ground up and probably not be as effective as if you had started with proven processes.
It’s like people who start a project and say “all these frameworks suck!!! I refuse use them! Bloat!!!”. Six months later you look at their codebase and surprise, they’ve created their own implicitly defined framework. They won’t call it a framework, but lo and behold they have a framework. Probably worse on every measurable dimension than what they would have had if they started with what’s out there too. Had they been honest with themselves, they could have saved a ton of trouble being explicit that there will be a framework no matter what and had been intentional about its creation/adoption.
"The word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products."
(https://pragdave.me/blog/2014/03/04/time-to-kill-agile.html)