I agree with Moxie. But I’d go even further: I think the real problem stems from the modern corporate structure.
We have this idea in modern management theory that responsibility and decision making must travel up the corporate hierarchy. That execs are best positioned to make product level decisions. Or decisions of any kind. And peons at the bottom - the “individual contributors” who do all the programming must know the least about the product.
But that’s completely backwards. The people on the ground in any organisation know the most about what’s really going on. If we disempower them, and try to turn software engineering into an assembly line process (jira tickets go in, code comes out) then of course programmers stop innovating. They’re explicitly, and intentionally distanced from their capacity to improve the product in creative ways.
To be clear, I also don’t think the answer is to pretend we’re all equal and have an entirely flat management hierarchy. Different people have different skills, and a healthy organisation needs lots of talents to thrive. But there’s lots of ways to do that which don’t disempower the people on the ground.
The book Reinventing Organisations is a gem of a book. It totally blew my mind on this stuff. It talks about all sorts of different ways that innovative companies have upended their corporate structures, and along the way have been able to get the best from their people. If / when I start a company, I’m tempted to make it required reading.
This is something I was actually shocked about coming from Government. People at the lower levels in a corporation just have no say, and might get lucky to hear about product direction and major decisions, often at far less detail than they actually need. The only way to really get in the know is to move up the chain, but those up the chain don't really seem to understand their own software products very well. Any discussion from an engineer is brushed off as being too "in the weeds" or "not high level enough," but it's not clear that those operating at a high level usually understand what's going on.
I really don't know what it is, but the corporate folks are just obsessed with hierarchy. It feels like it's in their blood. Even at the lower levels, they want to move up the chain so that they can have "real input," and they believe that culture must be set "from the top." Why? This is certainly one way to run things but it's not inherently the only way to run things. Individual teams _can_ set culture, and an individual team _could_ have better input on a strategic decision than the "decision makers." It just seems to be a forgone conclusion in the corporate world, and I can't see any reason for it other than status-seeking.
> corporate folks are just obsessed with hierarchy
You should read "The Dictator's Handbook." While it primarily applies to politicians in both democracies and authoritarian regimes, the thinking proposed in the book also applies to the corporate world. In other words, some people thrive on power, but these individuals are not always the best for the company due to a misalignment between their desires and the company's needs. Nevertheless, the system still works because... well, just read the book (or at least a summary of it)!
I'm somewhat pessimistic about corporate culture: companies have an instinctual drive to hunt for profit. If they didn't, they would go extinct by going bankrupt. This leads to a self-selection process that explains a lot. Combine this with the tendency of some people to strive for power—as you mentioned, "they want to move up the chain"—and we end up with a somewhat toxic mixture in today's economy.
Regarding the topic "Agile is killing software innovation," I would argue that it's not Agile itself that's stifling innovation, but rather the emergent structure and behavior of corporate entities. The combination of power-hungry individuals and profit-seeking companies stifles a lot of potential. It's highly inefficient and causes considerable misery, but it seems to function just well enough to keep the economy limping along.
One of the saving graces of corporate entities is their ability to buy up startups, for example.
That's an amazing book. Easily one of the most entertaining reads of that year for me. I remember it saying a lot about "inner circles" and the absolutely critical need for leadership to maintain a hold over them. So much so that it isn't that important for leadership to be good at the "job description" as long as they can identify the other centers of power and keep them happy.
I recently started work at a new company that tries to turn everyone in to a cog. No agency, every step requires other teams and people that may or may not respond. It's maddening.
It is almost like the modern software enterprise optimises towards minimising productivity, an emergent self-sabotaging. Everything is just avoiding responsibilities and blaming the proces, others or just the universe in general for things going to shit.
Anyway, I'm going to read that book now before i start my own company ;)
I strongly agree, but winning this cultural/political war is hard.
And it's an extra hard war to fight in part because of what Moxie is saying, which I appreciate. The agile style promotes a cancerous software growth. Delivery deliver deliver, all the time, features forever.
It's good to be value minded, but making good long term decisions, thinking through your software and where it's headed, making long term decisions, and having artifacts and buy in for these long term plans is setting yourself up to resist the rot and decay that agile so easily lets in.
Agule and books like team topologies don't really have any commitment to the software as a whole. The team has a compact to ship ship ship ship, and so does every other team. But it imagines them all as islands, with only some interdependence. I struggle to think of examples to make this clear, but good software is holistic, good software tries to slam concerns & create smoothness across joints.
When agile lets software grow to be so very discombobulated, even if the corporation wants to be bottom up, the chaotic software architecture keeps engineers from seeing and springing on good ideas. Our potential as engineers is defined in sizable part by what we can understand and how well we understand the internals of our software, how well everything fits together, how legible the system is. Agile actively destroys this kernel for understanding, sacrifices the engineer's greatest leverage, for short virtue, for the reward of always be shipping.
If agile hasn't so overwhelmingly become a license for teams & product to do whatever, so long as the features keep shipping, the engineers would be in a better stance to have executive will & function, be able to go out there & steer & direct; we'd deserve to be heard better. But we are kept low by the thrashing chaos that comes in the absence of thinking ahead. The technical inadequacy erodes our ability to make political/organizational progress.
It seems that you are mystifying agile as this all-powerful machine.
Most corporations are mediocre, slow to change, and their software projects are just not ideal. And agile is a mindset to accept this, focus on the people, both devs and stakeholders (who fund the project, and ultimately the company that pays their salary). Probably by now it's a cliche to reference the first line of the manifesto, but... people above process, etc.
Agile sets the stage, helps grease the gears, by default it doesn't keep the project "accountable".
How the actual plans come out of it is up to the actual team, the leadership (where the classic fish and head olfactory law applies).
Islands and cliques form naturally anyway. And in some sense it's okay, because making too big plans tends to fail. (Again, agile at best nudges folks to think about commitments, what their work items mean for the team, for other teams ... but if course the further the others the less relatable is the importance of their needs.)
> we'd deserve to be heard better
yes, of course. and other "bottom feeders" too (excuse my phrasing), and middle managers too ... alas corporations tend to be broken as I mentioned.
And as a consequence, I found that outside our vocal minority most people develop the "if I don't care it doesn't hurt" mindset, they do want to close tickets and nothing more.
> Delivery deliver deliver, all the time, features forever.
This is the mindset, and a huge problem. But Agile does not cause it. (Or solve it, of course.) Money causes it. Features bring money, more than other things, most of the time. Money in the short term. And few managers seem to care about the long term.
If anything, Agile is liked by managers because it claims to allow building features non-stop.
> We have this idea in modern management theory that responsibility and decision making must travel up the corporate hierarchy. That execs are best positioned to make product level decisions. Or decisions of any kind. And peons at the bottom - the “individual contributors” who do all the programming must know the least about the product.
The problem is that the decision making is not about the product. The decision making is about making money. Unfortunately, for a lot of companies, making money is not equal to building good product. IC is completely unequipped to make decisions that would maximise the company top/ bottom line (this is a descriptive statement, and it is not about the individual either: modern corporation is famously broken)
> IC is completely unequipped to make decisions that would maximise the company top/ bottom line
Right; companies want and expect ICs to keep their heads down and close tickets. Of course they’re ill equipped to make decisions outside the scope of their job role. If you expect nothing of someone, they will give you nothing in return.
> making money is not equal to building good product
I think in most cases, building a good product makes money in the long term. In the short term it might be possible to make money despite having a bad product, but as soon as the short term ends... look at Boeing and Intel.
This, short-term decisions win money in the short-term but businesses are always talking about retention and retention is most durable with a good product.
Yes, and this somewhat recent notion that corporations only exist to benefit their shareholders. Corporations exist in order to do the business that they’re in the business of doing, and their structure is such that it allows for the distribution of risk and the possibility of profit for investors. Ideally, people would make investment decisions based on whether the business was sound. Like, yeah, this business has tons of satisfied customers, is doing relatively efficient business and makes money doing it: I’ll invest because that company is a winner. Everything now, though, is entirely judged on growth and bottom lines that are easily manipulable quarter to quarter. You’re a successful corporation if number goes up. The financialization of literally everything erodes the real purposes the corporation exists. And the mindless repetition of the line that corporations exist simply to make profits for the benefit of shareholders is bullshit. How many times have we seen companies chasing those profits in such a way as to destroy the longer-term viability/success of their business? Yeah, they made profit for the shareholder in the short term. Hurray. And destroyed the business in the longer-term. That seems to me to be pretty anti-shareholder.
I don't think the structure is the problem. I think it's fine that the higher you go the bigger decisions and responsibility there is.
The part that I think is not working is that managers neglect one of the basic behaviors they should be engaging in. Weekly one-on-ones done properly. That's the place for information to go from directs to managers.
We have this idea in modern management theory that responsibility and decision making must travel up the corporate hierarchy. That execs are best positioned to make product level decisions. Or decisions of any kind. And peons at the bottom - the “individual contributors” who do all the programming must know the least about the product.
But that’s completely backwards. The people on the ground in any organisation know the most about what’s really going on. If we disempower them, and try to turn software engineering into an assembly line process (jira tickets go in, code comes out) then of course programmers stop innovating. They’re explicitly, and intentionally distanced from their capacity to improve the product in creative ways.
To be clear, I also don’t think the answer is to pretend we’re all equal and have an entirely flat management hierarchy. Different people have different skills, and a healthy organisation needs lots of talents to thrive. But there’s lots of ways to do that which don’t disempower the people on the ground.
The book Reinventing Organisations is a gem of a book. It totally blew my mind on this stuff. It talks about all sorts of different ways that innovative companies have upended their corporate structures, and along the way have been able to get the best from their people. If / when I start a company, I’m tempted to make it required reading.