Imagine a 5 year old code base that doesn't even have technical debt: it's a manifestation of the technical debt itself. It's hundreds of thousands of lines of code, mostly with "emergent behavior". You can't release without spending days on clicking through the RC system manually to check what's broken. Also, your product is extremely complex because apparently the barrier to entry in your field of business is very high.
And you have no users.
Then suddenly users arrive.
What it takes is 1) management buy-in (they should be desperate enough) 2) engineers who are not willing to make compromises.
Also, as usual in crisis management, people who successfully manage crises are generally ousted after the crisis is over because the personality needed to solve crises is not compatible with peacetime modus operandi.
The most positive change I've seen happen in an organization is through skunkworks projects, i.e. work you don't tell your boss about.
Too much technical debt, but the boss doesn't want to refactor? Refactor any code you were touching anyway.
Build process too complicated and brittle? Write build scripts, add error checking, and make other necessary changes to fix the problem.
My past bosses never asked "Where did you find the time to do that?" even if they even noticed that anything happened at all.
At one company I worked, we had no build server. The boss didn't see the need for it so he never approved it, but he also didn't reject it once we set it up.
Any organizational change that requires other to change their behavior or their opinions? It is very difficult to change. The only success I've had is by mentioning an idea multiple times and eventually my boss started thinking it was his idea.
Funny enough, at my last job the bosses never listened to anything their employees suggested. Whenever an outsider (journalist, user, family member, or outside consultant) suggested the same thing as the employees suggested, only then did the bosses consider it.
I don't know how common this is in other places, but that organization could only change through external, not internal, influences.
I remember reading something about it in the gaming industry; from memory it was written by a contractor brought into 3D games on tight deadlines to unstick them. And what they did was go in, ask the employees what was wrong, the employees told him everything that was wrong and what would fix it. They'd been trying to tell anyone who would listen, for months, and he was the first person who would listen. Then he turned around and told everything that was wrong to management, who were thrilled that the contractor could come in and so quickly tell them what was wrong and how to fix it, and give him the resources to do it because the deadline was imminent. The employees could lean on the consultant to cut through bureaucracy, and the consultant (who had skills) could sort out some intricate game engine performance problems, and the whole thing moved on. He did that almost routinely moving round big game companies from project to project.
I might be misremembering, but I can't find it in the HN search history.
Like an explanation given for how companies will let key employees quit, then excitedly pay more for new hires. New employees (or contractors) are all marketing about how great they are. Existing employees are all too obviously mere mortals with plenty of flaws. Of course getting rid of the ordinary employee and getting Superman looks like a good idea.
Manager spends time listening to employees complain: losing proposition, boring, time consuming, looks like doing nothing. Manager brings in amazing consultant who helps: manager looks good, saves effort, has a claim to need a bigger team and budget in future because evidence shows they just needed some extra help.
I’m actively working to change this as a CTO in a small digital agency. Can’t say that I’ve completely solved it but I’ve started challenging my peers when they use the words hope, belief and especially magic to describe what they’re up against. We’re professionals not hobbyists. Our codebases should get easier not harder to work with over time.
We have a sister organization which is larger and with their own CTO. I’ve tried to boil the ocean and inspire change across both companies, it was hard, for now I'm just focusing on trying to inoculate the new hires against accepting any sub-optimal bureaucracy. Though I see that need to find some way to train them to be so open and confident that they can start to challenge processes when they need to.
I have a lot of hope in the new hires though. Culture is hard to change but definitely doable. It’s all about who you hire, fire, praise and incentivize.
You can’t change an org from the outside, but it’s also difficult to change it from the inside.