1. Fast growing organizations struggle to keep up with the communication overhead when rapidly onboarding new engineers. Most common open source frameworks lack good interfaces for developing isolated components in the same project. In the short term it's easier to spin up a new project than defining and enforcing interface and dependency boundaries.
2. Cloud providers and consultants are incentivized to propagate the myth that distributed systems is the best solution for all problems.
3. Engineers looking to grow are incentivized to add popular new tools. In particular, the less equity you have in the company, the greater financial incentive you have to become an expert of a tool in high demand and land a job elsewhere with higher pay.
4. In my experience very few engineers learn the fundamentals of computers and systems. Instead they follow "gurus" that tell them the current "best practices" are. I think it's easier to feel you're doing a good job by making all your code comply to some style guide, or building systems with an architecture discussed in some cloud provider blog.
5. A VP of engineering I worked with told me in private that one of the reasons we were adding a lot of distributed systems components was so that we could sell ourselves as a tech company to VCs in the next funding round rather than a tech enabled business. I doubt that VCs care about this, but it's telling that a VP of eng thinks it matters.
6. If you start breaking up your monolith into a distributed system you won't feel the pain until you have several systems that are struggling to coordinate and keep data consistent. For the first few months or even years you'll only see the upsides of quicker iterations. It can be enough time that all the engineers that added the distributed systems got promoted and left for another job.
For companies growing quickly or large companies I don't see how you're able to mitigate the communication overhead without adding distributed systems. It allows different teams to ignore each other for the most part and respond to the market quicker. It's often easier for teams to re-build systems than trying coordinate with a different team that has different incentives.
But for all other companies I think people are adding distributed systems prematurely. But lots of individuals in the decision making chain are incentivized to add them. Unless you have an experienced CTO that can enforce a sane policy, it's inevitable that someone will add a distributed system without understanding the nuances that come with it.
1. Fast growing organizations struggle to keep up with the communication overhead when rapidly onboarding new engineers. Most common open source frameworks lack good interfaces for developing isolated components in the same project. In the short term it's easier to spin up a new project than defining and enforcing interface and dependency boundaries.
2. Cloud providers and consultants are incentivized to propagate the myth that distributed systems is the best solution for all problems.
3. Engineers looking to grow are incentivized to add popular new tools. In particular, the less equity you have in the company, the greater financial incentive you have to become an expert of a tool in high demand and land a job elsewhere with higher pay.
4. In my experience very few engineers learn the fundamentals of computers and systems. Instead they follow "gurus" that tell them the current "best practices" are. I think it's easier to feel you're doing a good job by making all your code comply to some style guide, or building systems with an architecture discussed in some cloud provider blog.
5. A VP of engineering I worked with told me in private that one of the reasons we were adding a lot of distributed systems components was so that we could sell ourselves as a tech company to VCs in the next funding round rather than a tech enabled business. I doubt that VCs care about this, but it's telling that a VP of eng thinks it matters.
6. If you start breaking up your monolith into a distributed system you won't feel the pain until you have several systems that are struggling to coordinate and keep data consistent. For the first few months or even years you'll only see the upsides of quicker iterations. It can be enough time that all the engineers that added the distributed systems got promoted and left for another job.
For companies growing quickly or large companies I don't see how you're able to mitigate the communication overhead without adding distributed systems. It allows different teams to ignore each other for the most part and respond to the market quicker. It's often easier for teams to re-build systems than trying coordinate with a different team that has different incentives.
But for all other companies I think people are adding distributed systems prematurely. But lots of individuals in the decision making chain are incentivized to add them. Unless you have an experienced CTO that can enforce a sane policy, it's inevitable that someone will add a distributed system without understanding the nuances that come with it.