As programmers we are ingrained to look further than the problem at hand. We try to see the bigger picture, solve problems in holistic ways if possible.
Often this can lead to over-engineering, creating more corner cases and ambiguity in requirements and reduction of test coverage. That is a trap to lok out for.
On the other hand it also can lead to better understanding of the problem space, increase understandability, and less need for later refactoring.
Just stating, 'don't look further than the problem you are required to solve' is not an excellent general rule.
> Just stating, 'don't look further than the problem you are required to solve' is not an excellent general rule.
It's still pretty good though; I think the only better formulation is "only implement a general solution if it's _simpler_ than the particular/specialized solution". Sometimes, that is indeed the case - but it's pretty rare. So if you want to keep things simple, "don't generalize until you have 3 examples" is a pretty solid rule-of-thumb.
But the main issue with that kind of remarks is that we are not omniscient demiurges. If we don’t try to implement two approaches, then we won’t be able to judge based on actual facts and metrics. If we already did that in the past often enough, sure we can gain some intuition on what will be the actual outcomes of the tradeoff we retain.
But paying the price to build such an expertise is also a tradeoff. In particular we might desire to provide the most sound and solid solution from a grand master quality level. But if the whole market is already in the hand of a few players that went with the old card of "occupy the territory first and (possibly) consider quality later", our grandiose art will be also a path to starvation.
> our grandiose art will be also a path to starvation.
That's what I'm saying, that in general the "grandiose art" _is_ a path to starvation, in a lot of business software at least. Because the actual (eventual) requirements are often unknowable at the start, and the initial assumptions are often invalid/false. So if you build your grandiose architecture based on wrong initial assumptions, it'll still lead to a subpar product even if the architecture was perfectly correct (given said assumptions). It's better to approach it with the idea that you're not writing code that needs to run 1 more than e.g. 1 year (but, you ARE writing code that you might need to maintain next year! that's a subtle but important difference. You write code to move fast now - not to enable you to move fast 1 year later; BUT, at the same time you write code that won't slow you down horrendously, in case you can no longer delete it 1 year later).
unless that lack of (intentional) lack of foresight by my superiors endud up with us having to rewrite a particular feature several times, over several years and it still doesn't work with acceptable performance, because it had to be shoehorned in later because we were shut down every time we said "but this is gonna make X impossible later!"
There you have the gambling, the risk, the reason why programming isn't mechanical.
One has to use judgement. Some generalisations are quite cheap, others very costly. It's always worth using the cheap ones and very important to use the expensive ones carefully. e.g. Use a named constant rather than a literal - that's hardly ever a bad generalisation...or making repeated code into functions.
OTOH, writing all your code in e.g. asynchronous python on the assumption that you might need it to be async next year....that's an expensive effort which may never ultimately be justified. You have to be fairly sure that you will want it.
It's also worthwhile to use languages that support change. So you can tell yourself that upgrading your code won't be an impossible pain in future. C++ is not one of those flexible languages so you feel pressure to get it right first time.
That's the thing, it might have been the right call. Take facebook: perhaps if you were on the original engineering team, you could've made the (correct) call that "developing this thing in php won't scale to billions of users! This is the wrong architecture!". And no doubt, a lot of hacks needed to be done, because it wasn't written "correctly" the first time.
On the other hand, if you had written it for 1 billion users from the start, chances are that it wouldn't have ever surpassed 1000k users. And that's a much worse problem to have.
---
To take it to your situation - perhaps the leaders really were incompetent; but also, perhaps if you spent more time to do it "right" earlier, the entire project would've been killed. It's hard to know what was the right call.... at least, the project is still alive, so the original call might not have been disastrous.
This was a critical piece of functionality. It was backup software, with deduplication. The feature in question was the ability to delete old backups. We were forced to release without it and then rush to implement something before customers ran out of disk space and started asking for refunds
I don’t think it makes sense to not think of generality options.
Often that helps us see a problem in more ways. Come at it from different directions. Generalizing mentally is map discovery,
But when it comes to implementation, any code that is there to enable unused generality is unused and unnecessary code. Any unused freedom of interface increases the chance of unintentional misuse or sleeper bugs.
The number of ways unnecessary code or loose constraints can be unhelpful is unlimited.
When it comes to optimizing, testing, maintaining, it is optimal to have the narrowest mission possible.
—
The exception is when designing a platform or library, with open ended reuse. Then value can genuinely be increased with greater applicability.
Then there will be generality vs. focused tradeoffs whose resolution will be very context dependent. On the subject matter and its potential market/users.
Good point. I guess it also depends on what are you further looking into. A few months ago I implemented a feature a client requested that was being charged by the hour. When code review came a more senior engineer suggested a huge refactor that in the end would leave cleaner code but required changing half of the small code base. I pushed back saying it wasn’t necessary due to the scale of the library we were working on. Turns out he took on the task to refactor it himself, broke both my code and preexisting code, and I had to go in and spend triple the amount of time debugging the refactor. A few months pass by and the whole code base will forcefully have to be refactored again to be merged into another code base.
He was looking further into making a clean code base, but I was looking further into saving our clients a few thousand bucks and delivering early. Which one was right? I’ll reflect on that after this new refactor and see how much the previous one helps (or doesn’t).
It think it's essential to understand the problem you are trying to solve - however often that problem is how to do something better in the specialised case than the current tools that support the general case.
ie why is anybody writing any new software these days when Powerpoint, Excel and Word cover all the use cases in the general case? ( I exaggerate for effect ).
Yeah, it's a real challenge to come up with low resolution rules/heuristics. That becomes much harder when you're tailoring your message to different communities / expertise levels.
I agree with you, but I also get where the author is coming from. Sometimes simple guard rails for new devs is a lot better than ultra detailed nuance that leaves them without a simple path forward.
In my experience I've witness horrible examples of both extremes. Overgeneralized yet poor solutions, and (often duplicated) hyper specific solutions. Both typically from less experienced folks. I think more communication and mentoring would really have helped more there.
As simple as possible, but no simpler
Let your code tell its own story
Localize complexity
Generalization takes three examples
Work backward from your result, not forward from your code
The first lesson of optimization is don't optimize
A good name is the best documentation
Bugs are contagious
Eliminate failure cases
Code that isn't running doesn't work
Sometimes you just need to hammer the nails
"
Sounds quite solid - as guidelines. My only rule for programming is, that the result is correct and usable.
I read it when I got stuck on a recent project, and paused after each chapter to review _all_ of the code (fortunately, it's a fairly small project) and to apply the principles from that chapter to the code, and it was a big help and got me over the hurdle to a working version.
A little micro-technique compatible with the "a good name is the best documentation" rule; create temporary well named boolean variables to "document" your branching logic. A toy example;
My arbitrary rule from recent experience is this: you do not need to lint your JSON by compiling and then running a Serde-based Rust application as part of your CI/CD pipeline for every single commit, when the project is entirely Python and Ansible for systems deployment.
When I worked at Bloomberg their first coding guideline was "Don't sweat the small stuff." I was mystified by that when I first read it. Then I sat in meetings where someone would drone on about how clever they thought their code was until someone asked "Is this small stuff?" It would shut them up so we could move on to more important matters.
1000%. The number of times I’ve tried to go straight to the correct well factored abstraction and it worked out are countable on one hand. This is one of those pieces of advise that I know to be true yet it takes continuous effort to enact.
There are several books that commonly get recommended for all around general programming knowledge (Pragmatic Programmer, A Philosophy Of Software Design, etc) Curious how this is any different!
I believe this is the type of book that I'd like to skim to see what it has to offer rather than just blindly buy it. This is how interpret some of those points:
* As simple as possible, but no simpler -> Don't get into fancy stuff like e.g: design patterns just because you learnt or know them? Same for other techniques depending your language and facilities? Don't use things just for using them.
* Let your code tell its own story -> Use verbs for functions and methods, nouns for variables and classes so code is understood with minimum effort as is being read?
* Generalization takes three examples -> Sure, first you write a solution, then copy paste it if needed elsewhere as you didn't expect to be reused, third time make an abstraction (function(s), method(s), etc) as it popped up once again?
* A good name is the best documentation -> Picking reasonably good names for all kind of identifiers will make code self-document and understand?
* Sometimes you just need to hammer the nails -> Sometimes you need to be pragmatic and get a thing done, maybe not in the best possible way?
* Code that isn't running doesn't work -> Remove all commented/dead code from a codebase as not only running but might also generate confusion to other people?
I do not want to diminish the content of the book in any way nor sound arrogant, but in the past I've seen similar titles and they weren't offering that much to what I already knew at the time.
Testing is running though but that's not the same as eliminating dead code. If you're writing a library/API, you might not personally use the code or hit the edge cases but other users will. If they aren't tested they don't work is my experience.
Any modern software design has to meet these 3-4 key aspects:
1. Make the software scalable w.r.t machines.
2. Make the software scalable w.r.t humans.
3. Make the software maintainable over time.
4. Make the software reusable to reduce startup costs.
Motivated by these, we make a lot of choices:
1. decompose a large complex problem into smaller, isolated, modular problems that can be worked on by different small teams, and run on different servers with inter-server communication.
2. organize code along with its documentation for easy understanding, readability and modifiability over time by same developers and different developers.
3. organize reusable and independently upgradable parts of the code for ease of upgrade with stable interfaces and stable test suites.
4. maintain the build/package/deploy toolchains for ease of underlying hardware and operating systems upgrades without affecting all of the code.
5. reuse is the only way to reduce costs – both time and effort – for anything. thinking carefully and setting up for reuse of services, systems, libraries, toolchains, processes, practices etc are critical to any large successful software project.
These are general rules for normal times. But there are inflection points when it is profitable to violate these rules.
Things that distort the cost/benefit tradeoffs and make it profitable to violate these rules:
1. When tech ecosystem is rapidly evolving and toolchains and libraries ecosystem isn't mature.
2. When time to market is super critical and if successful we will have more than enough money to deal with these problems later, and if we are late even with good software we would have failed and shut shop.
3. When we can't hire talented skilled engineers and still it is worthwhile to win in the short-term.
With these rules and violations, you can get stuck in an early local optima. A culture of continuous safe refactors is a super-power that can make you immune to it. Ossification of any kind is bad.
Often this can lead to over-engineering, creating more corner cases and ambiguity in requirements and reduction of test coverage. That is a trap to lok out for.
On the other hand it also can lead to better understanding of the problem space, increase understandability, and less need for later refactoring.
Just stating, 'don't look further than the problem you are required to solve' is not an excellent general rule.