You should probably not automate all toil. You should only automate toil when the toil cost/effect is more burdensome than the cost of automating it. All automation has a cost, and may or may not create value. Automation should have a positive and timely return on investment. If the ROI is 10 years down the road, you probably shouldn't automate it (yet). If there is a cheaper way to deal with the toil, explore that avenue first.
Several times in my career I worked on projects to reduce toil. Sometimes the project would fail because the time it took to work on them went well past the cost saving estimation. Sometimes they would be completed, but the value created was far less than their cost. And sometimes automation wasn't even the solution, and we just needed to change our process or system, or do some other manual thing that reduced the toil cost. Sometimes we chose to automate toil because we were afraid to take on a larger project we knew would make the toil unnecessary, so we paid for the automation and then later for rebuilding everything. Or toil was used as an excuse to justify a project that didn't really have to do with toil.
One of my biggest mistakes as an engineer was assumptions I made about my work that ended up creating more waste than value. Talk to an outsider about your plans and why you're doing it, take their advice seriously. And if your automation is optional, make sure you have buy-in before you start working on it; i've sunk months on things that nobody ended up using.
A great way to automate toil is incrementally. Typically you have a runbook with step-by-step instructions, and over time you automate one step, then another, etc. The investment is minimal and gradual, it can change over time, and you can target the costliest parts of the toil, optimizing value.
Some times, the process of automating something provides enough positive returns in and of itself. For example, you might learn how to do new things along the way. Or you might be able to give the task to some one new so they can learn.
Or maybe in the process of automating, you discover new things about the process itself and can improve it.
I agree that one should be careful to consider work priorities and return on investment, but there are often hidden returns to something like this that leaders don't understand and take into full account.
This misses the induced demand effect of dramatically reducing the cost of the task. There are many things that only happen occasionally because they are annoying and slow. If you reduce the friction suddenly everyone does it 10x per day and the whole company benefits from faster feedback loops.
> This misses the induced demand effect of dramatically reducing the cost of the task. There are many things that only happen occasionally because they are annoying and slow. If you reduce the friction suddenly everyone does it 10x per day and the whole company benefits from faster feedback loops.
But that assumes the task is still valuable at doing it 10 times more a day.
One key distinction to think about might be if your task is letting you reduce cost vs increase revenue.
A task that is a "cost" - e.g. if a user wants X, we need to do Y - likely won't need to be done 10x more frequently with 10x increased value if the demand for X hasn't changed. So you make it cheaper for us to do Y when X is desired, so the margin for X is increased, which still might make a ton of business sense, but the top-line boost to profitability is limited to the original manual cost of Y.
A task that is revenue-driving - e.g. "we have to do X any time we're putting together a sales deck for a new prospect" - can have a much higher flywheel effect. Can our existing sales team potentially now bring in 4 times more clients? That could be huge, and so you've both increased margin and top line.
It doesn't need to be as valuable to still be net positive though, because now it doesn't take human time, just computer time.
Imagine for an ML product, making an accuracy report. If it's slow and required lots of human time, you might do it once a quarter for releases to important customers. If it's cheap and quick then you can run it on every CI run to check for regressions before merging code. Sure, you run it maybe 1000x more and don't get 1000x the value.
But, critically, the value is not the cost savings of not having to run it manually per quarter, the value is the more stable product and avoiding spending time bisecting a quarter of engineering work to figure out where bugs were introduced. And this was enabled by automating.
Sure, if the cost of automating here is < the current cost of rework and investigation, you win. And dev time is expensive, so that sort of thing is usually an easy call.
Yeah, it doesn't need to continue to increase in value linearly with repeated runs, it's the summed value that matters.
The "cost" is fuzzy too, often - e.g. time and budget spent on reliability-focused engineers or active troubleshooting rarely drops to 0 if you don't automate anything. It might just make it more expensive to react to incidents!
Maybe turn it away from a "gate" question - "should this thing be automated" - and into a prioritization one - "we could automate so many things, which ones should we do first?"
I don't know whether it's visible from where you are sitting, but what you wrote is exactly contrary to the TFA. TFA implicitly starts from "what if we forget for a minute that ROI exists, and see where that leads us". (If it seems incredibly wasteful, stay with me to the end.) And where it led them is that the people like you and me are crucial parts of Investment (the I of ROI), and the people group together. The ones that gladly do 90% toil don't like to team up with those that prefer 10% toil.
Because the problem with ROI calculation is that for some areas the ideal amount of toil would be 99% and for other 1%. For both extremes, you'll bleed valuable people, so sometimes your Investment becomes "hire new team just for that" and the rest is peanuts.
To put the thing back on its feet first create a team that takes 50% toil and give them these areas that ROI-wise require approximately 50% toil. Call the team "SRE". Create a team that takes 10% toil. Give different areas. Create a team that takes 90% toil, etc.
An organization generally has fixed resources for automation investment. You should look beyond if the ROI justifies investment, to instead prioritizing the highest ROI items that are most likely to be successfully automated.
Several times in my career I worked on projects to reduce toil. Sometimes the project would fail because the time it took to work on them went well past the cost saving estimation. Sometimes they would be completed, but the value created was far less than their cost. And sometimes automation wasn't even the solution, and we just needed to change our process or system, or do some other manual thing that reduced the toil cost. Sometimes we chose to automate toil because we were afraid to take on a larger project we knew would make the toil unnecessary, so we paid for the automation and then later for rebuilding everything. Or toil was used as an excuse to justify a project that didn't really have to do with toil.
One of my biggest mistakes as an engineer was assumptions I made about my work that ended up creating more waste than value. Talk to an outsider about your plans and why you're doing it, take their advice seriously. And if your automation is optional, make sure you have buy-in before you start working on it; i've sunk months on things that nobody ended up using.
A great way to automate toil is incrementally. Typically you have a runbook with step-by-step instructions, and over time you automate one step, then another, etc. The investment is minimal and gradual, it can change over time, and you can target the costliest parts of the toil, optimizing value.