> So the data scientists have multiple laptops that they download the data to, then run the models overnight.
This haphazard way of running compute jobs really stuck out to me. I can't imagine doing things this way (rather than having a central compute cluster running SLURM or similar) at a company bigger than, say, a dozen people - much less the scale of Uber. What's the rationale? Even if it's just a cluster of 3 or 4 machines in a rack shoved in the corner, isn't that better than ... laptops?
It was easy to go to IT and say "get us a pile of laptops" and let the data scientists do their thing. It was hard to hire engineers to solve the problem (I was one of the engineers). They particular problem was far enough down the priority list that it took until 2016 to solve.
This haphazard way of running compute jobs really stuck out to me. I can't imagine doing things this way (rather than having a central compute cluster running SLURM or similar) at a company bigger than, say, a dozen people - much less the scale of Uber. What's the rationale? Even if it's just a cluster of 3 or 4 machines in a rack shoved in the corner, isn't that better than ... laptops?