> Machine learning isn't comparable to software development. It is a statistical modelling exercise.
It's neither of the two. Machine learning isn't comparable to any other human endeavor because in many cases, much more value comes out of the models than (seemingly) goes in.
LLMs for example are punching way above their weight. The ideas underlying their software implementations are extremely simple compared to the incredibly complex behavior they produce. Take some neural networks that can be explained to a bright high schooler, add a few more relatively basic ML concepts, then push an unfiltered dump of half the Internet into it, and suddenly you get a machine that talks like a human.
Obviously I'm simplifying here, but consider that state-of-the-art LLM architectures are still simple enough that they can be completely understood through a 10-hour online course, and can be implemented in a few hundred lines of PyTorch code. That's absolutely bananas considering that the end result is something that can write a poem about airplanes in the style of Beowulf.
Lots of problems have very simple solutions. And progress often means finding simpler solutions over time.
But coming up with those solutions, and debugging them, is what's hard.
For a comparison, have a look at how pistols got simpler over the last two hundred years. Have a look at the intricate mechanism of the P08 Luger https://www.youtube.com/watch?v=9adOzT_qMq0 and compare it to a modern pistol of your choice. (And the P08 Luger is already pretty late invention.)
Or have a look at modern Ikea furniture, which can be assembled and understood by untrained members of the general public. But designing new Ikea furniture is much harder.
It's neither of the two. Machine learning isn't comparable to any other human endeavor because in many cases, much more value comes out of the models than (seemingly) goes in.
LLMs for example are punching way above their weight. The ideas underlying their software implementations are extremely simple compared to the incredibly complex behavior they produce. Take some neural networks that can be explained to a bright high schooler, add a few more relatively basic ML concepts, then push an unfiltered dump of half the Internet into it, and suddenly you get a machine that talks like a human.
Obviously I'm simplifying here, but consider that state-of-the-art LLM architectures are still simple enough that they can be completely understood through a 10-hour online course, and can be implemented in a few hundred lines of PyTorch code. That's absolutely bananas considering that the end result is something that can write a poem about airplanes in the style of Beowulf.