A useful analogy is natural selection. The two features, mutation and selection pressure, are present in software. From the inside, the mutations seem rational, controlled, planned, but from the outside they are random. The selection pressure is itself complex, but success can be measured by "number of copies in the world", which is ultimately driven by social success and luck as much as anything. Software also has a (unique?) "win more" mechanic where a popular tool tends to get more popular. The pressure happens recursively at different scales, and sometimes very large branches tend to die off because of big topics like "lack memory safety" or "supply chain attacks".
Interestingly, language diversity seems driven by school curricula, which becomes comfort which becomes hiring practice, the change driven by academic boredom with a given language.
> Software also has a (unique?) "win more" mechanic where a popular tool tends to get more popular.
I don't think this is unique at all to software tools and is a general principle.
Rosen had a paper in 1981 talking about "The Economics of Superstars" [0]. I don't claim to have deep knowledge but the idea is that if an individual has limited resources (money, attention, etc.) and wants to ensure the maximum gain from their expenditure, then allocating resources to the superstar is a safer bet. This means that superstars get proportionally larger resources than what they might be valued at.
For example, if 120 consumer has $5 to spend on a song ($600 total), each consumer on an individual level might allocate their $5 on the known good superstar song. The superstar song might only be 2x better than the next best thing, so one might expect that the superstar song might, at most, get $400 of the total, but because of the superstar effect, the superstar song gets all $600.
I just watched an interview with Torvalds about Git. There were a lot of reasons why Git was so quickly adopted, not the least of which was functionality and how it leveraged a changing technology landscape, but a major factor was almost surely Torvalds celebrity status and blessing.
Interestingly, language diversity seems driven by school curricula, which becomes comfort which becomes hiring practice, the change driven by academic boredom with a given language.